勤俭持家

2016-11-30 21:05:25开始使用Linode VPS,这一晃几年过去了,我到现在还是在使用它运行一个商业网站!一个字评价 !从2016开机到2017才被重启了一次,官方发工单说需要提升性能重启配置下~

持续使用 Linode 4GB Plan + backup service2019-12-01,我决定升级为 Linode 8GB以应对业务量!这个时候 Linode备份的费用会从 $5/月上升为$10/月,这就有点小贵了...

于是,我本着勤俭持家的优良传统,开始寻找VPS自动备份的方案!寻觅了几天,终于确定了这么一个解决方案,开此文记录下。

备份方案

  1. 储存网盘:onedrive
  2. 监控备份:server酱(推送备份成功消息到微信)
  3. 整站备份:使用LNMP一键安装包自带的backup.sh脚本(每隔3天执行一次并上传到onedrive网盘)
  4. 增量备份:使用rclone sync命令(每天执行一次并上传到onedrive网盘)

通过 cron定时执行两条备份命令:

  • 第一条是rclone每天增量备份
  • 第二条是每隔3天的脚本备份全站

都是上传到onedrive网盘里,备份成功后推送消息到微信。

why onedrive?

  1. 能对接rclone命令行工具的网盘之一
  2. 桌面/手机客户端没有被墙
  3. 附带正版office 365
  4. 360企业同步网盘差不多到期了,工作资料转过来
  5. 1TB的容量,除了备份网站数据,还剩很大空间备份其他个人资料
  6. 能当电影离线下载盘,开全局梯子还能满速下载回来

实操记录

1.购买office 365 个人版

本来想用申请来的谷歌/微软无限量网盘,后面想想怕翻车,还是算了吧~

本来想在微软官网直接购买 ¥398/年,后来兜兜转转到淘宝的数码荔枝下单 ¥219/年,折合 ¥18.25/月,对比 linode $10 backup。嗯,真香~

2.安装和配置rclone

安装和配置rclone

#在centos7终端下运行命令
yum install -y rclone

#安装成功后开始对接onedrive网盘,继续在终端下运行命令
rclone config

选择n新建并命名,比如输入:o,回车

NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
No remotes found - make a new one
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> n
name> o

接着列出了所支持连接的网盘,选择onedrive,输入编号:22,回车

Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / 1Fichier
   \ "fichier"
 2 / Alias for an existing remote
   \ "alias"
 3 / Amazon Drive
   \ "amazon cloud drive"
 4 / Amazon S3 Compliant Storage Provider (AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, etc)
   \ "s3"
 5 / Backblaze B2
   \ "b2"
 6 / Box
   \ "box"
 7 / Cache a remote
   \ "cache"
 8 / Citrix Sharefile
   \ "sharefile"
 9 / Dropbox
   \ "dropbox"
10 / Encrypt/Decrypt a remote
   \ "crypt"
11 / FTP Connection
   \ "ftp"
12 / Google Cloud Storage (this is not Google Drive)
   \ "google cloud storage"
13 / Google Drive
   \ "drive"
14 / Google Photos
   \ "google photos"
15 / Hubic
   \ "hubic"
16 / JottaCloud
   \ "jottacloud"
17 / Koofr
   \ "koofr"
18 / Local Disk
   \ "local"
19 / Mail.ru Cloud
   \ "mailru"
20 / Mega
   \ "mega"
21 / Microsoft Azure Blob Storage
   \ "azureblob"
22 / Microsoft OneDrive
   \ "onedrive"
23 / OpenDrive
   \ "opendrive"
24 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
   \ "swift"
25 / Pcloud
   \ "pcloud"
26 / Put.io
   \ "putio"
27 / QingCloud Object Storage
   \ "qingstor"
28 / SSH/SFTP Connection
   \ "sftp"
29 / Transparently chunk/split large files
   \ "chunker"
30 / Union merges the contents of several remotes
   \ "union"
31 / Webdav
   \ "webdav"
32 / Yandex Disk
   \ "yandex"
33 / http Connection
   \ "http"
34 / premiumize.me
   \ "premiumizeme"
Storage> 22
** See help for pcloud backend at: https://rclone.org/pcloud/ **

这里client_id使用默认的(后面可以自己再设置,貌似上传下载速度可以再快些),回车

Pcloud App Client Id
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_id>

这里client_secret使用默认的,回车

Pcloud App Client Secret
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_secret> 

不选择自动配置,输入 n,回车

Remote config
Use auto config?
 * Say Y if not sure
 * Say N if you are working on a remote or headless machine
y) Yes
n) No
y/n> n

重点:因为我们是在命令行直接操作的,所以跳不出浏览器获取验证码;因此这一步你需要在本地电脑终端里输入 rclone authorize "onedrive",它就会自动启动浏览器访问网页,此时你只需要登录进去你的微软账号后,验证码就在那里了,复制后黏贴到 result>后面即可,回车

For this to work, you will need rclone available on a machine that has a web browser available.
Execute the following on your machine:
        rclone authorize "onedrive"
Then paste the result below:
result> 

此时连接上网盘后,选择 1,回车:

Choose a number from below, or type in an existing value
 1 / OneDrive Personal or Business
   \ "onedrive"
 2 / Sharepoint site
   \ "sharepoint"
 3 / Type in driveID
   \ "driveid"
 4 / Type in SiteID
   \ "siteid"
 5 / Search a Sharepoint site
   \ "search"
Your choice> 1

它查找到网盘信息后会列出来,比如这里输入 0,回车

Found 1 drives, please select the one you want to use:
0: OneDrive (business) id=b!Eqwertyuiopasdfghjklzxcvbnm-7mnbvcxzlkjhgfdsapoiuytrewqk
Chose drive to use:> 0

输入 y,回车

Found drive 'root' of type 'business', URL: https://org-my.sharepoint.com/personal/you/Documents
Is that okay?
y) Yes
n) No
y/n> y

最后会列出所有刚输入的信息让你确认,输入 y,回车后,再输入 q退出,即可完成配置了

--------------------
[remote]
type = onedrive
token = {"access_token":"youraccesstoken","token_type":"Bearer","refresh_token":"yourrefreshtoken","expiry":"2018-08-26T22:39:52.486512262+08:00"}
drive_id = b!Eqwertyuiopasdfghjklzxcvbnm-7mnbvcxzlkjhgfdsapoiuytrewqk
drive_type = business

--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y

继续在服务器终端上测试能否连接到网盘

rclone lsd o: -P -v
#回车后会列出网盘根目录下的所有文件夹名

你可以查看下生成的配置文件,换新服务器的话可以直接复制这个配置文件过去继续使用!

cat ~/.config/rclone/rclone.conf

3.挂载网盘

准备工作

1.在onedrive网盘里新建个文件夹,比如:backups

#可以直接在网盘里操作,也可以在服务器终端上
rclone mkdir o:backups

2.在服务器上新建个文件夹,比如:backups(路径自己定):

mkdir /home/backups

3.在服务器上安装 screen工具

yum install -y screen

4.在服务器上提前安装这个 fuse,不然挂载的时候会提示 Fatal error: failed to mount FUSE fs: fusermount: exec: "fusermount": executable file not found in $PATH

yum install -y fuse

手动挂载

我们现在要使用 screenmount命令,把网盘根目录下的 o:backups挂载到服务器的 /home/backups

在服务器终端输入命令如下:教程来源

/usr/bin/screen -d -m -S rcmount-odrive /usr/bin/rclone mount o:backups /home/backups --allow-other --allow-non-empty --vfs-cache-mode writes

#rcmount-odrive是screen在后台运行的任务窗口名字,你可以改为任意名字方便自己辨别
#o:backups是网盘的文件夹路径,你可以对应改为自己的
#/home/backups是服务器的文件夹路径,你可以对应改为自己的

再检查下是否挂载成功:

df -h

设置开机自启挂载

新建一个开机启动项,名字 rcmount.service要跟Description=name一致

  • 比如你命名rcmount-gdrive.service,里面的Description=rcmount-gdrive
  • 比如你命名rcmount-odrive.service,里面的Description=rcmount-odrive
vim /etc/systemd/system/rcmount.service

## 請輸入以下內容 ##
[Unit]
Description=rcmount
After=network.target

[Service]
User=root
Type=forking
ExecStart=/usr/bin/screen -d -m -S rcmount /usr/bin/rclone mount o:backups /home/backups --allow-other --allow-non-empty --vfs-cache-mode writes

ExecStop=ExecStop=/usr/bin/screen -S rcmount -X quit

[Install]
WantedBy=multi-user.target

Esc,:wq保存退出,接着运行启动项:

systemctl daemon-reload
systemctl enable rcmount.service
systemctl start rcmount.service

最后,你可以重启服务器看看开机有没有自动挂载到网盘。

4.注册serve酱

这里注册并绑定好微信推送:https://sc.ftqq.com/3.version,拿到链接:https://sc.ftqq.com/[SCKEY(登入后可见)].send?text=主人服务器又备份好啦~

text=这里可以自定义通知的标题消息

5.开始增量备份

网站数据量大的话,第一次备份并上传到网盘所花费的时间会很久(没有打包成压缩包的),所以建议开个screen任务窗口在后台运行命令,我们继续接下来的操作吧。第二次增量备份的话就很快了,因为它只会备份有改动的文件。

screen -S backup

#说明
#rclone sync 你的网站目录绝对路径 你的网盘绝对路径 && curl https://sc.ftqq.com/[SCKEY(登入后可见)].send?text=xxx站每日同步备份完成啦

#例子
rclone sync /home/wwwroot/www.sgq.com o:backups/sgq/www.sgq.com && curl https://sc.ftqq.com/keysxxxx.send?text=sgq每日同步备份完成啦

只有第一条命令执行成功了,它才会执行 &&后面的第二条微信通知命令。

6.backup.sh脚本

如果你的建站环境是使用LNMP一键安装包的话(不是的话也可以直接复制一份修改),可以到路径 /root/lnmp1.7/tools下找到该脚本(能实现全站&数据库打包成压缩包并备份),复制一份到家目录下:

cp ~/lnmp1.7/tools/backup.sh ~/

按照脚本头部说明,开始修改备份脚本:

vim ~/backup.sh
  • Backup_Home="上边步骤3新建的服务器备份文件夹的绝对路径"(即已挂载到onedrive网盘的目录,脚本执行时会直接上传到网盘的,所以我就没有启用脚本的FTP备份了)
  • Backup_Dir=("网站目录的绝对路径")
  • Backup_Database=("数据库名字")
  • MYSQL_PassWord='数据库密码'
  • -3day意思是备份保留3天,你可以修改成想要的保留天数
#IMPORTANT!!!Please Setting the following Values!

Backup_Home="/home/backups/"
MySQL_Dump="/usr/local/mysql/bin/mysqldump"
######~Set Directory you want to backup~######
Backup_Dir=("/home/wwwroot/vpser.net" "/home/wwwroot/lnmp.org")

######~Set MySQL Database you want to backup~######
Backup_Database=("lnmp" "vpser")

######~Set MySQL UserName and password~######
MYSQL_UserName='root'
MYSQL_PassWord='yourrootpassword'

######~Enable Ftp Backup~######
Enable_FTP=0
# 0: enable; 1: disable
######~Set FTP Information~######
FTP_Host='1.2.3.4'
FTP_Username='vpser.net'
FTP_Password='yourftppassword'
FTP_Dir="backup"

#Values Setting END!

原脚本

#!/bin/bash

#Funciont: Backup website and mysql database
#Author: licess
#Website: https://lnmp.org

#IMPORTANT!!!Please Setting the following Values!

Backup_Home="/home/backup/"
MySQL_Dump="/usr/local/mysql/bin/mysqldump"
######~Set Directory you want to backup~######
Backup_Dir=("/home/wwwroot/vpser.net" "/home/wwwroot/lnmp.org")

######~Set MySQL Database you want to backup~######
Backup_Database=("lnmp" "vpser")

######~Set MySQL UserName and password~######
MYSQL_UserName='root'
MYSQL_PassWord='yourrootpassword'

######~Enable Ftp Backup~######
Enable_FTP=0
# 0: enable; 1: disable
######~Set FTP Information~######
FTP_Host='1.2.3.4'
FTP_Username='vpser.net'
FTP_Password='yourftppassword'
FTP_Dir="backup"

#Values Setting END!

TodayWWWBackup=www-*-$(date +"%Y%m%d").tar.gz
TodayDBBackup=db-*-$(date +"%Y%m%d").sql
OldWWWBackup=www-*-$(date -d -3day +"%Y%m%d").tar.gz
OldDBBackup=db-*-$(date -d -3day +"%Y%m%d").sql

Backup_Dir()
{
    Backup_Path=$1
    Dir_Name=`echo ${Backup_Path##*/}`
    Pre_Dir=`echo ${Backup_Path}|sed 's/'${Dir_Name}'//g'`
    tar zcf ${Backup_Home}www-${Dir_Name}-$(date +"%Y%m%d").tar.gz -C ${Pre_Dir} ${Dir_Name}
}
Backup_Sql()
{
    ${MySQL_Dump} -u$MYSQL_UserName -p$MYSQL_PassWord $1 > ${Backup_Home}db-$1-$(date +"%Y%m%d").sql
}

if [ ! -f ${MySQL_Dump} ]; then  
    echo "mysqldump command not found.please check your setting."
    exit 1
fi

if [ ! -d ${Backup_Home} ]; then  
    mkdir -p ${Backup_Home}
fi

if [ ${Enable_FTP} = 0 ]; then
    type lftp >/dev/null 2>&1 || { echo >&2 "lftp command not found. Install: centos:yum install lftp,debian/ubuntu:apt-get install lftp."; }
fi

echo "Backup website files..."
for dd in ${Backup_Dir[@]};do
    Backup_Dir ${dd}
done

echo "Backup Databases..."
for db in ${Backup_Database[@]};do
    Backup_Sql ${db}
done

echo "Delete old backup files..."
rm -f ${Backup_Home}${OldWWWBackup}
rm -f ${Backup_Home}${OldDBBackup}

if [ ${Enable_FTP} = 0 ]; then
    echo "Uploading backup files to ftp..."
    cd ${Backup_Home}
    lftp ${FTP_Host} -u ${FTP_Username},${FTP_Password} << EOF
cd ${FTP_Dir}
mrm ${OldWWWBackup}
mrm ${OldDBBackup}
mput ${TodayWWWBackup}
mput ${TodayDBBackup}
bye
EOF

echo "complete."
fi

保存后即可执行该备份脚本了

./backup.sh

7.cron定时备份

完成了以上所有的工作并且测试没有问题后,便可以使用cron命令来添加定时任务,实现自动备份了。

crontab -e

写入以下两条定时任务

#每日22:20定时执行rclone sync备份到网盘,并推送通知到微信
0 22 * * * rclone sync /home/wwwroot/www.sgq.com o:backups/sgq/www.sgq.com && curl https://sc.ftqq.com/key.send?text=sgq每日同步备份完成啦

#每隔3天(在22:10)执行backup.sh脚本全站(包括数据库)备份到网盘,并推送通知到微信
10 22 */3 * * /root/backup.sh && curl https://sc.ftqq.com/key.send?text=sgq全站备份完成

crontab.png

微信通知效果:
serve酱.PNG

Last modification:January 9, 2021
如果觉得我的文章对你有用,请随意赞赏