鉴于上次的服务器崩溃事件,写一个数据自动备份脚本就显得尤为重要。折腾了一晚,最终实现效果是每晚三点自动备份数据库和网站等数据并使用 Openssl 加密归档,保留最近一个月的数据,通过开源的 rclone 工具同步到 Google Drive 和 Onedrive 上,这样就万无一失了🍟
1. Shell 脚本
脚本是自己使用,因此比较简单。
#!/bin/bash
# Writen by Atp on Nov 15, 2020
# Website: https://atpx.com
DATE=`date +%Y%m%d`
BACKUP_TEMP_PATH=/backup/tmp
# Database
DB_NAME=
DB_USR=
DB_PW=
DB_OUT_NAME="db_$DATE"
# Website
WEB_PATH=
WEB_OUT_NAME="web_$DATE"
# Archive
ARCHIVE_PATH=/backup/archives
ARCHIVE_LOG_FILE=/backup/log/backup.log
# Backup mysql
# mysqldump -u$DB_USR -p$DB_PW --databases $DB_NAME > $BACKUP_TEMP_PATH/$DB_OUT_NAME.sql
mysqldump --login-path=$DB_USR --databases $DB_NAME > $BACKUP_TEMP_PATH/$DB_OUT_NAME.sql
# Backup webdsite
cd $WEB_PATH
zip -rq $BACKUP_TEMP_PATH/$WEB_OUT_NAME.zip *
# Archive files
cd $BACKUP_TEMP_PATH
# tar cf - * | openssl enc -e -aes-128-cbc -pbkdf2 -k [password] -out $ARCHIVE_PATH/$DATE.tar.aes
tar cf - * | gpg -e -r [recipient] -o $ARCHIVE_PATH/$DATE.tar.gpg
# Backup the latest 30 days
rm $BACKUP_TEMP_PATH/*
find $ARCHIVE_PATH -name "*.tar.gpg" -type f -mtime +30 -exec rm {} \;
# Sync to Google Drive
rclone sync $ARCHIVE_PATH gdrive:[sync path]
# Sync to Microsoft OneDrive
rclone sync $ARCHIVE_PATH onedrive:[sync path]
# Output log file
echo -e "\e[1;32m---------------------------------------------\e[0m" >> $ARCHIVE_LOG_FILE
echo -e "Backup date: "$(date +%Y-%m-%d)"\t""Finish time: "$(date +%H:%M:%S) >> $ARCHIVE_LOG_FILE
echo -e "\e[1;32m---------------------------------------------\e[0m\n" >> $ARCHIVE_LOG_FILE
这里说明一下,使用mysqldump备份的时候,如果通过账号密码导出会提示使用明文密码不安全的警告,虽然不影响使用,但看着就不爽,因此改用了 MySQL5.7 之后提供的新方式,通过 --login-path=
的方式登录数据库,首先保存一个加密验证信息:
mysql_config_editor set --login-path=[name] --host=localhost --user=[usr] --password
查看所设置的信息:
mysql_config_editor print --all
删除指定信息:
mysql_config_editor remove --login-path=[usr]
值得一提的,我第一次设置的时候毫无疑问的失败了- -,一直无法登录,后来才知道所设置的用户密码中不能包括"#“符号,不然就会报错。
2. 配置 rclone
官方有一键安装脚本,适用于 Linux/macOS/BSD 系统:
curl https://rclone.org/install.sh | sudo bash
之后需要在服务器上配置 Google Dirve 和 OneDrive 的连接,输入 rclone config
命令,根据提示的信息进行操作,下面演示的版本为 v1.57.0,不同版本可能会略有出入,但思路是一样的。
Google Drive
- using defaults
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n # 输入 n,新建远程连接
name> gdrive # 输入自定义名称,用于区分不同连接
Option Storage.
Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value.
1 / 1Fichier
\ "fichier"
2 / Alias for an existing remote
\ "alias"
3 / Amazon Drive
\ "amazon cloud drive"
4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, SeaweedFS, and Tencent COS
\ "s3"
5 / Backblaze B2
\ "b2"
6 / Better checksums for other remotes
\ "hasher"
7 / Box
\ "box"
8 / Cache a remote
\ "cache"
9 / Citrix Sharefile
\ "sharefile"
10 / Compress a remote
\ "compress"
11 / Dropbox
\ "dropbox"
12 / Encrypt/Decrypt a remote
\ "crypt"
13 / Enterprise File Fabric
\ "filefabric"
14 / FTP Connection
\ "ftp"
15 / Google Cloud Storage (this is not Google Drive)
\ "google cloud storage"
16 / Google Drive
\ "drive"
17 / Google Photos
\ "google photos"
18 / Hadoop distributed file system
\ "hdfs"
19 / Hubic
\ "hubic"
20 / In memory object storage system.
\ "memory"
21 / Jottacloud
\ "jottacloud"
22 / Koofr
\ "koofr"
23 / Local Disk
\ "local"
24 / Mail.ru Cloud
\ "mailru"
25 / Mega
\ "mega"
26 / Microsoft Azure Blob Storage
\ "azureblob"
27 / Microsoft OneDrive
\ "onedrive"
28 / OpenDrive
\ "opendrive"
29 / OpenStack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
\ "swift"
30 / Pcloud
\ "pcloud"
31 / Put.io
\ "putio"
32 / QingCloud Object Storage
\ "qingstor"
33 / SSH/SFTP Connection
\ "sftp"
34 / Sia Decentralized Cloud
\ "sia"
35 / Sugarsync
\ "sugarsync"
36 / Tardigrade Decentralized Cloud Storage
\ "tardigrade"
37 / Transparently chunk/split large files
\ "chunker"
38 / Union merges the contents of several upstream fs
\ "union"
39 / Uptobox
\ "uptobox"
40 / Webdav
\ "webdav"
41 / Yandex Disk
\ "yandex"
42 / Zoho
\ "zoho"
43 / http Connection
\ "http"
44 / premiumize.me
\ "premiumizeme"
45 / seafile
\ "seafile"
Storage> 16 # 找到 Google Drive 的序号,我这里的版本是 16
Option client_id.
Google Application Client Id
Setting your own is recommended.
See https://rclone.org/drive/#making-your-own-client-id for how to create your own.
If you leave this blank, it will use an internal key which is low performance.
Enter a string value. Press Enter for the default ("").
client_id> # 一般留空,直接回车,如果你需要同步大量文件可以自己去 Google Cloud 申请一个 client_id 而不使用 rclone 的 api 减少限制
Option client_secret.
OAuth Client Secret.
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_secret> # 一般留空,直接回车
Option scope.
Scope that rclone should use when requesting access from drive.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value.
1 / Full access all files, excluding Application Data Folder.
\ "drive"
2 / Read-only access to file metadata and file contents.
\ "drive.readonly"
/ Access to files created by rclone only.
3 | These are visible in the drive website.
| File authorization is revoked when the user deauthorizes the app.
\ "drive.file"
/ Allows read and write access to the Application Data folder.
4 | This is not visible in the drive website.
\ "drive.appfolder"
/ Allows read-only access to file metadata but
5 | does not allow any access to read or download file content.
\ "drive.metadata.readonly"
scope> 1 # 根据你的需求选择访问权限,这里选择完全权限
Option root_folder_id.
ID of the root folder.
Leave blank normally.
Fill in to access "Computers" folders (see docs), or for rclone to use
a non root folder as its starting point.
Enter a string value. Press Enter for the default ("").
root_folder_id> # 一般留空,直接回车
Option service_account_file.
Service Account Credentials JSON file path.
Leave blank normally.
Needed only if you want use SA instead of interactive login.
Leading `~` will be expanded in the file name as will environment variables such as `${RCLONE_CONFIG_DIR}`.
Enter a string value. Press Enter for the default ("").
service_account_file> # 一般留空,直接回车
Edit advanced config?
y) Yes
n) No (default)
y/n> n # 输入 n,不进行高级设置
Use auto config?
* Say Y if not sure
* Say N if you are working on a remote or headless machine
y) Yes (default)
n) No
y/n> n # 输入 n
Option config_verification_code.
Verification code
Go to this URL, authenticate then paste the code here.
https://accounts.google.com/o/oauth2/xxxxxx
Enter a string value. Press Enter for the default ("").
config_verification_code> # 输入访问上述网址后的 token
Configure this as a Shared Drive (Team Drive)?
y) Yes
n) No (default)
y/n> n # 是否配置为团队盘
--------------------
[gdrive]
type = drive
scope = drive
token = {"access_token":""}
team_drive =
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y # 上述配置是否正确,输入 y 确认即可完成配置
Current remotes:
Name Type
==== ====
gdrive drive
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q # 输入 q 退出配置
OneDrive
连接 OneDrive 的 token 需要在本地 Windows 电脑上下载 Windows 版 rclone 获取,下载后解压,在 rclone 文件夹中的资源管理器地址栏输入 cmd,打开命令提示符,输入 rclone 配置中提示的命令:
rclone authorize "onedrive"
接下来会弹出浏览器,登录账号进行授权,然后返回 cmd 窗口会出现类似信息:
2021/11/27 21:53:38 NOTICE: Log in and authorize rclone for access
2021/11/27 21:53:38 NOTICE: Waiting for code...
2021/11/27 21:53:42 NOTICE: Got code
Paste the following into your remote machine --->
xxxx
<---End paste
复制保存对应 “xxxx” 的所有内容,接下来会用到,然后回到服务器上输入 rclone config
进行配置:
- using defaults
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config
n/s/q> n # 输入 n,新建远程连接
name> onedrive # 输入自定义名称,用于区分不同连接
Option Storage.
Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value.
1 / 1Fichier
\ "fichier"
2 / Alias for an existing remote
\ "alias"
3 / Amazon Drive
\ "amazon cloud drive"
4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, SeaweedFS, and Tencent COS
\ "s3"
5 / Backblaze B2
\ "b2"
6 / Better checksums for other remotes
\ "hasher"
7 / Box
\ "box"
8 / Cache a remote
\ "cache"
9 / Citrix Sharefile
\ "sharefile"
10 / Compress a remote
\ "compress"
11 / Dropbox
\ "dropbox"
12 / Encrypt/Decrypt a remote
\ "crypt"
13 / Enterprise File Fabric
\ "filefabric"
14 / FTP Connection
\ "ftp"
15 / Google Cloud Storage (this is not Google Drive)
\ "google cloud storage"
16 / Google Drive
\ "drive"
17 / Google Photos
\ "google photos"
18 / Hadoop distributed file system
\ "hdfs"
19 / Hubic
\ "hubic"
20 / In memory object storage system.
\ "memory"
21 / Jottacloud
\ "jottacloud"
22 / Koofr
\ "koofr"
23 / Local Disk
\ "local"
24 / Mail.ru Cloud
\ "mailru"
25 / Mega
\ "mega"
26 / Microsoft Azure Blob Storage
\ "azureblob"
27 / Microsoft OneDrive
\ "onedrive"
28 / OpenDrive
\ "opendrive"
29 / OpenStack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
\ "swift"
30 / Pcloud
\ "pcloud"
31 / Put.io
\ "putio"
32 / QingCloud Object Storage
\ "qingstor"
33 / SSH/SFTP Connection
\ "sftp"
34 / Sia Decentralized Cloud
\ "sia"
35 / Sugarsync
\ "sugarsync"
36 / Tardigrade Decentralized Cloud Storage
\ "tardigrade"
37 / Transparently chunk/split large files
\ "chunker"
38 / Union merges the contents of several upstream fs
\ "union"
39 / Uptobox
\ "uptobox"
40 / Webdav
\ "webdav"
41 / Yandex Disk
\ "yandex"
42 / Zoho
\ "zoho"
43 / http Connection
\ "http"
44 / premiumize.me
\ "premiumizeme"
45 / seafile
\ "seafile"
Storage> 27 # 找到 OneDrive 的序号,我这里的版本是 27
Option client_id.
Google Application Client Id
Setting your own is recommended.
See https://rclone.org/drive/#making-your-own-client-id for how to create your own.
If you leave this blank, it will use an internal key which is low performance.
Enter a string value. Press Enter for the default ("").
client_id> # 一般留空,直接回车
Option client_secret.
OAuth Client Secret.
Leave blank normally.
Enter a string value. Press Enter for the default ("").
client_secret> # 一般留空,直接回车
Option region.
Choose national cloud region for OneDrive.
Enter a string value. Press Enter for the default ("global").
Choose a number from below, or type in your own value.
1 / Microsoft Cloud Global
\ "global"
2 / Microsoft Cloud for US Government
\ "us"
3 / Microsoft Cloud Germany
\ "de"
4 / Azure and Office 365 operated by 21Vianet in China
\ "cn"
region> 1 # 选择你的 OneDrive 区域
Edit advanced config?
y) Yes
n) No (default)
y/n> n # 输入 n,不进行高级设置
Use auto config?
* Say Y if not sure
* Say N if you are working on a remote or headless machine
y) Yes (default)
n) No
y/n> n # 输入 n
Option config_token.
For this to work, you will need rclone available on a machine that has
a web browser available.
For more help and alternate methods see: https://rclone.org/remote_setup/
Execute the following on the machine with the web browser (same rclone
version recommended):
rclone authorize "onedrive"
Then paste the result.
Enter a string value. Press Enter for the default ("").
config_token> # 粘贴前面在 Windows 上获取的内容
Option config_type.
Type of connection
Enter a string value. Press Enter for the default ("onedrive").
Choose a number from below, or type in an existing value.
1 / OneDrive Personal or Business
\ "onedrive"
2 / Root Sharepoint site
\ "sharepoint"
/ Sharepoint site name or URL
3 | E.g. mysite or https://contoso.sharepoint.com/sites/mysite
\ "url"
4 / Search for a Sharepoint site
\ "search"
5 / Type in driveID (advanced)
\ "driveid"
6 / Type in SiteID (advanced)
\ "siteid"
/ Sharepoint server-relative path (advanced)
7 | E.g. /teams/hr
\ "path"
config_type> 1 # 输入你的 OneDrive 类型
Drive OK?
Found drive "root" of type "personal"
URL: https://onedrive.live.com/?cid=xxxxx
y) Yes (default)
n) No
y/n> y # 输入 y
--------------------
[onedrive]
type = onedrive
client_id =
client_secret =
token = {"access_token":"","expiry":""}
drive_id =
drive_type = personal
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y # 输入 y 确认
Current remotes:
Name Type
==== ====
gdrive drive
onedrive onedrive
e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> q # 输入 q 退出配置
测试配置
配置完成后,你可以同步一个简单的文件进行测试,比如在 /home/rclone-test
目录下新建一个 1.txt
文件,然后使用同步命令:
rclone sync /home/rclone-test onedrive:rclone-test/temp
然后登录你的 OneDrive 查看 rclone-test/temp
目录下是否有 1.txt
文件。
至于 rclone 完整的命令操作,可以查看官方文档。
3. 定时任务
设置好 rclone 之后就该定时执行备份脚本了,这里通过 Crontab 设置定时任务,每晚两点执行备份脚本,记得给脚本可执行权限:
crontab -e
00 2 * * * /backup/backup.sh
使用说明:
分 时 日 月 周 执行命令
第 1 列:分钟 0-59,每分钟用 * 或者 */1 表示,整点分钟数为 00 或 0
第 2 列:小时 0-23(0 表示 0 点)
第 3 列:日 1-31
第 4 列:月 1-12
第 5 列:星期 0-6(0 表示星期天)
第 6 列:运行的命令
Updated 2020-11-21
经研究发现,Openssl 并不适合用于文档加密,因此改用 GnuPG
如果你认为这篇文章还不错,可以考虑为我充电 ⚡️