Chiloh

Chiloh Wei

一个尝试理解世界,理解自己的人
jike
twitter

データ Rclone 定期同期

前回の記事:Typecho ブログスクリプトのバックアップと復元 では、スクリプトを使用してウェブサイトのデータベースを定期的にバックアップする方法について説明しました。今日は、Rclone を使用してバックアップデータを Google ドライブや阿里云 OSS などの S3 ストレージに自動的に同期する方法を共有します。

一、Rclone のインストール#

サーバーのターミナルにログインし、以下のコマンドを実行します:

curl https://rclone.org/install.sh | sudo bash

以前にcurlをインストールしていない場合は、以下のコマンドを実行してインストールを完了してください:

yum -y install curl

二、Rclone の設定#

インストールが成功したら、次のコマンドを入力します:

rclone config

以下の操作コマンドが表示されますので、指示に従って操作してください:

2021/05/09 12:48:01 NOTICE: Config file "/root/.config/rclone/rclone.conf" not found - using defaults
No remotes found - make a new one
n) New remote
s) Set configuration password
q) Quit config

nを入力して新規作成し、nameをカスタマイズした後、接続するストレージが表示されます:

name> Alibaba
Type of storage to configure.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / 1Fichier
   \ "fichier"
 2 / Alias for an existing remote
   \ "alias"
 3 / Amazon Drive
   \ "amazon cloud drive"
 4 / Amazon S3 Compliant Storage Providers including AWS, Alibaba, Ceph, Digital Ocean, Dreamhost, IBM COS, Minio, and Tencent COS
   \ "s3"
 5 / Backblaze B2
   \ "b2"
 6 / Box
   \ "box"
 7 / Cache a remote
   \ "cache"
 8 / Citrix Sharefile
   \ "sharefile"
 9 / Compress a remote
   \ "compress"
10 / Dropbox
   \ "dropbox"
11 / Encrypt/Decrypt a remote
   \ "crypt"
12 / Enterprise File Fabric
   \ "filefabric"
13 / FTP Connection
   \ "ftp"
14 / Google Cloud Storage (this is not Google Drive)
   \ "google cloud storage"
15 / Google Drive
   \ "drive"
16 / Google Photos
   \ "google photos"
17 / Hadoop distributed file system
   \ "hdfs"
18 / Hubic
   \ "hubic"
19 / In memory object storage system.
   \ "memory"
20 / Jottacloud
   \ "jottacloud"
21 / Koofr
   \ "koofr"
22 / Local Disk
   \ "local"
23 / Mail.ru Cloud
   \ "mailru"
24 / Mega
   \ "mega"
25 / Microsoft Azure Blob Storage
   \ "azureblob"
26 / Microsoft OneDrive
   \ "onedrive"
27 / OpenDrive
   \ "opendrive"
28 / OpenStack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
   \ "swift"
29 / Pcloud
   \ "pcloud"
30 / Put.io
   \ "putio"
31 / QingCloud Object Storage
   \ "qingstor"
32 / SSH/SFTP Connection
   \ "sftp"
33 / Sugarsync
   \ "sugarsync"
34 / Tardigrade Decentralized Cloud Storage
   \ "tardigrade"
35 / Transparently chunk/split large files
   \ "chunker"
36 / Union merges the contents of several upstream fs
   \ "union"
37 / Webdav
   \ "webdav"
38 / Yandex Disk
   \ "yandex"
39 / Zoho
   \ "zoho"
40 / http Connection
   \ "http"
41 / premiumize.me
   \ "premiumizeme"
42 / seafile
   \ "seafile"

阿里云 OSS を例に、ターミナルで4を入力し、次に2を選択し、以下の指示に従って操作します:

Storage> 4
** See help for s3 backend at: https://rclone.org/s3/ **

Choose your S3 provider.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / Amazon Web Services (AWS) S3
   \ "AWS"
 2 / Alibaba Cloud Object Storage System (OSS) formerly Aliyun
   \ "Alibaba"
 3 / Ceph Object Storage
   \ "Ceph"
 4 / Digital Ocean Spaces
   \ "DigitalOcean"
 5 / Dreamhost DreamObjects
   \ "Dreamhost"
 6 / IBM COS S3
   \ "IBMCOS"
 7 / Minio Object Storage
   \ "Minio"
 8 / Netease Object Storage (NOS)
   \ "Netease"
 9 / Scaleway Object Storage
   \ "Scaleway"
10 / StackPath Object Storage
   \ "StackPath"
11 / Tencent Cloud Object Storage (COS)
   \ "TencentCOS"
12 / Wasabi Object Storage
   \ "Wasabi"
13 / Any other S3 compatible provider
   \ "Other"
provider> 2
Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
Only applies if access_key_id and secret_access_key is blank.
Enter a boolean value (true or false). Press Enter for the default ("false").
Choose a number from below, or type in your own value
 1 / Enter AWS credentials in the next step
   \ "false"
 2 / Get AWS credentials from the environment (env vars or IAM)
   \ "true"
env_auth> 1
AWS Access Key ID.
Leave blank for anonymous access or runtime credentials.
Enter a string value. Press Enter for the default ("").
access_key_id> 
AWS Secret Access Key (password)
Leave blank for anonymous access or runtime credentials.
Enter a string value. Press Enter for the default ("").
secret_access_key> 
Endpoint for OSS API.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / East China 1 (Hangzhou)
   \ "oss-cn-hangzhou.aliyuncs.com"
 2 / East China 2 (Shanghai)
   \ "oss-cn-shanghai.aliyuncs.com"
 3 / North China 1 (Qingdao)
   \ "oss-cn-qingdao.aliyuncs.com"
 4 / North China 2 (Beijing)
   \ "oss-cn-beijing.aliyuncs.com"
 5 / North China 3 (Zhangjiakou)
   \ "oss-cn-zhangjiakou.aliyuncs.com"
 6 / North China 5 (Huhehaote)
   \ "oss-cn-huhehaote.aliyuncs.com"
 7 / South China 1 (Shenzhen)
   \ "oss-cn-shenzhen.aliyuncs.com"
 8 / Hong Kong (Hong Kong)
   \ "oss-cn-hongkong.aliyuncs.com"
 9 / US West 1 (Silicon Valley)
   \ "oss-us-west-1.aliyuncs.com"
10 / US East 1 (Virginia)
   \ "oss-us-east-1.aliyuncs.com"
11 / Southeast Asia Southeast 1 (Singapore)
   \ "oss-ap-southeast-1.aliyuncs.com"
12 / Asia Pacific Southeast 2 (Sydney)
   \ "oss-ap-southeast-2.aliyuncs.com"
13 / Southeast Asia Southeast 3 (Kuala Lumpur)
   \ "oss-ap-southeast-3.aliyuncs.com"
14 / Asia Pacific Southeast 5 (Jakarta)
   \ "oss-ap-southeast-5.aliyuncs.com"
15 / Asia Pacific Northeast 1 (Japan)
   \ "oss-ap-northeast-1.aliyuncs.com"
16 / Asia Pacific South 1 (Mumbai)
   \ "oss-ap-south-1.aliyuncs.com"
17 / Central Europe 1 (Frankfurt)
   \ "oss-eu-central-1.aliyuncs.com"
18 / West Europe (London)
   \ "oss-eu-west-1.aliyuncs.com"
19 / Middle East 1 (Dubai)
   \ "oss-me-east-1.aliyuncs.com"
endpoint> 8
Canned ACL used when creating buckets and storing or copying objects.

This ACL is used for creating objects and if bucket_acl isn't set, for creating buckets too.

For more info visit https://docs.aws.amazon.com/AmazonS3/latest/dev/acl-overview.html#canned-acl

Note that this ACL is applied when server-side copying objects as S3
doesn't copy the ACL from the source but rather writes a fresh one.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / Owner gets FULL_CONTROL. No one else has access rights (default).
   \ "private"
 2 / Owner gets FULL_CONTROL. The AllUsers group gets READ access.
   \ "public-read"
   / Owner gets FULL_CONTROL. The AllUsers group gets READ and WRITE access.
 3 | Granting this on a bucket is generally not recommended.
   \ "public-read-write"
 4 / Owner gets FULL_CONTROL. The AuthenticatedUsers group gets READ access.
   \ "authenticated-read"
   / Object owner gets FULL_CONTROL. Bucket owner gets READ access.
 5 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
   \ "bucket-owner-read"
   / Both the object owner and the bucket owner get FULL_CONTROL over the object.
 6 | If you specify this canned ACL when creating a bucket, Amazon S3 ignores it.
   \ "bucket-owner-full-control"
acl> 3
The storage class to use when storing new objects in OSS.
Enter a string value. Press Enter for the default ("").
Choose a number from below, or type in your own value
 1 / Default
   \ ""
 2 / Standard storage class
   \ "STANDARD"
 3 / Archive storage mode.
   \ "GLACIER"
 4 / Infrequent access storage mode.
   \ "STANDARD_IA"
storage_class> 
Edit advanced config? (y/n)
y) Yes
n) No (default)
y/n> n
Remote config
--------------------
[gdrive]
type = s3
provider = Alibaba
env_auth = false
endpoint = oss-cn-hongkong.aliyuncs.com
acl = public-read-write
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y

阿里云 OSS の ID とキーを入力する必要がありますが、最初はエンターを押してスキップし、阿里云で設定した後にこの部分を編集できます。その後、指示に従って阿里云 OSS のノード位置と読み書き権限を選択します。上記の選択を参考にしてください(香港ノード、公共読み書き権限の例)。他のノードの場合は、対応する番号を選択してください。

三、阿里云 API の設定#

阿里云のウェブサイトにログインし、右上のアバターをクリックしてアクセス制御に入り、新しいユーザーを作成し、プログラミングアクセスを選択し、AliyunOSSFullAccess権限を追加します。

ユーザー作成

これで阿里云 OSS サービスの ID とキーを取得できます。

四、Rclone の設定を続ける#

再度ターミナルでrclone configコマンドを入力し、以下の操作に従って編集モードに入り、阿里云 OSS の ID とキーを補充します。

Current remotes:

Name                 Type
====                 ====
Alibaba              s3

e) Edit existing remote
n) New remote
d) Delete remote
r) Rename remote
c) Copy remote
s) Set configuration password
q) Quit config
e/n/d/r/c/s/q> e
[Alibaba]
type = s3
provider = Alibaba
env_auth = false
endpoint = oss-cn-hongkong.aliyuncs.com
acl = public-read-write
--------------------
Edit remote
** See help for s3 backend at: https://rclone.org/s3/ **

Value "provider" = "Alibaba"
Edit? (y/n)>
y) Yes
n) No (default)
y/n> n
Value "env_auth" = "false"
Edit? (y/n)>
y) Yes
n) No (default)
y/n> y
Get AWS credentials from runtime (environment variables or EC2/ECS meta data if no env vars).
Only applies if access_key_id and secret_access_key is blank.
Enter a boolean value (true or false). Press Enter for the default ("false").
Choose a number from below, or type in your own value
 1 / Enter AWS credentials in the next step
   \ "false"
 2 / Get AWS credentials from the environment (env vars or IAM)
   \ "true"
env_auth> 1
Value "access_key_id" = ""
Edit? (y/n)>
y) Yes
n) No (default)
y/n> y
AWS Access Key ID.
Leave blank for anonymous access or runtime credentials.
Enter a string value. Press Enter for the default ("").
access_key_id> ここに取得した阿里云OSS IDを入力
Value "secret_access_key" = ""
Edit? (y/n)>
y) Yes
n) No (default)
y/n> y
AWS Secret Access Key (password)
Leave blank for anonymous access or runtime credentials.
Enter a string value. Press Enter for the default ("").
secret_access_key> ここに取得した阿里云OSS キーを入力
Value "endpoint" = "oss-cn-hongkong.aliyuncs.com"
Edit? (y/n)>
y) Yes
n) No (default)
y/n> n
Value "acl" = "public-read-write"
Edit? (y/n)>
y) Yes
n) No (default)
y/n> n
Value "storage_class" = ""
Edit? (y/n)
y) Yes
n) No (default)
y/n> n
Edit advanced config? (y/n)
y) Yes
n) No (default)
y/n> n
--------------------
[Alibaba]
type = s3
provider = Alibaba
env_auth = false
endpoint = oss-cn-hongkong.aliyuncs.com
acl = public-read-write
access_key_id = 阿里云OSS ID
secret_access_key = 阿里云OSSキー
--------------------
y) Yes this is OK (default)
e) Edit this remote
d) Delete this remote
y/e/d> y

これで Rclone の阿里云 OSS 接続設定がすべて完了しました。Google Drive など他のストレージに接続する場合も、同様の手順です。

五、スクリプトの修正#

バックアップスクリプトの後ろに以下を追加します:

rclone copy /path/backup Alibaba:BucketName

ここで/path/backupはデータバックアップのディレクトリを示し、Alibaba:BucketNameAlibabarclone操作中に新しく作成したremote名であり、後ろのBucketNameはあなたの阿里云 OSS のストレージバケット名です。

阿里云 OSS にデータを定期的に自動同期したい場合は、ターミナルでcrontab -eを入力し、次のように入力します:

0 18 * * 5 /bin/bash /root/jiaoben/backup.sh

ここで:0 18 * * 5は毎週金曜日の午後 6 時にコマンドを実行することを示し、root bashは root 権限で bash コマンドを実行することを示し、/path/backup.shはバックアップスクリプトの保存場所を示し、/path/wwwroot/domain.comはウェブサイトのルートディレクトリを示します。

読み込み中...
文章は、創作者によって署名され、ブロックチェーンに安全に保存されています。