rclone/docs/content/b2.md

4.7 KiB

title description date
B2 Backblaze B2 2016-06-15

Backblaze B2

B2 is Backblaze's cloud storage system.

Paths are specified as remote:bucket (or remote: for the lsd
command.) You may put subdirectories in too, eg remote:bucket/path/to/dir.

Here is an example of making a b2 configuration. First run

rclone config

This will guide you through an interactive setup process. You will
need your account number (a short hex number) and key (a long hex
number) which you can get from the b2 control panel.

No remotes found - make a new one
n) New remote
q) Quit config
n/q> n
name> remote
Type of storage to configure.
Choose a number from below, or type in your own value
 1 / Amazon Cloud Drive
   \ "amazon cloud drive"
 2 / Amazon S3 (also Dreamhost, Ceph)
   \ "s3"
 3 / Backblaze B2
   \ "b2"
 4 / Dropbox
   \ "dropbox"
 5 / Google Cloud Storage (this is not Google Drive)
   \ "google cloud storage"
 6 / Google Drive
   \ "drive"
 7 / Hubic
   \ "hubic"
 8 / Local Disk
   \ "local"
 9 / Microsoft OneDrive
   \ "onedrive"
10 / Openstack Swift (Rackspace Cloud Files, Memset Memstore, OVH)
   \ "swift"
11 / Yandex Disk
   \ "yandex"
Storage> 3
Account ID
account> 123456789abc
Application Key
key> 0123456789abcdef0123456789abcdef0123456789
Endpoint for the service - leave blank normally.
endpoint> 
Remote config
--------------------
[remote]
account = 123456789abc
key = 0123456789abcdef0123456789abcdef0123456789
endpoint = 
--------------------
y) Yes this is OK
e) Edit this remote
d) Delete this remote
y/e/d> y

This remote is called remote and can now be used like this

See all buckets

rclone lsd remote:

Make a new bucket

rclone mkdir remote:bucket

List the contents of a bucket

rclone ls remote:bucket

Sync /home/local/directory to the remote bucket, deleting any
excess files in the bucket.

rclone sync /home/local/directory remote:bucket

Modified time

The modified time is stored as metadata on the object as
X-Bz-Info-src_last_modified_millis as milliseconds since 1970-01-01
in the Backblaze standard. Other tools should be able to use this as
a modified time.

Modified times are used in syncing and are fully supported except in
the case of updating a modification time on an existing object. In
this case the object will be uploaded again as B2 doesn't have an API
method to set the modification time independent of doing an upload.

SHA1 checksums

The SHA1 checksums of the files are checked on upload and download and
will be used in the syncing process. You can use the --checksum flag.

Large files which are uploaded in chunks will store their SHA1 on the
object as X-Bz-Info-large_file_sha1 as recommended by Backblaze.

Versions

When rclone uploads a new version of a file it creates a new version
of it
.
Likewise when you delete a file, the old version will still be
available.

The old versions of files are visible in the B2 web interface, but not
via rclone yet.

Rclone doesn't provide any way of managing old versions (downloading
them or deleting them) at the moment. When you purge a bucket, all
the old versions will be deleted.

Transfers

Backblaze recommends that you do lots of transfers simultaneously for
maximum speed. In tests from my SSD equiped laptop the optimum
setting is about --transfers 32 though higher numbers may be used
for a slight speed improvement. The optimum number for you may vary
depending on your hardware, how big the files are, how much you want
to load your computer, etc. The default of --transfers 4 is
definitely too low for Backblaze B2 though.

Specific options

Here are the command line options specific to this cloud storage
system.

--b2-chunk-size valuee=SIZE

When uploading large files chunk the file into this size. Note that
these chunks are buffered in memory. 100,000,000 Bytes is the minimim
size (default 96M).

--b2-upload-cutoff=SIZE

Cutoff for switching to chunked upload (default 4.657GiB ==
5GB). Files above this size will be uploaded in chunks of
--b2-chunk-size. The default value is the largest file which can be
uploaded without chunks.

--b2-test-mode=FLAG

This is for debugging purposes only.

Setting FLAG to one of the strings below will cause b2 to return
specific errors for debugging purposes.

  • fail_some_uploads
  • expire_some_account_authorization_tokens
  • force_cap_exceeded

These will be set in the X-Bz-Test-Mode header which is documented
in the b2 integrations
checklist
.