nielash 87ec26001f local: add server-side copy with xattrs on macOS (part-fix #1710)
Before this change, macOS-specific metadata was not preserved by rclone, even for
local-to-local transfers (it does not use the "user." prefix, nor is Mac metadata
limited to xattrs.) Additionally, rclone did not take advantage of APFS's native
"cloning" functionality for fast and deduplicated transfers.

After this change, local (on macOS only) supports "server-side copy" similarly to
other remotes, and achieves this by using (when possible) macOS's native APFS
"cloning", which is the same underlying mechanism deployed when a user
duplicates a file via the Finder UI. This has several advantages over the
previous behavior:

- It is extremely fast (even large files can be cloned instantly)
- It is very efficient in terms of storage, as it automatically deduplicates when
possible (i.e. so that having two identical files does not consume more storage
than having just one.) (The concept is similar to a "hard link", but subsequent
modifications will not affect the original file.)
- It preserves Mac-specific metadata to the maximum degree, including not only
xattrs but also metadata not easily settable by other methods, including Finder
and Spotlight params.

When server-side "clone" is not available (for example, on non-APFS volumes), it
falls back to server-side "copy" (still preserving metadata but using more disk
storage.) It is only used when both remotes are local (and not wrapped by other
remotes, such as crypt.) The behavior of local on non-mac systems is unchanged.
2024-08-15 15:36:38 +01:00
2024-03-10 11:22:43 +00:00
2024-08-14 21:55:26 +01:00
2024-06-15 13:08:20 +01:00
2023-08-29 13:01:15 +01:00
2023-11-19 12:48:27 +00:00
2024-01-03 17:43:23 +00:00
2024-06-14 16:04:51 +01:00
2024-06-14 16:04:51 +01:00
2024-06-14 16:04:51 +01:00
2024-06-14 16:04:51 +01:00
2020-10-28 18:16:23 +00:00
2024-06-14 16:04:51 +01:00
2019-09-29 11:05:10 +01:00
2024-05-22 15:50:31 +01:00
2024-06-14 17:54:27 +01:00


rclone logo rclone logo

Website | Documentation | Download | Contributing | Changelog | Installation | Forum

Build Status Go Report Card GoDoc Docker Pulls

Rclone

Rclone ("rsync for cloud storage") is a command-line program to sync files and directories to and from different cloud storage providers.

Storage providers

  • 1Fichier 📄
  • Akamai Netstorage 📄
  • Alibaba Cloud (Aliyun) Object Storage System (OSS) 📄
  • Amazon S3 📄
  • ArvanCloud Object Storage (AOS) 📄
  • Backblaze B2 📄
  • Box 📄
  • Ceph 📄
  • China Mobile Ecloud Elastic Object Storage (EOS) 📄
  • Cloudflare R2 📄
  • Citrix ShareFile 📄
  • DigitalOcean Spaces 📄
  • Digi Storage 📄
  • Dreamhost 📄
  • Dropbox 📄
  • Enterprise File Fabric 📄
  • Fastmail Files 📄
  • FTP 📄
  • GoFile 📄
  • Google Cloud Storage 📄
  • Google Drive 📄
  • Google Photos 📄
  • HDFS (Hadoop Distributed Filesystem) 📄
  • HiDrive 📄
  • HTTP 📄
  • Huawei Cloud Object Storage Service(OBS) 📄
  • ImageKit 📄
  • Internet Archive 📄
  • Jottacloud 📄
  • IBM COS S3 📄
  • IONOS Cloud 📄
  • Koofr 📄
  • Leviia Object Storage 📄
  • Liara Object Storage 📄
  • Linkbox 📄
  • Linode Object Storage 📄
  • Magalu Object Storage 📄
  • Mail.ru Cloud 📄
  • Memset Memstore 📄
  • Mega 📄
  • Memory 📄
  • Microsoft Azure Blob Storage 📄
  • Microsoft Azure Files Storage 📄
  • Microsoft OneDrive 📄
  • Minio 📄
  • Nextcloud 📄
  • OVH 📄
  • Blomp Cloud Storage 📄
  • OpenDrive 📄
  • OpenStack Swift 📄
  • Oracle Cloud Storage 📄
  • Oracle Object Storage 📄
  • ownCloud 📄
  • pCloud 📄
  • Petabox 📄
  • PikPak 📄
  • Pixeldrain 📄
  • premiumize.me 📄
  • put.io 📄
  • Proton Drive 📄
  • QingStor 📄
  • Qiniu Cloud Object Storage (Kodo) 📄
  • Quatrix 📄
  • Rackspace Cloud Files 📄
  • RackCorp Object Storage 📄
  • Scaleway 📄
  • Seafile 📄
  • SeaweedFS 📄
  • SFTP 📄
  • SMB / CIFS 📄
  • StackPath 📄
  • Storj 📄
  • SugarSync 📄
  • Synology C2 Object Storage 📄
  • Tencent Cloud Object Storage (COS) 📄
  • Uloz.to 📄
  • Wasabi 📄
  • WebDAV 📄
  • Yandex Disk 📄
  • Zoho WorkDrive 📄
  • The local filesystem 📄

Please see the full list of all storage providers and their features

Virtual storage providers

These backends adapt or modify other storage providers

  • Alias: rename existing remotes 📄
  • Cache: cache remotes (DEPRECATED) 📄
  • Chunker: split large files 📄
  • Combine: combine multiple remotes into a directory tree 📄
  • Compress: compress files 📄
  • Crypt: encrypt files 📄
  • Hasher: hash files 📄
  • Union: join multiple remotes to work together 📄

Features

  • MD5/SHA-1 hashes checked at all times for file integrity
  • Timestamps preserved on files
  • Partial syncs supported on a whole file basis
  • Copy mode to just copy new/changed files
  • Sync (one way) mode to make a directory identical
  • Bisync (two way) to keep two directories in sync bidirectionally
  • Check mode to check for file hash equality
  • Can sync to and from network, e.g. two different cloud accounts
  • Optional large file chunking (Chunker)
  • Optional transparent compression (Compress)
  • Optional encryption (Crypt)
  • Optional FUSE mount (rclone mount)
  • Multi-threaded downloads to local disk
  • Can serve local or remote files over HTTP/WebDAV/FTP/SFTP/DLNA

Installation & documentation

Please see the rclone website for:

Downloads

License

This is free software under the terms of the MIT license (check the COPYING file included in this package).

Description
"rsync for cloud storage" - Google Drive, S3, Dropbox, Backblaze B2, One Drive, Swift, Hubic, Wasabi, Google Cloud Storage, Azure Blob, Azure Files, Yandex Files
Readme 362 MiB
Languages
Go 98.5%
Python 0.4%
Shell 0.4%
HTML 0.3%
JavaScript 0.2%
Other 0.1%