Nick Craig-Wood
40c02989f1
acd: Fix panic on token expiry - fixes #1117
2017-02-11 17:49:59 +00:00
Nick Craig-Wood
50e190ff54
cat: don't allocate buffers if not needed to reduce memory usage
2017-02-09 11:46:53 +00:00
Nick Craig-Wood
dd20a297d6
cat: Fix go routine leak
2017-02-09 11:25:36 +00:00
Nick Craig-Wood
c0ad29c06c
Clarify logging and docs for --no-traverse incompatibilities - fixes #1059
2017-02-08 22:35:12 +00:00
Nick Craig-Wood
d091d4a8bb
rclone cat: add --head, --tail, --offset, --count and --discard
...
Fixes #819
2017-02-08 08:09:41 +00:00
Nick Craig-Wood
381b845307
acd: Fix nil pointer deref after Move #1098
...
Don't attempt to read the info in moveNode as there are paths which
don't, read it again from the directory afterwards.
2017-02-04 12:56:21 +00:00
Nick Craig-Wood
48cdedc97b
Re-implement sync routine to work a directory at a time
...
Multiple directories (up to --checkers worth) are scanned at once.
This uses much less memory than the previous scheme - only the amount
of memory needed to hold an entire directory listing of objects.
For directory based remotes the speed is unchanged.
For bucket based remotes, instead of doing one API call to list the
whole bucket, it does multiple calls, one for each pseudo directory.
However these are done in parallel so in practice this seems to speed
up directory listings.
This replaces the existing sync method as it performs faster and uses
less memory.
The old sync method is available with the temporary --old-sync-method
flag.
Fixes #517
Fixes #439
Fixes #236
Fixes #1067
2017-02-04 10:30:25 +00:00
Nick Craig-Wood
7c6cd3a9e1
Make --delete-after the default and refactor --delete-{before,during,after} parsing
2017-02-04 10:30:25 +00:00
Nick Craig-Wood
bcdd73369f
Ignore --delete-before with --track-renames
2017-02-04 10:30:25 +00:00
Nick Craig-Wood
86bec20b56
sync: factor accumulating the rename checks
2017-02-04 10:30:25 +00:00
Nick Craig-Wood
c3b2b89473
Add ListDirSorted function to list a directory
...
* fix error return of readFilesFn also
2017-02-04 10:30:25 +00:00
Nick Craig-Wood
85f05c57d1
Clean empty directories between test runs
2017-02-04 10:30:25 +00:00
Nick Craig-Wood
16d91246c4
sftp: Fix remote race on creating directories
...
Because there is a period of time between checking a directory needs
creating and creating it, the leads to errors where directories are
attempting to be created twice.
Add locking on a per directory basis to fix while doing mkdir.
2017-02-04 10:29:46 +00:00
Nick Craig-Wood
726cb43be9
Complete SFTP remote #521
...
* Add unit tests
* Fix up remote so it passes tests
* Add docs
2017-02-04 10:29:46 +00:00
Nick Craig-Wood
288302c2cf
Make fallback purge delete empty directories too.
...
This was implemented to make the SFTP unit tests pass.
2017-02-04 10:29:46 +00:00
Nick Craig-Wood
609671aabc
Add Jack Schmidt to contributors
2017-02-04 10:29:46 +00:00
Jack Schmidt
b9a8315696
Basic SFTP support, Issue #521
2017-02-04 10:29:18 +00:00
Jack Schmidt
27e18b6efa
sftp: add required packages to vendor
2017-02-04 10:29:18 +00:00
Nick Craig-Wood
9d331ce04b
Implement --ignore-checksum flag
...
Fixes #793 Fixes #863 Fixes #981
2017-02-03 08:11:10 +00:00
Nick Craig-Wood
916569102c
b2: constrain memory usage when doing multipart uploads #439
...
Each part of a multipart upload takes 96M of memory, so we make sure
that we don't user more than `--transfers` * 96M of memory buffering
the multipart uploads.
This has the consequence that some uploads may appear to be at 0% for
a while, however they will get going eventually so this won't
re-introduce #731 .
2017-02-03 08:03:04 +00:00
Nick Craig-Wood
28f9b9b611
drive: detect files using file size as well as md5 - fixes 980
2017-02-03 08:00:03 +00:00
Nick Craig-Wood
7679620f4b
drive: Experimentally add --drive-list-chunk
2017-02-02 21:49:02 +00:00
Nick Craig-Wood
8a11da4e14
mount: Make fsync be a no-op for direectories too #1045
2017-02-02 21:31:41 +00:00
Nick Craig-Wood
f11867d810
Add Jon Yergatian to contributors
2017-02-02 21:14:24 +00:00
Jon Yergatian
6f8501e9a1
s3: Added ca-central-1 region
2017-02-02 21:14:10 +00:00
Nick Craig-Wood
37fe6d56e5
mount: fix docs for umount flags - fixes #1036
2017-01-30 18:17:16 +00:00
Nick Craig-Wood
ff8f11d79c
Add Károly Oláh to contributors
2017-01-29 20:55:21 +00:00
okaresz
cbc113492a
Add Drive specific option: --drive-skip-gdocs - fixes #1035
2017-01-29 20:53:51 +00:00
Nick Craig-Wood
74702554da
onedrive: use token renewer to stop auth errors on long uploads
...
Fixes #820
2017-01-29 20:45:45 +00:00
Nick Craig-Wood
bd29015022
Factor token renewer from amazonclouddrive to oauthutil
2017-01-29 20:45:44 +00:00
Nick Craig-Wood
2192805360
rclone config: when choosing from a list, allow the value to be entered
2017-01-29 15:51:26 +00:00
Nick Craig-Wood
db0b93c0ad
rclone config: allow rename and copy of remotes - fixes #641
2017-01-29 15:37:44 +00:00
Nick Craig-Wood
94947f2523
Implement -L, --copy-links flag to allow rclone to follow symlinks
...
Fixes #40
2017-01-29 13:43:20 +00:00
Nick Craig-Wood
29c6e22024
mount: Make fsync be a no-op rather than returning an error - fixes #1045
2017-01-29 11:29:42 +00:00
Nick Craig-Wood
390f3cf35b
crypt: Add --crypt-show-mapping to show encrypted file mapping
...
Fixes #1004
2017-01-29 10:14:17 +00:00
Nick Craig-Wood
20c033b484
Add a writing documentation section and update document
2017-01-29 09:47:47 +00:00
Nick Craig-Wood
8068ef96b6
Add Dario Giovannetti to contributors
2017-01-26 10:29:21 +00:00
Dario Giovannetti
9d36258923
Comply with XDG Base Directory specification
...
Fixes #868
2017-01-26 10:22:08 +00:00
Nick Craig-Wood
9fdeb82328
Fix tests under Windows
2017-01-20 17:12:05 +00:00
Nick Craig-Wood
2abfae283c
crypt: fix crypt writer getting stuck in a loop #902
...
This happened when the underlying reader returned io.ErrUnexpectedEOF.
The error handling for the call to io.ReadFull failed to take this
into account.
io.ErrUnexpectedEOF is reasonably common when SSL connections go wrong
when communicating with ACD, so it manifested itself as transfers from
non-encrypted ACD to encrypted ACD getting stuck.
2017-01-20 16:00:55 +00:00
Nick Craig-Wood
b6848a3edb
Fix race in Lister.Finished which was causing the tests to be unreliable
2017-01-19 20:11:17 +00:00
Nick Craig-Wood
e2bf9eb8e9
Implement --suffix for use with --backup-dir only #98
...
This also makes sure we remove files we are about to override in the
--backup-dir properly.
2017-01-19 20:11:17 +00:00
Nick Craig-Wood
a77659e47d
Make directory listing checks more reliable and easier to read
2017-01-19 20:11:17 +00:00
Nick Craig-Wood
e9da14ac2e
acd: After moving a file, wait for the file to no longer be in the directory
...
This fixes a Move followed quickly by a Copy updating the wrong file.
2017-01-19 20:11:17 +00:00
Nick Craig-Wood
a4bf22e620
b2: fix upload url not being refreshed properly #825
2017-01-17 17:34:21 +00:00
Nick Craig-Wood
a6b4065e13
mount: fix retry on network failure when reading off crypt - fixes #1042
2017-01-17 16:32:04 +00:00
Nick Craig-Wood
07ebf35987
Clarify what happens to files already in the --backup-dir DIR
2017-01-16 19:26:56 +00:00
Nick Craig-Wood
47ebd0789c
Make "make quicktest" ignore a config file if present for local running
...
This means "make quicktest" should give the same result as when run by
Travis.
2017-01-16 17:54:18 +00:00
Nick Craig-Wood
166fd50451
oauthutil: copy the config before modifying it
...
The stops simultaneous use of oauth configs with different client IDs
causing a problem.
2017-01-16 17:33:25 +00:00
Nick Craig-Wood
0604d3dbf2
acd, onedrive: make sure we have found the root before purging
...
If we don't, purge can try to trash the root node which fortunately
doesn't succeed.
2017-01-16 17:33:25 +00:00
