Commit Graph

877 Commits

Author SHA1 Message Date
Nick Craig-Wood
4f9e805d44 acd: Work around 408 REQUEST_TIMEOUT and 504 GATEWAY_TIMEOUT errors
Amazon Drive sometimes returns errors at the end of large uploads

  * 408 REQUEST_TIMEOUT
  * 504 GATEWAY_TIMEOUT
  * 500 Internal error

The file may have been uploaded correctly though, so, on error, wait
for up to 2 minutes for it to appear if it was fully
uploaded (configure timeout with --acd-upload-wait-time).

Issues: #601 #605 #606
2016-08-18 17:39:23 +01:00
Nick Craig-Wood
3f7107839e Add Per Cederberg to contributors 2016-08-18 17:10:50 +01:00
Per Cederberg
bb62c49489 New B2 API endpoint
Backblaze will change the authentication API endpoint on August 16, 2016. The old endpoint will be removed Feb 2nd 2017.

See https://help.backblaze.com/hc/en-us/articles/224959187-B2-Domain-Migration-Plan
2016-08-15 15:59:19 +02:00
Nick Craig-Wood
ae6018355c Correct parameter order for copy/sync etc 2016-08-06 00:07:36 +01:00
Nick Craig-Wood
0805ec051f Add BasicInfo interface shared between Dir and Object 2016-08-05 17:45:27 +01:00
Nick Craig-Wood
e27b91ffb8 Factor each commmand into its own package 2016-08-05 17:13:54 +01:00
Nick Craig-Wood
0a7b34eefc Move internals of rclone command into cmd so it can be imported externally 2016-08-04 22:33:46 +01:00
Nick Craig-Wood
549cac90af Use cobra autogenerated docs
* put the most up to date docs into the code
  * generate command docs using rclone gendocs
  * put command docs into own directory
  * remake them into MANUAL.md
2016-08-04 21:47:14 +01:00
Nick Craig-Wood
ba0b41dd92 Add gendocs command to rclone 2016-08-04 21:47:14 +01:00
Nick Craig-Wood
2df261e42b Add genautocomplete command to make bash completion script. 2016-08-04 21:47:14 +01:00
Nick Craig-Wood
38adb35abe Make dedupe take an optional mode parameter 2016-08-04 21:47:14 +01:00
Nick Craig-Wood
520ded60e3 Add memtest command for debugging purposes 2016-08-04 21:47:14 +01:00
Nick Craig-Wood
ae56df7d4f Add --dedupe-mode only to dedupe command 2016-08-04 21:47:14 +01:00
Nick Craig-Wood
412591dfaf Make rclone use cobra for command line parsing 2016-08-03 17:16:27 +01:00
Nick Craig-Wood
57f8f1ec92 b2: set maximum backoff to 5 Minutes #597 2016-08-01 22:57:52 +01:00
Nick Craig-Wood
f0434789cf Encourage using the latest version before submitting an issue. 2016-07-28 10:38:16 +01:00
Nick Craig-Wood
c2f6decb9c swift: note that tenant isn't optional for > v1 auth - fixes #563 2016-07-15 18:25:59 +01:00
Nick Craig-Wood
9eeed25418 local: fix filenames with invalid UTF-8 not being uploaded #568 2016-07-15 14:18:09 +01:00
Nick Craig-Wood
67562081f7 Version v1.32 2016-07-13 17:32:39 +01:00
Nick Craig-Wood
41917eb1f2 b2: Fix upload of files large files not in root - fixes #582 2016-07-13 15:28:39 +01:00
Nick Craig-Wood
c3e996f10f b2 doc fixes 2016-07-13 14:50:47 +01:00
Nick Craig-Wood
63f6827a0d Version v1.31 2016-07-13 12:28:01 +01:00
Nick Craig-Wood
96e2271cce Factor commands into Makefile 2016-07-13 12:25:19 +01:00
Nick Craig-Wood
ac3c83f966 Fix integration tests for drive 2016-07-12 21:38:15 +01:00
Nick Craig-Wood
b9c8e61d39 Explicitly check the state in tests after writing files
...otherwise Amazon Drive will fail.
2016-07-12 21:36:39 +01:00
Nick Craig-Wood
a6056408dd Fix move command - stop it running for overlapping fses - fixes #577
* Make move command check for overlapping remotes and refuse to run
  * Do copy/delete rather than all the copies then all the deletes
  * Doesn't purge the source - this was unexpected behaviour see #512 and #416
  * Add -list-retries flag to test suite to control retries

This changes the semantics of `move` slightly.  However it now errs on
the side of not deleting stuff.
2016-07-12 10:49:37 +01:00
Nick Craig-Wood
b9479cf7ab Implement --no-update-modtime flag - fixes #511 2016-07-12 10:46:45 +01:00
Nick Craig-Wood
452a5badc1 Add Stefan Weichinger to contributors 2016-07-11 15:32:58 +01:00
Stefan G. Weichinger
d645bf0966 Add basic info how to use ansible role for installation 2016-07-11 15:31:36 +01:00
Nick Craig-Wood
50addaa91e Add Antonio Messina to contributors 2016-07-11 15:22:17 +01:00
Antonio Messina
02a3bbaa3d swift: add support for non-default project domain.
With Keystone V3 both users and projects (a.k.a. tenants) can belong
to different domains. This change allow specifying different domains
for the user and the project.
2016-07-11 15:16:58 +01:00
Nick Craig-Wood
a20d80565b Tidy stats output - fixes #541 2016-07-11 13:04:30 +01:00
Nick Craig-Wood
56adb52a21 Rename Amazon Cloud Drive to Amazon Drive - fixes #532 2016-07-11 12:42:44 +01:00
Nick Craig-Wood
8c2fc6daf8 s3: Add instructions on how to use rclone with minio 2016-07-11 12:12:28 +01:00
Nick Craig-Wood
4bd9932703 Fix wording in verbose copy logs - fixes #574 2016-07-09 10:11:57 +01:00
Nick Craig-Wood
2a1d4b7563 s3: Add ap-northeast-2 (Seoul) and ap-south-1 (Mumbai) regions - fixes #567 2016-07-06 11:14:59 +01:00
Nick Craig-Wood
b394431f18 Improve --files-from docs - fixes #547 2016-07-05 12:33:59 +01:00
Nick Craig-Wood
cc628717d8 b2: Add --b2-versions flag so old versions can be listed and retreived. #420 2016-07-05 11:27:04 +01:00
Nick Craig-Wood
f3e00133a0 dropbox: Don't retry 461 errors - fixes #551
461 errors from dropbox indicate some sort of copyright violation.
2016-07-04 13:45:53 +01:00
Nick Craig-Wood
606961f49d b2: Treat 403 errors (eg cap exceeded) as fatal #420 2016-07-04 13:45:53 +01:00
Nick Craig-Wood
13591c7c00 Redo error handling for sync/copy/move
* Factor sync/copy/move into its own file
  * Make fatal errors abort the sync
  * Make Copy return errors
  * Make Sync/Copy/Move return the last Copy error if there was one
  * Prioritise returning Fatal errors
  * NoRetry errors are returned if no other types of errors
2016-07-04 13:45:53 +01:00
Nick Craig-Wood
28f4061892 Add two more classes of error Fatal and NoRetry
These are for remotes to signal that they have a fatal error and don't
want to continue (eg cap exceeded) or that a particular file shouldn't
be retried for some reason.
2016-07-04 13:45:52 +01:00
Nick Craig-Wood
018fe80bcb b2: cleanup old file versions - fixes #462 2016-07-02 17:03:08 +01:00
Nick Craig-Wood
0a43ff9c13 Modify interface for accounting to take a string not an fs.Object 2016-07-02 16:58:50 +01:00
Nick Craig-Wood
9aae143833 Implement cleanup command for emptying trash / removing old versions of files 2016-07-01 16:35:36 +01:00
Nick Craig-Wood
c8e2531c8b b2: make error handling compliant 2016-07-01 16:23:23 +01:00
Nick Craig-Wood
9290004bb8 pacer: make sleep get-able and set-able 2016-07-01 16:22:51 +01:00
Nick Craig-Wood
cbebefebc4 b2: Fix handling of token expiry #420
Found with --b2-test-mode expire_some_account_authorization_tokens
2016-07-01 11:47:42 +01:00
Nick Craig-Wood
6f3897ce2c b2: implement --b2-test-mode to set X-Bz-Test-Mode header #420 2016-07-01 11:30:09 +01:00
Nick Craig-Wood
ea5878f590 b2: set cutoff for chunked upload to 200MB #420
This is the value recommended in the b2 integration checklist:

https://www.backblaze.com/b2/docs/integration_checklist.html
2016-07-01 10:08:09 +01:00