Commit Graph

669 Commits

Author SHA1 Message Date
Nick Craig-Wood
ad85f6e413 Implement directory include filtering for efficiency
Fixes #395
2016-05-16 17:14:04 +01:00
Nick Craig-Wood
536526cc92 amazonclouddrive: Restart directory listings on error - fixes #475
Before this change rclone would retry only the page that was missing
from the directory listing.  However it turns out that on 429 errors
at least, that page is gone from the directory listing which results
in missing files in the list.  The workaround for this is to restart
the directory listing on any retryable errors.
2016-05-14 17:15:42 +01:00
Nick Craig-Wood
ac9c20b048 Make IsRetryError function 2016-05-14 17:11:19 +01:00
Nick Craig-Wood
2db35f0ce7 Dump out unexpected state in integration test 2016-05-07 21:19:26 +01:00
Nick Craig-Wood
dbfa7031d2 Factor Lister into own file, write tests and fix 2016-05-07 17:17:43 +01:00
Nick Craig-Wood
c2d0e86431 Add more tests for List() and fix resulting problems 2016-05-07 14:50:35 +01:00
Nick Craig-Wood
68ec6a9f5b Add a directory parameter to Fs.List() 2016-05-06 16:52:34 +01:00
Nick Craig-Wood
753b0717be Refactor the List and ListDir interface
Gives more accurate error propagation, control of depth of recursion
and short circuit recursion where possible.

Most of the the heavy lifting is done in the "fs" package, making file
system implementations a bit simpler.

This commit contains some code originally by Klaus Post.

Fixes #316
2016-05-06 16:52:34 +01:00
Nick Craig-Wood
3bdad260b0 Fix typo (thanks Saverio Proto) 2016-05-06 14:09:12 +01:00
Nick Craig-Wood
d205dc23e9 Fix oddities using a file in the root - fixes #471
* Check return from NewFsObject which caused nil ptr deref
  * Correct root directory from "" to string(os.PathSeparator) in getDirFile
2016-05-06 13:52:50 +01:00
Nick Craig-Wood
bdd26d71b2 Clarify swift errors - fixes #460 2016-05-02 12:34:15 +01:00
Nick Craig-Wood
8b2f6faf18 Re-enable OS X in travis tests 2016-05-01 13:13:20 +01:00
Nick Craig-Wood
7c01bbddf8 Normalise path names for OSX local filesystem
Fixes #194 Fixes #451 Fixes #463
2016-05-01 13:13:20 +01:00
Nick Craig-Wood
1752ee3c8b Retry errors which indicate the connection closed prematurely.
See discussion in #442
2016-04-29 17:29:34 +01:00
Nick Craig-Wood
5c2d8ffe33 Retry only the failing tests in the integration tests 2016-04-26 10:20:07 +01:00
Nick Craig-Wood
7fecd5c8c6 Add Leigh Klotz to contributors 2016-04-22 21:12:45 +01:00
Leigh Klotz
19b7ff12ad Doc updates for pasword prompt changes 2016-04-22 21:11:36 +01:00
Nick Craig-Wood
b053234eb1 Add Fabian Ruff to contributors 2016-04-22 21:02:54 +01:00
Fabian Ruff
640d7bd365 Add domain option for openstack (v3 auth) 2016-04-22 21:00:54 +01:00
Nick Craig-Wood
8af68e779f Add Michal Witkowski to contributors 2016-04-22 20:09:16 +01:00
Nick Craig-Wood
3a1198cac5 gcs: Don't configure the oauth token if service_account_file is supplied 2016-04-22 20:07:10 +01:00
Michal Witkowski
022ab4516d Add service account support for GCS 2016-04-22 19:53:27 +01:00
Nick Craig-Wood
17aac9b15f Note certificates FAQ works on Solaris too 2016-04-22 11:53:56 +01:00
Klaus Post
6c0c9abd57 Use "password:" instead of "password>" prompt
Fixes #410
2016-04-21 19:39:46 +01:00
Nick Craig-Wood
70496c15e1 Add Jim Tittsler to contributors 2016-04-21 19:37:41 +01:00
Jim Tittsler
8b61e68bb7 Fix doc typos. 2016-04-20 11:50:28 +09:00
Nick Craig-Wood
bb75d80d33 Fix frontmatter 2016-04-18 18:55:07 +01:00
Nick Craig-Wood
157d7d45f5 Version v1.29 2016-04-18 18:30:29 +01:00
Nick Craig-Wood
b5cba73cc3 Make test more reliable 2016-04-18 17:48:52 +01:00
Nick Craig-Wood
dd36264aad Add FAQ All my uploaded docx/xlsx/pptx files appear as archive/zip
Fixes #417
2016-04-12 21:41:24 +01:00
Nick Craig-Wood
ddb47758f3 drive: increase default chunk size to 8 MB and document - fixes #397 2016-04-12 21:33:55 +01:00
Nick Craig-Wood
9539bbf78a Fix appveyor build after vet removal from tools repo 2016-04-07 20:07:00 +01:00
Nick Craig-Wood
0f8e7c3843 Make rclone check obey the --size-only flag - fixes #419 2016-04-07 15:01:45 +01:00
Nick Craig-Wood
b835330714 Use "application/octet-stream" if mime.TypeByExtension returns invalid type
Fixes #424
2016-04-07 14:32:01 +01:00
Nick Craig-Wood
310db14ed6 Notes on --transfers and B2 2016-04-04 17:58:36 +01:00
Klaus Post
7f2e9d9a6b Require go v1.5 for compilation
Google cloud package requires go v1.5 to compile, so we need to require the same for rclone.

Fixes #408
2016-04-04 17:34:39 +01:00
Nick Craig-Wood
6cc9c09610 drive: preserve mime type on file update - fixes #417 2016-04-04 16:58:42 +01:00
Nick Craig-Wood
93c60c34e1 b2: Fix incorrect value of Precision - should be 1ms not 1s 2016-03-24 15:23:27 +00:00
Klaus Post
02c11dd4a7 Don't de-reference swift connection
The connection object contains a mutex, so it is good practice not to dereference it to a value.

Reported by Go tip "go vet".
2016-03-23 17:09:05 +00:00
Klaus Post
40dc575aa4 Update Travis CI
- Only use golint if version is > Go 1.4
- Add Go 1.6 and tip as test targets.
2016-03-23 17:07:26 +00:00
Klaus Post
f8101771c9 Disable keepalive to keep server from serving stale results.
Fixes issue #402

Bonus fix: Fix "multiple header writes" warning when no code is received.
2016-03-23 16:57:56 +00:00
Klaus Post
8f4d6973fb Fix missing "quit" option when there are no remotes. 2016-03-23 16:57:56 +00:00
Nick Craig-Wood
ced3a4bc19 Implement -I, --ignore-times for unconditional upload - fixes #311 2016-03-22 17:02:27 +00:00
Nick Craig-Wood
cb22583212 b2: Enable mod time syncing - fixes #348 2016-03-22 15:56:44 +00:00
Nick Craig-Wood
414b35ea56 Change the interface of SetModTime to return an error - #348 2016-03-22 15:56:44 +00:00
Nick Craig-Wood
f469905d07 dropbox: Note 10,000 files limitation on purge - fixes #374 2016-03-22 14:46:43 +00:00
Nick Craig-Wood
20f4b2c91d b2: update API to new version - fixes #393
* Make reading mod time and SHA1 much more efficient
    * removes an HTTP transaction to increase speed
  * Reduce memory usage of the objects
2016-03-22 14:39:56 +00:00
Nick Craig-Wood
37543bd1d9 b2: Fix parsing of mod time when not in metadata
This files this error `Failed to parse mod time string "":
"src_last_modified_millis" not found in metadata`.
2016-03-22 10:26:37 +00:00
Nick Craig-Wood
0dc0052e93 Note that filters must use / not \ - #394 2016-03-19 17:40:54 +00:00
Nick Craig-Wood
bd27473762 swift: Don't return an MD5SUM for static large objects - #392
* rename isManifest to isDynamicLargeObject for clarity
2016-03-17 17:36:20 +00:00