Nick Craig-Wood
2cfb3834f2
Log errors with %v
2016-06-18 09:36:47 +01:00
Nick Craig-Wood
df1092ef33
Change Fs.Put so that it must cope with existing files
...
This should fix duplicate files on drive and 409 errors on
amazonclouddrive however it will slow down the upload slightly as
another roundtrip will be needed.
None of the other Fses needed adjusting.
Fixes #483
2016-06-13 19:29:10 +01:00
Nick Craig-Wood
4c5b2833b3
Convert to using github.com/pkg/errors everywhere
2016-06-13 17:43:03 +01:00
Nick Craig-Wood
c6a79ff72d
Fix remaining places in listing where we were logging errors not returning them
2016-05-30 19:51:15 +01:00
Nick Craig-Wood
36700d36a7
Fix dropbox root directory listings
2016-05-16 17:54:59 +01:00
Nick Craig-Wood
c2d0e86431
Add more tests for List() and fix resulting problems
2016-05-07 14:50:35 +01:00
Nick Craig-Wood
68ec6a9f5b
Add a directory parameter to Fs.List()
2016-05-06 16:52:34 +01:00
Nick Craig-Wood
753b0717be
Refactor the List and ListDir interface
...
Gives more accurate error propagation, control of depth of recursion
and short circuit recursion where possible.
Most of the the heavy lifting is done in the "fs" package, making file
system implementations a bit simpler.
This commit contains some code originally by Klaus Post.
Fixes #316
2016-05-06 16:52:34 +01:00
Nick Craig-Wood
414b35ea56
Change the interface of SetModTime to return an error - #348
2016-03-22 15:56:44 +00:00
Nick Craig-Wood
7d4e143dee
Make it obvious that the client secrets are encrypted
2016-02-28 19:57:19 +00:00
Nick Craig-Wood
e08e35984c
Add help to remote chooser in rclone config - fixes #43
2016-02-21 13:40:16 +00:00
klauspost
ef06371c93
Create separate interface for object information.
...
Take out read-only information about a Fs in a separate struct to limit access.
See discussion at #282 .
2016-02-19 13:31:09 +00:00
Nick Craig-Wood
ccba859812
Test all available hashes for each remote
2016-01-23 09:10:36 +00:00
klauspost
78c3a5ccfa
Add support for multiple hash types.
...
Add support for multiple hash types with negotiation of common hash types for comparison.
Manually rebased version of #277 (see discussion there)
2016-01-11 13:39:33 +01:00
Nick Craig-Wood
c724d8f614
dropbox: Make file exclusion error controllable with -q #287
2016-01-10 11:49:04 +00:00
Nick Craig-Wood
b257de4aba
Be more constistent with naming in remotes
...
* External objects are called Fs and Object
* Object.fs always points to the Fs
2015-11-07 11:14:46 +00:00
Nick Craig-Wood
bf562d7373
Tweak wording on client/secret ids in acd, gcs and drive - fixes #155
2015-10-03 16:04:45 +01:00
Nick Craig-Wood
e9c915e6fe
Fix golint warnings
2015-09-22 18:47:16 +01:00
Nick Craig-Wood
2ed158aba3
Fixes from go vet and errcheck
2015-09-22 07:31:12 +01:00
Nick Craig-Wood
c98a51b26c
Lightly obscure secrets
2015-09-01 22:33:34 +01:00
Nick Craig-Wood
e2717a031e
Implement Mover and DirMover interfaces fixes #115
...
* unit tests
* local
* drive
* dropbox
2015-09-01 21:49:13 +01:00
Nick Craig-Wood
92745aa950
Add Root() to Fs interface
2015-09-01 21:49:13 +01:00
Nick Craig-Wood
0168f55f3e
Switch to spf13 fork of pflag - fixes #116
...
This supports --long value as well as --long=value which is as
expected for a unix utility.
2015-08-29 18:14:24 +01:00
Nick Craig-Wood
8b60ab86a1
dropbox: force use of our custom transport which makes timeouts work
2015-08-29 17:48:15 +01:00
Nick Craig-Wood
8e3d45d2dc
dropbox: increase chunk size to improve upload speeds - fixes #103
...
Chunks aren't buffered in memory, so chose 128M as the default size as
producing the maximum throughput. This takes the throughput from 78
kBytes/s to 4MBytes/s a 50x improvement!
2015-08-25 19:01:37 +01:00
Nick Craig-Wood
a96b522958
Implement server side copies if possible - fixes #99
...
Add optional fs.Copier interface
Implemented for
* swift
* s3
* drive
* dropbox
* google cloud storage
2015-08-23 21:18:38 +01:00
Nick Craig-Wood
fedf81c2b7
Add Name() to Fs interface to return name as passed to NewFs
2015-08-23 13:36:38 +01:00
Nick Craig-Wood
bc19e2d84b
dropbox: Issue an error message when trying to upload bad file name - fixes #108
2015-08-20 20:46:35 +01:00
Nick Craig-Wood
1d9e76bb0f
dropbox: remove datastore - Fixes #55 #84
...
This means that dropbox no longer stores MD5SUMs and modified times.
Fixup the tests so that blank MD5SUMs are ignored, and that if
Precision is set to a fs.ModTimeNotSupported, ModTimes can be ignored too.
This opens the door for other FSs which don't support metadata easily.
2015-08-17 17:26:36 +01:00
Leonid Shalupov
3fcff32524
do not print stats in quiet mode - fixes #70
...
...unless had some errors or stats interval requested.
Add fs.ErrorLog to differentiate between Logs which should be
suppressed and errors which shouldn't.
2015-08-10 11:17:54 +01:00
Nick Craig-Wood
472f065ce7
Make sizes in Dropbox 64 bit - fixes #80
2015-08-10 11:17:54 +01:00
Leonid Shalupov
754ce9dec6
dropbox: record names coming from dropbox API, fixes #53 case insensitivity causes duplicated files
2015-06-14 15:23:56 +01:00
Nick Craig-Wood
1d254a3674
Implement --timeout and --conntimeout - fixes #54
...
NB dropbox still to do
2015-05-10 11:29:55 +01:00
Nick Craig-Wood
51b24a1dc6
dropbox: adapt code to recent library changes
2014-12-23 13:55:22 +00:00
Nick Craig-Wood
d3c16608e4
Test Listing the Root of each Fs
2014-07-31 23:20:39 +01:00
Nick Craig-Wood
3e27ff1b95
Add Root List test and fs.Limited tests for single files
2014-07-31 21:35:29 +01:00
Nick Craig-Wood
c389616657
all: make private functions / variables / constant which shouldn't be public
2014-07-29 17:50:07 +01:00
Nick Craig-Wood
457ef2c190
Automatically generate the tests files for each Fs
2014-07-29 17:18:22 +01:00
Nick Craig-Wood
17ffb0855f
Fixes after running errcheck
2014-07-25 18:19:49 +01:00
Nick Craig-Wood
a5b0d88608
Make tests for each Fs
...
Factor tests out of rclonetest
2014-07-24 23:13:32 +01:00
Nick Craig-Wood
282cba20a0
swift, s3, dropbox: fix metadata read on Update()
...
This was causing changed files to be marked as corrupted on upload
2014-07-20 11:23:05 +01:00
Nick Craig-Wood
2479ce2c8e
dropbox: go1.1 compatibility
2014-07-19 15:48:40 +01:00
Nick Craig-Wood
dfc8a375f6
dropbox: Switch to using RFC3339 for time metadata
2014-07-15 19:27:42 +01:00
Nick Craig-Wood
7c9bdb4b7a
dropbox: make limited fs work (copy single file)
2014-07-15 19:27:42 +01:00
Nick Craig-Wood
f8bb0d9cc8
dropbox: remove metadata when we remove files
2014-07-15 19:27:42 +01:00
Nick Craig-Wood
b185e104ed
dropbox: Fix mkdir on already created directory
2014-07-15 19:27:42 +01:00
Nick Craig-Wood
e57a4c7c0c
dropbox: open the datastore in the background
2014-07-15 19:27:42 +01:00
Nick Craig-Wood
d2f187e1a1
dropbox: Use /delta to list objects - much quicker
...
Also fix major performance problem - re-reading entry each time!
2014-07-15 19:27:42 +01:00
Nick Craig-Wood
c9aca33030
dropbox: Fix concurrent access to Dropbox datastore and Lower case keys in datastore
2014-07-15 19:27:42 +01:00
Nick Craig-Wood
2b0911531c
dropbox: basics of metadata in Dropbox datastore working
2014-07-15 19:27:42 +01:00
Nick Craig-Wood
2149185fc2
dropbox: Initial support of full Fs interface
...
Still missing metadata support (eg SetModTime)
2014-07-15 19:27:42 +01:00
