Commit Graph

506 Commits

Author SHA1 Message Date
Nick Craig-Wood
639e812789 fs: integration tests: add SUMMARY heading for log scraping 2017-11-29 15:55:37 +00:00
Nick Craig-Wood
1c6cad2252 fs: integration tests: add 30 minute timeout per test 2017-11-29 13:51:17 +00:00
ishuah
aab8051f50 move: add --delete-empty-src-dirs flag - fixes #1854 2017-11-28 11:38:19 +03:00
Nick Craig-Wood
fdb01437d8 fs: Allow the http Transport to have an optional filter request function 2017-11-24 09:07:56 +00:00
Nick Craig-Wood
02ffd43572 fs: Save the config before asking for a token - fixes #1220
Before this if the client_id/client_secret was edited it would
disappear when asking for the new token.

This means the post config is done after the user has confirmed the
config is OK which can't be helped.
2017-11-23 14:01:32 +00:00
Nick Craig-Wood
e53892f53b fs,drive,dropbox: Make and use new RepeatableReader variants to lower memory use
RepeatableReaderSized has a pre-allocated buffer which should help
with memory usage - before it grew the buffer.  Since we know the size
of the chunks, pre-allocating it should be much more efficient.

RepeatableReaderBuffer uses the buffer passed in.

RepeatableLimit* are convenience funcitions for wrapping a reader in
an io.LimitReader and then a RepeatableReader with the same buffer
size.
2017-11-23 13:53:46 +00:00
ishuah
6c62fced60 move: fixed root source directories getting deleted after move - fixes #1849 2017-11-23 12:01:35 +03:00
Nick Craig-Wood
159fce0106 fs: fix --cache-dir to have some effect 2017-11-22 17:05:02 +00:00
Nick Craig-Wood
cbe5d7ce64 fs: Remove X-Auth-Token: from headers when dumping for swift 2017-11-21 17:32:07 +00:00
Nick Craig-Wood
1a65a4e769 fs: Add --dump flag, introduce --dump requests, responses and remove --dump-auth, --dump-filters
Now --dump-flag is written as --dump flag. This is a comma separated list which can contain

  * headers - HTTP headers as before
  * bodies  - HTTP bodies as before
  * requests - HTTP request bodies
  * responses - HTTP response bodies
  * auth - HTTP auth
  * filters - Filter rexeps

Leave --dump-headers and --dump-bodies for the time being but remove
the other --dump-* flags as they aren't used very often.
2017-11-21 17:32:07 +00:00
ishuah
b4aa920a3d stats: show the amount of data transferred in kb/mb - fixes #1167 2017-11-21 12:40:02 +03:00
Nick Craig-Wood
8951875c21 vfs,mount,cmount,mountlib: allow flags to be overriden by environment variables 2017-11-20 12:46:21 +00:00
Nick Craig-Wood
e946a8eab0 fs: Add CacheDir config variable 2017-11-20 12:00:32 +00:00
Nick Craig-Wood
46947b3b9b rcat: fix goroutine leak
This was leaking goroutines in the short file case beause it wasn't
calling Close() on the Account object.  This became apparent when
testing with mount.
2017-11-20 12:00:32 +00:00
Iakov Davydov
7d3abdc463 tests for --exclude-if-present 2017-11-17 21:51:11 +00:00
Iakov Davydov
538246f6c3 support exclude file in --fast-list mode 2017-11-17 21:51:11 +00:00
Iakov Davydov
557dd8f031 ListDirSorted check for excludefile 2017-11-17 21:51:11 +00:00
Iakov Davydov
37aaa19f3a new option: --exclude-if-present 2017-11-17 21:51:11 +00:00
Iakov Davydov
cef2e3bf83 path -> startPath in walkRDirTree (we need the path package) 2017-11-17 21:51:11 +00:00
Iakov Davydov
a3a436ce16 WalkRDirTree: return error if unknown item type 2017-11-17 21:51:11 +00:00
Iakov Davydov
5d05df3124 ListContainsExcludeFile: checks for exclude file in the list 2017-11-17 21:51:11 +00:00
Iakov Davydov
421ba84e12 DirTree.Prune: deletes several directories 2017-11-17 21:51:11 +00:00
Iakov Davydov
7ae7080824 FileExists check if a file exists 2017-11-17 21:51:11 +00:00
Remus Bunduc
016abf825e cache: first version 2017-11-15 15:23:21 +00:00
ishuah
aec2265be0 rclone: implement exit codes - #1136 2017-11-15 17:48:37 +03:00
Substantiel
2423fa40e2 config: add password sub command for setting obscured passwords 2017-11-15 14:44:45 +00:00
ishuah
c65734ee69 move: delete source directory after successful move - fixes #1642 2017-11-07 22:21:38 +00:00
Substantiel
7749157596 Add --auto-confirm flag 2017-11-05 21:56:50 +00:00
Nick Craig-Wood
cc7ed13b9b fs: factor test running code into fstest/run.go 2017-11-03 12:59:58 +00:00
ishuah
10f27e2ff2 allow trailing+leading whitespace for passwords - #1717
warn users when they enter passwords with leading/trailing whitespaces

Updated config_test.go, removing deprecated test case and updated TestReveal
2017-10-20 15:46:17 +01:00
Nick Craig-Wood
115053930e Make error messages less crypting when revealing an unobscured password - fixes #1743 2017-10-16 22:03:06 +01:00
Nick Craig-Wood
edfab09eb9 config: add sub commands for full config file management
Previously config sub commands were manually parsed rather than using
cobra.

Make config command have the following sub commands:

 * create    Create a new remote with name, type and options.
 * delete    Delete an existing remote <name>.
 * dump      Dump the config file as JSON.
 * edit      Enter an interactive configuration session.
 * file      Show path of configuration file in use.
 * providers List in JSON format all the providers and options.
 * show      Print (decrypted) config file, or the config for a single remote.
 * update    Update options in an existing remote.

The following changes were made to existing commands

 * listproviders was renamed to providers
 * listoptions was removed in favour of providing the output in providers
 * jsonconfig was renamed to create
 * an optional parameter was added to the show command
2017-10-14 11:50:41 +01:00
thierry
0575623dff Add config listproviders, listoptions, jsonconfig for automated config
Addition of a method listing the providers, a method listing the
options of a provider and method of manual configuration.
2017-10-13 17:17:36 +01:00
Nick Craig-Wood
fc8b13c993 moveto/copyto: Fix to allow copying to the same name - fixes #1736 2017-10-12 20:45:36 +01:00
Nick Craig-Wood
43ced30f11 fs: Add more errors to retry - fixes #1733 2017-10-10 19:51:02 +01:00
Nick Craig-Wood
3eb7f52e39 fs: Add "unexpected EOF reading trailer" as a retriable error - fixes #1730 2017-10-09 17:29:16 +01:00
Nick Craig-Wood
7f3dc9b5c4 Implement WebDAV remote #580
This has special knowledge of Owncloud and Nextcloud to enable more
functionality such as mod times.
2017-10-09 16:19:37 +01:00
Nick Craig-Wood
00fe6d95da fs: fix duplicate files causing spurious copies
Before this fix duplicate files (on Google Drive) caused the next file
to be spuriously copied.  `rclone dedupe` worked around the problem.
2017-10-02 16:52:53 +01:00
Nick Craig-Wood
9e9297838f Implement pcloud remote - #418 2017-10-01 11:37:35 +01:00
Nick Craig-Wood
1c10497b68 Start v1.38-DEV development 2017-09-30 15:16:09 +01:00
Nick Craig-Wood
d96e45ba5b Version v1.38 2017-09-30 14:20:43 +01:00
Nick Craig-Wood
657b3a674d fs: fix test_all -clean to run just one cleaning thread per remote 2017-09-30 11:07:09 +01:00
Nick Craig-Wood
2857ed5c35 fs: fix --immutable tests on remotes which don't have modtime 2017-09-28 08:56:30 +01:00
Jacob McNamee
2d8e75cab4 Implement --immutable option 2017-09-24 20:00:00 +01:00
Stefan
85877f3adc config: add show/file subcommands which print the config/its path (fixes #1086) 2017-09-19 17:59:19 +02:00
Nick Craig-Wood
87335de8a8 fs: fix filename normalization issues in the tests when running on OS X 2017-09-17 15:31:22 +01:00
Stefan Breunig
12405f9f41 fuse: re-use rcat to support uploads for all remotes (fixes #1672) 2017-09-16 22:49:08 +02:00
Stefan
234bfae0d5 b2: implement streaming upload of files with unknown length (see #1614) (closes #1686) 2017-09-16 22:43:48 +02:00
Nick Craig-Wood
4ac9a65049 fs: stop normalizing file names but do a normalized compare in the sync
This works by using a transform function to transform file names when
doing a compare when matching file names in a directory.  rclone now
UTF-8 normalizes the file names and does a case insensitive compare if
the destination remote is case insensitive.

This deprecates the --local-no-unicode-normalization flag.

Fixes #1477
2017-09-16 19:49:31 +01:00
Nick Craig-Wood
a8e41f081c fs: re-implement check and cryptcheck using the same traversal as sync
This makes them 100% consistent with sync and also make them use less
memory as they no longer build the whole tree in memory first.

Fixes #1657
2017-09-16 19:49:31 +01:00