Commit Graph

600 Commits

Author SHA1 Message Date
Matan Rosenberg
22f06590f7 genautocomplete: add support for fish shell 2020-04-29 16:19:35 +01:00
Nick Craig-Wood
f37af9afec lsjson: Add --hash-type parameter and use it in lsf to speed up hashing
Before this change if you specified --hash MD5 in rclone lsf it would
calculate all the hashes and just return the MD5 hash which was very
slow on the local backend.

Likewise specifying --hash on rclone lsjson was equally slow.

This change introduces the --hash-type flag (and corresponding
internal parameter) so that the hashes required can be selected in
lsjson.

This is used internally in lsf when the --hash parameter is selected
to speed up the hashing by only hashing with the one hash specified.

Fixes #4181
2020-04-29 16:09:45 +01:00
Kush
f555873f18
delete: added --rmdirs flag to delete directories as well - fixes #4055
If you supply the --rmdirs flag with delete command,
it will remove all empty directories along with it
leaving the root intact.
2020-04-29 12:15:30 +01:00
Nick Craig-Wood
1c8eab81a5 dbhashsum: hide the command now it is deprecated 2020-04-29 10:12:12 +01:00
Nick Craig-Wood
cbc5af329f cachestats: deprecate in favour of rclone backend stats cache: 2020-04-29 10:10:57 +01:00
Nick Craig-Wood
1aa1a2c174 backend: add new backend command for backend specific commands
These commands are for implementing backend specific
functionality. They have documentation which is placed automatically
into the backend doc.

There is a simple test for the feature in the backend tests.
2020-04-29 10:10:57 +01:00
Nick Craig-Wood
1f61027f51 rc: add -o/--opt and -a/--arg for more structured input 2020-04-29 09:42:31 +01:00
Nick Craig-Wood
e051a34fc1 dbhashsum: deprecate: use rclone hashsum DropboxHash instead 2020-04-23 11:13:13 +01:00
Nick Craig-Wood
d22e6f5a96 cryptcheck: remove duplicated debug OK message 2020-04-22 11:33:48 +01:00
Nick Craig-Wood
44b1a591a8 crypt: get rid of the unused Cipher interface as it obfuscated the code 2020-04-22 11:33:48 +01:00
Nick Craig-Wood
3f654dac37 mount: map more rclone errors into file systems errors
This improves the error reporting, in particular for
fs.ErrorPermissionDenied which was being reported as an IO error.
2020-04-21 16:31:43 +01:00
Nick Craig-Wood
eed9c5738d vfs: factor the vfs cache into its own package 2020-04-20 10:42:33 +01:00
Nick Craig-Wood
fd39cbc193 vfstests: move functional tests from mountlib and make them work with VFS
The tests are now run for the mount commands and for the plain VFS.

This makes the tests much easier to debug when running with a VFS than
through a mount.
2020-04-20 10:42:33 +01:00
Nick Craig-Wood
b25f5eb0d1 serve sftp: use VFS utility functions instead of own copy 2020-04-19 15:40:55 +01:00
Denis
31a1cc46b7
copyurl: add no-clobber flag - fixes #3950 2020-04-19 12:40:17 +01:00
Dan Dascalescu
756d47fb50 copy: fix typo 2020-04-15 17:08:44 +01:00
Jon Fautley
53874bd8ee cmd: add --error-on-no-transfer option
This allows rclone to exit with a non-zero return code if no files are
transferred. This is useful when calling rclone as part of a workflow/script
pipeline as it allows the end user to stop processing if no files have been
transferred.

NB: Enabling this option will return in rclone exiting non-zero if there are no
transfers. Depending on how your're currently using rclone in your scripts,
this may break existing setups!
2020-04-15 17:06:40 +01:00
Nick Craig-Wood
da41db4712 vfs,mount,cmount: report 1PB free for unknown disk sizes
Factor the logic into the VFS layer so we don't have to duplicate it
into mount and cmount.

See: https://forum.rclone.org/t/rclone-mount-question/15454/
2020-04-11 13:31:10 +01:00
Michael G
ec8a884787 doc: Clarify 'key' option for host key on serve sftp
The option --key would set the sftp host key. It could be mistaken for a default-user-key. Instead, explicitly call it 'host key' to avoid confusion.
2020-04-10 15:23:58 +01:00
Ankur Gupta
08c2cb784f
filter: Added --files-from-raw flag
--files-from parses input files by ignoring comments starting with # and ;
and stripping whitespace from start and end of strings.

The --files-from-raw flag was added that reads every line from the file ignoring
comment characters and not stripping whitespace while maintaining
backwards compatibility.

Fixes #3762
2020-04-03 10:36:24 +01:00
Nick Craig-Wood
66e08e0cc8 mount: warn if --allow-non-empty used on Windows and clarify docs 2020-03-31 12:16:03 +01:00
Nick Craig-Wood
243a868a5b touch: add --localtime flag to make --timestamp localtime not UTC
Fixes #4067
2020-03-23 17:12:56 +00:00
Nick Craig-Wood
36717c7d98 serve restic: fix tests after restic project removed vendoring 2020-03-18 16:50:01 +00:00
evileye
6ff0bb825e mount: fix fail because of too long volume name - fixes #4026 2020-03-05 13:57:20 +00:00
Nick Craig-Wood
2b268f9724 build: fixup formatting after go1.14 go fmt changes 2020-02-28 16:58:33 +00:00
Dan Walters
7586a345ff dlna: cds: use modification time as date in dlna metadata
We havn't been outputting anything for this until now, which leads to my
Samsung showing an epoch/1970 date for all files.
2020-02-27 18:05:18 +01:00
Nick Craig-Wood
e3c11c9ca1 mount: add --async-read flag to disable asynchronous reads
See: https://forum.rclone.org/t/constantly-high-iowait-add-log/14156
2020-02-27 16:12:33 +00:00
Nick Craig-Wood
17b4058ee9 mount: constrain to go1.13 or above otherwise bazil.org/fuse fails to compile 2020-02-26 21:26:46 +00:00
Nick Craig-Wood
9663f9b2ab mount: ignore --allow-root flag with a warning as it has been removed upstream
For background see: https://github.com/bazil/fuse/issues/144
2020-02-26 21:11:25 +00:00
Dan Walters
7e2568a312 dlna: cds: don't specify childCount at all when unknown
Basically, solving #3541 with a different approach - bringing in
the upstream upnpav module, and changing ChildCount from int to a
*int to avoid childCount="0" in the XML output when that value is
simply unknown.

Current approach is leading to some recursion issues and according
to the DLNA spec it shouldn't be necessary, anyway.
2020-02-25 08:41:00 +01:00
Nick Craig-Wood
ec1271818f mount2: hide mount2 command for the moment 2020-02-11 14:28:13 +00:00
Nick Craig-Wood
8318020387 Implement mount2 with go-fuse
This passes the tests and works efficiently with the non sequential vfs ReadAt fix.
2020-02-11 14:28:13 +00:00
Nick Craig-Wood
5c869d5bd3 cmd: make stats be printed on non-zero exit code
This also rationalises the exit sequence so that dumping
goroutines/open files happens regardless of exit error state.

See: https://forum.rclone.org/t/transfer-stats-on-non-0-exit/14211
2020-02-10 16:40:43 +00:00
Nick Craig-Wood
a54210a2e4 docs: restore missing mount --daemon docs
This was done as part of ebfeec9fb4 which unfortunately patched
the auto generated files.
2020-02-10 15:29:39 +00:00
Nick Craig-Wood
8b664c3ec5 docs: restore lost spelling fixes
These came from 3d424c6e08 which unfortunately got added the
docs to the auto generated files.
2020-02-10 15:29:39 +00:00
Nick Craig-Wood
feee92c790 docs: restore lost mount share docs
These came from 162fdfe455 which unfortunately added the docs to
the auto generated files.
2020-02-10 15:29:39 +00:00
Nick Craig-Wood
de93852512 docs: restore lost auth proxy logs
These came from f2a789ea98 which unfortunately added the docs to
the auto generated files.
2020-02-10 15:29:39 +00:00
Nick Craig-Wood
dfb710eab7 gendocs: add autogenerated header to all docs 2020-02-10 15:29:39 +00:00
Lars Lehtonen
4ee3c21a9d cmd/serve/ftp: replace deprecated os.SEEK_SET with io.SeekStart 2020-02-06 10:58:34 +00:00
Nick Craig-Wood
f80d98553a dbhashsum: stop it returning UNSUPPORTED on dropbox 2020-01-29 19:49:42 +00:00
Nick Craig-Wood
bfd9f32188 lsjson: add --no-mimetype flag, speed up lsf
Before this changed we unconditionally fetched the MimeType. On Some
backends like s3 and swift this takes an extra transaction which meant
that `lsf` on those backends was needlessly slow.

This adds an internal option so `lsf` can declare whether it wants
MimeTypes or not depending on whether the user asked for them and an
external flag `--no-mimetype` for `lsjson`.

See: https://forum.rclone.org/t/reliably-setup-incremental-updates/14006/8
2020-01-26 16:38:00 +00:00
landall
77e55b8265 hashsum: Add flag --base64 flag - fixes #3663
This flag can be used to output QuickXorHash in the same format as MS
Graph API.
2020-01-20 17:29:58 +00:00
Xiaoxing Ye
ccaca04a5d rcd: move webgui apart; option to disable browser
Fix #3601, #3785
2020-01-20 12:27:55 +00:00
Nick Craig-Wood
7cf8ea354c dedupe: add missing modes to help string 2020-01-19 11:09:45 +00:00
Nick Craig-Wood
422ad38e5b copyurl: add --stdout flag to write to stdout 2020-01-18 11:15:51 +00:00
Nick Craig-Wood
0b7f959433 cmount: when setting dates discard out of range dates
It appears that sometimes Windows/WinFSP/cgofuse sends dates which are
the epoch to rclone.  These dates appear as 1601-01-01 00:00:00 plus
or minus the timezone.

These dates aren't being sent from rclone.

This patch filters dates out before 1601-01-02 so rclone does not
attempt to set them.

See: https://forum.rclone.org/t/bug-corruption-of-modtime-via-vfs-layer/12204
See: https://forum.rclone.org/t/io-error-googleapi-error-403-insufficient-permission-insufficientpermissions/11372
See: https://github.com/billziss-gh/cgofuse/issues/35
2020-01-18 11:13:35 +00:00
Nick Craig-Wood
81002747c5 dedupe: implement keep smallest too
This is to help deduping google docs and their exported versions if
they accidentally get uploaded to the source again.

See: https://forum.rclone.org/t/my-stupidity-or-a-bug/13861
2020-01-17 13:08:37 +00:00
Nick Craig-Wood
b88dec51e5 proxy: replace use of bcrypt with sha256
Unfortunately bcrypt only hashes the first 72 bytes of a given input
which meant that using it on ssh keys which are longer than 72 bytes
was incorrect.

This swaps over to using sha256 which should be adequate for the
purpose of protecting in memory passwords where the unencrypted
password is likely in memory too.
2020-01-15 16:23:57 +00:00
Paul Tinsley
f2a789ea98 serve sftp: Add support for public key with auth proxy - fixes #3572 2020-01-15 16:23:57 +00:00
Nick Craig-Wood
ae340cf7d9 log: factor flags into logflags package - fixes #3792 2020-01-09 13:25:37 +00:00