github-mirror/rclone
2
0
Fork 0
mirror of https://github.com/rclone/rclone.git synced 2024-11-25 17:57:44 +08:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
7,164 Commits 177 Branches 114 Tags 280 MiB
9a66086fa0
Commit Graph

49 Commits

Author SHA1 Message Date
Nick Craig-Wood
d0d41fe847 rclone config redacted: implement support mechanism for showing redacted config 
This introduces a new fs.Option flag, Sensitive and uses this along
with IsPassword to redact the info in the config file for support
purposes.

It adds this flag into backends where appropriate. It was necessary to
add oauthutil.SharedOptions to some backends as they were missing
them.

Fixes #5209
 2023-07-07 16:25:14 +01:00
albertony
555def2da7 build: add package comments to silence revive linter 2022-08-28 13:43:51 +02:00
Nick Craig-Wood
6fd9e3d717 build: reformat comments to pass go1.19 vet 
See: https://go.dev/doc/go1.19#go-doc
 2022-08-05 16:35:41 +01:00
albertony
fdd2f8e6d2 Error strings should not be capitalized 
Reported by staticcheck 2022.1.2 (v0.3.2)

See: staticcheck.io
 2022-06-23 23:26:02 +02:00
albertony
ec117593f1 Fix lint issues reported by staticcheck 
Used staticcheck 2022.1.2 (v0.3.2)

See: staticcheck.io
 2022-06-13 21:13:50 +02:00
Nick Craig-Wood
e43b5ce5e5 Remove github.com/pkg/errors and replace with std library version 
This is possible now that we no longer support go1.12 and brings
rclone into line with standard practices in the Go world.

This also removes errors.New and errors.Errorf from lib/errors and
prefers the stdlib errors package over lib/errors.
 2021-11-07 11:53:30 +00:00
albertony
e2f47ecdeb docs: punctuation cleanup 
See #5538
 2021-10-20 22:56:19 +02:00
Ivan Andreev
24de896df2 build: apply gofmt from golang 1.17 2021-09-09 20:43:59 +03:00
albertony
2925e1384c Use binary prefixes for size and rate units 
Includes adding support for additional size input suffix Mi and MiB, treated equivalent to M.
Extends binary suffix output with letter i, e.g. Ki and Mi.
Centralizes creation of bit/byte unit strings.
 2021-04-27 02:25:52 +03:00
Nick Craig-Wood
a774f6bfdb qingstor: fix rclone cleanup 
This patch changes to using the default page limit for listing
unfinished multpart uploads rather than 1000. 1000 is the maximum
specified in the docs, but setting anything larger than 200 gives an
error.
 2021-01-21 17:35:31 +00:00
Nick Craig-Wood
d7cd35e2ca qingstor: fix error propagation in CleanUp 
Before this change errors cleaning multiple buckets were passing silently
 2021-01-21 17:35:31 +00:00
Nick Craig-Wood
9d574c0d63 fshttp: read config from ctx not passed in ConfigInfo #4685 2020-11-26 16:40:12 +00:00
Nick Craig-Wood
2e21c58e6a fs: deglobalise the config #4685 
This is done by making fs.Config private and attaching it to the
context instead.

The Config should be obtained with fs.GetConfig and fs.AddConfig
should be used to get a new mutable config that can be changed.
 2020-11-26 16:40:12 +00:00
Nick Craig-Wood
8b96933e58 fs: Add context to fs.Features.Fill & fs.Features.Mask #3257 #4685 2020-11-09 18:05:54 +00:00
Nick Craig-Wood
d846210978 fs: Add context to NewFs #3257 #4685 
This adds a context.Context parameter to NewFs and related calls.

This is necessary as part of reading config from the context -
backends need to be able to read the global config.
 2020-11-09 18:05:54 +00:00
Josh Soref
a089de0964 docs: grammar: uncountable: links 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref
3068ae8447 docs: grammar: count agreement: files 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref
e4a87f772f docs: spelling: e.g. 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref
d4f38d45a5 docs: spelling: high-speed 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref
bbe7eb35f1 docs: spelling: server-side 
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-28 18:16:23 +00:00
Josh Soref
d0888edc0a Spelling fixes 
Fix spelling of: above, already, anonymous, associated,
authentication, bandwidth, because, between, blocks, calculate,
candidates, cautious, changelog, cleaner, clipboard, command,
completely, concurrently, considered, constructs, corrupt, current,
daemon, dependencies, deprecated, directory, dispatcher, download,
eligible, ellipsis, encrypter, endpoint, entrieslist, essentially,
existing writers, existing, expires, filesystem, flushing, frequently,
hierarchy, however, implementation, implements, inaccurate,
individually, insensitive, longer, maximum, metadata, modified,
multipart, namedirfirst, nextcloud, obscured, opened, optional,
owncloud, pacific, passphrase, password, permanently, persimmon,
positive, potato, protocol, quota, receiving, recommends, referring,
requires, revisited, satisfied, satisfies, satisfy, semver,
serialized, session, storage, strategies, stringlist, successful,
supported, surprise, temporarily, temporary, transactions, unneeded,
update, uploads, wrapped

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
 2020-10-14 15:21:31 +01:00
Nick Craig-Wood
3a14b1d5a9 build: make rclone build with wasm 
Needed to drop
- azureblob backend
- cache backend
- qingstor backend
- cachestats command
- ncdu command
 2020-08-10 17:32:21 +01:00
Nick Craig-Wood
c820576329 fs: define SlowModTime and SlowHash features in the relevant backends 2020-06-30 12:01:36 +01:00
Martin Michlmayr
4aee962233 doc: fix typos throughout docs and code 2020-05-20 15:54:51 +01:00
Martin Michlmayr
fb169a8b54
doc: fix typos throughout docs 2020-05-19 12:02:44 +01:00
Nick Craig-Wood
472d4799d1 qingstor: make rclone cleanup remove pending multipart uploads older than 24h 2020-03-18 12:49:21 +00:00
Lars Lehtonen
219bd97e8a backend/qingstor: lint fix 2020-02-14 18:11:01 +00:00
Nick Craig-Wood
bafe7d5a73 backends: move encoding definitions from fs/encodings 2020-01-16 14:40:36 +00:00
Nick Craig-Wood
3c620d521d backend: adjust backends to have encoding parameter 
Fixes #3761
Fixes #3836
Fixes #3841
 2020-01-16 14:40:36 +00:00
Nick Craig-Wood
479c803fd9 vendor: update all dependencies 2019-11-14 21:51:34 +00:00
Nick Craig-Wood
d122d1d191 qingstor: use lib/encoder 2019-09-30 22:00:25 +01:00
Nick Craig-Wood
6f16588123 s3,b2,googlecloudstorage,swift,qingstor,azureblob: fixes after code review #3421 
- change the interface of listBuckets() removing dir parameter and adding context
- add makeBucket() and use in place of Mkdir("")
    - this fixes some corner cases in Copy/Update
- mark all the listed buckets OK in ListR

Thanks to @yparitcher for the review.
 2019-08-22 23:06:59 +01:00
Nick Craig-Wood
b619430bcf qingstor: make all operations work from the root #3421 2019-08-17 10:30:41 +01:00
Nick Craig-Wood
c2050172aa qingstor: upgrade to v3 SDK and fix listing loop 2019-08-14 16:15:34 +01:00
Nick Craig-Wood
e502be475a azureblob/b2/dropbox/gcs/koofr/qingstor/s3: fix 0 length files 
In 0386d22cc9 we introduced a test for 0 length files read the
way mount does.

This test failed on these backends which we fix up here.
 2019-08-06 15:18:08 +01:00
Nick Craig-Wood
57d5de6fba build: fix up package paths after repo move 
git grep -l github.com/ncw/rclone | xargs -d'\n' perl -i~ -lpe 's|github.com/ncw/rclone|github.com/rclone/rclone|g'
goimports -w `find . -name \*.go`
 2019-07-28 18:47:38 +01:00
Aleksandar Jankovic
f78cd1e043 Add context propagation to rclone 
- Change rclone/fs interfaces to accept context.Context
- Update interface implementations to use context.Context
- Change top level usage to propagate context to lover level functions

Context propagation is needed for stopping transfers and passing other
request-scoped values.
 2019-06-19 11:59:46 +01:00
Fabian Möller
a0d4c04687
backend: fix misspellings 2019-02-07 19:51:03 +01:00
Nick Craig-Wood
6a832b7173 qingstor: default --qingstor-upload-concurrency to 1 to work around bug 
If the upload concurrency is set > 1 then the hash becomes corrupted.
The upload is fine, and can be downloaded fine, however the hash is no
longer the md5sum of the object.  It is not known whether this is
rclone's fault or a bug at QingStor.
 2019-01-27 21:09:11 +00:00
Nick Craig-Wood
a2341cc412 qingstor: add upload chunk size/concurrency/cutoff control #2851 
* --upload-chunk-size
* --upload-concurrency
* --upload-cutoff
 2019-01-18 15:20:20 +00:00
Peter Kaminski
63a2a935fc fix typos in original files, per #2727 review request 2018-11-14 22:48:58 +00:00
Nick Craig-Wood
7194c358ad azureblob,b2,qingstor,s3,swift: remove leading / from paths - fixes #2484 2018-08-26 23:19:28 +01:00
Nick Craig-Wood
f3f48d7d49 Implement new backend config system 
This unifies the 3 methods of reading config

  * command line
  * environment variable
  * config file

And allows them all to be configured in all places.  This is done by
making the []fs.Option in the backend registration be the master
source of what the backend options are.

The backend changes are:

  * Use the new configmap.Mapper parameter
  * Use configstruct to parse it into an Options struct
  * Add all config to []fs.Option including defaults and help
  * Remove all uses of pflag
  * Remove all uses of config.FileGet
 2018-07-16 21:20:47 +01:00
Nick Craig-Wood
be54fd8f70 Remove builds conditional on go1.7 since that is now guaranteed #2154 
Old fallback code was deleted and the go1.7 style code inlined where
appropriate.
 2018-04-07 11:42:55 +01:00
Nick Craig-Wood
f3e982d3bf azureblob,b2,gcs,qingstor,s3,swift: Don't check for bucket/container presense if listing was OK 
In a typical rclone copy to a bucket/container based remote, before
this change we were doing a list, followed by a HEAD of the bucket to
check it existed before doing the copy.  The fact the list succeeded
means the bucket exists so mark it OK at that point.

Issue #1421
 2018-03-01 12:11:34 +00:00
Nick Craig-Wood
97c414f025 config/hash: rename more symbols after factoring into own package 2018-01-18 20:27:52 +00:00
Nick Craig-Wood
4b0c5f79b5 qingstor: Only support on go1.7+ 2018-01-16 17:05:26 +00:00
Nick Craig-Wood
11da2a6c9b Break the fs package up into smaller parts. 
The purpose of this is to make it easier to maintain and eventually to
allow the rclone backends to be re-used in other projects without
having to use the rclone configuration system.

The new code layout is documented in CONTRIBUTING.
 2018-01-15 17:51:14 +00:00
Nick Craig-Wood
b8b620f5c2 Move all backends into backend directory 2018-01-12 17:07:38 +00:00