Commit Graph

6963 Commits

Author SHA1 Message Date
Nick Craig-Wood
d0810b602a crypt: add --crypt-pass-bad-blocks to allow corrupted file output 2023-03-22 16:23:37 +00:00
Nick Craig-Wood
d5afcf9e34 crypt: try not to return "unexpected EOF" error
Before this change the code wasn't taking into account the error
io.ErrUnexpectedEOF that io.ReadFull can return properly. Sometimes
that error was being returned instead of a more specific and useful
error.

To fix this, io.ReadFull was replaced with the simpler
readers.ReadFill which is much easier to use correctly.
2023-03-22 16:23:37 +00:00
Nick Craig-Wood
07c4d95f38 crypt: fix tests assert.Error which should have been assert.EqualError 2023-03-22 16:23:37 +00:00
Nick Craig-Wood
fd83071b6b rc: fix operations/stat with trailing /
Before this change using operations/stat with a remote pointing to a
dir with a trailing / would return a null output rather than the
correct info.

This was because the directory was not found with a trailing slash in
the directory listing.

Fixes #6817
2023-03-22 16:22:45 +00:00
Nick Craig-Wood
e042d9089f fs: Fix interaction between --progress and --interactive
Before this change if both --progress and --interactive were set then
the screen display could become muddled.

This change makes --progress and --interactive use the same lock so
while rclone is asking for interactive questions, the progress will be
paused.

Fixes #6755
2023-03-22 16:18:41 +00:00
Nick Craig-Wood
cdfa0beafb lib/atexit: ensure OnError only calls cancel function once
Before this change the cancelFunc could be called twice, once while
handling the interrupt (CTRL-C) and once while unwinding the stack if
the function happened to finish.

This change ensure the cancelFunc is only called once by wrapping it
in a sync.Once
2023-03-22 12:50:58 +00:00
Nick Craig-Wood
ddb3b17e96 s3: fix hang on aborting multpart upload with iDrive e2
Apparently the abort multipart upload call doesn't return while
multipart uploads are in progress on iDrive e2.

This means that if we CTRL-C a multpart upload rclone hangs until the
all parts uploading have completed. However since rclone is uploading
multiple parts at once this doesn't happen until after the entire file
is uploaded.

This was fixed by cancelling the upload context which causes all the
uploads to stop instantly.
2023-03-22 12:50:58 +00:00
Nick Craig-Wood
32f71c97ea Add Zach Kipp to contributors 2023-03-22 12:50:58 +00:00
dependabot[bot]
53853116fb build(deps): bump github.com/aws/aws-sdk-go from 1.44.223 to 1.44.226
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.223 to 1.44.226.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.223...v1.44.226)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-22 11:51:03 +00:00
dependabot[bot]
a887856998 build(deps): bump github.com/oracle/oci-go-sdk/v65
Bumps [github.com/oracle/oci-go-sdk/v65](https://github.com/oracle/oci-go-sdk) from 65.32.1 to 65.33.0.
- [Release notes](https://github.com/oracle/oci-go-sdk/releases)
- [Changelog](https://github.com/oracle/oci-go-sdk/blob/master/CHANGELOG.md)
- [Commits](https://github.com/oracle/oci-go-sdk/compare/v65.32.1...v65.33.0)

---
updated-dependencies:
- dependency-name: github.com/oracle/oci-go-sdk/v65
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-22 11:50:24 +00:00
Zach Kipp
0df7466d2b cmd/rcd: Fix command docs to include command specific prefix (#6675)
This change addresses two issues with commands that re-used
flags from common packages:

1) cobra.Command definitions did not include the command specific
   prefix in doc strings.
2) Command specific flag prefixes were added after generating
   command doc strings.
2023-03-22 11:47:35 +00:00
eNV25
23579e3b99 cmd/ncdu: refactor redraw handling 2023-03-21 16:41:22 +00:00
Nick Craig-Wood
3affba6fa6 build: remove duplicate linux/arm64 build 2023-03-21 16:25:46 +00:00
Nick Craig-Wood
542677d807 s3: fix --s3-versions on individual objects
Before this fix attempting to access an s3 versioned object by name in
a subdirectory of root would not find the object.

This fixes the problem and introduced an integraton test.

See: https://forum.rclone.org/t/s3-versions-cant-retrieve-old-version/36900
2023-03-21 12:44:45 +00:00
Nick Craig-Wood
d481aa8613 Revert "s3: fix InvalidRequest copying to a locked bucket from a source with no MD5SUM"
This reverts commit e5a1bcb1ce.

This causes a lot of integration test failures so may need to be optional.
2023-03-21 11:43:43 +00:00
Nick Craig-Wood
15e633fa8b build: disable provenance in docker build
To attempt to fix this error:

buildx failed with: ERROR: failed to solve: missing provenance for owlcc15myb2dpmxrz6dl5bzqc
2023-03-20 18:09:54 +00:00
dependabot[bot]
732c24c624 build(deps): bump docker/build-push-action from 3 to 4
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 3 to 4.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-20 16:21:22 +00:00
Nick Craig-Wood
75dfdbf211 ci: revert revive settings back to fix lint
The upstream revive repo changed the default settings for this linter.
We use this through golangci-lint.

This change meant lots of errors appearing all at once. We should
probably fix these in due course, but for the time being this disables
those settings.

See: https://github.com/mgechev/revive/pull/799
2023-03-20 15:26:21 +00:00
asdffdsazqqq
5f07113a4b
docs: install: how to uninstall rclone via winget 2023-03-20 14:51:42 +00:00
Richard Tweed
6a380bcc67
build: fix dockerfile reference in beta image pipeline 2023-03-20 11:54:31 +00:00
dependabot[bot]
97276ce765 build(deps): bump google.golang.org/api from 0.112.0 to 0.114.0
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.112.0 to 0.114.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases)
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/main/CHANGES.md)
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.112.0...v0.114.0)

---
updated-dependencies:
- dependency-name: google.golang.org/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-17 16:25:06 +00:00
dependabot[bot]
a23a7a807f build(deps): bump github.com/klauspost/compress from 1.16.0 to 1.16.3
Bumps [github.com/klauspost/compress](https://github.com/klauspost/compress) from 1.16.0 to 1.16.3.
- [Release notes](https://github.com/klauspost/compress/releases)
- [Changelog](https://github.com/klauspost/compress/blob/master/.goreleaser.yml)
- [Commits](https://github.com/klauspost/compress/compare/v1.16.0...v1.16.3)

---
updated-dependencies:
- dependency-name: github.com/klauspost/compress
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-17 16:24:20 +00:00
dependabot[bot]
c6a4caaf7e build(deps): bump goftp.io/server
Bumps goftp.io/server from 0.4.2-0.20210615155358-d07a820aac35 to 1.0.0-rc1.

---
updated-dependencies:
- dependency-name: goftp.io/server
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-17 16:23:37 +00:00
dependabot[bot]
5574733dcb build(deps): bump github.com/oracle/oci-go-sdk/v65
Bumps [github.com/oracle/oci-go-sdk/v65](https://github.com/oracle/oci-go-sdk) from 65.32.0 to 65.32.1.
- [Release notes](https://github.com/oracle/oci-go-sdk/releases)
- [Changelog](https://github.com/oracle/oci-go-sdk/blob/master/CHANGELOG.md)
- [Commits](https://github.com/oracle/oci-go-sdk/compare/v65.32.0...v65.32.1)

---
updated-dependencies:
- dependency-name: github.com/oracle/oci-go-sdk/v65
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-17 16:22:54 +00:00
dependabot[bot]
49c21d0b6e build(deps): bump github.com/aws/aws-sdk-go from 1.44.218 to 1.44.223
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.218 to 1.44.223.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.218...v1.44.223)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-17 16:22:08 +00:00
eNV25
0ea2ce3674 cmd/ncdu: fix screen corruption when logging
Before this change if logs were not redirected, logging would
corrupt the terminal screen.

This commit stores the logs (max ~100 lines) in an array and
print them when the program exits.
2023-03-17 14:52:34 +00:00
dependabot[bot]
3ddf824251 build(deps): bump actions/setup-go from 3 to 4
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 3 to 4.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](https://github.com/actions/setup-go/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-03-17 14:41:49 +00:00
Nick Craig-Wood
68fdff3c27 build: ensure users with no secrets (dependabot) don't run android upload step 2023-03-17 14:04:46 +00:00
Nick Craig-Wood
c003485ae3 build: ensure users with no secrets (dependabot) don't run deploy step 2023-03-17 13:49:11 +00:00
Nick Craig-Wood
99d5080191 Add Richard Tweed to contributors 2023-03-17 13:49:11 +00:00
alankrit
2ad217eedd librclone:Added example on using librclone with golang. 2023-03-17 12:00:27 +00:00
albertony
a3eb7f1142 jottacloud: fix vfs writeback stuck in a failed upload loop with file versioning disabled
Avoid returning error when option no_versions and remove fail

Fixes #6857
2023-03-17 11:54:43 +00:00
Richard Tweed
6d620b6d88
build: update docker beta build to latest actions and to push to ghcr
* Add ghcr option for docker images
* Update to use the upstream build actions
* Add ability to push beta images manually.
2023-03-17 11:54:01 +00:00
Arnav Singh
9f8357ada7
sftp: fix using key_use_agent and key_file together needing private key file
When using ssh-agent to hold multiple keys, it is common practice to configure
openssh to use a specific key by setting the corresponding public key as
the `IdentityFile`. This change makes a similar behavior possible in rclone
by having it parse the `key_file` config as the public key when
`key_use_agent` is `true`.

rclone already attempted this behavior before this change, but it assumed that
`key_file` is the private key and that the public key is specified in
`${key_file}.pub`. So for parity with the openssh behavior, this change makes
rclone first attempt to read the public key from `${key_file}.pub` as before
(for the sake of backward compatibility), then fall back to reading it from
`key_file`.

Fixes #6791
2023-03-17 11:44:19 +00:00
Nick Craig-Wood
e5a1bcb1ce s3: fix InvalidRequest copying to a locked bucket from a source with no MD5SUM
Before this change, we would upload files as single part uploads even
if the source MD5SUM was not available.

AWS won't let you upload a file to a locket bucket without some sort
of hash protection of the upload which we don't have with no MD5SUM.

So we switch to multipart upload when the source does not have an
MD5SUM.

This means that if --s3-disable-checksum is set or we are copying from
a source with no MD5SUMs we will copy with multipart uploads.

This patch changes all uploads, not just those to locked buckets
because having no MD5SUM protection on uploads is undesirable.

Fixes #6846
2023-03-17 11:34:20 +00:00
Nick Craig-Wood
46484022b0 fs: add size to JSON logs when moving or copying an object #6849 2023-03-17 11:22:57 +00:00
Nick Craig-Wood
ab746ef891 Add Thibault Coupin to contributors 2023-03-17 11:22:57 +00:00
Paul
6241c1ae43 Add devnoname120 to contributors 2023-03-17 11:09:08 +00:00
Paul
0f8d3fe6a3 webdav: add support for chunked uploads — fix #3666
Co-authored-by: Thibault Coupin <thibault.coupin@gmail.com>
Co-authored-by: Nick Craig-Wood <nick@craig-wood.com>
2023-03-17 11:09:08 +00:00
Paul
07afb9e700 webdav: add chunking helper file 2023-03-17 11:09:08 +00:00
Thibault Coupin
3165093feb fstests: add option to skip chunked upload 2023-03-17 11:09:08 +00:00
Paul
4af0c1d902 rest: add optional GetBody function for HTTP call 2023-03-17 11:09:08 +00:00
Nick Craig-Wood
82f9554474 docs: note that rcat will retry chunks when multipart uploading
See: https://forum.rclone.org/t/the-rclone-rcat-reliability-for-the-uploading-files-to-s3/36830
2023-03-17 10:52:21 +00:00
Nick Craig-Wood
d8d53b7aa0 Add Christopher Merry to contributors 2023-03-17 10:52:21 +00:00
Nick Craig-Wood
8c9048259a Add Arnavion to contributors 2023-03-17 10:52:21 +00:00
Christopher Merry
0361acbde4 googlecloudstorage: added gcs requester pays 2023-03-16 17:13:37 +00:00
Aaron Gokaslan
f5bf0a48f3 uptobox: fix improper regex 2023-03-16 17:12:27 +00:00
albertony
cec843dd8c build: run workflow even if tag/branch name contains slash 2023-03-16 17:07:07 +00:00
Anthony Pessy
54a9488e59 s3: add GCS to provider list 2023-03-16 14:24:21 +00:00
Arnavion
29fe0177bd webdav: add "fastmail" provider for Fastmail Files
This provider:

- supports the `X-OC-Mtime` header to set the mtime

- calculates SHA1 checksum server side and returns it as a `ME:sha1hex` prop

To differentiate the new hasMESHA1 quirk, the existing hasMD5 and hasSHA1
quirks for Owncloud have been renamed to hasOCMD5 and hasOCSHA1.

Fixes #6837
2023-03-16 14:20:29 +00:00