Commit Graph

14 Commits

Author SHA1 Message Date
Nick Craig-Wood
0914ec316c b2: fix multipart upload retries #1733
Prior to this fix we were uploading 0 length bodies if a retry was
needed on a multipart upload chunk.  This gave this error `http:
ContentLength=268435496 with Body length 0`.

Fix by remaking the hash appending reader in the Call loop.  This is
inefficient in the face of retries, but these are uncommon.
2017-12-13 10:11:20 +00:00
Stefan
234bfae0d5 b2: implement streaming upload of files with unknown length (see #1614) (closes #1686) 2017-09-16 22:43:48 +02:00
Stefan Breunig
b510c70c1e b2: calculate missing hashes on the fly instead of spooling – fixes #1288 2017-08-12 12:57:34 +02:00
Nick Craig-Wood
384724fd11 rest, b2, onedrive: remove Absolute parameter from rest.Opts and replace with RootURL 2017-07-23 11:32:34 +01:00
Nick Craig-Wood
80f53176d9 Rename log functions and factor into own file 2017-02-11 17:54:50 +00:00
Nick Craig-Wood
916569102c b2: constrain memory usage when doing multipart uploads #439
Each part of a multipart upload takes 96M of memory, so we make sure
that we don't user more than `--transfers` * 96M of memory buffering
the multipart uploads.

This has the consequence that some uploads may appear to be at 0% for
a while, however they will get going eventually so this won't
re-introduce #731.
2017-02-03 08:03:04 +00:00
Nick Craig-Wood
4105da206a b2: reauth the account while doing uploads too #825
Originally it was thought the upload URL expiring would provide 401
errors so it was excluded from reauth when doing uploads, but on
re-reading the docs and looking at this issue it seems that 401 errors
are only caused by the account token expiring and not the upload token
expiring.

We will refresh both the upload token and account token on a 401 error
while uploading, and just the account token when we get a 401 at any
other time.
2016-11-07 13:30:51 +00:00
Nick Craig-Wood
544ca6035a b2: Make sure each upload has at least one upload slot - fixes #731 2016-10-17 16:48:11 +01:00
Nick Craig-Wood
037a000cc8 b2: fix stats accounting for upload - fixes #602 2016-08-22 21:19:38 +01:00
Nick Craig-Wood
41917eb1f2 b2: Fix upload of files large files not in root - fixes #582 2016-07-13 15:28:39 +01:00
Nick Craig-Wood
c8e2531c8b b2: make error handling compliant 2016-07-01 16:23:23 +01:00
Nick Craig-Wood
cbebefebc4 b2: Fix handling of token expiry #420
Found with --b2-test-mode expire_some_account_authorization_tokens
2016-07-01 11:47:42 +01:00
Nick Craig-Wood
46f8e50614 b2: Make upload multi-threaded - fixes #531 2016-07-01 10:04:52 +01:00
Nick Craig-Wood
2a46be8cf3 b2: implement large file uploading - fixes #456 2016-06-18 13:38:05 +01:00