YenForYang 269f90c1e4 drive: Fix buffering for single request upload for files smaller than --drive-upload-cutoff ... I discovered that `rclone` always upload in chunks of 16MiB whenever uploading a file smaller than `--drive-upload-cutoff`. This is undesirable since the purpose of the flag `--drive-upload-cutoff` is to *prevent* chunking below a certain file size. I realized that it wasn't `rclone` forcing the 16MiB chunks. The `google-api-go-client` forces a chunk size default of [`googleapi.DefaultUploadChunkSize`](32bf29c2e1/googleapi/googleapi.go (L55-L57)) bytes for resumable type uploads. This means that all requests that use `*drive.Service` directly for upload without specifying a `googleapi.ChunkSize` will be forced to use a *`resumable`* `uploadType` (rather than `multipart`) for files less than `googleapi.DefaultUploadChunkSize`. This is also noted directly in the Drive API client documentation [here](https://pkg.go.dev/google.golang.org/api/drive/v3@v0.44.0#FilesUpdateCall.Media). This fixes the problem by passing `googleapi.ChunkSize(0)` to `Media()` method calls, which is the only way to disable chunking completely. This is mentioned in the API docs [here](https://pkg.go.dev/google.golang.org/api/googleapi@v0.44.0#ChunkSize). The other alternative would be to pass `googleapi.ChunkSize(f.opt.ChunkSize)` -- however, I'm *strongly* in favor of *not* doing this for performance reasons. By not explicitly passing a `googleapi.ChunkSize(0)`, we effectively allow [`PrepareUpload()`](https://pkg.go.dev/google.golang.org/api/internal/gensupport@v0.44.0#PrepareUpload) to create a [`NewMediaBuffer`](https://pkg.go.dev/google.golang.org/api/internal/gensupport@v0.44.0#NewMediaBuffer) that copies the original `io.Reader` passed to `Media()` in order to check that its size is less than `ChunkSize`, which will unnecessarily consume time and memory. `minChunkSize` is also changed to be `googleapi.MinUploadChunkSize`, as it is something specified we have no control over.		2021-10-08 15:29:38 +01:00
..
test	drive: add --drive-import-formats	2018-09-25 15:20:23 +01:00
drive_internal_test.go	drive: constrain list by filter #5023	2021-10-07 22:11:22 +03:00
drive_test.go	build: fix up package paths after repo move	2019-07-28 18:47:38 +01:00
drive.go	drive: Fix buffering for single request upload for files smaller than --drive-upload-cutoff	2021-10-08 15:29:38 +01:00
upload.go	backends: Add context checking to remaining backends #4504	2021-03-16 16:17:22 +00:00