Commit Graph

18 Commits

Author SHA1 Message Date
Aleksandar Jankovic
8243ff8bc8 accounting: isolate stats to groups
Introduce stats groups that will isolate accounting for logically
different transferring operations. That way multiple accounting
operations can be done in parallel without interfering with each other
stats.

Using groups is optional. There is dedicated global stats that will be
used by default if no group is specified. This is operating mode for CLI
usage which is just fire and forget operation.

For running rclone as rc http server each request will create it's own
group. Also there is an option to specify your own group.
2019-07-28 14:48:19 +01:00
Aleksandar Jankovic
be0464f5f1 accounting: change stats interface
This is done to make clear ownership over accounting object and prepare
for removing global stats object.

Stats elapsed time calculation has been altered to account for actual
transfer time instead of stats creation time.
2019-07-28 14:48:19 +01:00
Aleksandar Jankovic
f78cd1e043 Add context propagation to rclone
- Change rclone/fs interfaces to accept context.Context
- Update interface implementations to use context.Context
- Change top level usage to propagate context to lover level functions

Context propagation is needed for stopping transfers and passing other
request-scoped values.
2019-06-19 11:59:46 +01:00
Fabian Möller
552eb8e06b vfs: try to seek buffer on read only files 2018-08-17 18:10:28 +01:00
Fabian Möller
bd3ad1ac3e vfs: add option to read source files in chunks 2018-05-05 12:49:42 +01:00
Nick Craig-Wood
e5be471ce0 Use io.SeekStart/End/Current constants now for go1.7+ #2154 2018-04-07 11:42:36 +01:00
Nick Craig-Wood
7fb53a031c vfs: don't cache the object in read and read/write handles
This should help with inconsistent reads when the source object changes.
2018-03-04 11:20:22 +00:00
Nick Craig-Wood
44276db454 vfs: make -ve sized files appear as 0 size. #320
This means that Google docs will no longer appear as huge files in
`rclone mount`.  They will not be downloadable, though sometimes
trying twice will work.
2018-01-31 16:22:05 +00:00
Nick Craig-Wood
11da2a6c9b Break the fs package up into smaller parts.
The purpose of this is to make it easier to maintain and eventually to
allow the rclone backends to be re-used in other projects without
having to use the rclone configuration system.

The new code layout is documented in CONTRIBUTING.
2018-01-15 17:51:14 +00:00
Nick Craig-Wood
76b151984c vfs: cache the size of the object in the read handle 2017-11-20 17:57:13 +00:00
Nick Craig-Wood
60945d0a37 vfs: remove misleading comment 2017-11-20 12:46:23 +00:00
Nick Craig-Wood
c31556c6d1 vfs: Make sure all public methods are locked in Read and Write Handle 2017-11-04 10:24:10 +00:00
Nick Craig-Wood
2083ac6e2a vfs: add ECLOSED and tidy errors 2017-11-04 10:24:10 +00:00
Nick Craig-Wood
5634659ea3 mount,vfs: unify Read and Write handles in preparation for ReadWrite handles 2017-11-04 10:24:10 +00:00
Nick Craig-Wood
e18122e88b vfs: add tests and subsequent fixes
* Tests for VFS layer
  * Small fixes found during testing
  * Fix Close, Flush and Release behaviour for ReadFileHandle and WriteFileHandle
  * Fix nil object bugs on File
2017-11-04 10:24:10 +00:00
Nick Craig-Wood
a5dc62f6c1 vfs: Make file handles compatible with OS
* Implement directory handles
  * Unify OpenFile
  * Add all the methods to match *os.File
  * Add StatParent and Rename methods to VFS
2017-11-03 13:00:00 +00:00
Nick Craig-Wood
e8883e9fdb vfs: factor flags into vfsflags and remove global variables 2017-11-03 12:59:59 +00:00
Nick Craig-Wood
c1aaff220d Factor new vfs module out of cmd/mountlib
This is an OS style file system abstraction with directory caching
used in mount, cmount, serve webdav and serve http.
2017-11-03 12:59:59 +00:00