rclone/vfs
Leo Luan 8f23cae1c0 vfs: Add cache reset for --vfs-cache-max-size handling at cache poll interval
The vfs-cache-max-size parameter is probably confusing to many users.
The cache cleaner checks cache size periodically at the --vfs-cache-poll-interval
(default 60 seconds) interval and remove cache items in the following order.

(1) cache items that are not in use and with age > vfs-cache-max-age
(2) if the cache space used at this time still is larger than
vfs-cache-max-size, the cleaner continues to remove cache items that are
not in use.

The cache cleaning process does not remove cache items that are currently in use.
If the total space consumed by in-use cache items exceeds vfs-cache-max-size, the
periodical cache cleaner thread does not do anything further and leaves the in-use
cache items alone with a total space larger than vfs-cache-max-size.

A cache reset feature was introduced in 1.53 which resets in-use (but not dirty,
i.e., not being updated) cache items when additional cache data incurs an ENOSPC
error.  But this code was not activated in the periodical cache cleaning thread.

This patch adds the cache reset step in the cache cleaner thread during cache
poll to reset cache items until the total size of the remaining cache items is
below vfs-cache-max-size.
2021-04-26 17:55:52 +01:00
..
test_vfs lib/random: unify random string generation into random.String 2019-08-06 12:44:08 +01:00
vfscache vfs: Add cache reset for --vfs-cache-max-size handling at cache poll interval 2021-04-26 17:55:52 +01:00
vfscommon vfs: --vfs-used-is-size to report used space using recursive scan (#4043) 2021-02-17 23:36:13 +03:00
vfsflags vfs: --vfs-used-is-size to report used space using recursive scan (#4043) 2021-02-17 23:36:13 +03:00
vfstest mount: more user friendly mounting as network drive on windows 2020-12-28 13:59:34 +00:00
dir_handle_test.go vfs: add --vfs-writeback option to delay writes back to cloud storage 2020-06-30 12:01:36 +01:00
dir_handle.go Spelling fixes 2020-10-14 15:21:31 +01:00
dir_test.go vfs: fix Create causing windows explorer to truncate files on CTRL-C CTRL-V 2021-03-31 14:48:02 +01:00
dir.go vfs: fix Create causing windows explorer to truncate files on CTRL-C CTRL-V 2021-03-31 14:48:02 +01:00
errors_test.go vfs: add tests and subsequent fixes 2017-11-04 10:24:10 +00:00
errors.go build: drop support for go1.8 2019-04-15 21:49:58 +01:00
file_test.go vfs: fix modtime set if --vfs-cache-mode writes/full and no write 2021-03-16 13:36:48 +00:00
file.go vfs: fix modtime set if --vfs-cache-mode writes/full and no write 2021-03-16 13:36:48 +00:00
help.go vfs: --vfs-used-is-size to report used space using recursive scan (#4043) 2021-02-17 23:36:13 +03:00
make_open_tests.go vfs: bring DO NOT EDIT comments in line with "go help generate" 2020-04-30 12:24:44 +01:00
open_test.go vfs: bring DO NOT EDIT comments in line with "go help generate" 2020-04-30 12:24:44 +01:00
rc_test.go vfs: fix vfs/refresh calls with fs= parameter 2020-11-07 14:26:33 +00:00
rc.go vfs: fix vfs/refresh calls with fs= parameter 2020-11-07 14:26:33 +00:00
read_test.go vfs: add --vfs-writeback option to delay writes back to cloud storage 2020-06-30 12:01:36 +01:00
read_write_test.go vfs: Make tests run reliably 2020-06-30 12:01:36 +01:00
read_write.go vfs: fix modtime set if --vfs-cache-mode writes/full and no write 2021-03-16 13:36:48 +00:00
read.go fs: deglobalise the config #4685 2020-11-26 16:40:12 +00:00
vfs_case_test.go vfs: add --vfs-writeback option to delay writes back to cloud storage 2020-06-30 12:01:36 +01:00
vfs_test.go Spelling fixes 2020-10-14 15:21:31 +01:00
vfs.go vfs: --vfs-used-is-size to report used space using recursive scan (#4043) 2021-02-17 23:36:13 +03:00
vfstest_test.go mount: change interface of mount commands to take mount options 2020-07-24 10:48:51 +01:00
vstate_string.go vfs: fix virtual entries causing deleted files to still appear 2020-11-10 16:47:25 +00:00
write_test.go vfs: add --vfs-writeback option to delay writes back to cloud storage 2020-06-30 12:01:36 +01:00
write.go docs: spelling: etc. 2020-10-28 18:16:23 +00:00