This change adds the --direct-io flag to the mount. This means the
page cache is completely bypassed for reads and writes. No read-ahead
takes place. Shared mmap is disabled.
This is useful to accurately read files which may change length
frequently on the source.
Before this fix, we told cgofuse/WinFSP that the backend was case
insensitive but didn't implement the Getpath backend function to
return the normalised case of a file.
Resently cgofuse started implementing case insensitive files properly
but since we hadn't implemented Getpath, the file names were taking
the default of all in UPPER CASE.
This patch implements Getpath for cgofuse which fixes the case
problems.
This problem came to light when we upgraded cgofuse and WinFSP (to
1.12) which had the code to implement Getpath.
Fixes#6682
This is possible now that we no longer support go1.12 and brings
rclone into line with standard practices in the Go world.
This also removes errors.New and errors.Errorf from lib/errors and
prefers the stdlib errors package over lib/errors.
Before this change when reading directories we would use the directory
handle and the Readdir(-1) call on the directory handle. This worked
fine for the first read, but if the directory was read again on the
same handle Readdir(-1) returns nothing (as per its design).
It turns out that macOS leaves the directory handle open and just
re-reads the data from it, so this problem causes directories to start
out full then subsequently appear empty.
macOS/OSXFUSE is passing an offset of 0 to the Readdir call telling
rclone to seek in the directory, but we've told FUSE that we can't
seek by always returning ofst=0 in the fill function.
This fix works around the problem by reading the directory from the
path each time, ignoring the actual handle. This should be no less
efficient.
We will return an ESPIPE if offset is ever non 0.
There are possible corner cases reading deleted directories which this
ignores.
Before this change Windows would read a directory then immedately stat
every item in the directory.
After this change we return the stat information along with the
directory which stops so many callbacks.
Before this change, files of unknown size (eg Google Docs) would
appear in file listings with 0 size and would only allow 0 bytes to be
read.
This change sets the direct_io flag in the FUSE return which bypasses
the cache for these files. This means that they can be read properly.
This is compatible with some, but not all applications.
OSX FUSE only supports 32 bit number of blocks which means that block
counts have been wrapping. This causes f_bavail to be 0 which in turn
causes problems with programs like borg backup.
Fixes#2356
Previously this was adding it in to all file opens which was causing
inefficiencies under Windows where it stats the file using
open/fstat/close.
This change will make stat operations run much quicker under Windows
as they won't have to open the underlying file
This problem was introduced in61b6159a05336bd7ba105766de2d2ff171f7fb81
where we added O_CREATE to all file opens and creates.
Before this change Open("name", os.O_RDONLY|os.O_TRUNC) would have
truncated the file. This is what Linux does, but is counterintuitive.
POSIX states this is undefined, so return an error in this case
instead. This preserves the invariant O_RDONLY => file is not
changed.
The purpose of this is to make it easier to maintain and eventually to
allow the rclone backends to be re-used in other projects without
having to use the rclone configuration system.
The new code layout is documented in CONTRIBUTING.
On Windows the fuse.O_* flags do not have the same values as the
os.O_* flags so translate between the two representations. They are
mostly the same which is why this hasn't caused a problem before.
* Tests for VFS layer
* Small fixes found during testing
* Fix Close, Flush and Release behaviour for ReadFileHandle and WriteFileHandle
* Fix nil object bugs on File
