Antonio SJ Musumeci
2696079601
break fs.hpp up into separate files
2020-08-18 19:28:36 -04:00
Antonio SJ Musumeci
ec15872a1f
cleanup function signatures and definitions
2020-08-18 19:28:36 -04:00
Antonio SJ Musumeci
6cc6524997
change category to enum class
2020-08-18 19:28:36 -04:00
Antonio SJ Musumeci
0371b047f9
change from fasthash64 to wyhash
...
wyhash has very good performance for small keys (our usecase) and
works on platforms with alignment concerns.
A user had an issue where fasthash64 lead to misaligned reads and
signal exceptions. wyhash does not have such issues and is faster.
2020-08-06 19:00:05 -04:00
Antonio SJ Musumeci
30d13b7f77
inodecalc: add 32bit versions of hashs
2020-08-03 19:42:06 -04:00
Antonio SJ Musumeci
5989d41a86
readdir: use getdents64 for compatibility with ARM64
2020-08-02 18:43:09 -04:00
Antonio SJ Musumeci
0709b2e447
add missing options to usage
2020-07-30 16:55:18 -04:00
Antonio SJ Musumeci
15a0aedd25
cleanup: move some config data structures to separate files
2020-07-29 21:42:43 -04:00
Antonio SJ Musumeci
a93bd9f7fa
freebsd: misc cleanups to get freebsd compiling
2020-07-29 13:08:38 -04:00
Antonio SJ Musumeci
139e61efaa
nfsopenhack: remove empty file check
2020-07-26 17:11:06 -04:00
Antonio SJ Musumeci
93218a343a
NFS open/creat hack
...
A hack to work around non-POSIX NFS <-> FUSE behavior where it turns
an open(O_EXCL|O_CREAT,0444) into multiple calls that results in
EACCES.
2020-07-23 19:36:10 -04:00
Antonio SJ Musumeci
3ec137c4ad
policy: add "most shared path" policies
...
Like path preserving but walks back the path till a match is found. Should
cover the usecase where someone wants a "less strict" form of path
preservation.
2020-07-21 16:31:08 -04:00
Andrea Gelmini
3a6738475a
Fix typos
2020-07-20 13:01:33 +02:00
Andrea Gelmini
468d42088a
Removed duplicated include
2020-07-20 12:51:44 +02:00
Antonio SJ Musumeci
08d267fc61
moveonenospc: enhance the feature to allow using a policy
...
Just like functions you can now set a policy for moveonfreenospc. This
allows for more flexibility.
For backwards compatibility moveonfreenospc=true is converted to
moveonfreenospc=mfs.
minfreespace does apply which is slightly different from original behavior.
2020-07-15 19:33:05 -04:00
Antonio SJ Musumeci
c0990644e8
remove unnecessary libfuse flags nopath, nullpath_ok, and utime_omit_ok
2020-07-08 19:25:23 -04:00
Antonio SJ Musumeci
b4397f7f2e
add 'inodecalc' option to allow selection of inode calculation algo
2020-06-28 20:02:07 -04:00
Antonio SJ Musumeci
dbdd3e22fc
additional readdir refactor cleanup
...
Differences between readdir and getdents is minimal at best. Leaving code
for now to allow for possible expansion later.
2020-06-27 23:17:21 -04:00
Antonio SJ Musumeci
54c41c4994
rework config management
...
* Add standard way to do str2val and val2str conversion
* Add support for a config file
* Add support for reading and writing settings via ioctl
2020-06-18 18:54:25 -04:00
Antonio SJ Musumeci
698c4147fc
fix getdent name length calculation
2020-03-12 23:45:20 -04:00
Antonio SJ Musumeci
3a46ec9fab
fix filename hashing error
2020-03-07 12:09:40 -05:00
Antonio SJ Musumeci
9b2634a1e5
fix name length calculation for musc
2020-02-29 17:15:29 -05:00
Antonio SJ Musumeci
62873d2d3b
use getdents64 on linux
2020-02-28 20:34:12 -05:00
Antonio SJ Musumeci
d119807adb
restructure readdir, add readdir_plus
...
Does not enable READDIR_AUTO. Might add in the future.
2020-02-22 19:45:52 -05:00
Antonio SJ Musumeci
a646fe016d
change inode conversion algo to reduce collision
2020-02-12 21:05:52 -05:00
Antonio SJ Musumeci
576ff3694e
add cache.writeback to xattrs
2020-02-12 19:48:37 -05:00
Antonio SJ Musumeci
903d39f968
add writeback caching
2020-02-09 21:08:53 -05:00
Antonio SJ Musumeci
4d82ed9324
fix short writes on >2GB files when cloning file
2019-10-14 23:32:51 -04:00
Antonio SJ Musumeci
0fffabfbe3
only return 1 branch for rand/eprand policies
2019-09-07 17:25:06 -04:00
Antonio SJ Musumeci
50ad648ce4
initialize mutex to fix lockup
2019-09-07 14:10:36 -04:00
Antonio SJ Musumeci
4c4c27a93e
set uid & gid when calling ioctl
...
Turns out certain ioctl calls check the effective user id (FS_IOC_SETFLAGS).
2019-06-17 13:37:15 -04:00
Antonio SJ Musumeci
5ca928e436
accept old arguments for backwards compatibility
2019-06-10 20:20:41 -04:00
Antonio SJ Musumeci
7cbd88ac81
allow setting of 'max_pages' (via 'fuse_msg_size')
...
Linux 4.20 and above allow setting the number of pages per FUSE message
upto 256 (4K * 256 = 1MiB). This can greatly increase read and write
speeds depending on the workload.
2019-06-03 21:01:19 -04:00
Antonio SJ Musumeci
8cb7195c3e
add copy_file_range support
2019-06-03 07:12:05 -04:00
Antonio SJ Musumeci
529a953e30
add file caching across opens and runtime control
2019-05-30 00:10:08 -04:00
Antonio SJ Musumeci
ddf6a2f105
make async_read optional again
2019-05-28 15:57:50 -04:00
Antonio SJ Musumeci
2323c16316
add readdir caching
2019-05-23 22:11:33 -04:00
Antonio SJ Musumeci
1baa706d37
add symlink caching
2019-05-22 19:25:38 -04:00
Antonio SJ Musumeci
61cded5b34
fix for unlink race condition
2019-05-21 22:57:02 -04:00
Antonio SJ Musumeci
5f2221155c
ensure parallel dirops is enabled if capable
2019-05-20 00:36:42 -04:00
Antonio SJ Musumeci
2b019b84b0
ensure async_aio is enabled if capable
2019-05-20 00:18:56 -04:00
trapexit
af1c0d78e8
Merge pull request #621 from trapexit/posix-acl
...
add support for POSIX ACLs
2019-05-19 18:26:49 -04:00
Antonio SJ Musumeci
80d56ac94e
add support for POSIX ACLs
2019-05-19 18:17:55 -04:00
Antonio SJ Musumeci
1ca70521a4
remove 'remote' flock support
2019-05-19 17:34:56 -04:00
Antonio SJ Musumeci
e052446713
fix setting of fsname
2019-05-12 14:07:21 -04:00
Antonio SJ Musumeci
5883020784
tweak docs
2019-05-10 11:54:41 -04:00
Antonio SJ Musumeci
6ecc618d83
remove .fuse_hidden file creation
...
addresses #521
mergerfs is written using libfuse's high level API. The HLAPI forces a certain
behavior with regard to unlinking or rename overwriting of open files. It
renames the file to .fuse_hiddenXXXXXX and when incoming requests refer
to the unlinked file it will use path based calls and use that path.
This behavior leads to rmdirs failing or some software noticing
the .fuse_hidden files.
This patch changes the HLAPI and removes entirely the .fuse_hidden behavior.
See the README for more.
2019-04-13 21:41:51 -04:00
Antonio SJ Musumeci
2a075ead92
fall back to other file copy methods in clonefile
2019-04-13 15:08:13 -04:00
Antonio SJ Musumeci
b69819e4b2
ioctl: don't set outbufsz when not needed
2019-03-20 10:18:55 -04:00
Antonio SJ Musumeci
0918dfd117
make attr, entry, negative_entry cache timeouts runtime configurable
2019-02-23 01:03:20 -05:00
Antonio SJ Musumeci
95c0cc741d
replace libfuse's autoconf with makefile
2019-02-18 21:36:32 -05:00
Antonio SJ Musumeci
7bf607bb6b
fix FS_IOC_{GET,SET}{FLAGS,VERSION} ioctl calls
2019-02-17 11:48:19 -05:00
Antonio SJ Musumeci
940c323251
misc fixes for FreeBSD
2019-02-06 23:20:28 -05:00
Antonio SJ Musumeci
5ab7d2dd72
remove O_LARGEFILE
2019-02-06 21:06:11 -05:00
Antonio SJ Musumeci
9d9ee7b71d
general code cleanup
2019-01-31 22:47:09 -05:00
Antonio SJ Musumeci
c5b2415daf
remove defaults, hard code atomic_o_trunc, big_writes, and default_permissions
...
`defaults` is a value used by all filesystems and isn't passed through to
mergerfs when mounting via the fstab or the mount command. This led
to inconsistent application of options. atomic_o_trunc, big_writes, and
default_permissions should be enabled all the time anyway and splice_*
can lead to issues so they are not always enabled.
2019-01-30 22:46:21 -05:00
Antonio SJ Musumeci
5be7e007ce
add statfs cache
2019-01-23 23:06:07 -05:00
Antonio SJ Musumeci
7a057daa0c
add policy cache for 'open'
...
A fusepath -> basepath cache for `open` to limit the overhead of FUSE in 'open, read/write, close' patterns (such as Transmission).
2019-01-22 09:21:15 -05:00
Antonio SJ Musumeci
9fd3b968fc
make ioctl on directories use open policy
2019-01-18 22:41:52 -05:00
Antonio SJ Musumeci
87b2795f2b
set direct_io per open/create, now runtime configurable
2019-01-11 15:23:14 -05:00
Antonio SJ Musumeci
10f7f8bdea
optimize link_cow eligibility check
2019-01-04 10:21:28 -05:00
Antonio SJ Musumeci
dfa1c1ad6f
check minfreespace on newest policy create
2018-12-12 15:53:53 -05:00
Antonio SJ Musumeci
6ae68465cf
fix building on alpine w/ musl
2018-11-28 17:08:40 -05:00
trapexit
3c1f19e85a
Merge pull request #534 from trapexit/xattr
...
change xattr setting notsup to nosys
2018-11-19 23:32:07 -05:00
Antonio SJ Musumeci
7d9458fdd1
change xattr setting notsup to nosys
2018-11-19 23:30:39 -05:00
Antonio SJ Musumeci
43b676a6e2
fix indexing of truncate targets
2018-11-19 23:25:16 -05:00
Antonio SJ Musumeci
c46134c03a
fix building on platforms without O_PATH
2018-11-06 07:59:28 -05:00
Antonio SJ Musumeci
3631ab7df2
optimize readdir file dedup
...
Use fasthash64 to hash filenames to uint64_t and store in khash
set. Significantly reduces malloc/free'ing and memory usage.
2018-11-06 00:05:30 -05:00
Antonio SJ Musumeci
680f8194f9
add ability to change statfs behavior
2018-11-05 21:33:27 -05:00
Antonio SJ Musumeci
7524e57262
rename NW (no write) to NC (no create)
2018-11-01 23:37:45 -04:00
Antonio SJ Musumeci
b55ebba4ed
add tagging branches RW/RO/NW
...
This allows users to tag a branch as readonly or not for writing regardless
of how the filesystem is mounted. Should simplify deployments and offer
more flexibility.
2018-11-01 23:13:49 -04:00
Antonio SJ Musumeci
9e0ab1f518
misc cleanups
2018-10-24 22:38:36 -04:00
Antonio SJ Musumeci
8a48b74cb4
policy return cleanup
2018-10-15 19:03:02 -04:00
Antonio SJ Musumeci
85026d5780
add FICLONE and copy_file_range to clonefile
...
If available FICLONE and copy_file_range will be tried in addition to sendfile
when copying data between two files. The fallback is a tradition read/write
loop. On systems that support these it should improve performance.
2018-10-12 10:18:49 -04:00
Antonio SJ Musumeci
1885a82cce
remove libattr dependency
2018-10-09 13:50:45 -04:00
Antonio SJ Musumeci
65f482e483
add ability to turn on/off xattr support at runtime
2018-10-09 08:41:31 -04:00
Antonio SJ Musumeci
8d1a1564bf
fix building on certain platforms
2018-10-09 08:23:37 -04:00
Antonio SJ Musumeci
93f7d7d927
add link_cow feature
...
When enabled if a regular file is opened which has a link count > 1 it will
copy the file to a temporary file and rename over the original. Effectively
breaking the link. This behavior is similar to cow-shell and other LD_PRELOAD
based "CoW" solutions.
2018-10-06 22:05:05 -04:00
Antonio SJ Musumeci
9afefef032
keep literal when glob fails
2018-10-01 16:56:40 -04:00
Antonio SJ Musumeci
f8563369bc
add security_capability option
2018-09-30 23:30:53 -04:00
Andrey Mazo
ec6adaa376
options: move "-o threads=" help text to libfuse/
...
Parsing of and acting upon "-o threads" option is done in the bundled `libfuse/`,
so move the corresponding help text there too.
This make help text consistent with behavior
when building against system libfuse,
which doesn't support this option.
2018-08-19 21:04:35 +03:00
Antonio SJ Musumeci
9830e29945
fix version generation
2018-03-09 21:06:00 -05:00
Antonio SJ Musumeci
cfe9c28a10
ignore clonepath metadata errors. fixes #470
2018-03-08 16:55:35 -05:00
Antonio SJ Musumeci
f48c16d162
stop clonepath at base directory. fixes #467
2018-03-07 18:46:32 -05:00
Antonio SJ Musumeci
1d6d22727c
call 32bit versions of set/geteuid on 32bit platforms
2017-10-17 22:21:29 -04:00
Antonio SJ Musumeci
75ed37a11a
add setting of thread pool size
2017-07-02 23:50:39 -04:00
Antonio SJ Musumeci
0708110ec1
improve khash performance
2017-07-01 11:53:35 -04:00
Antonio SJ Musumeci
a7b126cb48
use pragma once in headers
2017-06-30 11:38:34 -04:00
Antonio SJ Musumeci
a2bddec8c7
add 'ignore path preserving on rename' feature
2017-06-30 00:01:00 -04:00
Antonio SJ Musumeci
aea2b40dda
use temp files (then rename) when moving files for moveonenospc
2017-06-28 21:17:47 -04:00
Antonio SJ Musumeci
bfd410def2
use fusepath from fileinfo. closes #417
2017-06-24 17:03:41 -04:00
Antonio SJ Musumeci
7296d3d90d
add time.h to fix compiling on some platforms
2017-06-15 19:35:28 -04:00
Antonio SJ Musumeci
de0985c135
add libfuse 2.9.7 to repo and build against libfuse.a
2017-06-02 11:25:25 -04:00
Antonio SJ Musumeci
87c2f2f9dc
add nullrw feature to facilitate benchmarking
2017-05-26 18:11:18 -04:00
Antonio SJ Musumeci
6a7675f2bf
symlinkify: file -> symlink-to-original-file after timeout
2017-05-05 00:14:04 -04:00
Antonio SJ Musumeci
8ba3a08f40
make dropcacheonclose runtime configurable
2017-04-21 16:58:39 -04:00
Antonio SJ Musumeci
ccaa458739
better handle incomplete reads/writes in copying files
2017-04-13 12:55:34 -04:00
Antonio SJ Musumeci
162b99e6b8
enable nopath and nullpath_ok
2017-04-12 13:52:06 -04:00
Antonio SJ Musumeci
f15437c713
tweak movefile behavior
2017-04-11 15:24:09 -04:00
Antonio SJ Musumeci
2fbeb67dd9
hide fs::fadvise as it's not used directly
2017-04-11 09:42:33 -04:00
Antonio SJ Musumeci
8b976ab3c1
support older libfuse without utime_omit_ok flag
2017-04-11 09:23:40 -04:00
Antonio SJ Musumeci
1a1fa06aa3
fadvise cleanup
2017-04-11 09:02:05 -04:00
Antonio SJ Musumeci
617195d71e
enable utime_omit_ok flag
2017-04-10 21:36:56 -04:00
Antonio SJ Musumeci
be3eb7e3ed
work around getgrouplist signature difference on osx
2017-04-07 16:23:18 -04:00
Antonio SJ Musumeci
0600734942
handle 32bit and 64bit inode recalculation
2017-04-07 09:47:19 -04:00
Antonio SJ Musumeci
9d0798dbfb
restructure fadvise
2017-04-07 00:12:57 -04:00
Antonio SJ Musumeci
e2acffe55b
restructure fallocate abstraction
2017-04-06 23:35:31 -04:00
Antonio SJ Musumeci
42d454ac27
abstract futimesat
2017-04-06 23:13:02 -04:00
Antonio SJ Musumeci
0b2bf17cd7
abstract access to highres atime/mtime
2017-04-06 22:16:17 -04:00
Antonio SJ Musumeci
e20d566f96
use correct integer types
2017-04-06 21:29:39 -04:00
Antonio SJ Musumeci
c043ef95a3
make fs::attr return ENOTSUP on EINVAL #381
2017-04-03 21:11:20 -04:00
Antonio SJ Musumeci
6aa62d03ff
add option to drop file caches before closing files
2017-02-18 16:06:29 -05:00
Antonio SJ Musumeci
492d895632
check metadata on chown/chmod errors when cloning
2017-02-15 09:08:27 -05:00
Antonio SJ Musumeci
9cc9bb9de2
misc document updates
2017-01-31 19:10:54 -05:00
Antonio SJ Musumeci
a60d815e92
add ifndefs to all headers
2017-01-17 11:45:44 -05:00
Antonio SJ Musumeci
e93c946198
limit need to explicitly call .c_str()
2017-01-17 11:45:44 -05:00
Antonio SJ Musumeci
7b4e1ea36d
remove clone command
2017-01-17 11:45:44 -05:00
Antonio SJ Musumeci
726b88e039
restructure error calculation
2017-01-17 11:45:43 -05:00
Antonio SJ Musumeci
d67d5dec77
check for system.posix_acl_default before setting umask
2017-01-16 12:19:58 -05:00
Antonio SJ Musumeci
1aa76a54c2
use different read and write functions when using direct_io
2016-12-20 17:06:34 -05:00
Antonio SJ Musumeci
67b48fcb3a
compute inode in readdir
2016-12-16 10:57:24 -05:00
Antonio SJ Musumeci
c8fa51c81a
support setting of inodes (using use_ino option)
...
creates a 64bit inode value from the underlying device value + original inode
final_ino = orig_ino | (dev << 32)
not perfect but given few filesystems use 64bit inodes nor is st_dev more than 16bit usually it should be fine
2016-12-14 18:53:21 -05:00
Antonio SJ Musumeci
822204ff34
replace std::set with klib's khash to increase readdir performance
2016-12-14 17:24:48 -05:00
Antonio SJ Musumeci
35075bb0cd
return clonepath errors
...
currently the error is ignored and it was expected the primary call would
fail. problem is it returns confusing errors as a result. (eg ENOENT vs EPERM)
2016-12-06 16:48:52 -05:00
Antonio SJ Musumeci
3c5351af93
ignore filesystems which return zeros for statfs. closes #335
2016-11-15 23:27:17 -05:00
Antonio SJ Musumeci
157dae0d2f
define O_LARGEFILE and O_NOATIME if needed
2016-10-28 22:04:15 -04:00
Antonio SJ Musumeci
00c814dc58
consolidate and simplify utime
2016-10-28 22:04:09 -04:00
Antonio SJ Musumeci
6d6fb45a3b
check if fdatasync is available and return ENOSYS if not
2016-10-27 16:59:04 -04:00
Antonio SJ Musumeci
d0b6cd1f38
further abstraction of system calls
2016-10-20 16:51:49 -04:00
Antonio SJ Musumeci
1dc7bff6e6
wrap most posix filesystem functions
2016-10-19 16:58:52 -04:00
Antonio SJ Musumeci
8f594e17d7
add flock
2016-10-19 11:58:44 -04:00
Antonio SJ Musumeci
3fb7f8919a
add EDQUOT to errors which trigger moveonenospc
2016-10-06 13:54:57 -04:00
Antonio SJ Musumeci
d9a79062b8
use SYS_setgroup32 syscall if available. closes #319
2016-09-19 17:04:11 -04:00
Antonio SJ Musumeci
7e423cd9ce
small tweaks to build on Debian kFreeBSD
2016-09-14 08:36:06 -04:00
Antonio SJ Musumeci
0395e7c776
fix futimes version of utimes wrapper
2016-09-13 21:30:45 -04:00
Antonio SJ Musumeci
93923177ee
fix #define typo
2016-09-13 17:40:23 -04:00
Antonio SJ Musumeci
1513c92cbc
abstract posix_fadvise
2016-09-13 17:36:28 -04:00
Antonio SJ Musumeci
1a698e5ef5
rename include cpp files to have icpp extension
2016-08-26 14:41:56 -04:00
Antonio SJ Musumeci
2ee6b4f581
include sys/types.h to pick up ssize_t
2016-08-26 13:09:15 -04:00
Antonio SJ Musumeci
709dda5069
support systems without ENODATA
2016-08-26 12:59:32 -04:00
Antonio SJ Musumeci
49474f01ca
make futimes crossplatform
2016-08-07 14:42:54 -04:00
Antonio SJ Musumeci
f7547d7cf5
Merge pull request #299 from trapexit/realpath
...
use dynamic buffer for realpath
2016-08-04 17:14:09 -04:00
Antonio SJ Musumeci
bb02ab2cba
Merge pull request #298 from trapexit/fssendfile
...
split sendfile wrapper into separate files
2016-08-04 17:13:28 -04:00
Antonio SJ Musumeci
34d38cb0fb
split sendfile wrapper into separate files
2016-08-04 17:11:23 -04:00
Antonio SJ Musumeci
192a9d54d1
make fs_attr compile on unsupported platforms
2016-08-04 17:10:02 -04:00
Antonio SJ Musumeci
40574bd138
use dynamic buffer for realpath
2016-08-04 14:52:27 -04:00
Antonio SJ Musumeci
45f757dc60
add osx version of fallocate
2016-08-02 17:14:45 -04:00
Antonio SJ Musumeci
0fceb8e0a3
add epall and eprand policies
2016-08-01 16:08:02 -04:00
Antonio SJ Musumeci
7634eb1440
replace nonstandard eaccess with POSIX.1-2008 faccessat
2016-07-29 12:39:44 -04:00
Antonio SJ Musumeci
a93ab6c2f5
add existing path first found policy. closes #289
2016-07-11 20:42:56 -04:00
