github-mirror/mergerfs
2
0
Fork 0
mirror of https://github.com/trapexit/mergerfs.git synced 2024-12-04 06:34:05 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
1,233 Commits 30 Branches 95 Tags 27 MiB
aa128fa808
Commit Graph

384 Commits

Author SHA1 Message Date
Antonio SJ Musumeci
9effeb6869 Add "per-process" file caching feature 
Allows chosing page caching based on process name.
 2023-02-18 01:16:36 -05:00
Antonio SJ Musumeci
2d9c915641 Return NOTTY for all btrfs ioctl calls 
mergerfs tries to pass through ioctl calls but due to the complexity
of btrfs calls their is risk of segfaults. In the future it may be
required to whitelist all supported ioctl calls.
 2023-02-11 10:07:55 -05:00
Antonio SJ Musumeci
9adddfc66d Add option to lazy umount target mount point 2023-02-02 21:20:23 -05:00
Antonio SJ Musumeci
3e313cd605 Forbid adding mountpoint to branch list 2023-01-29 11:51:56 -05:00
Antonio SJ Musumeci
92ccfd3c89 Hard code setting of allow_other, ignore when set 2023-01-26 23:37:37 -05:00
Antonio SJ Musumeci
a7113bb163 Allow setting of scheduling priority 2023-01-25 23:38:31 -05:00
Antonio SJ Musumeci
939eb3996a Add option to wait for branches to become new mounts 
branches-mount-timeout=UINT64 in seconds (default: 0)
 2023-01-25 11:59:14 -05:00
Antonio SJ Musumeci
3ee93d4c56 Remove 'nonempty' argument 
This makes it like all other filesystems and brings it into alignment
with libfuse3's behavior.
 2023-01-16 23:33:36 -05:00
Antonio SJ Musumeci
d11807ef7c Remove use_ino option and make behavior same as if set 2023-01-16 16:21:44 -05:00
Antonio SJ Musumeci
24423b8d2a Add async message processing 2023-01-16 13:15:53 -05:00
Antonio SJ Musumeci
a6ca96fa08 Fix printf for 32bit systems 2022-12-08 22:04:15 -05:00
Antonio SJ Musumeci
7606430188 Remove libfuse abstraction in prep for adding request data 2022-11-26 23:30:27 -06:00
Antonio SJ Musumeci
32e2c9c48e Tweaks for 32bit systems 2022-11-05 23:28:10 -04:00
Antonio SJ Musumeci
3b0d7fb26e Fix setting of stat vars for symlinkify 2022-10-30 22:27:33 -04:00
Antonio SJ Musumeci
46ce28b149 Error when given invalid policy names 2022-08-06 23:01:23 -04:00
Andrea Gelmini
338e559e53
Fix typos 2022-06-22 11:51:23 +02:00
Antonio SJ Musumeci
c9a935835f Fix query of attr during symlink 
Using getattr when follow-symlink is enabled causes invalid type to the kernel
if symlink pointed to non-symlink.
 2022-03-19 16:18:46 -04:00
Antonio SJ Musumeci
c43b464d6c Call FUSE getattr rather than syscall for extra logic 
While more efficient calling regular getattr (stat) leads to incorrect
information being returned and would not work correctly when using
symlinkify.
 2021-12-18 17:07:23 -05:00
Antonio SJ Musumeci
e256c88e4c Get attrs for link, not target 2021-12-16 19:11:29 -05:00
Antonio SJ Musumeci
ee8532c70d Ensure server handling of locks is disabled 2021-11-27 00:01:12 -05:00
Antonio SJ Musumeci
18dead4d86 Add new debug printing routines 2021-10-27 22:12:10 -04:00
Antonio SJ Musumeci
5f737cb7bf Add option to log node memory usage metrics 2021-10-11 11:57:16 -04:00
Antonio SJ Musumeci
313aa2a1d3 Update ghc::filesystem and nonstd::optional 2021-09-30 19:43:05 -04:00
Antonio SJ Musumeci
8def16a811 Fix infinite loop in mfs action policy 2021-08-29 12:33:52 -04:00
Antonio SJ Musumeci
43a6d66e3c Major cleanup of libfuse to remove unneeded features 
* Remove request interrupt code. Required tracking of all requests unnecesssarily.
* Remove all debugging printing. Have plans to do full replacement.
* Remove deprecated functions.
* Remove unneeded error checking.
* Remove "userdata" which was unused.
* Remove allow_root feature.
 2021-08-29 00:05:30 -04:00
trapexit
3b8d495c43
Merge pull request #940 from trapexit/ioctl-runtime-removal 
Remove config ioctl calls
 2021-08-03 17:02:01 -04:00
Antonio SJ Musumeci
61c2187c14 Remove config ioctl calls 
While they were going to be the new way of interacting with the runtime config
I've decided to use toml based config for mergerfs v3. In that case all updates
will happen through the config file and ioctl will only be used to load/reload
the config rather than a key:value like API.
 2021-08-02 16:56:20 -04:00
trapexit
192e4a4af8
Merge pull request #939 from trapexit/update-nonstd-optional 
Update nonstd::optional to v3.4.0
 2021-08-02 16:29:40 -04:00
Antonio SJ Musumeci
7013ff9aeb Update nonstd::optional to v3.4.0 2021-08-02 16:10:56 -04:00
Antonio SJ Musumeci
3ce0dc56e4 Update ghc::filesystem to v1.5.8 2021-08-02 16:06:43 -04:00
Antonio SJ Musumeci
098f353bf1 pfrd: fix mod by zero error when all branches are filtered 2021-06-25 09:17:16 -04:00
Antonio SJ Musumeci
8adebc9489 new features: follow-symlinks, rename-exdev, link-exdev 
* follow-symlinks: allows mergerfs to transparently follow symlinks
* link-exdev: in the event a link returns EXDEV create a symlink instead
* rename-exdev: in the event a rename returns EXDEV move the oldpath and
  create a symlink for the newpath
 2021-02-11 23:24:08 -05:00
Antonio SJ Musumeci
d337574665 fix parsing of relative branch paths 2021-02-11 19:22:17 -05:00
trapexit
cbea2ba91b
Merge branch 'master' into readdir_rewind_fix 2021-02-09 18:13:08 -05:00
Antonio SJ Musumeci
f6e37672b8 reset dentry buffer when rewind'ed 2021-02-09 18:09:28 -05:00
Antonio SJ Musumeci
538467b86d config: rework global config, remove rwlock, make branches RCU like 
Also added unit tests. Should have done separately but found a number of bugs.
 2021-01-14 21:02:10 -05:00
Antonio SJ Musumeci
f50812a9ce dirname should not return an empty string 2020-12-18 16:04:17 -05:00
Antonio SJ Musumeci
44a98910ba msp policies stopped before root path 2020-12-17 19:55:49 -05:00
Antonio SJ Musumeci
7edd3c6cf3 config: fix invalid error check when parsing config file 2020-11-28 20:26:18 -05:00
Antonio SJ Musumeci
1b26f4908e general cleanup, slight memory reduction 2020-11-19 22:58:52 -05:00
Antonio SJ Musumeci
c3fffefbc1 msp policies: used wrong path to check existance 2020-11-19 18:34:28 -05:00
Antonio SJ Musumeci
04844420dc wyhash: use safety mode 1 
mode 0 is not intended for unaligned buffers
 2020-09-24 08:45:34 -04:00
Antonio SJ Musumeci
5a3184359d rework some function error handling 
chmod, chown, removexattr, setxattr, truncate, utimens:
* if no errors: return 0
* if no successes: return first error
* if file acted on was the same as related search function: return its value
* return 0

rmdir, unlink:
* if no errors: return 0
* return first error
 2020-09-23 00:01:25 -04:00
Antonio SJ Musumeci
610c75fa90 properly return const ref from tofrom string wrapper 2020-09-19 18:57:13 -04:00
Antonio SJ Musumeci
767039c492 option_parser: return 0 when requesting help or version 2020-09-10 21:58:23 -04:00
Antonio SJ Musumeci
fc3453932a branches: add per branch minfreespace w/ original value as default 
example: /mnt/disk0=RW,1G:/mnt/disk1=RW,2G
 2020-09-04 21:04:45 -04:00
Antonio SJ Musumeci
6311df774d optionally use lchmod depending on if on Linux or not (BSD) 2020-08-26 12:44:32 -04:00
Antonio SJ Musumeci
046844083f add {,ep,msp}pfrd policies 
Percentage Free Random Distribution

Chooses a random branch based on the available space percentage free.

IE: if branch A has 1G free and branch B has 2G then B should be chosen
twice as often.
 2020-08-25 22:19:01 -04:00
Antonio SJ Musumeci
7fd629ebf3 add #warning to make it more obvious what versions of functions are used 2020-08-18 19:28:36 -04:00
Antonio SJ Musumeci
2fe20b888e clean up and separate out fs_* files 2020-08-18 19:28:36 -04:00
Antonio SJ Musumeci
2696079601 break fs.hpp up into separate files 2020-08-18 19:28:36 -04:00
Antonio SJ Musumeci
ec15872a1f cleanup function signatures and definitions 2020-08-18 19:28:36 -04:00
Antonio SJ Musumeci
6cc6524997 change category to enum class 2020-08-18 19:28:36 -04:00
Antonio SJ Musumeci
0371b047f9 change from fasthash64 to wyhash 
wyhash has very good performance for small keys (our usecase) and
works on platforms with alignment concerns.

A user had an issue where fasthash64 lead to misaligned reads and
signal exceptions. wyhash does not have such issues and is faster.
 2020-08-06 19:00:05 -04:00
Antonio SJ Musumeci
30d13b7f77 inodecalc: add 32bit versions of hashs 2020-08-03 19:42:06 -04:00
Antonio SJ Musumeci
5989d41a86 readdir: use getdents64 for compatibility with ARM64 2020-08-02 18:43:09 -04:00
Antonio SJ Musumeci
0709b2e447 add missing options to usage 2020-07-30 16:55:18 -04:00
Antonio SJ Musumeci
15a0aedd25 cleanup: move some config data structures to separate files 2020-07-29 21:42:43 -04:00
Antonio SJ Musumeci
a93bd9f7fa freebsd: misc cleanups to get freebsd compiling 2020-07-29 13:08:38 -04:00
Antonio SJ Musumeci
139e61efaa nfsopenhack: remove empty file check 2020-07-26 17:11:06 -04:00
Antonio SJ Musumeci
93218a343a NFS open/creat hack 
A hack to work around non-POSIX NFS <-> FUSE behavior where it turns
an open(O_EXCL|O_CREAT,0444) into multiple calls that results in
EACCES.
 2020-07-23 19:36:10 -04:00
Antonio SJ Musumeci
3ec137c4ad policy: add "most shared path" policies 
Like path preserving but walks back the path till a match is found. Should
cover the usecase where someone wants a "less strict" form of path
preservation.
 2020-07-21 16:31:08 -04:00
Andrea Gelmini
3a6738475a Fix typos 2020-07-20 13:01:33 +02:00
Andrea Gelmini
468d42088a Removed duplicated include 2020-07-20 12:51:44 +02:00
Antonio SJ Musumeci
08d267fc61 moveonenospc: enhance the feature to allow using a policy 
Just like functions you can now set a policy for moveonfreenospc. This
allows for more flexibility.

For backwards compatibility moveonfreenospc=true is converted to
moveonfreenospc=mfs.

minfreespace does apply which is slightly different from original behavior.
 2020-07-15 19:33:05 -04:00
Antonio SJ Musumeci
c0990644e8 remove unnecessary libfuse flags nopath, nullpath_ok, and utime_omit_ok 2020-07-08 19:25:23 -04:00
Antonio SJ Musumeci
b4397f7f2e add 'inodecalc' option to allow selection of inode calculation algo 2020-06-28 20:02:07 -04:00
Antonio SJ Musumeci
dbdd3e22fc additional readdir refactor cleanup 
Differences between readdir and getdents is minimal at best. Leaving code
for now to allow for possible expansion later.
 2020-06-27 23:17:21 -04:00
Antonio SJ Musumeci
54c41c4994 rework config management 
* Add standard way to do str2val and val2str conversion
* Add support for a config file
* Add support for reading and writing settings via ioctl
 2020-06-18 18:54:25 -04:00
Antonio SJ Musumeci
698c4147fc fix getdent name length calculation 2020-03-12 23:45:20 -04:00
Antonio SJ Musumeci
3a46ec9fab fix filename hashing error 2020-03-07 12:09:40 -05:00
Antonio SJ Musumeci
9b2634a1e5 fix name length calculation for musc 2020-02-29 17:15:29 -05:00
Antonio SJ Musumeci
62873d2d3b use getdents64 on linux 2020-02-28 20:34:12 -05:00
Antonio SJ Musumeci
d119807adb restructure readdir, add readdir_plus 
Does not enable READDIR_AUTO. Might add in the future.
 2020-02-22 19:45:52 -05:00
Antonio SJ Musumeci
a646fe016d change inode conversion algo to reduce collision 2020-02-12 21:05:52 -05:00
Antonio SJ Musumeci
576ff3694e add cache.writeback to xattrs 2020-02-12 19:48:37 -05:00
Antonio SJ Musumeci
903d39f968 add writeback caching 2020-02-09 21:08:53 -05:00
Antonio SJ Musumeci
4d82ed9324 fix short writes on >2GB files when cloning file 2019-10-14 23:32:51 -04:00
Antonio SJ Musumeci
0fffabfbe3 only return 1 branch for rand/eprand policies 2019-09-07 17:25:06 -04:00
Antonio SJ Musumeci
50ad648ce4 initialize mutex to fix lockup 2019-09-07 14:10:36 -04:00
Antonio SJ Musumeci
4c4c27a93e set uid & gid when calling ioctl 
Turns out certain ioctl calls check the effective user id (FS_IOC_SETFLAGS).
 2019-06-17 13:37:15 -04:00
Antonio SJ Musumeci
5ca928e436 accept old arguments for backwards compatibility 2019-06-10 20:20:41 -04:00
Antonio SJ Musumeci
7cbd88ac81 allow setting of 'max_pages' (via 'fuse_msg_size') 
Linux 4.20 and above allow setting the number of pages per FUSE message
upto 256 (4K * 256 = 1MiB). This can greatly increase read and write
speeds depending on the workload.
 2019-06-03 21:01:19 -04:00
Antonio SJ Musumeci
8cb7195c3e add copy_file_range support 2019-06-03 07:12:05 -04:00
Antonio SJ Musumeci
529a953e30 add file caching across opens and runtime control 2019-05-30 00:10:08 -04:00
Antonio SJ Musumeci
ddf6a2f105 make async_read optional again 2019-05-28 15:57:50 -04:00
Antonio SJ Musumeci
2323c16316 add readdir caching 2019-05-23 22:11:33 -04:00
Antonio SJ Musumeci
1baa706d37 add symlink caching 2019-05-22 19:25:38 -04:00
Antonio SJ Musumeci
61cded5b34 fix for unlink race condition 2019-05-21 22:57:02 -04:00
Antonio SJ Musumeci
5f2221155c ensure parallel dirops is enabled if capable 2019-05-20 00:36:42 -04:00
Antonio SJ Musumeci
2b019b84b0 ensure async_aio is enabled if capable 2019-05-20 00:18:56 -04:00
trapexit
af1c0d78e8
Merge pull request #621 from trapexit/posix-acl 
add support for POSIX ACLs
 2019-05-19 18:26:49 -04:00
Antonio SJ Musumeci
80d56ac94e add support for POSIX ACLs 2019-05-19 18:17:55 -04:00
Antonio SJ Musumeci
1ca70521a4 remove 'remote' flock support 2019-05-19 17:34:56 -04:00
Antonio SJ Musumeci
e052446713 fix setting of fsname 2019-05-12 14:07:21 -04:00
Antonio SJ Musumeci
5883020784 tweak docs 2019-05-10 11:54:41 -04:00
Antonio SJ Musumeci
6ecc618d83 remove .fuse_hidden file creation 
addresses #521

mergerfs is written using libfuse's high level API. The HLAPI forces a certain
behavior with regard to unlinking or rename overwriting of open files. It
renames the file to .fuse_hiddenXXXXXX and when incoming requests refer
to the unlinked file it will use path based calls and use that path.

This behavior leads to rmdirs failing or some software noticing
the .fuse_hidden files.

This patch changes the HLAPI and removes entirely the .fuse_hidden behavior.

See the README for more.
 2019-04-13 21:41:51 -04:00
Antonio SJ Musumeci
2a075ead92 fall back to other file copy methods in clonefile 2019-04-13 15:08:13 -04:00
Antonio SJ Musumeci
b69819e4b2 ioctl: don't set outbufsz when not needed 2019-03-20 10:18:55 -04:00
Antonio SJ Musumeci
0918dfd117 make attr, entry, negative_entry cache timeouts runtime configurable 2019-02-23 01:03:20 -05:00