github-mirror/mergerfs
2
0
Fork 0
mirror of https://github.com/trapexit/mergerfs.git synced 2024-11-26 10:13:50 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
1,033 Commits 30 Branches 95 Tags 26 MiB
77e6f400b5
Commit Graph

318 Commits

Author SHA1 Message Date
Antonio SJ Musumeci
c46134c03a fix building on platforms without O_PATH 2018-11-06 07:59:28 -05:00
Antonio SJ Musumeci
3631ab7df2 optimize readdir file dedup 
Use fasthash64 to hash filenames to uint64_t and store in khash
set. Significantly reduces malloc/free'ing and memory usage.
 2018-11-06 00:05:30 -05:00
Antonio SJ Musumeci
680f8194f9 add ability to change statfs behavior 2018-11-05 21:33:27 -05:00
Antonio SJ Musumeci
7524e57262 rename NW (no write) to NC (no create) 2018-11-01 23:37:45 -04:00
Antonio SJ Musumeci
b55ebba4ed add tagging branches RW/RO/NW 
This allows users to tag a branch as readonly or not for writing regardless
of how the filesystem is mounted. Should simplify deployments and offer
more flexibility.
 2018-11-01 23:13:49 -04:00
Antonio SJ Musumeci
9e0ab1f518 misc cleanups 2018-10-24 22:38:36 -04:00
Antonio SJ Musumeci
8a48b74cb4 policy return cleanup 2018-10-15 19:03:02 -04:00
Antonio SJ Musumeci
85026d5780 add FICLONE and copy_file_range to clonefile 
If available FICLONE and copy_file_range will be tried in addition to sendfile
when copying data between two files. The fallback is a tradition read/write
loop. On systems that support these it should improve performance.
 2018-10-12 10:18:49 -04:00
Antonio SJ Musumeci
1885a82cce remove libattr dependency 2018-10-09 13:50:45 -04:00
Antonio SJ Musumeci
65f482e483 add ability to turn on/off xattr support at runtime 2018-10-09 08:41:31 -04:00
Antonio SJ Musumeci
8d1a1564bf fix building on certain platforms 2018-10-09 08:23:37 -04:00
Antonio SJ Musumeci
93f7d7d927 add link_cow feature 
When enabled if a regular file is opened which has a link count > 1 it will
copy the file to a temporary file and rename over the original. Effectively
breaking the link. This behavior is similar to cow-shell and other LD_PRELOAD
based "CoW" solutions.
 2018-10-06 22:05:05 -04:00
Antonio SJ Musumeci
9afefef032 keep literal when glob fails 2018-10-01 16:56:40 -04:00
Antonio SJ Musumeci
f8563369bc add security_capability option 2018-09-30 23:30:53 -04:00
Andrey Mazo
ec6adaa376
options: move "-o threads=" help text to libfuse/ 
Parsing of and acting upon "-o threads" option is done in the bundled `libfuse/`,
so move the corresponding help text there too.
This make help text consistent with behavior
when building against system libfuse,
which doesn't support this option.
 2018-08-19 21:04:35 +03:00
Antonio SJ Musumeci
9830e29945 fix version generation 2018-03-09 21:06:00 -05:00
Antonio SJ Musumeci
cfe9c28a10 ignore clonepath metadata errors. fixes #470 2018-03-08 16:55:35 -05:00
Antonio SJ Musumeci
f48c16d162 stop clonepath at base directory. fixes #467 2018-03-07 18:46:32 -05:00
Antonio SJ Musumeci
1d6d22727c call 32bit versions of set/geteuid on 32bit platforms 2017-10-17 22:21:29 -04:00
Antonio SJ Musumeci
75ed37a11a add setting of thread pool size 2017-07-02 23:50:39 -04:00
Antonio SJ Musumeci
0708110ec1 improve khash performance 2017-07-01 11:53:35 -04:00
Antonio SJ Musumeci
a7b126cb48 use pragma once in headers 2017-06-30 11:38:34 -04:00
Antonio SJ Musumeci
a2bddec8c7 add 'ignore path preserving on rename' feature 2017-06-30 00:01:00 -04:00
Antonio SJ Musumeci
aea2b40dda use temp files (then rename) when moving files for moveonenospc 2017-06-28 21:17:47 -04:00
Antonio SJ Musumeci
bfd410def2 use fusepath from fileinfo. closes #417 2017-06-24 17:03:41 -04:00
Antonio SJ Musumeci
7296d3d90d add time.h to fix compiling on some platforms 2017-06-15 19:35:28 -04:00
Antonio SJ Musumeci
de0985c135 add libfuse 2.9.7 to repo and build against libfuse.a 2017-06-02 11:25:25 -04:00
Antonio SJ Musumeci
87c2f2f9dc add nullrw feature to facilitate benchmarking 2017-05-26 18:11:18 -04:00
Antonio SJ Musumeci
6a7675f2bf symlinkify: file -> symlink-to-original-file after timeout 2017-05-05 00:14:04 -04:00
Antonio SJ Musumeci
8ba3a08f40 make dropcacheonclose runtime configurable 2017-04-21 16:58:39 -04:00
Antonio SJ Musumeci
ccaa458739 better handle incomplete reads/writes in copying files 2017-04-13 12:55:34 -04:00
Antonio SJ Musumeci
162b99e6b8 enable nopath and nullpath_ok 2017-04-12 13:52:06 -04:00
Antonio SJ Musumeci
f15437c713 tweak movefile behavior 2017-04-11 15:24:09 -04:00
Antonio SJ Musumeci
2fbeb67dd9 hide fs::fadvise as it's not used directly 2017-04-11 09:42:33 -04:00
Antonio SJ Musumeci
8b976ab3c1 support older libfuse without utime_omit_ok flag 2017-04-11 09:23:40 -04:00
Antonio SJ Musumeci
1a1fa06aa3 fadvise cleanup 2017-04-11 09:02:05 -04:00
Antonio SJ Musumeci
617195d71e enable utime_omit_ok flag 2017-04-10 21:36:56 -04:00
Antonio SJ Musumeci
be3eb7e3ed work around getgrouplist signature difference on osx 2017-04-07 16:23:18 -04:00
Antonio SJ Musumeci
0600734942 handle 32bit and 64bit inode recalculation 2017-04-07 09:47:19 -04:00
Antonio SJ Musumeci
9d0798dbfb restructure fadvise 2017-04-07 00:12:57 -04:00
Antonio SJ Musumeci
e2acffe55b restructure fallocate abstraction 2017-04-06 23:35:31 -04:00
Antonio SJ Musumeci
42d454ac27 abstract futimesat 2017-04-06 23:13:02 -04:00
Antonio SJ Musumeci
0b2bf17cd7 abstract access to highres atime/mtime 2017-04-06 22:16:17 -04:00
Antonio SJ Musumeci
e20d566f96 use correct integer types 2017-04-06 21:29:39 -04:00
Antonio SJ Musumeci
c043ef95a3 make fs::attr return ENOTSUP on EINVAL #381 2017-04-03 21:11:20 -04:00
Antonio SJ Musumeci
6aa62d03ff add option to drop file caches before closing files 2017-02-18 16:06:29 -05:00
Antonio SJ Musumeci
492d895632 check metadata on chown/chmod errors when cloning 2017-02-15 09:08:27 -05:00
Antonio SJ Musumeci
9cc9bb9de2 misc document updates 2017-01-31 19:10:54 -05:00
Antonio SJ Musumeci
a60d815e92 add ifndefs to all headers 2017-01-17 11:45:44 -05:00
Antonio SJ Musumeci
e93c946198 limit need to explicitly call .c_str() 2017-01-17 11:45:44 -05:00
Antonio SJ Musumeci
7b4e1ea36d remove clone command 2017-01-17 11:45:44 -05:00
Antonio SJ Musumeci
726b88e039 restructure error calculation 2017-01-17 11:45:43 -05:00
Antonio SJ Musumeci
d67d5dec77 check for system.posix_acl_default before setting umask 2017-01-16 12:19:58 -05:00
Antonio SJ Musumeci
1aa76a54c2 use different read and write functions when using direct_io 2016-12-20 17:06:34 -05:00
Antonio SJ Musumeci
67b48fcb3a compute inode in readdir 2016-12-16 10:57:24 -05:00
Antonio SJ Musumeci
c8fa51c81a support setting of inodes (using use_ino option) 
creates a 64bit inode value from the underlying device value + original inode

final_ino = orig_ino | (dev << 32)

not perfect but given few filesystems use 64bit inodes nor is st_dev more than 16bit usually it should be fine
 2016-12-14 18:53:21 -05:00
Antonio SJ Musumeci
822204ff34 replace std::set with klib's khash to increase readdir performance 2016-12-14 17:24:48 -05:00
Antonio SJ Musumeci
35075bb0cd return clonepath errors 
currently the error is ignored and it was expected the primary call would
fail. problem is it returns confusing errors as a result. (eg ENOENT vs EPERM)
 2016-12-06 16:48:52 -05:00
Antonio SJ Musumeci
3c5351af93 ignore filesystems which return zeros for statfs. closes #335 2016-11-15 23:27:17 -05:00
Antonio SJ Musumeci
157dae0d2f define O_LARGEFILE and O_NOATIME if needed 2016-10-28 22:04:15 -04:00
Antonio SJ Musumeci
00c814dc58 consolidate and simplify utime 2016-10-28 22:04:09 -04:00
Antonio SJ Musumeci
6d6fb45a3b check if fdatasync is available and return ENOSYS if not 2016-10-27 16:59:04 -04:00
Antonio SJ Musumeci
d0b6cd1f38 further abstraction of system calls 2016-10-20 16:51:49 -04:00
Antonio SJ Musumeci
1dc7bff6e6 wrap most posix filesystem functions 2016-10-19 16:58:52 -04:00
Antonio SJ Musumeci
8f594e17d7 add flock 2016-10-19 11:58:44 -04:00
Antonio SJ Musumeci
3fb7f8919a add EDQUOT to errors which trigger moveonenospc 2016-10-06 13:54:57 -04:00
Antonio SJ Musumeci
d9a79062b8 use SYS_setgroup32 syscall if available. closes #319 2016-09-19 17:04:11 -04:00
Antonio SJ Musumeci
7e423cd9ce small tweaks to build on Debian kFreeBSD 2016-09-14 08:36:06 -04:00
Antonio SJ Musumeci
0395e7c776 fix futimes version of utimes wrapper 2016-09-13 21:30:45 -04:00
Antonio SJ Musumeci
93923177ee fix #define typo 2016-09-13 17:40:23 -04:00
Antonio SJ Musumeci
1513c92cbc abstract posix_fadvise 2016-09-13 17:36:28 -04:00
Antonio SJ Musumeci
1a698e5ef5 rename include cpp files to have icpp extension 2016-08-26 14:41:56 -04:00
Antonio SJ Musumeci
2ee6b4f581 include sys/types.h to pick up ssize_t 2016-08-26 13:09:15 -04:00
Antonio SJ Musumeci
709dda5069 support systems without ENODATA 2016-08-26 12:59:32 -04:00
Antonio SJ Musumeci
49474f01ca make futimes crossplatform 2016-08-07 14:42:54 -04:00
Antonio SJ Musumeci
f7547d7cf5 Merge pull request #299 from trapexit/realpath 
use dynamic buffer for realpath
 2016-08-04 17:14:09 -04:00
Antonio SJ Musumeci
bb02ab2cba Merge pull request #298 from trapexit/fssendfile 
split sendfile wrapper into separate files
 2016-08-04 17:13:28 -04:00
Antonio SJ Musumeci
34d38cb0fb split sendfile wrapper into separate files 2016-08-04 17:11:23 -04:00
Antonio SJ Musumeci
192a9d54d1 make fs_attr compile on unsupported platforms 2016-08-04 17:10:02 -04:00
Antonio SJ Musumeci
40574bd138 use dynamic buffer for realpath 2016-08-04 14:52:27 -04:00
Antonio SJ Musumeci
45f757dc60 add osx version of fallocate 2016-08-02 17:14:45 -04:00
Antonio SJ Musumeci
0fceb8e0a3 add epall and eprand policies 2016-08-01 16:08:02 -04:00
Antonio SJ Musumeci
7634eb1440 replace nonstandard eaccess with POSIX.1-2008 faccessat 2016-07-29 12:39:44 -04:00
Antonio SJ Musumeci
a93ab6c2f5 add existing path first found policy. closes #289 2016-07-11 20:42:56 -04:00
Antonio SJ Musumeci
43cbd9c670 move size calculations to use uint64_t. fixes #287 2016-07-11 09:04:47 -04:00
Antonio SJ Musumeci
cb35a374ac rework fallocate logic 2016-05-11 13:19:44 -04:00
Antonio SJ Musumeci
23b8e4582e fix ioctl on directories 2016-05-11 11:22:00 -04:00
Antonio SJ Musumeci
be6341e6f0 create eplus (existing path, least used space) policy. closes #273 2016-05-07 15:15:51 -04:00
Antonio SJ Musumeci
f7d3e8bf47 create lus (least used space) policy. closes #273 2016-05-07 14:59:42 -04:00
Antonio SJ Musumeci
070ed08caa properly check errors of xattr. closes #255 2016-03-10 18:37:09 -05:00
Antonio SJ Musumeci
2061211894 fix rename failing on non-path preserving policies 2016-03-06 02:21:23 -05:00
Antonio SJ Musumeci
12cf57dc5a re-add minfreespace check to epmfs policy 2016-03-04 13:52:31 -05:00
Antonio SJ Musumeci
4ecf3c5797 clearly separate usage of statvfs from stat for file existance 
Used statvfs to also check for existance vs lstat. On dead symlinks
this resulted in ENOENT making certain functions fail.

closes #239
 2016-02-29 23:04:59 -05:00
Antonio SJ Musumeci
779143f4b6 add minfreespace checks to policy ff's create and remove fwfs 
fwfs is now covered by ff with the minfreespace and readonly checks
 2016-02-29 18:06:59 -05:00
Antonio SJ Musumeci
14886a27f4 add readonly and minfreespace filters to all policy for creates. closes #236 2016-02-29 10:15:53 -05:00
Antonio SJ Musumeci
9819cf6133 fix clonepath being called on wrong source 2016-02-26 17:16:09 -05:00
Antonio SJ Musumeci
e593927095 normalize error handling in rename policy 2016-02-24 17:10:56 -05:00
Antonio SJ Musumeci
7c85cd906a ff policy tweaks 2016-02-24 12:40:02 -05:00
Antonio SJ Musumeci
5cf3bb7c98 override standard libfuse version flag 2016-02-24 06:36:33 -05:00
Antonio SJ Musumeci
25a039942c minor tweaks to filesystem utility functions 2016-02-23 12:28:20 -05:00
Antonio SJ Musumeci
792c9b9808 use stat/2 rather than statvfs/2 to find file drive 2016-02-23 12:06:16 -05:00
Antonio SJ Musumeci
d4ec341c4f remove unnecessary policies 2016-02-21 18:24:15 -05:00
Antonio SJ Musumeci
5813d1e477 ignore drives mounted as readonly from create policies. closes #224 2016-02-21 14:34:40 -05:00
Antonio SJ Musumeci
f3e75a0544 use stat.st_dev to uniquely identify mounts for statfs. closes #213 2016-02-15 22:55:47 -05:00
Antonio SJ Musumeci
b3248a8f78 simplify policies 2016-02-15 02:06:58 -05:00
Antonio SJ Musumeci
7bf1ca45e4 add existing path, least free space policy. closes #216 2016-02-12 10:06:41 -05:00
Antonio SJ Musumeci
6086620e23 use references to srcmounts rather than copies 2016-02-11 21:09:31 -05:00
Antonio SJ Musumeci
f779f82ed6 fix statvfs drive dedup. closes #209 2016-01-22 10:15:24 -05:00
Antonio SJ Musumeci
ea325755a7 make symlink function like mknod/mkdir. closes #204 2016-01-20 17:55:20 -05:00
Antonio SJ Musumeci
1d1694bbb5 fix indexing of mknod targets. closes #202 2016-01-20 17:24:50 -05:00
Antonio SJ Musumeci
25265f4e85 dedup based on full statvfs struct rather than fsid. closes #183 2016-01-20 16:44:05 -05:00
Antonio SJ Musumeci
62f8fc524e have link act similar to rename 
closes #193
 2016-01-19 16:08:02 -05:00
Antonio SJ Musumeci
242af777be move from MIT to ISC license. closes #194 2016-01-14 16:56:38 -05:00
Antonio SJ Musumeci
4c77ac4d24 all action functions return success should at least one succeed. closes #189 2016-01-14 07:58:57 -05:00
Antonio SJ Musumeci
a3e6a0352d rework rename algo to minimize likelihood of EXDEV being returned. closes #187 2016-01-12 16:23:12 -05:00
Antonio SJ Musumeci
eb6d1a1bc2 change to using template for policy class 2015-12-29 16:14:37 -05:00
Antonio SJ Musumeci
c731b7035c fix building without xattr 2015-12-14 18:48:04 -05:00
Antonio SJ Musumeci
9e24796429 add mergerfs pid to xattrs 2015-12-07 21:20:12 -05:00
Antonio SJ Musumeci
5e880bddba use SYS_setgroups rather than setgroups 2015-10-29 23:20:37 -04:00
Antonio SJ Musumeci
93cedabe9c fix misc issues flagged by clang scan-build 2015-10-22 18:06:11 -04:00
Antonio SJ Musumeci
1c7de2d83e fix minor integer casting issues 2015-10-13 23:33:33 -04:00
Antonio SJ Musumeci
58446f9124 misc fixes to compile on older platforms 2015-10-12 19:51:57 -04:00
Antonio SJ Musumeci
40f569be37 rewrite gid cache system 
Uses fixed storage so as to not require C++11 local thread storage or memory allocation.
 2015-10-11 15:01:57 -04:00
Antonio SJ Musumeci
53e3284b26 remove usage of UINT32_MAX macro 2015-10-01 10:26:12 -04:00
Antonio SJ Musumeci
09ffc8c903 provide usage text and version info. closes #146 2015-09-29 07:29:10 -04:00
Antonio SJ Musumeci
5808ab7795 move on enospc when writing feature. closes #141 
This feature mimics the standard mhddfs behavior but is more thorough.
If a write fails and the errno is set to ENOSPC then mergerfs will (if
the feature is enabled) attempt to move the file to the drive with
the most free space but only if it has enough room for the file plus
the amount to be written. If that transfer is successful it will then
unlink the original file and attempt the previously failed write again.

The copy includes copying the path and file including the acls, owners,
attributes, extended attributes, and timestamps.
 2015-09-25 15:14:17 -04:00
Antonio SJ Musumeci
a960a7ef46 cleanup controlfile manipulation 2015-09-14 22:40:49 -04:00
Antonio SJ Musumeci
e0cf97261c include default_permissions in default arguments 
closes #130
 2015-09-14 22:12:24 -04:00
Antonio SJ Musumeci
3163258a33 make changing credentials opportunistic + per thread setgroups cache 
closes #129,#131
 2015-09-14 19:00:07 -04:00
Antonio SJ Musumeci
ce9352987c realpath'ize all source mount points. closes #117 2015-09-06 18:24:36 -04:00
Antonio SJ Musumeci
2d899479c3 fix non-suffixed setxattr of user.mergerfs.minfreespace 2015-09-05 23:08:11 -04:00
Antonio SJ Musumeci
b22528bac4 add user.mergerfs.version xattr 2015-09-05 21:48:15 -04:00
Antonio SJ Musumeci
e377d54c93 add user.mergerfs.policies xattr 2015-09-05 19:49:08 -04:00
Antonio SJ Musumeci
8178bf518c refactor and simplify getxattr for user.mergerfs.\* 2015-09-03 16:56:54 -04:00
Antonio SJ Musumeci
bc77b0fd2a add minfreespace check to epmfs create policy 2015-09-01 22:28:35 -04:00
Antonio SJ Musumeci
1f1e481075 rework rename 
Return EXDEV if directories of tragets differ. If the old and new path exist in the same directory first rename each old found by the policy and then unlink/rmdir any new on a drive which didn't rename. The unlink/rmdir will occur only if there were no rename errors. Any failures of unlink/rmdir are ignored. The last rename error is returned.
 2015-09-01 18:32:31 -04:00
Antonio SJ Musumeci
7a93198d4a forgot to add einval to policy 2015-08-30 17:08:24 -04:00
Antonio SJ Musumeci
bbc75f6ff1 create errno policies for simulating errors. closes #107 2015-08-27 22:38:23 -04:00
Antonio SJ Musumeci
126df0fbf1 fix epmfs failing to pick the existing path. closes #102 2015-08-24 15:51:17 -04:00
Antonio SJ Musumeci
8767db971a remove unused variable 2015-08-05 11:59:39 -04:00
Antonio SJ Musumeci
838c081b8c Merge pull request #94 from trapexit/config-cleanup 
config get and struct naming cleanup
 2015-08-05 11:51:33 -04:00
Antonio SJ Musumeci
267f2d291c move requesting of FUSE flags to init from cli args 2015-08-05 11:50:21 -04:00
Antonio SJ Musumeci
f130d07fd8 config get and struct naming cleanup 2015-08-05 09:28:53 -04:00
Antonio SJ Musumeci
52d80299b5 passthrough ioctl args without processing. closes #90 2015-08-05 09:18:34 -04:00
Antonio SJ Musumeci
c60d03831f use gte rather than gt for mtime comparisons. fixes #91 2015-08-04 14:03:44 -04:00
Antonio SJ Musumeci
80b2c35175 add creation of full path for open 2015-07-16 16:57:01 -04:00
Antonio SJ Musumeci
983fa9100d change fuse functions to use the fuse namespace 2015-07-15 10:53:13 -04:00
Antonio SJ Musumeci
efadd2c8d8 Merge pull request #86 from trapexit/readdir-cleanup 
remove unused readdir function
 2015-07-15 10:51:43 -04:00
Antonio SJ Musumeci
e5359ebd05 remove unused readdir function 2015-07-15 10:50:25 -04:00
Antonio SJ Musumeci
f00cd1467e use pthread_getugid_np instead of gete{u,g}id on OSX. fixes #84 2015-07-15 10:39:47 -04:00