github-mirror/mergerfs
2
0
Fork 0
mirror of https://github.com/trapexit/mergerfs.git synced 2024-11-25 17:57:41 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

remove defaults, hard code atomic_o_trunc, big_writes, and default_permissions

Browse Source 
`defaults` is a value used by all filesystems and isn't passed through to
mergerfs when mounting via the fstab or the mount command. This led
to inconsistent application of options. atomic_o_trunc, big_writes, and
default_permissions should be enabled all the time anyway and splice_*
can lead to issues so they are not always enabled.
This commit is contained in:
Antonio SJ Musumeci 2019-01-30 21:58:48 -05:00
parent 21d690a210
commit c5b2415daf
5 changed files with 35 additions and 58 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
README.md
View File

@ -63,12 +63,11 @@ mergerfs does **not** support the copy-on-write (CoW) behavior found in **aufs**
### mount options
* **defaults**: a shortcut for FUSE's **atomic_o_trunc**, **auto_cache**, **big_writes**, **default_permissions**, **splice_move**, **splice_read**, and **splice_write**. These options seem to provide the best performance.
* **allow_other**: a libfuse option which allows users besides the one which ran mergerfs to see the filesystem. This is required for most use-cases.
* **direct_io**: causes FUSE to bypass caching which can increase write speeds at the detriment of reads. Note that not enabling `direct_io` will cause double caching of files and therefore less memory for caching generally (enable **dropcacheonclose** to help with this problem). However, `mmap` does not work when `direct_io` is enabled.
* **minfreespace=value**: the minimum space value used for creation policies. Understands 'K', 'M', and 'G' to represent kilobyte, megabyte, and gigabyte respectively. (default: 4G)
* **moveonenospc=true|false**: when enabled (set to **true**) if a **write** fails with **ENOSPC** or **EDQUOT** a scan of all drives will be done looking for the drive with the most free space which is at least the size of the file plus the amount which failed to write. An attempt to move the file to that drive will occur (keeping all metadata possible) and if successful the original is unlinked and the write retried. (default: false)
* **use_ino**: causes mergerfs to supply file/directory inodes rather than libfuse. While not a default it is generally recommended it be enabled so that hard linked files share the same inode value.
* **use_ino**: causes mergerfs to supply file/directory inodes rather than libfuse. While not a default it is recommended it be enabled so that linked files share the same inode value.
* **hard_remove**: force libfuse to immedately remove files when unlinked. This will keep the `.fuse_hidden` files from showing up but if software uses an opened but unlinked file in certain ways it could result in errors.
* **dropcacheonclose=true|false**: when a file is requested to be closed call `posix_fadvise` on it first to instruct the kernel that we no longer need the data and it can drop its cache. Recommended when **direct_io** is not enabled to limit double caching. (default: false)
* **symlinkify=true|false**: when enabled (set to **true**) and a file is not writable and its mtime or ctime is older than **symlinkify_timeout** files will be reported as symlinks to the original files. Please read more below before using. (default: false)
@ -99,7 +98,7 @@ To make it easier to include multiple branches mergerfs supports [globbing](http
Each branch can have a suffix of `=RW` (read / write), `=RO` (read-only), or `=NC` (no create). These suffixes work with globs as well and will apply to each path found. `RW` is the default behavior and those paths will be eligible for all policy categories. `RO` will exclude those paths from `create` and `action` policies (just as a filesystem being mounted `ro` would). `NC` will exclude those paths from `create` policies (you can't create but you can change / delete).
```
$ mergerfs -o defaults,allow_other,use_ino /mnt/disk\*:/mnt/cdrom /media/drives
# mergerfs -o allow_other,use_ino /mnt/disk\*:/mnt/cdrom /media/drives
```
The above line will use all mount points in /mnt prefixed with **disk** and the **cdrom**.
@ -107,8 +106,8 @@ The above line will use all mount points in /mnt prefixed with **disk** and the
To have the pool mounted at boot or otherwise accessable from related tools use **/etc/fstab**.
```
# <file system> <mount point> <type> <options> <dump> <pass>
/mnt/disk*:/mnt/cdrom /media/drives fuse.mergerfs defaults,allow_other,use_ino 0 0
# <file system> <mount point> <type> <options> <dump> <pass>
/mnt/disk*:/mnt/cdrom /media/drives fuse.mergerfs allow_other,use_ino 0 0
```
**NOTE:** the globbing is done at mount or xattr update time (see below). If a new directory is added matching the glob after the fact it will not be automatically included.
@ -588,8 +587,8 @@ done
# TIPS / NOTES
* The recommended base options are **defaults,allow_other,direct_io,use_ino**. (**use_ino** will only work when used with mergerfs 2.18.0 and above.)
* Run mergerfs as `root` unless you're merging paths which are owned by the same user otherwise strange permission issues may arise.
* **use_ino** will only work when used with mergerfs 2.18.0 and above.
* Run mergerfs as `root` (with **allow_other**) unless you're merging paths which are owned by the same user otherwise strange permission issues may arise.
* https://github.com/trapexit/backup-and-recovery-howtos : A set of guides / howtos on creating a data storage system, backing it up, maintaining it, and recovering from failure.
* If you don't see some directories and files you expect in a merged point or policies seem to skip drives be sure the user has permission to all the underlying directories. Use `mergerfs.fsck` to audit the drive for out of sync permissions.
* Do **not** use `direct_io` if you expect applications (such as rtorrent) to [mmap](http://linux.die.net/man/2/mmap) files. It is not currently supported in FUSE w/ `direct_io` enabled. Enabling `dropcacheonclose` is recommended when `direct_io` is disabled.
@ -758,7 +757,7 @@ https://lkml.org/lkml/2016/9/14/527
[25192.603193] [<ffffffff810a0730>] ? kthread_create_on_node+0x1e0/0x1e0
```
There is a bug in the kernel. A work around appears to be turning off `splice`. Add `no_splice_write,no_splice_move,no_splice_read` to mergerfs' options. Should be placed after `defaults` if it is used since it will turn them on. This however is not guaranteed to work.
There is a bug in the kernel. A work around appears to be turning off `splice`. Don't add the `splice_*` arguments or add `no_splice_write,no_splice_move,no_splice_read`. This, however, is not guaranteed to work.
#### rm: fts_read failed: No such file or directory
@ -830,7 +829,7 @@ Below is an example of mhddfs and mergerfs setup to work similarly.
`mhddfs -o mlimit=4G,allow_other /mnt/drive1,/mnt/drive2 /mnt/pool`
`mergerfs -o minfreespace=4G,defaults,allow_other,category.create=ff /mnt/drive1:/mnt/drive2 /mnt/pool`
`mergerfs -o minfreespace=4G,allow_other,category.create=ff /mnt/drive1:/mnt/drive2 /mnt/pool`
#### Why use mergerfs over aufs?
@ -935,9 +934,9 @@ For non-Linux systems mergerfs uses a read-write lock and changes credentials on
# PERFORMANCE TWEAKING
* try adding (or removing) `direct_io`
* try adding (or removing) `auto_cache` / `noauto_cache` (included in `defaults`)
* try adding (or removing) `kernel_cache` (don't use the underlying filesystems directly if enabling `kernel_cache`)
* try adding (or removing) `splice_move`, `splice_read`, and `splice_write` (all three included in `defaults`)
* try adding (or removing) `auto_cache`
* try adding (or removing) `kernel_cache`
* try adding (or removing) `splice_move`, `splice_read`, and `splice_write`
* try increasing cache timeouts `attr_timeout`, `entry_timeout`, `ac_attr_timeout`, `negative_timeout`
* try changing the number of worker threads
* try disabling `security_capability` or `xattr`

44
man/mergerfs.1
View File

@ -78,11 +78,6 @@ so you can mix rw and ro drives.
.SH OPTIONS
.SS mount options
.IP \[bu] 2
\f[B]defaults\f[]: a shortcut for FUSE\[aq]s \f[B]atomic_o_trunc\f[],
\f[B]auto_cache\f[], \f[B]big_writes\f[], \f[B]default_permissions\f[],
\f[B]splice_move\f[], \f[B]splice_read\f[], and \f[B]splice_write\f[].
These options seem to provide the best performance.
.IP \[bu] 2
\f[B]allow_other\f[]: a libfuse option which allows users besides the
one which ran mergerfs to see the filesystem.
This is required for most use\-cases.
@ -112,8 +107,8 @@ write retried.
.IP \[bu] 2
\f[B]use_ino\f[]: causes mergerfs to supply file/directory inodes rather
than libfuse.
While not a default it is generally recommended it be enabled so that
hard linked files share the same inode value.
While not a default it is recommended it be enabled so that linked files
share the same inode value.
.IP \[bu] 2
\f[B]hard_remove\f[]: force libfuse to immedately remove files when
unlinked.
@ -252,7 +247,7 @@ can\[aq]t create but you can change / delete).
.IP
.nf
\f[C]
$\ mergerfs\ \-o\ defaults,allow_other,use_ino\ /mnt/disk\\*:/mnt/cdrom\ /media/drives
#\ mergerfs\ \-o\ allow_other,use_ino\ /mnt/disk\\*:/mnt/cdrom\ /media/drives
\f[]
.fi
.PP
@ -264,8 +259,8 @@ tools use \f[B]/etc/fstab\f[].
.IP
.nf
\f[C]
#\ <file\ system>\ \ \ \ \ \ \ \ <mount\ point>\ \ <type>\ \ \ \ \ \ \ \ \ <options>\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ <dump>\ \ <pass>
/mnt/disk*:/mnt/cdrom\ \ /media/drives\ \ fuse.mergerfs\ \ defaults,allow_other,use_ino\ \ \ 0\ \ \ \ \ \ \ 0
#\ <file\ system>\ \ \ \ \ \ \ \ <mount\ point>\ \ <type>\ \ \ \ \ \ \ \ \ <options>\ \ \ \ \ \ \ \ \ \ \ \ \ <dump>\ \ <pass>
/mnt/disk*:/mnt/cdrom\ \ /media/drives\ \ fuse.mergerfs\ \ allow_other,use_ino\ \ \ 0\ \ \ \ \ \ \ 0
\f[]
.fi
.PP
@ -1203,13 +1198,12 @@ done
.fi
.SH TIPS / NOTES
.IP \[bu] 2
The recommended base options are
\f[B]defaults,allow_other,direct_io,use_ino\f[].
(\f[B]use_ino\f[] will only work when used with mergerfs 2.18.0 and
above.)
\f[B]use_ino\f[] will only work when used with mergerfs 2.18.0 and
above.
.IP \[bu] 2
Run mergerfs as \f[C]root\f[] unless you\[aq]re merging paths which are
owned by the same user otherwise strange permission issues may arise.
Run mergerfs as \f[C]root\f[] (with \f[B]allow_other\f[]) unless
you\[aq]re merging paths which are owned by the same user otherwise
strange permission issues may arise.
.IP \[bu] 2
https://github.com/trapexit/backup\-and\-recovery\-howtos : A set of
guides / howtos on creating a data storage system, backing it up,
@ -1537,11 +1531,9 @@ https://lkml.org/lkml/2016/9/14/527
.PP
There is a bug in the kernel.
A work around appears to be turning off \f[C]splice\f[].
Add \f[C]no_splice_write,no_splice_move,no_splice_read\f[] to
mergerfs\[aq] options.
Should be placed after \f[C]defaults\f[] if it is used since it will
turn them on.
This however is not guaranteed to work.
Don\[aq]t add the \f[C]splice_*\f[] arguments or add
\f[C]no_splice_write,no_splice_move,no_splice_read\f[].
This, however, is not guaranteed to work.
.SS rm: fts_read failed: No such file or directory
.PP
Not \f[I]really\f[] a bug.
@ -1670,7 +1662,7 @@ Below is an example of mhddfs and mergerfs setup to work similarly.
.PP
\f[C]mhddfs\ \-o\ mlimit=4G,allow_other\ /mnt/drive1,/mnt/drive2\ /mnt/pool\f[]
.PP
\f[C]mergerfs\ \-o\ minfreespace=4G,defaults,allow_other,category.create=ff\ /mnt/drive1:/mnt/drive2\ /mnt/pool\f[]
\f[C]mergerfs\ \-o\ minfreespace=4G,allow_other,category.create=ff\ /mnt/drive1:/mnt/drive2\ /mnt/pool\f[]
.SS Why use mergerfs over aufs?
.PP
aufs is mostly abandoned and no longer available in many distros.
@ -1867,14 +1859,12 @@ assuming there are few users.
.IP \[bu] 2
try adding (or removing) \f[C]direct_io\f[]
.IP \[bu] 2
try adding (or removing) \f[C]auto_cache\f[] / \f[C]noauto_cache\f[]
(included in \f[C]defaults\f[])
try adding (or removing) \f[C]auto_cache\f[]
.IP \[bu] 2
try adding (or removing) \f[C]kernel_cache\f[] (don\[aq]t use the
underlying filesystems directly if enabling \f[C]kernel_cache\f[])
try adding (or removing) \f[C]kernel_cache\f[]
.IP \[bu] 2
try adding (or removing) \f[C]splice_move\f[], \f[C]splice_read\f[], and
\f[C]splice_write\f[] (all three included in \f[C]defaults\f[])
\f[C]splice_write\f[]
.IP \[bu] 2
try increasing cache timeouts \f[C]attr_timeout\f[],
\f[C]entry_timeout\f[], \f[C]ac_attr_timeout\f[],

5
src/init.cpp
View File

@ -28,10 +28,11 @@ namespace mergerfs
{
ugid::init();
conn->want |= FUSE_CAP_ASYNC_READ;
conn->want |= FUSE_CAP_ATOMIC_O_TRUNC;
conn->want |= FUSE_CAP_BIG_WRITES;
conn->want |= FUSE_CAP_DONT_MASK;
#ifdef FUSE_CAP_IOCTL_DIR
conn->want |= FUSE_CAP_IOCTL_DIR;
#endif
return &Config::get_writable();
}

4
src/ioctl.cpp
View File

@ -62,7 +62,6 @@ namespace local
return local::ioctl(fi->fd,cmd_,data_);
}
#ifdef FUSE_IOCTL_DIR
#ifndef O_NOATIME
#define O_NOATIME 0
@ -119,7 +118,6 @@ namespace local
cmd_,
data_);
}
#endif
}
namespace mergerfs
@ -134,10 +132,8 @@ namespace mergerfs
unsigned int flags_,
void *data_)
{
#ifdef FUSE_IOCTL_DIR
if(flags_ & FUSE_IOCTL_DIR)
return local::ioctl_dir(ffi_,cmd_,data_);
#endif
return local::ioctl_file(ffi_,cmd_,data_);
}

17
src/option_parser.cpp
View File

@ -52,11 +52,9 @@ void
set_option(fuse_args &args,
const std::string &option_)
{
string option;
option = "-o" + option_;
fuse_opt_insert_arg(&args,1,option.c_str());
fuse_opt_add_arg(&args,"-o");
fuse_opt_add_arg(&args,option_.c_str());
}
static
@ -103,12 +101,8 @@ void
set_default_options(fuse_args &args)
{
set_option(args,"atomic_o_trunc");
set_option(args,"auto_cache");
set_option(args,"big_writes");
set_option(args,"default_permissions");
set_option(args,"splice_move");
set_option(args,"splice_read");
set_option(args,"splice_write");
}
static
@ -240,7 +234,7 @@ parse_and_process_arg(Config &config,
fuse_args *outargs)
{
if(arg == "defaults")
return (set_default_options(*outargs),0);
return 0;
else if(arg == "direct_io")
return (config.direct_io=true,0);
@ -363,10 +357,6 @@ usage(void)
"mergerfs options:\n"
" <srcpaths> ':' delimited list of directories. Supports\n"
" shell globbing (must be escaped in shell)\n"
" -o defaults Default FUSE options which seem to provide the\n"
" best performance: atomic_o_trunc, auto_cache,\n"
" big_writes, default_permissions, splice_read,\n"
" splice_write, splice_move\n"
" -o func.<f>=<p> Set function <f> to policy <p>\n"
" -o category.<c>=<p> Set functions in category <c> to <p>\n"
" -o cache.open=<int> 'open' policy cache timeout in seconds.\n"
@ -488,6 +478,7 @@ namespace mergerfs
opts,
::option_processor);
set_default_options(args);
set_fsname(args,config.branches);
set_subtype(args);
}