Lots of misc README updates

man/mergerfs.1

@@ -1,7 +1,7 @@
 .\"t
 .\" Automatically generated by Pandoc 2.9.2.1
 .\"
-.TH "mergerfs" "1" "2023-02-23" "mergerfs user manual" ""
+.TH "mergerfs" "1" "" "mergerfs user manual" ""
 .hy
 .SH NAME
 .PP
@@ -74,12 +74,12 @@ A         +      B        =       C
 \f[R]
 .fi
 .PP
-mergerfs does \f[B]NOT\f[R] support the copy-on-write (CoW) or whiteout
+mergerfs does \f[B]not\f[R] support the copy-on-write (CoW) or whiteout
 behaviors found in \f[B]aufs\f[R] and \f[B]overlayfs\f[R].
 You can \f[B]not\f[R] mount a read-only filesystem and write to it.
 However, mergerfs will ignore read-only drives when creating new files
 so you can mix read-write and read-only drives.
-It also does \f[B]NOT\f[R] split data across drives.
+It also does \f[B]not\f[R] split data across drives.
 It is not RAID0 / striping.
 It is simply a union of other filesystems.
 .SH TERMINOLOGY
@@ -110,6 +110,38 @@ start with one of the following option sets.
 .SS You don\[cq]t need \f[C]mmap\f[R]
 .PP
 \f[C]cache.files=off,dropcacheonclose=true,category.create=mfs\f[R]
+.SS Command Line
+.PP
+\f[C]mergerfs -o create.files=partial,dropcacheonclose=true,category.create=mfs /mnt/hdd0:/mnt/hdd1 /media\f[R]
+.SS /etc/fstab
+.PP
+\f[C]/mnt/hdd0:/mnt/hdd1 /media fuse.mergerfs create.files=partial,dropcacheonclose=true,category.create=mfs 0 0\f[R]
+.SS systemd mount
+.PP
+https://github.com/trapexit/mergerfs/wiki/systemd
+.IP
+.nf
+\f[C]
+[Unit]
+Description=mergerfs service
+
+[Service]
+Type=simple
+KillMode=none
+ExecStart=/usr/bin/mergerfs \[rs]
+  -f \[rs]
+  -o create.files=partial \[rs]
+  -o dropcacheonclose=true \[rs]
+  -o category.create=mfs \[rs]
+  /mnt/hdd0:/mnt/hdd1 \[rs]
+  /media
+ExecStop=/bin/fusermount -uz /media
+Restart=on-failure
+
+[Install]
+WantedBy=default.target
+\f[R]
+.fi
 .PP
 See the mergerfs wiki for real world
 deployments (https://github.com/trapexit/mergerfs/wiki/Real-World-Deployments)
@@ -168,6 +200,11 @@ The request will succeed but do nothing.
 Useful for benchmarking mergerfs.
 (default: false)
 .IP \[bu] 2
+\f[B]lazy-umount-mountpoint=BOOL\f[R]: mergerfs will attempt to
+\[lq]lazy umount\[rq] the mountpoint before mounting itself.
+Useful when performing live upgrades of mergerfs.
+(default: false)
+.IP \[bu] 2
 \f[B]ignorepponrename=BOOL\f[R]: Ignore path preserving on rename.
 Typically rename and link act differently depending on the policy of
 \f[C]create\f[R] (read below).
@@ -364,11 +401,22 @@ Use \f[C]async_read=false\f[R] instead.
 \f[B]splice_write\f[R]: deprecated - Does nothing.
 .IP \[bu] 2
 \f[B]splice_move\f[R]: deprecated - Does nothing.
+.IP \[bu] 2
+\f[B]allow_other\f[R]: deprecated - mergerfs always sets this FUSE
+option as normal permissions can be used to limit access.
+.IP \[bu] 2
+\f[B]use_ino\f[R]: deprecated - mergerfs should always control inode
+calculation so this is enabled all the time.
 .PP
 \f[B]NOTE:\f[R] Options are evaluated in the order listed so if the
 options are \f[B]func.rmdir=rand,category.action=ff\f[R] the
 \f[B]action\f[R] category setting will override the \f[B]rmdir\f[R]
 setting.
+.PP
+\f[B]NOTE:\f[R] Always look at the documentation for the version of
+mergerfs you\[cq]re using.
+Not all features are available in older releases.
+Use \f[C]man mergerfs\f[R] or find the docs as linked in the release.
 .SS Value Types
 .IP \[bu] 2
 BOOL = `true' | `false'
@@ -403,7 +451,7 @@ calculation and a individual minfreespace value.
 The values are set by prepending an \f[C]=\f[R] at the end of a branch
 designation and using commas as delimiters.
 Example: /mnt/drive=RW,1234
-.SS branch type
+.SS branch mode
 .IP \[bu] 2
 RW: (read/write) - Default behavior.
 Will be eligible in all policy categories.
@@ -417,7 +465,7 @@ NC: (no-create) - Will be excluded from \f[C]create\f[R] policies.
 You can\[cq]t create on that branch but you can change or delete.
 .SS minfreespace
 .PP
-Same purpose as the global option but specific to the branch.
+Same purpose and syntax as the global option but specific to the branch.
 If not set the global value is used.
 .SS globbing
 .PP
@@ -428,12 +476,12 @@ the shell itself will apply the glob itself.\f[R]
 .IP
 .nf
 \f[C]
-# mergerfs /mnt/disk\[rs]*:/mnt/cdrom /media/drives
+# mergerfs /mnt/hdd\[rs]*:/mnt/ssd /media
 \f[R]
 .fi
 .PP
 The above line will use all mount points in /mnt prefixed with
-\f[B]disk\f[R] and the \f[B]cdrom\f[R].
+\f[B]hdd\f[R] and \f[B]ssd\f[R].
 .PP
 To have the pool mounted at boot or otherwise accessible from related
 tools use \f[B]/etc/fstab\f[R].
@@ -441,7 +489,7 @@ tools use \f[B]/etc/fstab\f[R].
 .nf
 \f[C]
 # <file system>        <mount point>  <type>         <options>             <dump>  <pass>
-/mnt/disk*:/mnt/cdrom  /mnt/pool      fuse.mergerfs  minfreespace=16G      0       0
+/mnt/hdd*:/mnt/ssd    /media          fuse.mergerfs  minfreespace=16G      0       0
 \f[R]
 .fi
 .PP
@@ -532,7 +580,7 @@ This tuple is not client facing.
 The inode that is presented to the client is passed through the kernel
 uninterpreted.
 .PP
-From FUSE docs regarding \f[C]use_ino\f[R]:
+From FUSE docs for \f[C]use_ino\f[R]:
 .IP
 .nf
 \f[C]
@@ -879,7 +927,7 @@ a branch.
 \f[B]ENOENT\f[R] will be returned if no eligible branch is found.
 .SS Path Preservation
 .PP
-Policies, as described below, are of two basic types.
+Policies, as described below, are of two basic classifications.
 \f[C]path preserving\f[R] and \f[C]non-path preserving\f[R].
 .PP
 All policies which start with \f[C]ep\f[R] (\f[B]epff\f[R],
@@ -897,8 +945,6 @@ With the \f[C]msp\f[R] or \f[C]most shared path\f[R] policies they are
 defined as \f[C]path preserving\f[R] for the purpose of controlling
 \f[C]link\f[R] and \f[C]rename\f[R]\[cq]s behaviors since
 \f[C]ignorepponrename\f[R] is available to disable that behavior.
-In mergerfs v3.0 the path preserving behavior of rename and link will
-likely be separated from the policy all together.
 .SS Policy descriptions
 .PP
 A policy\[cq]s behavior differs, as mentioned above, based on the
@@ -906,7 +952,6 @@ function it is used with.
 Sometimes it really might not make sense to even offer certain policies
 because they are literally the same as others but it makes things a bit
 more uniform.
-In mergerfs 3.0 this might change.
 .PP
 .TS
 tab(@);
@@ -1249,10 +1294,99 @@ For create functions \f[C]mkdir\f[R], \f[C]mknod\f[R], and
 \f[C]symlink\f[R] which don\[cq]t return a file descriptor and therefore
 can have \f[C]all\f[R] or \f[C]epall\f[R] policies it will return
 success if any of the calls succeed and an error otherwise.
-.SH BUILD / UPDATE
+.SH INSTALL
+.PP
+https://github.com/trapexit/mergerfs/releases
+.PP
+If your distribution\[cq]s package manager includes mergerfs check if
+the version is up to date.
+If out of date it is recommended to use the latest release found on the
+release page.
+Details for common distros are below.
+.SS Debian
+.PP
+Most Debian installs are of a stable branch and therefore do not have
+the most up to date software.
+While mergerfs is available via \f[C]apt\f[R] it is suggested that uses
+install the most recent version available from the releases
+page (https://github.com/trapexit/mergerfs/releases).
+.SS prebuilt deb
+.IP
+.nf
+\f[C]
+wget https://github.com/trapexit/mergerfs/releases/download/<ver>/mergerfs_<ver>.debian-<rel>_<arch>.deb
+dpkg -i mergerfs_<ver>.debian-<rel>_<arch>.deb
+\f[R]
+.fi
+.SS apt
+.IP
+.nf
+\f[C]
+sudo apt install -y mergerfs
+\f[R]
+.fi
+.SS Ubuntu
+.PP
+Most Ubuntu installs are of a stable branch and therefore do not have
+the most up to date software.
+While mergerfs is available via \f[C]apt\f[R] it is suggested that uses
+install the most recent version available from the releases
+page (https://github.com/trapexit/mergerfs/releases).
+.SS prebuilt deb
+.IP
+.nf
+\f[C]
+wget https://github.com/trapexit/mergerfs/releases/download/<version>/mergerfs_<ver>.ubuntu-<rel>_<arch>.deb
+dpkg -i mergerfs_<ver>.ubuntu-<rel>_<arch>.deb
+\f[R]
+.fi
+.SS apt
+.IP
+.nf
+\f[C]
+sudo apt install -y mergerfs
+\f[R]
+.fi
+.SS Raspberry Pi OS
+.PP
+Effectively the same as Debian or Ubuntu.
+.SS Fedora
+.IP
+.nf
+\f[C]
+wget https://github.com/trapexit/mergerfs/releases/download/<ver>/mergerfs-<ver>.fc<rel>.<arch>.rpm
+sudo rpm -i mergerfs-<ver>.fc<rel>.<arch>.rpm
+\f[R]
+.fi
+.SS CentOS / Rocky
+.IP
+.nf
+\f[C]
+wget https://github.com/trapexit/mergerfs/releases/download/<ver>/mergerfs-<ver>.el<rel>.<arch>.rpm
+sudo rpm -i mergerfs-<ver>.el<rel>.<arch>.rpm
+\f[R]
+.fi
+.SS ArchLinux
+.IP "1." 3
+Setup AUR
+.IP "2." 3
+Install \f[C]mergerfs\f[R]
+.SS Other
+.PP
+Static binaries are provided for situations where native packages are
+unavailable.
+.IP
+.nf
+\f[C]
+wget https://github.com/trapexit/mergerfs/releases/download/<ver>/mergerfs-static-linux_<arch>.tar.gz
+sudo tar xvf mergerfs-static-linux_<arch>.tar.gz -C /
+\f[R]
+.fi
+.SH BUILD
 .PP
 \f[B]NOTE:\f[R] Prebuilt packages can be found at and recommended for
 most users: https://github.com/trapexit/mergerfs/releases
+.PP
 \f[B]NOTE:\f[R] Only tagged releases are supported.
 \f[C]master\f[R] and other branches should be considered works in
 progress.
@@ -1276,7 +1410,7 @@ $ make deb
 $ sudo dpkg -i ../mergerfs_<version>_<arch>.deb
 \f[R]
 .fi
-.SS RHEL / CentOS /Fedora
+.SS RHEL / CentOS / Rocky / Fedora
 .IP
 .nf
 \f[C]
@@ -1287,7 +1421,7 @@ $ su -
 # rpm -i rpmbuild/RPMS/<arch>/mergerfs-<version>.<arch>.rpm
 \f[R]
 .fi
-.SS Generically
+.SS Generic
 .PP
 Have git, g++, make, python installed.
 .IP
@@ -1339,6 +1473,8 @@ continue to run even if the software using it stop or are restarted.
 To work around this you can use a \[lq]lazy umount\[rq].
 Before mounting over top the mount point with the new instance of
 mergerfs issue: \f[C]umount -l <mergerfs_mountpoint>\f[R].
+Or you can let mergerfs do it by setting the option
+\f[C]lazy-umount-mountpoint=true\f[R].
 .SH RUNTIME CONFIG
 .SS .mergerfs pseudo file
 .IP
@@ -1825,8 +1961,6 @@ enable (or disable) page caching (\f[C]cache.files\f[R])
 .IP \[bu] 2
 enable \f[C]cache.writeback\f[R]
 .IP \[bu] 2
-enable \f[C]cache.open\f[R]
-.IP \[bu] 2
 enable \f[C]cache.statfs\f[R]
 .IP \[bu] 2
 enable \f[C]cache.symlinks\f[R]
@@ -1856,6 +1990,8 @@ increase readahead on all devices:
 .PP
 If you come across a setting that significantly impacts performance
 please contact trapexit so he may investigate further.
+Please test both against your normal setup, a singular branch, and with
+\f[C]nullrw=true\f[R]
 .SH BENCHMARKING
 .PP
 Filesystems are complicated.
@@ -1959,19 +2095,16 @@ echo 3 | sudo tee /proc/sys/vm/drop_caches
 .fi
 .SH TIPS / NOTES
 .IP \[bu] 2
-This document is very literal and thorough.
+This document is literal and thorough.
-Unless there is a bug things work as described.
 If a suspected feature isn\[cq]t mentioned it doesn\[cq]t exist.
 If certain libfuse arguments aren\[cq]t listed they probably
 shouldn\[cq]t be used.
 .IP \[bu] 2
 Ensure you\[cq]re using the latest version.
-Few distros have the latest version.
 .IP \[bu] 2
-Run mergerfs as \f[C]root\f[R] unless you\[cq]re merging paths which are
+Run mergerfs as \f[C]root\f[R].
-owned exclusively and fully by the same user otherwise strange
+mergerfs is designed and intended to be run as \f[C]root\f[R] and may
-permission issues may arise.
+exibit incorrect behavior if run otherwise..
-mergerfs is designed and intended to be run as \f[C]root\f[R].
 .IP \[bu] 2
 If you don\[cq]t see some directories and files you expect, policies
 seem to skip branches, you get strange permission errors, etc.
@@ -2077,8 +2210,8 @@ noforget
 inodecalc=path-hash
 .SS rtorrent fails with ENODEV (No such device)
 .PP
-Be sure to set \f[C]cache.files=partial|full|auto-full\f[R] or turn off
+Be sure to set \f[C]cache.files=partial|full|auto-full|per-processe\f[R]
-\f[C]direct_io\f[R].
+or turn off \f[C]direct_io\f[R].
 rtorrent and some other applications use
 mmap (http://linux.die.net/man/2/mmap) to read and write to files and
 offer no fallback to traditional methods.
@@ -2232,45 +2365,6 @@ There might be a hack to work around this (make mergerfs read the
 would be limited to Linux and the /etc/group DB.
 Preferably users would mount in the host group file into the containers
 or use a standard shared user & groups technology like NIS or LDAP.
-.SS mergerfs or libfuse crashing
-.PP
-First\&... always upgrade to the latest version unless told otherwise.
-.PP
-If using mergerfs below 2.22.0:
-.PP
-If suddenly the mergerfs mount point disappears and
-\f[C]Transport endpoint is not connected\f[R] is returned when
-attempting to perform actions within the mount directory \f[B]and\f[R]
-the version of libfuse (use \f[C]mergerfs -v\f[R] to find the version)
-is older than \f[C]2.9.4\f[R] its likely due to a bug in libfuse.
-Affected versions of libfuse can be found in Debian Wheezy, Ubuntu
-Precise and others.
-.PP
-In order to fix this please install newer versions of libfuse.
-If using a Debian based distro (Debian,Ubuntu,Mint) you can likely just
-install newer versions of
-libfuse (https://packages.debian.org/unstable/libfuse2) and
-fuse (https://packages.debian.org/unstable/fuse) from the repo of a
-newer release.
-.PP
-If using mergerfs at or above 2.22.0:
-.PP
-First upgrade if possible, check the known bugs section, and contact
-trapexit.
-.SS mergerfs appears to be crashing or exiting
-.PP
-There seems to be an issue with Linux version \f[C]4.9.0\f[R] and above
-in which an invalid message appears to be transmitted to libfuse (used
-by mergerfs) causing it to exit.
-No messages will be printed in any logs as it\[cq]s not a proper crash.
-Debugging of the issue is still ongoing and can be followed via the
-fuse-devel
-thread (https://sourceforge.net/p/fuse/mailman/message/35662577).
-.SS rm: fts_read failed: No such file or directory
-.PP
-Please update.
-This is only happened to mergerfs versions at or below v2.25.x and will
-not occur in more recent versions.
 .SH FAQ
 .SS How well does mergerfs scale? Is it \[lq]production ready?\[rq]
 .PP
@@ -2352,7 +2446,7 @@ the documentation will be improved.
 .PP
 That said, for the average person, the following should be fine:
 .PP
-\f[C]-o cache.files=off,dropcacheonclose=true,category.create=mfs\f[R]
+\f[C]cache.files=off,dropcacheonclose=true,category.create=mfs\f[R]
 .SS Why are all my files ending up on 1 drive?!
 .PP
 Did you start with empty drives?
@@ -2470,7 +2564,7 @@ Some of these also affect the use of mergerfs from container platforms
 such as Docker.
 .SS Why use FUSE? Why not a kernel based solution?
 .PP
-As with any two solutions to a problem there are advantages and
+As with any solutions to a problem there are advantages and
 disadvantages to each one.
 .PP
 A FUSE based solution has all the downsides of FUSE:
@@ -2497,8 +2591,6 @@ Allows more flexibility in design and features
 .IP \[bu] 2
 Overall easier to write, secure, and maintain
 .IP \[bu] 2
-Ability to run without root access or need to change the kernel
-.IP \[bu] 2
 Much lower barrier to entry (getting code into the kernel takes a lot of
 time and effort initially)
 .PP
@@ -2517,33 +2609,6 @@ to using \f[C]fuse.mergerfs\f[R] as the type in \f[C]/etc/fstab\f[R].
 If \f[C]mergerfs\f[R] doesn\[cq]t work as a type it could be due to how
 the \f[C]mount.mergerfs\f[R] tool was installed.
 Must be in \f[C]/sbin/\f[R] with proper permissions.
-.SS Why was libfuse embedded into mergerfs?
-.IP "1." 3
-A significant number of users use mergerfs on distros with old versions
-of libfuse which have serious bugs.
-Requiring updated versions of libfuse on those distros isn\[cq]t
-practical (no package offered, user inexperience, etc.).
-The only practical way to provide a stable runtime on those systems was
-to \[lq]vendor\[rq] / embed the library into the project.
-.IP "2." 3
-mergerfs was written to use the high level API.
-There are a number of limitations in the HLAPI that make certain
-features difficult or impossible to implement.
-While some of these features could be patched into newer versions of
-libfuse without breaking the public API some of them would require hacky
-code to provide backwards compatibility.
-While it may still be worth working with upstream to address these
-issues in future versions, since the library needs to be vendored for
-stability and compatibility reasons it is preferable / easier to modify
-the API.
-Longer term the plan is to rewrite mergerfs to use the low level API.
-.SS Why did support for system libfuse get removed?
-.PP
-See above first.
-.PP
-If/when mergerfs is rewritten to use the low-level API then it\[cq]ll be
-plausible to support system libfuse but till then it\[cq]s simply too
-much work to manage the differences across the versions.
 .SS Why was splice support removed?
 .PP
 After a lot of testing over the years splicing always appeared to be at
@@ -2733,11 +2798,6 @@ Disable file caching.
 If you aren\[cq]t using applications which use \f[C]mmap\f[R] it\[cq]s
 probably simpler to just disable it all together.
 The kernel won\[cq]t send the requests when caching is disabled.
-.SS What are these .fuse_hidden files?
-.PP
-Please upgrade.
-mergerfs >= 2.26.0 will not have these temporary files.
-See the notes on \f[C]unlink\f[R].
 .SS It\[cq]s mentioned that there are some security issues with mhddfs. What are they? How does mergerfs address them?
 .PP
 mhddfs (https://github.com/trapexit/mhddfs) manages running as
@@ -2789,11 +2849,11 @@ previously encountered questions/issues.
 \f[B]Please make sure you are using the latest
 release (https://github.com/trapexit/mergerfs/releases) or have tried it
 in comparison. Old versions, which are often included in distros like
-Debian and Ubuntu, are not ever going to be updated and your bug may
+Debian and Ubuntu, are not ever going to be updated and the issue you
-have been addressed already.\f[R]
+are encountering may have been addressed already.\f[R]
 .PP
 \f[B]For commercial support or feature requests please contact me
-directly.\f[R]
+directly. (mailto:support@spawn.link)\f[R]
 .SS Information to include in bug reports
 .IP \[bu] 2
 Information about the broader problem along with any attempted
@@ -2801,7 +2861,7 @@ solutions. (https://xyproblem.info)
 .IP \[bu] 2
 Solution already ruled out and why.
 .IP \[bu] 2
-Version of mergerfs: \f[C]mergerfs -V\f[R]
+Version of mergerfs: \f[C]mergerfs --version\f[R]
 .IP \[bu] 2
 mergerfs settings / arguments: from fstab, systemd unit, command line,
 OMV plugin, etc.
@@ -2848,21 +2908,26 @@ etc.
 .IP \[bu] 2
 github.com: https://github.com/trapexit/mergerfs/issues
 .IP \[bu] 2
-email: trapexit\[at]spawn.link
-.IP \[bu] 2
 discord: https://discord.gg/MpAr69V
 .IP \[bu] 2
-twitter: https://twitter.com/_trapexit
+reddit: https://www.reddit.com/r/mergerfs
-.IP \[bu] 2
+.SS Donations
-reddit: https://www.reddit.com/user/trapexit
-.SS Support development
-.PP
-This software is released under the very liberal ISC license and is
-therefore free to use for personal or commercial uses.
-That said if you like this software and have the means please consider
-supporting its development.
 .PP
 https://github.com/trapexit/support
+.PP
+Development and support of a project like mergerfs requires a
+significant amount of time and effort.
+The software is released under the very liberal ISC license and is
+therefore free to use for personal or commercial uses.
+.PP
+If you are a personal user and find mergerfs and its support valuable
+and would like to support the project financially it would be very much
+appreciated.
+.PP
+If you are using mergerfs commercially please consider sponsoring the
+project to ensure it continues to be maintained and receive updates.
+If custom features are needed feel free to contact me
+directly (mailto:support@spawn.link).
 .SH LINKS
 .IP \[bu] 2
 https://spawn.link
@@ -2876,5 +2941,3 @@ https://github.com/trapexit/mergerfs-tools
 https://github.com/trapexit/scorch
 .IP \[bu] 2
 https://github.com/trapexit/bbf
-.SH AUTHORS
-Antonio SJ Musumeci <trapexit@spawn.link>.