github-mirror/mergerfs
2
0
Fork 0
mirror of https://github.com/trapexit/mergerfs.git synced 2024-11-22 11:39:52 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity

misc README updates

Browse Source
This commit is contained in:
Antonio SJ Musumeci 2018-12-12 13:12:20 -05:00
parent fee9a449ba
commit 40969402e8
2 changed files with 114 additions and 102 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

87
README.md
View File

@ -1,6 +1,6 @@
% mergerfs(1) mergerfs user manual
% Antonio SJ Musumeci <trapexit@spawn.link>
% 2018-10-31
% 2018-12-12
# NAME
@ -124,22 +124,22 @@ By enabling `nullrw` mergerfs will work as it always does **except** that all re
Example:
```
$ dd if=/dev/zero of=/path/to/mergerfs/mount/benchmark ibs=1M obs=512 count=1024
$ dd if=/dev/zero of=/path/to/mergerfs/mount/benchmark ibs=1M obs=512 count=1024 conv=fdatasync
1024+0 records in
2097152+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 15.4067 s, 69.7 MB/s
$ dd if=/dev/zero of=/path/to/mergerfs/mount/benchmark ibs=1M obs=1M count=1024
$ dd if=/dev/zero of=/path/to/mergerfs/mount/benchmark ibs=1M obs=1M count=1024 conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.219585 s, 4.9 GB/s
$ dd if=/path/to/mergerfs/mount/benchmark of=/dev/null bs=512 count=102400
$ dd if=/path/to/mergerfs/mount/benchmark of=/dev/null bs=512 count=102400 conv=fdatasync
102400+0 records in
102400+0 records out
52428800 bytes (52 MB, 50 MiB) copied, 0.757991 s, 69.2 MB/s
$ dd if=/path/to/mergerfs/mount/benchmark of=/dev/null bs=1M count=1024
$ dd if=/path/to/mergerfs/mount/benchmark of=/dev/null bs=1M count=1024 conv=fdatasync
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.18405 s, 5.8 GB/s
@ -163,6 +163,7 @@ Policies, when called to create, will ignore drives which are readonly. This all
When using policies which are based on a device's available space the base path provided is used. Not the full path to the file in question. Meaning that sub mounts won't be considered in the space calculations. The reason is that it doesn't really work for non-path preserving policies and can lead to non-obvious behaviors.
#### Function / Category classifications
| Category | FUSE Functions |
@ -174,6 +175,7 @@ When using policies which are based on a device's available space the base path
Due to FUSE limitations **ioctl** behaves differently if its acting on a directory. It'll use the **getattr** policy to find and open the directory before issuing the **ioctl**. In other cases where something may be searched (to confirm a directory exists across all source mounts) **getattr** will also be used.
#### Path Preservation
Policies, as described below, are of two core types. `path preserving` and `non-path preserving`.
@ -184,29 +186,38 @@ A path preserving policy will only consider drives where the relative path being
When using non-path preserving policies paths will be cloned to target drives as necessary.
#### Filters
Policies basically search branches and create a list of files / paths for functions to work on. The policy is responsible for filtering and sorting. The policy type defines the sorting but filtering is mostly uniform as described below.
* No **search** policies filter.
* All **action** policies will filter out branches which are mounted **readonly** or tagged as **RO (readonly)**.
* All **create** policies will filter out branches which are mounted **readonly**, tagged **RO (readonly)** or **NC (no create)**, or has available space less than **minfreespace**.
If all branches are filtered an error will be returned. Typically **EROFS** or **ENOSPC** depending on the reasons.
#### Policy descriptions
All **create** policies will filter out branches which are mounted **read only** or tagged as **read only** or **no create**. All **action** policies will filter out branches which are mounted or tagged as **read only**.
If all branches are filtered an error will be returned. Typically EROFS or ENOSPC.
| Policy | Description |
|------------------|------------------------------------------------------------|
| all | Search category: acts like **ff**. Action category: apply to all found. Create category: for **mkdir**, **mknod**, and **symlink** it will apply to all branches. **create** works like **ff**. It will exclude branches with free space less than **minfreespace**. |
| epall (existing path, all) | Search category: acts like **epff**. Action category: apply to all found. Create category: for **mkdir**, **mknod**, and **symlink** it will apply to all existing paths found. **create** works like **epff**. Excludes branches with free space less than **minfreespace**. |
| epff (existing path, first found) | Given the order of the drives, as defined at mount time or configured at runtime, act on the first one found where the relative path already exists. For **create** category functions it will exclude branches with free space less than **minfreespace**. |
| eplfs (existing path, least free space) | Of all the drives on which the relative path exists choose the drive with the least free space. For **create** category functions it will exclude those with free space less than **minfreespace**. |
| eplus (existing path, least used space) | Of all the drives on which the relative path exists choose the drive with the least used space. For **create** category functions it will exclude those with free space less than **minfreespace**. |
| epmfs (existing path, most free space) | Of all the drives on which the relative path exists choose the drive with the most free space. For **create** category functions it will exclude those with free space less than **minfreespace**. |
| eprand (existing path, random) | Calls **epall** and then randomizes. Otherwise behaves the same as **epall**. |
| erofs | Exclusively return **-1** with **errno** set to **EROFS** (Read-only filesystem). |
| ff (first found) | Given the order of the drives, as defined at mount time or configured at runtime, act on the first one found. For **create** category functions it will exclude those with free space less than **minfreespace**. |
| lfs (least free space) | Pick the drive with the least available free space. For **create** category functions it will exclude those with free space less than **minfreespace**. |
| lus (least used space) | Pick the drive with the least used space. For **create** category functions it will exclude those with free space less than **minfreespace**. |
| mfs (most free space) | Pick the drive with the most available free space. |
| newest | Pick the file / directory with the largest mtime. For **create** category functions it will exclude those with free space less than **minfreespace**. |
| all | Search category: same as **epall**. Action category: same as **epall**. Create category: for **mkdir**, **mknod**, and **symlink** it will apply to all branches. **create** works like **ff**. |
| epall (existing path, all) | Search category: same as **epff** (but more expensive because it doesn't stop after finding a valid branch). Action category: apply to all found. Create category: for **mkdir**, **mknod**, and **symlink** it will apply to all found. **create** works like **epff** (but more expensive because it doesn't stop after finding a valid branch). |
| epff (existing path, first found) | Given the order of the branches, as defined at mount time or configured at runtime, act on the first one found where the relative path exists. |
| eplfs (existing path, least free space) | Of all the branches on which the relative path exists choose the drive with the least free space. |
| eplus (existing path, least used space) | Of all the branches on which the relative path exists choose the drive with the least used space. |
| epmfs (existing path, most free space) | Of all the branches on which the relative path exists choose the drive with the most free space. |
| eprand (existing path, random) | Calls **epall** and then randomizes. |
| erofs | Exclusively return **-1** with **errno** set to **EROFS** (readonly filesystem). |
| ff (first found) | Search category: same as **epff**. Action category: same as **epff**. Create category: Given the order of the drives, as defined at mount time or configured at runtime, act on the first one found. |
| lfs (least free space) | Search category: same as **eplfs**. Action category: same as **eplfs**. Create category: Pick the drive with the least available free space. |
| lus (least used space) | Search category: same as **eplus**. Action category: same as **eplus**. Create category: Pick the drive with the least used space. |
| mfs (most free space) | Search category: same as **epmfs**. Action category: same as **epmfs**. Create category: Pick the drive with the most available free space. |
| newest | Pick the file / directory with the largest mtime. |
| rand (random) | Calls **all** and then randomizes. |
#### Defaults ####
| Category | Policy |
@ -215,6 +226,7 @@ If all branches are filtered an error will be returned. Typically EROFS or ENOSP
| create | epmfs |
| search | ff |
#### rename & link ####
**NOTE:** If you're receiving errors from software when files are moved / renamed then you should consider changing the create policy to one which is **not** path preserving, enabling `ignorepponrename`, or contacting the author of the offending software and requesting that `EXDEV` be properly handled.
@ -255,19 +267,22 @@ The above behavior will help minimize the likelihood of EXDEV being returned but
**link** uses the same basic strategy.
#### readdir ####
[readdir](http://linux.die.net/man/3/readdir) is different from all other filesystem functions. While it could have it's own set of policies to tweak its behavior at this time it provides a simple union of files and directories found. Remember that any action or information queried about these files and directories come from the respective function. For instance: an **ls** is a **readdir** and for each file/directory returned **getattr** is called. Meaning the policy of **getattr** is responsible for choosing the file/directory which is the source of the metadata you see in an **ls**.
#### statvfs ####
[statvfs](http://linux.die.net/man/2/statvfs) normalizes the source drives based on the fragment size and sums the number of adjusted blocks and inodes. This means you will see the combined space of all sources. Total, used, and free. The sources however are dedupped based on the drive so multiple sources on the same drive will not result in double counting it's space. Filesystems mounted further down the tree of the branch will not be included.
# BUILDING
**NOTE:** Prebuilt packages can be found at: https://github.com/trapexit/mergerfs/releases
First get the code from [github](http://github.com/trapexit/mergerfs).
First get the code from [github](https://github.com/trapexit/mergerfs).
```
$ git clone https://github.com/trapexit/mergerfs.git
@ -277,11 +292,8 @@ $ wget https://github.com/trapexit/mergerfs/releases/download/<ver>/mergerfs-<ve
#### Debian / Ubuntu
```
$ sudo apt-get -y update
$ sudo apt-get -y install git make
$ cd mergerfs
$ make install-build-pkgs
$ # build-essential git g++ debhelper python automake libtool lsb-release
$ sudo tools/install-build-pkgs
$ make deb
$ sudo dpkg -i ../mergerfs_version_arch.deb
```
@ -289,11 +301,8 @@ $ sudo dpkg -i ../mergerfs_version_arch.deb
#### Fedora
```
$ su -
# dnf -y update
# dnf -y install git make
# cd mergerfs
# make install-build-pkgs
# # rpm-build gcc-c++ which python automake libtool gettext-devel
# tools/install-build-pkgs
# make rpm
# rpm -i rpmbuild/RPMS/<arch>/mergerfs-<verion>.<arch>.rpm
```
@ -819,17 +828,21 @@ Filesystems are very complex and difficult to debug. mergerfs, while being just
This software is free to use and released under a very liberal license. That said if you like this software and would like to support its development donations are welcome.
* PayPal: trapexit@spawn.link
* Patreon: https://www.patreon.com/trapexit
* SubscribeStar: https://www.subscribestar.com/trapexit
* Bitcoin (BTC): 12CdMhEPQVmjz3SSynkAEuD5q9JmhTDCZA
* Bitcoin Cash (BCH): 1AjPqZZhu7GVEs6JFPjHmtsvmDL4euzMzp
* Ethereum (ETH): 0x09A166B11fCC127324C7fc5f1B572255b3046E94
* Litecoin (LTC): LXAsq6yc6zYU3EbcqyWtHBrH1Ypx4GjUjm
* Ripple (XRP): rNACR2hqGjpbHuCKwmJ4pDpd2zRfuRATcE
* PayPal: trapexit@spawn.link
* Patreon: https://www.patreon.com/trapexit
# LINKS
* http://github.com/trapexit/mergerfs
* http://github.com/trapexit/mergerfs-tools
* http://github.com/trapexit/scorch
* http://github.com/trapexit/backup-and-recovery-howtos
* https://spawn.link
* https://github.com/trapexit/mergerfs
* https://github.com/trapexit/mergerfs-tools
* https://github.com/trapexit/scorch
* https://github.com/trapexit/bbf
* https://github.com/trapexit/backup-and-recovery-howtos

129
man/mergerfs.1
View File

@ -1,7 +1,7 @@
.\"t
.\" Automatically generated by Pandoc 1.19.2.4
.\"
.TH "mergerfs" "1" "2018\-10\-31" "mergerfs user manual" ""
.TH "mergerfs" "1" "2018\-12\-12" "mergerfs user manual" ""
.hy
.SH NAME
.PP
@ -314,22 +314,22 @@ Example:
.IP
.nf
\f[C]
$\ dd\ if=/dev/zero\ of=/path/to/mergerfs/mount/benchmark\ ibs=1M\ obs=512\ count=1024
$\ dd\ if=/dev/zero\ of=/path/to/mergerfs/mount/benchmark\ ibs=1M\ obs=512\ count=1024\ conv=fdatasync
1024+0\ records\ in
2097152+0\ records\ out
1073741824\ bytes\ (1.1\ GB,\ 1.0\ GiB)\ copied,\ 15.4067\ s,\ 69.7\ MB/s
$\ dd\ if=/dev/zero\ of=/path/to/mergerfs/mount/benchmark\ ibs=1M\ obs=1M\ count=1024
$\ dd\ if=/dev/zero\ of=/path/to/mergerfs/mount/benchmark\ ibs=1M\ obs=1M\ count=1024\ conv=fdatasync
1024+0\ records\ in
1024+0\ records\ out
1073741824\ bytes\ (1.1\ GB,\ 1.0\ GiB)\ copied,\ 0.219585\ s,\ 4.9\ GB/s
$\ dd\ if=/path/to/mergerfs/mount/benchmark\ of=/dev/null\ bs=512\ count=102400
$\ dd\ if=/path/to/mergerfs/mount/benchmark\ of=/dev/null\ bs=512\ count=102400\ conv=fdatasync
102400+0\ records\ in
102400+0\ records\ out
52428800\ bytes\ (52\ MB,\ 50\ MiB)\ copied,\ 0.757991\ s,\ 69.2\ MB/s
$\ dd\ if=/path/to/mergerfs/mount/benchmark\ of=/dev/null\ bs=1M\ count=1024
$\ dd\ if=/path/to/mergerfs/mount/benchmark\ of=/dev/null\ bs=1M\ count=1024\ conv=fdatasync
1024+0\ records\ in
1024+0\ records\ out
1073741824\ bytes\ (1.1\ GB,\ 1.0\ GiB)\ copied,\ 0.18405\ s,\ 5.8\ GB/s
@ -443,16 +443,26 @@ path being accessed already exists.
.PP
When using non\-path preserving policies paths will be cloned to target
drives as necessary.
.SS Policy descriptions
.SS Filters
.PP
All \f[B]create\f[] policies will filter out branches which are mounted
\f[B]read only\f[] or tagged as \f[B]read only\f[] or \f[B]no
create\f[].
Policies basically search branches and create a list of files / paths
for functions to work on.
The policy is responsible for filtering and sorting.
The policy type defines the sorting but filtering is mostly uniform as
described below.
.IP \[bu] 2
No \f[B]search\f[] policies filter.
.IP \[bu] 2
All \f[B]action\f[] policies will filter out branches which are mounted
or tagged as \f[B]read only\f[].
\f[B]readonly\f[] or tagged as \f[B]RO (readonly)\f[].
.IP \[bu] 2
All \f[B]create\f[] policies will filter out branches which are mounted
\f[B]readonly\f[], tagged \f[B]RO (readonly)\f[] or \f[B]NC (no
create)\f[], or has available space less than \f[B]minfreespace\f[].
.PP
If all branches are filtered an error will be returned.
Typically EROFS or ENOSPC.
Typically \f[B]EROFS\f[] or \f[B]ENOSPC\f[] depending on the reasons.
.SS Policy descriptions
.PP
.TS
tab(@);
@ -466,102 +476,91 @@ _
T{
all
T}@T{
Search category: acts like \f[B]ff\f[].
Action category: apply to all found.
Search category: same as \f[B]epall\f[].
Action category: same as \f[B]epall\f[].
Create category: for \f[B]mkdir\f[], \f[B]mknod\f[], and
\f[B]symlink\f[] it will apply to all branches.
\f[B]create\f[] works like \f[B]ff\f[].
It will exclude branches with free space less than
\f[B]minfreespace\f[].
T}
T{
epall (existing path, all)
T}@T{
Search category: acts like \f[B]epff\f[].
Search category: same as \f[B]epff\f[] (but more expensive because it
doesn\[aq]t stop after finding a valid branch).
Action category: apply to all found.
Create category: for \f[B]mkdir\f[], \f[B]mknod\f[], and
\f[B]symlink\f[] it will apply to all existing paths found.
\f[B]create\f[] works like \f[B]epff\f[].
Excludes branches with free space less than \f[B]minfreespace\f[].
\f[B]symlink\f[] it will apply to all found.
\f[B]create\f[] works like \f[B]epff\f[] (but more expensive because it
doesn\[aq]t stop after finding a valid branch).
T}
T{
epff (existing path, first found)
T}@T{
Given the order of the drives, as defined at mount time or configured at
runtime, act on the first one found where the relative path already
exists.
For \f[B]create\f[] category functions it will exclude branches with
free space less than \f[B]minfreespace\f[].
Given the order of the branches, as defined at mount time or configured
at runtime, act on the first one found where the relative path exists.
T}
T{
eplfs (existing path, least free space)
T}@T{
Of all the drives on which the relative path exists choose the drive
Of all the branches on which the relative path exists choose the drive
with the least free space.
For \f[B]create\f[] category functions it will exclude those with free
space less than \f[B]minfreespace\f[].
T}
T{
eplus (existing path, least used space)
T}@T{
Of all the drives on which the relative path exists choose the drive
Of all the branches on which the relative path exists choose the drive
with the least used space.
For \f[B]create\f[] category functions it will exclude those with free
space less than \f[B]minfreespace\f[].
T}
T{
epmfs (existing path, most free space)
T}@T{
Of all the drives on which the relative path exists choose the drive
Of all the branches on which the relative path exists choose the drive
with the most free space.
For \f[B]create\f[] category functions it will exclude those with free
space less than \f[B]minfreespace\f[].
T}
T{
eprand (existing path, random)
T}@T{
Calls \f[B]epall\f[] and then randomizes.
Otherwise behaves the same as \f[B]epall\f[].
T}
T{
erofs
T}@T{
Exclusively return \f[B]\-1\f[] with \f[B]errno\f[] set to
\f[B]EROFS\f[] (Read\-only filesystem).
\f[B]EROFS\f[] (readonly filesystem).
T}
T{
ff (first found)
T}@T{
Given the order of the drives, as defined at mount time or configured at
runtime, act on the first one found.
For \f[B]create\f[] category functions it will exclude those with free
space less than \f[B]minfreespace\f[].
Search category: same as \f[B]epff\f[].
Action category: same as \f[B]epff\f[].
Create category: Given the order of the drives, as defined at mount time
or configured at runtime, act on the first one found.
T}
T{
lfs (least free space)
T}@T{
Pick the drive with the least available free space.
For \f[B]create\f[] category functions it will exclude those with free
space less than \f[B]minfreespace\f[].
Search category: same as \f[B]eplfs\f[].
Action category: same as \f[B]eplfs\f[].
Create category: Pick the drive with the least available free space.
T}
T{
lus (least used space)
T}@T{
Pick the drive with the least used space.
For \f[B]create\f[] category functions it will exclude those with free
space less than \f[B]minfreespace\f[].
Search category: same as \f[B]eplus\f[].
Action category: same as \f[B]eplus\f[].
Create category: Pick the drive with the least used space.
T}
T{
mfs (most free space)
T}@T{
Pick the drive with the most available free space.
Search category: same as \f[B]epmfs\f[].
Action category: same as \f[B]epmfs\f[].
Create category: Pick the drive with the most available free space.
T}
T{
newest
T}@T{
Pick the file / directory with the largest mtime.
For \f[B]create\f[] category functions it will exclude those with free
space less than \f[B]minfreespace\f[].
T}
T{
rand (random)
@ -721,7 +720,7 @@ included.
\f[B]NOTE:\f[] Prebuilt packages can be found at:
https://github.com/trapexit/mergerfs/releases
.PP
First get the code from github (http://github.com/trapexit/mergerfs).
First get the code from github (https://github.com/trapexit/mergerfs).
.IP
.nf
\f[C]
@ -734,11 +733,8 @@ $\ wget\ https://github.com/trapexit/mergerfs/releases/download/<ver>/mergerfs\-
.IP
.nf
\f[C]
$\ sudo\ apt\-get\ \-y\ update
$\ sudo\ apt\-get\ \-y\ install\ git\ make
$\ cd\ mergerfs
$\ make\ install\-build\-pkgs
$\ #\ build\-essential\ git\ g++\ debhelper\ python\ automake\ libtool\ lsb\-release
$\ sudo\ tools/install\-build\-pkgs
$\ make\ deb
$\ sudo\ dpkg\ \-i\ ../mergerfs_version_arch.deb
\f[]
@ -748,11 +744,8 @@ $\ sudo\ dpkg\ \-i\ ../mergerfs_version_arch.deb
.nf
\f[C]
$\ su\ \-
#\ dnf\ \-y\ update
#\ dnf\ \-y\ install\ git\ make
#\ cd\ mergerfs
#\ make\ install\-build\-pkgs
#\ #\ rpm\-build\ gcc\-c++\ which\ python\ automake\ libtool\ gettext\-devel
#\ tools/install\-build\-pkgs
#\ make\ rpm
#\ rpm\ \-i\ rpmbuild/RPMS/<arch>/mergerfs\-<verion>.<arch>.rpm
\f[]
@ -1753,6 +1746,12 @@ This software is free to use and released under a very liberal license.
That said if you like this software and would like to support its
development donations are welcome.
.IP \[bu] 2
PayPal: trapexit\@spawn.link
.IP \[bu] 2
Patreon: https://www.patreon.com/trapexit
.IP \[bu] 2
SubscribeStar: https://www.subscribestar.com/trapexit
.IP \[bu] 2
Bitcoin (BTC): 12CdMhEPQVmjz3SSynkAEuD5q9JmhTDCZA
.IP \[bu] 2
Bitcoin Cash (BCH): 1AjPqZZhu7GVEs6JFPjHmtsvmDL4euzMzp
@ -1762,18 +1761,18 @@ Ethereum (ETH): 0x09A166B11fCC127324C7fc5f1B572255b3046E94
Litecoin (LTC): LXAsq6yc6zYU3EbcqyWtHBrH1Ypx4GjUjm
.IP \[bu] 2
Ripple (XRP): rNACR2hqGjpbHuCKwmJ4pDpd2zRfuRATcE
.IP \[bu] 2
PayPal: trapexit\@spawn.link
.IP \[bu] 2
Patreon: https://www.patreon.com/trapexit
.SH LINKS
.IP \[bu] 2
http://github.com/trapexit/mergerfs
https://spawn.link
.IP \[bu] 2
http://github.com/trapexit/mergerfs\-tools
https://github.com/trapexit/mergerfs
.IP \[bu] 2
http://github.com/trapexit/scorch
https://github.com/trapexit/mergerfs\-tools
.IP \[bu] 2
http://github.com/trapexit/backup\-and\-recovery\-howtos
https://github.com/trapexit/scorch
.IP \[bu] 2
https://github.com/trapexit/bbf
.IP \[bu] 2
https://github.com/trapexit/backup\-and\-recovery\-howtos
.SH AUTHORS
Antonio SJ Musumeci <trapexit@spawn.link>.