We use optimistic concurrency when rewriting the history file to
minimize the lock scope. Unfortunately, old.mtime == new.mtime
does not imply that file is unchanged; we don't have guarantees
on the granularity of the modification time timestamp, see
https://stackoverflow.com/questions/14392975/timestamp-accuracy-on-ext4-sub-millsecond
So let's lock before reading any old contents and use the other
"write-to-tempfile-and-rename" code path only when locking fails.
Potentially fixes#10300
(untested) which probably happens because read_zero_padded() attempts to
read bytes that have not been flushed yet.
No functional change, since with the parent commit, we no longer treat
"DirRemoteness::local" different from "DirRemoteness::remote", but we might
do so in future, so make sure we don't give a false positive here.
Non-Linux systems have ST_LOCAL or MNT_LOCAL, so no unknowns there.
See #10434
mmap() fails with ENODEV on remote file systems. This means we always fail
to read any old history on network file systems on Linux (except on the file
systems we recognize which are NFS, SMB and CIFS).
Untested, so I'm not sure if this works.
Fixes#10434
We no longer use RAII for enabling/disabling these, so a full object is
overkill. Additionally this object doesn't allow us to recover from the case
where we receive SIGTERM while inside terminal_protocols_{enable,disable}.
We can simply run disable another time since they're idempotent. Untested.
When I run a command with leading space, it is not added to the on-disk
history. However we still call History::save(). After 25 of such calls,
we rewrite the history file (even though nothing was written by us).
This is annoying when diagnosing #10300 where the history of the current
shell (but not other shells) is broken; because the history rewrite will
make the problem go away. Let's not save in this case, to make it easier to
run commands to inspect the state of the history file.
Given a history like
- cmd: echo OLD
when: 1726157160
\x00\x00\x00- cmd: echo leading NUL bytes
when: 1726157160
- cmd: echo NEW
when: 1726157223
offset_of_next_item() happily records 3 items even though the second item
is corrupted.
decode_item() fails which makes the caller stop loading any older items --
we got knee capped.
Avoid this horrible failure mode by skipping over these items already in
offset computation. For now we still lose the corrupted item itself.
In future we should probably try to delete the NUL bytes or avoid the
corruption in the first place.
See #10300 and others.
This should make the sort have a strict weak ordering, which rust
requires since 1.81 (or it will panic).
Note: This changes the order, but that's *fine* since the current
order is random weirdness anyway.
Fixes#10763
Commit b00899179 (Don't indent multi-line quoted strings; do indent inside
(), 2024-04-28) made parse_util_compute_indents() crash on `echo "$()"'x`.
After recursively indenting the command substitution, we indent the "'x
suffix. We skip the quoted part by setting "done=2". Later we wrongly
index "self.indents[done..range.start+offset+1]" (= "self.indents[2..1]").
Fix this by making sure that "start >= done", thus not setting any indents
for the quoted suffix. There is no need to do so; only the first character
in each line needs an indent.
In particular, this fixes the case
ctrl-r foo ctrl-r
where foo substring-matches no more than one page's worth of results.
The second attempt will fall back to subsequence matching which is wrong.
In case a terminal resize[1] causes us
to repaint a multi-line prompt that changes width like
function fish_prompt
for i in 1 2 3
random choice 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaa' 'bbbbbbbbbbb'
end
end
we add a clr_eol after each line[2] , to make sure
that a "b" line does not have leftover "a" letters
(80aaae5b7 (Clear to end of each line in left prompt, 2020-10-25)).
Unfortunately, if a prompt line takes up all the columns, clr_eol will
wrongly clear the last column. Reproduce with
function fish_prompt
string repeat $COLUMNS -
echo "$PWD> "
end
and observe that the last "-" is missing.
Previous (reverted) attempt d3ceba107 (Clear to eol before outputting line
in multi-line prompt, 2021-05-17) found the right fix but had an off-by-one
error which reintroduced the leftover "a" letters in the "random choice"
prompt above.
Given prompt string "aa\nbb\ncc", it wrongly printed
clr_eol "aa" clr_eol "\nbb" "\ncc"
Observe that the first line is cleared twice, while the second line is
never cleared. Fix that.
[1]: or an async "commandline -f repaint" triggered by a uvar change /
async prompt update
[2]: except after the last line where we probably already emit clr_eol
elsewhere..
Alternative fix: emit both clr_eol and clr_bol *before* drawing the current
line. However, if fish and the terminal disagree on character width, that
approach might erase too much.
Closes#8164
I guess it's nice to know that these two are the same but that info is not
needed here, it just adds confusion. The user must have pressed ctrl-j if
we get here, so echo that back.
See the parent commit.
This is just too confusing; \b sounds like it would map backspace but it's
actually just ctrl-h. Backspace is a different key ("bind backspace"),
so let's move away from \b.
Reproduce by typing ctrl-h in fish_key_reader, or, for even more confusion,
use a terminal like tmux and type ctrl-backspace which also sends ctrl-h.
I've thought about changing \b (and its aliases like \ch and \x08) to mean
backspace but that seems like unnecessary breakage, since they all already
mean ctrl-h, and can usually be mapped independent of backspace.
See the discussion in #10738
Make fish-printf no longer depend on the widestring crate, as other clients
won't use it; instead this is an optional feature.
Make format strings a generic type, so that both narrow and wide strings can
serve. This removes a lot of the complexity around converting from narrow to
wide.
Add a README.md to this crate.
This gave a weird error when you did e.g. `math Foo / 6`:
"Missing Operator" and only the "F" marked.
Adding an operator here anywhere won't help, so calling this an
"Unknown function" is closer to the truth. We also get nicer markings
because we know the extent of the identifier.
Fix 7308dbc7a (fish_indent: Prevent overwriting file with identical content,
2024-07-21) in a different way by passing O_TRUNC again.
If we don't want regressions we could use code review.
We used deque in C++ because this vector may be large, and so it avoids
repeated re-allocations. But VecDeque is different in Rust - it's contiguous -
so there's no benefit. Just use Vec.
For implementation reasons, we special-case cd in several ways
1. it gets different completions (handle_as_special_cd)
2. when highlighting, we honor CDPATH
3. we discard autosuggestions from history that don't have valid path arguments
There are some third-party tools like zoxide that redefine cd ("function cd
--wraps ...; ...; end"). We can't support this in general but let's try to
make an effort.
zoxide tries to be a superset of cd, so special case 1 is still
valid but 2 and 3 are not, because zoxide accepts some paths
that cd doesn't accept.
Let's add a hack to detect when "cd" actually means something else by checking
if there is any --wraps argument.
A cleaner solution is definitely possible but more effort.
Closes#10719
In some cases we add the wildcard twice.
$ fish -c '../jj; complete -C"ls cli/*/conf/tem"'
cli/*/*/config/templates.toml
Fix that. Test in the next commit.
There seems to be another bug in 3.7.1 where we fail to apply this completion
to the command line. This appears fixed. (FWIW we might want to revert
the quoting change in completion_apply_to_command_line(), maybe that one
accidentally fix this).
Fixes#10703
The HashMap is used to generate the __fish_describe_command integration
completions. Given the nature of the allocations and the numbers that we use, a
BTreeMap would theoretically perform better. Benchmarks show a 2-9%
improvement in completion times consistently in favor of BTreeMap.
Warnings were appearing under GCC 13.2
(void) alone is insufficient under modern compilers, workaround with logical
negation taken from GCC bug tracker.
Worth including because mold is rather popular in the rust world and because the
bug affects mold versions coincident with the development of the fish rust port.
The bug affects all currently released versions of mold from 2.30.0 (Mar 2024)
onwards under at least FreeBSD (though quite likely other platforms as well).
See https://github.com/rui314/mold/issues/1338 for reference.
The previous control flow logic wasn't sound and would leave the shell in a hung
state when `break` would be encountered.
The behavior is now straightforward, the shell reads until <Enter> or <q> is
pressed, at which point it aborts.
This was based on a misunderstanding.
On musl, 64-bit time_t on 32-bit architectures was introduced in version 1.2.0,
by introducing new symbols. The old symbols still exist, to allow programs compiled against older versions
to keep running on 1.2.0+, preserving ABI-compatibility. (see musl commit 38143339646a4ccce8afe298c34467767c899f51)
Programs compiled against 1.2.0+ will get the new symbols, and will therefore think time_t is 64-bit.
Unfortunately, rust's libc crate uses its own definition of these types, and does not check for musl version.
Currently, it includes the pre-1.2.0 32-bit type.
That means:
- If you run on a 32-bit system like i686
- ... and compile against a C-library other than libc
- ... and pass it a time_t-containing struct like timespec or stat
... you need to arrange for that library to be built against musl <1.2.0.
Or, as https://github.com/ericonr/rust-time64 says:
> Therefore, for "old" 32-bit targets (riscv32 is supposed to default to time64),
> any Rust code that interacts with C code built on musl after 1.2.0,
> using types based on time_t (arguably, the main ones are struct timespec and struct stat) in their interface,
> will be completely miscompiled.
However, while fish runs on i686 and compiles against pcre2, we do not pass pcre2 a time_t.
Our only uses of time_t are confined to interactions with libc, in which case with musl we would simply use the legacy ABI.
I have compiled an i686 fish against musl to confirm and can find no issue.
This reverts commit 55196ee2a0.
This reverts commit 4992f88966.
This reverts commit 46c8ba2c9f.
This reverts commit 3a9b4149da.
This reverts commit 5f9e9cbe74.
This reverts commit 338579b78c.
This reverts commit d19e5508d7.
This reverts commit b64045dc18.
Closes#10634
Add kqueue-based uvar notifier for BSD
Tested under FreeBSD 13.3.
This also works under all versions of macOS, and has some
benefits over the current notifyd choice.
Mutex is used because of the non-mut `notification_fd_became_readable()` `&self`
reference, but contention is not expected.
This restores a hack to trigger a command line repaint when "$fish_color_*" or
"$fish_pager_color_*" changes. These allow the command line to react immediately
to changes in other sessions or web_config.
This was removed in ff62d172e5 but there does not
appear to be a handler which actually redraws these.
Revert of ff62d172e5
The generated assembly is more or less the same and the previously generated
version had been manually verified, but this PR removes the usage of
`MaybeUninit::assume_init()` and replaces it with direct pointer writes.
This should result in no observable change: it continues to pass the functional
tests and benchmarks identically. The safety of the new code has been verified
with Miri.
[0]: https://github.com/mqudsi/fish-yaml-unescape-benchmark
This does not work as-is ("CSI a" is shift-up, not up).
I'm not sure if we want to implement these.
It's not a regression so there is no pressure.
This reverts commit 350598cb99.
byte_to_symbol was broken because it didn't iterate by byte, it
iterated by rust-char, which is a codepoint.
So it failed for everything outside of ascii and, because of a
mistaken bound, ascii chars from 0x21 to 0x2F ("!" to "/" - all the punctuation).
char_to_symbol will print printable codepoints as-is and
others escaped. This is okay - something like `decoded from: +` or
`decoded from: ö` is entirely understandable, there is no need to tell
you that "ö" is \xc3\xb6.
This reverts commit 423e5f6c03.
Array starts at 0, goes up to 27, that's 28 entries... *BUT* we also
need the catch-all entry after, so it's 29.
To be honest there's got to be a better way to write this.
WezTerm supports CSI u but unfortunately, typing single quote on a German
keyboard makes WezTerm send what gets decoded as `shift-'`.
This is bad, so disable it until this is fixed. In future we should maybe
add a runtime option to allow the user to override this decision.
See #10663
The \e\e\[A style is bad but iTerm and putty (alt-left) use it.
The main motivation for this change is to improve fish_key_reader output.
Part of #10663
As of rust 1.78, the Unix stdlib implementation is affected by the same issue:
pub fn sleep(dur: Duration) {
let mut secs = dur.as_secs();
let mut nsecs = dur.subsec_nanos() as _;
// If we're awoken with a signal then the return value will be -1 and
// nanosleep will fill in `ts` with the remaining time.
unsafe {
while secs > 0 || nsecs > 0 {
let mut ts = libc::timespec {
tv_sec: cmp::min(libc::time_t::MAX as u64, secs) as libc::time_t,
tv_nsec: nsecs,
};
secs -= ts.tv_sec as u64;
let ts_ptr = core::ptr::addr_of_mut!(ts);
if libc::nanosleep(ts_ptr, ts_ptr) == -1 {
assert_eq!(os::errno(), libc::EINTR);
secs += ts.tv_sec as u64;
nsecs = ts.tv_nsec;
} else {
nsecs = 0;
}
}
}
}
Note that there is a small behavior change here -- sleep() will continue
after signals; I'm not sure if we want that but it seems harmless?
Part of #10634
iTerm2 deviates from protocol, so back out c3c832761 (Stop using stack for
kitty progressive enhancement, 2024-08-03) in that case.
Note that we use several ways of detecting iTerm2 (ITERM_PROFILE,
TERM_PROGRAM=iTerm.app, ITERM_SESSION_ID).
LC_TERMINAL seems superior because it works over ssh.
This new one should hopefully go away eventually.
Today fish pushes/pops kitty progressive enhancements everytime control is
transfered to/from fish. This constitutes a regression relative to 3.7.1:
$ fish
$ ssh somehost fish
(network disconnect, now we missed our chance to pop from the stack)
$ bash # or some ncurses application etc
(keyboard shortcuts like ctrl-p are broken)
When invoking bash, we pop one entry off the stack but there is another one.
There seems to be a simple solution: don't use the stack but always reset
the current set of flags. Do that since I did not find a strong use case
for using the stack[1] (Note that it was recommended by terminal developers
to use the stack, so I might be wrong).
Note that there is still a regression if the outer shell is bash.
[1]: https://github.com/kovidgoyal/kitty/issues/7603#issuecomment-2256949384Closes#10603
According to the discussion in #2315, we adopt TTY modes for external commands
mainly for "stty". If our child process crashes (or SSH disconnect), we
might get weird modes. Let's ignore the modes in the failure case.
Co-authored-by: Johannes Altmanninger <aclopte@gmail.com>
Part of #10603
Run
printf \Xf6 | wl-copy # ö in ISO-8859-1
LANG=de_DE LC_ALL=$LANG gnome-terminal -- build/fish
and press ctrl-v. The pasted data looks like this:
$ set data (wl-paste -n 2>/dev/null | string collect -N)
$ set -S data
$data: set in local scope, unexported, with 1 elements
$data[1]: |\Xf6|
we pass $data directly to "commandline -i", which is supposed to insert it
into the commandline verbatim. What's actually inserted is "�".
This is because of all of:
1. We never decode "\Xf6 -> ö" in this scenario. Decoding it -- like we do
for non-pasted keyboard input -- would fix the issue.
2. We've switched to using Rust's char, which, for better or worse, disallows
code points that are not valid in Unicode (see b77d1d0e2 (Stop crashing
on invalid Unicode input, 2024-02-27)). This means that we don't simply
store \Xf6 as '\u{00f6}'. Instead we use our PUA encoding trick, making it
\u{f6f6} internally.
3. Finally, b77d1d0e2 renders reserved codepoints (which includes PUA chars)
using the replacement character � (sic). This was deemed more
user-friendly than printing an invalid character (which is probably not
mapped to a glyph). Yet it causes problems here: since we think that
\u{f6f6} is garbage, we try to render the replacement character. Apparently
that one is not defined(?) in ISO-8859-1; we get "�".
Fix this regression by removing the replacement character feature.
In future we should maybe decode pasted input instead. We could do that
lazily in "commandline -i", or eagerly in "set data (wl-paste ...)".
Commit 29f2da8d1 (Toggle terminal protocols lazily, 2024-05-16) made it so
the wildcard expansion in "echo **" (in a large directory tree) can't be
canceled with ctrl-c. Fix this by disabling terminal protocols already at
expansion time (not waiting until execution).
Using
SHELL=$(command -v fish) mc
Midnight Commander will spawn a fish child with
"function fish_prompt;"
"echo \"$PWD\">&%d; fish_prompt_mc; kill -STOP %%self; end\n",
So fish_prompt will SIGSTOP itself using an uncatchable signal.
On ctrl-o, mc will send SIGCONT to give back control to the shell.
Another ctrl-o will be intercepted by mc to put the shell back to sleep.
Since mc wants to intercept at least ctrl-o -- also while fish is in control
-- we can't use the CSI u encoding until mc either understands that, or uses
a different way of passing control between mc and fish.
Let's disable it for now.
Note that mc still uses %self but we've added a feature flag
to disable that. So if you use "set fish_features all"
you'll want to add a " no-remove-percent-self". A patch
to make mc use $fish_pid has been submitted upstream at
https://lists.midnight-commander.org/pipermail/mc-devel/2024-July/011226.html.
Closes#10640
When applying a wildcard, it's important to keep track of the files that have
been visited, to avoid symlink loops. Previously fish used a FileId for the
purpose. However FileId also includes richer information like modification time;
thus if a file is modified during wildcard expansion then fish may believe that
the file is different and visit it twice.
The richer information like modification time is important for atomic file
writes but should be ignored for wildcard expansion; just use the (dev, inode)
pair instead.
This also somewhat reduces our reliance on struct stat, but we still need it for
fstatat which Rust does not expose.
Returns 0 (true) in case an autosuggestion is currently being displayed.
This was first requested in #5000 then again in #10580 after the existing
workaround for this missing functionality was broken as part of a change to the
overall behavior of `commandline` (for the better).
Since we have a mix of both 0-based and 1-based line numbers in the code base,
we can now distinguish between them by type alone. Also stop using 0 as a
placeholder value for "no line number available" except in explicit helper
functions such as `get_lineno_for_display()`.
I've often needed a way to get the last bit of performance out of unwieldy
completions that involve a lot of string processing (apt completions come to
mind, and I ran into it just now with parsing man pages for kldload
completions).
Since many times we are looking for just one exact string in the haystack, an
easy optimization here is to introduce a way for `string match` or `string
replace` to early exit after a specific number of matches (typically one) have
been found.
Depending on the size of the input, this can be a huge boon. For example,
parsing the description from FreeBSD kernel module man pages with
zcat /usr/share/man/man4/zfs.4.gz | string match -m1 '.Nd *'
runs 35% faster with -m1 than without, while processing all files under
/usr/share/man/man4/*.4.gz in a loop (so a mix of files ranging from very short
to moderately long) runs about 10% faster overall with -m1.
Part of #1842
It's like jump-to-matching-bracket, but jumps right before the bracket
I will use it to mimic vi 'ab' and 'ib' text objects in the next commit
Given complicated semantics of jump-till-matching-bracket, an alternative name
could be 'jump-inside-matching-brackets'. But that would make names non-symmetrical.
I'm not sure what is worse.
Part of #1842
Split to:
- jump_and_remember_last_jump. What previously was called jump, now called
jump_and_remember_last_jump
- jump. Only jump, don't remember last jump. Now it's also possible to pass
vector of targets
The commit is pure refactoring, no functional changes are introduced.
The refactoring is needed for the next commits
Command completion descriptions were not being generated from `apropos`. Well,
they were being generated but that was not being correctly used by fish core.
Not sure when this was broken, but there's a possibility it was during the rust
port.
In addition to simply not working, it seems the old code tried to avoid
allocations but String::split_at_mut() allocates a new string (since one
allocation from the global allocator can't be split into two allocations to be
freed separately). Use `String::as_mut_utfstr()` before splitting the &wstr
instead of splitting the &str to actually do this alloc-free.
The "principal" parser is the one and only today; in the future we hope to
have multiple parsers to execute fish script in parallel.
Having a globally accessible "principle" parser is suspicious; now we can
get rid of it.
The "principal" environment stack was the one that was associated with the
"principal" parser and would dispatch changes like to TZ, etc.
This was always very suspicious, as a global; now we can remove it.
