This concerns how fish prevents its own fds from interfering with
user-defined fd redirections, like `echo hi >&5`. fish has historically
done this by tracking all user defined redirections when running a job,
and ensuring that pipes are not assigned the same fds. However this is
annoying to pass around - it means that we have to thread user-defined
redirections into pipe creation.
Take a page from zsh and just ensure that all pipes we create have fds in
the "high range," which here means at least 10. The primary way to do this
is via the F_DUPFD_CLOEXEC syscall, which also sets CLOEXEC, so we aren't
invoking additional syscalls in the common case. This will free us from
having to track which fds are in user-defined redirections.
fds.h will centralize logic around working with file descriptors. In
particular it will be the new home for logic around moving fds to high
unused values, replacing the "avoid conflicts" logic.
This needs to be rewritten, I'm pretty sure we have like 6 of these
kinds of ad-hoc "is this quoted" things lying around.
But for now, at least don't just check if the *previous* character was
a backslash.
Fixes#7685.
Previously we sometimes wanted to access an io_buffer_t to append to it
directly, but that's no longer true; all we really care about is its
separated_buffer_t. Make io_bufferfill_t::finish return the
separated_buffer directly, simplifying call sites. No user visible changes
expected here.
This concerns builtins writing to an io_buffer_t. io_buffer_t is how fish
captures output, especially in command substitutions:
set STUFF (string upper stuff)
Recall that io_buffer_t fills itself by reading from an fd (typically
connected to stdout of the command). However if our command is a builtin,
then we can write to the buffer directly.
Prior to this change, when a builtin anticipated writing to an
io_buffer_t, it would first write into an internal buffer, and then after
the builtin was finished, we would copy it to the io_buffer_t. This was
because we didn't have a polymorphic receiver for builtin output: we
always buffered it and then directed it to the io_buffer_t or file
descriptor or stdout or whatever.
Now that we have polymorphpic io_streams_t, we can notice ahead of time
that the builtin output is destined for an internal buffer and have it
just write directly to that buffer. This saves a buffering step, which is
a nice simplification.
While the user waits at the prompt, fish is waiting in select(), on stdin.
The sigio based universal notifier interrupts select() by arranging for a
signal to be delivered, which causes select() to return with EINTR.
However we weren't polling the notifier at that point so we would not
notice uvar changes, until we got some real input.
I didn't notice this when testing, because my testing was changing fish
prompt colors which updated the prompt for other reasons.
Fixes#7671.
Now that we have multiple clients of count_preceding_backslashes, factor
it out from fish_indent into wcstringutil.h, and then use the shared
implementation.
This goes to a separate file because that makes option parsing easier
and allows profiling both at the same time.
The "normal" profile now contains only the profile data of the actual
run, which is much more useful - you can now profile a function by
running
fish -C 'source /path/to/thing' --profile /tmp/thefunction.prof -c 'thefunction'
and won't need to filter out extraneous information.
Expansion parses slices like "$PATH[1..2]", but so does "set" when assigning
"set PATH[1..2] . .". Commit be06f842a ("Allow to omit indices in index
range expansions") forgot the latter.
Just like OPOST this just breaks output for anything not prepared for
it. Fish itself might work with it (and #4505 recommends it), but external commands are broken.
You'll see output like
foo
⏎
from `echo foo`.
Fixes#4873.
Continuation of #7133.
If given a windows path like `F:\foo`, this currently ends up
assert()ing in path_normalize_for_cd.
Instead, since these paths violate a bunch of assumptions we make, we
reject them and fall back on getting $PWD via getcwd() (which should
give us a nice proper unixy path).
Fixes#7636.
This isn't tested because it would require a system where a windowsy
path passes paths_are_same_file, and on the unix systems we run our
tests that's impossible as far as I can tell?
This used to print a literal DEL character in the output for `bind`,
which wouldn't actually show up and made it hard to figure out what
the key was.
So we just escape it back to how we actually used it - `\x7f`.
Fixes#7631.
This sometimes fails on github actions with ASAN. I am assuming that's
because the ctrl-c happens *before* the process has had a chance to
start.
So we do what we do and increase the delay.
These are a foreground and a background color. Now I see the point in
not naming them "foreground_color" and "background_color", but at
least "fg" and "bg" should do, right?
Prior to this change, histories were immortal and allocated with either
unique_ptr or just leaked via new. But this can result in races in the
path detection test, as the destructor races with the pointer-captured
history. Switch to using shared_ptr.
A weird interaction between grouped short options and our weird option
parsing that puts unknown options back:
```
echo "-n foo"
```
would see the `-n`, turn off printing newlines, interpret the " " as
another grouped short option, see that there is no short option for
space and put the entire token back on the arguments pile.
So it would print "-n foo" *without a newline*.
Fix this by keeping an old state of the options around and reverting
it when putting options back.
The alternative is *probably* to forbid the " " short option in
wgetopt, then check if an option group contains it and error out, but
this should only really be a problem in `echo` because that is,
AFAICT, the only thing that puts the options back.
Fixes#7614
When adding a command to history, we first expand its arguments to see
if any arguments are paths which refer to files. If so, we will only
autosuggest that command from history if the files are still valid. For
example, if the user runs `rm ./file.txt` then we will remember that
`./file.txt` referred to a file, and then only autosuggest that if the file
is present again.
Prior to this change we only performed simple expansion relative to the
working directory. This change extends it to variables and tilde
expansion. For example we will now apply the same hinting for
`rm ~/file.txt`
Fixes#7582
This removes the 100 msec timeout from io_buffer_t. We no longer need to
periodically wake up to check if a command substitution is finished,
because we get explicitly poked when that happens.
io_buffer_t is used to buffer output from a command substitution, so we
can split it into arguments. Typically io_buffer_t reads from its pipe
until it gets EOF and then stops reading. However it may be that the
cmdsub ends but EOF is not delivered because the stdout of the cmdsub
escaped with a background process.
Prior to this change we would wake up every 100 msec (select timeout) to
check if the cmdsub is finished. However this 100 msec adds latency if a
background process is launched from e.g. fish_prompt.
Switch to the new poke() function. Now when the cmdsub is finished, it
pokes its item, which explicitly wakes it up. This removes the extra
latency.
Fixes#7559
In preparation for fixing #7559, add a function poke_item to fd_monitor.
fd_monitor has a list of file descriptors, and invokes a callback when an
fd becomes readable. With this change, we assign each item a unique ID and
return it when the item is added; the ID may then be used to invoke the
callback explicitly.
The idea is that we can stop reading from the pipe associated with the
cmdsub when the job is finished, even if the pipe is still open.
This allows for multiple edits to be undone/redone in one go, as if they
were one edit.
Useful when a function is editing the commandline buffer via scripted
changes or via a keybinding so the internal changes to the buffer can be
abstracted away.
(Having extreme difficulty getting pexpect to play nice with the concept
of undo/redo...)
Currently binding `exit` to a key checks too late that it's exitted,
so it leaves the shell hanging around until the user does an execute
or similar.
As I understand it, the `exit` builtin is supposed to only exit the
current "thread" (once that actually becomes a thing), and the
bindings would probably run in a dedicated one, so the simplest
solution here is to just add an `exit` bind function.
Fixes#7604.
Prior to this change, `fish_private_mode` worked by just suppressing
history outright. With this change, `fish_private_mode` can be toggled on
and off. Commands entered while `fish_private_mode` is set are stored but
in memory only; they are not written to disk.
Fixes#7590Fixes#7589
Commands that start with a space should not be written to the history
file. Prior to this change, that was implemented by simply not adding them
to history. Items with leading spaces were simply dropped.
With this change, we add a 'history_persistence_mode_t' to
history_item_t, which tracks how the item persists. Items with leading
spaces are now marked as "ephemeral": they can be recovered via up arrow,
until the user runs another command, or types a space and hits return.
This matches zsh's HIST_IGNORE_SPACE feature.
Fixes#1383
Don't go into implicit interactive mode without ever executing
anything - not even `exit` or reacting to ctrl-d. That just renders
the shell useless and unquittable.
It was always a bit ridiculous that argparse required `X-longflag` if
that "X" short flag was never actually used anywhere.
Since the short letter is for getopt's benefit, we can hack around
this with our old friend: Unicode Private Use Areas.
We have a counter, starting at 0xE000 and going to 0xF8FF, that counts
up for all options that don't have a short flag and provides one. This
gives us up to 6400 long-only options.
6.4K should be enough for everybody.
A mildly interesting one is the call to test_wchar2utf8 with a non-null
pointer ("u1"/"dst") but 0 length. In this case we relied on malloc(0)
returning non-null which is not guaranteed.
src/fish_tests.cpp:1619:23: warning: Call to 'malloc' has an allocation
size of 0 bytes [clang-analyzer-optin.portability.UnixAPI]
mem = (char *)malloc(dlen);
^
test_wchar2utf8(w1, sizeof(w1) / sizeof(*w1), u1, 0, 0, 0,
"invalid params, dst is not NULL");
Prior to this change, a glob like `**/file.txt` would only match
`file.txt` in subdirectories; the `**` must match at least one directory.
This is historical behavior.
With this change we move a little closer to bash's implementation by
allowing a literal `**` segment to match in the current directory. That
is, `**/foo` will match both `foo` and `bar/foo`, while `b**/foo` will
only match `bar/foo`.
Fixes#7222.
Before running a command, or before importing a command from bash history,
we perform error checking. As part of error checking we expand commands
including variables and globs. If the glob is very large, like `/**`, then
we could hang expanding it.
One fix would be to limit the amount of expansion from the glob, but
instead let's just not expand command globs when performing error checking.
Fixes#7407
If the user types something like `/**`, prior to this change we would
attempt to expand it in the background for both highlighting and
autosuggestions. This could thrash your disk and also consume a lot of
memory.
Add a a field to operation_context_t to allow specifying a limit, and add
a "default background" limit of 512 items.
Historically fish has not supported tab completing or autosuggesting
wildcards with **. Prior to this fix, we would test every file match,
discover the ** wildcard, and then ignore it. Instead look for **
wildcards at the top level.
This prevents autosuggesting with /** from chewing up your disk.
When a completion's "--arguments" script ran, it would clobber $status with its value,
so when you repainted your prompt, it would now show the completion
script's status rather than the status of what you last ran.
Solve this by just storing the status and restoring it - other places
do this by calling exec_subshell with apply_exit_status set to false,
which does basically the same thing. We can't use it here because we
don't want to run a "full" script, we only want the arguments to be
expanded, without a "real" command.
No, I have no idea how to test this automatically.
Fixes#7555.
This has one functional difference, in that we now report non-EACCESS
errors even for relative paths. I consider that to be a plus.
Some other sites might benefit from this, let's look into that later.
1. This should be using our wcstod_l on platforms where we need
it (for some reason it wasn't picking it up on FreeBSD?)
2. This purports to have a "fast path". I like fast paths.
Locale-wise, we're only interested in one thing:
"." is the radix character when interpreting numbers
And for that it's enough to just use our c-locale, like elsewhere.
This saves a bunch of switching locale back and forth, and simplifies
the code.
Prior to this change, the functions in exec.cpp would return true or false
and it was not clear what significance that value had.
Switch to an enum to make this more explicit. In particular we have the
idea of a "pipeline breaking" error which should us to skip processes
which have not yet launched; if no process launches then we can bail out
to a different path which avoids reaping processes.
This would tell you a function was "Defined in - @ line 1" for every
function defined via `source`.
Really, ideally we'd figure out where the *source* call was, but that'
much more complicated, so we just give a comprehensible message.
This matches what we do in --profile's output:
```
> source /home/alfa/.config/fish/config.fish
--> set -gx XDG_CACHE_HOME /home/alfa/.cache
--> set -gx XDG_CONFIG_HOME /home/alfa/.config
--> set -gx XDG_DATA_HOME /home/alfa/.local/share
```
instead of
```
+ source /home/alfa/.config/fish/config.fish
+++ set -gx XDG_CACHE_HOME /home/alfa/.cache
+++ set -gx XDG_CONFIG_HOME /home/alfa/.config
+++ set -gx XDG_DATA_HOME /home/alfa/.local/share
```
Prior to this change, if fish were launched connected to a tty but not as
pgroup leader, it would attempt to become pgroup leader only if
--interactive is explicitly passed. But bash will unconditionally attempt
to become pgroup leader if launched interactively. This can result in
scenarios where fish is running interactively but in another pgroup. The
most obvious impact is that control-C will result in the pgroup leader
(not fish) exiting and make fish orphaned.
Switch to matching the bash behavior here - we will always try to become
pgroup leader if interactive.
Fixes#7060.
And again clang-format does something I don't like:
- if (found != end && std::strncmp(found->name, name, len) == 0 && found->name[len] == 0) return found;
+ if (found != end && std::strncmp(found->name, name, len) == 0 && found->name[len] == 0)
+ return found;
I *know* this is a bit of a long line. I would still quite like having
no brace-less multi-line if *ever*. Either put the body on the same
line, or add braces.
Blergh
When globbing, we have a base directory (typically $PWD) and a path
component relative to that. As PWD is "virtual" it may be a symlink. Prior
to this change we would use wrealpath to resolve symlinks before opening
the directory during a glob, but this call to wrealpath consumed roughly
half of the time during globbing, and is conceptually unnecessary as
opendir will resolve symlinks for us.
Remove it. This may have funny effects if the user's PWD is an unlinked
directory, but it roughly doubles the speed of a glob like `echo ~/**`.
This adds the ability to limit how many expansions are produced. For
example if $big contains 10 items, and is Cartesian-expanded as
$big$big$big$big... 10 times, we would naviely get 10^10 = 10 billion
results, which fish can't actually handle. Implement this in
completion_receiver_t, which now can return false to indicate an overflow.
The initial expansion limit 'k_default_expansion_limit' is set as 512k
items. There's no way for users to change this at present.
This switches certain uses from just appending to a list to using
completion_receiver_t, in preparation for limiting how many completions
may be produced. Perhaps in time this could also be used for "streaming"
completions.
completion_receiver_t wraps a completion list; it will centralize logic
around adding completions and most importantly it will enforce that we
do not exceed our expansion limit.
The pager cleanup missed that the existing token could already include active (as in unescaped) expansions, and just escaped them all.
This means things like `ls ~/<TAB>` would escape the `~`, which is obviously wrong and makes it awkward to use.
This reverts commit b38a23a46d.
I fully expect that we'll try again, but there's no use in keeping master broken while that happens.
Fixes#7526.
E.g. if we do `string match -q`, and we find a match, nothing about
the input can change anything, so we quit early.
This is mainly useful for performance, but it also allows `string`
with `-q` to be used with infinite input (e.g. `yes`).
Alternative to #7495.
warn_unused_result is the persistent one that won't go away with a
simple `(void)write(...)` and needs to be assigned to a variable (that
must then also be declared unused or else you'll get a warning about
_that_).
"smartcase" performs case-insensitive matching if the input string is all
lowercase, and case-sensitive matching otherwise. When completing e.g.
files, we will now show both case sensitive and insensitive completions if
the input string does not contain uppercase characters.
This is a delicate fix in an interactive component with low test coverage.
It's likely something will regress here.
Fixes#3978
This is an attempt to simplfy some completion logic. It mainly refactors
reader_data_t::handle_completions such that all completions have the token
prepended; this attempts to simplify the logic since now all completions
replace the token. It also changes how the pager prefix works. Previously
the pager prefix was an extra string that was prepended to all
completions. In the new model the completions already have the prefix
prepended and the prefix is used only for certain width calculations.
This is a somewhat frightening change in an interactive component with
low test coverage. It tweaks things like how long completions are
ellipsized. Buckle in!
In preparation for introducing "smart case", refactor string fuzzy
matching. Specifically split out the case folding and match type into
separate fields, so that we can introduce more case folding types without
a combinatoric explosion.
This is used to decide which fuzzy match is better, however it is used
only in wildcard expansion and not in actual completion ranking or
anywhere else where it could matter. Try removing the compare() call
and implementation.
What compare() did specially was compare distances, e.g. it ranks
lib as better than libexec when expanding /u/l/b. But the tests did not
exercise this so it's hard to know if it's working. In preparation for a
refactoring, remove it.
When fish presents an autosuggestion, there is some logic around whether
to retain it or discard it as the user types "into" it. Prior to this
change, we would retain the autosuggestion if the user's input text is a
case-insensitive prefix of the autosuggestion. This is reasonable for
certain case-insensitive autosuggestions like files, but it is confusing
especially for history items, e.g. `git branch -d ...` and `git branch -D
...` should not be considered a match.
With this change, when we compute the autosuggestion we record whether it
is "icase", and that controls whether the autosuggestion permits a
case-insensitive extension.
This addresses part of #3978.
The code to override the `(status current-command) was present`, but not
handled in either the default `fish_title` function or the fallback.
Closes#7444.
Currently a bit limited, unfortunately printf's `%a` specifier is
absolutely unreadable.
So we add `hex` and `octal` with `0x` and `0` prefixes respectively,
and also take a number but currently only allow 16 and 8.
The output is truncated to integer, so scale values other than 0 are
invalid and 0 is implied.
The docs mention this may change.
Prior to this change, we would run fish_prompt and then afterwards set
the shell modes. For users with an initially slow prompt, this would
mean that characters would be echoed to the tty until after the prompt
completes.
Reorder these so that we set the tty mode first. This implies we will
run the prompt in shell mode, but this was already the case up until
2a3677b386.
Fixes#7489. Note that the prior commit e0cedd4ad2 is also necessary
here, as that fixed an extra prompt execution.
The new commandline switch `string match --regex --import` will import
as fish variables any named capture groups with the matched captures as
the value(s).
Previously this parameter was used to more-eagerly restore the terminal
mode. This was the basis for #2214. However now we restore the mode
from the reader instead, so we can remove this unused parameter.
Prior to this fix, when key binding is a script command (i.e. not a
readline command), fish would run that key binding using fish's shell
tty modes. Switch to using the external tty modes. This re-fixes
issue #2214.
Prior to this change, when a process resumes because it is brought back
to the foreground, we would reset the terminal attributes to shell mode.
This fixed#2114 but subtly introduced #7483.
This backs out 9fd9f70346, re-introducing #2114 and re-fixing #7483.
A followup fix will re-fix #2114; these are broken out separately for
bisecting purposes.
Fixes#7483.
Prior to this change, for bindings which have script commands, the
inputter would execute them directly. However an upcoming fix for #7483
will require more integration with the reader. Switch to a new model where
the reader passes in a function to use for executing script commands.
