io_buffer_t is used to buffer output from a command substitution, so we
can split it into arguments. Typically io_buffer_t reads from its pipe
until it gets EOF and then stops reading. However it may be that the
cmdsub ends but EOF is not delivered because the stdout of the cmdsub
escaped with a background process.
Prior to this change we would wake up every 100 msec (select timeout) to
check if the cmdsub is finished. However this 100 msec adds latency if a
background process is launched from e.g. fish_prompt.
Switch to the new poke() function. Now when the cmdsub is finished, it
pokes its item, which explicitly wakes it up. This removes the extra
latency.
Fixes#7559
In preparation for fixing #7559, add a function poke_item to fd_monitor.
fd_monitor has a list of file descriptors, and invokes a callback when an
fd becomes readable. With this change, we assign each item a unique ID and
return it when the item is added; the ID may then be used to invoke the
callback explicitly.
The idea is that we can stop reading from the pipe associated with the
cmdsub when the job is finished, even if the pipe is still open.
This allows for multiple edits to be undone/redone in one go, as if they
were one edit.
Useful when a function is editing the commandline buffer via scripted
changes or via a keybinding so the internal changes to the buffer can be
abstracted away.
(Having extreme difficulty getting pexpect to play nice with the concept
of undo/redo...)
Currently binding `exit` to a key checks too late that it's exitted,
so it leaves the shell hanging around until the user does an execute
or similar.
As I understand it, the `exit` builtin is supposed to only exit the
current "thread" (once that actually becomes a thing), and the
bindings would probably run in a dedicated one, so the simplest
solution here is to just add an `exit` bind function.
Fixes#7604.
Prior to this change, `fish_private_mode` worked by just suppressing
history outright. With this change, `fish_private_mode` can be toggled on
and off. Commands entered while `fish_private_mode` is set are stored but
in memory only; they are not written to disk.
Fixes#7590Fixes#7589
Commands that start with a space should not be written to the history
file. Prior to this change, that was implemented by simply not adding them
to history. Items with leading spaces were simply dropped.
With this change, we add a 'history_persistence_mode_t' to
history_item_t, which tracks how the item persists. Items with leading
spaces are now marked as "ephemeral": they can be recovered via up arrow,
until the user runs another command, or types a space and hits return.
This matches zsh's HIST_IGNORE_SPACE feature.
Fixes#1383
Don't go into implicit interactive mode without ever executing
anything - not even `exit` or reacting to ctrl-d. That just renders
the shell useless and unquittable.
It was always a bit ridiculous that argparse required `X-longflag` if
that "X" short flag was never actually used anywhere.
Since the short letter is for getopt's benefit, we can hack around
this with our old friend: Unicode Private Use Areas.
We have a counter, starting at 0xE000 and going to 0xF8FF, that counts
up for all options that don't have a short flag and provides one. This
gives us up to 6400 long-only options.
6.4K should be enough for everybody.
A mildly interesting one is the call to test_wchar2utf8 with a non-null
pointer ("u1"/"dst") but 0 length. In this case we relied on malloc(0)
returning non-null which is not guaranteed.
src/fish_tests.cpp:1619:23: warning: Call to 'malloc' has an allocation
size of 0 bytes [clang-analyzer-optin.portability.UnixAPI]
mem = (char *)malloc(dlen);
^
test_wchar2utf8(w1, sizeof(w1) / sizeof(*w1), u1, 0, 0, 0,
"invalid params, dst is not NULL");
Prior to this change, a glob like `**/file.txt` would only match
`file.txt` in subdirectories; the `**` must match at least one directory.
This is historical behavior.
With this change we move a little closer to bash's implementation by
allowing a literal `**` segment to match in the current directory. That
is, `**/foo` will match both `foo` and `bar/foo`, while `b**/foo` will
only match `bar/foo`.
Fixes#7222.
Before running a command, or before importing a command from bash history,
we perform error checking. As part of error checking we expand commands
including variables and globs. If the glob is very large, like `/**`, then
we could hang expanding it.
One fix would be to limit the amount of expansion from the glob, but
instead let's just not expand command globs when performing error checking.
Fixes#7407
If the user types something like `/**`, prior to this change we would
attempt to expand it in the background for both highlighting and
autosuggestions. This could thrash your disk and also consume a lot of
memory.
Add a a field to operation_context_t to allow specifying a limit, and add
a "default background" limit of 512 items.
Historically fish has not supported tab completing or autosuggesting
wildcards with **. Prior to this fix, we would test every file match,
discover the ** wildcard, and then ignore it. Instead look for **
wildcards at the top level.
This prevents autosuggesting with /** from chewing up your disk.
When a completion's "--arguments" script ran, it would clobber $status with its value,
so when you repainted your prompt, it would now show the completion
script's status rather than the status of what you last ran.
Solve this by just storing the status and restoring it - other places
do this by calling exec_subshell with apply_exit_status set to false,
which does basically the same thing. We can't use it here because we
don't want to run a "full" script, we only want the arguments to be
expanded, without a "real" command.
No, I have no idea how to test this automatically.
Fixes#7555.
This has one functional difference, in that we now report non-EACCESS
errors even for relative paths. I consider that to be a plus.
Some other sites might benefit from this, let's look into that later.
1. This should be using our wcstod_l on platforms where we need
it (for some reason it wasn't picking it up on FreeBSD?)
2. This purports to have a "fast path". I like fast paths.
Locale-wise, we're only interested in one thing:
"." is the radix character when interpreting numbers
And for that it's enough to just use our c-locale, like elsewhere.
This saves a bunch of switching locale back and forth, and simplifies
the code.
Prior to this change, the functions in exec.cpp would return true or false
and it was not clear what significance that value had.
Switch to an enum to make this more explicit. In particular we have the
idea of a "pipeline breaking" error which should us to skip processes
which have not yet launched; if no process launches then we can bail out
to a different path which avoids reaping processes.
This would tell you a function was "Defined in - @ line 1" for every
function defined via `source`.
Really, ideally we'd figure out where the *source* call was, but that'
much more complicated, so we just give a comprehensible message.
This matches what we do in --profile's output:
```
> source /home/alfa/.config/fish/config.fish
--> set -gx XDG_CACHE_HOME /home/alfa/.cache
--> set -gx XDG_CONFIG_HOME /home/alfa/.config
--> set -gx XDG_DATA_HOME /home/alfa/.local/share
```
instead of
```
+ source /home/alfa/.config/fish/config.fish
+++ set -gx XDG_CACHE_HOME /home/alfa/.cache
+++ set -gx XDG_CONFIG_HOME /home/alfa/.config
+++ set -gx XDG_DATA_HOME /home/alfa/.local/share
```
Prior to this change, if fish were launched connected to a tty but not as
pgroup leader, it would attempt to become pgroup leader only if
--interactive is explicitly passed. But bash will unconditionally attempt
to become pgroup leader if launched interactively. This can result in
scenarios where fish is running interactively but in another pgroup. The
most obvious impact is that control-C will result in the pgroup leader
(not fish) exiting and make fish orphaned.
Switch to matching the bash behavior here - we will always try to become
pgroup leader if interactive.
Fixes#7060.
And again clang-format does something I don't like:
- if (found != end && std::strncmp(found->name, name, len) == 0 && found->name[len] == 0) return found;
+ if (found != end && std::strncmp(found->name, name, len) == 0 && found->name[len] == 0)
+ return found;
I *know* this is a bit of a long line. I would still quite like having
no brace-less multi-line if *ever*. Either put the body on the same
line, or add braces.
Blergh
When globbing, we have a base directory (typically $PWD) and a path
component relative to that. As PWD is "virtual" it may be a symlink. Prior
to this change we would use wrealpath to resolve symlinks before opening
the directory during a glob, but this call to wrealpath consumed roughly
half of the time during globbing, and is conceptually unnecessary as
opendir will resolve symlinks for us.
Remove it. This may have funny effects if the user's PWD is an unlinked
directory, but it roughly doubles the speed of a glob like `echo ~/**`.
This adds the ability to limit how many expansions are produced. For
example if $big contains 10 items, and is Cartesian-expanded as
$big$big$big$big... 10 times, we would naviely get 10^10 = 10 billion
results, which fish can't actually handle. Implement this in
completion_receiver_t, which now can return false to indicate an overflow.
The initial expansion limit 'k_default_expansion_limit' is set as 512k
items. There's no way for users to change this at present.
This switches certain uses from just appending to a list to using
completion_receiver_t, in preparation for limiting how many completions
may be produced. Perhaps in time this could also be used for "streaming"
completions.
completion_receiver_t wraps a completion list; it will centralize logic
around adding completions and most importantly it will enforce that we
do not exceed our expansion limit.
The pager cleanup missed that the existing token could already include active (as in unescaped) expansions, and just escaped them all.
This means things like `ls ~/<TAB>` would escape the `~`, which is obviously wrong and makes it awkward to use.
This reverts commit b38a23a46d.
I fully expect that we'll try again, but there's no use in keeping master broken while that happens.
Fixes#7526.
E.g. if we do `string match -q`, and we find a match, nothing about
the input can change anything, so we quit early.
This is mainly useful for performance, but it also allows `string`
with `-q` to be used with infinite input (e.g. `yes`).
Alternative to #7495.
warn_unused_result is the persistent one that won't go away with a
simple `(void)write(...)` and needs to be assigned to a variable (that
must then also be declared unused or else you'll get a warning about
_that_).
"smartcase" performs case-insensitive matching if the input string is all
lowercase, and case-sensitive matching otherwise. When completing e.g.
files, we will now show both case sensitive and insensitive completions if
the input string does not contain uppercase characters.
This is a delicate fix in an interactive component with low test coverage.
It's likely something will regress here.
Fixes#3978
This is an attempt to simplfy some completion logic. It mainly refactors
reader_data_t::handle_completions such that all completions have the token
prepended; this attempts to simplify the logic since now all completions
replace the token. It also changes how the pager prefix works. Previously
the pager prefix was an extra string that was prepended to all
completions. In the new model the completions already have the prefix
prepended and the prefix is used only for certain width calculations.
This is a somewhat frightening change in an interactive component with
low test coverage. It tweaks things like how long completions are
ellipsized. Buckle in!
In preparation for introducing "smart case", refactor string fuzzy
matching. Specifically split out the case folding and match type into
separate fields, so that we can introduce more case folding types without
a combinatoric explosion.
This is used to decide which fuzzy match is better, however it is used
only in wildcard expansion and not in actual completion ranking or
anywhere else where it could matter. Try removing the compare() call
and implementation.
What compare() did specially was compare distances, e.g. it ranks
lib as better than libexec when expanding /u/l/b. But the tests did not
exercise this so it's hard to know if it's working. In preparation for a
refactoring, remove it.
When fish presents an autosuggestion, there is some logic around whether
to retain it or discard it as the user types "into" it. Prior to this
change, we would retain the autosuggestion if the user's input text is a
case-insensitive prefix of the autosuggestion. This is reasonable for
certain case-insensitive autosuggestions like files, but it is confusing
especially for history items, e.g. `git branch -d ...` and `git branch -D
...` should not be considered a match.
With this change, when we compute the autosuggestion we record whether it
is "icase", and that controls whether the autosuggestion permits a
case-insensitive extension.
This addresses part of #3978.
The code to override the `(status current-command) was present`, but not
handled in either the default `fish_title` function or the fallback.
Closes#7444.
Currently a bit limited, unfortunately printf's `%a` specifier is
absolutely unreadable.
So we add `hex` and `octal` with `0x` and `0` prefixes respectively,
and also take a number but currently only allow 16 and 8.
The output is truncated to integer, so scale values other than 0 are
invalid and 0 is implied.
The docs mention this may change.
Prior to this change, we would run fish_prompt and then afterwards set
the shell modes. For users with an initially slow prompt, this would
mean that characters would be echoed to the tty until after the prompt
completes.
Reorder these so that we set the tty mode first. This implies we will
run the prompt in shell mode, but this was already the case up until
2a3677b386.
Fixes#7489. Note that the prior commit e0cedd4ad2 is also necessary
here, as that fixed an extra prompt execution.
The new commandline switch `string match --regex --import` will import
as fish variables any named capture groups with the matched captures as
the value(s).
Previously this parameter was used to more-eagerly restore the terminal
mode. This was the basis for #2214. However now we restore the mode
from the reader instead, so we can remove this unused parameter.
Prior to this fix, when key binding is a script command (i.e. not a
readline command), fish would run that key binding using fish's shell
tty modes. Switch to using the external tty modes. This re-fixes
issue #2214.
Prior to this change, when a process resumes because it is brought back
to the foreground, we would reset the terminal attributes to shell mode.
This fixed#2114 but subtly introduced #7483.
This backs out 9fd9f70346, re-introducing #2114 and re-fixing #7483.
A followup fix will re-fix #2114; these are broken out separately for
bisecting purposes.
Fixes#7483.
Prior to this change, for bindings which have script commands, the
inputter would execute them directly. However an upcoming fix for #7483
will require more integration with the reader. Switch to a new model where
the reader passes in a function to use for executing script commands.
The string "%ls is %ls", which is printed when `type <command>` is ran
for a command in PATH, couldn't be localized, since it was missing _()
around it.
It may happen that the user types an abbreviation and then hits return.
Prior to this commit, we would perform a form of syntax highlighting
that does not require I/O, so as to not block the user. However this
could cause invalid commands to be colored as valid.
More generally if the user has e.g a slow NFS mount, then syntax
highlighting may lag behind the user's typing, and be incorrect at the
time the user hits return. This is an unavoidable race, since proper
syntax highlighting may take arbitrarily long.
Introduce a new function `finish_highlighting_before_exec`, which waits
for any outstanding syntax highlighting to complete, BUT has a timeout
(250 milliseconds). After this, it falls back to the no-I/O variant, which
colors all commands as valid and nothing as paths.
Fixes#7418Fixes#5912
In some cases the completion we come up with may be unexpected, e.g.
if you have files like
/etc/realfile
and
/etc/wrongfile
and enter "/etc/gile", it will accept "wrongfile" because "g" and
"ile" are in there - it's a substring insertion match.
The underlying cause was a typo, so it should be easy to go back.
So we do a bit of magic and let "cancel" undo, but only right after a
completion was accepted via complete or complete-and-search.
That means that just reflexively pressing escape would, by default, get you back to
the old token and let you fix your mistake.
We don't do this when the completion was accepted via the pager,
because 1. there's more of a chance to see the problem there and 2.
it's harder to redo in that case.
Fixes#7433.
This was typically overridden by "too many/few arguments", but it's
actually incorrect:
sin(55
has the correct number of arguments to `sin`, but it's lacking
the closing `)`.
fish_user_paths is a fish-specific variable that can be persisted by
making it a universal variable or by making it a global variable set at
startup in `config.fish`.
Since it is not defined in a clean installation, a user could
inadvertently create it as `set -Ux fish_user_paths ....` the first
time, creating a horrible, ugly, self-loathing mess that will have you
chasing ghosts and bisecting for naught once fish re-imports
fish_user_paths as a *global* variable that shadows the universal one.
While that is true for any universal variable that is re-imported as a
global variable, only fish_user_paths has the potential to really screw
things up because we also re-export PATH based off of its value in turn.
This fixes up the SIGIO notifier in preparation for using it on BSD. It
removes the reliance on the signal's si_code, which is not available in
BSD, and it properly handles the BSD behavior where SIGIO is delivered on
a read even if the read returns EAGAIN.
Fix an error caused by `exec_job()` assuming a job launched with the
intention of being backgrounded would have a pgid assigned in all cases,
without considering the status of `exec_error` which could have resulted
in the job failing before it was launched into its own process group.
Fixes (but doesn't close) #7423 - that can be closed if this assertion
failure doesn't happen in any released fish versions.
It is apparently possible to launch fish such that its pid owns the tty,
but its pid is in a different pgroup. In that case, do not attempt to stop
with SIGTTIN; instead simply attempt to place fish in its own pgroup.
Fixes#7388
`complete_param_expand` knows how to handle cases like `foo=br` so we
don't need to bother sending just the `br` part. Furthermore, sending
just `br` is incorrect because we will end up replacing the entirety of
`foo=br` with the result of the completion. That is, `foo=br` will be
replaced with `bar` instead of being completed to `foo=bar`.
This switch is no longer necessary when only one command is given.
Internally completions are stored separately for each command,
so we only every print one command name per "complete" line anyway.
On WSL1, fcntl(F_SETOWN) will fail and this would report an error.
Suppress this error message since it is not very interesting.
The effect is to disable real-time universal variable propagation.
Introduce a new strategy for notifying other fish processes of universal
variable changes, as a planned replacement for the complex
strategy_named_pipe. The new strategy still uses a named pipe, but instead
of select() on it, it arranges for SIGIO to be delivered when data is
available. If a SIGIO has been seen since the last check, it means the file
needs to be re-read.
When expanding a string, you may or may not want to generate
descriptions alongside the expanded string. Usually you don't want to
but descriptions were opt out. This commit makes them opt in.
If the padding is not divisible by the char's width without remainder,
we pad the remainder with spaces, so the total width of the output is correct.
Also add completions, changelog entry, adjust documentation, add examples
with emoji and some tests. Apply some minor style nitpicks and avoid extra
allocations of the input strings.
Improves on #7328.
I believe this is the correct behavior, simply skip all whitespace before
a word. Try with
./fish -C 'bind \ef forward-bigword; bind \eb backward-bigword; bind \ed kill-bigword; bind \cw backward-kill-bigword'
Also unrelated formatting fixes. I don't think a CI failure on unformatted
code is warranted but I wish it could do that behind the scenes.
For example "grep --color"<TAB> can complete to "grep --color=". Don't add
a space in this case; we do the same for arguments that end in =.
In GNU-style getopt, equal sign means that the flag has an argument. Without
the = it would not consume the next argument as opposed to Python's argparse.
This was a weird special behavior where we'd put the commandline on a
new line if it wrapped *and* the prompt was > 33% of the screen.
It seems to be more confusing than anything.
Fixes#5118.
Prior to this change, tab completing with a variable assignment like
`VAR=val cmd<tab>` would parse out and apply VAR=val, then recursively
invoke completions. This caused some awkwardness around the wrap chain -
if a wrapped command had a variable completion we risked infinite
recursion. A secondary problem is that we would run any command
substitutions inside variable assignment, which the user does not expect
to run until pressing enter.
With this change, we explicitly track variable assignments encountered
during tab completion, including both those explicitly given on the
command line and those found during wrap chain walk. We then apply them
while suppressing command substitutions.
In preparation for applying variable assignments (VAR=VAL cmd), separate
them out from the command when performing completions. This includes both
those that the user typed, and any that come about through
completion --wraps.
When completing and walking a wrap chain, we pass around a lot of
information. Factor this together into a new struct custom_arg_data_t
which reduces the number of parameters needed.
The "wrap chain" refers to a sequence of commands which wrap other
commands, for completion purposes. One possibility is that a wrap chain
will produce a combinatorial explosion or even an infinite loop, so there
needs to be logic to prevent that. Part of that logic is encapsulated in a
visited set (wrap_chain_visited_set_t) to prevent exploring the same item
twice.
Prior to this change, we stored pairs (command, wrapped_command). But we
only really need to store the wrapped command. Switch to that.
One consequence is that if a command wraps another command in more than
one way, we won't explore both ways. This seems unlikely in practice.
Detect recursive calls to builtin complete and the internal completion in
the same place.
In 0a0149cc2 (Prevent infinite recursion when completion wraps variable assignment)
we don't print an error when completing certain aliases like:
alias vim "A=B vim"
But we also gave no completions.
We could make this case work, but I think that trying to salvage situations
like this one is way too complex. Instead, let the user know by printing an
error. Not sure if the style of the error fits.
We could add some heuristic to alias to not add --wraps in some cyclic cases.
The lambda has grown way too big, and it was not easy to see what the inputs
and outputs are. We always use the same visitor, so the function parameter
is not necessary.
This reads any additional positional arguments given to `fish -c` into
$argv.
We don't handle the first argument specially (as `$0`) as that's confusing and
doesn't seem very useful.
Fixes#2314.
This allows
bind -k backspace suppress-autosuggestion or backward-delete-char
To remove the suggestion on the first press and then delete
chars.
Note: This requires that we then don't reenable suggestions
immediately afterwards. Currently we don't after deletion.
Fixes#1419.
This makes history searches case-insensitive, unless the search string
contains an uppercase character.
This is what vim calls "smartcase".
Fixes#7273.
Closes#7344
Apply a targeted fix to the place where complete() is called to handle nested
variable assignments. Sadly, reporting an error is probably not okay here,
because people might legitimately use aliases like:
alias vim "A=B command vim"
This is all a bit ugly, and I hope to find a cleaner solution. Supporting
completions on commandlines like `x=$PWD cd $x/ ` is a nice feature but it
comes with some complexity.
This is too important to not be one.
For one if it couldn't be loaded for any reason it would
break a lot of fish scripts.
Also this is faster by ~20x.
Fixes#7342
4f0ade7a73 broke the tests when LANG was
C, so the MB_CUR_MAX==1 path wasn't working.
Seemingly that cast is doing some work here?
Just revert that bit for now, since this path is unimportant
anyway (please, please, please, please use a unicode capable locale).
This is a reimplementation of the "vectorized" ASCII detection
from str2wcs_internal. This handles the case where only part of
a string is ASCII. It also avoids pointer overflow issues and improves
commenting.
Prior to this change, str2wcs_internal had an optimization for ASCII
inputs. However the main cost was the repeated bounds checks when
performing push_back() on the resulting wcstring.
Switch to determining the number of ASCII characters, and then appending
those all in one go. This improves the time in the 'convert_ascii' test
from ~450 usec to ~75 usec.
When pressing tab repeatedly, completions only computed on the first one. This
is because the old logic assumed that completions are present if the last
key was tab. Recompute them if there are no completions at all.
Fixes#6863
Since builtins don't actually have the streams connected, but instead
read input via the io_streams_t objects, this would just always say
what *fish's* fds were.
Instead, pass along some of the stream data to check those
specifically - nobody cares that `test`s fd 0 *technically* is stdin.
What they want to know is that, if they used another program in that
place, it would connect to the TTY.
This is pretty hacky - I abused static variables for this, but
since it's two bools and an int it's probably okay.
See #1228.
Fixes#4766.
(regression from d415350aaf)
This is important especially in e.g. the new Windows Terminal, because
for some reason that lets the tab stick around if the process exited
with a non-zero status.
Will add tests as soon as I figure out how.
Taken from GNU realpath, this one makes realpath not resolve symlinks.
It still makes paths absolute and handles duplicate and trailing
slashes.
(useful in fish_add_path)
With a commandline like
```
a b c d
```
and the cursor at the beginning, this would eat "a b", which isn't a
sensible bigword.
Bigword should be "a word, with optional leading whitespace".
This was caused by an overly zealous state-machine that always ate one
char and only *then* started eating leading whitespace.
Instead eat *a character*, and if it was whitespace go on eating
whitespace, and if it was a printable go straight to only eating
printables.
Fixes#7325.
