Keeps the location of original function definition, and also stores
where it was copied. `functions` and `type` show both locations,
instead of none. It also retains the line numbers in the stack trace.
By default, fish does not complete files that have leading dots, unless the
wildcard itself has a leading dot. However this also affected completions;
for example `git add` would not offer `.gitlab-ci.yml` because it has a
leading dot.
Relax this for custom completions. Default file expansion still
suppresses leading dots, but now custom completions can create
leading-dot completions and they will be offered.
Fixes#3707.
When we draw the prompt, we move the cursor to the actual
position *we* think it is by issuing a carriage return (via
`move(0,0)`), and then going forward until we hit the spot.
This helps when the terminal and fish disagree on the width of the
prompt, because we are now definitely in the correct place, so we can
only overwrite a bit of the prompt (if it renders longer than we
expected) or leave space after the prompt. Both of these are benign in
comparison to staircase effects we would otherwise get.
Unfortunately, midnight commander ("mc") tries to extract the last
line of the prompt, and does so in a way that is overly naive - it
resets everything to 0 when it sees a `\r`, and doesn't account for
cursor movement. In effect it's playing a terminal, but not committing
to the bit.
Since this has been an open request in mc for quite a while, we hack
around it, by checking the $MC_SID environment variable.
If we see it, we skip the clearing. We end up most likely doing
relative movement from where we think we are, and in most cases it
should be *fine*.
This is early work but I guess there's no harm in pushing it?
Some thoughts on the conventions:
Types that live only inside Rust follow Rust naming convention
("FeatureMetadata").
Types that live on both sides of the language boundary follow the existing
naming ("feature_flag_t").
The alternative is to define a type alias ("using feature_flag_t =
rust::FeatureFlag") but that doesn't seem to be supported in "[cxx::bridge]"
blocks. We could put it in a header ("future_feature_flags.h").
"feature_metadata_t" is a variant of "FeatureMetadata" that can cross
the language boundary. This has the advantage that we can avoid tainting
"FeatureMetadata" with "CxxString" and such. This is an experimental approach,
probably not what we should do in general.
The initial port of feature flags requires a global initialization. Since
fish_indent accesses feature flags, let's make sure to initialize them here.
In future, we can stop initializing things fish_indent doesn't need (like
the topic monitor) but that's no big deal. Global initialization should
always be a benign addition.
The original implementation without the test took me 3 hours (first time
seriously looking into this)
The functions take "wcharz_t" for smooth integration with existing C++ callers.
This is at the expense of Rust callers, which would prefer "&wstr". Would be
nice to declare a function parameter that accepts both but I don't think
that really works since "wcharz_t" drops the lifetime annotation.
This works around an autocxx limitations where different types cannot
have the same name even if they live in different namespace.
ast::job_t conflicts with job_t.
This translated ctrl-k to "\v", which is a "vertical tab", and ctrl-l
to "\f" and ctrl-g to "\a".
There is no "vertical tab" or "alarm" or "\f" *key*, so these
shouldn't be translated. Just drop these and call them `\ck` and such.
(vertical tab specifically is utterly useless and I would be okay with
dropping it entirely, I have never seen it used anywhere)
Commit 3b30d92b6 (Commit transient edit when closing pager, 2022-08-31)
inadvertently introduced two regressions to history search:
1. It made Escape keeps the selected history entry,
instead of restoring the commandline before history search.
2. It made history search commands add undo entries.
Fix both of this issues.
Inadvertently broken in a2d816710f,
this made `cd .` no longer offer `cd ../` (same for general file completions
like `ls .`, which only offers dotfiles)
This meant we didn't actually do our weird en/decoding scheme for e.g.
a C locale, which meant that, when you then switch to a proper locale
the previous variables were broken.
I don't know how to test this automatically - none of my attempts seem
to ever *fail* with the old code, here's what you'd do manually:
- Run fish with an actual C locale (LC_ALL=C
fish_allow_singlebyte_locale=1 fish)
- `set -gx foo 💩`
- `set -e LC_ALL`
- `echo $foo` outputs "💩" if it works and "ð⏎" if it's broken.
Fixes#2613
This means cleaning out old universal variables is now just:
```fish
abbr --erase (abbr --list)
```
which makes upgrading much easier.
Note that this erases the currently defined variable and/or any
universal. It doesn't stop at the former because that makes it *easy*
to remove the universals (no running `abbr --erase` twice), and it
doesn't care about globals because, well, they would be gone on
restart anyway.
Fixes#9468.
Like I mentioned in #9089, 12 entries is a bit few.
So, instead, we do like we do for completions before disclosing and
pick half the screen (but at least X, in this case 12).
This avoids filling the entire screen, and will avoid an unsightly "X
more entries" (which requires scrolling down to fully disclose)
because it matches what the pager does.
Note: For multiline commands we can be pushed further upwards, and in
case of a multi-column layout we could fit more lines. That would
require asking the pager to fit as many as possible and give us back
the index of the last matching entry and rewinding the history search.
That's gonna be left as an exercise for later if it turns out to be necessary.
This now means `abbr --add` has two modes:
```fish
abbr --add name --function foo --regex regex
```
```fish
abbr --add name --regex regex replacement
```
This is because `--function` was seen to be confusing as a boolean flag.
Unfortunately print_hints was true *by default* - so for all builtins
that didn't pass it it would now be false instead.
This resulted in the trailer missing, which includes the line number
and context. So if you ran a script that includes `bind -M` the error
message would now just be "bind: -M: option requires an argument",
with no indication as to where.
This reverts commit 8a50d47a46.
The print_hints variable was always false, so just remove it.
This caused a cascade of other changes where the parser_t variable
becomes unused, so remove it from the call sites.
No functional change expected here.
When we insert characters that don't yet have highlighting, we use the
highlighting to the left, unless there is nothing to our left. The logic to
check if we are the leftmost character uses an overly loose comparison. Let's
make it more specific.
No functional change.
When there are multiple event handlers for a single event, we would print
the same log statement twice. Let's add the function name to make this
less confusing.
This would print
```
abbr -a -- dotdot --regex ^\\.\\.+\$ --function multicd
```
which expands "dotdot" to "--regex ^\\.\\.+\$...".
Instead, we move the name to right before the replacement, and move
the `--` before that:
```
abbr -a --regex ^\\.\\.+\$ --function -- dotdot multicd
```
It might be possible to improve that, but this at least round-trips.
Historical behavior is to stop option parsing at the first non-option argument.
Since we have added more options, it seemed impractical to keep that behavior.
However people are using options in their abbr expansions ("abbr e emacs
-nw"). To support this, we ignore options. However, we only ignore them
if they are not valid "abbr" options. Let's ignore all options in the
expansion definition, which is a small price to pay to keep most existing
configurations working.
Fixes#9410
This does not fix other cases which used to work, like
abbr x -unknown
Those are hopefully not used by anyone, so I don't think we need to maintain
support for that.
Enhances abbreviations with extra features
- global abbreviations
- trigger on regex match as alternative to literal match
- the ability to expand abbreviations with a user-defined function
- the ability to set cursor position after expansion
Also default the marker to '%'. So you may write:
abbr -a L --position anywhere --set-cursor "% | less"
or set an explicit marker:
abbr -a L --position anywhere --set-cursor=! "! | less"
This renames abbreviation triggers from `--trigger-on entry` and
`--trigger-on exec` to `--on-space` and `--on-enter`. These names are less
precise, as abbreviations trigger on any character that terminates a word
or any key binding that triggers exec, but they're also more human friendly
and that's a better tradeoff.
set-cursor enables abbreviations to specify the cursor location after
expansion, by passing in a string which is expected to be found in the
expansion. For example you may create an abbreviation like `L!`:
abbr L! --position anywhere --set-cursor ! "! | less"
and the cursor will be positioned where the "!" is after expansion, with
the "| less" appearing to its right.
This adds support for the `--function` option of abbreviations, so that the
expansion of an abbreviation may be generated dynamically via a fish
function.
Prior to this change, abbreviations were stored as fish variables, often
universal. However we intend to add additional features to abbreviations
which would be very awkward to shoe-horn into variables.
Re-implement abbreviations using a builtin, managing them internally.
Existing abbreviations stored in universal variables are still imported,
for compatibility. However new abbreviations will need to be added to a
function. A follow-up commit will add it.
Now that abbr is a built-in, remove the abbr function; but leave the
abbr.fish file so that stale files from past installs do not override
the abbr builtin.
This allows adjusting a pattern string so that it matches an entire
string, by wrapping the regex in a group like ^(?:...)$
This is a workaround for the fact that PCRE2_ENDANCHORED is unavailable
on PCRE2 prior to 2017, so we have to adjust the pattern instead.
Also introduce an overload of match() which creates its own
match_data_t.
We have had multiple crashes for relative CDPATH entries. Commit 5e274066e
(Always return absolute path in path_get_cdpath, 2019-10-17) tried to fix
all of them but it failed to do justice to its title. Let's fix this to
actually return absolute paths, always. Take care to to normalize the path
because it is used for autosuggestions. The normalization is mostly relevant
for CDPATH=. (the default) but it doesn't hurt others.
Closes#9407
wopterr was a feature to allow wgetopt to emit error messages; but we do
not use this and never will. Remove its support. No functional change
expected here.
We wrongly highlight this as prefix when actually the trailing slash should
invalidate it. Turns out path normalization drops the slash, so let's
sidestep that.
Fixes#9394
The "flag" field enables an option to discover which flag it was invoked
with. However in practice none of our options use multiple flags so this
parameter was always nullptr. Remove it and fix up all the builtins to
stop passing this.
No functional change here.
I believe this should be identical to the previous code and handle the same
cases (I'm guessing going by the comment that this came from a C codebase
without `bool` types).
The problem with the previous code is that it tripped up the `clangd` analyzer
into thinking `assert()` expressions can/should be simplified via DeMorgan's to
improve readability (because it was seeing the fully expanded macro).
The tty_ownership test was sometimes failing. In this test,
`fish_test_helper` creates a child and transfers the tty to it,
"abandoning" the tty. In some cases, the child was running before the
parent; the child claims the tty. When the parent tries to transfer it to
the child, it get SIGTTIN and stops. Fix this by ignoring SIGTTIN and
SIGTTOU.
This only affects macOS and BSDs.
The stack overflow tests are too slow without this.
This is because the tests are essentially quadratic: with 500 jobs, and
each job attempts to reap all jobs.
Inside a comment we offer plain file completions (or command completions if
the comment is in command position). However these completions are broken
because they don't consider any of the surrounding characters. For example
with a command line
echo # comment
^ cursor
we suggest file completions and insert them as
echo # comsomefile ment
Providing completions inside comments does not seem useful and it can be
misleading. Let's remove the completions; this should communicate better that
we are in a free-form comment that's not subject to fish syntax.
Closes#9320
When unsetting, the scope indicates the scope that was *removed* not
set, so the warning is incorrectly triggered. If anything, the confusion
is now removed or we emit a warning that the variable is still present
in another scope (but don't do that!).
Closes#9338.
This fixes#9321
IEEE Std 1003.1-2017 Issue 6 added optional error condition
[EINVAL] for if no conversion could be performed.
Switch back to wcstoimax/wcstoumax: do not work around the old FreeBSD
8 issue.
Add a test for printf '%d %d' 1 2 3
This addresses a long-standing TODO where `complete -C` output isn't
deduplicated.
With this patch, the same deduplication and sort procedure that is run on actual
pager completions is also executed for `complete -C` completions (with a `-C`
payload specified).
This makes it possible to use `complete -C` to test what completions will
actually be generated by the completions pager instead of it displaying
something completely divorced from reality, improving the productivity of fish
completions developers.
Note that completions that wouldn't be shown in the pager are also omitted from
the results, e.g. `test/buildroot/` and `test/fish_expand_test/` are omitted
from the check matches in `checks/complete_directories.fish` because even if
they were generated, the pager wouldn't have shown them. This again makes
reasoning about and debugging completions much easier and more sane.
This reverts commit 1c92d4c5db and
reintroduces support for trivially copyable `maybe_t` impls but with a
GCC version check to disable the optimization for GNU GCC compiler
versions 9 and below.
GCC 8.3.0 armhf builds seem to have a problem with the trivially
copyable `maybe_t` impl that introduces odd heisenbugs that cause the
tests to fail. GDB reveals that `maybe_t` function parameters received
in the callee differ from what was passed-in by the caller.
This behavior appears to be (but has not been confirmed as) a
platform-specific compiler bug. Under the same system (32-bit Debian 10
armhf), compiling with clang 7.0.1 does not result in any bugs and
causes all the tests to pass while compiling with GCC 10.2 under 32-bit
Debian 11 armhf also doesn't run into any problems, so just expand the
existing GCC version check that gates support for trivially copyable
`maybe_t` impls to encompass both the troublesome GCC 8 version and the
untested GCC 9 version.
This reverts commit 9d303a74e3.
This reverts commit 0305c842e6.
9d303a7 broke 32-bit armhf builds for unknown reasons, specifically in
settings where a trivial copy of `maybe_t<int>` was performed. A caller
would pass a literal int in the place of a `maybe_t<int>` parameter and
the callee would see a populated `maybe_t` but with a value of `0`
rather than the actual value that was passed in. It was too painful to
debug to a resolution under qemu.
Fixes ommitted newline char shown after complete -n'(foo)'
Also axes the 'contains syntax errors' line before the error.
Update tests
before
> complete -n'(foo)'
complete: Condition '(foo)' contained a syntax error
complete: Command substitutions not allowed⏎
after
> complete -n'(foo)'
complete: -n '(foo)': command substitutions not allowed here
This is a salvage of the "no functional changes" part of #9221, and cherry-picks
storing completion entries in a vector instead of a linked list. The legacy
"reverse intuitive" group ordering is kept by iterating in reverse order.
Tests pass but don't actually cover group order, which needs another test.
Makes it possible to retrieve the currently executing command line as
opposed to the currently executing command (`status current-command`).
Closes#8905.
There should be no functional changes in this commit.
The global variable `$_` set in the parser variables by `reader.cpp` and
read by the `status` builtin was deprecated in fish 2.0 but kept around
internally because there's no good way to store/share/forward parser
variables.
A new enum is added that identifies the status variable and they are
stored in a private array in the parser. There is no need for
synchronization because they are only set during job init and never
thereafter. This is currently asserted via ASSERT_IS_MAIN_THREAD() but
that assert can be dropped in the interest of making the parser possible
to clone and use from worker threads.
The old `$_` global variable is still kept for backwards compatibility,
though it will be dropped in a future release.
As the user is typing an argument, fish continually checks if the input is
the prefix of a valid file path. If yes, the input is underlined.
The same prefix-logic is used for all tokens on the command line, even for
"finished" tokens. This means we highlight any token that happens to be
a prefix of a valid file path. We actually want this to only apply to the
token that the user is currently typing.
Let's use the prefix-logic only for tokens adjacent to the cursor. This should
better match user expectations (and reduce IO traffic). I don't think this is
the perfect criteria but I don't know how else we can determine if a token is
"unfinished".
When visiting the "cd" node, we mark invalid paths as error, but don't
underline valid paths. This works fine most of the time because we later
underline paths (for any command, not just "cd").
However the latter check fails to honor CDPATH. Let's correct that, which
also allows to simplify the logic.
The next commit wants to move the "Underline every valid path" logic into the
visit() methods. The logic currently polls the cancel checker before checking
each path. If that's valid, it should probably have the same behavior inside
visit(). Since we currently can't cancel an AST-visitation, the next best
thing seems to suspend all IO operations, the rest should be very fast anyway.
I'm not sure if the motivation is strong enough; a conceivable alternative
would be to stop using the cancel checker altogether for highlighting.
When passing a value of type maybe_t<size_t>, clangd complains:
Parameter 'cursor' is passed by value and only copied once; consider
moving it to avoid unnecessary copies (fix available)
We get this warning because maybe_t<size_t> is not trivially copyable
because it has a user-defined destructor and copy-constructor. Let's remove
them if the contained type is trivially copyable, to avoid such warnings.
No functional change.
The destructor is equivalent to the compiler-generated one. The user-defined
destructor prevents maybe_t<size_t> from bearing the predicate "trivially
copyable". Let's remove it. No functional change.
It seems to have originally been thought that the only possible way a stack
overflow could happen is via function calls, but there are other possibilities.
Issue #9302 reports how `eval` can be abused to recursively execute a string
substitution ad infinitum, triggering a stack overflow in fish.
This patch extends the stack overflow check to also check the current
`eval_level` against a new constant `FISH_MAX_EVAL_DEPTH`, currently set to a
conservative but hopefully still fair limit of 500. For future reference, with
the default stack size for the main/foreground thread of 8 MiB, we actually have
room for a stack depth around 2800, but that's only with extremely minimal state
stored in each stack frame.
I'm not entirely sure why we don't check `eval_depth` regardless of block type;
it can't be for performance reasons since it's just a simple integer comparison
- and a ridiculously easily one for the branch predictor handle, at that - but
maybe it's to try and support non-recursive nested execution blocks of greater
than `FISH_MAX_STACK_DEPTH`? But even without recursion, the stack can still
overflow so may be we should just bump the limit up some (to 500 like the new
`FISH_MAX_EVAL_DEPTH`?) and check it all the time?
Closes#9302.
A `block_t` instance is allocated for each live block type in memory when
executing a script or snippet of fish code. While many of the items in a
`block_t` class are specific to a particular type of block, the overhead of
`maybe_t<event_t>` that's unused except in the relatively extremely rare case of
an event block is more significant than the rest, given that 88 out of the 216
bytes of a `block_t` are set aside for this field that is rarely used.
This patch reorders the `block_t` members by order of decreasing alignment,
bringing down the size to 208 bytes, then changes `maybe_t<event_t>` to
`shared_ptr<event_t>` instead of allocating room for the event on the stack.
This brings down the runtime memory size of a `block_t` to 136 bytes for a 37%
reduction in size.
I would like to investigate using inheritance and virtual methods to have a
`block_t` only include the values that actually make sense for the block rather
than always allocating some sort of storage for them and then only sometimes
using it. In addition to further reducing the memory, I think this could also be
a safer and saner approach overall, as it would make it very clear when and
where we can expect each block_type_type_t-dependent member to be present and
hold a value.
This is a false positive as a result of disabling TLS support in LSAN due to an
incompatibility with newer versions of glibc.
Also remove the older workaround (because it didn't work).
When there are multiple screens worth of output and `history` is writing to the
pager, pressing Ctrl-C at the end of a screen doesn't exit the pager (`q` is
needed for that) but previously caused fish to emit an error ("write:
Interrupted system call) until we starting silently handling SIGINT in
`fd_output_stream_t::append()`.
This patch makes `history` detect when the `append()` call returns with an error
and causes it to end early rather than repeatedly trying (and failing) to write
to the output stream.
If EINTR caused by SIGINT is encountered while writing to the
`fd_output_stream_t` output fd, mark the output stream as errored and return
false to the caller but do not visibly complain.
Addressing the outstanding TODO notwithstanding, this is needed to avoid
littering the tty with spurious errors when the user hits Ctrl-C to abort a
long-running builtin's output (w/ the primary example being `history`).
Up to now, in normal locales \x was essentially the same as \X, except
that it errored if given a value > 0x7f.
That's kind of annoying and useless.
A subtle change is that `\xHH` now represents the character (if any)
encoded by the byte value "HH", so even for values <= 0x7f if that's
not the same as the ASCII value we would diverge.
I do not believe anyone has ever run fish on a system where that
distinction matters. It isn't a thing for UTF-8, it isn't a thing for
ASCII, it isn't a thing for UTF-16, it isn't a thing for any extended
ASCII scheme - ISO8859-X, it isn't a thing for SHIFT-JIS.
I am reasonably certain we are making that same assumption in other
places.
Fixes#1352
Closes#9240.
Squash of the following commits (in reverse-chronological order):
commit 03b5cab3dc40eca9d50a9df07a8a32524338a807
Author: Mahmoud Al-Qudsi <mqudsi@neosmart.net>
Date: Sun Sep 25 15:09:04 2022 -0500
Handle differently declared posix_spawnxxx_t on macOS
On macOS, posix_spawnattr_t and posix_spawn_file_actions_t are declared as void
pointers, so we can't use maybe_t's bool operator to test if it has a value.
commit aed83b8bb308120c0f287814d108b5914593630a
Author: Mahmoud Al-Qudsi <mqudsi@neosmart.net>
Date: Sun Sep 25 14:48:46 2022 -0500
Update maybe_t tests to reflect dynamic bool conversion
maybe_t<T> is now bool-convertible only if T _isn't_ already bool-convertible.
commit 2b5a12ca97b46f96b1c6b56a41aafcbdb0dfddd6
Author: Mahmoud Al-Qudsi <mqudsi@neosmart.net>
Date: Sun Sep 25 14:34:03 2022 -0500
Make maybe_t a little harder to misuse
We've had a few bugs over the years stemming from accidental misuse of maybe_t
with bool-convertible types. This patch disables maybe_t's bool operator if the
type T is already bool convertible, forcing the (barely worth mentioning) need
to use maybe_t::has_value() instead.
This patch both removes maybe_t's bool conversion for bool-convertible types and
updates the existing codebase to use the explicit `has_value()` method in place
of existing implicit bool conversions.
The parent commit made the destructor of the DIR* member close it if necessary
(i.e. only if it's not null). This means that we can use the same logic in
the move constructor (where the source DIR* is null) and for move assignment
(where it might not be).
No functional change.
dir_iter_t closes its DIR* member in two places: the move assignment and
the destructor. Simplify this by closing it in the destructor of the DIR*
member which is called in both places. Use std::unique_ptr, which is shorter
than a dedicated wrapper class. Conveniently, it calls the deleter only if
the pointer is not-null. Unfortunately, std::unique_ptr requires explicit
conversion to DIR* when interacting with C APIs but it's probably still
better than a wrapper class.
This means that the noncopyable_t annotation is now implied due to the
unique_ptr member.
Additionally, we could probably remove the user-declared move constructor
and move assignment (the compiler-generated ones should be good enough). To
be safe, keep them around since they also erase the fd (though I hope we
don't rely on that behavior anywhere).
We should perhaps remove the user-declared destructor entirely but
dir_iter_t::entry_t also has one, I'm not sure why. Maybe there's a good
reason, like code size.
No functional change.
This was recently converted to a while-loop. However, we only
loop in a specific case when (by hitting "continue") so a
loop condition is not necessary.
No functional change.
We forgot to decode (i.e. turn into nice wchar_t codepoints)
"byte_literal" escape sequences. This meant that e.g.
```fish
string match ö \Xc3\Xb6
math 5 \X2b 5
```
didn't work, but `math 5 \x2b 5` did, and would print the wonderful
error:
```
math: Error: Missing operator
'5 + 5'
^
```
So, instead, we decode eagerly.
descend_unique_hierarchy is used for the cd autosuggestion: if a directory
contains exactly one subdirectory and no other entries, then propose that
as part of the cd autosuggestion.
This had a bug: if the subdirectory is a symlink to the parent, we would
chase that, going around the loop suggesting a longer path until we hit
PATH_MAX.
Fix this by using the new API which provides the inode "for free," and
track whether we've seen this inode before. This is technically too
conservative since the inode may be for a directory on a different device,
but devices are not available for free so this would incur a cost. In
practice encountering the same inode twice with different devices in a
unique hierarchy is unlikely, and should it happen the consequences are
merely cosmetic: we fail to suggest a longer path.
This introduces dir_iter_t, a new class for iterating the contents of a
directory. dir_iter_t encapsulates the logic that tries to avoid using
stat() to determine the type of a file, when possible.
While we hardcode the return values for the rest of our builtins, the `return`
builtin bubbles up whatever the user returned in their fish script, allowing
invalid return values such as negative numbers to make it into our C++ side of
things.
In creating a `proc_status_t` from the return code of a builtin, we invoke
W_EXITCODE() which is a macro that shifts left the return code by some amount,
and left-shifting a negative integer is undefined behavior.
Aside from causing us to land in UB territory, it also can cause some negative
return values to map to a "successful" exit code of 0, which was probably not
the fish script author's intention.
This patch also adds error logging to help catch any inadvertent additions of
cases where a builtin returns a negative value (should one forget that unix
return codes are always positive) and an assertion protecting against UB.
This was always the case if HAVE_TEXT wasn't defined, but if it was then we were
coercing the result of `_C()` to a `const wchar_t *` pointer, because we were
returning the address of a constant zero-length wchar_t pointer. This reserves a
local static `wcstring` variable that we can return as the "no text" sentinel
and bubbles back the `wcstring` reference rather than decomposing it into a
pointer.
This is a prerequisite for a bigger change I'm working on.
It's gone from 136 bytes to a 128 bytes by rearranging the items in order of
decreasing alignment requirements. While this reduces the memory consumption
slightly (by around 6%) for each completion we have in-memory, that translates
to only around ~8KiB of savings for a command with 1000 possible completions,
which is nice but ultimately not that big of a deal.
The bigger benefit is that a single `complete_entry_t` might now fit in a cache
line, hopefully making the process of testing completions for matches more
cache friendly (and maybe even faster).
The impact here depends on the command and how much output it
produces.
It's possible to get up to 1.5x - `string upper` being a good example,
or a no-op `string match '*'`.
But the more the command actually needs to do, the less of an effect
this has.
This basically immediately issues a "write()" if it's to a pipe or the
terminal.
That means we can reduce syscalls and improve performance, even by
doing something like
```c++
streams.out.append(somewcstring + L"\n");
```
instead of
```c++
streams.out.append(somewcstring);
streams.out.push_back(L'\n');
```
Some benchmarks of the
```fish
for i in (string repeat -n 2000 \n)
$thing
end
```
variety:
1. `set` (printing variables) sped up 1.75x
2. `builtin -n` 1.60x
3. `jobs` 1.25x (with 3 jobs)
4. `functions` 1.20x
5. `math 1 + 1` 1.1x
6. `pwd` 1.1x
Piping yields similar results, there is no real difference when
outputting to a command substitution.
This writes the output once per argument instead of once per format or
escaped char.
An egregious case:
```fish
printf (string repeat -n 200 \\x7f)%s\n (string repeat -n 2000 aaa\n)
```
Has been sped up by ~20x by reducing write() calls from 40000 to 200.
Even a simple
```fish
printf %s\n (string repeat -n 2000 aaa\n)
```
should now be ~1.2x faster by issuing 2000 instead of 4000 write
calls (the `\n` was written separately!).
This at least halves the number of "write()" calls we do if it goes to
a pipe or the terminal, or reduces them by 75% if there is a
description.
This makes
```fish
complete -c foo -xa "(seq 50000)"
complete -C"foo "
```
faster by 1.33x.
This uses wreaddir_resolving, which tries to use the dirent d_type
field if it exists. In that way, it can skip the `stat` to determine
if the given file is a directory.
This allows `cd` completions to skip stat in most cases:
```fish
strace -Ce newfstatat fish --no-config -c 'complete -C"cd /tmp/completion_test/"' >/dev/null
```
prints before:
```
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
100,00 0,002627 2 1033 4 newfstatat
```
after:
```
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
100,00 0,000054 1 31 3 newfstatat
```
for a directory with 1000 subdirectories.
(just `fish --no-config -c exit` does 26 newfstatat)
This should improve the situation with slow filesystems like fuse or
network fsen.
In case we have no d_type, we use `stat`, which would yield about the
same results.
The worst case is that we need directories *and* descriptions or the
"executable" flag (which we don't currently check for cd, if I read
this right?).
This flag determines whether or not more shortopt switches will be offered up as
potential completions (vs only the payload for the last-parsed shortopt switch).
Previously, it was being stomped before it was determined whether or not two
`complete` rules with different `result_mode.requires_param` values were
actually resolved against the current command line or not, and the last
evaluated completion rule would win out.
There are two changes here:
* `last_option_requires_param` is only assigned if all associated conditions for
a potential completion are also met, and
* If already assigned by a conflicting rule (which can only be user/developer
error), `last_option_requires_param` is allowed to change from true to false
but not the other way around (i.e. in case of a conflict, generate both
payloads and other shortopt completions)
The first change is immediately noticeable and affects many of our own
completions, see the discussion in #9221 for an example regarding `git` where
`-c` has any of about a million different possible meanings depending on which
completion preconditions have been met. The second change should only happen if
a dev/user mistakenly enters a `complete -c ...` rule for the same shortopt more
than once, both with conditions matching, sometimes requiring an argument and
not sometimes not. It should be a rare occurence.
This reverts commit 3d8f98c395.
In addition to the issues mentioned on the GitHub page for this commit,
it also broke the CentOS 7 build.
Note one can locally test the CentOS 7 build via:
./docker/docker_run_tests.sh ./docker/centos7.Dockerfile
Be more careful with sign extension issues stemming from the differences in how
an untyped literal is promoted to an integer vs how a typed (and signed) `char`
is promoted to an integer.
Also convert some `const[expr] static xxx` to `const[expr] xxx` where it makes
sense to let the compiler deduce on its own whether or not to allocate storage
for a constant variable rather than imposing our view that it should have STATIC
storage set aside for it.
A few call sites were not making use of the `XXX_LEN` definitions and were
calling `strlen(XXX)` - these have been updated to use `const_strlen(XXX)`
instead.
I'm not sure if any toolchains will have raise any issues with these changes...
CI will tell!
The optimization takes references to strings which are stored in a vector,
and stores those references in a set; but the strings are simultaneously
being moved within the vector, which may invalidate those references.
It's probably safe if you work through which particular strings are being
moved, but as a matter of principle we shouldn't take references to elements
of a vector while the vector is being rearranged, absenet a clear improvement
on a benchmark.
This reverts commit d5561623aa.
Whenever the command line changes, we redraw it with the previously computed
syntax highlighting. At the same time we start recomputing highlighting in
a background thread.
On some systems, the highlighting computation is slow, so the stale syntax
highlighting is visible.
The stale highlighting was computed for an old commandline. When the user
had inserted or deleted some characters in the middle, then the highlighting
is wrong for the characters to the right. This is because the characters
to the right have shifted but the highlighting hasn't. Fix this by also
shifting highlighting.
This means that text that was alrady highlighted will use the same
highlighting until a new one is computed. Newly inserted text uses the color
left of the cursor.
This is implemented by giving editable_line_t ownership of the highlighting.
It is able to perfectly sync text and highlighting; they will invariably
have the same length.
Fixes#9180
While its true that we only ever call this with temporaries, there is no
fundamental reason for this restriction. Taking by value is simpler and
more flexible. I think it does not change the generated code.
No functional change.
The idea for this function was that it stands as the one place that modifies
the text without push_edit. In practice I don't think it helps.
No functional change.
In theory this does less work so we should generally use this style.
In practice it looks uglier so I'm not sure. Maybe wait for stdlib ranges...
No functional change.
It turns out there *is* an obviously portable way... except it's
not-so-obviously not portable after all.
POSIX specifies that sigqueue(2) can be used to validate pid and signo
separately, returning EINVAL in the specific case of an invalid or unsupported
signal number. This would be perfect... if only it were actually implemented.
It seems that the WSLv1 implementation of pselect(2) does not check for
undelivered signals after the temporary sigmask is un-applied from the thread in
question.
When fish runs with job control enabled, it transfers ownership of the
tty to a child process, and then reclaims the tty after the process
exits. If job control is disabled then fish does not transfer or reclaim
the tty.
It may happen that the child process creates a pgroup and then transfers
the tty to it. In that case fish will not attempt to reclaim the tty, as
fish did not transfer it. Then when fish reads from stdin it will
receive SIGTTIN instead of data.
Fix this by unconditionally claiming the tty in readline().
Fixes#9181
This errored out *later* because the result was infinite or NaN, but
it didn't actually stop evaluation.
I'm not sure if there is a way to get floating point math to turn an
infinity back into something that doesn't depend on a literal
infinity, but division by zero conceptually isn't a thing we can
support.
There's entire branches of maths dedicated to figuring out what
dividing by "basically zero" means and we don't have to get into it.
This is essentially the inverse of `string pad`.
Where that adds characters to get up to the specified width,
this adds an ellipsis to a string if it goes over a specific maximum width.
The char can be given, but defaults to our ellipsis string.
("…" if the locale can handle it and "..." otherwise)
If the ellipsis string is empty, it just truncates.
For arguments given via argv, it goes line-by-line,
because otherwise length makes no sense.
If "--no-newline" is given, it adds an ellipsis instead and removes all subsequent lines.
Like pad and `length --visible`, it goes by visible width,
skipping recognized escape sequences, as those have no influence on width.
The default target width is the shortest of the given widths that is non-zero.
If the ellipsis is already wider than the target width,
we truncate instead. This is safer overall, so we don't e.g. move into a new line.
This is especially important given our default ellipsis might be width 3.
When selecting items in the pager, only the latest of those items is kept
in the edit history, as so-called transient edit. Each new transient edit
evicts any old transient edit (via undo).
If the pager is closed by a command that performs another transient edit
(like history-token-search-backward) we thus inadvertently undo (= remove)
the token inserted by the pager. Fix this by closing a transient edit
session when closing the pager. Token search will start its own session.
Fixes#9160
strncpy will fill the entire buffer with NUL.
In this case we have a 128 byte buffer and write "empty" - 5 bytes -
into it.
So now instead of writing 6 bytes it'll write 128 bytes. Especially
wasteful because we already did memset before
This fixes a crash when you open the history pager and then do
history-token-search-backward (e.g. alt+. or alt-up).
It would sometimes crash because the `colors.at(i)` was an
out-of-bounds access.
Note: This might still leave the highlighting offset in some
cases (not quite sure why), but at least it doesn't *crash*, and the
search generally *works*.
This reverts commit 3e556b984c.
Revert "Further fix the issue and add the assert that'd have prevented it."
This reverts commit 056502001e.
Revert "Fix actual issue with allow_use_posix_spawn."
This reverts commit 85b9f3c71f.
Revert "Stop using posix_spawn when it is not allowed"
This reverts commit 9c896e1990.
Revert "don't even set up a fish_use_posix_spawn handler if unsupported"
This reverts commit 8b14ac4a9c.
Commit 8b14ac4a9c started using
posix_spawn even if allow_use_posix_spawn() returns false. Stop doing
that.
This may be reproduced with:
./docker/docker_run_tests.sh ./docker/centos7.Dockerfile
as centos7 has a too-old glibc.
Let's hope this doesn't causes build failures for e.g. musl: I just
know it's good on macOS and our Linux CI.
It's been a long time.
One fix this brings, is I discovered we #include assert.h or cassert
in a lot of places. If those ever happen to be in a file that doesn't
include common.h, or we are before common.h gets included, we're
unawaringly working with the system 'assert' macro again, which
may get disabled for debug builds or at least has different
behavior on crash. We undef 'assert' and redefine it in common.h.
Those were all eliminated, except in one catch-22 spot for
maybe.h: it can't include common.h. A fix might be to
make a fish_assert.h that *usually* common.h exports.
This is a *tiny* commit code-wise, but the explanation is a bit
longer.
When I made string read in chunks, I picked a chunk size from bash's
read, under the assumption that they had picked a good one.
It turns out, on the (linux) systems I've tested, that's simply not
true.
My tests show that a bigger chunk size of up to 4096 is better *across
the board*:
- It's better with very large inputs
- It's equal-to-slightly-better with small inputs
- It's equal-to-slightly-better even if we quit early
My test setup:
0. Create various fish builds with various sizes for
STRING_CHUNK_SIZE, name them "fish-$CHUNKSIZE".
1. Download the npm package names from
https://github.com/nice-registry/all-the-package-names/blob/master/names.json (I
used commit 87451ea77562a0b1b32550124e3ab4a657bf166c, so it's 46.8MB)
2. Extract the names so we get a line-based version:
```fish
jq '.[]' names.json | string trim -c '"' >/tmp/all
```
3. Create various sizes of random extracts:
```fish
for f in 10000 1000 500 50
shuf /tmp/all | head -n $f > /tmp/$f
end
```
(the idea here is to defeat any form of pattern in the input).
4. Run benchmarks:
hyperfine -w 3 ./fish-{128,512,1024,2048,4096}"
-c 'for i in (seq 1000)
string match -re foot < $f
end; true'"
(reduce the seq size for the larger files so you don't have to wait
for hours - the idea here is to have some time running string and not
just fish startup time)
This shows results pretty much like
```
Summary
'./fish-2048 -c 'for i in (seq 1000)
string match -re foot < /tmp/500
end; true'' ran
1.01 ± 0.02 times faster than './fish-4096 -c 'for i in (seq 1000)
string match -re foot < /tmp/500
end; true''
1.02 ± 0.03 times faster than './fish-1024 -c 'for i in (seq 1000)
string match -re foot < /tmp/500
end; true''
1.08 ± 0.03 times faster than './fish-512 -c 'for i in (seq 1000)
string match -re foot < /tmp/500
end; true''
1.47 ± 0.07 times faster than './fish-128 -c 'for i in (seq 1000)
string match -re foot < /tmp/500
end; true''
```
So we see that up to 1024 there's a difference, and after that the
returns are marginal. So we stick with 1024 because of the memory
trade-off.
----
Fun extra:
Comparisons with `grep` (GNU grep 3.7) are *weird*. Because you both
get
```
'./fish-4096 -c 'for i in (seq 100); string match -re foot < /tmp/500; end; true'' ran
11.65 ± 0.23 times faster than 'fish -c 'for i in (seq 100); command grep foot /tmp/500; end''
```
and
```
'fish -c 'for i in (seq 2); command grep foot /tmp/all; end'' ran
66.34 ± 3.00 times faster than './fish-4096 -c 'for i in (seq 2);
string match -re foot < /tmp/all; end; true''
100.05 ± 4.31 times faster than './fish-128 -c 'for i in (seq 2);
string match -re foot < /tmp/all; end; true''
```
Basically, if you *can* give grep a lot of work at once (~40MB in this
case), it'll churn through it like butter. But if you have to call it
a lot, string beats it by virtue of cheating.
Rephrase this to more explicitly indicate that the uvar actually
was successfully set. I believe the prior phrasing can leave some
ambiguity as far as wether set just failed with an error, whether it
has done anything or not.
Now uses the same macro other builtins use for a missing -e arg,
and the error message show the short or long option as it was used.
e.g. before
$ set -e
set: Erase needs a variable name
after
$ set --erase
set: --erase: option requires an argument
$ set -e
set: -e: option requires an argument
Intern'd strings were intended to be "shared" to reduce memory usage but
this optimization doesn't carry its weight. Remove it. No functional
change expected.
We store filenames in function definitions to indicate where the
function comes from. Previously these were intern'd strings. Switch them
to a shared_ptr<wcstring>, intending to remove intern'd strings.
The history pager will show multiline commands in single-line cells.
We escape newline characters as \\n but that looks awkward if the next line
starts with a letter. Let's render control characters using their corresponding
symbol from the Control Pictures Unicode block.
This means there is also no need to escape backslashes, which further improves
the history pager - now the rendering has exactly as many backslashes as
the eventual command.
This means that (multiline) commands in the history pager will be rendered
with the same amount of characters as are in the actual command (unless
they contain funny nonprintables). This makes it easy for the next commit
to highlight multiline commands correctly in the history pager.
The font size for these symbols (for example ␉) is quite small, but that's
okay since for the proposed uses it's not so important that they readable.
The important thing is that the stand out from surrounding text.
This checked specifically for "| and" and "a=b" and then just gave the
error without a caret at all.
E.g. for a /tmp/broken.fish that contains
```fish
echo foo
echo foo | and cat
```
This would print:
```
/tmp/broken.fish (line 3): The 'and' command can not be used in a pipeline
warning: Error while reading file /tmp/broken.fish
```
without any indication other than the line number as to the location
of the error.
Now we do
```
/tmp/broken.fish (line 3): The 'and' command can not be used in a pipeline
echo foo | and cat
^~^
warning: Error while reading file /tmp/broken.fish
```
Another nice one:
```
fish --no-config -c 'echo notprinted; echo foo; a=b'
```
failed to give the error message!
(Note: Is it really a "warning" if we failed to read the one file we
wer told to?)
We should check if we should either centralize these error messages
completely, or always pass them and remove this "code" system, because
it's only used in some cases.
This skipped printing a "^" line if the start or length of the error
was longer than the source.
That seems like the correc thing at first glance, however it means
that the caret line isn't skipped *if the file goes on*.
So, for example
```fish
echo "$abc["
```
by itself, in a file or via `fish -c`, would not print an error, but
```fish
echo "$abc["
true
```
would. That's not a great way to print errors.
So instead we just.. imagine the start was at most at the end.
The underlying issue why `echo "$abc["` causes this is that `wcstol`
didn't move the end pointer for the index value (because there is no
number there). I'd fix this, but apparently some of
our recursive variable calls absolutely rely on this position value.
This makes the awkward case
fish: Unexpected end of string, square brackets do not match
echo f[oo # not valid, no matching ]
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
(that `]` is simply the last character on the line, it's firmly in a comment)
less awkward by only marking the starting brace.
The implementation here is awkward mostly because the tok_t
communicates two things: The error location and how to carry on.
So we need to store the error length separately, and this is the first
time we've done so.
It's possible we can make this simpler.
This makes it so instead of marking the error location with a simple
`^`, we mark it with a caret, then a run of `~`, and then an ending `^`.
This makes it easier to see where exactly an error occured, e.g. which
command substitution was meant.
Note: Because this uses error locations that haven't been exposed like
that, it's likely to shake out weirdnesses and inaccuracies. For that
reason I've not adjusted the tests yet.
This stops us from loading the completions for e.g. `./foo` if there
is no `foo` in path.
This is because the completion scripts will call an unqualified `foo`,
and then error out.
This of course means if the script would work because it never calls
the command, we still don't load it.
Pathed completions via `complete --path` should be unaffected because
they aren't autoloaded anyway.
Workaround for #3117Fixes#9133
This was misguidedly "fixed" in
9e08609f85, which made printf error out
with any "-"-prefixed words as the first argument.
Note: This means currently `printf --help` doesn't print the help.
This also matches `echo`, and we currently don't have anything to make
a literal `--help` execute a builtin help except for keywords. Oh well.
Fixes#9132
* Disclose pager to screen height immediately
This removes that bit where we only show 4 rows at most at first,
instead we disclose between half of terminal height up to the full terminal height (but still at least 4 rows).
This results in less pressing of tab to get the other results, and
better visibility of results.
Unlike moving it to the actual top of the screen, it's not as jarring and doesn't push terminal history off-screen as much.
Fixes#2698
This used to be kept, so e.g. testing it with
fish_read_limit=5 echo (string repeat -n 10 a)
would cause the prompt and such to error as well.
Also there was no good way to get back to the default value
afterwards.
* string repeat: Don't allocate repeated string all at once
This used to allocate one string and fill it with the necessary
repetitions, which could be a very very large string.
Now, it instead uses one buffer and fills it to a chunk size,
and then writes that.
This fixes:
1. We no longer crash with too large max/count values. Before they
caused a bad_alloc because we tried to fill all RAM.
2. We no longer fill all RAM if given a big-but-not-too-big value. You
could've caused fish to eat *most* of your RAM here.
3. It can start writing almost immediately, instead of waiting
potentially minutes to start.
Performance is about the same to slightly faster overall.
This newline apparently dates back to when we required all statements to
be terminated; but our AST no longer requires that so we can remove
this. No functional change expected here.
The previous fix was reverted because it broke another scenario. Add tests
for both scenarios.
The first test exposes another problem: autosuggestions are sometimes not
recomputed after selecting the first completion with Tab Tab. Fix that too.
This makes it easy to see where the individual commands start. Perhaps we
can get rid of this once we have syntax highlighting for the commands in
the history pager, or if we add timestamps as descriptions.
Note that every change to the search field still starts a new search, from
the end of history. We could change this in future but it's unclear to me
what the expected behavior is. I don't find the traditional readline behavior
very intuitive.
This reimplements ridiculousfish/control_r which is a more future-proof
approach than #6686.
Pressing Control+R shows history in our pager and allows to search filter
commands with the pager search field.
On the surface, this works just like in other shells; though there are
some differences.
- Our pager shows multiple results at a time.
- Other shells allow to use up arrow/down arrow to select adjacent entries
in history. Shouldn't be hard to implement but the hidden state might
confuse users and it doesn't play well with up-or-search, so this is
left out.
Users might expect the history pager to use subsequence matching (fuzzy
matching) like the completion pager, however due to the history pager design it
uses substring matching. We could change this in future, however that means
we would also want to change the ordering from "reverse-chronological" to
"longest common subsequence" (e.g. what fuzzy finders do), because otherwise
a query "fis" might give this ordering:
fsck /dev/disk/by-partlabel/Linux\x20filesystem
fish
which is probably not what the user wants.
The pager shows only a small number of history items at a time. This is
because, as explained above, the history pager does not support subsequence
matching, so navigating it does not scale well.
Closes#602
The next patch wants to add state that should be reset when we clear the
pager, which will happen in this function.
This reverts commit b25b291d38.
No functional change.
The pager's rendering_needs_update() function detects some but not all
scenarios where a rendering is stale. In particular, it does not compare
the completion strings.
To make this work, we manually invalidate the pager rendering whenever we
update completion strings. The history pager needs the same functionality,
so let's move it into the pager.
No functional change.
This addresses code review feedback to not couple the purely visual
concept of cursor style with the logical concept of the selection size.
Instead this now uses a dedicated variable
`$fish_select_char_after_cursor` to determine whether to extend the
selection beyond the cursor:
* fish_select_char_after_cursor = 1 or unset -> extend selection
* all other cases -> place the selection end that the cursor
This fixes the handling of the right end of the selection. Currently the
right end is considered to be at the cursor position + 1. When using a
`block` or `underline` cursor this is arguably correct, because the
cursor has a width of 1 and spans from the current position to the next:
```
x x [x x x̲] x
```
This is incorrect though (or at least very unintuitive), when using a
`line` cursor:
```
x x [x x|x] x
```
This commit changes the strategy for determining the end of the
selection in the following way:
* If the current cursor as determined by `$fish_cursor_<bind_mode>` is
set to `line`, then a cursor width of `0` is assumed.
* In all other cases, including `block` and `underscore` as well as when
no value is set we retain the previous behavior of assuming a cursor
width of `1`.
```
x x [x x x̲] x
x x [x x|]x x
```
This change should not affect many users, because the selection is
probably used most by vi-mode users, who are also likely to use a
block cursor.
We use "c > 0" but we actually mean "c != 0". The former looks like the
other code path handles negative c. Yet if c is negative, our code would
print a single escaped byte (\xXY) which is wrong because a negative value
has "sizeof wchar_t" bytes which is at least 2.
I think on platforms with 16-bit wchar_t it's possible that we actually
get a negative value but I haven't checked.
Since the fix for #3892, this escaping style escapes
\n to \\n
as well as
\\ to \\\\
\' to \\'
I believe these two are the only printable characters that are escaped with
ESCAPE_NO_PRINTABLES.
The rationale is probably to keep the encoding unambiguous and reversible.
However that doesn't justify escaping the single quote. Probably this was
an accident, so let's revert that part.
This has the nice effect that single quotes will no longer be escaped
when rendered in the completion pager (which is consistent with other
special characters). Try it:
complete : -a "aaa\'\; aaaa\'\;" -f
Also this makes the error output of builtin bind consistent:
$ bind -e --preset \;
$ bind -e --preset \'
$ bind \;
bind: No binding found for sequence “;”
$ bind \'
bind: No binding found for sequence “'”
the last line is clearly better than the old version:
bind: No binding found for sequence “\'”
In general, the fact that ESCAPE_NO_PRINTABLES escapes the (printable)
backslash is weird but I guess it's fine because it looks more consistent to
users, even though the result is an undocumented subset of the fish language.
ESCAPE_ALL is not really a helpful name. Also it's the most common flag.
Let's make it the default so we can remove this unhelpful name.
While at it, let's add a default value for the flags argument, which helps
most callers.
The absence of ESCAPE_ALL makes it only escape nonprintable characters
(with some exceptions). We use this for displaying strings in the completion
pager as well as for the human-readable output of "set", "set -S", "bind"
and "functions".
No functional change.
When listing variables, "set" tries to escape variable names.
Since variable names cannot have special characters, this doesn't do anything.
The escaping is one of the few places that does not use ESCAPE_ALL. This has
complex behavior; let's alleviate the problem by getting rid of this call.
No functional change.
Or should we stop using it?
I'm fine with either always or never using auto-formatting but our current
way of using it only sometimes is confusing.
No functional change.
Almost all edits to our commandline are funneled through
reader_data_t::push_edit(). Notable exceptions are undo/redo (which move
across existing edits instead). Due to an oversight, undo/redo fail to
trigger commandline update hooks. Fix that.
Our behavior of triggering hooks only for the search field looks weird. I
reckon that the command line eventually catches up, but this means we trigger
some hooks redundantly. Once we figure that out we can remove the new function.
command_line_has_transient_edit tracks the actual command line, not the
pager search field. We accidentally reset it after modifying the search field
which causes unexpected behavior - the commandline added by the completion
pager remains even after I press Escape.
If the completion pager renders as
foo1 bar1 baz1 qux1
foo2 bar2 baz2
foo3 bar3 baz3
and we go backwards from "foo1" (using left arrow), we'll end up at "baz3",
not "qux1". Pretty smart!
If however we go backwards once more, nothing happens.
The root cause is that there are two different kinds of selection indices:
the one before rendering (9/qux1) and the one after we cleverly subtract
the half-filled last column (8/baz3). The backwards movement ends up
decrementing the first, so it moves from 9 to 8 and nothing changes in
the rendering.
Fix this by using the selection index that we actually rendered.
There is another caller that relies on the old behavior of using the unrendered
selection index. Make it use a dedicated overload that does not depend on
the rendering.
Otherwise realpath would add the cwd, which would be broken if fish
ever cd'd.
We could add the original cwd, but even that isn't enough, because we
need *the parent's* idea of cwd and $PATH.
Or, alternatively, what we need is for the OS to give us the actual
path to ourselves.
get_executable_path says: "This needs to be realpath'd"
So how about we do that? The only other place we use it is fish.cpp,
and we realpath it there already.
See #9085
Our pager computes the selected completion based on its rendering. The number
of rows affect the selection, in particular when moving left from the top
left cell. This computation breaks if the number of rows is zero, which
happens in at least
two scenarios:
1. If the completion pager was not shown (as is the case for complete-or-search)
2. If the search field had filtered away every candidate but not anymore.
I believe in these scenarios the selected completion index is always 0,
so let's fix the selection for that case.
Probably too minor for a changelog entry.
Closes#9080
Posix allows this as an alternative with the same semantics for read.
Found in conjunction with #9067.
Should be no functional difference on other systems.
The wait status value, which we also use internally, is read by a
bunch of macros.
Unfortunately because we want to *create* such a value, and some
systems lack the "W_EXITCODE" macro to do that, we need to figure out
how it's encoded.
So we simply check a specific value, and assume the encoding from
that.
On Haiku the return status is in the lower byte, on other systems it's
typically the upper byte.
TODO: Test on musl (that's the other system without W_EXITCODE).
Fixes#9067
This was an inadvertent change from
cc632d6ae9.
Because we used wgetcwd directly before, we always got the "physical"
resolved $PWD.
There's an argument to be made to use the logical $PWD here as well
but I prefer not to make changes lik that in a random commit without
good reason.
This can be used to print the modification time, like `stat` with some
options.
The reason is that `stat` has caused us a number of portability
headaches:
1. It's not available everywhere by default
2. The versions are quite different
For instance, with GNU stat it's `stat -c '%Y'`, with macOS it's `stat
-f %m`.
So now checking a cache file can be done just with builtins.
These are non-POSIX extensions other test(1) utilities implement,
which compares the modification time of two files as proposed for
fish in #3589: testing if one file is newer than another file.
-ef is a common extension to test(1) which checks if two paths refer
to the same file, by comparing the dev and inode numbers.
As explained by the comment, this was dead code. If it were ever executed,
it would cause very weird behavior because it would make some completions
randomly affect others.
Let's just print a warning (maybe this is better than crashing?).
Previously, the search text is used to find out which part of the
updated command line should be highlighted during a history search. This
approach will cause the incorrect part to be highlighted when the line
contains multiple instances of the search text.
To address this, we have to find out exactly where to highlight, i.e.
the offset of the current token in the command line (0 if not a token
search) plus the offset of the search text in the match.
This function is supposed to return "the next directory". Because this
is imperfect, it only tries to.
Except it went to all the trouble of figuring out the type and then
just... returned it anyway.
This has nice speedups in globs with directory components like `*/` or
`**`. I have observed 1.1x to 2.0x.
We could also return when we know it's definitely a directory and then
skip a stat() later, but preliminary testing seemed to show that's not
worth much.
Take advantage of additional cleanup unlocked by this refactoring,
including eliminating unneeded error returns and simplifying some
control flow.
No user-visible behavior change expected here.
This switches builtin_string from using PCRE2 directly, to using the new re
component. This simplifies some code and removes redundancy.
No user-visible behavior change expected here.
This migrates our PCRE2 dependency from builtin/string.cpp to new files
re.h/re.cpp, allowing regexes to be used in other places in fish.
No user-visible behavior change expected here.
This switches the flag_to_function from a map to just an ordinary switch
statement. This saves some memory/startup time and removes some
relocations. No functional change here.
This adds a line to `set --show`s output like
```
$PATH: originally inherited as |/home/alfa/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/var/lib/flatpak/exports/bin|
```
to help with debugging.
Note that this means keeping an additional copy of the original
environment around. At most this would be one ARG_MAX's worth, which
is about 2M.
It's still useful without, for instance to implement a command that
takes no options, or to check min-args or max-args.
(technically no optspecs, no min/max args and --ignore-unknown does
nothing, but that's a very specific error that we don't need to forbid)
Fixes#9006
Resolves this warning:
> warning: 'sprintf' is deprecated: This function is provided for compatibility reasons only. Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
If you run an initial command via `fish -c`, and that command is
cancelled e.g. via control-C, then ensure that the cancellation signal
is cleared before running config files.
Fixes#9024
This concerns what happens if the user types e.g. `grep --i` and grep or
its completions have not yet been loaded. Previously we would "bounce to
the main thread" from within the autosuggestion thread to load grep's
completions. However under concurrent execution, this may deadlock as the
main thread is waiting for something else.
In the new implementation, complete simply records the commands that it
would autoload, and returns them back to the caller, where the caller can
decide how to handle them.
In general iothread_perform_on_main risks deadlock under concurrent
execution and we should try to get rid of it.
There should be no user-visible change from this fix.
The last remnant of the old debug system, this was only used in
show_stackframe.
Because that's only ever called with an "E" level currently I've
removed the level argument entirely. If it's needed we'd have to pass
a flog category here.
The fix for #3481 caused us to save the screen status after external
commands were run, fixing an unnecessary abandon-line when switching
modes. But we may also run commands not directly as part of a binding,
but instead via an on-variable event, e.g. for fish_bind_mode.
Extend this fix to all bindings, guarded by changes to exec_count. Now
any time an external command runs as part of a binding we should pick up
changes to the tty and not abandon the line.
Fixes#3481 again.
In b0084c3fc4, we refactored out event handlers get removed. But this
also caused us to remove "one-shot" handlers even if they have not yet
been fired. Fix this.
This concerns running a key binding which invokes a command. If that
command modifies the tty, then fish will spot the modification later and
then react to it by redrawing the prompt. However tty modifications may
be benign or desirable; for example switching the cursor from a line to
a block. Fix this by re-fstating the tty after running external
commands.
Fixes#3481
Previously, `kill-whole-line` kills the line and its following
newline. This is insufficient when we are on the last line, because
it would not actually clear the line. The cursor would stay on the
line, which is not the correct behavior for bindings like `dd`.
Also, `cc` in vi-mode used `kill-whole-line`, which is not correct
because it should not remove any newlines. We have to introduce
another special input function (`kill-inner-line`) to fix this.
When the user adds a completion for a command, we push it to the front
of the completion list so it appears first; for that reason we don't
want to use a vector. However we can do better than std::list; try using
std::forward_list which is singly linked. No functional change here (but
we will see if this breaks any old platforms in which case it's fine to
revert this).
Prior to this change, the list of completions was stored as a
std::unordered_set, using some funny comparators and suspicious
const_cast to make it map-like. Use a real map instead, simplifying
the code. No functional change here.
Prior to this commit, setting a universal variable may trigger syncing
against the file which will modify other universal variables. But if we
want to support multiple environments we need the parser to decide when to
sync uvars. Shift the decision of when to sync to the parser itself. When a
universal variable is modified, now we just set a flag and it's up to the
(main) parser when to pick it up. This is hopefully just a refactoring with
no user-visible changes.
This makes it so `complete -c foo -n test1 -n test2` registers *both*
conditions, and when it comes time to check the candidate, tries both,
in that order. If any fails it stops, if all succeed the completion is offered.
The reason for this is that it helps with caching - we have a
condition cache, but conditions like
```fish
test (count (commandline -opc)) -ge 2; and contains -- (commandline -opc)[2] length
test (count (commandline -opc)) -ge 2; and contains -- (commandline -opc)[2] sub
```
defeats it pretty easily, because the cache only looks at the entire
script as a string - it can't tell that the first `test` is the same
in both.
So this means we separate it into
```fish
complete -f -c string -n "test (count (commandline -opc)) -ge 2; and contains -- (commandline -opc)[2] length" -s V -l visible -d "Use the visible width, excluding escape sequences"
+complete -f -c string -n "test (count (commandline -opc)) -ge 2" -n "contains -- (commandline -opc)[2] length" -s V -l visible -d "Use the visible width, excluding escape sequences"
```
which allows the `test` to be cached.
In tests, this improves performance for the string completions by 30%
by reducing all the redundant `test` calls.
The `git` completions can also greatly benefit from this.
This adds a path builtin to deal with paths.
It offers the following subcommands:
filter to go through a list of paths and only print the ones that pass some filter - exist, are a directory, have read permission, ...
is as a shortcut for filter -q to only return true if one of the paths passed the filter
basename, dirname and extension to print certain parts of the path
change-extension to change the extension to a different one (as a string operation)
normalize and resolve to canonicalize the paths in various flavors
sort to sort paths, also only using the basename or dirname as a key
The definition of "extension" here was carefully considered and should line up with how extensions are actually used - ~/.bashrc doesn't have an extension, but ~/.conf.d does (".d").
These subcommands all compose well - they can read from arguments or stdin (like string), they can use null-delimited input or output (input is autodetected - if a NULL happens in the first PATH_MAX bytes it switches automatically).
It is both a failglob exception (so like set if a glob passed to it fails it just doesn't get any arguments for it instead of triggering an error), and passes output to command substitution buffers explicitly split (like string split0) so newlines are easy to handle.
This would still remove non-existent paths, which isn't a strict
inversion and contradicts the docs.
Currently, to only allow paths that exist but don't pass a type check,
you'd have to filter twice:
path filter -Z foo bar | path filter -vfz
If a shortcut for this becomes necessary we can add it later.
This is now added to the two commands that definitely deal with
relative paths.
It doesn't work for e.g. `path basename`, because after removing the
dirname prepending a "./" doesn't refer to the same file, and the
basename is also expected to not contain any slashes.
Because we now count the extension including the ".", we print an
empty entry.
This makes e.g.
```fish
set -l base (path change-extension '' $somefile)
set -l ext (path extension $somefile)
echo $base$ext
```
reconstruct the filename, and makes it easier to deal with files with
no extension.
This means "../" components are cancelled out even after non-existent
paths or files.
(the alternative is to error out, but being able to say `path resolve
/path/to/file/../../` over `path resolve (path dirname
/path/to/file)/../../` seems worth it?)
This sorts paths by basename, dirname or full path - in future
possibly size or age.
It takes --invert to invert the sort and "--what=basename|dirname|..."
to specify what to sort
This can be used to implement better conf.d sorting, with something
like
```fish
set -l sourcelist
for file in (path sort --what=basename $__fish_config_dir/conf.d/*.fish $__fish_sysconf_dir/conf.d/*.fish $vendor_confdirs/*.fish)
```
which will iterate over the files by their basename. Then we keep a
list of their basenames to skip over anything that was already
sourced, like before.
The recent change to skip the newline for `string` changed this, and
it also hit builtin path (which is in development separately, so it's
not like it broke master).
Let's pick a good default here.
This just goes back until it finds an existent path, resolves that,
and adds the normalized rest on top.
So if you try
/bin/foo/bar////../baz
and /bin exists as a symlink to /usr/bin, it would resolve that, and
normalize the rest, giving
/usr/bin/foo/baz
(note: We might want to add this to realpath as well?)
This includes the "." in what `path extension` prints.
This allows distinguishing between an empty extension (just `.`) and a
non-existent extension (no `.` at all).
These are short flags for "--perm=read" and "--type=link" and such.
Not every type or permission has a shorthand - we don't want "-s" for
"suid". So just the big three each get one.
This is needed because you might feasibly give e.g. `path filter`
globs to further match, and they might already present no results.
It's also well-handled since path simply does nothing if given no paths.
These were officially called "--null-input", but I just used
"--null-in" everywhere, which worked because getopt allows unambiguous abbreviations.
But since *I* couldn't keep it straight and the "put" is just
superfluous, let's remove it.
This is theoretically sound, because a path can only be PATH_MAX - 1
bytes long, so at least the PATH_MAXest byte needs to be a NULL.
The one case this could break is when something has a NULL-output mode
but doesn't bother printing the NULL for only one path, and that path
contains a newline. So we leave --null-in there, to force it on.
This adds a "path" builtin that can handle paths.
Implemented so far:
- "path filter PATHS", filters paths according to existence and optionally type and permissions
- "path base" and "path dir", run basename and dirname, respectively
- "path extension PATHS", prints the extension, if any
- "path strip-extension", prints the path without the extension
- "path normalize PATHS", normalizes paths - removing "/./" components
- and such.
- "path real", does realpath - i.e. normalizing *and* link resolution.
Some of these - base, dir, {strip-,}extension and normalize operate on the paths only as strings, so they handle nonexistent paths. filter and real ignore any nonexistent paths.
All output is split explicitly, so paths with newlines in them are
handled correctly. Alternatively, all subcommands have a "--null-input"/"-z" and "--null-output"/"-Z" option to handle null-terminated input and create null-terminated output. So
find . -print0 | path base -z
prints the basename of all files in the current directory,
recursively.
With "-Z" it also prints it null-separated.
(if stdout is going to a command substitution, we probably want to
skip this)
All subcommands also have a "-q"/"--quiet" flag that tells them to skip output. They return true "when something happened". For match/filter that's when a file passed, for "base"/"dir"/"extension"/"strip-extension" that's when something about the path *changed*.
Filtering
---------
`filter` supports all the file*types* `test` has - "dir", "file", "link", "block"..., as well as the permissions - "read", "write", "exec" and things like "suid".
It is missing the tty check and the check for the file being non-empty. The former is best done via `isatty`, the latter I don't think I've ever seen used.
There currently is no way to only get "real" files, i.e. ignore links pointing to files.
Examples
--------
> path real /bin///sh
/usr/bin/bash
> path extension foo.mp4
mp4
> path extension ~/.config
(nothing, because ".config" isn't an extension.)
This teaches `--on-signal SIGINT` (and by extension `trap cmd SIGINT`)
to work properly in scripts, not just interactively. Note any such
function will suppress the default behavior of exiting. Do this for
SIGTERM as well.
s_observed_signals is used to inform the signal handler which signals may
have --on-signal functions attached to them, as an optimization. Prior to
this change it was latched: once we started observing a signal we assume we
will keep observing that signal. Make it properly increment and decrement,
in preparation for making trap work non-interactively.
Like `set` and `read` before it, `eval` can be used to set variables,
and so it can't be shadowed by a function without loss of
functionality.
So this forbids it.
Incidentally, this means we will no longer try to autoload an
`eval.fish` file that's left over from an old version, which would
have helped with #8963.
This concerns what happens if one event handler removes another, when
both are responding to the same event. Previously we had a "double lock"
where we would traverse the list twice. Now track directly in the
handler when it is removed; this simplifies the code a lot. No
functional changes expected here.
Hitting tab on "echo **" will often result in more than 256 matches.
Commit 143757e8c (Expand wildcards on tab, 2021-11-27) describes this scenario
> If the expansion would produce more than 256 items, we flash the command
> line and do nothing, since it would make the commandline overfull.
Yet we actually erase the "**" token, which seems wrong since we already
flash the command line. Fix this, at the cost of making the code a bit uglier.
I tried to write a test in tests/pexpects/wildcard_tab.py but that doesn't
seem to work because pexpect provides only a "dumb" terminal. I wonder if we
can test what we write to the screen without depending on a terminal emulator.
c4fb857dac (in 3.4.1) introduced a regression where process_exit
events would only fire once the job itself is complete. Allow
process_exit events to fire before that. Fixes#8914.
This is after we've tried to find the interpreter, so we would already
have complained about e.g. /usr/bin/pthyon not existing.
Realistically the most common case here is things that don't start
with a shebang like ELFs. Writing special extraction code here is
overkill, and I can't see a good function to do it for us.
But this should point you in the right direction.
Fixes#8938
This gets the passwd entry for $USER (if it is set). If that gives the
same uid that geteuid() gives us, we assume the data is correct.
If not, we reset $USER (and $HOME if it's empty) from the passwd value for our UID.
This allows using $USER in a prompt even if you've `su`d. Bash gets around this by having a special escape in its $PS1 DSL that checks passwd instead.
Fixes#8583
This reverts commit ccb6cb1abe.
CI fails with
/home/runner/work/fish-shell/fish-shell/src/autoload.cpp:148:1: error: function ‘autoload_t::autoload_t(autoload_t&&)’ defaulted on its redeclaration with an exception-specification that differs from the implicit exception-specification ‘’
148 | autoload_t::autoload_t(autoload_t &&) noexcept = default;
| ^~~~~~~~~~
make[2]: *** [CMakeFiles/fishlib.dir/build.make:96: CMakeFiles/fishlib.dir/src/autoload.cpp.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:369: CMakeFiles/fishlib.dir/all] Error 2
make: *** [Makefile:139: all] Error 2
Not sure what's wrong - it compiles fine on my machine. Will check later.
Even though we disable exceptions, we use noexcept in some
places to enable certain optimizations in std::vector, see
https://en.cppreference.com/w/cpp/utility/move_if_noexcept.
Some methods have noexcept only at their declaration (or only at the
definition). This will be an error when compiling with "g++ -std=c++17". Make
both signatures match.
This cleans up the path_get_path function which is used to resolve a
command name against $PATH, by removing the dependence on errno and
being explicit about which error is returned.
Should be no user-visible change here.
Curses variables like `enter_italics_mode` are secretly defined to
dereference through the `cur_term` variable. Be sure we do not read or
write these curses variables if cur_term is NULL. See #8873, #8875.
Add a regression test.
Apple's terminfo has missing support for enter_italics_mode,
exit_italics_mode, and enter_dim_mode. Previously we would hack in such
support in set_color; migrate that to init_curses so we do it up-front
instead of opportunistically.
To recap, this means `&` in the middle of a word no longer
backgrounds.
So:
```fish
echo foo&bar # prints foo&bar
echo foo& bar # backgrounds an echo that prints "foo" and runs "bar"
```
This can no longer be changed. If "no-stderr-nocaret" is in
$fish_features it will simply be ignored.
The "^" redirection that was deprecated in fish 3.0 is now gone for good.
Note: For testing reasons, it can still be set _internally_ by running
"feature_flags_t::set". We simply shouldn't do that.
If we get an E2BIG while executing a process, we check how large the
exported variables are. We already did this, but then immediately
added it to the total.
So now we keep the tally just for the variables around, and if it's
over half (which is an atypical value if your system has an ARG_MAX of
2MB), we mention that in the error.
Figuring out which variable is too big (in case it's just one) is probably too complicated,
but we can at least complain if things seem suspect.
Untested because I don't know *how* to do so portably
Prior to this change, if you tab-completed a token with a wildcard (glob), we
would invoke ordinary completions. Instead, expand the wildcard, replacing
the wildcard with the result of expansions. If the wildcard fails to expand,
flash the command line to signal an error and do not modify it.
Example:
> touch file(seq 4)
> echo file*<tab>
becomes:
> echo file1 file2 file3 file4
whereas before the tab would have just added a space.
Some things to note:
1. If the expansion would produce more than 256 items, we flash the command
line and do nothing, since it would make the commandline overfull.
2. The wildcard token can be brought back through Undo (ctrl-Z).
3. This only kicks in if the wildcard is in the "path component
containing the cursor." If the wildcard is in a previous component,
we continue using completions as normal.
Fixes#954.
When fish expands a string that starts with a tilde, like `~/stuff/*`, it
first must resolve the tilde (e.g. to the user's home directory) before
passing it to wildcard expansion. The wildcard expansion will produce full
paths like `/home/user/stuff/file`. fish then "unexpands" the home directory
back to a tilde.
Previously this was only used during completions, but in the next commit
we plan to use it for string expansions as well.
Rationalize this behavior by adding an explicit flag to request it and
explain some subtleties about completions.
This commit was problematic for a few reasons:
1. It silently changed the behavior of argparse, by switching which
characters were replaced with `_` from non-alphanumeric to punctuation.
This is a potentially breaking change and there doesn't appear to be any
justification for it.
2. It combines a one-line if with a multi-line else which we should try
to avoid.
This reverts commit 63bd4eda55.
This reverts commit 4f835a0f0f.
These macros were historically used only in internal error messages which
should never happen! Now we are able to enforce they never happen at
compile time so we can remove them.
No functional change here.
If we ever need any of these... they're in this commit:
fish_wcswidth_visible()
status_cmd_opts_t::feature_name
completion_t::is_naturally_less_than()
parser_t::set_empty_var_and_fire()
parser_t::get_block_desc()
parser_keywords_skip_arguments()
parser_keywords_is_block()
job_t::has_internal_proc()
fish_wcswidth_visible()
When you do
```fish
set foo-bar baz
```
"foo-baz" isn't usable as a variable *name*. When you just say the
"variable" is invalid that could also be interpreted to be a special
type of variable or something.
String tokens are subdivided by command substitutions. Some syntax errors
can occur in the gap between two command substitutions. Make the caret point
to the start of that gap, instead of the token start.
When expanding command substitutions, we use a naïve way of detecting whether
the cmdsub has the optional leading dollar. We check if the last character was
a dollar, which breaks if it's an escaped dollar. We wrongly expand
\$(echo "") to the empty string. Fix this by checking if the dollar was escaped.
The parse_util_* functions have a bunch of output parameters. We should
return a parameter bag instead (I think I tried once and failed).
Given
set var a
echo "$var$(echo b)"
the double-quoted string is expanded right-to-left, so we construct an
intermediate "$varb". Since the variable "varb" is undefined, this wrongly
expands to the empty string (should be "ab"). Fix this by isolating the
expanded command substitution internally. We do the same when handling
unquoted command substitutions.
Fixes#8849
1. Bravely use a real enum for has_arg, despite the warnings.
2. Use some C++11 initializers so we don't have to pass an int for this
parameter.
No functional change expected here.
If the history file is larger than 4GB on a 32 bit system, fish will
refuse to read it. However the check was incorrect because it cast the
file size to size_t, which may be 32 bit. Switch to using uint64.
fd_monitor is used when an external command pipes into a buffer, e.g. for
command substitutions. It monitors the read end of the external command's
pipe in the background, and fills the buffer as data arrives. fd_monitor is
multiplexed, so multiple buffers can be monitored at once by a single
thread.
It may happen that there's no active buffer fill; in this case fd_monitor
wants to keep its thread alive for a little bit in case a new one arrives.
This is useful for e.g. handling loops where you run the same command
multiple times.
However there was a bug due to a refactoring which caused fd_monitor to
exit too aggressively. This didn't affect correctness but it meant more
thread creation and teardown.
Fix this; this improves the aliases.fish benchmark by about 20 msec.
No need to changelog this IMO.
This was already apparently supposed to work, but didn't because we
just overrode errno again.
This now means that, if a correctly named candidate exists, we don't
start the command-not-found handler.
See #8804
This used to call exec_subshell, which has two issues:
1. It creates a command substitution block which shows up in a stack
trace
2. It does much more work than necessary
This removes a useless "in command substitution" from an error message
in an autoloaded file, and it speeds up autoloading a bit (not
measurable in actual benchmarks, but microbenchmarks are 2x).
We need special handling when reporting backtraces for commands run
during startup, i.e. config.fish. Previously we had a global variable;
make it local to the parser to eliminate a global.
No functional change here.
Cancellation groups were meant to reflect the following idea: if you ran a
simple block:
begin
cmd1
cmd2
end
then under job control, cmd1 and cmd2 would get separate groups; however if
either exits due to SIGINT or SIGQUIT we also want to propagate that to the
outer block. So the outermost block and its interior jobs would share a
cancellation group. However this is more complex than necessary; it's
sufficient for the execution context to just store an int internally.
This ought not to affect anything user-visible.
Currently, when a variable like $fish_color_command is set but empty:
set -g fish_color_command
what happens is that highlight parses it and ends up with a "normal"
color.
Change it so instead it sees that the variable is empty and goes
on to check the fallback variable, e.g. fish_color_normal.
That makes it easier to make themes that override variables.
This means that older themes that expect an empty variable to be
"normal" need to be updated to set it to "normal".
Following from this, we could make writing .theme files easier by no
longer requiring them to list all variables with specific values.
Either the theme reader could be updated to implicitly set known color
variables to empty, or the themes could feature empty values.
See #8787.
fish reads the tty modes at startup, and tries to restore them to the
original values on exit, to be polite. However this causes problems when
fish is run in a pipeline with another process which also messes with the
tty modes. Example:
fish -c 'echo foo' | vim -
Here vim's manipulation of the tty would race with fish, and often vim
would end up with broken modes.
Only restore the tty if we are interactive. Fixes#8705.
This is a big cleanup to how tty transfer works. Recall that when job
control is active, we transfer the tty to jobs via tcsetpgrp().
Previously, transferring was done "as needed" in continue_job. That is, if
we are running a job, and the job wants the terminal and does not have it,
we will transfer the tty at that point.
This got pretty weird when running mixed pipelines. For example:
cmd1 | func1 | cmd2
Here we would run `func1` before calling continue_job. Thus the tty
would be transferred by the nested function invocation, and also restored
by that invocation, potentially racing with tty manipulation from cmd1 or
cmd2.
In the new model, migrate the tty transfer responsibility outside of
continue_job. The caller of continue_job is then responsible for setting up
the tty. There's two places where this gets done:
1. In `exec_job`, where we run a job for the first time.
2. In `builtin_fg` where we continue a stopped job in the foreground.
Fixes#8699
This is a cleanup of job groups, rationalizing a bunch of stuff. Some
notable changes (none user-visible hopefully):
1. Previously, if a job group wanted a pgid, then we would assign it to the
first process to run in the job group. Now we deliberately mark which
process will own the pgroup, via a new `leads_pgrp` flag in process_t. This
eliminates a source of ambiguity.
2. Previously, if a job were run inside fish's pgroup, we would set fish's
pgroup as the group of the job. But this meant we had to check if the job
had fish's pgroup in lots of places, for example when calling tcsetpgrp.
Now a job group only has a pgrp if that pgrp is external (i.e. the job is
under job control).
* Turn on default bindings for --no-config mode
The fallback bindings are super awkward to use.
This was called out specifically in #7921, I'm going for the targeted
fix for now.
* Only change keybindings when interactive
That's also when we'd source them normally.
These were changed in fish 3.0 in December 2018.
This means upgrading from fish 2.7.1 or earlier to the next fish
version will require users to set their universal variable again.
This just defines a constant to whichever tparm implementation we're
using (either the actual, working one the system provides, or our
kludge to paper over Solaris' inadequacies).
This means that there won't be so much ping-ponging of what "tparm"
stands for. "tparm" is the system's function. Only we don't use it,
just like we don't use wcstod directly.
Fixes#8780
* New -n flag for string join command.
This is an argument that excludes empty result items. Fixes#8351
* New documentation for string-join.
The new argument --no-empty was added at string-join manpage.
* New completions for the new -n flag for string join.
* Remove the documentation of the new -n flag of string join0
The reason to remove this new argument in the join0 is that this flag basically doesn't make any difference in the join0.
* Refactor the validation for the string join.
The string join command was using the length of the argument, this commit changes the validation to use the empty function.
* Revert #4b56ab452
The reason for the revert is thath the build broke on the ubuntu in the Github actions.
* Revert #e72e239a1
The reason the compilation on GitHub broke is that the test was weird, it didn't even run it, Common CI systems are typically very very resource-constrained.
* Resolve conflicts in the string-join.rst.
* Resolve conflicts in the "string-join.rst".
commit #1242d0fd7 not fixed all conflicts.
This is supposed to detect color escape sequences, to figure out how
long an escape sequence is, for use in width calculations.
However, the typical color sequences are already taken care of by
is_csi_style_escape_seq because they look like a csi sequence starting
with `\e[` and ending in `m`.
In the entire terminfo database shipped with ncurses 6.3, these are
the terminals that have non-csi color sequences:
at-color
atari-color
atari_st-color
d220-dg
d230-dg
d230c-dg
d430-dg
d430-unix
d430-unix-25
d430-unix-s
d430-unix-sr
d430-unix-w
d430c-dg
d430c-unix
d430c-unix-25
d430c-unix-s
d430c-unix-sr
d430c-unix-w
d470-dg
d470c-dg
dg+fixed
dgmode+color
dgmode+color8
dgunix+fixed
emu
fbterm
i3164
ibm3164
linux-m1b
linux-m2
minitel1
minitel1b
putty-m1b
putty-m2
st52-color
tt52
tw52
tw52-color
xterm-8bit
Most of these were discontinued in the 90s and their manufacturers no
longer exist (like Data General, which went defunct in 1999). The last one is a special mode for xterm that is
fundamentally UTF-8 incompatible because it encodes a CSI as \X9b.
The linux/putty m1b and m2 entries (also for minitel) don't support
color to begin with and the sequences they have in their terminfo
entries are control characters anyway, so the calculation would still
add up.
In turn, what we gain from this is much faster width calculations with
unrecognized escapes -
e.g. `string length -V \efoo` is sped up by a factor of 20.
An alternative would be to skip this if max_colors is > 16 as that is
the most any of these entries can do. The runtime scales linearly with
the number of colors so on those systems it would be reasonably quick anyway.
But given just *how* outdated these are I believe it is okay to just
remove support outright. I do not believe anyone has ever run fish on
any of these.
* Implement fish_wcstod_underscores
* Add fish_wcstod_underscores unit tests
* Switch to using fish_wcstod_underscores in tinyexpr
* Add tests for math builtin underscore separator functionality
* Add documentation for underscore separators for math builtin
* Add a changelog entry for underscore numeric separators
We can't always read in chunks because we often can't bear to
overread:
```fish
echo foo\nbar | begin
read -l foo
read -l bar
end
```
needs to have the first read read `foo` and the second read `bar`. So
here we can only read one byte at a time.
However, when we are directly redirected:
```fish
echo foo | read foo
```
we can, because the data is only for us anyway. The stream will be
closed after, so anything not read just goes away. Nobody else is
there to read.
This dramatically speeds up `read` of long lines through a pipe. How
much depends on the length of the line.
With lines of 5000 characters it's about 15x, with lines of 50
characters about 2x, lines of 5 characters about 1.07x.
See #8542.
This is the simple fix - if we have no valid digit, we have nothing to
return. So instead of returning a NULL, we return an error.
This is already the case for invalid octal escapes (like `\777`).
Fixes#8545
This reverts commits:
2d9e51b43ed1d9f147ec346ce8081b
The box drawing because it's entangled with the rest and we don't
currently use this anywhere I know of. Nor was it gated on terminfo,
so it could have broken things, for subjectively little gain.
Fixes#8727.
This updates widechar_width.h to one generated from
15e782aa3df9dfef436516f66f745a90b421329.
The change here is a rationalization of doublewide vs widened-in-9.
Many emoji have been moved to widened-in-9 because we now use the
correct version (this uses the *emoji* version, and emoji version 3.0
corresponds to Unicode 9).
The new --escape option means that -C is not necessarily the last option;
We have this scenario where we produce a bogus error
$ fish -c 'complete -C --escape'
complete: --escape: option requires an argument
--escape doesn't take arguments, so let the error message say -C.
A history search ends when you move the cursor, but the commandline inserted by
history search is still marked as transient. This means that the next history
search will clear the transient commandline. This means we are dropping an undo
point, for example:
echo 11
echo 1
echo autosuggestion
echo^P # commandline is "echo 1"
^A # stop history search
^P # commandline is "echo 11"
^Z # Bug: commandline goes back to "echo", but it should be "echo 1"
In the worst case, we are switching from line-search to token-search (see
the attached test case). Clearing the transient edit means the line is gone
and only the token is left on the command line.
fish outputs the result of fish_title inside an escape sequence, which
happens to be terminated by \a (BEL). It may happen that the initial
output is interrupted; fish then emits the closing BEL and that makes an
annoying beep. Output the fish_title all at once, even if a signal is
delivered (so we don't get "stuck inside" the sequence).
This is related to #8628 in that it's a "torn escape sequence."
Say the user has a multi-char binding (typically an escape sequence), and a
signal arrives partway through the binding. The signal has an event handler
which enques some readline event, for example, `repaint`. Prior to this
change, the readline event would cause the multi-char binding to fail. This
would cause bits of the escape sequence to be printed to the screen.
Fix this by noticing when a sequence was "interrupted" by a non-char event,
and then rotating a sequence of such interruptions to the front of the
queue.
Fixes#8628
readch_timed is called after reading the escape character \x1b. The escape
char may be a standalone key press or part of an escape sequence; fish
waits for a little bit (per the fish_escape_delay_ms variable) to see if
something else arrives, before treating it as standalone escape-key press.
It may happen that a signal is delivered while fish waits. Prior to this
change we would treat this signal as a "nothing was read" event, causing
escape to be wrongly treated as standalone.
Avoid this by using pselect() with a full signal mask, to ensure this call
completes.
check_exit events are generated to give the reader a chance to respond to
commands, or otherwise to return control to the reader loop. Prior to this
change they were being passed to match key bindings. This is useless since
no key binding can match a check_exit event. FLOG noisily complains about
unmatched events. So just don't pass these to mapping_execute.
The previous commit added transient commandlines when completing
commands with variable overrides. Transient commandlines require a
parser, but perform_one_completion_cd_test() asked for completions
without giving a parser, which is only okay when asking for
autosuggestions (like perform_one_autosuggestion_cd_test() does).
Let's pass a parser to fix the test.
Today, a command like "var=val status " has custom completions
because we skip over the var=val variable override when detecting
the command token.
However if the custom completions read the commandline state (via
"commandline -opc") they do see they variable override, which breaks
them, most likely. Try "a=b git ".
For completions of wrapped commands, we already set a transient
commandline. Do the same for commands with leading variable overrides;
then git completions for "a=b git " will think the commandline is
"git ".
This updates widecharwidth to
6d3d55b419db93934517cb568d1a3d95909b4c7b, which includes the same
Hangul Jamo check in a separate table.
This should slightly speed up most width calculation because we no
longer need to do it for most chars, including the overwhelmingly
common ascii ones.
Also the range is increased and should better match reality.
`read` allows specifying the initial command line text. This was
text got accidentally ignored starting in a32248277f. Fix this
regression and add a test.
Fixes#8633
Both constant values and functions are represented as `te_fun_t`.
This struct defines `operator()` which evaluates the function with the
given arguments.
Previously, when we got an unknown option with --ignore-unknown, we
would increment woptind but still try to read the same contents.
This means in e.g.
```
argparse -i h -- -ooo -h
```
The `-h` would also be skipped as an option, because after the first
`-o` getopt reads the other two `-o` and skips that many options.
This could be handled more extensively in wgetopt, but the simpler fix
is to just skip to the next argv entry once we have an unknown option
- there's nothing more we can do with it anyway!
Additionally, document this and clearly explain that we currently
don't transform the option.
Fixes#8637
The sole notifiers test recreated the uvar directory, so if it was
called while the universal test was running it would stop it from
completing correctly.
This happened reasonably often on Ubuntu with tsan on Github Actions.
Cygwin tests are failing because cygwin has a low limit of only 64 fds in
select(). Extend select_wrapper_t to also support using poll(), according to
a FISH_USE_POLL new define. All systems now use poll() except for Mac.
Rename select_wrapper_t to fd_readable_set_t since now it may not wrap
select().
This allows the deep-cmdsub.fish test to pass on Cygwin.
This affects the caret position. In an expression like
123 456
we previously reported:
123 456
^ missing operator
Now we do:
123 456
^ missing operator
We do it on the first space, which should be acceptable.
(no need for a changelog entry, we have already ignored #8511)
This unit test was passing 0 instead of a pointer to indicate the end of
a varargs; this might fail on 64 bit, and indeed did fail on Cygwin. This
fixes the Cygwin expand test.
Only show the shebang warning for .fish commands.
Use the phrase "interpreter directive" as the formal name for the
shebang.
Switch from windows to Windows for the operating system.
There is no undefined behavior in closing a moved pipe, since the
move constructor simply sets the fd to -1, which is ignored by close().
The move constructor of autoclose_fd_t is "fully specified" (like
unique_ptr).
It's good practice to eagerly close pipes which may be inherited by
child processes, since otherwise the writer may not get EPIPE correctly.
Closing the pipe explicitly makes it clear that the pipe does not stay
open across continue_job().
This reverts commit c014c23662.
"not not return 34" exits with 34, not 1. This behavior is pretty
surprising but benign. I think it's very unlikely that anyone relies
on the opposite behavior, because using two "not" decorators in one
job is weird, and code that compares not's raw exit code is rare.
The behavior doesn't match our docs, but it's not worth changing the
docs because that would confuse newcomers. Add a test to cement the
behavior and a comment to explain this is intentional.
I considered adding the comment at
parse_execution_context_t::populate_not_process where this behavior
is implemented but the field defintion seems even better, because I
expect programmers to read that first.
Closes#8377
When we execute something and it doesn't have a shebang, typically we
fall back on running it with /bin/sh. For .fish scripts, we still
refuse to do this (assuming that /bin/sh won't handle .fish scripts properly).
Only the error wasn't great. So we now explicitly mention when there's
a missing shebang, and point towards the shebang line otherwise.
Seems like size_t is unnecessarily large as well, as elsewhere
in the code we are clamping down to uint32_t / source_offset_t.
This makes tok_t more like 16 bytes. More cleanup seems desirable,
this is not very well hamrnoized across our code base.
Instead of 7a80ad74f, which adds ifdeffery, we simply drop the
variables we don't care about. This leaves two presumably
glibc-specific variables, but drops 5 variables like LC_MONETARY, so
it's overall a win.
This reverts commit 7a80ad74f4.
The builtin history delete call has some code that removes a leading and
trailing quote from its arguments. This code dates back to ec34f2527a,
when the builtin was introduced. It seems wrong and tests pass
without it. Let's bravely remove it.
Use the remaining_to_disclose count to determine if all completions
are shown (allows consistent behavior between short and long completion
lists).
Closes#8485
Commit e40eba358 (Treat text following quoted command substitution
as quoted) made parse_util_locate_cmdsubst_range() aware of quoted
command substitutions, by skipping surrounding text via quote_end().
However, it was not quite right. We fail to properly parse
two consecutive command substitutions in the same string,
because we don't maintain the quoting context across calls to
parse_util_locate_cmdsubst_range(). Let's track that bit in a
parameter. This allows us to get rid of the quote_end() hack.
Also apply this to the other place where we call
parse_util_locate_cmdsubst_range() in a loop (highlighting).
Fixes#8500
This fixes a regression about where we report errors:
echo error(here
old: ^
fixed: ^
Commit 0c22f67bd (Remove the old parser bits, 2020-07-02) removed
uses of "error_offset_within_token" so we always report errors at
token start. Add it back, hopefully restoring the 3.1.2 behavior.
Note that for cases like
echo "$("
we report "unbalanced quotes" because we treat the $( as double
quote. Giving a better error seems hard because of the ambguity -
we don't know if quote is meant to be inside or outside the command
substitution.
If you make a script called `foo` somewhere in $PATH, and did not give
it a shebang, this would end up calling
sh foo
instead of
sh /usr/bin/foo
which might not match up.
Especially if the path is e.g. `--version` or `-` that would end up
being misinterpreted *by sh*.
So instead we simply pass the actual_cmd to sh, because we need it
anyway to get it to fail to execute before.
I think the auto-all-the-things here was a making this a little
hard to follow, so replace these things that will be used in printf
with what they really are. And change the * lengths to ints.
should clear up the alerts.
When the completion pager fills up all lines of the screen, we subtract
from the pager size the number of lines occupied by the prompt +
command line buffer (typically 1), so the command line is always
visible. However, we only subtract the number of lines *before* the
cursor, so on some multiline commandlines we draw a pager that is
too large for our screen, clobbering the commandline rendering.
Fix this by counting all lines.
Fixes#8509
Possibly fixes#8405
As seen in
https://stackoverflow.com/questions/70139844/how-to-execute-custom-fish-scripts-in-custom-path-folder,
making a shebang like
#!usr/bin/fish
won't work, and will error with the default "file does not exist"
error *pointing to the file, not the interpreter*.
Detect that interpreter properly.
We might want to make this an even more specific error, but now it
says
```
exec: Failed to execute process '/home/alfa/.local/bin/borken.fish': The file specified the interpreter 'usr/bin/fish', which is not an executable command.
```
Which is okay.
A command like "printf nonewline | sed s/x/y/" does not print a
concluding newline, whereas "printf nnl | string replace x y" does.
This is an edge case -- usually the user input does have a newline at
the end -- but it seems still better for this command to just forward
the user's data.
Teach most string subcommands to check if stdin is missing the trailing
newline, and stop adding one in that case.
This does not apply when input is read from commandline arguments.
* Most subcommands stop adding the final newline, because they don't
really care about newlines, so besides their normal processing,
they just want to preserve user input. They are:
* string collect
* string escape/unescape
* string join¹
* string lower/upper
* string pad
* string replace
* string repeat
* string sub
* string trim
* string match keeps adding the newline, following "grep". Additionally,
for string match --regex, it's important to output capture groups
separated by newlines, resulting in multiple output lines for an
input line. So it is not obvious where to leave out the newline.
* string split/split0 keep adding the newline for the same reason --
they are meant to output multiple elements for a single input line.
¹) string join0 is not changed because it already printed a trailing
zero byte instead of the trailing newline. This is consistent
with other tools like "find -print0".
Closes#3847
A completion entry like «complete -a '\\~'» results in completions
that insert \~ into the command line. However we usually want to
insert ~, but there is no way to do that.
There are a couple of longstanding issues about completion escaping
[1]. Until we fix those in a general way, fix the common case by
never escaping tildes when applying custom completions to the command
line. This is a hack but will probably work out fine because we don't
expect literal tildes in arguments.
The tilde is included in completions for cdh, or
__fish_complete_suffix, which simply forwards results from "complete
-C". Revert a workaround to cdh that expanded ~, because we can now
render that without escaping.
Closes#4570, #8441
[ja: tweak patch and commit message]
[1]: https://github.com/fish-shell/fish-shell/pull/8441#discussion_r748803338
A «complete -C '~/fish-shell/build/fish '» fails to load custom
completions because we do not expand the ~, so
complete_param_for_command() thinks that this command is invalid.
Expand command tokens before loading custom completions.
Fixes#8442
Currently,
set -q --unpath PATH
simply ignores the "--unpath" bit (and same for "--path").
This changes it, so just like exportedness you can check pathness.
* fish_key_reader: Simplify default output
It now only prints the bind statement. Timing information and such is
relegated to a separate "verbose" mode.
* Adjust fish_key_reader docs
* Adjust tests
What this did was
1. Find directory
2. Turn name into wcstring and return it
3. Turn name back into string for some operations
Instead, let's unglue the wcstringing from this, return the narrow
string and then widen it when we need.
This didn't even mention that it was a script file, it was just
filename: File not found
Which would be rather confusing if e.g. someone forgot that
`--profile` requires an argument.
This finds the first broken component, to help people figure out where
they misspelt something.
E.g.
```
echo foo >/usr/lob/systemd/system/machines.target.wants/var-lib-machines.mount
```
will now show:
```
warning: Path '/usr/lob' does not exist
```
which would help with seeing that it should be "/usr/lib".
Commit fe63c8ad3 (Shadow/override iswdigit instead of changing it at
individual call sites, 2021-10-04) added our own implementation of
iswdigit() to common.h. The "include-what-you-use" rule means that
files that use iswdigit() should now include common.h. Do that.
A variable may be broken across multiple lines with a backslash, for
example:
> echo $FISH_\
VERSION
Teach syntax highlighting about this line breaking. Fixes#8444
check_global_scope_exists is meant to warn if the user creates a
universal variable shadowing a global. In practice it always returned
success (though it may print an error). Remove its return value and
clean up the call sites. Also rename it to
`warn_if_uvar_shadows_global`. No functional change in this commit.
On a commandline like "ls arg" (cursor at end) we do not expand
abbrevations on enter. OTOH, on "ls " we do expand. This can be
frustrating because it means that the two obvious ways to suppress
abbrevation expansion (C-Space or post-expansion C-Z) cannot be used to
suppress expansion of a command without arguments. (One workaround is
"ls #".)
Only expand-on-execute if the cursor is at the command name (no space
in between).
This is a strict improvement for realistic scenarios, because if there
is a space, the user has already expressed the intent to not expand
the abbreviation. (I hope no one is using recursive abbreviations.)
Closes#8423
This allows to disable autosuggestions in config or with
fish -C 'set -g fish_autosuggestion_enabled 0'
instead of only in existing interactive sessions.
I'm not sure if passing the env var table is actually necessary here,
since we already have a reader.
This allows rebinding escape in the user list without breaking e.g.
arrow keys (which send escape and then `[A` and similar, so escape is
a prefix of them).
Fixes#8428.
+ No functional change here, just renames and #include changes.
+ CMake can't have slashes in the target names. I'm suspciious of
that weird machinery for test, but I made it work.
+ A couple of builtins did not include their own headers, that
is no longer the case.
Very slight performance increase (1% when parsing *all .fish scripts
in fish-shell*), but this removes a useless variable and some
.c_str()inging.
Theoretically it should also remove some wcslen() calls, but those
seem to be optimized out?
This fixes printing octal and hex values that are negative or larger
than UINT_MAX.
Negative values get a leading -, like:
> math --base hex -10
-0xa
Fixes#8417.
(or use the correct specifiers for the type if we can.)
These are hard to track down because we can't get compile-time
warnings for the wprintf family of in libc like is possible for
the narrow versions.
- Introduce BUILTIN_ERR_COMBO2_EXCLUSIVE
- Distill generally more terse, unambiguous error descriptions.
Remember English is not everyone's language.
- Do not capitalize sentence fragments
- Use the modality where problem input is in a %s: prefix, then
is explained.
- Do not address the user (the "You cannot do ..." kraderism)
- Spell out 'arguments' rather than 'args' for consistency
- Mention 'function' as a scope
Watching for exit events is rare, so check if we have any exit events
before actually emitting them. This saves about 2% of time in
external_cmds benchmark.
This untangles some of the complicated logic and loops around posting
job exit events, and invoking the fish_job_summary function. No
functional change here (hopefully).
Prior to this change, job_t::is_stopped() returned true if there were
zero running processes in the job. This meant that completed jobs were
reported as stopped. Stop doing this, it's a footgun.
Exited processes generate event_t::process_exit if they exit with a
nonzero status. Prior to this change, to avoid sending duplicate events,
we would clear the status. This is ugly since we're lying about the
process exit status. Use a real flag to prevent sending duplicate
notifications.
This adds a variable, $fish_autosuggestion_enabled.
When set to 0, it will turn off autosuggestions/highlighting.
Setting it to anything else will enable it (which also
means this remains enabled by default).
