Commit Graph

3777 Commits

Author SHA1 Message Date
ridiculousfish
7e77dc8964 Add a test for round-tripping characters in the private use area
I wrote this test believing that fish had a bug, but actually fish is
behaving correctly here. Still the test is nice so I am checking it in.
2021-02-17 12:29:51 -08:00
ridiculousfish
c0b39aaeb7 Do not reset terminal color when donating term for running key bindings
fish maintains two tty modes: one for itself and one for external
commands. The external command mode is also used when executing
fish-script key bindings, which was added in 5f16a299a7 (note that
commit had the wrong issue, the correct issue is #2114).

Prior to this fix, when switching to external modes, we would also reset
the tty's foreground color. This bumped tty's timestamp, causing us to
believe that the tty had been modified, and then repainting the prompt. If
the prompt were multi-line, we would repaint the whole prompt starting
from its second line, leaving a trailing line above it.

It would be reasonable to save the tty timestamp after resetting the
color, but given that using external modes for keybindings is new, it's
better to instead not reset the color in this case. So migrate the color
resetting to only when we run external commands.

Fixes #7722
2021-02-17 11:10:51 -08:00
Mahmoud Al-Qudsi
75af89699a Patch fish_tests to work with changed const_strlen requirements 2021-02-17 12:55:21 -06:00
Mahmoud Al-Qudsi
70b06a899d Bypass recursion in case of single trailing nul in const_strlen()
The default case for string literals like `"foo"` is a single trailing
nul, and that's what we have almost everywhere. By checking the
second-to-last index for a non-nul byte, we can skip the recursive
invocation, thus speeding up compilation that teeny, tinsy bit faster.
2021-02-17 12:26:40 -06:00
Mahmoud Al-Qudsi
071b9b13cd Reverse const_strlen() recursion logic
Rather than making the run-time complexity of the algorithm 𝒪(n) where n
is the length of the string, make it 𝒪(k) where k is the number of
trailing nul bytes.

The second parameter `index` with a default non-value is in lieu of a
helper function that would have had a name like `count_trailing_nuls()`.
2021-02-17 12:26:40 -06:00
Fabian Homborg
313b70a0c2 math: Set LC_NUMERIC to C again
e94f86e6d2 removed it in favor of using
fish_wcstod, but this broke the *output* - math currently prints
numbers with "," and then can't read them.

So we partially revert it until we come up with something better.
Maybe set $LC_NUMERIC globally inside fish?
2021-02-17 09:07:40 +01:00
Johannes Altmanninger
c0af8dae20 fish_indent: fix extra indent of continuation lines inside blocks
fish_indent used to increment the indentation level whenever we saw an escaped
newline.  This broke because of recent changes to parse_util_compute_indents().
Since parse_util_compute_indents() function already indents continuations
there is not much to do for fish_indent - we can simply query the indentation
level of the newline.  Reshuffle the code since we need to pass the offset
of the newline. Maybe this can even be simplified further.

Fixes #7720
2021-02-16 18:39:03 +01:00
Johannes Altmanninger
444c05dfb1 Do not indent after escaped newline in comment
We do something similar in fish_indent.  This fixes the spurious indent
after comments in share/completions/emerge.fish.

See #7720
2021-02-16 18:39:03 +01:00
Fabian Homborg
00fabb4fe1 Enhance fallback keybindings a bit
Bind \cc like normal, since we now no longer use a function, and bind
some important control bindings like \cs and the ever-important emacs \cb/f/p/n.

What really kills the usability here is the up-line vs up-or-search.
2021-02-15 20:54:35 +01:00
Fabian Homborg
65a760528a Skip figuring out config with --no-execute
Cuts the time to check all our fish scripts in the
check-all-fish-files.fish test roughly in half, from 3.3s
to 1.8s.
2021-02-15 18:59:54 +01:00
Fabian Homborg
c83436d050 Don't overwrite IXON/IXOFF for startup modes
We actually restore those before exit, so this would force-disable
flow control whenever fish exits.
2021-02-15 17:30:50 +01:00
Fabian Homborg
1426d61bc9 Stop disabling VSTOP/VSTART
Without flow control enabled these won't be interpreted any way.
2021-02-15 17:30:50 +01:00
Fabian Homborg
2be720b6cc Set flow control settings also in the shell
Since, unlike e.g. OPOST, this can sometimes be useful, just copy
whatever flow control settings the terminal ends up with.

We still *default* flow control to off (because it's an awful default
and allows us to bind ctrl-s), but if the user decides to enable it so
be it.

Note that it's _possible_ flow control ends up enabled accidentally, I
doubt this happens much and it won't render the shell unusable (and
good terminals might even tell you you've stopped the app).

Fixes #7704
2021-02-15 17:30:50 +01:00
Fabian Homborg
c27fb9b802 source: Escape filenames in errors
Otherwise this would look weird if you had, say, a tab in there.

See #7716.

(note that this doesn't handle e.g. zero-width-joiners, because those
aren't currently escaped. we might want to add an escape mode for
unprintable characters, but for combining codepoints that's tricky!)
2021-02-15 17:08:26 +01:00
Mahmoud Al-Qudsi
ef5db47cf7 Make const_strlen return an unsigned value 2021-02-14 11:42:26 -06:00
Fabian Homborg
552c7bc40b Don't add a space if completion only added a single "/"
This added a space if only one character was added, e.g.

```fish
cd dev<TAB>
```

would complete to

```fish
cd dev/<SPACE>
```

which makes picking deeper directories awkward.

So just go back to the old behavior of doing it for any length.

This is a regression from e27d97b02e.

cc @krobelus
2021-02-14 13:50:43 +01:00
ridiculousfish
9a165b93fb handle_builtin_output to take io_chain by const reference
There was no reason for this to be a pointer or mutable.
2021-02-13 20:05:33 -08:00
ridiculousfish
ee15bc2a36 Add a variant of valid_var_name which accepts const wchar_t *
This avoids creating some unnecessary strings.
2021-02-13 18:49:43 -08:00
Johannes Altmanninger
5b3a466fa9 Refactor: collapse if statements 2021-02-13 09:01:41 +01:00
Johannes Altmanninger
5e8a248758 Indent escaped newlines
Similar to what fish_indent does. After typing "echo \" and hitting return,
the cursor will be indented.

A possible annoyance is that when you have multiple indented lines

	echo 1 \
	    2 \
	    3 \
	    4 \

If you remove lines in the middle with Control-k, the lines below
the deleted one will start jumping around, as they are disconnected
from and reconnected to "echo".
2021-02-13 09:01:41 +01:00
Johannes Altmanninger
511747d59e Indent only leaf nodes and in-between gaps
Probably not necessary for the next commit, but this way feels more logical
2021-02-13 09:01:41 +01:00
Johannes Altmanninger
7ee4a3b40d Indent empty lines inside block 2021-02-13 09:01:41 +01:00
Johannes Altmanninger
fffcdf8792 Highlight redirection target as valid if it contains a to-be-defined variable
If a variable is undefined, but it looks like it will be defined by the
current command line, assume the user knows what they are doing.
This should cover most real-world occurrences.

Closes #6654
2021-02-13 08:59:54 +01:00
Ethel Morgan
5a0aa7824f Saturate exit codes to 255 for all builtins
After commit 6dd6a57c60, 3 remaining
builtins were affected by uint8_t overflow: `exit`, `return`, and
`functions --query`.

This commit:
- Moves the overflow check from `builtin_set_query` to `builtin_run`.
- Removes a conflicting int -> uint8_t conversion in `builtin_return`.
- Adds tests for the 3 remaining affected builtins.
- Simplifies the wording for the documentation for `set --query`.
- Does not change documentation for `functions --query`, because it does
  not state the exit code in its API.
- Updates the CHANGELOG to reflect the change to all builtins.
2021-02-13 08:41:51 +01:00
Johannes Altmanninger
e27d97b02e Do not add spaces after completions ending in "-"
Some programs use this to separate things in a word, see
https://github.com/spf13/cobra/pull/1249#discussion_r563605048

Require the token to be at least length 2 for the no-space behavior,
for completions of "-" like for python.
2021-02-13 08:13:31 +01:00
Johannes Altmanninger
b5df9a7137 Fix a compiler warning about comparison of different signedness 2021-02-13 08:07:20 +01:00
Fabian Homborg
a503371c57 termsize: Default to 80x24 when the terminal says 0 again
This was lost in
6bdbe732e40c2e325aa15fcf0f28ad0dedb3a551..c7160d7cb4970c2a03df34547f357721cb5e88db.

Note that we only print a term-support flog message for now, the
warning seems a bit much.

Fixes #7709.
2021-02-12 18:25:21 +01:00
Fabian Homborg
056153eddd Remove stray comment
[ci skip]
2021-02-11 18:44:53 +01:00
ridiculousfish
17d6aa054b exec_internal_builtin_proc to stop returning failure
Now that closing stdin is no longer an error for builtins, the function
exec_internal_builtin_proc cannot fail. Make it return void instead.
2021-02-10 17:43:12 -08:00
ridiculousfish
84d59accfc builtins to allow stdin to be closed
Prior to this fix, if stdin were explicitly closed, then builtins would
silently fail. For example:

    count <&-

would just fail with status 1. Remove this limitation and allow each
builtin to handle a closed stdin how it sees fit.
2021-02-10 17:43:10 -08:00
ridiculousfish
f239329f33 Attempt to fix the 32 bit fd_monitor test
Speculatively the fd_monitor thread is not scheduled, or we are awoken
early. Add a loop to ensure it gets run.

This is an attempt at #7699
2021-02-10 12:28:34 -08:00
Johannes Altmanninger
51850e9303 Output human-readable descriptions of some parse errors
"echo > )" provokes one of these.
2021-02-09 22:38:16 +01:00
Johannes Altmanninger
86707378cc Also allow unclosed quotes in some places
See #7693
2021-02-09 22:38:16 +01:00
Johannes Altmanninger
38b95defbd Inside an unclosed subshell, do not report other parse errors
In an interactive shell, typing "for x in (<RET>" would print an error:

	fish: Expected end of the statement, but found a parse_token_type_t::tokenizer_error

Our tokenizer converts "(" into a special error token, hence this message.
Fix two cases by not reporting errors, but only if we allow parsing incomplete
input. I'm not really sure if this is necessary, but it's sufficient.

Fixes #7693
2021-02-09 22:19:42 +01:00
ridiculousfish
e423a58e24 Add a thread yield to topic monitor torture test
This speeds up the test by about 5 msec.
2021-02-08 14:06:47 -08:00
ridiculousfish
4b9a096cf2 builtins to sometimes not buffer when writing to a pipe
Prior to this change, if you pipe a builtin to another process, it would
be buffered. With this fix the builtin will write directly to the pipe if
safe (that is, if the other end of the pipe is owned by some external
process that has been launched).

Most builtins do not produce a lot of output so this is somewhat tricky to
reproduce, but it can be done like so:

     bash -c 'for i in {1..500}; do echo $i ; sleep .5; done' |
	   string match --regex '[02468]' |
	   cat

Here 'string match' is filtering out numbers which contain no even digits.
With this change, the numbers are printed as they come, instead of
buffering all the output.

Note that bcfc54fdaa fixed this for the case where the
builtin outputs to stdout directly. This fix extends it to all pipelines
that include only one fish internal process.
2021-02-08 14:22:02 -08:00
ridiculousfish
171d09288b Rename allow_buffering to piped_output_needs_buffering
This makes the variable's role clear. It controls whether output to a
pipe must be buffered to avoid deadlock.
2021-02-08 14:22:02 -08:00
Mahmoud Al-Qudsi
2d39568ec4 Statically assert the sort order of more lists
Add compile-time checks to ensure list of string subcommands, builtins,
and electric variables are kept in asciibetical order to facilitate
binary search lookups.
2021-02-08 15:31:49 -06:00
Mahmoud Al-Qudsi
cb3ab80cab Use const_strlen in a few different places
This may slightly improve performance by allowing the compiler greater
visibility into what is happing on top of not executing at runtime in
some hot paths, but more importantly, it gets rid of magic constants in a
few different places.
2021-02-08 15:16:21 -06:00
Mahmoud Al-Qudsi
5d4c24bae1 Refactor color.h/color.cpp
* Use `uint8_t` instead of `unsigned char`
* Statically assert the sort order for `named_colors`
* Use constexpr for array lengths
2021-02-08 15:16:21 -06:00
Mahmoud Al-Qudsi
5fda1e05dc Statically assert the sort order of input_function_metadata_t 2021-02-08 15:16:20 -06:00
Mahmoud Al-Qudsi
2df16b53ff Use thread-local vectors for caching peeked events
These functions are called in the event queue hot path every time an
input event takes place. If we could guarantee a maximum length of
non-char (i.e. readline) events in the queue, we could use
`event_queue_peeker_t` with a fixed storage size of, e.g., 32 events,
but I'm not sure what a reasonable number would in fact be, so I'm just
changing these to use a thread-local vector that will re-use its
previous heap allocation in subsequent invocations rather than thrashing
the heap.
2021-02-08 15:16:20 -06:00
Mahmoud Al-Qudsi
2c1764fd45 Convert more event queue push_front loops to insert_front 2021-02-08 15:16:20 -06:00
Mahmoud Al-Qudsi
5c014e129a Reduce input latency searching for readline function mappings
The lookups are executed on all input events, so they are worth
optimizing.

Cache the list of names, use binary search to get a function code from a
name, and stop enumerating mappings after `has_function` and `has_command`
have been determined.
2021-02-08 15:16:20 -06:00
Ethel Morgan
6dd6a57c60 Saturate return value in builtin_set_query
builtin_set_query returns the number of missing variables. Because the
return value passed to the shell is an 8-bit unsigned integer, if the
number of missing variables is a multiple of 256, it would overflow to 0.

This commit saturates the return value at 255 if there are more than 255
missing variables.
2021-02-08 20:38:56 +01:00
Johannes Altmanninger
e40850ab89 Format fish_tests.cpp 2021-02-08 07:31:33 +01:00
ridiculousfish
50a7798041 Elimiate static variables inside builtin_test
builtin_test stashes some variables in statics, to support
the `test -t` expression. However this will cause conflicts with
concurrent execution, where we may want to run two `test` expressions at
once. Do the grunt work of threading the data into all places it needs
to go.
2021-02-07 17:41:36 -08:00
ridiculousfish
40d8e7e983 Correct the sense of a test for builtin stdin fds
fish isn't quite sure what to do if the user specifies an fd redirection
for builtins. For example `source <&5` could potentially just read from
an arbitrary file descriptor internal to fish, like the history file.

fish has some lame code that tries to detect these, but got the sense
wrong. Fix it so that fd redirections for builtins are restricted to
range 0 through 2.
2021-02-07 16:21:33 -08:00
ridiculousfish
17707065b8 Remove the io_pipe_t parameter from exec_internal_builtin_proc
This parameter describes if stdin has a pipe, but that can be easily
inferred from the io_chain. Remove it in the interest of parsimony.
2021-02-07 16:03:58 -08:00
Fabian Homborg
b3626d48e7 Highlight keywords differently
This introduces a new variable $fish_color_keyword that will be used
to highlight keywords. If it's not defined, we fall back on
$fish_color_command as before.

An issue here is that most of our keywords have this weird duality of
also being builtins *if* executed without an argument or with
`--help`.

This means that e.g.

    if

is highlighted as a command until you start typing

    if t

and then it turns keyword.
2021-02-07 21:18:51 +01:00
ridiculousfish
0f1281bec6 Unify thread sanitizer detection
We now have two files that need to know if thread sanitizer is enabled. They
can share the detection code.
2021-02-07 10:59:10 -08:00
ridiculousfish
ced56d492f Disable iothread pool wait-around under TSan
The iothread pool has a feature where, if the thread is emptied, some
threads will choose to wait around in case new work appears, up to a
certain amount of time (500 msec). This prevents thrashing where new
threads are rapidly created and destroyed as the user types. This is
implemented via `std::condition_variable::wait_for`. However this function
is not properly instrumented under Thread Sanitizer (see
https://github.com/google/sanitizers/issues/1259) so TSan reports false
positives. Just disable this feature under TSan.
2021-02-07 10:59:10 -08:00
ridiculousfish
a942df3886 Use fd_event_signaller_t in fd_monitor_t
fd_monitor_t allows observing a collection of fds. It also has its own
fd, which it uses to awaken itself when there are changes. Switch to
using fd_event_signaller_t instead of a pipe; this reduces the number of
file descriptors and is more efficient under Linux.
2021-02-07 10:59:10 -08:00
ridiculousfish
e004930947 Use fd_event_signaller in iothread completions
This simplifies how iothread notices when there are completions ready to
run.
2021-02-07 10:59:10 -08:00
ridiculousfish
8066428feb Add fd_event_signaller_t
fd_event_signaller_t exists to expose eventfd under Linux. This is a
more lightweight way of signalling events than using a pipe.
2021-02-07 10:59:10 -08:00
Fabian Homborg
78358ab351 Make disable_mouse_tracking inaccessible
This isn't something you want to bind, it's only a readline symbol as
a hack, so we shouldn't expose it to the user.
2021-02-07 19:50:56 +01:00
ridiculousfish
aac5862a67 Use vectors, not queues, in iothread main thread requests
queues use std::deque under the hood which is more expensive than a vector.
We always consume the entire queue so there is no advantage to use deque here.
Just use a vector.
2021-02-06 16:19:21 -08:00
ridiculousfish
76833cf6af Use futures in perform_on_main_thread
Replace the complicated implementation which shared a condition variable, with
one which just uses std::future<void>. This may allocate more condition
variables but is much simpler.
2021-02-06 16:19:21 -08:00
Mahmoud Al-Qudsi
eecc223c51 Recognize and disable mouse-tracking CSI events
Fish was previously oblivious to the existence of mouse-tracking ANSI
escapes; this was mostly OK because they're disabled by default and we
don't enable them, but if a TUI application that turned on mouse
reporting crashed or exited without turning mouse reporting off, fish
would be left in an unusable state as all mouse reporting CSI sequences
would be posted to the prompt.

This can be tested by executing `printf '\x1b[?1003h'` at the prompt,
then clicking with any mouse button anywhere within the terminal window.
Previously, this would have resulted in seeming garbage being spewed to
the prompt; now, fish detects the mouse tracking CSIs posted to stdin by
the terminal emulator and a) ignores them to prevent invalid input, as
well as b) posts the CSI needed to disable future mouse tracking events
from being emitted on subsequent mouse interactions (until re-enabled).

Note that since we respond to a mouse tracking CSI rather than
pre-emptively disable mouse reporting, we do not need to do any sort of
feature detection to determine whether or not the terminal supports
mouse reporting (otherwise, if it didn't support it and we posted the
CSI anyway, we'd end up with exactly the kind of cruft posted to the
prompt that we're trying to avoid).

Fixes #4873
2021-02-06 17:22:59 -06:00
Mahmoud Al-Qudsi
cc392b3774 Add RAII-based event_queue_peeker_t helper
This is a stack-allocating utility class to peek up to N
characters/events out of an `event_queue_t` object. The need for a
hard-coded maximum peek length N at each call site is to avoid any heap
allocation, as this would be called in a hot path on every input event.
2021-02-06 17:18:53 -06:00
Mahmoud Al-Qudsi
c203c88c66 Add and use event_queue_t::insert_front()
This allows directly inserting multiple characters/events in one go at
the front of the input queue, instead of needing to add them one-by-one
in reverse order.

In addition to improving performance in case of fragmented dequeue
allocation, this also is less error prone since a dev need not remember
to use reverse iterators when looping over a vector of peeked events.
2021-02-06 17:18:53 -06:00
Mahmoud Al-Qudsi
ea1a4b7932 Explicitly annotate intentional switch fallthrough
This silences a very useful warning in GCC 10.
2021-02-06 17:03:23 -06:00
ridiculousfish
b7e892d545 next_thread_id to use atomics, not locks
We have multiple places where we use std::atomic<uint64_t>, so let's use it
in next_thread_id too.
2021-02-06 14:27:08 -08:00
ridiculousfish
fe334bf620 Remove scoped_rlock
It is unused.
2021-02-06 14:27:08 -08:00
ridiculousfish
9c238385f0 Fix binary_semaphore_t under non-Linux TSan
Under non-Linux builds, binary_semaphore is implemented with a
self-pipe. When TSan is active we mark the pipe as non-blocking as TSan
cannot interrupt read (but can interrupt select). However we weren't
properly testing for EAGAIN leading to an assertion failure.

Allow looping on EAGAIN.
2021-02-06 14:41:50 -08:00
ridiculousfish
736e344727 assert_is_locked to take std::mutex, not void *
It's unclear why this had the void* cast.
2021-02-06 14:24:45 -08:00
ridiculousfish
98b0ef532f io_buffer_t to store a promise, not a future, to satisfy TSan
io_buffer_t is a buffer that fills itself by reading from a file
descriptor (typically a pipe). When the file descriptor is widowed, the
operation completes, and it reports completion by marking a
`std::promise<void>`. The "main thread" waits for this by waiting on the
promise's future. However TSan was reporting that the future's destructor
races with its promise's wait method. It's not obvious if this is valid,
but we can fix it by keeping the promise alive until the io_buffer_t is
deallocated.

This fixes the TSan issues reported under
`complete_background_fillthread_and_take_buffer` for #7681 (but there
are other unresolved issues).
2021-02-06 13:28:01 -08:00
Fabian Homborg
caca4fec22 fds.h: Add missing types.h include
Broke the build on FreeBSD because that defines mode_t there.
2021-02-06 19:59:53 +01:00
ridiculousfish
b5716e97cc Remove fd_set_t
Now that we no longer need to worry about pipes conflicting with
user-specified redirections, we can remove fd_set_t.
2021-02-05 18:14:50 -08:00
ridiculousfish
b79ec0122a Use pipe2 when creating pipes if avaialble
This allows us to avoid marking the pipe as CLOEXEC in some cases,
saving a system call.
2021-02-05 17:58:08 -08:00
ridiculousfish
97f29b1f4d Pipe fds to move to the "high range"
This concerns how fish prevents its own fds from interfering with
user-defined fd redirections, like `echo hi >&5`. fish has historically
done this by tracking all user defined redirections when running a job,
and ensuring that pipes are not assigned the same fds. However this is
annoying to pass around - it means that we have to thread user-defined
redirections into pipe creation.

Take a page from zsh and just ensure that all pipes we create have fds in
the "high range," which here means at least 10. The primary way to do this
is via the F_DUPFD_CLOEXEC syscall, which also sets CLOEXEC, so we aren't
invoking additional syscalls in the common case. This will free us from
having to track which fds are in user-defined redirections.
2021-02-05 17:58:08 -08:00
ridiculousfish
6c4f2622ef iothread's notify pipes to use make_autoclose_pipes
This allows it to take advantage of the upcoming high-range fd changes.
2021-02-05 17:58:08 -08:00
ridiculousfish
4b4bf541d1 Migrate more fd-concerned functions from wutil into fds
Functions like wopen_cloexec have a new home in fds.cpp. This is in
preparation for reworking how internal fds avoid conflict with user fds.
2021-02-05 17:58:08 -08:00
ridiculousfish
6588cf35f4 Move autoclose_pipes_t from io.h to fds.h 2021-02-05 17:58:08 -08:00
ridiculousfish
be9375e914 Migrate autoclose_fd_t to new file fds.h
fds.h will centralize logic around working with file descriptors. In
particular it will be the new home for logic around moving fds to high
unused values, replacing the "avoid conflicts" logic.
2021-02-05 17:58:08 -08:00
Fabian Homborg
b5305ce3d3 Handle backslashes properly in locate_brackets_of_type
This needs to be rewritten, I'm pretty sure we have like 6 of these
kinds of ad-hoc "is this quoted" things lying around.

But for now, at least don't just check if the *previous* character was
a backslash.

Fixes #7685.
2021-02-05 22:03:13 +01:00
ridiculousfish
97bde2f2bf Further refactoring of io_buffer_t
Previously we sometimes wanted to access an io_buffer_t to append to it
directly, but that's no longer true; all we really care about is its
separated_buffer_t. Make io_bufferfill_t::finish return the
separated_buffer directly, simplifying call sites. No user visible changes
expected here.
2021-02-04 17:14:46 -08:00
ridiculousfish
258149fe2e Improve locking discipline in io_buffer_t
Previously we had a lock that was taken in an ad-hoc manner. Switch to
using owning_lock.
2021-02-04 17:03:54 -08:00
ridiculousfish
8bcc8c1a36 Further cleanup of separated_buffer_t and io_buffer_t
Remove some clinging tendrils of life as a template object.
2021-02-04 16:43:47 -08:00
ridiculousfish
cbf10971f0 Reorganize separated_buffer_t
Move private bits to the bottom and do some other mild cleanup.
2021-02-04 16:06:28 -08:00
ridiculousfish
d578f8d136 separated_buffer_t to accept strings by rvalue reference
This saves a copy in some cases.
2021-02-04 16:02:40 -08:00
ridiculousfish
032467f338 separated_buffer_t to stop being a template
Now that we no longer construct wide separated buffers, it doesn't have
to be templatized.
2021-02-04 15:32:11 -08:00
ridiculousfish
7d494eab5c builtins to write to buffers directly
This concerns builtins writing to an io_buffer_t. io_buffer_t is how fish
captures output, especially in command substitutions:

    set STUFF (string upper stuff)

Recall that io_buffer_t fills itself by reading from an fd (typically
connected to stdout of the command). However if our command is a builtin,
then we can write to the buffer directly.

Prior to this change, when a builtin anticipated writing to an
io_buffer_t, it would first write into an internal buffer, and then after
the builtin was finished, we would copy it to the io_buffer_t. This was
because we didn't have a polymorphic receiver for builtin output: we
always buffered it and then directed it to the io_buffer_t or file
descriptor or stdout or whatever.

Now that we have polymorphpic io_streams_t, we can notice ahead of time
that the builtin output is destined for an internal buffer and have it
just write directly to that buffer. This saves a buffering step, which is
a nice simplification.
2021-02-04 15:21:32 -08:00
ridiculousfish
cd9a035f02 Add a string_output_stream_t to collect builtin output
This is used when creating a function; this breaks a dependency on the
more complicated buffered_output_stream_t to ease refactoring.
2021-02-04 14:12:14 -08:00
ridiculousfish
fc97151aec Add a variant of wcs2string which accepts a ptr, length pair
This will be useful when refactoring separated buffers.
2021-02-04 13:28:48 -08:00
ridiculousfish
86a12e1abd separated_buffer_t::append to stop being a template
In preparation for simplifying how builtins write to buffers, make
append an ordinary function rather than a template function.
2021-02-04 13:19:11 -08:00
ridiculousfish
7e2a538300 create_output_stream_for_builtin to accept read limit directly
This avoids requiring passing in a parser.
2021-02-03 19:00:04 -08:00
ridiculousfish
2d78c9a0d9 Poll the uvar notifier when the reader is interrupted by a signal
While the user waits at the prompt, fish is waiting in select(), on stdin.
The sigio based universal notifier interrupts select() by arranging for a
signal to be delivered, which causes select() to return with EINTR.
However we weren't polling the notifier at that point so we would not
notice uvar changes, until we got some real input.

I didn't notice this when testing, because my testing was changing fish
prompt colors which updated the prompt for other reasons.

Fixes #7671.
2021-01-31 15:42:35 -08:00
ridiculousfish
409ed7d6d0 Factor out count_preceding_backslashes
Now that we have multiple clients of count_preceding_backslashes, factor
it out from fish_indent into wcstringutil.h, and then use the shared
implementation.
2021-01-30 16:20:20 -08:00
Shizcow
cff5aa9130 Ensure escaped trailing spaces are not trimmed 2021-01-30 15:57:29 -08:00
Fabian Homborg
594d51e7eb Add a separate --profile-startup option to profile startup
This goes to a separate file because that makes option parsing easier
and allows profiling both at the same time.

The "normal" profile now contains only the profile data of the actual
run, which is much more useful - you can now profile a function by
running

   fish -C 'source /path/to/thing' --profile /tmp/thefunction.prof -c 'thefunction'

and won't need to filter out extraneous information.
2021-01-29 20:46:34 +01:00
Fabian Homborg
005d3a5981 Enable strict-aliasing and implicit-fallthrough warnings
GCC needs to have the comment *right before* the case label... blergh
2021-01-29 18:23:30 +01:00
Fabian Homborg
4e8c0f757d complete: Don't require a parameter with --force-files
A classic fallthrough problem!

This is why I want to enable -Wimplicit-fallthrough
2021-01-29 18:23:29 +01:00
Johannes Altmanninger
062f24d91b builtin set: make slice index range optional, like in slice expansion
Expansion parses slices like "$PATH[1..2]", but so does "set" when assigning
"set PATH[1..2] . .".  Commit be06f842a ("Allow to omit indices in index
range expansions") forgot the latter.
2021-01-28 07:19:38 +01:00
Fabian Homborg
275534b1b3 read: Remove unused short options
This has both "m" and "B" in the short options but did nothing with
them, so it would assert() out.

Fixes #7659.
2021-01-26 07:06:25 +01:00
Fabian Homborg
baa9b21a6f type: Only print function path with "--path"
Fixes #7653.
2021-01-24 15:31:39 +01:00
Fabian Homborg
88a84bd988 reader: Force ONLCR on for fish and external commands
Just like OPOST this just breaks output for anything not prepared for
it. Fish itself might work with it (and #4505 recommends it), but external commands are broken.

You'll see output like

foo
   ⏎

from `echo foo`.

Fixes #4873.

Continuation of #7133.
2021-01-18 21:00:08 +01:00
Fabian Homborg
8b133833fa Don't inherit windows paths for $PWD
If given a windows path like `F:\foo`, this currently ends up
assert()ing in path_normalize_for_cd.

Instead, since these paths violate a bunch of assumptions we make, we
reject them and fall back on getting $PWD via getcwd() (which should
give us a nice proper unixy path).

Fixes #7636.

This isn't tested because it would require a system where a windowsy
path passes paths_are_same_file, and on the unix systems we run our
tests that's impossible as far as I can tell?
2021-01-17 23:08:04 +01:00
Fabian Homborg
932074f06c escape_string_script: Escape DEL as \x7f
This used to print a literal DEL character in the output for `bind`,
which wouldn't actually show up and made it hard to figure out what
the key was.

So we just escape it back to how we actually used it - `\x7f`.

Fixes #7631.
2021-01-16 12:49:49 +01:00
Fabian Homborg
a4f5dd5054 set: Move the new values
A C++ special!

This makes

```fish
set -l var (seq 1 10000)
set -l v
for f in $var
    set -a v $f
end
```

~15% faster by removing allocations.
2021-01-15 21:00:25 +01:00
ridiculousfish
7a0bddfcfa Teach string repeat to handle multiple arguments
Each argument in string repeat is handled independently, except that the
--no-newline option applies only to the last newline.

Fixes #5988
2021-01-11 17:00:06 -08:00
ridiculousfish
290d1f2cd6 Mild refactoring of builtin_string repeat
Preparation for fixing issue 5988; no behavior change expected here.
2021-01-11 16:52:39 -08:00
ridiculousfish
1d4883d810 Remove an unnecessary 'using' declaration
This was just redundant with the struct tag.
2021-01-11 15:23:52 -08:00
ridiculousfish
7207a205f2 Switch history races test to use threads instead of processes
This avoids issues with ASan and TSan whose allocators do not properly
clean up in atfork, leading to deadlocks in child processes.
2021-01-11 12:44:21 -08:00
Fabian Homborg
3fc9c0b38c tests: Increase cancellation delay
This sometimes fails on github actions with ASAN. I am assuming that's
because the ctrl-c happens *before* the process has had a chance to
start.

So we do what we do and increase the delay.
2021-01-11 21:00:33 +01:00
Fabian Homborg
7bf2b9fd43 output: Rename some variables
These are a foreground and a background color. Now I see the point in
not naming them "foreground_color" and "background_color", but at
least "fg" and "bg" should do, right?
2021-01-11 20:56:15 +01:00
Fabian Homborg
f7b2bf8229 output: Simplify some duplicated code
Becomes a bit boring after a while
2021-01-11 20:53:11 +01:00
Fabian Homborg
19efd22468 env: Setup $HOME/$USER *before* the config directories
They are based on $HOME, so setting $HOME has to be done first.

Fixes #7620

(untested because I'm assuming common CI systems have weird $HOME settings)
2021-01-11 18:51:47 +01:00
ridiculousfish
e8c9da100c Track histories with shared_ptr
Prior to this change, histories were immortal and allocated with either
unique_ptr or just leaked via new. But this can result in races in the
path detection test, as the destructor races with the pointer-captured
history. Switch to using shared_ptr.
2021-01-09 17:02:11 -08:00
ridiculousfish
e062a07a97 Revert "Stop using unique_ptr to store histories"
This reverts commit 6f91195f40.
This triggered ASan complaints due to leaks.
2021-01-09 17:02:11 -08:00
ridiculousfish
87dacc0e95 Improve formatting and layout of history path detection test 2021-01-09 17:02:11 -08:00
ridiculousfish
884eb2b198 Remove an unused static variable 2021-01-09 17:02:11 -08:00
ridiculousfish
89687e7db7 Fix a warning building on Linux
Initialize saved_errno
2021-01-09 13:14:54 -08:00
Fabian Homborg
1dd776ec99 echo: Don't interpret and print options
A weird interaction between grouped short options and our weird option
parsing that puts unknown options back:

```
echo "-n foo"
```

would see the `-n`, turn off printing newlines, interpret the " " as
another grouped short option, see that there is no short option for
space and put the entire token back on the arguments pile.

So it would print "-n foo" *without a newline*.

Fix this by keeping an old state of the options around and reverting
it when putting options back.

The alternative is *probably* to forbid the " " short option in
wgetopt, then check if an option group contains it and error out, but
this should only really be a problem in `echo` because that is,
AFAICT, the only thing that puts the options back.

Fixes #7614
2021-01-09 08:50:30 +01:00
ridiculousfish
3c3d09b65f Fix a tsan warning in features_t 2021-01-08 19:36:56 -08:00
ridiculousfish
6f91195f40 Stop using unique_ptr to store histories
These register shutdown dtors, which cause tsan to complain.
2021-01-08 14:14:05 -08:00
ridiculousfish
bee8e8f6f7 Expand more when performing history path detection
When adding a command to history, we first expand its arguments to see
if any arguments are paths which refer to files. If so, we will only
autosuggest that command from history if the files are still valid. For
example, if the user runs `rm ./file.txt` then we will remember that
`./file.txt` referred to a file, and then only autosuggest that if the file
is present again.

Prior to this change we only performed simple expansion relative to the
working directory. This change extends it to variables and tilde
expansion. For example we will now apply the same hinting for
`rm ~/file.txt`

Fixes #7582
2021-01-08 12:58:34 -08:00
ridiculousfish
4faebf74e6 Remove 100 msec timeout from io_buffer_t
This removes the 100 msec timeout from io_buffer_t. We no longer need to
periodically wake up to check if a command substitution is finished,
because we get explicitly poked when that happens.
2021-01-07 12:07:06 -08:00
ridiculousfish
d5d09c993e io_buffer_t to explicitly poke its item when closing
io_buffer_t is used to buffer output from a command substitution, so we
can split it into arguments. Typically io_buffer_t reads from its pipe
until it gets EOF and then stops reading. However it may be that the
cmdsub ends but EOF is not delivered because the stdout of the cmdsub
escaped with a background process.

Prior to this change we would wake up every 100 msec (select timeout) to
check if the cmdsub is finished. However this 100 msec adds latency if a
background process is launched from e.g. fish_prompt.

Switch to the new poke() function. Now when the cmdsub is finished, it
pokes its item, which explicitly wakes it up. This removes the extra
latency.

Fixes #7559
2021-01-07 11:54:31 -08:00
ridiculousfish
fd08b660c0 Add a poke function to fd_monitor
In preparation for fixing #7559, add a function poke_item to fd_monitor.

fd_monitor has a list of file descriptors, and invokes a callback when an
fd becomes readable. With this change, we assign each item a unique ID and
return it when the item is added; the ID may then be used to invoke the
callback explicitly.

The idea is that we can stop reading from the pipe associated with the
cmdsub when the job is finished, even if the pipe is still open.
2021-01-07 11:51:04 -08:00
Mahmoud Al-Qudsi
7669e8e497 Add concept of edit groups
This allows for multiple edits to be undone/redone in one go, as if they
were one edit.

Useful when a function is editing the commandline buffer via scripted
changes or via a keybinding so the internal changes to the buffer can be
abstracted away.

(Having extreme difficulty getting pexpect to play nice with the concept
of undo/redo...)
2021-01-05 15:43:34 -06:00
Fabian Homborg
6eeb8861e7 Add exit bind function
Currently binding `exit` to a key checks too late that it's exitted,
so it leaves the shell hanging around until the user does an execute
or similar.

As I understand it, the `exit` builtin is supposed to only exit the
current "thread" (once that actually becomes a thing), and the
bindings would probably run in a dedicated one, so the simplest
solution here is to just add an `exit` bind function.

Fixes #7604.
2021-01-04 09:45:34 +01:00
Fabian Homborg
85ba2ed790 type: Add missing newline
Otherwise this would print

    # Defined interactivelyfunction foo

for interactively defined functions.
2021-01-03 17:48:25 +01:00
ridiculousfish
118f710e99 Allow fish_private_mode to change at runtime
Prior to this change, `fish_private_mode` worked by just suppressing
history outright. With this change, `fish_private_mode` can be toggled on
and off. Commands entered while `fish_private_mode` is set are stored but
in memory only; they are not written to disk.

Fixes #7590
Fixes #7589
2021-01-02 22:01:47 -08:00
ridiculousfish
9fdc4f903b Explicitly track persistence mode in history_item_t
Commands that start with a space should not be written to the history
file. Prior to this change, that was implemented by simply not adding them
to history. Items with leading spaces were simply dropped.

With this change, we add a 'history_persistence_mode_t' to
history_item_t, which tracks how the item persists. Items with leading
spaces are now marked as "ephemeral": they can be recovered via up arrow,
until the user runs another command, or types a space and hits return.
This matches zsh's HIST_IGNORE_SPACE feature.

Fixes #1383
2021-01-02 21:31:19 -08:00
ridiculousfish
cdf05325ed Reorganize history_item_t
Move the private bits to the bottom of the class and other mild
refactoring. No user visible behavior change expected.
2021-01-02 19:51:16 -08:00
Fabian Homborg
cf8219e3ce Exit if --no-execute is enabled don't interactively read from the terminal
Don't go into implicit interactive mode without ever executing
anything - not even `exit` or reacting to ctrl-d. That just renders
the shell useless and unquittable.
2021-01-01 21:22:52 +01:00
Fabian Homborg
7ea8e20623
argparse: Make short flag names optional (#7585)
It was always a bit ridiculous that argparse required `X-longflag` if
that "X" short flag was never actually used anywhere.

Since the short letter is for getopt's benefit, we can hack around
this with our old friend: Unicode Private Use Areas.

We have a counter, starting at 0xE000 and going to 0xF8FF, that counts
up for all options that don't have a short flag and provides one. This
gives us up to 6400 long-only options.

6.4K should be enough for everybody.
2021-01-01 11:37:25 +01:00
ridiculousfish
792abf61ec Attempt to fix the tsan build
Deliberately leak the shared thread pool to avoid shutdown dtor registration
and tsan complaints at exit.
2020-12-31 17:03:53 -08:00
ridiculousfish
f03ff8cd00 Add a test for history path detection
This will support history path detection improvements in a future
commit.
2020-12-30 00:44:25 -08:00
Johannes Altmanninger
801955851b Workaround clang-tidy incorrectly assuming null
This silences a false positive linter warning about a null dereference.
2020-12-29 16:31:43 +01:00
Johannes Altmanninger
8fc9b9d61b Address some minor lints
A mildly interesting one is the call to test_wchar2utf8 with a non-null
pointer ("u1"/"dst") but 0 length. In this case we relied on malloc(0)
returning non-null which is not guaranteed.

	src/fish_tests.cpp:1619:23: warning: Call to 'malloc' has an allocation
	size of 0 bytes [clang-analyzer-optin.portability.UnixAPI]
	        mem = (char *)malloc(dlen);
	                      ^
	test_wchar2utf8(w1, sizeof(w1) / sizeof(*w1), u1, 0, 0, 0,
			"invalid params, dst is not NULL");
2020-12-29 16:31:43 +01:00
Johannes Altmanninger
69a9785f50 Refactor: pass by value, not reference, to enable move semantics
clang-tidy wrote:
> warning: passing result of std::move() as a const reference argument;
> no move will actually happen [performance-move-const-arg]
2020-12-29 16:31:43 +01:00
ridiculousfish
43505f7077 Allow ** glob segments to match zero directories
Prior to this change, a glob like `**/file.txt` would only match
`file.txt` in subdirectories; the `**` must match at least one directory.
This is historical behavior.

With this change we move a little closer to bash's implementation by
allowing a literal `**` segment to match in the current directory. That
is, `**/foo` will match both `foo` and `bar/foo`, while `b**/foo` will
only match `bar/foo`.

Fixes #7222.
2020-12-28 23:51:18 -08:00
ridiculousfish
df73964ced Clean up some comments around wildcard expansion 2020-12-28 23:51:18 -08:00
Johannes Altmanninger
322ceb7ab4 builtin realpath: use absolute path also with -s/--no-symlinks
The old test needs to be changed because $XDG_DATA_HOME can be relative.

Fixes #7574
2020-12-24 08:53:08 +01:00
ridiculousfish
e43913a547 Stop expanding globs in command position when performing error checking
Before running a command, or before importing a command from bash history,
we perform error checking. As part of error checking we expand commands
including variables and globs. If the glob is very large, like `/**`, then
we could hang expanding it.

One fix would be to limit the amount of expansion from the glob, but
instead let's just not expand command globs when performing error checking.

Fixes #7407
2020-12-22 12:38:51 -08:00
ridiculousfish
a8080e8e6f Allow specifying a limit on number of expansion in operation_context
If the user types something like `/**`, prior to this change we would
attempt to expand it in the background for both highlighting and
autosuggestions. This could thrash your disk and also consume a lot of
memory.

Add a a field to operation_context_t to allow specifying a limit, and add
a "default background" limit of 512 items.
2020-12-22 12:38:51 -08:00
ridiculousfish
0f2d73e4a3 Remove a stale comment 2020-12-22 12:38:51 -08:00
ridiculousfish
c2c729352e Eagerly abort wildcard completions for ** wildcards
Historically fish has not supported tab completing or autosuggesting
wildcards with **. Prior to this fix, we would test every file match,
discover the ** wildcard, and then ignore it. Instead look for **
wildcards at the top level.

This prevents autosuggesting with /** from chewing up your disk.
2020-12-22 12:38:51 -08:00
ridiculousfish
10362a70df Clean up parse_error_offset_source_start
Use range-based for loops and relax the requirement that we have an
error list.
2020-12-22 12:38:51 -08:00
ridiculousfish
38a30d1798 Mark subclasses of io_data_t as final 2020-12-19 20:06:36 -08:00
ridiculousfish
6f2e377fcc Clean up some unnecessary variable names in maybe.h 2020-12-19 16:10:40 -08:00
ridiculousfish
90f4c458e5 Rename insert_line_above to insert_line_over
This is for symmetry with insert_line_under. See #7442.
2020-12-19 14:31:33 -08:00
ridiculousfish
2d2efc8b2e Implement o and O bindings for vi mode
Credit to @joallard for the patch. Fixes #7442
2020-12-19 14:28:00 -08:00
Fabian Homborg
7e7355bde1 Restore $status after expanding completions
When a completion's "--arguments" script ran, it would clobber $status with its value,
so when you repainted your prompt, it would now show the completion
script's status rather than the status of what you last ran.

Solve this by just storing the status and restoring it - other places
do this by calling exec_subshell with apply_exit_status set to false,
which does basically the same thing. We can't use it here because we
don't want to run a "full" script, we only want the arguments to be
expanded, without a "real" command.

No, I have no idea how to test this automatically.

Fixes #7555.
2020-12-19 11:37:01 +01:00
Fabian Homborg
31166f4731 Simplify some duplicated path checks
This has one functional difference, in that we now report non-EACCESS
errors even for relative paths. I consider that to be a plus.

Some other sites might benefit from this, let's look into that later.
2020-12-15 18:15:59 +01:00
Fabian Homborg
0f6669f43c Stop using env_var_t::to_list in a few places
We don't need the entire list in modifiable form here - some just needs
the size, the others can just get a reference.
2020-12-15 15:47:44 +01:00
Fabian Homborg
14908322a9 Also include fallback.h
GRrrrrr
2020-12-14 23:23:00 +01:00
Fabian Homborg
0f5a226e2f math: Use fish_wcstod instead
1. This should be using our wcstod_l on platforms where we need
it (for some reason it wasn't picking it up on FreeBSD?)

2. This purports to have a "fast path". I like fast paths.
2020-12-14 23:09:01 +01:00
Fabian Homborg
3af07e6c6e math: Wcharify the error message
Dunno, this seems to work, but then this is the sort of thing
that *seems* to work.
2020-12-14 23:02:54 +01:00
Fabian Homborg
e94f86e6d2 math: Use wcstod_l
Locale-wise, we're only interested in one thing:

"." is the radix character when interpreting numbers

And for that it's enough to just use our c-locale, like elsewhere.

This saves a bunch of switching locale back and forth, and simplifies
the code.
2020-12-14 22:58:47 +01:00
Fabian Homborg
97cd87f3b2 math: Use wchar
This was doing a bunch of work narrowing strings for no reason.
2020-12-14 22:54:53 +01:00
Fabian Homborg
6e9364ab50 fish_indent: Change --debug-level to --debug with flog categories
The "debug-level" flag makes little sense since we have no more
debug *levels* left.
2020-12-14 19:36:18 +01:00
ridiculousfish
36766ea3d7 Correct $status for certain pipeline-aborting failures
If we refused to launch a job because of a "pipeline aborting" error,
then it's the caller's responsibility to set $status.

Fixes #7540
2020-12-13 17:33:34 -08:00
ridiculousfish
2caeec24f7 Tighten up pipeline-aborting errors
Prior to this change, the functions in exec.cpp would return true or false
and it was not clear what significance that value had.

Switch to an enum to make this more explicit. In particular we have the
idea of a "pipeline breaking" error which should us to skip processes
which have not yet launched; if no process launches then we can bail out
to a different path which avoids reaping processes.
2020-12-13 17:30:26 -08:00
ridiculousfish
e5cff1a2db Fix some warnings from gcc
Use ignored_result instead of void casts, to satisfy the gcc.
2020-12-13 15:35:59 -08:00
Fabian Homborg
b7f47344b0 Print nicer "defined in" for functions defined on stdin/via source
This would tell you a function was "Defined in - @ line 1" for every
function defined via `source`.

Really, ideally we'd figure out where the *source* call was, but that'
much more complicated, so we just give a comprehensible message.
2020-12-11 23:09:16 +01:00
Fabian Homborg
425dabd6b1 Change fish_trace prefix to "->" instead of plusses
This matches what we do in --profile's output:

```
> source /home/alfa/.config/fish/config.fish
--> set -gx XDG_CACHE_HOME /home/alfa/.cache
--> set -gx XDG_CONFIG_HOME /home/alfa/.config
--> set -gx XDG_DATA_HOME /home/alfa/.local/share
```

instead of

```
+ source /home/alfa/.config/fish/config.fish
+++ set -gx XDG_CACHE_HOME /home/alfa/.cache
+++ set -gx XDG_CONFIG_HOME /home/alfa/.config
+++ set -gx XDG_DATA_HOME /home/alfa/.local/share
```
2020-12-11 21:24:33 +01:00
Fabian Homborg
ff62d172e5 Stop repainting in C++
We already have a variable handler, there is no need to repaint twice.
2020-12-11 18:43:04 +01:00
ridiculousfish
a2e486966a Always become pgroup leader in interactive mode
Prior to this change, if fish were launched connected to a tty but not as
pgroup leader, it would attempt to become pgroup leader only if
--interactive is explicitly passed. But bash will unconditionally attempt
to become pgroup leader if launched interactively. This can result in
scenarios where fish is running interactively but in another pgroup. The
most obvious impact is that control-C will result in the pgroup leader
(not fish) exiting and make fish orphaned.

Switch to matching the bash behavior here - we will always try to become
pgroup leader if interactive.

Fixes #7060.
2020-12-06 13:42:35 -08:00
ridiculousfish
5f131878a9 Buffer in outputter_t::term_puts
We were calling write() once for each character; buffer these instead.
2020-12-06 13:42:35 -08:00
Fabian Homborg
ab5d7f80d0 Restyle codebase
And again clang-format does something I don't like:

-    if (found != end && std::strncmp(found->name, name, len) == 0 && found->name[len] == 0) return found;
+    if (found != end && std::strncmp(found->name, name, len) == 0 && found->name[len] == 0)
+        return found;

I *know* this is a bit of a long line. I would still quite like having
no brace-less multi-line if *ever*. Either put the body on the same
line, or add braces.

Blergh
2020-12-06 15:39:54 +01:00
Fabian Homborg
aa895645dd Add string to reserved keywords
Since `string match` now creates variables, wrapping `string`
necessarily breaks things, so we need to disallow it.

See #7459, #7509.
2020-12-06 15:39:49 +01:00
ridiculousfish
fbeff2e751 Fix the build when gettext is disabled
When gettext is disabled, completions descriptions get passed as
const wcstring & which breaks the build. Accept the descriptions
by value instead.
2020-12-05 14:26:07 -08:00
ridiculousfish
91503151c9 Bravely remove a call to wrealpath in globbing
When globbing, we have a base directory (typically $PWD) and a path
component relative to that. As PWD is "virtual" it may be a symlink. Prior
to this change we would use wrealpath to resolve symlinks before opening
the directory during a glob, but this call to wrealpath consumed roughly
half of the time during globbing, and is conceptually unnecessary as
opendir will resolve symlinks for us.

Remove it. This may have funny effects if the user's PWD is an unlinked
directory, but it roughly doubles the speed of a glob like `echo ~/**`.
2020-12-05 14:04:45 -08:00
ridiculousfish
594a6a35e8 Adopt expansion limits in wildcard expansions
This prevents e.g. `count /**` from consuming all of your memory.

Fixes #7226
2020-12-05 13:21:46 -08:00
ridiculousfish
f11a60473a Introduce expansion limits
This adds the ability to limit how many expansions are produced. For
example if $big contains 10 items, and is Cartesian-expanded as
$big$big$big$big... 10 times, we would naviely get 10^10 = 10 billion
results, which fish can't actually handle. Implement this in
completion_receiver_t, which now can return false to indicate an overflow.

The initial expansion limit 'k_default_expansion_limit' is set as 512k
items. There's no way for users to change this at present.
2020-12-05 13:19:07 -08:00
ridiculousfish
48567c37de Adopt completion_receiver_t more widely
This switches certain uses from just appending to a list to using
completion_receiver_t, in preparation for limiting how many completions
may be produced. Perhaps in time this could also be used for "streaming"
completions.
2020-12-05 13:18:14 -08:00
ridiculousfish
245f264c04 Remove a suspicious 'unused' declaration for wildcard_complete_internal
This function is used and so is its return value, at all call sites.
2020-12-05 11:46:01 -08:00
ridiculousfish
af3383e727 Introduce completion_receiver_t
completion_receiver_t wraps a completion list; it will centralize logic
around adding completions and most importantly it will enforce that we
do not exceed our expansion limit.
2020-12-05 11:46:01 -08:00
Fabian Homborg
7cefe598e9 Don't use KERN_PROC_PATHNAME on NetBSD
This returns the wrong thing and breaks the tests.

Since it's not super important anyway, just disable it and go back to
/proc, that works.
2020-12-05 14:43:08 +01:00
Fabian Homborg
02efce51a9 string match: Only import variables for the first matching argument
This makes it work the same whether it quits early (with "-q") or not,
and it's generally nice to nail this down.

See #7495.
2020-12-04 18:45:08 +01:00
Fabian Homborg
b9b84e63bf Revert "Attempt to simplify how completions get presented in the pager"
The pager cleanup missed that the existing token could already include active (as in unescaped) expansions, and just escaped them all.

This means things like `ls ~/<TAB>` would escape the `~`, which is obviously wrong and makes it awkward to use.

This reverts commit b38a23a46d.

I fully expect that we'll try again, but there's no use in keeping master broken while that happens.

Fixes #7526.
2020-12-04 16:44:48 +01:00
Fabian Homborg
720982a3cb string: Quit early if --quiet is satisfied
E.g. if we do `string match -q`, and we find a match, nothing about
the input can change anything, so we quit early.

This is mainly useful for performance, but it also allows `string`
with `-q` to be used with infinite input (e.g. `yes`).

Alternative to #7495.
2020-12-01 18:55:01 +01:00
Mahmoud Al-Qudsi
8aac537191 Silence GCC warn_unused_result warnings in tests
warn_unused_result is the persistent one that won't go away with a
simple `(void)write(...)` and needs to be assigned to a variable (that
must then also be declared unused or else you'll get a warning about
_that_).
2020-11-29 18:12:09 -06:00
ridiculousfish
74b298a6f9 Fix a gcc warning about comparison of different signedness 2020-11-29 14:06:04 -08:00
ridiculousfish
f5f0f98991 Remove expand_flag::skip_jobs
It was unused.
2020-11-29 14:01:13 -08:00
ridiculousfish
2b8d2deb0c Introduces "smartcase" completions
"smartcase" performs case-insensitive matching if the input string is all
lowercase, and case-sensitive matching otherwise. When completing e.g.
files, we will now show both case sensitive and insensitive completions if
the input string does not contain uppercase characters.

This is a delicate fix in an interactive component with low test coverage.
It's likely something will regress here.

Fixes #3978
2020-11-29 12:40:01 -08:00
ridiculousfish
b38a23a46d Attempt to simplify how completions get presented in the pager
This is an attempt to simplfy some completion logic. It mainly refactors
reader_data_t::handle_completions such that all completions have the token
prepended; this attempts to simplify the logic since now all completions
replace the token. It also changes how the pager prefix works. Previously
the pager prefix was an extra string that was prepended to all
completions. In the new model the completions already have the prefix
prepended and the prefix is used only for certain width calculations.

This is a somewhat frightening change in an interactive component with
low test coverage. It tweaks things like how long completions are
ellipsized. Buckle in!
2020-11-29 12:35:18 -08:00
ridiculousfish
4b947e0a23 Refactor string fuzzy matching
In preparation for introducing "smart case", refactor string fuzzy
matching. Specifically split out the case folding and match type into
separate fields, so that we can introduce more case folding types without
a combinatoric explosion.
2020-11-29 12:35:18 -08:00
ridiculousfish
20b98294ba Bravely remove string_fuzzy_match_t::compare
This is used to decide which fuzzy match is better, however it is used
only in wildcard expansion and not in actual completion ranking or
anywhere else where it could matter. Try removing the compare() call
and implementation.

What compare() did specially was compare distances, e.g. it ranks
lib as better than libexec when expanding /u/l/b. But the tests did not
exercise this so it's hard to know if it's working. In preparation for a
refactoring, remove it.
2020-11-29 12:35:18 -08:00
ridiculousfish
ac1ee6f1fd Make fuzzy_match_type_t an enum class
Also rename it to fuzzy_type_t and shorten some of its values.
2020-11-29 12:35:18 -08:00
ridiculousfish
9144141ded Migrate string_fuzzy_match from common.h to wcstringutil.h
This is a more appropriate location for this functionality.
Also take this opportunity to clean up subsequence_in_string.
2020-11-29 12:35:18 -08:00
ridiculousfish
639cd66ba1 Conditionally make autosuggestions case sensitive
When fish presents an autosuggestion, there is some logic around whether
to retain it or discard it as the user types "into" it. Prior to this
change, we would retain the autosuggestion if the user's input text is a
case-insensitive prefix of the autosuggestion. This is reasonable for
certain case-insensitive autosuggestions like files, but it is confusing
especially for history items, e.g. `git branch -d ...` and `git branch -D
...` should not be considered a match.

With this change, when we compute the autosuggestion we record whether it
is "icase", and that controls whether the autosuggestion permits a
case-insensitive extension.

This addresses part of #3978.
2020-11-29 12:35:18 -08:00
ridiculousfish
1094b95b6f Mild refactoring of autosuggestions
Rather than storing an autosuggestion as a string, store a struct.
This is preparing to conditionalize autosuggestion case sensitivity.
2020-11-29 12:35:18 -08:00
ridiculousfish
d9ebe13cb4 Reorganize and improve commenting of autosuggest_validate_from_history
No behavior change expected here.
2020-11-29 12:35:18 -08:00
Johannes Altmanninger
7da93e2617 builtin functions: don't mix up multiple arguments
This regressed in 2e38cf2a which is contained in 2.6.0.

Fixes #7515
2020-11-29 06:35:02 +01:00
Johannes Altmanninger
79df29e8fd Remove some dead functions in the highlighter
These are replaced by "visit" methods.
2020-11-29 05:59:16 +01:00
Mahmoud Al-Qudsi
aa0bfa0eb8 Minor cleanup
clangd needs to respect clang-format files when inserting headers up
top.

[ci skip]
2020-11-28 01:00:27 -06:00
Mahmoud Al-Qudsi
a3cb1e2dcd Fix setting terminal title after fg
The code to override the `(status current-command) was present`, but not
handled in either the default `fish_title` function or the fallback.

Closes #7444.
2020-11-28 00:56:10 -06:00
Fabian Homborg
5872f4522d math: Add --base option
Currently a bit limited, unfortunately printf's `%a` specifier is
absolutely unreadable.

So we add `hex` and `octal` with `0x` and `0` prefixes respectively,
and also take a number but currently only allow 16 and 8.

The output is truncated to integer, so scale values other than 0 are
invalid and 0 is implied.

The docs mention this may change.
2020-11-27 19:33:27 +01:00
ridiculousfish
30c7a17302 Set tty to shell mode before running fish_prompt
Prior to this change, we would run fish_prompt and then afterwards set
the shell modes. For users with an initially slow prompt, this would
mean that characters would be echoed to the tty until after the prompt
completes.

Reorder these so that we set the tty mode first. This implies we will
run the prompt in shell mode, but this was already the case up until
2a3677b386.

Fixes #7489. Note that the prior commit e0cedd4ad2 is also necessary
here, as that fixed an extra prompt execution.
2020-11-26 16:47:08 -08:00
ridiculousfish
e0cedd4ad2 Remove exec_prompt call from read_push
This prompt execution does not appear to be necessary, and spoils the
upcoming fix for #7489
2020-11-26 16:38:33 -08:00
Mahmoud Al-Qudsi
5ddafb3b79 Add support for importing named regex matches
The new commandline switch `string match --regex --import` will import
as fish variables any named capture groups with the matched captures as
the value(s).
2020-11-26 14:41:00 -06:00
Mahmoud Al-Qudsi
282fb14dcf Get rid of magic numbers in report_match() result
Replace with a class-local `enum class` instance.
2020-11-26 14:38:04 -06:00
Fabian Homborg
a14e64ed6c math: Don't override errors with "unexpected token"
As always, we want to give the most specific error we can.

Fixes #7508
2020-11-26 12:41:19 +01:00
Fabian Homborg
903a9fbf0c math: Don't match longer function names
The comparison here is a bit naive, so "n" matches "ncr", so
technically

   math 'n(2, 3)'

is equivalent to

   math 'ncr(2, 3)'

Work towards #7508.
2020-11-26 12:37:09 +01:00
ridiculousfish
7c4891407f Remove restore_attrs from terminal_return_from_job_group function
Previously this parameter was used to more-eagerly restore the terminal
mode. This was the basis for #2214. However now we restore the mode
from the reader instead, so we can remove this unused parameter.
2020-11-23 19:36:55 -08:00
ridiculousfish
5f16a299a7 Use external mode for term when running key bindings
Prior to this fix, when key binding is a script command (i.e. not a
readline command), fish would run that key binding using fish's shell
tty modes. Switch to using the external tty modes. This re-fixes
issue #2214.
2020-11-23 19:36:39 -08:00
ridiculousfish
db514df95b Stop setting tty back to shell mode when a fg proc completes
Prior to this change, when a process resumes because it is brought back
to the foreground, we would reset the terminal attributes to shell mode.
This fixed #2114 but subtly introduced #7483.

This backs out 9fd9f70346, re-introducing #2114 and re-fixing #7483.
A followup fix will re-fix #2114; these are broken out separately for
bisecting purposes.

Fixes #7483.
2020-11-23 19:36:39 -08:00
ridiculousfish
e9b683dee1 Refactor how inputter handles script commands
Prior to this change, for bindings which have script commands, the
inputter would execute them directly. However an upcoming fix for #7483
will require more integration with the reader. Switch to a new model where
the reader passes in a function to use for executing script commands.
2020-11-23 19:36:39 -08:00
ridiculousfish
994f95b845 Move inputter_t private bits to the bottom of the class
Just a reorganization to clarify what parts are the interface.
2020-11-23 19:36:39 -08:00
ridiculousfish
050a211838 Clarify the role of the 'in' param in inputter_t constructor 2020-11-23 19:36:39 -08:00
ridiculousfish
48c50d202b Save a string allocation in expand_arguments_from_nodes
This function is called a lot; we can save a little bit of memory here.
2020-11-23 19:36:39 -08:00
Fabian Homborg
2e55e34544 Reformat 2020-11-22 14:39:48 +01:00
Mahmoud Al-Qudsi
3699e50e00 Explicitly check for KERN_PROC_PATHNAME
While FreeBSD, DragonflyBSD, and NetBSD have KERN_PROC_PATHNAME,
OpenBSD does not.
2020-11-20 15:49:57 -06:00
Mahmoud Al-Qudsi
be78e9dc28 fixup! Unify handling of BSD systems where applicable 2020-11-20 15:06:19 -06:00
Mahmoud Al-Qudsi
06f1b34553 Correct reporting of setpgid (parent vs child)
Previously, it always said "own process" (e.g. child error).
2020-11-20 14:22:42 -06:00
Mahmoud Al-Qudsi
e4c052330f Handle ESRCH from setpgid(2) on FreeBSD 2020-11-20 14:18:02 -06:00
Mahmoud Al-Qudsi
76faee71a5 Unify handling of BSD systems where applicable 2020-11-20 14:11:03 -06:00
Fabian Homborg
3c14d310a0 reader: Stop converting to wchar and back to wcstring
This called completion_insert with a wchar_t*, which was then passed
to a function that takes a wcstring.
2020-11-15 15:27:19 +01:00
Fabian Homborg
263ef55ae6 reader: Use erase directly
No need to use a separate reference.

Also no need to erase from begin(), just use the indices.
2020-11-15 15:26:52 +01:00
Fabian Homborg
3eba6c5d5a signal: Remove redundant set 2020-11-15 15:20:55 +01:00
Fabian Homborg
c23fc9a365 builtin_test: Exit early on float parsing error
cppcheck complains about a possible null-dereference.
2020-11-15 15:15:20 +01:00
Fabian Homborg
a8f259f685 Remove unused debug_escape function 2020-11-15 15:15:10 +01:00
Fabian Homborg
9c2d22e452 Remove debug_stack_frames
This was unused with FLOG. We leave the option stubbed out for now, so
we don't error out for well-meaning calls.
2020-11-15 11:32:52 +01:00
Fabian Homborg
95e86cf2d2 Remove the old debug macro and impl
This should make calling `debug()` impossible. Some of the other
bits remain, to be removed later.
2020-11-15 11:28:01 +01:00
Fabian Homborg
b32540f346 proc: Remove unused function 2020-11-15 11:26:45 +01:00
Fabian Homborg
01cd6385ff reader: Make param const
const good! Fabian like const!
2020-11-15 11:26:15 +01:00
Fabian Homborg
9f924f37fb reader: Pre-increment
If we don't do it now, static analysis things are just gonna bug us
until someone does it.
2020-11-15 11:25:45 +01:00
Fabian Homborg
575d574691 ast: Remove unused variable 2020-11-15 11:19:23 +01:00
Érico Rolim
21041e3cc7 src: don't split wide char strings that are used with gettext.
A bug in xgettext leads to the generation of useless msgids in the po
files for these strings.
2020-11-13 14:34:42 +01:00
Érico Rolim
6e9d5c00e5 src/builtin_type.cpp: add missing gettext call.
The string "%ls is %ls", which is printed when `type <command>` is ran
for a command in PATH, couldn't be localized, since it was missing _()
around it.
2020-11-13 14:31:40 +01:00
ridiculousfish
17fc542082 Revert "Stop caching line breaks in the prompt calculation"
This reverts commit a2ff32d904.

Per comments on the commit, the original code had correct handling of line
breaks inside escapes.
2020-11-12 10:55:11 -08:00
Fabian Homborg
5ee3eeff5d Remove the final two debug() calls 2020-11-07 10:20:52 +01:00
Mahmoud Al-Qudsi
640f4444f5 Disable SIGIO notifier on WSL
It currently does not trigger the uvar notifier and fails the automated
tests.

See #7429.
2020-11-06 20:49:44 -06:00
Fabian Homborg
65c5433662 Delete unused field
Fixes #7456.

[ci skip]
2020-11-06 17:44:09 +01:00
ridiculousfish
d3192d37a2 Allow timing-out I/O-able syntax highlighting after expanding abbreviation
It may happen that the user types an abbreviation and then hits return.
Prior to this commit, we would perform a form of syntax highlighting
that does not require I/O, so as to not block the user. However this
could cause invalid commands to be colored as valid.

More generally if the user has e.g a slow NFS mount, then syntax
highlighting may lag behind the user's typing, and be incorrect at the
time the user hits return. This is an unavoidable race, since proper
syntax highlighting may take arbitrarily long.

Introduce a new function `finish_highlighting_before_exec`, which waits
for any outstanding syntax highlighting to complete, BUT has a timeout
(250 milliseconds). After this, it falls back to the no-I/O variant, which
colors all commands as valid and nothing as paths.

Fixes #7418
Fixes #5912
2020-11-05 20:07:05 -08:00
ridiculousfish
c861fdadcf Remove return value from iothread_perform
It was not actually used by any test.
2020-11-05 19:28:26 -08:00
ridiculousfish
a2ff32d904 Stop caching line breaks in the prompt calculation
These are fast enough to find on demand.
2020-11-01 14:45:35 -08:00
Soumya
80aaae5b74 Clear to end of each line in left prompt 2020-11-01 13:29:26 -08:00
Fabian Homborg
2a07673561 Don't call a variable "stdin"
Musl has a macro that interferes.
2020-10-31 18:15:19 +01:00
Fabian Homborg
0951a706cf Let read read from fds other than 0
This allows

read </dev/tty

to work.

Fixes #7358
2020-10-31 13:39:20 +01:00
Fabian Homborg
d334dc6643 Let cancel after an unambiguous completion was accepted undo it
In some cases the completion we come up with may be unexpected, e.g.
if you have files like

/etc/realfile

and

/etc/wrongfile

and enter "/etc/gile", it will accept "wrongfile" because "g" and
"ile" are in there - it's a substring insertion match.

The underlying cause was a typo, so it should be easy to go back.

So we do a bit of magic and let "cancel" undo, but only right after a
completion was accepted via complete or complete-and-search.

That means that just reflexively pressing escape would, by default, get you back to
the old token and let you fix your mistake.

We don't do this when the completion was accepted via the pager,
because 1. there's more of a chance to see the problem there and 2.
it's harder to redo in that case.

Fixes #7433.
2020-10-30 19:37:44 +01:00
Johannes Altmanninger
5ff2d38d4c builtin time: print help on invalid syntax
I always mix up the order with variable assignments.
2020-10-26 19:25:41 +01:00
Fabian Homborg
a84d57b02b math: Actually report closing paren error
This was typically overridden by "too many/few arguments", but it's
actually incorrect:

    sin(55

has the correct number of arguments to `sin`, but it's lacking
the closing `)`.
2020-10-26 18:13:43 +01:00
Rosen Penev
cef84cf2c2 clang-tidy: use append
Found with performance-inefficient-string-concatenation

Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-10-25 22:54:51 -07:00
Rosen Penev
334be56021 run codebase through clang-tidy
Signed-off-by: Rosen Penev <rosenp@gmail.com>
2020-10-25 22:48:49 -07:00
Mahmoud Al-Qudsi
332287708b Prevent fish from re-importing an exported fish_user_paths
fish_user_paths is a fish-specific variable that can be persisted by
making it a universal variable or by making it a global variable set at
startup in `config.fish`.

Since it is not defined in a clean installation, a user could
inadvertently create it as `set -Ux fish_user_paths ....` the first
time, creating a horrible, ugly, self-loathing mess that will have you
chasing ghosts and bisecting for naught once fish re-imports
fish_user_paths as a *global* variable that shadows the universal one.

While that is true for any universal variable that is re-imported as a
global variable, only fish_user_paths has the potential to really screw
things up because we also re-export PATH based off of its value in turn.
2020-10-25 21:45:45 -05:00
ridiculousfish
a34b9036ba Enable SIGIO notifier on FreeBSD
Now that SIGIO works on FreeBSD, switch to that uvar notifier
2020-10-25 14:53:56 -07:00
ridiculousfish
e669c41d1b Fix up SIGIO notifier tests
FreeBSD has the behavior where SIGIO is delivered on a read. Teach the
tests how to handle this behavior.
2020-10-25 14:53:31 -07:00
ridiculousfish
8bb20a8d91 Remove use of POLL_IN in SIGIO uvar notifier
This fixes up the SIGIO notifier in preparation for using it on BSD. It
removes the reliance on the signal's si_code, which is not available in
BSD, and it properly handles the BSD behavior where SIGIO is delivered on
a read even if the read returns EAGAIN.
2020-10-25 14:52:37 -07:00
Mahmoud Al-Qudsi
3652bcf731 fixup! Fix assertion failure on job redirection error 2020-10-24 17:59:11 -05:00
Mahmoud Al-Qudsi
64671c64a1 Fix assertion failure on job redirection error
Fix an error caused by `exec_job()` assuming a job launched with the
intention of being backgrounded would have a pgid assigned in all cases,
without considering the status of `exec_error` which could have resulted
in the job failing before it was launched into its own process group.

Fixes (but doesn't close) #7423 - that can be closed if this assertion
failure doesn't happen in any released fish versions.
2020-10-24 16:15:40 -05:00
ridiculousfish
bbbbe00c81 Handle being launched with control of the tty, but not in own pgroup
It is apparently possible to launch fish such that its pid owns the tty,
but its pid is in a different pgroup. In that case, do not attempt to stop
with SIGTTIN; instead simply attempt to place fish in its own pgroup.

Fixes #7388
2020-10-18 14:37:31 -07:00
Fabian Homborg
5abc6060a4 Disable sigio notifier on FreeBSD
This fails for unknown reasons.

Not looking like a great *replacement* for the named pipe now, tbh.

See #7400.
2020-10-14 20:40:15 +02:00
Collin Styles
da0acb28ba Don't chomp foo= when completing foo=br
`complete_param_expand` knows how to handle cases like `foo=br` so we
don't need to bother sending just the `br` part. Furthermore, sending
just `br` is incorrect because we will end up replacing the entirety of
`foo=br` with the result of the completion. That is, `foo=br` will be
replaced with `bar` instead of being completed to `foo=bar`.
2020-10-14 18:35:54 +02:00
Johannes Altmanninger
ddf0a8e461 Refactor: slightly rework how variables are assigned during completion
Seems better since we now have two callers. This would be a good use case for
[[nodiscard]].
2020-10-10 13:00:47 +02:00
Johannes Altmanninger
8ef8fb3d94 Refactor: omit parens in lambdas with no parameters
TIL []{} is a thing.  We already do that in some places, so this improves
consistency, although it may be less obvious.
2020-10-10 13:00:24 +02:00