Commit Graph

3170 Commits

Author SHA1 Message Date
ridiculousfish
3532a955a6 Use parse_util_locate_cmdsubst_range when validating arguments
Removes another usage of parse_util_locate_cmdsubst().
2020-07-14 15:34:26 -07:00
ridiculousfish
68d256a91c Use parse_util_locate_cmdsubst_range in highlighting 2020-07-14 15:27:14 -07:00
ridiculousfish
9cade52c77 Clean up expand_cmdsubst somewhat
Eliminate some of the pointer arithmetic and switch to
parse_util_locate_cmdsubst_range. It's still a pretty ugly function.
2020-07-14 15:19:10 -07:00
ridiculousfish
37dc554fe1 Revert "Remove unnecessary owning_lock usages"
This reverts commit 3a5585df95.

This reverts a change that removed a lock. It's indeed true that in master,
fish script is bound to the main thread. But I'm working to remove that
limitation and these locks are important in that future.
2020-07-12 18:56:39 -07:00
ridiculousfish
4904e4bc41 Fix autosuggestion validation
When switching to the new ast, commands that were not decorated
statements (like function declarations) would be rejected from
autosuggestion validation because we could not find a command. Stop
rejecting them.
2020-07-12 18:47:33 -07:00
Mahmoud Al-Qudsi
3a5585df95 Remove unnecessary owning_lock usages
The owning locks were added after the original code and decorated with
comments indicating they are thread-safe, even though they're only ever
used from the main thread. Presuming the intent was to make future
manipulation of the code safer rather than to actually make use of any
thread safety guarantees, these have been wrapped in a new
`thread_exclusive` type which always calls ASSERT_IS_MAIN_THREAD.

The benefit is that this does not perform a syscall to lock a mutex
each time the variables are accessed.
2020-07-12 20:21:28 -05:00
Mahmoud Al-Qudsi
6d45dd837f Remove trailing ", " in functions output
When executed interactively and not piped, `functions` adds a comma as a
separator between each result. This removes the separator after the last
item.
2020-07-12 19:14:35 -05:00
Mahmoud Al-Qudsi
f1951fed3a Fix suggestionOK variable name 2020-07-12 19:14:35 -05:00
Mahmoud Al-Qudsi
73bf9dd784 Stop calling path_get_path for builtins and functions
highlight.cpp was blindly calling path_get_path for each head command
typed at the prompt which triggers a lot of syscalls via waccess.

It's still going to do that while commands are being composed, but now
it won't if we can make a cheap lookup to the builtins/functions hash
table and can determine that it's a valid command before inspecting the
filesystem.
2020-07-12 19:14:35 -05:00
ridiculousfish
3319e308d0 Make ast::node_t non-virtual
Eliminate its vtable to save 8 bytes per node, which is a lot!
2020-07-12 16:57:30 -07:00
ridiculousfish
a8eb2a6813 Make union_ptr_t's constructor statically type safe
Ensure it cannot be constructed from the wrong node type.
2020-07-12 16:57:30 -07:00
ridiculousfish
8d37be2916 ast lists to use new[] instead of vector
Because the list is not changed after construction, we do not need
the vector's capacity field. This reduces the size of lists from 48
to 32 bytes.
2020-07-12 16:57:30 -07:00
ridiculousfish
c12ab7a674 Rearrange ast::node_t fields to improve packing 2020-07-12 16:57:30 -07:00
ridiculousfish
487de1e6c3 Reduce copying in parse_util_detect_errors
Allow parse_util_detect_errors to accept an already-parsed ast. This
eliminates a copy of the source, which is helpful when executing large
scripts.
2020-07-12 16:57:30 -07:00
ridiculousfish
dfeec433d8 Reduce allocation churn in parse_util_detect_errors
Reuse a single string for storage.
2020-07-12 16:57:30 -07:00
ridiculousfish
9ee5075fc3 Reformat CPP files 2020-07-12 12:21:25 -07:00
ridiculousfish
2a4c545b21 Rework how signals trigger cancellation
When fish receives a "cancellation inducing" signal (SIGINT in particular)
it has to unwind execution - for example while loops or whatever else that
is executing. There are two ways this may come about:

1. The fish process received the signal
2. A child process received the signal

An example of the second case is:

    some_command | some_function

Here `some_command` is the tty owner and so will receive control-C, but
then fish has to cancel function execution.

Prior to this change, these were handled uniformly: both would just set a
cancellation signal inside the parser. However in the future we will have
multiple parsers and it may not be obvious which one to set the flag in.
So instead distinguish these cases: if a process receives SIGINT we mark
the signal in its job group, and if fish receives it we set a global
variable.
2020-07-12 12:16:01 -07:00
ridiculousfish
12d0afa929 Fix some build warnings in fish_tests 2020-07-12 11:41:06 -07:00
ridiculousfish
2e5222ffe8 Finish renaming job tree to job group
Some "tree" terminology was still there.
2020-07-11 17:05:42 -07:00
ridiculousfish
765c48afa4 Migrate the notion of 'foreground' from job to job group
Whether a job is foreground is a property of its pgid, so it belongs
naturally on the job group.
2020-07-11 17:01:52 -07:00
ridiculousfish
225470493b Make parse_token_type_t an enum class
Improves type safety.
2020-07-09 14:22:04 -07:00
ridiculousfish
7ea396ab3f Remove lrand48 checks and support
lrand48 is no longer used.
2020-07-08 11:00:12 -07:00
ridiculousfish
35cb449aa1 Make parse_statement_decoration_t a class enum 2020-07-07 16:28:39 -07:00
ridiculousfish
71a8eb0aa4 parsed_source_t to hold an ast directly instead of through unique_ptr
We have untangled the dependency loop and so now parsed_source_t no longer
requires indirection.
2020-07-07 16:16:45 -07:00
ridiculousfish
5308223212 Migrate next_parse_token into token_stream_t
Cleaning up parse_tree.cpp with an eye to remove it.
2020-07-07 14:01:01 -07:00
ridiculousfish
72e35af381 Remove preceding_escaped_nl
It's no longer necessary for fish_indent
2020-07-07 13:48:35 -07:00
Mahmoud Al-Qudsi
8d25ed962c Add early abortion of completion match attempt 2020-07-06 23:08:19 -05:00
Mahmoud Al-Qudsi
02d0380e6b Make fish_exit workaround for forced exit not SIGHUP-specific 2020-07-05 23:17:21 -05:00
Mahmoud Al-Qudsi
04c6442dcc Allow fish_exit to run even on fish SIGHUP
We were previously aborting the main event loop before calling fish_exit
in the event of a SIGHUP. This patch causes the SIGHUP to be stored in a
separate state variable from a regular "must exit" condition so the
associated event can be fired before we terminate the loop.

All streams are redirected before the event is called to prevent a
SIGTTIN/SIGTTOU due to the user script reading/writing from a disposed
tty.

Closes #7014
2020-07-05 22:18:21 -05:00
Mahmoud Al-Qudsi
791c4fb1dd Fix unused return value errors in fish tests
This error only happens on recent versions of gcc, see previous
commit e6bb7fc973 for more info.

Instead of using `ignore_result()` here, I've added a `system_assert`
function/macro that mimics the behavior of all the other `system()`
calls in the file.
2020-07-05 22:18:21 -05:00
ridiculousfish
bd4c4a9a9c Stop weak linking pre 10.9 macOS symbols
Previously fish weak-linked wcsdup, wcsncasecmp, and wcscasecmp.
This enabled fish to be used on 10.6. However the minimum Mac version
is now 10.9, where these symbols are available.
2020-07-05 12:49:29 -07:00
ridiculousfish
6976d0ee7e Simplify infinite loop fix when parsing "a="
This reworks the "a=" detection to be simpler.
If we detect a variable assignment that produces an error,
simply consume it.

We also take the opportunity to not highlight it as an error,
and add some tests.

Original commit is 1ca05d32d3.
2020-07-05 12:15:18 -07:00
ridiculousfish
23224f71ce Make some variables local which did not need to be static 2020-07-05 12:15:18 -07:00
Johannes Altmanninger
1ca05d32d3 Fix infinite loop when parsing "a="
Typing that command in an interactive prompt would make the highlighter thread
eat up CPU and memory.  Probably not the right fix; I think the token should
already have been consumed when the error is detected, then there is no need
to consume it when unwinding.
2020-07-05 13:19:48 +02:00
Johannes Altmanninger
ada03d3509 Correct pager size when command line soft-wraps
When selectiong a large completion entry in the pager, it would clobber the
prompt. To reproduce, first run this command

	complete -c : -xa '(
		# completion entries that,  when applied to the commandline
		# need one, two, or three lines respectively
		echo 1
		echo 2(string repeat -n (math $COLUMNS - 5) x)
		echo 3(string repeat -n $COLUMNS x)
		printf %s\n n(seq $LINES)
	)'

then type ": " and hit Tab repeatedly. When cycling through completion
entries, observe that fish always tries to render the pager with the same
size, even though the number of lines occupied by the command line buffer
changes due to soft wrapping.

Fix this by rendering the pager after the command line has been rendered, so
we know how many lines we have left.
2020-07-05 08:55:17 +02:00
Mahmoud Al-Qudsi
e628ba51e7 Remove repeated calculation of fixed string's length in a loop 2020-07-04 20:38:06 -05:00
Mahmoud Al-Qudsi
b8a16a8ba0 Convert highlighted_char_t to a struct 2020-07-04 20:23:50 -05:00
ridiculousfish
0c22f67bde Remove the old parser bits
Now that everything has been migrated to the new AST, remove as much of
the parse_tree bits as possible
2020-07-04 14:58:05 -07:00
ridiculousfish
3534c07584 Adopt the new AST in parse_execution
parse_execution is what turns a parsed tree into jobs, etc. Switch it from
parse_tree to the new AST.
2020-07-04 14:58:05 -07:00
ridiculousfish
6c6088f45c Adopt the new AST in fish_tests
This switches fish_tests from parse_tree to the new AST.
2020-07-04 14:58:05 -07:00
ridiculousfish
886603b2ca Adopt the new AST in fish_indent
This switches fish_indent from parsing with parse_tree
to the new ast.

This is the most difficult transition because the new ast retains less
lexical information than the old parse tree. The strategy is:

1. Use parse_util_compute_indents to compute indenting for each token.

2. Compute the "gap text" between the text of significant tokens. This
contains whitespace, comments, etc.

3. "Fix up" the gap text while leaving the significant tokens alone.
2020-07-04 14:58:05 -07:00
ridiculousfish
6b24edccf6 Adopt the new AST in add_pending_with_file_detection
This switches add_pending_with_file_detection from parsing with parse_tree
to the new ast.
2020-07-04 14:58:05 -07:00
ridiculousfish
202fdfa54a Adopt the new AST in parse_util_detect_errors
This switches parse_util_detect_errors from parsing with parse_tree to
the new ast.
2020-07-04 14:58:05 -07:00
ridiculousfish
7bea5ffa2e Adopt the new AST in parse_util_compute_indents
This switches parse_util_compute_indents from parsing with parse_tree to
the new ast.

It also reworks the parse_util_compute_indents tests, because
parse_util_compute_indents will be the backing for fish_indent.
2020-07-04 14:58:05 -07:00
ridiculousfish
358d7072a2 Adopt the new AST in bash history import
This switches bash history importing from parsing with parse_tree to the
new ast.
2020-07-04 14:58:05 -07:00
ridiculousfish
46c4ec8d68 Adopt the new AST in completion argument lists
This switches completion argument list expansion from parsing with
parse_tree to the new ast.
2020-07-04 14:58:05 -07:00
ridiculousfish
0d4f9c6220 Adopt the new AST in abbreviation expansion
This switches abbreviation expansion from parsing with parse_tree to the
new ast.
2020-07-04 14:58:05 -07:00
ridiculousfish
8d9725c301 Adopt the new AST in highlighting
This switches syntax highlighting from parsing with parse_tree to the new
ast.
2020-07-04 14:58:05 -07:00
ridiculousfish
4d4455007d Introduce a new fish ast
This is the first commit of a series intended to replace the existing
"parse tree" machinery. It adds a new abstract syntax tree and uses a more
normal recursive descent parser.

Initially there are no users of the new ast. The following commits will
replace parse_tree -> ast for all usages.
2020-07-04 14:58:02 -07:00
ridiculousfish
45c9e3b0f1 parsed_source_ref to always make a job_list
Removed an unnecessary param in preparation for more changes.
2020-07-04 14:51:15 -07:00
ridiculousfish
e530163078 Revert "fix unreachable code warning"
This reverts commit 202fe39d34.

If mkostemp is a weak symbol and is null, then the code will
be reachable.
2020-07-04 14:49:05 -07:00
ridiculousfish
90d8df8128 Use _POSIX_VDISABLE instead of \0 to disable control functions
Prior to this commit, fish used NUL ('\0') to disable control
functions (for example, the function that generates SIGTSTP).
However NUL may in fact be bindable and is on macOS via
control-space.

Use instead _POSIX_VDISABLE if defined and not -1.
2020-07-01 22:33:31 -07:00
Fabian Homborg
77f412af1b fixup! Also clear suggestions
Also #7145.
2020-07-01 21:00:16 +02:00
Fabian Homborg
eb35975c0f Make cancel-commandline actual bind function
This was always awkward as fish script, and had problems with
interrupting the autoloading.

Note that we still leave the old function intact to facilitate easier
upgrading for now.

Fixes #7145.
2020-07-01 20:56:56 +02:00
Johannes Altmanninger
76e0875c8f Apply clang-format 10 and selected lints from "make lint-all" 2020-07-01 00:44:06 +02:00
Johannes Altmanninger
25fe353187 Readline command beginning-of-history visits the oldest search match
Previously it would do the same as end-of-history
2020-07-01 00:40:32 +02:00
Johannes Altmanninger
e5b8035fea Fix updating command line on Page-Down
Page-Down seems to deactivate history search, so trying to undo
would leave the command line in an inconsistent state.

Fixes #7162 which was introduced in
12a9cb29 Fix assertion failure on page up / page down
2020-07-01 00:40:32 +02:00
Johannes Altmanninger
2b3b460264 Make reader_history_search_t::matches a vector instead of a deque
It is used exclusively as vector at the moment since we only ever append
at the end.  Making it a deque would be useful when allowing to edit the
search string and subsequently resume the search at an arbitrary position
in the history.
2020-07-01 00:40:32 +02:00
Johannes Altmanninger
971c2eb668 history search: do not move the cursor after failing backward search
When editing a multiline command line and pressing "up" with the cursor at the
first line, fish attempts a hsitory search. If the search fails, don't move
the cursor to the end of the multiline command because this can be annoying
when the user does not actually want to perform a history search.
2020-07-01 00:40:32 +02:00
Johannes Altmanninger
4f796df87c Clarify comment 2020-07-01 00:40:32 +02:00
Mahmoud Al-Qudsi
0b55f08de2 Speed up executable command completions
This brings down the number of syscalls per potential completion result
from three or four to just one.
2020-06-29 09:27:49 -05:00
Mahmoud Al-Qudsi
48b59cc194 Fix detection of executables with 007 mode
It's not surprising that this never came up, as I cannot imagine a more
useless chmod value. Perhaps in the context of nologin or something?
*shrug*
2020-06-28 23:06:44 -05:00
Mahmoud Al-Qudsi
f5b431c21b Remove std::move blocking potential copy elision 2020-06-28 18:08:13 -05:00
Mahmoud Al-Qudsi
0414be75cc fixup! set_color: Don't error with "-b" and no argument 2020-06-26 19:46:23 -05:00
Hugo Gualandi
de9e8cb897 Fix binfmts.h typo in a comment
The name of the header file is binfmts.h, with an "s" at the end.
2020-06-26 21:00:25 +02:00
Fabian Homborg
86a6a205e7 set_color: Don't error with "-b" and no argument
We already don't print an error with just `set_color`, so it should be
consistent.

Fixes #7154

(also removes an impossible exit)
2020-06-26 20:59:17 +02:00
Mahmoud Al-Qudsi
4ba9e70f9a Remove extraneous line break in the middle of an error message 2020-06-25 23:01:59 -05:00
Mahmoud Al-Qudsi
34b918d0a0 Reduce unneeded calls to tcgetpgrp
After profiling bottlenecks in job execution, the calls to `tcgetpgrp`
were identified to take a good amount of the execution time. Collecting
metrics on which branches were taken revealed that in all "normal"
cases, there is no benefit to calling `tcgetpgrp` before calling
`tcsetpgrp` as it can instead be called only in the error case to
determine what sort of error handling behavior should be applied.

This makes the best-case scenario of a single syscall much more likely
than in the previous situation.
2020-06-25 22:46:09 -05:00
Mahmoud Al-Qudsi
c3849ebeba Convert var_table_t to an unordered_map
Profiling revealed string comparison in variable lookups to be a
significant hotspot. This change causes `make test` to complete ~4.5%
faster per `hyperfine`.
2020-06-25 00:56:49 -05:00
Mahmoud Al-Qudsi
a5be15da69 Optimize lookup of electric variables 2020-06-24 22:46:02 -05:00
Mahmoud Al-Qudsi
9bddd78239 fixup! Eliminate recursive calculation of string length in wildcard matching
This was originally comparing two pointers for equality but after the
refactor to wcstring it ended up comparing a const string pointer to the
_contents_ of the wcstring.
2020-06-24 21:53:05 -05:00
Mahmoud Al-Qudsi
bfa17ecff6 Fix call site for wildcard_match_internal 2020-06-24 21:23:18 -05:00
Mahmoud Al-Qudsi
a6f1e6119b Eliminate recursive calculation of string length in wildcard matching 2020-06-24 17:21:18 -05:00
Fabian Homborg
149a0b98af Another formatting run
I really kinda hate how insistent clang-format is to have line
breaks *IFF THE LINE IS TOO LONG*.

Like... lemme just add a break if it looks better, will you?

But it is the style at this time, so we shall tie an onion to our
belt.
2020-06-24 20:43:56 +02:00
Fabian Homborg
d0c1f2a2cc Remove duplicated line
Oops.
2020-06-24 18:33:26 +02:00
Fabian Homborg
3d2dc856ab Disable the SUSP character
This makes binding \cz possible.

We already ignore the SIGTSTP signal it sends, so until now it was useless.

(also STOP and START for good measure, but since we disable flow
control in fish anyway these already shouldn't have been sent)

Fixes #7152
2020-06-24 18:26:17 +02:00
Mahmoud Al-Qudsi
8b910d6de2 Further optimize performance of terminal output
Profiling shows that parsing color names still took much longer than it
should.

wcscasecmp is so slow that using it directly causes `try_parse_special`
to consume up to 3% of all of fish's cpu time due to extremely
inefficient invariant case lookups for wide characters (tested: Fedora
Server 32 w/ glibc 2.31 with -O2).
2020-06-22 23:39:51 -05:00
Mahmoud Al-Qudsi
3c90b6c57d Re-implement optimizations to color lookup by name
Hopefully without the null-termination issues this time around. Fixed by simply
cloning the str object as-is then transforming it.
2020-06-20 17:00:30 -05:00
Fabian Homborg
7447c8faae Also undo changes to ICRNL and INLCR
These control the disambiguation between ctrl-j and ctrl-m.

This can cause the enter key to send a ctrl-m, which programs might be
unprepared for.

(This is why you need to do `stty sane<ctrl-j>`)
2020-06-20 23:40:37 +02:00
Fabian Homborg
30eb4d8b0d Revert "Optimize lookup of colors by name"
Yeah, there's some weirdness here with ASAN that I can't reproduce locally, so we revert it for now.

This reverts commit a6efe0f0e4.
2020-06-20 22:35:14 +02:00
Fabian Homborg
40de4ef764 Color lookup: Use wcsncmp to avoid looking at garbage
The `reserve` here can, under certain circumstances, reserve more than
strictly needed.

The simple workaround is to just never look at more than we feed in.

(really what we'd *want* is to look at the length of the *color
names*, but those are wchar, so length lookup is crappy NULL-lookup)
2020-06-20 22:13:54 +02:00
Mahmoud Al-Qudsi
a6efe0f0e4 Optimize lookup of colors by name
This was profiled to be a hotspot during startup. The usage of
wcscasecmp in a loop caused repeated transforms to lowercase, which is
incredibly slow for wide characters.
2020-06-20 13:57:24 -05:00
Mahmoud Al-Qudsi
f2d5f95396 Merge sigint_checker_t generalizations from #7060
This makes it possible to expand the signals checked by the type. I can't merge
the sigttin fixes for #7060 yet because they introduce new breakage, but this
will make merging any future fix easier.
2020-06-20 11:27:15 -05:00
Fabian Homborg
e9e23a8333 Force some terminal modes even for external commands
In #7133, neovim crashing caused "OPOST" to be turned off, which
caused a weird staircase display.

So we just force a set of settings that don't seem useful to change to
avoid breaking the terminal with something like that.

Fixes #7133.
2020-06-19 21:09:08 +02:00
Fabian Homborg
a121833e88 argparse: Only print stacktrace when it's an error in argparse usage
A broken/missing optspec or `--` is a bug in the script using
argparse, an unknown option or invalid argument is a bug in using that script.

So in the former case print a stacktrace, because the person writing
the `argparse` call is at fault, in the latter don't.

Fixes #6703.
2020-06-17 20:05:48 +02:00
ridiculousfish
e91a06b764 Correct a misspeeling 2020-06-09 19:57:04 -07:00
ridiculousfish
269f907f2f Use inline ivar initialization in parse_node_t 2020-06-09 15:16:31 -07:00
ridiculousfish
19293ec2d6 Make parse_keyword_t an enum class 2020-06-09 15:13:02 -07:00
ridiculousfish
b4351c5927 Clean up posix_spawn code paths
Prior to this change, the posix_spawn code paths used a fair amount of
manual management around its allocated structures (attrs and file actions).
Encapsulate this into a new class that manages memory management and error
handling.
2020-06-09 14:59:06 -07:00
ridiculousfish
c7160d7cb4 Eliminate the termsize handling from common.h
Finish the transition to termsize.h. Remove the scary termsize bits
from common.cpp, which can throw off events at arbitrary calls and are
dangerously reentrant. Migrate everyone to the new termsize.h.
2020-06-07 20:00:42 -07:00
ridiculousfish
6bdbe732e4 Adopt termsize_t in the pager 2020-06-07 20:00:42 -07:00
ridiculousfish
db909605b8 Migrate reformat_for_screen to new termsize container 2020-06-07 20:00:42 -07:00
ridiculousfish
340c8490f6 Introduce termsize_container_t
fish's handling of terminal sizes is currently rather twisted. The
essential problem is that the terminal size may change at any point from a
SIGWINCH, and common_get_{width,height} may modify it and post variable
change events from arbitrary locations.

Tighten up the semantics. Assign responsibility for managing the tty size
to a new class, `termsize_container_t`. Rationalize locking and reentrancy.

Explicitly nail down the relationship between $COLUMNS/$LINES and the tty
size. The new semantics are: whatever changed most recently takes
precendence.
2020-06-07 20:00:42 -07:00
ridiculousfish
d5a239e59e Bravely stop attempting to modify the terminal size
Prior to this fix, fish would attempt to resize the terminal via
TIOCSWINSZ, which was added as part of #3740. In practice this probably
never did anything useful since generally only the tty master can use
this. Remove the support and note it in the changelog.
2020-06-07 20:00:42 -07:00
ridiculousfish
df618a0768 Migrate DFLT_TERM from common.h to env.cpp
There's no reason every .cpp file needs to see these values.
2020-06-07 20:00:41 -07:00
ridiculousfish
7b486b4634 Factor s_reset better
Prior to this fix, s_reset would attempt to reset the screen, optionally
using the PROMPT_SP hack to go to the next line. This in turn required
passing in the screen width even if it wasn't needed (because we were
not going to abandon the line). Factor this into two functions:

- s_reset_line which does not apply the hack
- s_reset_abandoning_line which applies the PROMPT_SP hack
2020-06-07 20:00:41 -07:00
ridiculousfish
fc42516dfb Unwind some calls to common_get_width from inside screen
common_get_width will "lazily" decide the screen width, which means
changing the environment variable stack. This is a surprising thing
to do from the middle of screen rendering.

Switch to passing in widths explicitly to screen.
2020-06-07 20:00:41 -07:00
ridiculousfish
0673d86242 Remove a dead overload of s_reset
This function was not actually implemented.
2020-06-07 20:00:41 -07:00
ridiculousfish
812cc1dbaf Clean up line_t
Use a single allocation instead of two for text and colors.
Comment and tighten up its methods.
2020-06-07 20:00:41 -07:00
ridiculousfish
5429b54258 Remove k_invalid_termsize 2020-06-07 20:00:41 -07:00