Commit Graph

8989 Commits

Author SHA1 Message Date
Mahmoud Al-Qudsi
a2dda29cf6 Fix build on macOS Yosemite
Older versions of the macOS dev toolchain use MAP_ANON instead of
MAP_ANONYMOUS. We already do this elsewhere.
2018-10-28 12:48:27 -05:00
Mahmoud Al-Qudsi
203de775d0 Fix hang when piping from function to process and exceeding pipe buffer
This is an opposite case from the usual "pipe into grep-the-function"
where my `pbpaste` emitted a lot of content exceeding the OS pipe
buffer. The `block_on_fg` condition was just `send_sigcont` in the
original job control rewrite, and it was incorrect to sub it for
WAIT_BY_PROCESS on its own.

However, this requires always blocking when select_try returns an
interrupted/incomplete read or else fish doesn't block and stays running
in a tight loop in the background (and incorrectly writing to a terminal
it doesn't own under higher debug levels), which I *think* is OK.
2018-10-28 10:35:51 -05:00
Mahmoud Al-Qudsi
1015e74480 Treat _ and - alike for case-insensitive fuzzy matching
Closes #3584.
2018-10-28 10:35:32 -05:00
ridiculousfish
e09e1e8e41 Fix funced when using the built-in editor 2018-10-27 17:19:43 -07:00
Mahmoud Al-Qudsi
17049ce919 Reuse std::locale() across calls within single ifind()
Instantiate the std:locale instance used within the character comparison
callback outside the lambda and take a reference to it instead of
creating the locale object for each character in the sequence.

This is part of a very tight loop with lots of inputs during the
evaluation of fuzzy string matches for completions/autosuggestions and
is worth optimizing.
2018-10-27 18:51:59 -05:00
Mahmoud Al-Qudsi
0d8334a31b Fix hup_background_jobs (née kill_background_jobs) implementation
This was introduced in 1b1bc28c0a but did
not cause any problems until the job control refactor, which caused it
to attempt to signal the calling `exec` builtin's own (invalid) pgrp
with SIGHUP.

Also improved debugging for `j->signal()` failures by printing the
signal we tried sending in case of error, rename the function to
`hup_background_jobs`, and move it from `reader.h`/`reader.cpp` to
`proc.h`/`proc.cpp`.
2018-10-27 18:01:38 -05:00
Mahmoud Al-Qudsi
4d3b56c151 Associate external commands in functions with extant pgrps
When a function is encountered by exec_job, a new context is created for
its execution from the ground up, with a new job and all, ultimately
resulting in a recursive call to exec_job from the same (main) thread.

Since each time exec_job encounters a new job with external commands
that needs terminal control it creates a new pgrp and gives it control
of the terminal (tcsetpgrp & co), this effectively takes control away
from the previously spawned external commands which may be (and likely
are) expecting to still have terminal access.

This commit attempts to detect when such a situation arises by handling
recursive calls to exec_job (which can only happen if the pipeline
included a function) by borrowing the pgrp from the (necessarily still
active) parent job and spawning new external commands into it.

When a parent job spawns new jobs due to the evaluation of a new
function (which shouldn't be the case in the first place), we end up
with two distinct jobs sharing one pgrp (to fix #3952). This can lead to
early termination of a pgrp if finished parent job children are reaped
before future processes in either the parent or future child jobs can
join it.

While the parent job is under construction, require that waitpid(2)
calls for the child job be done by process id and not job pgrp.

Closes #3952.
2018-10-27 18:01:38 -05:00
Mahmoud Al-Qudsi
419d7a5138 Don't decompose shared_ptr to raw pointer for exec_job 2018-10-27 18:01:38 -05:00
Mahmoud Al-Qudsi
008eef50f3 Speed up process_mark_finished_children calls
Use SIGCHLD to determine whether or not waitpid(2) calls can be elided,
but only with extreme caution. If we receive SIGCHLD but are not able to
reap all jobs, we need to iterate through them again.

For this to work, we need to make sure that we reap all children that we
can reap after a SIGCHLD, i.e. it's not OK to just reap the first and
return or else we can never clear the dirty state flag.

In all cases, as expensive as a call to waitpid() may be, if a child
process is available for reaping it is always cheaper to wait on it then
reap it than to call select_try() and end up timing out.
2018-10-27 18:01:38 -05:00
Mahmoud Al-Qudsi
c7f5e58927 More graceful handling of setpgid(2) failure in child_set_group()
Handle EPERM (WSL only?) and EINTR by retrying.
2018-10-27 18:01:38 -05:00
Mahmoud Al-Qudsi
3afcca3114 Drop keepalive process even for WSL
Windows 10 17763 Redstone 5 (October 2018 Update) officially brings
zombie support (first introduced in 17713) to the general public.

See https://docs.microsoft.com/en-us/windows/wsl/release-notes#build-17763-1809
2018-10-27 18:01:38 -05:00
Mahmoud Al-Qudsi
319d1b81fb Add note about PROCESS_EXIT still being used 2018-10-27 18:01:38 -05:00
Mahmoud Al-Qudsi
8072900e16 Change control flow in job_continue()
The old code was rather haphazard with regards to error control, and
would make mutable changes before operations that could fail without any
viable error handling options.
2018-10-27 18:01:38 -05:00
Mahmoud Al-Qudsi
39a05a359a Overhaul continue_job() and try_select()
Convert `select_try()` to return a well-defined enum describing its
state, and handle each of the three possible cases with clear reasons
why we are blocking or not blocking in each subsequent call to
`process_mark_finished_children()`.
2018-10-27 18:01:38 -05:00
Mahmoud Al-Qudsi
1bfbed94ae Clean up terminal_give_to_job()
* Use the newly-introduced signal_block_t RAII wrapper
* Remove EINTR loops as all signals are blocked
* Clean up control flow thanks to RAII wrappers
* Rename parameter to clarify what it does and update docs accordingly
* Update outdated comments referencing SIGSTOP code that was removed a
  long time ago.
* Remove no-op CHECK_BLOCK() call
2018-10-27 18:01:38 -05:00
Mahmoud Al-Qudsi
bd122aa433 Add RAII wrapper for signal_block/signal_unblock 2018-10-27 18:01:38 -05:00
Mahmoud Al-Qudsi
f9118d964e Clean up job flags, status helpers, and instance helper methods
* Convert JOB_* enums to scoped enums
* Convert standalone job_is_* functions to member functions
* Convert standalone job_{promote, signal, continue} to member functions
* Convert standolen job_get{,_from_pid} to `job_t` static functions
* Reduce usage of JOB_* enums outside of proc.cpp by using new
  `job_t::is_foo()` const helper methods instead.

This patch is only a refactor and should not change any functionality or
behavior (both observed and unobserved).
2018-10-27 18:01:38 -05:00
Mahmoud Al-Qudsi
e753581df7 Bring some consistency and rationale to debug log levels
* Debug level 3: describe all commands being executed (this is, after all,
a shell and one can argue that this is the most important debug
information avaliable)
* Debug level 4: details of execution, mainly fork vs no-fork and io
handling

Also introduced j->preview() to print a short descriptor of the job
based on the head of the first process so we don't overwhelm with
needless repitition, but also so that we don't have to rely on
distinguishing between repeated, non-unique/non-monotonic job ids that
are often recycled within a single "execution cycle" (pressing enter
once).
2018-10-27 18:01:38 -05:00
Mahmoud Al-Qudsi
0ff24b35a1 Overhaul behavior of process_mark_finished_children()
Per @ridiculousfish's suggestions in #5219,
`process_mark_finished_children()` has been updated to work in an easier-
to-follow manner. Its behavior is now straight forward, it always checks
for finished processes but only blocks if `block_on_fg` is true.

We're not using the SIGCHLD count in s_sigchld_generation_cnt for
anything any more, as it's not actually a reliable metric since we can
experience one SIGCHLD as a result of two processes exiting (see #1768),
but only reap one of them if the other is in a not-fully-constructed job
(see #5219), a state we cannot possibly detect without calling
`waitpid()` on all child processes, which we are explicitly avoiding.
2018-10-27 18:01:38 -05:00
Mahmoud Al-Qudsi
54050bd4c5 Convert job list to a dequeue
We never insert elements into the middle of a job list, only move
elements to the top. While that can be done "efficiently" with a list, it
can be done faster with a deque, which also won't thrash the cache when
enumerating over jobs.

This speeds up enumeration in the critical path in
`process_mark_finished_children()`.
2018-10-27 18:01:38 -05:00
Mahmoud Al-Qudsi
d467bb58d9 Replace pid/pgid -2 with INVALID_PID 2018-10-27 18:01:38 -05:00
Mahmoud Al-Qudsi
af0c8d51e0 Overhaul job and terminal control
* Instead of reaping all child processes when we receive a SIGCHLD, try
reaping only processes belonging to process groups from fully-
constructed jobs, which should eliminate the need for the keepalive
process entirely (WSL's lack of zombies not withstanding) as now
completed processes are not reaped until the job has been fully
constructed (i.e.  all processes launched), which means their process
group should still be around for new processes to join.

* When `tcgetpgrp()` calls return 0, attempt to `tcsetpgrp()` before
invoking failure handling code.

* When forking a builtin and not running interactively, do not bail if
unable to set/restore terminal attributes.

Fixes #4178. Fixes #3805. Fixes #5210.
2018-10-27 18:01:38 -05:00
ridiculousfish
9454397e4c Correctly split path environment variables about colons
As noted in #5271
2018-10-27 15:20:32 -07:00
ridiculousfish
47890389e1 Merge branch 'uvar_path_support'
Support for --path and --unpath in universal variables.
Closes  #5271
2018-10-27 01:24:27 -07:00
ridiculousfish
5899694233 Allow setting universal path variables
Support for path and unpath in universal variables.
Fixes #5271
2018-10-27 01:05:00 -07:00
ridiculousfish
9690ac5974 Rename fish_universal_variables file to fish_variables
This is to avoid development versions of fish 3.0 freaking out when the
file format is changed. We now have better support for for future universal
variable formats so it's unlikely we'll have to change the file name again.
2018-10-26 16:06:50 -07:00
ridiculousfish
7b5296817a Teach universal variables to not overwrite future file formats 2018-10-26 16:06:50 -07:00
ridiculousfish
d18e2d970c Switch to new universal variable format
Example line:

SETUVAR --export foo:bar
2018-10-26 16:06:50 -07:00
ridiculousfish
11c77abc8c Make universal variable matching case sensitive 2018-10-26 16:06:50 -07:00
ridiculousfish
fe485f2485 Adopt populate_varabless in universal variables 2018-10-26 16:06:50 -07:00
ridiculousfish
adc69f94da Support parsing the new universal variable format 2018-10-26 16:06:50 -07:00
ridiculousfish
d98874bd08 Improve testability factoring of env_universal_t 2018-10-26 16:06:47 -07:00
Fabian Homborg
121d61cf31 Revert "help: Use variable-as-command instead of eval"
We do a bunch of escaping before to make `eval` work, and that needs to be removed as well or fragment-urls don't work.

This reverts commit e9568069a7.
2018-10-26 10:52:02 +02:00
Fabian Homborg
c78e6a3ccf edit_command_buffer: repaint
Thanks @amosbird on gitter.
2018-10-25 17:34:01 +02:00
Fabian Homborg
fb0c1460a9 __fish_complete_pgrep: Truncate to 15 characters
Fixes #4132.
2018-10-25 12:10:17 +02:00
Mahmoud Al-Qudsi
5ad292328a Add dynamic clang completions via clang --autocomplete
Use clang/clang++'s own autocompletion support to complete arguments. It
is rather convoluted as clang generates autocompletions for a portion of
the current token rather than the entire token, e.g. while `--st` will
autocomplete to `--std=` (which is fine by fish), `--std=g` will
autocomplete to `gnu...` without the leading `--std=` which breaks fish'
support for the completion.

Additionally, on systems where clang/clang++ is the system compiler
(such as FreeBSD), it is very often for users to invoke a newer version
of clang/clang++ installed as clang[++]-NN instead of clang. Using a
monkey-patched version of `complete -p` to support that without breaking
(future) completions for commands like `clang-format`.

Closes #4174.
2018-10-24 23:14:55 -05:00
Mahmoud Al-Qudsi
f695284a84 Add fish private mode info to changelog and manual 2018-10-24 19:33:48 +02:00
Mahmoud Al-Qudsi
ebb3a3a16e Set $fish_private_mode and show a message on private mode startup
The message can be localized and is set as a global variable shadowing
the universally-defined $fish_greeting.
2018-10-24 19:33:48 +02:00
Mahmoud Al-Qudsi
379f44fabe Add a --private option to launch fish in private mode
In private mode, access to previous history is blocked and new history
does not persist and is only available for the duration of the current
session.

This mode can be used when it is not desirable for commandline history
to leak into a session, e.g. via autocomplete or when it is desirable to
test the behavior of fish in the absence of history items without
permanently clearing the history.

I'm sure there are a lot more features that can be incorporated into
private mode, such as restricting access to certain user-specific
configuration files, etc.

This addresses a lot of the concerns raised in #1363 (which was later
changed to track mosh-specific problems). See also #102.
2018-10-24 19:33:48 +02:00
Fabian Homborg
b427cd1823 ls.fish: Use gdircolors if available
See #5278.
2018-10-24 19:27:51 +02:00
Fabian Homborg
afc82ff23e Math: Truncate integers (scale == 0)
Fixes #5251.
2018-10-24 18:53:33 +02:00
Fabian Homborg
c5d72332ba io: Explicitly reset discard flag
When we discard output because there's been too much, we print a
warning, but subsequent uses of the same buffer still discard.

Now we explicitly reset the flag, so we warn once and everything works
normal after.

Fixes #5267.
2018-10-24 16:59:24 +02:00
Fabian Homborg
809998a9a5 completions/git: Use argparse
This is a tiny bit faster, but mostly it's more concise and extendable.
2018-10-24 12:26:01 +02:00
Fabian Homborg
381215ef07 completions/git: Let git glob files
Fixes #5229.
2018-10-24 12:22:31 +02:00
Fabian Homborg
6bc2e15953 reader: Stringify completion_apply_to_command_line 2018-10-24 11:28:55 +02:00
Fabian Homborg
410e13dd74 expand: Stringify get_home_directory_name
This is actually nicer than the pointilistic version.
2018-10-23 19:10:14 +02:00
Fabian Homborg
884b4c9a61 parse_util: Remove some wcharisms 2018-10-23 19:10:14 +02:00
Fabian Homborg
f64a87a374 path: Make working_directory wcstring
Kinda weird that that one was a wchar_t*
2018-10-23 19:10:14 +02:00
Fabian Homborg
7533fa89d4 complete: Stringify 2018-10-23 19:10:14 +02:00
Fabian Homborg
57cf9055d5 FAQ: Add pkg-config/splitting question
Fixes #4855.
2018-10-23 16:55:44 +02:00