Commit Graph

13581 Commits

Author SHA1 Message Date
ridiculousfish
98b0ef532f io_buffer_t to store a promise, not a future, to satisfy TSan
io_buffer_t is a buffer that fills itself by reading from a file
descriptor (typically a pipe). When the file descriptor is widowed, the
operation completes, and it reports completion by marking a
`std::promise<void>`. The "main thread" waits for this by waiting on the
promise's future. However TSan was reporting that the future's destructor
races with its promise's wait method. It's not obvious if this is valid,
but we can fix it by keeping the promise alive until the io_buffer_t is
deallocated.

This fixes the TSan issues reported under
`complete_background_fillthread_and_take_buffer` for #7681 (but there
are other unresolved issues).
2021-02-06 13:28:01 -08:00
Fabian Homborg
432f005859 Disable Github Actions tsan
This was updated and now always fails, but it always did so - you can
test it with 3.1.2 as well, it's just not happy with the iothread
stuff.

Because it's super easy to test this locally this disables the github
actions test so it doesn't complain *constantly*.

See #7681
2021-02-06 21:32:42 +01:00
Fabian Homborg
caca4fec22 fds.h: Add missing types.h include
Broke the build on FreeBSD because that defines mode_t there.
2021-02-06 19:59:53 +01:00
ridiculousfish
b5716e97cc Remove fd_set_t
Now that we no longer need to worry about pipes conflicting with
user-specified redirections, we can remove fd_set_t.
2021-02-05 18:14:50 -08:00
ridiculousfish
b79ec0122a Use pipe2 when creating pipes if avaialble
This allows us to avoid marking the pipe as CLOEXEC in some cases,
saving a system call.
2021-02-05 17:58:08 -08:00
ridiculousfish
97f29b1f4d Pipe fds to move to the "high range"
This concerns how fish prevents its own fds from interfering with
user-defined fd redirections, like `echo hi >&5`. fish has historically
done this by tracking all user defined redirections when running a job,
and ensuring that pipes are not assigned the same fds. However this is
annoying to pass around - it means that we have to thread user-defined
redirections into pipe creation.

Take a page from zsh and just ensure that all pipes we create have fds in
the "high range," which here means at least 10. The primary way to do this
is via the F_DUPFD_CLOEXEC syscall, which also sets CLOEXEC, so we aren't
invoking additional syscalls in the common case. This will free us from
having to track which fds are in user-defined redirections.
2021-02-05 17:58:08 -08:00
ridiculousfish
6c4f2622ef iothread's notify pipes to use make_autoclose_pipes
This allows it to take advantage of the upcoming high-range fd changes.
2021-02-05 17:58:08 -08:00
ridiculousfish
4b4bf541d1 Migrate more fd-concerned functions from wutil into fds
Functions like wopen_cloexec have a new home in fds.cpp. This is in
preparation for reworking how internal fds avoid conflict with user fds.
2021-02-05 17:58:08 -08:00
ridiculousfish
6588cf35f4 Move autoclose_pipes_t from io.h to fds.h 2021-02-05 17:58:08 -08:00
ridiculousfish
be9375e914 Migrate autoclose_fd_t to new file fds.h
fds.h will centralize logic around working with file descriptors. In
particular it will be the new home for logic around moving fds to high
unused values, replacing the "avoid conflicts" logic.
2021-02-05 17:58:08 -08:00
ridiculousfish
f0d07f9b1c Fix git test when run from git
Prior to this change, the checks/git.fish test would fail if run from a
git interactive rebase (such as via `git rebase -i --exec 'ninja test'`),
because git itself would inject stuff into the environment. Teach the git
test how to clean up its environment first before running.
2021-02-05 17:58:02 -08:00
Fabian Homborg
b5305ce3d3 Handle backslashes properly in locate_brackets_of_type
This needs to be rewritten, I'm pretty sure we have like 6 of these
kinds of ad-hoc "is this quoted" things lying around.

But for now, at least don't just check if the *previous* character was
a backslash.

Fixes #7685.
2021-02-05 22:03:13 +01:00
Fabian Homborg
c8a91cb067 docs: Fix link in bind
Found while replacing links with :ref: roles, which are checked.
2021-02-05 20:19:28 +01:00
Fabian Homborg
2a9edfe26e help: Better handle builtins when docs aren't installed
__fish_print_commands just prints the commands we have man pages for,
and help uses that to figure out whether it should link
a command or a section. If the docs aren't installed it won't find
anything.

At least check the builtins, because we document them and it's easy.

This probably needs to be added at build time - glob
doc_src/cmds/*.rst.
2021-02-05 17:19:07 +01:00
Fabian Homborg
b70600e070 docs: Remove errant space 2021-02-05 17:11:29 +01:00
Shun Sakai
9147a30926 Update CHANGELOG 2021-02-05 12:25:04 +01:00
Shun Sakai
d8530257c7 Add completions for libavif 2021-02-05 12:25:04 +01:00
Shun Sakai
4d48720d6a Update CHANGELOG 2021-02-05 12:24:23 +01:00
Shun Sakai
060e796577 Add completions for the JPEG XL Reference Software 2021-02-05 12:24:23 +01:00
Michael Jarvis
496d7c44a1 Fix sphinx doc warning
~/src/fish-shell/doc_src/cmds/argparse.rst:103: WARNING: Literal block ends without a blank line; unexpected unindent.
~/src/fish-shell/doc_src/cmds/argparse.rst:103: WARNING: Literal block ends without a blank line; unexpected unindent.
2021-02-05 11:07:58 +01:00
ridiculousfish
97bde2f2bf Further refactoring of io_buffer_t
Previously we sometimes wanted to access an io_buffer_t to append to it
directly, but that's no longer true; all we really care about is its
separated_buffer_t. Make io_bufferfill_t::finish return the
separated_buffer directly, simplifying call sites. No user visible changes
expected here.
2021-02-04 17:14:46 -08:00
ridiculousfish
258149fe2e Improve locking discipline in io_buffer_t
Previously we had a lock that was taken in an ad-hoc manner. Switch to
using owning_lock.
2021-02-04 17:03:54 -08:00
ridiculousfish
8bcc8c1a36 Further cleanup of separated_buffer_t and io_buffer_t
Remove some clinging tendrils of life as a template object.
2021-02-04 16:43:47 -08:00
ridiculousfish
cbf10971f0 Reorganize separated_buffer_t
Move private bits to the bottom and do some other mild cleanup.
2021-02-04 16:06:28 -08:00
ridiculousfish
d578f8d136 separated_buffer_t to accept strings by rvalue reference
This saves a copy in some cases.
2021-02-04 16:02:40 -08:00
ridiculousfish
032467f338 separated_buffer_t to stop being a template
Now that we no longer construct wide separated buffers, it doesn't have
to be templatized.
2021-02-04 15:32:11 -08:00
ridiculousfish
7d494eab5c builtins to write to buffers directly
This concerns builtins writing to an io_buffer_t. io_buffer_t is how fish
captures output, especially in command substitutions:

    set STUFF (string upper stuff)

Recall that io_buffer_t fills itself by reading from an fd (typically
connected to stdout of the command). However if our command is a builtin,
then we can write to the buffer directly.

Prior to this change, when a builtin anticipated writing to an
io_buffer_t, it would first write into an internal buffer, and then after
the builtin was finished, we would copy it to the io_buffer_t. This was
because we didn't have a polymorphic receiver for builtin output: we
always buffered it and then directed it to the io_buffer_t or file
descriptor or stdout or whatever.

Now that we have polymorphpic io_streams_t, we can notice ahead of time
that the builtin output is destined for an internal buffer and have it
just write directly to that buffer. This saves a buffering step, which is
a nice simplification.
2021-02-04 15:21:32 -08:00
ridiculousfish
cd9a035f02 Add a string_output_stream_t to collect builtin output
This is used when creating a function; this breaks a dependency on the
more complicated buffered_output_stream_t to ease refactoring.
2021-02-04 14:12:14 -08:00
ridiculousfish
fc97151aec Add a variant of wcs2string which accepts a ptr, length pair
This will be useful when refactoring separated buffers.
2021-02-04 13:28:48 -08:00
ridiculousfish
86a12e1abd separated_buffer_t::append to stop being a template
In preparation for simplifying how builtins write to buffers, make
append an ordinary function rather than a template function.
2021-02-04 13:19:11 -08:00
ridiculousfish
7e2a538300 create_output_stream_for_builtin to accept read limit directly
This avoids requiring passing in a parser.
2021-02-03 19:00:04 -08:00
Fabian Homborg
3eef295990 docs/argparse: Remove more of the vestigial shortopt mentions
It should only be mentioned as a backwards-compatibility measure,
because it is useless - not even the short flag variable is set.
2021-02-03 19:13:53 +01:00
Fabian Homborg
cf4f8ae929 Revert "Color "ip" output if available"
This reverts commit abb59a6ec9.

This is still buffered if piped to another function, like with the default `grep`.

See #5340, #5356.
2021-02-02 18:44:44 +01:00
Fabian Homborg
d2d18e2a6a docs: Remove references to read history
This hasn't been kept since #5904 in 3.1.0.
2021-02-02 09:42:57 +01:00
Fabian Homborg
bb1aa5e72f docs: Make more code lines shorter 2021-02-02 08:35:38 +01:00
Fabian Homborg
8bb3d1198f docs: Make code lines shorter 2021-02-02 08:29:31 +01:00
Johannes Altmanninger
29ee4d318d Do use autogenerated completions for external git subcommands
Some third party Git tools provide a man page, which we can at least use
for completing options.

The old logic excluded all generated completions for Git subcommands.
Instead, try to load completions for all available external subcommands.
We can use $PATH/git-* because /bin/git-add and friends were removed in Git
1.6.0 in 2008.

Closes #4358 (the "git-foo" wrapping was added in #7652)
2021-02-02 07:54:28 +01:00
Johannes Altmanninger
abb59a6ec9 Color "ip" output if available
This is supported since iproute2 v4.19.0

Closes #5340
2021-02-02 06:39:40 +01:00
Fabian Homborg
2faf814da4 docs: Point away from set -x
This is a common anti-pattern, we should try to get people to do `set -gx`.
2021-02-01 18:12:24 +01:00
Fabian Homborg
97be837ff5 Update pcre2 to 10.36
This performs *most* of the pcreectomy of b418e36f22.

It removes the tests and docs and all the large files, but it does
*not* touch any of the files except for making Find_Package
quiet (783a895b11) or remove the AUTHORS and similar files as
they are very small.

This seems much easier, cleaner, nicer and has 90% of the effect of
the old - the size now is 2.7MB instead of 2.1MB, down from 10MB.

Fixes #7599
2021-02-01 17:37:03 +01:00
ridiculousfish
2d78c9a0d9 Poll the uvar notifier when the reader is interrupted by a signal
While the user waits at the prompt, fish is waiting in select(), on stdin.
The sigio based universal notifier interrupts select() by arranging for a
signal to be delivered, which causes select() to return with EINTR.
However we weren't polling the notifier at that point so we would not
notice uvar changes, until we got some real input.

I didn't notice this when testing, because my testing was changing fish
prompt colors which updated the prompt for other reasons.

Fixes #7671.
2021-01-31 15:42:35 -08:00
ridiculousfish
e4a993c581 Use xar-based workflow for creating Mac packages
As spotted in #7656, macOS installer files built on Big Sur fail signature
verification on macOS 10.11. This is because Big Sur productsign no longer
supplies the SHA-1 hash, and 10.11 does not know how to read the SHA-256
hash.

Replace the productsign flow with a flow based on
http://users.wfu.edu/cottrell/productsign/productsign_linux.html . This
uses the xar tool to digitally sign the installer packages, with both
SHA-1 and SHA-256 hashes.

The xar tool is somewhat tricky to build, so is checked in (as binary!)
compiled for Mac.

To build a Mac package, run make_pkg.sh (which invokes the signing flow)
followed by mac_notarize.sh which adds the notarization.
2021-01-31 14:07:49 -08:00
Fabian Homborg
6442dc96d6 docs: Add a loops section to index
This was only in the tutorial - we really should improve the split here.
2021-01-31 13:15:03 +01:00
Fabian Homborg
7eb616b787 docs: Make some things subsections
E.g. autoloading and aliases are both about functions, variable scope
and overrides are both about variables.

It makes sense to group these together, and this might allow us to
collapse some of the TOC later.
2021-01-31 12:34:47 +01:00
Fabian Homborg
02c11e1db5 docs: Put variable expansion before command substitution
That's the order the parent section lists it in.
2021-01-31 12:30:20 +01:00
Fabian Homborg
892330b904 docs: Drop "Other features" header
This has two features now, there's no need to group it.
2021-01-31 11:59:43 +01:00
Fabian Homborg
00fc56f3e1 docs: Drop weird local table of contents from "Installation" section
This is about a page long, it doesn't need links.
2021-01-31 11:57:51 +01:00
Fabian Homborg
ad560e2b80 docs: Expand a bit on the alias/abbr thing
Also move abbr explanation to interactive use (as abbrs are purely an
interactive concept)

(also add an example to tilde expansion, not making a separate commit
for that)
2021-01-31 11:56:19 +01:00
Fabian Homborg
8e8349d714 docs: Unify job control
Remove the redundant "running multiple programs" section and merge the
"job control" and "background" sections.
2021-01-31 11:49:54 +01:00
Fabian Homborg
9fead046b5 docs: Explain the man/help split in the help section 2021-01-31 11:41:46 +01:00