We have now entirely switched the script tests to littlecheck.
Note: This adjusts the complete_directories test, because it removes a
directory that was created before by a .in test. There's no real
change in behavior.
This does require the test directory be cleaned, or the tests will fail.
test_util gets to stay for a while longer, because it sets up the
testing env (locale and such).
This, together with the other testX, really just tests some basic
syntax. So let's just call it "basic".
Note that this file uses escaped newlines on purpose, so restyling it
would currently break it. I'm not sure what the best thing to do here is.
Instead of invoking littlecheck.py independently for each file, pass
all files at once. This amortizes the Python startup cost, and reduces
the total test time by ~15 seconds (!).
This isn't quite the old-style test, but it checks some of the line
continuation stuff.
Note that littlecheck ignores leading whitespace, so testing the
actual indentation requires some more effort.
Things like
```fish
\
echo foo
```
or
```fish
echo foo; \
echo bar
```
are a formatting blunder and should be handled.
This makes it so the escaped newline is removed, and the
semicolon/token_type_end handling will then put the statements on
different lines.
One case this doesn't handle brilliantly is an escaped newline after a
pipe:
```fish
echo foo | \
cat
```
is turned into
```fish
echo foo | cat
```
which here works great, but in long pipelines can cause issues.
Pipes at the end of the line cause fish to continue parsing on the
next line, so this can just be written as
```fish
echo foo |
cat
```
for now.
This tries to see if quotes guard some expansion from happening. If it
detects a "weird" character it'll leave the quotes in place, even in
some cases where it might not trigger.
So
for i in 'c' 'color'
turns into
for i in c color
The rationale here is that these quotes are useless, wasting
space (and line length), but more importantly that they are
superstitions. They don't do anything, but look like they do.
The counter argument is that they can be kept in case of later
changes, or that they make the intent clear - "this is supposed to be
a string we pass".
The problem is that under TSAN, the timing of signals becomes very weird and
exposes some real race conditions. We will need to re-design how signal
event handlers work.
This test launches two background processes and is sensitive to
interleaving of output. Fix it so that newlines are not output by
the background process.
Hopefully this fixes the flakiness of this test.
f8ba0ac5bf introduced a bug where INT handlers would themselves be
cancelled, due to the signal. Defer processing handlers until the
parser is ready to execute more fish script.
Fixes the interactive case of #6649.
If a background process runs a fish function which launches another
background process, ensure that these background procs get different
pgroups. Add a test for it.
This executes `fish --no-execute` a whole bunch of times in order to
find syntax errors in our fish scripts.
tests/ is exempt because it contains syntax errors on purpose.
This is a great idea in principle, but it takes ~4s on my system.
It used to error out when a command wasn't known, even when it was a
function that would only be discovered via autoloading.
Now we just accept that a command doesn't exist when no-execute is
given - we're not gonna execute it anyway.
Also, in the same breath stop counting empty commands after expansion
and empty wildcard expansions as errors - these depend on runtime
values, so we can't verify them without executing.
Fixes#977.
(note that it still executes "time", but that's another commit)
Appending to an fd doesn't really make sense, but we allowed the
syntax previously and it was actually used.
It's not too harmful to allow it, so let's just do that again.
For the record: Zsh also allows it, bash doesn't.
Fixes#6614
Glob ordering is used in a variety of places, including figuring out
conf.d and really needs to be stable.
Other ordering, like completions, is really just cosmetic and can
change if it makes for a nicer experience.
So we uncouple it by copying the wcsfilecmp from 3.0.2, which will
return the ordering to what it was in that release.
Fixes#6593
The `function --on-job-exit caller` feature allows a command substitution
to observe when the parent job exits. This has never worked very well - in
particular it is based on job IDs, so a function that observes this will
run multiple times. Implement it properly.
Do this by having a not-recycled "internal job id".
This is only used by psub, but ensure it works properly none-the-less.
This one tests a bunch of separate stuff, so we put it into a few
different files.
The main, new one is "slices.fish", which tests various index expressions.
This makes two changes:
1. Remove the 'brace_text_start' idea. The idea of 'brace_text_start' was
to prevent emitting `BRACE_SPACE` at the beginning or end of an item. But
we later strip these off anyways, so there is no apparent benefit. If we
are not doing brace expansion, this prevented emitting whitespace at the
beginning or end of an item, leading to #6564.
2. When performing brace expansion, only stomp the space character with
`BRACE_SPACE`; do not stomp newlines and tabs. This is because the fix in
came from a newline or tab literal, then we would have effectively
replaced a newline or tab with a space, so this is important for #6564 as
well. Moreover, it is not easy to place a literal newline or tab inside a
brace expansion, and users who do probably do not mean for it to be
stripped, so I believe this is a good change in general.
Fixes#6564
Just another version of the error. We still want to get a bug if it
ever triggers a *wrong* error, so we still list all the options
instead of going for `.*option:.*Z.*`.
Fixes#6554
Solaris/OpenIndiana/Illumos `rm` checks that and errors out.
In these cases we don't actually need it to be a part of $PWD as
it's just for cleanup, so we `cd` out before.
See #5472
See 1ee57e9244Fixes#6555Fixes#6558
Prior to this fix, fish was rather inconsistent in when $status gets set
in response to an error. For example, a failed expansion like "$foo["
would not modify $status.
This makes the following inter-related changes:
1. String expansion now directly returns the value to set for $status on
error. The value is always used.
2. parser_t::eval() now directly returns the proc_status_t, which cleans
up a lot of call sites.
3. We expose a new function exec_subshell_for_expand() which ignores
$status but returns errors specifically related to subshell expansion.
4. We reify the notion of "expansion breaking" errors. These include
command-not-found, expand syntax errors, and others.
The upshot is we are more consistent about always setting $status on
errors.
macOS `mktemp -d` likes to return symlinks. Guard against that possibility.
That allows the test to succeed when run directly, instead of through the
build target.
It was possible to start the new job and execute `jobs` again before
the job died (or we noticed it did), so the test would fail.
To properly test, we need to ensure the job has been removed. `wait`
should do it.
complete -C'echo $HOM ' would complete $HOM instead of a new token.
Fixes another regression introduced in
6fb7f9b6b - Fix completion for builtins with subcommands
It's now good enough to do so.
We don't allow grid-alignment:
```fish
complete -c foo -s b -l barnanana -a '(something)'
complete -c foo -s z -a '(something)'
```
becomes
```fish
complete -c foo -s b -l barnanana -a '(something)'
complete -c foo -s z -a '(something)'
```
It's just more trouble than it is worth.
The one part I'd change:
We align and/or'd parts of an if-condition with the in-block code:
```fish
if true
and false
dosomething
end
```
becomes
```fish
if true
and false
dosomething
end
```
but it's not used terribly much and if we ever fix it we can just
reindent.
for-loops that were not inside a function could overwrite global
and universal variables with the loop variable. Avoid this by making
for-loop-variables local variables in their enclosing scope.
This means that if someone does:
set a global
for a in local; end
echo $a
The local $a will shadow the global one (but not be visible in child
scopes). Which is surprising, but less dangerous than the previous
behavior.
The detection whether the loop is running inside a function was failing
inside command substitutions. Remove this special handling of functions
alltogether, it's not needed anymore.
Fixes#6480
Store the entire function declaration, not just its job list.
This allows us to extract the body of the function complete with any
leading comments and indents.
Fixes#5285
In particular, this allows `true && time true`, or `true; and time true`,
and both `time not true` as well as `not time true` (like bash).
time is valid only as job _prefix_, so `true | time true` could call
`/bin/time` (same in bash)
See discussion in #6442
Extend the commit 8e17d29e04 to block processes, for example:
begin ; stuff ; end
or if/while blocks as well.
Note there's an existing optimization where we do not create a job for a
block if it has no redirections.
job_promote attempts to bring the most recently "touched" job to the front
of the job list. It did this via:
std::rotate(begin, job, end)
However this has the effect of pushing job-1 to the end. That is,
promoting '2' in [1, 2, 3] would result in [2, 3, 1].
Correct this by replacing it with:
std::rotate(begin, job, job+1);
now we get the desired [2, 1, 3].
Also add a test.
It looks like the last status already contains the signal that cancelled
execution.
Also make `fish -c something` always return the last exit status of
"something", instead of hardcoded 127 if exited or signalled.
Fixes#6444
This was previously required so that, if there was a redirection to a
file, we would fork a process to create the file even if there was no
output. For example `echo -n >/tmp/file.txt` would have to create
file.txt even though it would be empty.
However now we open the file before fork, so we no longer need special
logic around this.
Do this only when splitting on IFS characters which usually contains
whitespace characters --- read --delimiter is unchanged; it still
consumes no more than one delimiter per variable. This seems better,
because it allows arbitrary delimiters in the last field.
Fixes#6406
This adds a test for the obscure case where an fd is redirected to
itself. This is tricky because the dup2 will not clear the CLO_EXEC bit.
So do it manually; also posix_spawn can't be used in this case.
The IO cleanup left file redirections open in the child. For example,
/bin/cmd < file.txt would redirect stdin but also leave the file open.
Ensure these get closed properly.
Prior to this fix, a job would hold onto any IO redirections from its
parent. For example:
begin
echo a
end < file.txt
The "echo a" job would hold a reference to the I/O redirection.
The problem is that jobs then extend the life of pipes until the job is
cleaned up. This can prevent pipes from closing, leading to hangs.
Fix this by not storing the block IO; this ensures that jobs do not
prolong the life of pipes.
Fixes#6397
(command pwd) uses the system's implementation of pwd. At least the GNU
coreutils implementation defaults to -P, which resulted in symlinks being
expanded when switching between directories with nextd/prevd.
This did some weird unescaping to try to extract the first word.
So we're now more likely to be *correct*, and the alias benchmark is
about 20% *faster*.
Call it a win-win.
This splits a string into variables according to the shell's
tokenization rules, considering quoting, escaping etc.
This runs an automatic `unescape` on the string so it's presented like
it would be passed to the command. E.g.
printf '%s\n' a\ b
returns the tokens
printf
%s\n
a b
It might be useful to add another mode "--tokenize-raw" that doesn't
do that, but this seems to be the more useful of the two.
Fixes#3823.
This added the function offset *again*, but it's already included in
the line for the current file.
And yes, I have explicitly tested a function file with a function
defined at a later line.
Fixes#6350
Since #6287, bare variable assignments do not parse, which broke
the "Unsupported use of '='" error message.
This commit catches parse errors that occur on bare variable assignments.
When a statement node fails to parse, then we check if there is at least one
prefixing variable assignment. If so, we emit the old error message.
See also #6347
This adds initial support for statements with prefixed variable assignments.
Statments like this are supported:
a=1 b=$a echo $b # outputs 1
Just like in other shells, the left-hand side of each assignment must
be a valid variable identifier (no quoting/escaping). Array indexing
(PATH[1]=/bin ls $PATH) is *not* yet supported, but can be added fairly
easily.
The right hand side may be any valid string token, like a command
substitution, or a brace expansion.
Since `a=* foo` is equivalent to `begin set -lx a *; foo; end`,
the assignment, like `set`, uses nullglob behavior, e.g. below command
can safely be used to check if a directory is empty.
x=/nothing/{,.}* test (count $x) -eq 0
Generic file completion is done after the equal sign, so for example
pressing tab after something like `HOME=/` completes files in the
root directory
Subcommand completion works, so something like
`GIT_DIR=repo.git and command git ` correctly calls git completions
(but the git completion does not use the variable as of now).
The variable assignment is highlighted like an argument.
Closes#6048
Presently the completion engine ignores builtins that are part of the
fish syntax. This can be a problem when completing a string that was
based on the output of `commandline -p`. This changes completions to
treat these builtins like any other command.
This also disables generic (filename) completion inside comments and
after strings that do not tokenize.
Additionally, comments are stripped off the output of `commandline -p`.
Fixes#5415Fixes#2705
PATH and CDPATH have special behavior around empty elements. Express this
directly in env_stack_t::set rather than via variable dispatch; this is
cleaner.
This adds support for `fish_trace`, a new variable intended to serve the
same purpose as `set -x` as in bash. Setting this variable to anything
non-empty causes execution to be traced. In the future we may give more
specific meaning to the value of the variable.
The user's prompt is not traced unless you run it explicitly. Events are
also not traced because it is noisy; however autoloading is.
Fixes#3427
Until now, something like
`math '7 = 2'`
would complain about a "missing" operator.
Now we print an error about logical operators not being supported and
point the user towards `test`.
Fixes#6096
The `--entire` would enable output even though the `--quiet` should
have silenced it. These two don't make any sense together so print an
error, because the user could have just left off the `-q`.
This spewed errors because the `math` invocation got no second
operand:
Testing file checks/sigint.fish ... math: Error: Too few arguments
'1571487730 -'
but only if the `date` didn't do milliseconds, which is the case on
FreeBSD and NetBSD.
(also force the variable to be global - we don't want to have a
universal causing trouble here)
Universal exported variables (created by `set -xU`) used to show up
both as universal and global variable in child instances of fish.
As a result, when changing an exported universal variable, the
new value would only be visible after a new login (or deleting the
variable from global scope in each fish instance).
Additionally, something like `set -xU EDITOR vim -g` would be imported
into the global scope as a single word resulting in failures to
execute $EDITOR in fish.
We cannot simply give precedence to universal variables, because
another process might have exported the same variable. Instead, we
only skip importing a variable when it is equivalent to an exported
universal variable with the same name. We compare their values after
joining with spaces, hence skipping those imports does not change the
environment fish passes to its children. Only the representation in
fish is changed from `"vim -g"` to `vim -g`.
Closes#5258.
This eliminates the issue #5348 for universal variables.
Consider a group of short options, like -xzPARAM, where x and z are options and z takes an argument.
This commit enables completion of the argument to the last option (z), both within the same
token (-xzP) or in the next one (-xz P).
complete -C'-xz' will complete only parameters to z.
complete -C'-xz ' will complete only parameters to z if z requires a parameter
otherwise, it will also complete non-option parameters
To do so this implements a heuristic to differentiate such strings from single long options. To
detect whether our token contains some short options, we only require the first character after the
dash (here x) to be an option. Previously, all characters had to be short options. The last option
in our example is z. Everything after the last option is assumed to be a parameter to the last
option.
Assume there is also a single long option -x-foo, then complete -C'-x' will suggest both -x-foo and
-xy. However, when the single option x requires an argument, this will not suggest -x-foo.
However, I assume this will almost never happen in practise since completions very rarely mix
short and single long options.
Fixes#332
This stops reading argument names after another option appears. It does not break any previous uses and in fact fixes uses like
```fish
function foo --argument-names bar --description baz
```
* `function` command handles options after argument names (Fixes#6186)
* Removed unneccesary test
I got tired of seeing ' ... ok (0 sec)' so now with GNU date/gdate
installed there is millisecond output shown. One can get rough
nanoseconds from gdate.
This sequence can be generatd by control-spacebar. Allow it to be bound
properly.
To do this we must be sure that we never round-trip the key sequence
through a C string.
Meaning empty variables, command substitutions that don't print
anything.
A switch without an argument
```fish
switch
case ...
end
```
is still a syntax error, and more than one argument is still a runtime
error.
The none-argument matches either an empty-string `case ''` or a
catch-all `case '*'`.
Fixes#5677.
Fixes#4943.
This test uses universal variables, and so it can fail when run
multiple times.
It might be a good idea to do this in general, but for now let's just
try it here.
Previously when propagating explicitly separated output, we would early-out
if the buffer was empty, where empty meant contains no characters. However
it may contain one or more empty strings, in which case we should propagate
those strings.
Remove this footgun "empty" function and handle this properly.
Fixes#5987
I tested this manually (`littlecheck.py -s fish=fish tests/checks/eval.fish`) from the base directory, which means I got
"tests/checks/eval", while the real test gets "checks/eval".
I then reran `make test_fishscript`, but that didn't pull in the
updated test - we should really handle that better.
I'm gonna add more tests to this and I don't want to touch the old stuff.
Notice that this needs to have the output of the complete_directories
test adjusted because this one now runs later.
That's something we should take into account in future.
history now often writes to the history file asynchronously, but the history
test expects to find the text in the file immediately after running the
command. Hack a bit in history to make this test more reliable.
This required a bit of thinking.
What we do is we have one test that fakes $HOME, and then we do the
various config tests there.
The fake config we have is reused and we exercise all of the same codepaths.
This prints a green "ok" with the duration, just like the rest of the
tests.
Note that this clashes a bit with
https://github.com/ridiculousfish/littlecheck/pull/3.
(also don't check for python again and again and again)
This is a bit weird sometimes, e.g. to test the return status (that
fish actually *returns $status*), we use a #RUN line with %fish
invoking %fish, so we can use the substitution.
Still much nicer.
The missing scripts are those that rely on config.
Especially as, in this case, the documentation is quite massive.
Caught by porting string's test to littlecheck.
See #3404 - this was already supposed to be included.
This is a nice test (ha!) for how this works and what littlecheck can
do for us.
1. Input is now the actual file, not "Standard Input" anymore. So
any errors mentioning that now include the filename.
2. Regex are really nice for filenames, but especially for line
numbers
3. It's much nicer to have the output where it's created, instead of
needing to follow three files at the same time.
Instead of requiring a flag to enable newline trimming, invert it so the
flag (now `--no-trim-newlines`) disables newline trimming. This way our
default behavior matches that of sh's `"$(cmd)"`.
Also change newline trimming to trim all newlines instead of just one,
again to match sh's behavior.
The `string collect` subcommand behaves quite similarly in practice to
`string split0 -m 0` in that it doesn't split its output, but it also
takes an optional `--trim-newline` flag to trim a single trailing
newline off of the output.
See issue #159.
This adds support for .check files inside the tests directory. .check
files are tests designed to be run with littlecheck.
Port printf test to littlecheck and remove the printf.in test.
It's always a bit annoying that `*` requires quoting.
So we allow "x" as an alternative, only it needs to be followed by
whitespace to distinguish it from "0x" hexadecimal notation.
This makes the following changes:
1. Events in background threads are executed in those threads, instead of
being silently dropped
2. Blocked events are now per-parser instead of global
3. Events are posted in builtin_set instead of within the environment stack
The last one means that we no longer support event handlers for implicit
sets like (example) argv. Instead only the `set` builtin (and also `cd`)
post variable-change events.
Events from universal variable changes are still not fully rationalized.
When setting a variable without a specified scope, we should give priority
to an existing local or global above an existing universal variable with
the same name.
In 16fd780484 there was a regression that
made universal variables have priority.
Fixes#5883
Brace expansion with single words in it is quite useless - `HEAD@{0}`
expanding to `HEAD@0` breaks git.
So we complicate the rule slightly - if there is no variable expansion
or "," inside of braces, they are just treated as literal braces.
Note that this is technically backwards-incompatible, because
echo foo{0}
will now print `foo{0}` instead of `foo0`. However that's a
technicality because the braces were literally useless in that case.
Our tests needed to be adjusted, but that's because they are meant to
exercise this in weird ways.
I don't believe this will break any code in practice.
Fixes#5869.
This read something like `o=!_validate_int`, and the flag modifier
reading kept the pointer after the `!`, so it created a long flag
called `_validate_int`, which meant it would not only error out form
```fish
argparse 'i=!_validate_int' 'o=!_validate_int' -- $argv
```
with "Long flag '_validate_int' already defined", but also set
$_flag_validate_int.
Fixes#5864.
As mentioned in #2900, something like
```fish
test -n "$var"; and set -l foo $var
```
is sufficiently idiomatic that it should be allowable.
Also fixes some additional weirdness with semicolons.
This removes semicolons at the end of the line and collapses
consecutive ones, while replacing meaningful semicolons with newlines.
I.e.
```fish
echo;
```
becomes
```fish
echo
```
but
```fish
echo; echo
```
becomes
```fish
echo
echo
```
Fixes#5859.
This keeps all unknown options in $argv, so
```fish
argparse -i a/alpha -- -a banana -o val -w
```
results in $_flag_a set to banana, and $argv set to `-o val -w`.
This allows users to use multiple argparse passes, or to simply avoid
specifying all options e.g. in completions - `systemctl` has 46 of
them, most not having any effect on the completions.
Fixes#5367.
This is a long-standing issue with how `complete --do-complete` does
its argument parsing: It takes an optional argument, so it has to be
attached to the token like `complete --do-complete=foo` or (worse)
`complete -Cfoo`.
But since `complete` doesn't take any bare arguments otherwise (it
would error with "too many arguments" if you did `complete -C foo`) we
can just take one free argument as the argument to `--do-complete`.
It's more of a command than an option anyway, since it entirely
changes what the `complete` call _does_.
