There's no reason to inject prefix into our newly allocated str after storing
pattern in there. Just allocate with the needed capacity up front and then
insert in the correct order.
The new asan exit handlers are called to get proper ASAN leak reports (as
calling _exit(0) skips the LSAN reporting stage and exits with success every
time).
They are no-ops when not compiled for ASAN.
This ports some signal setup and handling bits to Rust.
The signal handling machinery requires walking over the list of known signals;
that's not supported by the Signal type. Rather than duplicate the list of
signals yet again, switch back to a table, as we had in C++.
This also adds two further pieces which were neglected by the Signal struct:
1. Localize signal descriptions
2. Support for integers as the signal name
This allows the rust code to free up C++ resources allocated for a callback even
when the callback isn't executed (as opposed to requiring the callback to run
and at the end of the callback cleaning up all allocated resources).
Also add type-erased destructor registration to callback_t. This allows for
freeing variables allocated by the callback for debounce_t's
perform_with_callback() that don't end up having their completion called due to
a timeout.
Largely routine but for the trampolines in iothread.h and iothread.cpp which
were a real PITA to get correct w/ all their variants.
Integration is complete with all old code ripped out and the tests using the
rust version of the code.
Like the WSL check, this was incorrectly assuming WSL implies
cfg(windows) when it's actually picked up as Linux.
Also, improve over the C++ code by not relying on the build-time WSL
status to determine if we are running on WSL at runtime since it's often
the case that the fish binaries are built on a non-WSL host (for
packaging) then executed on a WSL only at runtime.
(But it's ok to assume if fish has been built for Windows or not Linux
that it will either be run or not run on top of a Win32 character device
system.)
Also, port of the comment and relevant WSL and fish issue links over
from the CPP codebase for posterity.
* Since we already have an allocation of length wstr.len(), it's
probably better to allocate the result (which is strictly less than or
equal to the input length) up-front rather than risk thrashing the Vec
allocation,
* There's no need to compare c2 against '\0' since that will just cause
to_digit(16) to return None anyway,
* Our convert_hex() specialization of to_digit(16) that only checks
capital letters A-F without also checking lowercase a-f isn't
significantly faster than just use to_digit(16), and we already assert
that the input *wasn't* a lowercase a-f before making the call, so
there's no point in using a special function to handle that.
Prior to this change, wstr::split had two weird behaviors:
1. Splitting an empty string would yield nothing, rather than an empty
string.
2. Splitting a string with the separator character as last character
would not yield an empty string.
For example L!("x:y:").split(':') would return ["x", "y"] instead of
what it does in C++, which is ["x", "y", ""].
Fix these.
This reverts commit 76dc849fca.
The warning added in that commit is incorrect. The functions
unescape_string_url and unescape_string_var will not panic, because
char_at() return 0 if the index is equal to its length.
This reverts commit f9c92753c4.
This commit attempted to replace exit_without_destructors() with
std::process::exit; however this is wrong for two reasons:
1. std::process::exit() runs Rust runtime cleanup stuff we don't want
2. std::process::exit() invokes destructors, meaning atexit handlers,
which we don't want.
The type system no longer guarantees that the input string is nul-terminated,
meaning accessing beyond the range-checked `i` a char-at-a-time is no longer
safe. (In C++, we would either be using a plain C string which is always
nul-terminated or we would be using (w)string::cstr() which similarly grants
access to its nul-terminated buffer.)
Aside from that, there's no need to explicitly check `if c2 == '\0'` because
'\0' is not a valid hex digit so the `?` tacked on to `convert_hex_digit(c2)?`
will abort and return `None` anyway.
convert_hex_digit() is not appreciably faster than char::to_digit(16) and makes
the code less maintainable since it encodes certain assumptions; since it's also
not used consistently just drop it in favor of the std fn.
Since the output string (per the decode logic) is always shorter than or equal
to the input string, just reserve the input string size upfront to prevent vec
reallocations.
Somewhat counter-intuitively, this code is active when compiling under *Linux*
and is always false when compiling under Windows. The logic was incorrectly
reversed before (it's easier to reason about when you realize that fish doesn't
even compile under Windows because it uses tons of libc functions).
As the code was actually never compiled, it wasn't actually tested for validity
either and there were some issues that prevented it from compiling that have
since been fixed. The logic has also been adjusted a bit to make it possible to
use the rust-native int parsing instead of `libc::strtod()`.
The code has been changed to use `once_cell::race::OnceBool` instead of
`once_cell::sync::Lazy<T>` which imposes a greater runtime burden with locking
and other overhead. We don't care if the code runs more than once on init (if
calls were to race, though they probably don't) - just that the code isn't
subsequently executed on each call. The `once_cell::race` module is a better fit
here, though it doesn't expose the ergonomic `Lazy<T>` façade around its types.
is_main_thread() and co were previously ported to threads.rs, so remove the
duplicate code and move everything else related to threads there as well. No
need for common.rs to be as long as our old common.cpp!
I left #[deprecated] stubs in common.rs to help redirect anyone porting code
over that we can remove after the port has finished.
Additionally, the fork guards had previously been left as a todo!() item but I
ported that over. They're all called from the now-central threads::init()
function so there isn't a need to call each individual thread-management-fn
manually.
The decision was made a while back to try and embrace/use the native rust thread
functionality and utilities so the manual thread management code has been ripped
out and was replaced with code that marshals the native rust values instead. The
values won't line up with what the C++ code sees, but it never lined up anyway
since each was using a separate counter to keep track of the values.
Except for the indent visitor bits.
Tests for parse_util_detect_errors* are not ported yet because they depend
on expand.h (and operation_context.h which depends on env.h).
This caused math to assert out because it never wrote into the buffer.
Now, presumably it wrote somewhere but I don't know where, so fixing
this seems like a good idea.
Fixes#9735.
The translation is fairly direct though it adds some duplication, for example
there are multiple "match" statements that mimic function overloading.
Rust has no overloading, and we cannot have generic methods in the Node trait
(due to a Rust limitation, the error is like "cannot be made into an object")
so we include the type name in method names.
Give clients like "indent_visitor_t" a Rust companion ("IndentVisitor")
that takes care of the AST traversal while the AST consumption remains
in C++ for now. In future, "IndentVisitor" should absorb the entirety of
"indent_visitor_t". This pattern requires that "fish_indent" be exposed
includable header to the CXX bridge.
Alternatively, we could define FFI wrappers for recursive AST traversal.
Rust requires we separate the AST visitors for "mut" and "const"
scenarios. Take this opportunity to concretize both visitors:
The only client that requires mutable access is the populator. To match the
structure of the C++ populator which makes heavy use of function overloading,
we need to add a bunch of functions to the trait. Since there is no other
mutable visit, this seems acceptable.
The "const" visitors never use "will_visit_fields_of()" or
"did_visit_fields_of()", so remove them (though this is debatable).
Like in the C++ implementation, the AST nodes themselves are largely defined
via macros. Union fields like "Statement" and "ArgumentOrRedirection"
do currently not use macros but may in future.
This commit also introduces a precedent for a type that is defined in one
CXX bridge and used in another one - "ParseErrorList". To make this work
we need to manually define "ExternType".
There is one annoyance with CXX: functions that take explicit lifetime
parameters require to be marked as unsafe. This makes little sense
because functions that return `&Foo` with implicit lifetime can be
misused the same way on the C++ side.
One notable change is that we cannot directly port "find_block_open_keyword()"
(which is used to compute an error) because it relies on the stack of visited
nodes. We cannot modify a stack of node references while we do the "mut"
walk. Happily, an idiomatic solution is easy: we can tell the AST visitor
to backtrack to the parent node and create the error there.
Since "node_t::accept_base" is no longer a template we don't need the
"node_visitation_t" trampoline anymore.
The added copying at the FFI boundary makes things slower (memcpy dominates
the profile) but it's not unusable, which is good news:
$ hyperfine ./fish.{old,new}" -c 'source ../share/completions/git.fish'"
Benchmark 1: ./fish.old -c 'source ../share/completions/git.fish'
Time (mean ± σ): 195.5 ms ± 2.9 ms [User: 190.1 ms, System: 4.4 ms]
Range (min … max): 193.2 ms … 205.1 ms 15 runs
Benchmark 2: ./fish.new -c 'source ../share/completions/git.fish'
Time (mean ± σ): 677.5 ms ± 62.0 ms [User: 665.4 ms, System: 10.0 ms]
Range (min … max): 611.7 ms … 805.5 ms 10 runs
Summary
'./fish.old -c 'source ../share/completions/git.fish'' ran
3.47 ± 0.32 times faster than './fish.new -c 'source ../share/completions/git.fish''
Leftovers:
- Enum variants are still snakecase; I didn't get around to changing this yet.
- "ast_type_to_string()" still returns a snakecase name. This could be
changed since it's not user visible.
A JobId is not supposed to convert to other types.
Since this type is defined as NonZeroU32 (which cannot be -1), we need to
add some conversion functions to match the C++ behavior.
Overall, it would have been better to keep using the C++ type.
The conversion to usize is used for array accesses, so negative values
would cause crashes either way. Let's do it earlier so we can get rid of
the suspect C-style cast.
This allows us to use the scoped push in more scenarios by appeasing the
borrow checker.
Use it in a couple of places instead of ScopeGuard. Hopefully this is makes
porting easier.
Even though we generally dont' want to use this type (because it's immutable),
it can be advantageous when working with the std::fs API. This is because
it implements "AsRef<Path>" which neither of CString and Vec<u8> do.
This is basically a subset of type, so we might as well.
To be clear this is `command -s` and friends, if you do `command grep` that's
handled as a keyword.
One issue here is that we can't get "one path or not" because I don't
know how to translate a maybe_t? Do we need to make it a shared_ptr instead?
Most of it is duplicated, hence untested.
Functions like mbrtowc are not exposed by the libc crate, so declare them
ourselves.
Since we don't know the definition of C macros, add two big hacks to make
this work:
1. Replace MB_LEN_MAX and mbstate_t with values (resp types) that should
be large enough for any implementation.
2. Detect the definition of MB_CUR_MAX in the build script. This requires
more changes for each new libc. We could also use this approach for 1.
Additionally, this commit brings a small behavior change to
read_unquoted_escape(): we cannot decode surrogate code points like \UDE01
into a Rust char, so use � (\UFFFD, replacement character) instead.
Previously, we added such code points to a wcstring; looks like they were
ignored when printed.
wcs2string converts a wide string to a narrow one. The result is
null-terminated and may also contain interior null-characters.
std::string allows this.
Rust's null-terminated string, CString, does not like interior null-characters.
This means we will need to use Vec<u8> or OsString for the places where we
use interior null-characters.
On the other hand, we want to use CString for places that require a
null-terminator, because other Rust types don't guarantee the null-terminator.
Turns out there is basically no overlap between the two use cases, so make
it two functions. Their equivalents in Rust will have the same name, so
we'll only need to adjust the type when porting.
Existing C++ code didn't use a function for this but simply added
ENCODE_DIRECT_BASE. In Rust that's more verbose because char won't do
arithmetics, hence the function.
We'll add a dual function for decoding, so let's rename this.
BTW we should get rid of the "wchar" naming, it's just "char" in Rust.
Prior to this change, wcstoi("0x") would fail with missing digits.
However strtoul will "backtrack" to return just the 0 and leave the x as
the remainder. Implement this behavior.
Prior to this change, wcstoi() would return an error if the requested
type were unsigned, and the input had a leading minus sign. However this
causes problems for printf, which expects strtoul behavior.
Add "modulo base" behavior which wraps the negative value to positive.
Factor this into an option; the default is False (but code which
previously used strtoull directly should set it to true).
fish_wcstoi_partial is like fish_wcstoi: it converts from a string to an
int optionally inferring the radix. fish_wcstoi_partial also returns the
number of characters consumed.
Unfortunately we cannot use wide string literals in match statements
(not sure if there's an easy fix).
Because of this, I converted the input to UTF-8 so we could use the match
statement. This conversion is confusing, let's skip it.
Everything but signal handlers has been changed to use `Signal` instead of
`c_int` or `i32` signal values.
Event handlers are using `usize` to match C++, at least for now.
Signal is a newtype around NonZeroI32. We could use NonZeroU8 since all signal
values comfortably fit, but using i32 lets us avoid a fallible attempt at
narrowing values returned from the system as integers to the narrower u8 type.
Known signals are explicitly defined as constants and can be matched against
with equality or with pattern matching in a `match` block. Unknown signal values
are passed-through without causing any issues.
We're using per-OS targeting to enable certain libc SIGXXX values - we could
change this to dynamically detecting what's available in build.rs but then it
might not match what libc exposes, still giving us build failures.
This should be used in lieu of manually targeting individual operating systems
when using features shared by all BSD families.
e.g. instead of
#[cfg(any(target_os = "freebsd", target_os = "dragonflybsd", ...))]
fn foo() { }
you would use
#[cfg(feature = "bsd")]
fn foo() { }
This feature is automatically detected at build-time (see build.rs changes) and
should *not* be enabled manually. Additionally, this feature may not be used to
conditionally require any other dependency, as that isn't supported for
auto-enabled features.
Just address two clippy lints that are fallout from changing the signal type.
There's no longer any need to convert these (which gets rid of an unwrap).
Due to limitations imposed by the borrow checker, there are very few places
where we will be able to use the `ScopedPush` class ported over from the C++
codebase (once you capture the value w/ a `ScopedPush` you can't access the
value - or the mutable reference you used to reach it! - until the `ScopedPush`
object goes out of scope).
This alternative requires binding the previous values to a variable and manually
restoring them in the callback passed to the `ScopeGuard` constructor, but will
work with rust's borrow and `&mut` paradigm.
Currently the `autocxx` generated code does not produce any code intelligence
because `rust-analyzer` can't find the generated code since it's not in the
workspace. Here, we detect `rust-analyzer` by checking for a `RUSTC_WRAPPER`
environment variable containing `rust-analyzer` and changing (or avoid changing)
the output directory accordingly.
Closes#9654.
This was added to support signals; however we are unlikely to use this
for anything else. Remove it; just use a u64 to report signals that have
been set.
This optimizes over both the rust rewrite and the original C++ code. The rust
rewrite saw `std::bitset` replaced with `[bool; 65]` which could result in a
lot of memory copy bandwidth each time we checked for and received no signals.
The original C++ code would iterate over all signal slots to see if any were
set. The code now returns a single u64 and only checks slots that are known to
have signals via an intelligent `Iterator` impl.
You can now use a reference to CxxWString or an allocated UniquePtr<CxxWString>
to get an &wstr temporary to use without having to allocate again (e.g. via
`from_ffi()`).
wchar.rs should not import let alone reexport FFI strings.
Stop re-exporting utf32str! because we use L! instead.
In wchar_ffi.rs, stop re-exporting cxx::CxxWString because that hasn't
seen adoption.
I think we should use re-exports only for aliases like "wstr" or for aliases
into internal modules.
So I'd probably remove `pub use wchar_ffi::wcharz_t = crate::ffi::wcharz_t`
as well.
bool_assert_comparison is stupid, the reason they give is "it's shorter". Well,
`assert!(!foo)` is nowhere near as readable as `assert_eq!(foo, false)` because
of the ! noise from the macro.
Uninlined format args is a stupid lint that Rust actually walked back when they
made it an official warning because you still have to use a mix of inlined and
un-inlined format args (the latter of which won't complain) since only idents
can be inlined.
This shows some of the ugliness of the rust borrow checker when it comes to
safely implementing any sort of recursive access and the need to be overly
explicit about which types are actually used across threads and which aren't.
We're forced to use an `Arc` for `ItemMaker` (née `item_maker_t`) because
there's no other way to make it clear that its lifetime will last longer than
the FdMonitor's. But once we've created an `Arc<T>` we can't call
`Arc::get_mut()` to get an `&mut T` once we've created even a single weak
reference to the Arc (because that weak ref could be upgraded to a strong ref at
any time). This means we need to finish configuring any non-atomic properties
(such as `ItemMaker::always_exit`) before we initialize the callback (which
needs an `Arc<ItemMaker>` to do its thing).
Because rust doesn't like self-referential types and because of the fact that we
now need to create both the `ItemMaker` and the `FdMonitorItem` separately
before we set the callback (at which point it becomes impossible to get a
mutable reference to the `ItemMaker`), `ItemMaker::item` is dropped from the
struct and we instead have the "constructor" for `ItemMaker` take a reference to
an `FdMonitor` instance and directly add itself to the monitor's set, meaning we
don't need to move the item out of the `ItemMaker` in order to add it to the
`FdMonitor` set later.
We were only using their ffi implementations which are automatically
exported/public, but the actual functions we would need if we were to use
FdMonitor and co. in native rust code were either private or missing convenient
wrappers.
The existing code is kept, but a rusty version of these functions is added for
code that needs them.
These should only be temporarily used when porting 1-to-1 from C++; we should
use the std library's `read()` and `write_all()` methods instead in the future.
By extracting the equivalent of i32::cmp() into its own const function,
it becomes a lot easier to see what is happening and the logic can be
more direct.
These will be used in the parser.
Maybe this type should be a struct with boolean fields. The current way has
the upside that the usage is exactly the same as in C++.
For some reason this error is triggered by tests after the Rust port of
ast.cpp. Might want to get to the bottom of this but moving it back
to match the original C++ logic fixes it.
This is one of the few warnings we disable due to false positives. Let's also
disable it in the preprocessing steps needed for the Rust build.
Other warnings we ignore are -Wno-address -Wunused-local-typedefs and
-Wunused-macros. I didn't add them here because I don't expect that they
will be triggered by the headers we give to cxx.
Prior to this fix, the Rust FLOG output was regressed from C++, because
it put quotes around strings. However if we used Display, we would fail
to FLOG non-display types like ThreadIDs.
There is apparently no way in Rust to write a function which formats a
value preferentially using Display, falling back to Debug.
Fix this by introducing two new traits, FloggableDisplay and
FloggableDebug. FloggableDisplay is implemented for all Display types,
and FloggableDebug can be "opted into" for any Debug type:
impl FloggableDebug for MyType {}
Both traits have a 'to_flog_str' function. FLOG brings them both into
scope, and Rust figures out which 'to_flog_str' gets called.
* wutil: Rewrite `wrealpath` in Rust
* Reduce use of FFI types in `wrealpath`
* Addressed PR comments regarding allocation
* Replace let binding assignment with regular comparison
