Change how math rounds integer results

We need our `math` builtin to behave like `bc` with respect to rounding
floating point values to integer to avoid breaking to many existing
uses. So when scale is zero round down to the nearest integer.

Another change for #3157.
This commit is contained in:
Kurtis Rader 2017-08-23 17:31:04 -07:00
parent 24d251ff4b
commit b816cd6d50
2 changed files with 6 additions and 2 deletions

View File

@ -15,7 +15,7 @@ The `math` command can evaluate multiple expressions separated by commas. The re
The following options are available:
- `-sN` or `--scale=N` sets the scale of the result. `N` must be an integer and defaults to zero (rounded to the nearest integer).
- `-sN` or `--scale=N` sets the scale of the result. `N` must be an integer and defaults to zero. A scale of zero causes results to be rounded down to the nearest integer. So `3/2` returns `1` rather than `2` which `1.5` would normally round to. This is for compatibility with `bc` which was the basis for this command prior to fish 3.0.0. Scale values greater than zero causes the result to be rounded using the usual rules to the specified number of decimal places.
\subsection return-values Return Values

View File

@ -133,7 +133,11 @@ static int evaluate_expression(wchar_t *cmd, parser_t &parser, io_streams_t &str
int nNum;
mu::value_type *v = p.Eval(nNum);
for (int i = 0; i < nNum; ++i) {
streams.out.append_format(L"%.*lf\n", opts.scale, v[i]);
if (opts.scale == 0) {
streams.out.append_format(L"%ld\n", static_cast<long>(v[i]));
} else {
streams.out.append_format(L"%.*lf\n", opts.scale, v[i]);
}
}
return STATUS_CMD_OK;