Commit Graph

1276 Commits

Author SHA1 Message Date
Daniel Shahaf
ffed6e1654 On the feature/redrawhook branch, advise its users that they can switch back to master.
Fixes #655.
2020-08-09 11:07:04 +00:00
Daniel Shahaf
7cc6226477 docs: Track making the new codepath conditional upon the 'memo=' feature.
The new codepath is used in 5.8.0.2 and newer; see zsh-syntax-highlighting.zsh:417.
2020-08-09 10:56:18 +00:00
Daniel Shahaf
cba4a1bc2e On the feature/redrawhook branch, changelog: Add entries for issues fixed by this branch.
This covers issues referenced from #245 or labelled "feature:redrawhook" or "widget-wrapping".  See #749 for details.
2020-07-14 23:20:29 +00:00
Daniel Shahaf
637e1c702e Merge remote-tracking branch 'origin/master' into feature/redrawhook
* origin/master:
  Post-release version number bump.
  Tag version 0.8.0-alpha1-pre-redrawhook.
  brackets: Optimize the character iteration
2020-07-14 22:49:17 +00:00
Daniel Shahaf
2d60a47cc4 Post-release version number bump. 2020-07-14 21:33:52 +00:00
Daniel Shahaf
00a5fd11eb Tag version 0.8.0-alpha1-pre-redrawhook. 2020-07-14 21:31:39 +00:00
Daniel Shahaf
cdd7f899cb brackets: Optimize the character iteration
Test case: See the first comment of issue #295.

Before:
num  calls                time                       self            name
-----------------------------------------------------------------------------------
 1)   14         284.67    20.33   82.84%    284.67    20.33   82.84%  _zsh_highlight_highlighter_brackets_paint
 2)   15         313.72    20.91   91.30%     26.62     1.77    7.75%  _zsh_highlight
 3)   14         318.68    22.76   92.74%      7.41     0.53    2.16%  _zsh_highlight_call_widget

After:
num  calls                time                       self            name
-----------------------------------------------------------------------------------
 1)   15          57.25     3.82   45.33%     28.30     1.89   22.41%  _zsh_highlight
 2)   14          81.96     5.85   64.90%     27.02     1.93   21.40%  _zsh_highlight_call_widget
 3)   14          26.31     1.88   20.84%     26.31     1.88   20.84%  _zsh_highlight_highlighter_brackets_paint
 4)    2          10.66     5.33    8.44%      8.97     4.49    7.10%  VCS_INFO_formats
 5)    2          25.92    12.96   20.52%      7.72     3.86    6.11%  VCS_INFO_get_data_git

(That's not in my everyday configuration but in my minimal testing
setup, which runs vcs_info with the default settings.)
2020-07-14 20:19:40 +00:00
Daniel Shahaf
cb33cc0081 On the feature/redrawhook branch, change the detection of the 'memo=' feature to avoid a catch-22. 2020-07-14 02:37:41 +00:00
Daniel Shahaf
9ce3540d74 Merge remote-tracking branch 'origin/master' into feature/redrawhook
* origin/master:
  Revert "driver: Move the initialization of $zsh_highlight__memo_feature out of the entry point function."
2020-07-14 02:31:24 +00:00
Daniel Shahaf
d62baa50f4 Revert "driver: Move the initialization of $zsh_highlight__memo_feature out of the entry point function."
This reverts commit 8a1bd7c8b6.

The detection only works correctly from a widget function, so the change
was wrong: it caused zsh_highlight__memo_feature to be set to false
every time.
2020-07-14 02:29:46 +00:00
Daniel Shahaf
59cb9a560d driver: Make the redrawhook codepath conditional upon the memo= feature.
Fixes #579 (zsh-autosuggestions interoperability).
Fixes #735 (ditto).

See https://github.com/zsh-users/zsh-syntax-highlighting/issues/579#issuecomment-650126055
See https://github.com/zsh-users/zsh-autosuggestions/issues/529#issuecomment-650481227
2020-07-14 02:13:51 +00:00
Daniel Shahaf
7b863fb413 Merge remote-tracking branch 'origin/master' into feature/redrawhook
* origin/master:
  driver: Move the initialization of $zsh_highlight__memo_feature out of the entry point function.
2020-07-14 02:08:20 +00:00
Daniel Shahaf
8a1bd7c8b6 driver: Move the initialization of $zsh_highlight__memo_feature out of the entry point function.
This is needed for feature/redrawhook to be able to use it.
2020-07-14 02:07:32 +00:00
Daniel Shahaf
daf0d94bae On the feature/redrawhook branch, move the changelog entry to the current release's section. 2020-07-14 02:00:45 +00:00
Daniel Shahaf
c28312b9d3 Merge remote-tracking branch 'origin/master' into feature/redrawhook
* origin/master: (297 commits)
  driver: Follow-up to grandparent: Have all test suite entry points declare the mock $region_highlight.
  Use the new, unreleased zsh 'memo=' feature to remove only our own entries from $region_highlight.
  driver: Stop re-declaring $region_highlight.  It's unneeded.
  docs: regexp highlighter: Fix a wrong associative array name in the example.
  docs: Fix obs-repository link
  tests: Fix a wrong value of $PREBUFFER in a test, and add checks to prevent this from recurring.
  test harness: Fix use of an undefined variable in an error message.
  'main': Don't progress the $in_redirection staller while $in_param.
  tests: Add an XFail test for issue #712.
  'main': Highlight the parentheses of array assignments as reserved words.
  CI += zsh-5.8
  main: Add tests for arithmetic expansion
  main: Add arithmetic substitution highlighting
  changelog.md: Restore vertical whitespace before section headers.
  'main': Fix issue #677, concerning multiline aliases.
  changelog: Update through HEAD.
  'main': Further optimize argument parsing.
  'main': Optimize a hot path.
  tests: Add a performance testing script, for measuring the performance of the 'main' highlighter on a large file.
  changelog: Update through HEAD.
  test harness: Print the expected-v.-actual on every failure, not just upon cardinality failures.
  Document ZSH_HIGHLIGHT_MAXLENGTH.
  'main': Fix the last commit's bug concerning parameter elision not happening in redirects in command position.
  'main': Add a test for parameter elision not happening in redirects in command position.
  'main': Fix regression in zsh 5.3.1 and older: all precmd hooks later than z-sy-h would be aborted.
  changelog += WARN_NESTED_VAR fixes (#727, #731)
  'main': Fix a regression caused by the great-grandparent commit's WARN_NESTED_VAR fix.
  'main': Don't run `_zsh_highlight_main__type` on every non-command word.
  'make perf': Show only a cumulative datum per highligher, rather than per test file.
  'main': Don't trip WARN_NESTED_VAR.
  'main': Follow-up to previous: Document the version number, and deduplicate some option letters.
  'main': precommands += strace
  editorconfig: Fix Makefile settings
  Fix typo
  Bump copyright years.
  driver: Fix "_zsh_highlight:3: read-only variable: ret" warnings when POSIX_BUILTINS is set.
  tests: Add a test for the infinite loop fixed by each of the last two commits.
  'main': Fix expansion of positional parameters in `_zsh_highlight_main_highlighter__try_expand_parameter`.
  'main': Fix an infinite loop.
  'main': precommands += ionice(1) (from util-linux)
  driver: Simplify initialization of $zsyh_user_options in the fallback codepath.
  driver: Make sure we don't change the return value in a called function.
  'main': Make logic more robust.  No functional change.
  'main': Break out an anonymous function into a named function.
  Fix typos in comments.
  main: Add test for issue #713
  'main': Support the 'env' precommand.
  test harness: Fix the pretty-printer's padding implementation.
  Revert "test harness: Rewrite the columnar pretty-printer without external tools." and "travis: Remove bsdmainutils since column(1) has been removed, three commits ago."
  changelog: Update through HEAD.
  'main': Correctly highlight '&&' and '||' inside '[[ … ]]' conditions.
  'main': Highlight reserved words following assignments as errors.
  tests: Add tests for issue #461.
  test harness: Output the time information to the same place the test name was printed to.
  test harness: Stringify values in a more readable manner.
  tests: Add a unit test for a path specified with mixed quoting.
  tests: Add a test for issue #498, which has already been fixed.
  tests: Test that global qualifiers and command substitutions aren't evaluated.
  'main': Don't consider path_prefix in alias expansions.
  'main': Add a test for aliases to AUTO_CD directories.
  'main': Let AUTO_CD directories be highlighted with their own style.
  'main': Add an auxiliary variable for readability.
  'main': In command position, do not highlight directories (unless AUTO_CD is set) and non-executable files.
  'main': Extend tests to capture the current behaviour.
  'main': Add an XFail test for issue #202.
  'main': Highlight errors from the EQUALS option.
  'main': Let the type determination ignore global aliases when it ignores regular ones.
  'main': Add a regression test for parameters that expand to global aliases.
  'main': Enable the zsh/parameter codepath of global aliases highlighting.
  changelog: Update through HEAD.
  travis: Remove bsdmainutils since column(1) has been removed, three commits ago.
  'main': Highlight global aliases
  tests: Record current behaviour on global aliases.
  test harness: Rewrite the columnar pretty-printer without external tools.
  test harness: Fix an issue with the pretty-printed $expected_region_highlight/$region_highlight diffing.
  'main': Support the "close file descriptor" and "coproc" redirection syntaxes
  tests: Add a test for the "close file descriptor" and "coproc" redirection syntaxes
  tests: Fix the test added in the last commit.
  tests: Add a test for issue #705, concerning continuation lines.
  test harness: Let tests fail early by exiting non-zero or by setting a flag.
  test harness: Print the test name when $skip_test is set.
  test harness: Remove a bogus check.
  test harness: Fix $skip_test support, broken yesterday.
  travis: Install bsdmainutils to provide column(1).
  test harness: When the cardinality check fails, pretty-print \$expected_region_highlight and \$region_highlight.
  test harness: Don't leak options from test files to the test harness.
  test harness: Fix test failures under zsh 5.0.8 and older.
  'main': Fix a bug manifesting under zsh 5.2 and older.
  'main': Don't highlight arithmetic expansions as command substitutions.
  tests: Add a test documenting the current state, prior to introducing #704.
  test harness: Change cardinality check semantics
  test harness: No-op change to minimize the next diff.
  'main': Document additional meanings of the 'S' $braces_stack flag.
  'main': When the redirection operator '>&' or '<&' is followed by a positive integer, do not consider that as a filename; it's always a file descriptor.
  'main': Add $last_arg for "lookbehind".
  noop: Clarify comment.
  'main': Honour the MULTIOS option when applying the 'globbing' style.
  'main': Document what $in_redirection is currently used for.
  'main': The optimized cmdsubst input syntax doesn't glob.
  changelog: Fix markup.
  ...
2020-07-14 01:57:55 +00:00
Daniel Shahaf
d9a7963970 driver: Follow-up to grandparent: Have all test suite entry points declare the mock $region_highlight.
tests/generate.zsh already does this.
2020-07-14 01:57:02 +00:00
Daniel Shahaf
810c2dcede Use the new, unreleased zsh 'memo=' feature to remove only our own entries from $region_highlight.
Fixes #418 (interoperability issue with other plugins).
2020-07-14 00:32:36 +00:00
Daniel Shahaf
075c852924 driver: Stop re-declaring $region_highlight. It's unneeded. 2020-07-14 00:32:36 +00:00
Nuri Jung
fb929edc30 docs: regexp highlighter: Fix a wrong associative array name in the example.
Factored out from #747.
2020-07-01 12:55:43 +00:00
Il Harper
91d2eeaf23 docs: Fix obs-repository link 2020-06-15 00:23:05 -05:00
Daniel Shahaf
f6471dbec3 tests: Fix a wrong value of $PREBUFFER in a test, and add checks to prevent this from recurring.
Discussed: https://github.com/zsh-users/zsh-syntax-highlighting/pull/706#issuecomment-642072978
2020-06-11 08:14:07 +00:00
Daniel Shahaf
f6f7a91818 test harness: Fix use of an undefined variable in an error message.
The error message uses «$1», but it is always unset at that point.
Furthermore, the NO_UNSET (-u) option is in effect.  Therefore, when the
error message was to be printed, zsh's NO_UNSET error message was
printed instead:
.
    (anon):20: 1: parameter not set

That error message was printed to stderr, and was caught by the caller
within the test harness:
.
    Bail out! On './highlighters/main/test-data/path_prefix3.zsh': output on stderr

Thus, the bug being fixed would not caused false positive or false
negative test results, but only wrong error messages.

Follow-up to 0.7.1-71-ge6eea1f, "test harness: Don't leak options from
test files to the test harness".
2020-06-11 08:09:45 +00:00
Daniel Shahaf
6fd92e1bbd 'main': Don't progress the $in_redirection staller while $in_param.
Fixes #712.
2020-06-08 14:38:59 +00:00
Daniel Shahaf
792c065acb tests: Add an XFail test for issue #712.
The second test point passes on 0.7.1, but the third does not.
2020-06-08 14:38:43 +00:00
Daniel Shahaf
0f11d80968 'main': Highlight the parentheses of array assignments as reserved words.
Fixes #585.
2020-06-08 14:23:43 +00:00
Daniel Shahaf
00c0c76550 CI += zsh-5.8
Fixes #740.
2020-05-26 17:52:17 +00:00
Matthew Martin
870bccf8ff main: Add tests for arithmetic expansion 2020-05-23 20:29:09 -05:00
Matthew Martin
a238647df9 main: Add arithmetic substitution highlighting
Closes #607 #649 #704
2020-05-23 20:29:00 -05:00
Daniel Shahaf
f8b1470314 changelog.md: Restore vertical whitespace before section headers. 2020-05-22 04:55:17 +00:00
Daniel Shahaf
ade4b28d2d 'main': Fix issue #677, concerning multiline aliases.
The fix is to exempt such aliases from the empty commands sanity check.
2020-05-22 04:54:10 +00:00
Daniel Shahaf
8211a95421 changelog: Update through HEAD. 2020-05-22 04:44:49 +00:00
Daniel Shahaf
d4df626c03 'main': Optimize the unquoted arguments characterwise walker
Fixes #730.

% repeat 3 { zsh -f tests/test-zprof.zsh main | tee … | grep -w _zsh_highlight | head -n1 }
19)    1       26787.11 26787.11  100.00%      5.33     5.33    0.02%  _zsh_highlight
19)    1       26900.64 26900.64  100.00%      5.45     5.45    0.02%  _zsh_highlight
19)    1       26915.76 26915.76  100.00%      5.47     5.47    0.02%  _zsh_highlight

Merge remote-tracking branch 'danielsh/perf-argument-1-v2'

* danielsh/perf-argument-1-v2:
  'main': Further optimize argument parsing.
  'main': Optimize a hot path.
  tests: Add a performance testing script, for measuring the performance of the 'main' highlighter on a large file.
2020-05-22 04:43:04 +00:00
Daniel Shahaf
8f5d74d219 'main': Further optimize argument parsing.
% repeat 3 { zsh -f tests/test-zprof.zsh main | tee … | grep -w _zsh_highlight | head -n1 }
18)    1       26895.86 26895.86  100.00%      6.35     6.35    0.02%  _zsh_highlight
19)    1       27399.11 27399.11  100.00%      5.52     5.52    0.02%  _zsh_highlight
19)    1       27027.58 27027.58  100.00%      5.66     5.66    0.02%  _zsh_highlight

----

This commit has been rebased.  The above statistics were measured after
the rebase.  The below statistics had been measured before the rebase.

num  calls                time                       self            name
-----------------------------------------------------------------------------------
 1)    3       25689.17  8563.06   98.15%  18422.01  6140.67   70.38%  _zsh_highlight_main_highlighter_highlight_list
 2) 32390        5706.13     0.18   21.80%   2315.68     0.07    8.85%  _zsh_highlight_main_highlighter_highlight_argument
19)    1       26173.33 26173.33  100.00%      5.27     5.27    0.02%  _zsh_highlight

Interestingly, if I make the change in this diff to
_zsh_highlight_main_highlighter_highlight_double_quote —

>     diff --git a/highlighters/main/main-highlighter.zsh b/highlighters/main/main-highlighter.zsh
>     index da6ab2b..bb17618 100644
>     --- a/highlighters/main/main-highlighter.zsh
>     +++ b/highlighters/main/main-highlighter.zsh
>     @@ -1462,10 +1462,13 @@ _zsh_highlight_main_highlighter_highlight_double_quote()
>        local i j k ret style
>        reply=()
>
>     -  for (( i = $1 + 1 ; i <= $#arg ; i += 1 )) ; do
>     +  (( i = $1 ))
>     +  while (( ++i <= $#arg )); do
>     +    i=${arg[(ib.i.)[\"\`\$\\\\${histchars[1]}]]}
>          (( j = i + start_pos - 1 ))
>          (( k = j + 1 ))
>          case "$arg[$i]" in
>     +      ("")  break;;
>            ('"') break;;
>            ('`') saved_reply=($reply)
>                  _zsh_highlight_main_highlighter_highlight_backtick $i

— it actually makes things measurably slower (!), even on input that has
a large number of pasted double-quoted strings: on «BUFFER=": ${(r.8*1500..foo"bar".):-}"»
the slowdown is (1123.24ms / 1091.06ms = 1.0295).  Therefore, I won't be
committing that change.
2020-05-22 04:36:59 +00:00
Daniel Shahaf
700c0e18fe 'main': Optimize a hot path.
% git co HEAD^ && repeat 3 { zsh -f tests/test-zprof.zsh main | tee … | grep -w _zsh_highlight | head -n1 }
HEAD is now at 64e3651 'main': Optimize a hot path.
19)    1       28765.13 28765.13  100.00%      5.57     5.57    0.02%  _zsh_highlight
19)    1       28566.46 28566.46  100.00%      5.91     5.91    0.02%  _zsh_highlight
19)    1       28248.12 28248.12  100.00%      5.57     5.57    0.02%  _zsh_highlight

----

This commit has been rebased.  The above statistics were measured after
the rebase.  The below statistics had been measured before the rebase.

Before this patch:

    num  calls                time                       self            name
    -----------------------------------------------------------------------------------
     1)    3       33410.81 11136.94   98.51%  19277.07  6425.69   56.84%  _zsh_highlight_main_highlighter_highlight_list
    19)    1       33916.21 33916.21  100.00%      5.27     5.27    0.02%  _zsh_highlight

With this patch:

    num  calls                time                       self            name
    -----------------------------------------------------------------------------------
     1)    3       27167.49  9055.83   98.17%  18754.77  6251.59   67.77%  _zsh_highlight_main_highlighter_highlight_list
    19)    1       27674.40 27674.40  100.00%      5.39     5.39    0.02%  _zsh_highlight

And if test-zprof.zsh is changed to not set interactivecomments:

    num  calls                time                       self            name
    -----------------------------------------------------------------------------------
     1) 13360       36029.12     2.70   83.56%  30304.23     2.27   70.28%  _zsh_highlight_main_highlighter_highlight_argument
    21)    1       43117.76 43117.76  100.00%      4.52     4.52    0.01%  _zsh_highlight

    num  calls                time                       self            name
    -----------------------------------------------------------------------------------
     1) 13360       14782.89     1.11   68.12%   9163.42     0.69   42.23%  _zsh_highlight_main_highlighter_highlight_argument
    21)    1       21699.93 21699.93  100.00%      4.17     4.17    0.02%  _zsh_highlight
2020-05-22 04:32:39 +00:00
Daniel Shahaf
a50647e77b tests: Add a performance testing script, for measuring the performance of the 'main' highlighter on a large file.
% git co HEAD^ && repeat 3 { zsh -f tests/test-zprof.zsh main | tee … | grep -w _zsh_highlight | head -n1 }
HEAD is now at f1948df tests: Add a performance testing script, for measuring the performance of the 'main' highlighter on a large file.
19)    1       34378.97 34378.97  100.00%      5.43     5.43    0.02%  _zsh_highlight
19)    1       34058.34 34058.34  100.00%      5.50     5.50    0.02%  _zsh_highlight
19)    1       34364.80 34364.80  100.00%      5.36     5.36    0.02%  _zsh_highlight
2020-05-22 04:31:48 +00:00
Daniel Shahaf
b253a8b86a changelog: Update through HEAD.
The great-grandparent commit, "a3ae74 'main': Fix the last commit's bug
concerning parameter elision not happening in redirects in command
position.", is not added because it's not a change with respect
to 0.7.1.
2020-05-22 03:30:49 +00:00
Daniel Shahaf
4dd4797ae0 test harness: Print the expected-v.-actual on every failure, not just upon cardinality failures.
I was looking into something and wanted to see how a the second word in the
array was highlighted, even though the failure was on the third word.
2020-05-22 03:27:48 +00:00
Daniel Shahaf
5171ec524f Document ZSH_HIGHLIGHT_MAXLENGTH.
Fixes #698.
2020-05-22 02:23:18 +00:00
Daniel Shahaf
ea3ae74164 'main': Fix the last commit's bug concerning parameter elision not happening in redirects in command position. 2020-05-13 12:51:52 +00:00
Daniel Shahaf
41b8a74692 'main': Add a test for parameter elision not happening in redirects in command position.
Will be fixed in the next commit.
2020-05-13 12:51:40 +00:00
Daniel Shahaf
3a4b212c7d 'main': Fix regression in zsh 5.3.1 and older: all precmd hooks later than z-sy-h would be aborted.
In those versions of zsh, «[[ -o nosuchoption ]]» is regarded as
a syntax error.  In newer zsh versions, it merely returns non-zero
(specifically, it returns 3, unlike «[[ -o unsetoption ]]» which
returns 1).

Fixes #732.
Fixes #733.
2020-05-06 20:27:55 +00:00
Daniel Shahaf
0582ea1910 changelog += WARN_NESTED_VAR fixes (#727, #731) 2020-05-05 17:59:51 +00:00
Daniel Shahaf
e65ebf0466 'main': Fix a regression caused by the great-grandparent commit's WARN_NESTED_VAR fix.
An error message was emitted on versions of zsh that don't have the
WARN_NESTED_VAR option.

Fixes #731.
2020-05-05 14:31:42 +00:00
Daniel Shahaf
06710f3780 'main': Don't run _zsh_highlight_main__type on every non-command word.
Fixes #728, the performance regression from 0.7.1.
2020-05-04 18:35:15 +00:00
Daniel Shahaf
343ec1061f 'make perf': Show only a cumulative datum per highligher, rather than per test file.
The overall per-highlighter duration should be less prone to random
noise than the multitude of per-test-file figures.
2020-05-04 17:48:47 +00:00
Daniel Shahaf
b08d508cd8 driver: Fix a bug that prevented subsequent, third-party zle-line-pre-redraw hooks from running.
Without this patch, `_zsh_highlight` was invoked by add-zle-hook-widget
with `$?` being non-zero (see add-zle-hook-widget:48-52).  Since
`_zsh_highlight` preserves `$?` from its caller's point of view,
add-zle-hook-widget saw a non-zero exit code from `_zsh_highlight` and
did not run any the remaining zle-line-pre-redraw hooks.

See https://github.com/zsh-users/zsh-syntax-highlighting/issues/579#issuecomment-623576907.
2020-05-04 16:45:21 +00:00
Daniel Shahaf
16d818a21f 'main': Don't trip WARN_NESTED_VAR.
Fixes #727.
2020-05-04 13:27:33 +00:00
Daniel Shahaf
f1b9fbbaf0 'main': Follow-up to previous: Document the version number, and deduplicate some option letters. 2020-04-20 11:11:29 +00:00
Daniel Shahaf
8d32609a7b 'main': precommands += strace 2020-04-20 11:09:24 +00:00
Daniel Shahaf
f5d1be7ec2 editorconfig: Fix Makefile settings 2020-04-12 02:59:39 +00:00