Commit Graph

30 Commits

Author SHA1 Message Date
Daniel Shahaf
3ff5bec82e test harness: Let tests fail early by exiting non-zero or by setting a flag.
Fixes #609.
2020-03-19 00:37:21 +00:00
Daniel Shahaf
61c1cfe99f test harness: Change cardinality check semantics
The cardinality check shall —

- if the test sets \$expected_mismatch, be XFail;
- elif any test points is XFail, be skipped;
- else, be expected to pass.

To test this, change «6 * 9» to «6 + 9» in test-data/arith1.zsh that
will be added in the after-next (grandchild) commit.
2020-03-17 15:05:22 +00:00
Daniel Shahaf
f63f07417d Merge remote-tracking branch 'danielsh/tests-skip-cardinality-v1'
* danielsh/tests-skip-cardinality-v1:
  tests: Minor documentation readability tweak
  Add a test for issue #641.5, using the infrastructure added in the previous commits.
  tests: Skip cardinality tests whenever any test point is expected to fail.
  tests: Make $expected_mismatch skip the cardinality check, rather than consider it an expected failure.
  tests: Include the name of the 'cardinality check' test point in the output
2020-03-15 18:38:26 +00:00
Daniel Shahaf
8e3578240c tests harness docs: Add paragraph breaks. 2020-02-28 22:36:57 +00:00
Daniel Shahaf
b7592e581d tests: Minor documentation readability tweak 2019-11-10 11:49:47 +00:00
Daniel Shahaf
4952325051 tests: Skip cardinality tests whenever any test point is expected to fail.
When writing an expected-to-fail test case, the cardinality of $region_highlight
at the time the test is written may differ from the cardinality it will have
once the bug is fixed.  For example, with issue #641.5, the current highlighting
is ['nice', 'x=y', 'y', 'ls'] — four elements — but the correct highlighting
would have three elements: ['nice', 'x=y', 'ls'].  There is no point in reporting
a separate test failure for the cardinality check in this case, nor for 'ls' being
highlighted as 'command' rather than 'default'.

At the same time, in other cases the current and correct highlighting may have the
same number of elements (for example, this would be the case for a hypothetical
"the command word is highlighted as an alias rather than a function" bug).  Thus,
the previous commit, q.v..
2019-11-10 11:49:13 +00:00
Daniel Shahaf
d5a4a6e195 tests: Make $expected_mismatch skip the cardinality check, rather
than consider it an expected failure.

With this change, if $expected_region_highlight and $region_highlight
coincidentally have the same number of elements, the test won't be considered
to fail.

This is useful in conjunction with the next commit, q.v..

At this time, no tests set $expected_mismatch explicitly.  However, the
commit after next (this commit's grandchild) will add a test that will
set $expected_mismatch implicitly, using the functionality in the next
commit (this commit's child).
2019-11-10 11:48:40 +00:00
Sean Wei
4fb570e104
docs: Enable Syntax Highlighting for Code Snippits 2019-01-13 16:12:41 +08:00
Daniel Shahaf
864864442e tests: Allow marking the cardinality check as TODO (XFail).
Needed for next commit.
2018-12-29 11:31:18 +00:00
Matthew Martin
a9be0975c8 tests: Directly diff expected_region_highlight against region_highlight 2018-02-10 14:49:07 -06:00
Matthew Martin
f4d37b74cc tests: Add ability to skip tests 2017-11-06 07:08:53 -06:00
Matthew Martin
038409c10d tests: Run tests with WARN_CREATE_GLOBAL 2017-11-05 15:04:49 -06:00
Daniel Shahaf
add6825898 dev tools: Extend tests/generate.zsh. 2016-07-29 19:02:40 +00:00
Daniel Shahaf
8013dc3b8d dev tools: Add a script that generates a test-data file. 2016-07-29 18:58:37 +00:00
Matthew Martin
364f206a54 docs: Give example of test that modifies its environment 2016-05-12 22:48:07 -05:00
Matthew Martin
19acd8e844 docs: Mention $BUFFER 2016-05-12 22:48:03 -05:00
Matthew Martin
13018f3dd7 docs: Document use of NONE in expected_region_highlight 2016-05-12 22:47:55 -05:00
Daniel Shahaf
9b64ad750f tests: Add a 'print failures only' mode to 'make test', called 'make quiet-test'.
Fixes zsh-users/zsh-syntax-highlighting#262.

Currently, 'make quiet-test' uses Perl.  However, since it is considered a development
tool rather than a user-facing tool, users and downstream packages needn't install Perl.
Furthermore, even this dev-only dependency may be dropped in the future.

The only difference between tests/tap-filter here and the one in the issue is using
a `cat` subshell v. using 'undef $/; <STDIN>'.
2016-01-02 21:22:01 +00:00
Daniel Shahaf
b989ae23d0 docs copyedit: Rewrap to 80 columns, part 3. 2015-11-24 06:06:16 +00:00
Daniel Shahaf
3f163a2fd4 docs copyedit: Rewrap to 80 columns, part 2. 2015-11-24 06:06:16 +00:00
Daniel Shahaf
9ce97cd64d docs copyedit: minor. 2015-11-24 06:06:16 +00:00
Daniel Shahaf
49f7beecb5 docs copyedit: Review toplevel and tests README.md files. 2015-11-24 06:06:16 +00:00
Daniel Shahaf
6c033e62e9 docs copyedit: Whitespace. 2015-11-24 06:06:16 +00:00
Daniel Shahaf
c015339202 tests: Provide an independent, auto-cleaned working directory to each test.
Fixes zsh-users/zsh-syntax-highlighting#182.
Prerequisite for testing issue #228.

* tests/test-highlighting.zsh
  (run_test): Move functionality to run_test_internal; make run_test be a wrapper
    that handles creating and cleaning up the tempdir.

* tests/README.md: Document the new feature.

* "highlighters/main/test-data/path-space- .zsh"
* highlighters/main/test-data/path-tilde-named.zsh
* highlighters/main/test-data/path.zsh
    Change test data to not depend on being run from the source directory.
2015-11-16 22:54:52 +00:00
Daniel Shahaf
b5d02a2f49 tests: Document test isolation, implemented some time ago. 2015-11-16 22:31:18 +00:00
Daniel Shahaf
4513eaea71 'make perf': New target. 2015-10-27 09:47:09 +02:00
Matthew Martin
d99aa58aaa test harness: Run each test as a single subshell.
That allows tests to be completely independent of each other, so tests that
change global state — such as modify environment variables, define functions or
aliases, or hash commands — will not affect other tests, without needing an
explicit cleanup step.

This enables testing path-tilde-home with and without $HOME set, which is part
of issue #216.

While at it, convert the test harness to TAP.  This fixes issue #180 by adding
support for "not ok 42 # TODO" output.

This commit assumes that 'grep' supports POSIX-compliant -q and -v flags.

Patch-by: Matthew Martin <phy1729@gmail.com>
2015-10-23 03:31:00 +00:00
Daniel Shahaf
c46b8d169e test harness: Fix off-by-one discrepancy between observed and expected.
Fixes issue #195.
2015-09-12 20:48:25 +00:00
Julien Nicoulaud
ef4f5ed638 Fix READMEs 2011-06-12 23:16:41 +02:00
Julien Nicoulaud
0772ddd346 Big refactoring.
* Don't override user defined styles
* Better modularisation of highlighters
* Allow to define which highlighters are activated
* Allow to define the order in which they are defined
* Minor performance optimizations
* Fixed some variables leak
* Improve documentation
* Brackets highlighter: use ZSH_HIGHLIGHT_STYLES instead of a specific array
2011-06-12 22:57:14 +02:00