Commit Graph

27 Commits

Author SHA1 Message Date
Mahmoud Al-Qudsi
2cdacbdce4 Restore sanity to line continuations
Line continuations (i.e. escaped new lines) now make sense again. With
the smart pipe support (pipes continue on to next line) recently added,
this hack to have continuations ignore comments makes no sense.

This is valid code:

```fish
echo hello |
# comment here
tr -d 'l'
```

this isn't:
```fish
echo hello | \
# comment here
tr -d 'l'
```

Reverts @snnw's 318daaffb2
Closes #2928. Closes #2929.
2018-03-12 08:12:23 -05:00
Mahmoud Al-Qudsi
00f95a978e Make { and } valid, first-class tokenizer elements 2018-03-11 19:36:10 -05:00
Mahmoud Al-Qudsi
f508a1f274 Reset tokenizer state on start and improve slice error detection 2018-03-11 12:13:55 -05:00
Mahmoud Al-Qudsi
fb1c7a3d68 Simplify token parser 2018-03-10 18:42:56 -06:00
ridiculousfish
8ded041352 Add && and || support to tokenizer 2018-03-05 12:20:56 -08:00
ridiculousfish
c4d903ff98 Rationalize how the parser reports tokenizer errors
Remove the unnecessary SQUASH_ERROR flag and correctly report errors
generated from the tokenizer.
2018-02-23 17:28:12 -08:00
ridiculousfish
0950c35eb2 Reduce the amount of copying when the parser drives the tokenizer 2018-02-23 15:58:13 -08:00
ridiculousfish
99fb7bb6aa Refactor how redirections are represented by the tokenizer
Prior to this fix, each redirection type was a separate token_type.
Unify these under a single type TOK_REDIRECT and break the redirection
type out into a new sub-type redirection_type_t.
2018-02-23 15:19:58 -08:00
ridiculousfish
6673fe5457 Clean up tokenizer implementation
Rather than storing a bunch of "next_foo" fields, simply populate the
tok_t directly.
2018-02-23 14:31:13 -08:00
ridiculousfish
0f62161b2b Clean up tokenizer error handling. 2018-02-19 16:31:39 -08:00
ridiculousfish
f30bf40300 Clean up comment parsing in tokenizer
Unify the show_comments and non-show_comments path.
2018-02-19 16:20:47 -08:00
ridiculousfish
b13ee818d2 Some early cleanup of tokenizer
Prior to this the tokenizer ran "one ahead", where tokenizer_t::next()
would in fact return the last-parsed token. Switch to parsing on demand
instead of running one ahead; this is simpler and prepares for tokenizer
changes.
2018-02-19 15:10:59 -08:00
ridiculousfish
fd2a0dffa9 Reflect newlines after pipes in fish grammar
The previous attempt to support newlines after pipes changed the lexer to
swallow newlines after encountering a pipe. This has two problems that are
difficult to fix:

1. comments cannot be placed after the pipe
2. fish_indent won't know about the newlines, so it will erase them

Address these problems by removing the lexer behavior, and replacing it
with a new parser symbol "optional_newlines" allowing the newlines to be
reflected directly in the fish grammar.
2018-02-18 14:44:08 -08:00
slama
38418d6356 Enable multi-line edit when the line ends with a pipe (#1285) 2018-02-18 13:01:38 -08:00
Aaron Gyes
fa78a7101c Make IWYU output in lint.cpp less messy
And re-run IWYU, adjust #includes.
2016-06-23 17:26:08 -07:00
Kurtis Rader
c14bac4284 restyle tokenizer module to match project style
Reduces lint errors from 70 to 46 (-34%). Line count from 1158 to 936 (-19%).

Another step in resolving issue #2902.
2016-05-03 15:00:44 -07:00
Kurtis Rader
1f06e5f0b9 add better support for IWYU and fix things
Remove the "make iwyu" build target. Move the functionality into the
recently introduced lint.fish script. Fix a lot, but not all, of the
include-what-you-use errors. Specifically, it fixes all of the IWYU errors
on my OS X server but only removes some of them on my Ubuntu 14.04 server.

Fixes #2957
2016-04-26 15:02:22 -07:00
ridiculousfish
10f3ea0008 Mark a bunch of constructors as explicit
This prevents undesired implicit conversions
2016-02-27 19:38:15 -08:00
ridiculousfish
aa76f64a94 Correct the caret position for unbalanced square brackets 2015-08-10 19:30:21 -07:00
ridiculousfish
e34a8da5d7 Correct the positioning of the error caret
When an error occurs midway through a token, like abc(def,
make the caret point at the location of the error (i.e. the paren)
instead of at the beginning of the token.
2015-08-10 18:31:20 -07:00
David Adam
a6a16808e3 Merge branch 'iwyu'
http://include-what-you-use.org/
2015-07-29 09:30:19 +08:00
ridiculousfish
c9349f2ef6 Continue to refine tokenizer interface
Migrate some functions into tokenizer instance methods
2015-07-26 00:58:32 -07:00
ridiculousfish
4ebaa7b6bd Continue migration to the new tokenizer interface 2015-07-26 00:12:36 -07:00
ridiculousfish
618896c043 Early reworking of tokenizer interface 2015-07-25 23:05:47 -07:00
ridiculousfish
0dbd83ffaf Remove some dead code 2015-07-25 22:47:44 -07:00
David Adam
3929e9de0e Merge branch 'master' into iwyu 2015-07-26 10:20:13 +08:00
ridiculousfish
b4f53143b0 Migrate source files into src/ directory
This change moves source files into a src/ directory,
and puts object files into an obj/ directory. The Makefile
and xcode project are updated accordingly.

Fixes #1866
2015-07-24 00:59:27 -07:00