Fix two whitespace bugs in the lexer.

The first bug was not allowing whitespace between '#' and the
directive name.

The second bug was swallowing a terminating newline along with any
trailing whitespace on a line.

With these two fixes, and the previous commit to stop emitting SPACE
tokens, the recently added extra-whitespace test now passes.
This commit is contained in:
Carl Worth 2010-05-14 17:29:24 -07:00
parent 81f01432bd
commit e36a4d5be9

View file

@ -39,7 +39,7 @@ SPACE [[:space:]]
NONSPACE [^[:space:]]
NEWLINE [\n]
HSPACE [ \t]
HASH ^{HSPACE}*#
HASH ^{HSPACE}*#{HSPACE}*
IDENTIFIER [_a-zA-Z][_a-zA-Z0-9]*
TOKEN [^[:space:](),]+
@ -109,6 +109,6 @@ TOKEN [^[:space:](),]+
return NEWLINE;
}
{SPACE}+
{HSPACE}+
%%