glcpp: Fix so that trailing punctuation does not prevent macro expansion

The trick here is that flex always chooses the rule that matches the most
text. So with a input text of "two:" which we want to be lexed as an
IDENTIFIER token "two" followed by an OTHER token ":" the previous OTHER
rule would match longer as a single token of "two:" which we don't want.

We prevent this by forcing the OTHER pattern to never match any
characters that appear in other constructs, (no letters, numbers, #,
_, whitespace, nor any punctuation that appear in CPP operators).

Fixes bug #44764:

	GLSL preprocessor doesn't replace defines ending with ":"
	https://bugs.freedesktop.org/show_bug.cgi?id=44764

Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>

NOTE: This is a candidate for stable release branches.
This commit is contained in:
Carl Worth 2012-01-21 09:24:11 -08:00
parent e931b0e05b
commit 7ab1c7f792

View file

@ -70,7 +70,15 @@ HSPACE [ \t]
HASH ^{HSPACE}*#{HSPACE}*
IDENTIFIER [_a-zA-Z][_a-zA-Z0-9]*
PUNCTUATION [][(){}.&*~!/%<>^|;,=+-]
OTHER [^][(){}.&*~!/%<>^|;,=#[:space:]+-]+
/* The OTHER class is simply a catch-all for things that the CPP
parser just doesn't care about. Since flex regular expressions that
match longer strings take priority over those matching shorter
strings, we have to be careful to avoid OTHER matching and hiding
something that CPP does care about. So we simply exclude all
characters that appear in any other expressions. */
OTHER [^][_#[:space:]#a-zA-Z0-9(){}.&*~!/%<>^|;,=+-]
DIGITS [0-9][0-9]*
DECIMAL_INTEGER [1-9][0-9]*[uU]?