Section 3.3 (Preprocessor) of the GLSL 1.30 spec (and later) and the
GLSL ES spec (all versions) say:
"All macro names containing two consecutive underscores ( __ ) are
reserved for future use as predefined macro names. All macro names
prefixed with "GL_" ("GL" followed by a single underscore) are also
reserved."
The intention is that names containing __ are reserved for internal use
by the implementation, and names prefixed with GL_ are reserved for use
by Khronos. Since every extension adds a name prefixed with GL_ (i.e.,
the name of the extension), that should be an error. Names simply
containing __ are dangerous to use, but should be allowed. In similar
cases, the C++ preprocessor specification says, "no diagnostic is
required."
Per the Khronos bug mentioned below, a future version of the GLSL
specification will clarify this.
Signed-off-by: Ian Romanick <ian.d.romanick@intel.com>
Cc: "9.2 10.0 10.1" <mesa-stable@lists.freedesktop.org>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Tested-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Anuj Phogat <anuj.phogat@gmail.com>
Tested-by: Darius Spitznagel <d.spitznagel@goodbytez.de>
Cc: Tapani Pälli <lemody@gmail.com>
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=71870
Bugzilla: Khronos #11702
This is the innocent-looking but killer test case to verify the bug fixed in
the preceding commit.
Reviewed-by: Jordan Justen <jordan.l.justen@intel.com>
The preprocessor currently accepts multiple else/elif-groups
per if-section. The GLSL-preprocessor is defined by the C++
specification, which defines the following parse-rule:
if-section:
if-group elif-groups(opt) else-group(opt) endif-line
This clearly only allows a single else-group, that has to come
after any elif-groups.
So let's modify the code to follow the specification. Add test
to prevent regressions.
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Carl Worth <cworth@cworth.org>
Cc: 10.0 <mesa-stable@lists.freedesktop.org>
The preprocessor has always replaced multi-line comments with a single space
character, (as required by the specification), but as of commit
bd55ba568b the lexer also emitted a NEWLINE
token for each newline within the comment, (in order to preserve line
numbers).
The emitting of NEWLINE tokens within the comment broke the rule of "replace a
multi-line comment with a single space" as could be exposed by code like the
following:
#define FOO a/*
*/b
FOO
Prior to commit bd55ba568b, this code defined
the macro FOO as "a b" as desired. Since that commit, this code instead
defines FOO as "a" and leaves a stray "b" in the output.
In this commit, we fix this by not emitting the NEWLINE tokens while lexing
the comment, but instead merely counting them in the commented_newlines
variable. Then, when the lexer next encounters a non-commented newline it
switches to a NEWLINE_CATCHUP state to emit as many NEWLINE tokens as
necessary (so that subsequent parsing stages still generate correct line
numbers).
Of course, it would have been more clear if we could have written a loop to
emit all the newlines, but flex conventions prevent that, (we must use
"return" for each token we emit).
It similarly would have been clear to have a new rule restricted to the
<NEWLINE_CATCHUP> state with an action much like the body of this if
condition. The problem with that is that this rule must not consume any
characters. It might be possible to write a rule that matches a single
lookahead of any character, but then we would also need an additional rule to
ensure for the <EOF> case where there are no additional characters available
for the lookahead to match.
Given those considerations, and given that the SKIP-state manipulation already
involves a code block at the top of the lexer function, before any rules, it
seems best to me to go with the implementation here which adds a similar
pre-rule code block for the NEWLINE_CATCHUP.
Finally, this commit also changes the expected output of a few, existing glcpp
tests. The change here is that the space character resulting from the
multi-line comment is now emitted before the newlines corresponding to that
comment. (Previously, the newlines were emitted first, and the space character
afterward.)
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=72686
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Ian Romanick <ian.d.romanick@intel.com>
To trigger the bug, it suffices to have a line-continuation followed by
a newline and then a non-line-continuation backslash.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
First we test that line continuations are honored within a comment, (as
recently changed in glcpp), then we test that line continuations can be
disabled via an option within the context. This is tested via the new support
for a test-specific command-line option passed to glcpp.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
This will allow the test exercising disabled line continuations to arrange
for the --disable-line-continuations argument to be passed to the standalone
glcpp.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
This test file is very similar to test 113-line-and-file-macros but uses token
pasting for cleaner quiz answers (without spaces between the digits). This
test passes thanks to the recent addition of support for pasting INTEGER
tokens, (but would have failed without that).
(Note that this test is distinct from test 059-token-pasting-integer which
pastes integers parsed from the source. Those are parsed to INTEGER_STRING
tokens and are already pasted correctly as verified by that test. The only way
to generate the INTEGER tokens which currently fail to paste is with an
internal define such as __LINE__ that results in an integer.)
Reviewed-by: Matt Turner <mattst88@gmail.com>
The current code lets a few invalid pastes through, such as an string pasted
onto the end of an integer. Extend the invalid-paste test to catch some of
these.
Reviewed-by: Matt Turner <mattst88@gmail.com>
These tokens are easy to expand by just looking at the current, tracked
location values, (and no need to look anything up in the hash table).
Add a test which verifies __LINE__ with several values, (and verifies __FILE__
for the single value of 0). Our testing framework isn't sophisticated enough
here to have a test with multiple file inputs.
This commit fixes part of es3conform's preprocess16_frag test.
Reviewed-by: Matt Turner <mattst88@gmail.com>
And add test cases to ensure that this works
- 110 verifies that glcpp rejects #elif<digits> which glcpp
previously accepted.
- 111 verifies that glcpp accepts #if followed immediately by
(, +, -, !, or ~.
- 112 does the same as 111 but for #elif.
See 17f9beb6 for #if change.
Reviewed-by: Carl Worth <cworth@cworth.org>
By moving the HASH_LINE rule out of control_line: and into line:, we avoid
adding control_line's additional \n (as seen in the first hunk).
mattst88: Carl and I determined independently of Fabian that the 091
test needed to be modified identically to this, and our patch to fix the
test was more complicated.
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=51506
Reviewed-by: Matt Turner <mattst88@gmail.com>
Reviewed-by: Carl Worth <cworth@cworth.org>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Previously we were accepting garbage after #else and #endif tokens when
the previous preprocessor conditional evaluated to false (eg, #if 0).
When the preprocessor hits a false conditional, it switches the lexer
into the SKIP state, in which it ignores non-control tokens. The parser
pops the SKIP state off the stack when it reaches the associated #elif,
#else, or #endif. Unfortunately, that meant that it only left the SKIP
state after the lexing the entire line containing the #token and thus
would accept garbage after the #token.
To fix this we use a mid-rule, which is executed immediately after the
#token is parsed.
NOTE: This is a candidate for the stable branch
Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=56442
Fixes: preprocess17_frag.test from oglconform
Reviewed-by: Carl Worth <cworth@cworth.org> (glcpp-parse.y)
Acked-by: Ian Romanick <ian.d.romanick@intel.com>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
The most recent commit adds support for comments and macro expansion
on #line directives. Add testing to verify the new features.
Signed-off-by: Carl Worth <cworth@cworth.org>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
The GLSL specification requires that #line directives be interpreted
after macro expansion. Our existing implementation of #line macros in
the lexer prevents conformance on this point.
Moving the handling of #line from the lexer to the parser gives us the
macro expansion we need. An additional benefit is that the
preprocessor also now supports comments on the same line as #line
directives.
Finally, the preprocessor now emits the (fully-macro-expanded) #line
directives into the output. This allows the full GLSL compiler to also
see and interpret these directives so it can also generate correct
line numbers in error messages.
Signed-off-by: Carl Worth <cworth@cworth.org>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
A strict reading of the GLSL specification would have this be an
error, but we've received reports from users who expect the
preprocessor to interepret undefined macros as 0. This is the standard
behavior of the rpeprocessor for C, and according to these user
reports is also the behavior of other OpenGL implementations.
So here's one of those cases where we can make our users happier by
ignoring the specification. And it's hard to imagine users who really,
really want to see an error for this case.
The two affected tests cases are updated to reflect the new behavior.
Signed-off-by: Carl Worth <cworth@cworth.org>
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
This test cases exposes a bug as described in this bug report:
"ralloc.c:78: get_header: Assertion `info->canary == 0x5A1106'
failed." when using a macro in GLSL
https://bugs.freedesktop.org/show_bug.cgi?id=45082
Clearly, some memory is getting (incorrectly) freed on the first macro
invocation, leading to problems with the second macro invocation.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
This demonstrates a bug that was recently triggered in piglit.
Here is the original bug report (containing a test case almost identical
to this one):
https://bugs.freedesktop.org/show_bug.cgi?id=44764
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Success was (tests-passed AND valgrind-tests-passed) but this meant that
if the valgrind tests weren't run it would be considered a failure.
The logic is now (tests-passed AND (!valgrind OR valgrind-tests-passed))
which lets us return success if the valgrind tests aren't run.
Reviewed-by: Eric Anholt <eric@anholt.net>
Tested-by: Eric Anholt <eric@anholt.net>
Signed-off-by: Matt Turner <mattst88@gmail.com>
As written, this test correctly raises an error for #elif being used
with an undefined macro (and not as an argument to "defined"). If the
preceding #if were '#if 1' then this diagnositc would correctly be
hidden. That allows code such as the following to not raise an error:
#ifndef MAYBE_UNDEFINED
#elif MAYBE_UNDEFINED < 5
...
#endif
So this test case is working as expected already. We add it here just
to improve test coverage.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Carl Worth <cworth@cworth.org>
The specification reserves any macro name containing two consecutive
underscores, (anywhere within the name). Previously, we only raised
this error for macro names that started with two underscores.
Fix the implementation to check for two underscores anywhere, and also
update the corresponding 086-reserved-macro-names test.
This also fixes the following two piglit tests:
spec/glsl-1.30/preprocessor/reserved/double-underscore-02.frag
spec/glsl-1.30/preprocessor/reserved/double-underscore-03.frag
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Reviewed-by: Eric Anholt <eric@anholt.net>
Signed-off-by: Carl Worth <cworth@cworth.org>
Apparently we never implemented this, (but we've got a GLSL 1.30 test
in piglit that is exercising this case).
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Carl Worth <cworth@cworth.org>
This is something that piglit is exercising that currently fails.
Reviewed-by: Kenneth Graunke <kenneth@whitecape.org>
Signed-off-by: Carl Worth <cworth@cworth.org>
It's clear enough that the current segmentation fault isn't what we
want. And it's also very easy to know what we do want here, (just
check with any functional C preprocessor such as "gcc -E").
Add the desired output as an expected file so that the test suite
gives useful output, (showing the omitted output and the segfault),
rather than just reporting "No such file" for the expected file.
The common case for this test suite is to quickly test that everything
returns the correct results. In this case, the second run of the test
suite under valgrind was just annoying, (and the user would often
interrupt it).
Now, do what is wanted in the common case by default (just run the
test suite), and require a run with "glcpp-test --valgrind" in order
to test with valgrind.
The expected file here captures the current behavior of glcpp (which
is to generate an obscure "syntax error, unexpected $end" diagnostic
for this case).
It would certainly be better for glcpp to generate a nicer diagnostic,
(such as "missing closing parenthesis in function-like macro
definition" or so), but the current behavior is at least correct, and
expected. So we can make the test suite more useful by marking the
current behavior as expected.
The expected file here captures the current behavior of glcpp (which
is to generate a division-by-zero error) for this case.
It's easy to argue that it should be short-circuiting the evaluation
and not generating the diagnostic (which happens to be what gcc does).
But it doesn't seem like we should force this behavior on our
pre-processor, (and, as always, the GLSL specification of the
pre-processor is too vague on this point).
This test is behaving just fine already---it's generating an informative
diagnostic, ("error: division by 0 in preprocessor directive"), so adding
this in the expected file makes things pass.
The expected result has been out of sync with what glcpp produces for
some time; glcpp's actual result seems to be correct and is very close to
GCC's cpp. Updating this will make it easier to catch regressions in
upcoming commits.
This reverts commit d3df641f0a.
The original commit had sat unpushed on my machine for months. By the
time I found it again, I had forgotten that we had decided not to use
this change after all, (the relevant test was removed long ago).
The GLSL specification is vague here, (just says "as is standard for
C++"), though the C specifications seem quite clear that this should
be an error.
However, an existing piglit test (CorrectPreprocess11.frag) expects
this to be a warning, not an error, so we change this, and document in
README the deviation from the specification.
We recently added several tests that intentionally trigger
preprocessor errors. During valgrind-based testing, our test script
was noticing the non-zero return value from the preprocessor and
incorrectly flagging the valgrind-based test as failing.
To fix this, we make valgrind return an error code that is otherwise
unused by the preprocessor.
This test exposes two current bugs:
1. The source number is not being correctly emitted in error
messages (instead, it's always 0).
2. A directive of "#line 0" is resulting in the following
parse error:
preprocessor error: Invalid tokens after #
This directive is already implemented nicely, but wasn't previously tested.
It will be convenient to use this directive in further tests that rely
on error messages, (such as ensuring that #line correctly sets the line
number in the error message).
The specification says that redefining a macro is an error, unless the
new definitions is identical to the old one, (identical replacement
lists but ignoring differing amounts of whitespace).
In commit 6be3a8b70a, the #version directive
was fixed to stop generating a spurious newline. Here we simply update
the expected result for the single test which includes a #version directive.
This was previously being appended to the output string *after* a copy
of the supposedly final string was made and handed to the caller. So
the diagnostic was never actually visible to the user.
We fix this by moving the check for an unterminated #if from
glcpp_parser_destroy to the calling function, preprocess.
This fixes the test case 083-unterminated-if.c.
