allow show_token() on TOKEN_ZERO_IDENT
PATCH 1/1 allow show_token() on TOKEN_ZERO_IDENT
From: Luc Van Oostenryck <[email protected]>
TOKEN_ZERO_IDENTs are created during the evaluation of pre-processor expressions but which otherwise are normal idents and were first tokenized as TOKEN_IDENTs.
As such, they could perfectly be displayed by show_token() but are not. So, in error messages they are displayed as "unhandled token type '4'", which is not at all informative.
Fix this by letting show_token() process them like usual TOKEN_IDENTs. Idem for quote_token().
--- tokenize.c | 2 ++ 1 file changed, 2 insertions(+)
tokenize.c +2 -0
@@ -201,6 +201,7 @@ const char *show_token(const struct token *token)return "end-of-input" ;case TOKEN_IDENT : +case TOKEN_ZERO_IDENT :return show_ident (token->ident);case TOKEN_NUMBER :
@@ -259,6 +260,7 @@ const char *quote_token(const struct token *token)return "syntax error" ;case TOKEN_IDENT : +case TOKEN_ZERO_IDENT :return show_ident (token->ident);case TOKEN_NUMBER :
Signature · 1 line
2.36.1
Quoted 11 lines · Luc Van Oostenryck
TOKEN_ZERO_IDENTsare created during the evaluation of pre-processor expressions but which otherwise are normal idents and were first tokenized asTOKEN_IDENTs.As such, they could perfectly be displayed by
show_token()but are not. So, in error messages they are displayed as "unhandled token type '4'", which is not at all informative.Fix this by letting
show_token()process them like usualTOKEN_IDENTs. Idem forquote_token().
Ack.
I do wonder if it should be marked somehow as being that special case. The main reason for 'show_token()' is debugging, after all, and TOKEN_ZERO_IDENT does have magical properties in how it either silently expands to the constant '0', or it generates a warning about undefined preprocessor symbol.
But considering that we've apparently reported it as "unhandled token type '4'" since 2005, I guess it's not exactly a big deal.
Linus
Quoted 15 lines · Linus Torvalds
Quoted 11 lines · Luc Van Oostenryck
TOKEN_ZERO_IDENTsare created during the evaluation of pre-processor expressions but which otherwise are normal idents and were first tokenized asTOKEN_IDENTs.As such, they could perfectly be displayed by
show_token()but are not. So, in error messages they are displayed as "unhandled token type '4'", which is not at all informative.Fix this by letting
show_token()process them like usualTOKEN_IDENTs. Idem forquote_token().Ack.
Quoted 8 lines
I do wonder if it should be marked somehow as being that special case. The main reason for '
show_token()' is debugging, after all, andTOKEN_ZERO_IDENTdoes have magical properties in how it either silently expands to the constant '0', or it generates a warning about undefined preprocessor symbol.But considering that we've apparently reported it as "unhandled token type '4'" since 2005, I guess it's not exactly a big deal.
Yes, I first thought to do so but then choose not because I could not convince myself that its special property was irrelevant in warning/error messages. It looks to me more as an internal thing, more semantical than lexical, and a non-faithful representation would be confusing in messages.
For context, the input text I had (from GCC's testsuite) was:
#define empty #if empty#cpu(m68k) #endif
and the error message sparse issued was:
error: garbage at end: #unhandled token type '4' (unhandled token type '4' )
with this patch it's:
error: garbage at end: #cpu(m68k)
-- Luc