scanner: fix reading of really long line
PATCH 1/1 scanner: fix reading of really long line
Current code is causing a failure in adding a set containing a really long list of elements. The failure occurs as soon as the line is longer than flex read buffer.
When a line is longer than scanner buffer size, the code in YY_INPUT forces a rewind to the beginning of the string because it does not find a end of line. The result is that the string is never parsed.
This patch updates the code by rewinding till we found a space.
--- src/scanner.l | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
src/scanner.l +2 -1
@@ -47,7 +47,8 @@ clearerr(yyin); \ } \ if (result > 1) { \ - while (result > 1 && buf[result - 1] != '\n') \ + while (result > 1 && \ + (buf[result - 1] != '\n' && buf[result - 1] != ' ')) \ result--, n++; \ result--, n++; \ fseek(yyin, -n, SEEK_CUR); \
Signature · 1 line
2.1.3
Pablo Neira Ayuso
maintainerQuoted 9 lines · Eric Leblond
Current code is causing a failure in adding a set containing a really long list of elements. The failure occurs as soon as the line is longer than flex read buffer.
When a line is longer than scanner buffer size, the code in
YY_INPUTforces a rewind to the beginning of the string because it does not find a end of line. The result is that the string is never parsed.This patch updates the code by rewinding till we found a space.
But the part that didn't fit in will be ignore, right? So the user will lose some elements in the set?
If so, I think it's better to spot an error via YY_FATAL_ERROR to indicate that the line is too long to the user.