Skip to content

grep: better support invalid UTF-8 haystacks

Ævar Arnfjörð Bjarmason git 3 versions New

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v5 Latest2 patches 0 repliesNew
  2. v42 patches 5 repliesSuperseded
  3. v34 patches 2 repliesSuperseded

Patches in v5 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/2grep/pcre2 tests: don't rely on invalid UTF-8 data testt/t7812-grep-icase-non-ascii.sh+1 -60 replies
2/2grep/pcre2: better support invalid UTF-8 haystacksMakefile grep.c grep.h +4 more+81 -20 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v5 yet.

Read the v5 thread

Cover letter [PATCH v5 0/2] grep: better support invalid UTF-8 haystacks

Fixes a version comparison typo/thinko, pointed out by Ramsay Jones.

Ævar Arnfjörð Bjarmason (2):
grep/pcre2 tests: don't rely on invalid UTF-8 data test grep/pcre2: better support invalid UTF-8 haystacks

 Makefile                        |  1 +
 grep.c                          | 18 ++++++++++-
 grep.h                          |  4 +++
 t/helper/test-pcre2-config.c    | 12 ++++++++
 t/helper/test-tool.c            |  1 +
 t/helper/test-tool.h            |  1 +
 t/t7812-grep-icase-non-ascii.sh | 53 ++++++++++++++++++++++++++++-----
 7 files changed, 82 insertions(+), 8 deletions(-)
 create mode 100644 t/helper/test-pcre2-config.c

Range-diff:

1:  699bb6b324 = 1:  699bb6b324 grep/pcre2 tests: don't rely on invalid UTF-8 data test
2:  e4807d6879 ! 2:  04c87c04d7 grep/pcre2: better support invalid UTF-8 haystacks
    @@ grep.c: static void compile_pcre2_pattern(struct grep_pat *p, const struct grep_
     -		options |= PCRE2_UTF;
     +		options |= (PCRE2_UTF | PCRE2_MATCH_INVALID_UTF);
     +
    -+	if (PCRE2_MATCH_INVALID_UTF &&
    -+	    options & (PCRE2_UTF | PCRE2_CASELESS) &&
    -+	    !(PCRE2_MAJOR >= 10 && PCRE2_MAJOR >= 36))
    -+		/* Work around https://bugs.exim.org/show_bug.cgi?id=2642 fixed in 10.36 */
    -+		options |= PCRE2_NO_START_OPTIMIZE;
    ++	/* Work around https://bugs.exim.org/show_bug.cgi?id=2642 fixed in 10.36 */
    ++	if (PCRE2_MATCH_INVALID_UTF && options & (PCRE2_UTF | PCRE2_CASELESS)) {
    ++		struct strbuf buf;
    ++		int len;
    ++		int err;
    ++
    ++		if ((len = pcre2_config(PCRE2_CONFIG_VERSION, NULL)) < 0)
    ++			BUG("pcre2_config(..., NULL) failed: %d", len);
    ++		strbuf_init(&buf, len + 1);
    ++		if ((err = pcre2_config(PCRE2_CONFIG_VERSION, buf.buf)) < 0)
    ++			BUG("pcre2_config(..., buf.buf) failed: %d", err);
    ++		if (versioncmp(buf.buf, "10.36") < 0)
    ++			options |= PCRE2_NO_START_OPTIMIZE;
    ++		strbuf_release(&buf);
    ++	}
      
      	p->pcre2_pattern = pcre2_compile((PCRE2_SPTR)p->pattern,
      					 p->patternlen, options, &error, &erroffset,
Signature · 1 line
2.29.2.222.g5d2a92d10f8