[PATCH 0/3] fuzz: port OSS-Fuzz tests back to Git

Josh Steadmon

author

2024-10-10 21:11 UTC #0 lore raw

cover fuzz: port OSS-Fuzz tests back to Git

Git's fuzz tests are run continuously as part of OSS-Fuzz [1]. Several additional fuzz tests have been contributed directly to OSS-Fuzz; however, these tests are vulnerable to bitrot because they are not built during Git's CI runs, and thus breaking changes are much less likely to be noticed by Git contributors.

OSS-Fuzz's recommended setup is for tests to live in the repository of the code they test and to be built along with other tests [1].

Port some of these tests back to the Git project, so that they can be built and tested during our normal development workflow and CI, and as such avoid future bitrot.

[1] https://google.github.io/oss-fuzz/advanced-topics/ideal-integration/

Eric Sesterhenn (3):
fuzz: port fuzz-credential-from-url-gently from OSS-Fuzz
fuzz: port fuzz-parse-attr-line from OSS-Fuzz
fuzz: port fuzz-url-decode-mem from OSS-Fuzz

 Makefile                                   |  3 ++
 attr.c                                     | 38 +------------------
 attr.h                                     | 43 ++++++++++++++++++++++
 ci/run-build-and-minimal-fuzzers.sh        | 15 +++++++-
 oss-fuzz/.gitignore                        |  3 ++
 oss-fuzz/fuzz-credential-from-url-gently.c | 32 ++++++++++++++++
 oss-fuzz/fuzz-parse-attr-line.c            | 39 ++++++++++++++++++++
 oss-fuzz/fuzz-url-decode-mem.c             | 43 ++++++++++++++++++++++
 8 files changed, 177 insertions(+), 39 deletions(-)
 create mode 100644 oss-fuzz/fuzz-credential-from-url-gently.c
 create mode 100644 oss-fuzz/fuzz-parse-attr-line.c
 create mode 100644 oss-fuzz/fuzz-url-decode-mem.c

base-commit: 159f2d50e75c17382c9f4eb7cbda671a6fa612d1

Signature · 1 line

2.47.0.rc1.288.g06298d1525-goog

Josh Steadmon

author

2024-10-10 21:12 UTC #1 lore raw

PATCH 1/3 fuzz: port fuzz-credential-from-url-gently from OSS-Fuzz

From: Eric Sesterhenn <[email protected]>

Git's fuzz tests are run continuously as part of OSS-Fuzz [1]. Several additional fuzz tests have been contributed directly to OSS-Fuzz; however, these tests are vulnerable to bitrot because they are not built during Git's CI runs, and thus breaking changes are much less likely to be noticed by Git contributors.

Port one of these tests back to the Git project:
fuzz-credential-from-url-gently

This test was originally written by Eric Sesterhenn as part of a security audit of Git [2]. It was then contributed to the OSS-Fuzz repo in commit c58ac4492 (Git fuzzing: uncomment the existing and add new targets. (#11486), 2024-02-21) by Jaroslav Lobačevski. I (Josh Steadmon) have verified with both Eric and Jaroslav that they're OK with moving this test to the Git project.

[1] https://github.com/google/oss-fuzz
[2] https://ostif.org/wp-content/uploads/2023/01/X41-OSTIF-Gitlab-Git-Security-Audit-20230117-public.pdf

Co-authored-by Jaroslav Lobačevski <[email protected]>

Co-authored-by Josh Steadmon <[email protected]>

Signed-off-by Josh Steadmon <[email protected]>

---
 Makefile                                   |  1 +
 ci/run-build-and-minimal-fuzzers.sh        | 13 +++++++--
 oss-fuzz/.gitignore                        |  1 +
 oss-fuzz/fuzz-credential-from-url-gently.c | 32 ++++++++++++++++++++++
 4 files changed, 45 insertions(+), 2 deletions(-)
 create mode 100644 oss-fuzz/fuzz-credential-from-url-gently.c

Makefile +1 -0

index e298c8b55e..3ce391062f 100644
@@ -2378,6 +2378,7 @@ endif
 FUZZ_OBJS += oss-fuzz/dummy-cmd-main.o
 FUZZ_OBJS += oss-fuzz/fuzz-commit-graph.o
 FUZZ_OBJS += oss-fuzz/fuzz-config.o
+FUZZ_OBJS += oss-fuzz/fuzz-credential-from-url-gently.o
 FUZZ_OBJS += oss-fuzz/fuzz-date.o
 FUZZ_OBJS += oss-fuzz/fuzz-pack-headers.o
 FUZZ_OBJS += oss-fuzz/fuzz-pack-idx.o

ci/run-build-and-minimal-fuzzers.sh +11 -2

index af8065f349..d9d3ad23c7 100755
@@ -13,8 +13,17 @@ group "Build fuzzers" make \
 	LIB_FUZZING_ENGINE="-fsanitize=fuzzer,address" \
 	fuzz-all
 
-for fuzzer in commit-graph config date pack-headers pack-idx ; do
+fuzzers="
+commit-graph \
+config \
+credential-from-url-gently \
+date \
+pack-headers \
+pack-idx \
+"
+
+for fuzzer in $fuzzers ; do
 	begin_group "fuzz-$fuzzer"
-	./oss-fuzz/fuzz-$fuzzer -verbosity=0 -runs=1 || exit 1
+	echo ./oss-fuzz/fuzz-$fuzzer -verbosity=0 -runs=1 || exit 1
 	end_group "fuzz-$fuzzer"
 done

oss-fuzz/.gitignore +1 -0

index a877c11f42..2cfc845b20 100644
@@ -1,5 +1,6 @@
 fuzz-commit-graph
 fuzz-config
+fuzz-credential-from-url-gently
 fuzz-date
 fuzz-pack-headers
 fuzz-pack-idx

oss-fuzz/fuzz-credential-from-url-gently.c new file +32 -0

new file mode 100644
index 0000000000..c872f9ad2d
@@ -0,0 +1,32 @@
+#include "git-compat-util.h"
+#include <stddef.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <string.h>
+#include <stdio.h>
+#include "credential.h"
+
+int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);
+
+int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
+{
+	struct credential c;
+	char *buf;
+
+	buf = malloc(size + 1);
+	if (!buf)
+		return 0;
+
+	memcpy(buf, data, size);
+	buf[size] = 0;
+
+	// start fuzzing
+	credential_init(&c);
+	credential_from_url_gently(&c, buf, 1);
+
+	// cleanup
+	credential_clear(&c);
+	free(buf);
+
+	return 0;
+}

Signature · 1 line

2.47.0.rc1.288.g06298d1525-goog

Oswald Buddenhagen

2024-10-11 09:13 UTC #2 lore raw

Re: [PATCH 1/3] fuzz: port fuzz-credential-from-url-gently from OSS-Fuzz

just some nits:

ci/run-build-and-minimal-fuzzers.sh group "Build fuzzers" make \ · line 23

Quoted 9 lines · Josh Steadmon

+++ b/ci/run-build-and-minimal-fuzzers.sh
ci/run-build-and-minimal-fuzzers.sh +8 -0
+fuzzers="
+commit-graph \
+config \
+credential-from-url-gently \
+date \
+pack-headers \
+pack-idx \
+"

the trailing space-backslashes can be left out, which would make the code less noisy. then the variable would contain linebreaks instead of spaces, which the for loop would be just fine with (as $IFS contains LF).

Quoted 2 lines

+for fuzzer in $fuzzers ; do

the space before the semicolon seems excessive.

Review

Quoted 3 lines

+++ b/oss-fuzz/fuzz-credential-from-url-gently.c
(quoted diff) +2 -0
+int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);
+

are these seemingly redundant prototypes meant to suppress compiler warnings?

Quoted 2 lines

oss-fuzz/fuzz-credential-from-url-gently.c new file +2 -0
+int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
+{

Junio C Hamano

2024-10-11 16:35 UTC #3 lore raw

Re: [PATCH 1/3] fuzz: port fuzz-credential-from-url-gently from OSS-Fuzz

Quoted 13 lines · Oswald Buddenhagen

just some nits:
ci/run-build-and-minimal-fuzzers.sh group "Build fuzzers" make \ · line 23
Quoted 6 lines · Josh Steadmon
+++ b/ci/run-build-and-minimal-fuzzers.sh
ci/run-build-and-minimal-fuzzers.sh +4 -0
+fuzzers="
+commit-graph \
...
+pack-idx \
+"
the trailing space-backslashes can be left out, which would make the code less noisy. then the variable would contain linebreaks instead of spaces, which the for loop would be just fine with (as $IFS contains LF).

Perfect.

Thanks.

Josh Steadmon

author

2024-10-14 20:35 UTC #4 lore raw

Re: [PATCH 1/3] fuzz: port fuzz-credential-from-url-gently from OSS-Fuzz

In reply to Oswald Buddenhagen

Quoted 16 lines · Oswald Buddenhagen

just some nits:
ci/run-build-and-minimal-fuzzers.sh group "Build fuzzers" make \ · line 23
Quoted 9 lines · Josh Steadmon
+++ b/ci/run-build-and-minimal-fuzzers.sh
ci/run-build-and-minimal-fuzzers.sh +8 -0
+fuzzers="
+commit-graph \
+config \
+credential-from-url-gently \
+date \
+pack-headers \
+pack-idx \
+"
the trailing space-backslashes can be left out, which would make the code less noisy. then the variable would contain linebreaks instead of spaces, which the for loop would be just fine with (as $IFS contains LF).

Fixed in V2.

Quoted 3 lines

Quoted 2 lines
+for fuzzer in $fuzzers ; do
the space before the semicolon seems excessive.

Fixed in V2.

Quoted 5 lines

Review
Quoted 3 lines
+++ b/oss-fuzz/fuzz-credential-from-url-gently.c
(quoted diff) +2 -0
+int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);
+
are these seemingly redundant prototypes meant to suppress compiler warnings?

Yes, unfortunately we get complaints without them.

Quoted 2 lines

oss-fuzz/fuzz-credential-from-url-gently.c line 12
Quoted 2 lines
oss-fuzz/fuzz-credential-from-url-gently.c new file +2 -0
+int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
+{

Thanks for the review!

Josh Steadmon

author

2024-10-14 20:43 UTC #5 lore raw

Re: [PATCH 1/3] fuzz: port fuzz-credential-from-url-gently from OSS-Fuzz

In reply to Josh Steadmon

Quoted 14 lines · Josh Steadmon

ci/run-build-and-minimal-fuzzers.sh +5 -1

index af8065f349..d9d3ad23c7 100755
@@ -13,8 +13,17 @@ group "Build fuzzers" make \
 	LIB_FUZZING_ENGINE="-fsanitize=fuzzer,address" \
 	fuzz-all
 
-for fuzzer in commit-graph config date pack-headers pack-idx ; do
+fuzzers="
+commit-graph \
+config \
+credential-from-url-gently \
+date \

ci/run-build-and-minimal-fuzzers.sh group "Build fuzzers" make \ · line 29

Quoted 10 lines

ci/run-build-and-minimal-fuzzers.sh +6 -1

+pack-headers \
+pack-idx \
+"
+
+for fuzzer in $fuzzers ; do
 	begin_group "fuzz-$fuzzer"
-	./oss-fuzz/fuzz-$fuzzer -verbosity=0 -runs=1 || exit 1
+	echo ./oss-fuzz/fuzz-$fuzzer -verbosity=0 -runs=1 || exit 1
 	end_group "fuzz-$fuzzer"
 done

I'm not sure how this "echo" got into the diff here, but I've removed it in V2.

Josh Steadmon

author

2024-10-10 21:12 UTC #6 lore raw

PATCH 2/3 fuzz: port fuzz-parse-attr-line from OSS-Fuzz

In reply to Josh Steadmon

From: Eric Sesterhenn <[email protected]>

Git's fuzz tests are run continuously as part of OSS-Fuzz [1]. Several additional fuzz tests have been contributed directly to OSS-Fuzz; however, these tests are vulnerable to bitrot because they are not built during Git's CI runs, and thus breaking changes are much less likely to be noticed by Git contributors.

Port one of these tests back to the Git project:
fuzz-parse-attr-line

This test was originally written by Eric Sesterhenn as part of a security audit of Git [2]. It was then contributed to the OSS-Fuzz repo in commit c58ac4492 (Git fuzzing: uncomment the existing and add new targets. (#11486), 2024-02-21) by Jaroslav Lobačevski. I (Josh Steadmon) have verified with both Eric and Jaroslav that they're OK with moving this test to the Git project.

[1] https://github.com/google/oss-fuzz
[2] https://ostif.org/wp-content/uploads/2023/01/X41-OSTIF-Gitlab-Git-Security-Audit-20230117-public.pdf

Co-authored-by Jaroslav Lobačevski <[email protected]>

Co-authored-by Josh Steadmon <[email protected]>

Signed-off-by Josh Steadmon <[email protected]>

---
 Makefile                            |  1 +
 attr.c                              | 38 +------------------------
 attr.h                              | 43 +++++++++++++++++++++++++++++
 ci/run-build-and-minimal-fuzzers.sh |  1 +
 oss-fuzz/.gitignore                 |  1 +
 oss-fuzz/fuzz-parse-attr-line.c     | 39 ++++++++++++++++++++++++++
 6 files changed, 86 insertions(+), 37 deletions(-)
 create mode 100644 oss-fuzz/fuzz-parse-attr-line.c

Makefile +1 -0

index 3ce391062f..141e194bf5 100644
@@ -2382,6 +2382,7 @@ FUZZ_OBJS += oss-fuzz/fuzz-credential-from-url-gently.o
 FUZZ_OBJS += oss-fuzz/fuzz-date.o
 FUZZ_OBJS += oss-fuzz/fuzz-pack-headers.o
 FUZZ_OBJS += oss-fuzz/fuzz-pack-idx.o
+FUZZ_OBJS += oss-fuzz/fuzz-parse-attr-line.o
 .PHONY: fuzz-objs
 fuzz-objs: $(FUZZ_OBJS)

attr.c +1 ~~-37~~

index 06b5b5e55e..f7898285c5 100644
@@ -259,42 +259,6 @@ const struct git_attr *git_attr(const char *name)
 	return git_attr_internal(name, strlen(name));
 }
 
-/* What does a matched pattern decide? */
-struct attr_state {
-	const struct git_attr *attr;
-	const char *setto;
-};
-
-struct pattern {
-	const char *pattern;
-	int patternlen;
-	int nowildcardlen;
-	unsigned flags;		/* PATTERN_FLAG_* */
-};
-
-/*
- * One rule, as from a .gitattributes file.
- *
- * If is_macro is true, then u.attr is a pointer to the git_attr being
- * defined.
- *
- * If is_macro is false, then u.pat is the filename pattern to which the
- * rule applies.
- *
- * In either case, num_attr is the number of attributes affected by
- * this rule, and state is an array listing them.  The attributes are
- * listed as they appear in the file (macros unexpanded).
- */
-struct match_attr {
-	union {
-		struct pattern pat;
-		const struct git_attr *attr;
-	} u;
-	char is_macro;
-	size_t num_attr;
-	struct attr_state state[FLEX_ARRAY];
-};
-
 static const char blank[] = " \t\r\n";
 
 /* Flags usable in read_attr() and parse_attr_line() family of functions. */

@@ -353,7 +317,7 @@ static const char *parse_attr(const char *src, int lineno, const char *cp,
 	return ep + strspn(ep, blank);
 }
 
-static struct match_attr *parse_attr_line(const char *line, const char *src,
+struct match_attr *parse_attr_line(const char *line, const char *src,
 					  int lineno, unsigned flags)
 {
 	size_t namelen, num_attr, i;

attr.h +43 -0

47 lines not on this page · raw message

ci/run-build-and-minimal-fuzzers.sh +1 -0

index d9d3ad23c7..60fe8b0dfc 100755
@@ -20,6 +20,7 @@ credential-from-url-gently \
 date \
 pack-headers \
 pack-idx \
+parse-attr-line \
 "
 
 for fuzzer in $fuzzers ; do

oss-fuzz/.gitignore +1 -0

index 2cfc845b20..ec185f061c 100644
@@ -4,3 +4,4 @@ fuzz-credential-from-url-gently
 fuzz-date
 fuzz-pack-headers
 fuzz-pack-idx
+fuzz-parse-attr-line

oss-fuzz/fuzz-parse-attr-line.c new file +39 -0

new file mode 100644
index 0000000000..45a4c4e53c
@@ -0,0 +1,39 @@
+#include "git-compat-util.h"
+#include <stddef.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <string.h>
+#include "attr.h"
+
+int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);
+
+int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
+{
+	struct match_attr *res;
+	char *buf;
+
+	buf = malloc(size + 1);
+	if (!buf)
+		return 0;
+
+	memcpy(buf, data, size);
+	buf[size] = 0;
+
+	res = parse_attr_line(buf, "dummy", 0, 0);
+
+	if (res) {
+		int j;
+		for (j = 0; j < res->num_attr; j++) {
+			const char *setto = res->state[j].setto;
+			if (ATTR_TRUE(setto) || ATTR_FALSE(setto) ||
+				ATTR_UNSET(setto))
+				;
+			else
+				free((char *)setto);
+		}
+		free(res);
+	}
+	free(buf);
+
+	return 0;
+}

Signature · 1 line

2.47.0.rc1.288.g06298d1525-goog

Josh Steadmon

author

2024-10-10 21:12 UTC #7 lore raw

PATCH 3/3 fuzz: port fuzz-url-decode-mem from OSS-Fuzz

In reply to Josh Steadmon

From: Eric Sesterhenn <[email protected]>

Git's fuzz tests are run continuously as part of OSS-Fuzz [1]. Several additional fuzz tests have been contributed directly to OSS-Fuzz; however, these tests are vulnerable to bitrot because they are not built during Git's CI runs, and thus breaking changes are much less likely to be noticed by Git contributors.

Port one of these tests back to the Git project:
fuzz-url-decode-mem

This test was originally written by Eric Sesterhenn as part of a security audit of Git [2]. It was then contributed to the OSS-Fuzz repo in commit c58ac4492 (Git fuzzing: uncomment the existing and add new targets. (#11486), 2024-02-21) by Jaroslav Lobačevski. I (Josh Steadmon) have verified with both Eric and Jaroslav that they're OK with moving this test to the Git project.

[1] https://github.com/google/oss-fuzz
[2] https://ostif.org/wp-content/uploads/2023/01/X41-OSTIF-Gitlab-Git-Security-Audit-20230117-public.pdf

Co-authored-by Jaroslav Lobačevski <[email protected]>

Co-authored-by Josh Steadmon <[email protected]>

Signed-off-by Josh Steadmon <[email protected]>

---
 Makefile                            |  1 +
 ci/run-build-and-minimal-fuzzers.sh |  1 +
 oss-fuzz/.gitignore                 |  1 +
 oss-fuzz/fuzz-url-decode-mem.c      | 43 +++++++++++++++++++++++++++++
 4 files changed, 46 insertions(+)
 create mode 100644 oss-fuzz/fuzz-url-decode-mem.c

Makefile +1 -0

index 141e194bf5..2cd7a6e003 100644
@@ -2383,6 +2383,7 @@ FUZZ_OBJS += oss-fuzz/fuzz-date.o
 FUZZ_OBJS += oss-fuzz/fuzz-pack-headers.o
 FUZZ_OBJS += oss-fuzz/fuzz-pack-idx.o
 FUZZ_OBJS += oss-fuzz/fuzz-parse-attr-line.o
+FUZZ_OBJS += oss-fuzz/fuzz-url-decode-mem.o
 .PHONY: fuzz-objs
 fuzz-objs: $(FUZZ_OBJS)

ci/run-build-and-minimal-fuzzers.sh +1 -0

index 60fe8b0dfc..08c556c818 100755
@@ -21,6 +21,7 @@ date \
 pack-headers \
 pack-idx \
 parse-attr-line \
+url-decode-mem \
 "
 
 for fuzzer in $fuzzers ; do

oss-fuzz/.gitignore +1 -0

index ec185f061c..f2d74de457 100644
@@ -5,3 +5,4 @@ fuzz-date
 fuzz-pack-headers
 fuzz-pack-idx
 fuzz-parse-attr-line
+fuzz-url-decode-mem

oss-fuzz/fuzz-url-decode-mem.c new file +43 -0

43 lines not on this page · raw message

Signature · 1 line

2.47.0.rc1.288.g06298d1525-goog

Junio C Hamano

2024-10-10 21:34 UTC #8 lore raw

In reply to Josh Steadmon

Quoted 6 lines · Josh Steadmon

Git's fuzz tests are run continuously as part of OSS-Fuzz [1]. Several additional fuzz tests have been contributed directly to OSS-Fuzz; however, these tests are vulnerable to bitrot because they are not built during Git's CI runs, and thus breaking changes are much less likely to be noticed by Git contributors.

Quoted 8 lines

OSS-Fuzz's recommended setup is for tests to live in the repository of the code they test and to be built along with other tests [1].
Port some of these tests back to the Git project, so that they can be built and tested during our normal development workflow and CI, and as such avoid future bitrot.
[1] https://google.github.io/oss-fuzz/advanced-topics/ideal-integration/

Thanks! Very much appreciated.

Quoted 20 lines

Eric Sesterhenn (3):
fuzz: port fuzz-credential-from-url-gently from OSS-Fuzz
fuzz: port fuzz-parse-attr-line from OSS-Fuzz
fuzz: port fuzz-url-decode-mem from OSS-Fuzz

 Makefile                                   |  3 ++
 attr.c                                     | 38 +------------------
 attr.h                                     | 43 ++++++++++++++++++++++
 ci/run-build-and-minimal-fuzzers.sh        | 15 +++++++-
 oss-fuzz/.gitignore                        |  3 ++
 oss-fuzz/fuzz-credential-from-url-gently.c | 32 ++++++++++++++++
 oss-fuzz/fuzz-parse-attr-line.c            | 39 ++++++++++++++++++++
 oss-fuzz/fuzz-url-decode-mem.c             | 43 ++++++++++++++++++++++
 8 files changed, 177 insertions(+), 39 deletions(-)
 create mode 100644 oss-fuzz/fuzz-credential-from-url-gently.c
 create mode 100644 oss-fuzz/fuzz-parse-attr-line.c
 create mode 100644 oss-fuzz/fuzz-url-decode-mem.c

base-commit: 159f2d50e75c17382c9f4eb7cbda671a6fa612d1

Josh Steadmon

author

2024-10-14 21:04 UTC #9 lore raw

v2 cover fuzz: port OSS-Fuzz tests back to Git

In reply to Josh Steadmon

Git's fuzz tests are run continuously as part of OSS-Fuzz [1]. Several additional fuzz tests have been contributed directly to OSS-Fuzz; however, these tests are vulnerable to bitrot because they are not built during Git's CI runs, and thus breaking changes are much less likely to be noticed by Git contributors.

OSS-Fuzz's recommended setup is for tests to live in the repository of the code they test and to be built along with other tests [1].

Port some of these tests back to the Git project, so that they can be built and tested during our normal development workflow and CI, and as such avoid future bitrot.

[1] https://google.github.io/oss-fuzz/advanced-topics/ideal-integration/

Changes in V2:
* Remove an errant echo in ci/run-build-and-minimal-fuzzers.sh
* Fix some style nits in ci/run-build-and-minimal-fuzzers.sh
* Fix whitespace errors in attr.{h,c}

Eric Sesterhenn (3):
fuzz: port fuzz-credential-from-url-gently from OSS-Fuzz
fuzz: port fuzz-parse-attr-line from OSS-Fuzz
fuzz: port fuzz-url-decode-mem from OSS-Fuzz

 Makefile                                   |  3 ++
 attr.c                                     | 40 +-------------------
 attr.h                                     | 43 ++++++++++++++++++++++
 ci/run-build-and-minimal-fuzzers.sh        | 13 ++++++-
 oss-fuzz/.gitignore                        |  3 ++
 oss-fuzz/fuzz-credential-from-url-gently.c | 32 ++++++++++++++++
 oss-fuzz/fuzz-parse-attr-line.c            | 39 ++++++++++++++++++++
 oss-fuzz/fuzz-url-decode-mem.c             | 43 ++++++++++++++++++++++
 8 files changed, 177 insertions(+), 39 deletions(-)
 create mode 100644 oss-fuzz/fuzz-credential-from-url-gently.c
 create mode 100644 oss-fuzz/fuzz-parse-attr-line.c
 create mode 100644 oss-fuzz/fuzz-url-decode-mem.c

Range-diff against v1:
1: 625b8d607e ! 1: 04225dd421 fuzz: port fuzz-credential-from-url-gently from OSS-Fuzz

    @@ ci/run-build-and-minimal-fuzzers.sh: group "Build fuzzers" make \
      
     -for fuzzer in commit-graph config date pack-headers pack-idx ; do
     +fuzzers="
    -+commit-graph \
    -+config \
    -+credential-from-url-gently \
    -+date \
    -+pack-headers \
    -+pack-idx \
    ++commit-graph
    ++config
    ++credential-from-url-gently
    ++date
    ++pack-headers
    ++pack-idx
     +"
     +
    -+for fuzzer in $fuzzers ; do
    ++for fuzzer in $fuzzers; do
      	begin_group "fuzz-$fuzzer"
    --	./oss-fuzz/fuzz-$fuzzer -verbosity=0 -runs=1 || exit 1
    -+	echo ./oss-fuzz/fuzz-$fuzzer -verbosity=0 -runs=1 || exit 1
    + 	./oss-fuzz/fuzz-$fuzzer -verbosity=0 -runs=1 || exit 1
      	end_group "fuzz-$fuzzer"
    - done
     
      ## oss-fuzz/.gitignore ##
     @@

2: 770fe27c67 ! 2: 16c6573190 fuzz: port fuzz-parse-attr-line from OSS-Fuzz
@@ attr.c: static const char *parse_attr(const char *src, int lineno, const char *c
}

-static struct match_attr *parse_attr_line(const char *line, const char *src,
+- int lineno, unsigned flags)
+struct match_attr *parse_attr_line(const char *line, const char *src,
- int lineno, unsigned flags)
++ int lineno, unsigned flags)
{
size_t namelen, num_attr, i;
+ const char *cp, *name, *states;

## attr.h ##
@@ attr.h: int git_attr_system_is_enabled(void);
@@ attr.h: int git_attr_system_is_enabled(void);
+};
+
+struct match_attr *parse_attr_line(const char *line, const char *src,
-+ int lineno, unsigned flags);
++ int lineno, unsigned flags);
+
#endif /* ATTR_H */

## ci/run-build-and-minimal-fuzzers.sh ##
-@@ ci/run-build-and-minimal-fuzzers.sh: credential-from-url-gently \
- date \
- pack-headers \
- pack-idx \
-+parse-attr-line \
+@@ ci/run-build-and-minimal-fuzzers.sh: credential-from-url-gently
+ date
+ pack-headers
+ pack-idx
++parse-attr-line
"

- for fuzzer in $fuzzers ; do
+ for fuzzer in $fuzzers; do

## oss-fuzz/.gitignore ##
@@ oss-fuzz/.gitignore: fuzz-credential-from-url-gently
3: 4c9813313c ! 3: ec6b5eed71 fuzz: port fuzz-url-decode-mem from OSS-Fuzz
@@ Makefile: FUZZ_OBJS += oss-fuzz/fuzz-date.o

## ci/run-build-and-minimal-fuzzers.sh ##
-@@ ci/run-build-and-minimal-fuzzers.sh: date \
- pack-headers \
- pack-idx \
- parse-attr-line \
-+url-decode-mem \
+@@ ci/run-build-and-minimal-fuzzers.sh: date
+ pack-headers
+ pack-idx
+ parse-attr-line
++url-decode-mem
"

- for fuzzer in $fuzzers ; do
+ for fuzzer in $fuzzers; do

## oss-fuzz/.gitignore ##
@@ oss-fuzz/.gitignore: fuzz-date

base-commit: 159f2d50e75c17382c9f4eb7cbda671a6fa612d1

Signature · 1 line

2.47.0.rc1.288.g06298d1525-goog

Josh Steadmon

author

2024-10-14 21:04 UTC #10 lore raw

PATCH v2 1/3 fuzz: port fuzz-credential-from-url-gently from OSS-Fuzz

From: Eric Sesterhenn <[email protected]>

Git's fuzz tests are run continuously as part of OSS-Fuzz [1]. Several additional fuzz tests have been contributed directly to OSS-Fuzz; however, these tests are vulnerable to bitrot because they are not built during Git's CI runs, and thus breaking changes are much less likely to be noticed by Git contributors.

Port one of these tests back to the Git project:
fuzz-credential-from-url-gently

This test was originally written by Eric Sesterhenn as part of a security audit of Git [2]. It was then contributed to the OSS-Fuzz repo in commit c58ac4492 (Git fuzzing: uncomment the existing and add new targets. (#11486), 2024-02-21) by Jaroslav Lobačevski. I (Josh Steadmon) have verified with both Eric and Jaroslav that they're OK with moving this test to the Git project.

[1] https://github.com/google/oss-fuzz
[2] https://ostif.org/wp-content/uploads/2023/01/X41-OSTIF-Gitlab-Git-Security-Audit-20230117-public.pdf

Co-authored-by Jaroslav Lobačevski <[email protected]>

Co-authored-by Josh Steadmon <[email protected]>

Signed-off-by Josh Steadmon <[email protected]>

---
 Makefile                                   |  1 +
 ci/run-build-and-minimal-fuzzers.sh        | 11 +++++++-
 oss-fuzz/.gitignore                        |  1 +
 oss-fuzz/fuzz-credential-from-url-gently.c | 32 ++++++++++++++++++++++
 4 files changed, 44 insertions(+), 1 deletion(-)
 create mode 100644 oss-fuzz/fuzz-credential-from-url-gently.c

Makefile +1 -0

index e298c8b55e..3ce391062f 100644
@@ -2378,6 +2378,7 @@ endif
 FUZZ_OBJS += oss-fuzz/dummy-cmd-main.o
 FUZZ_OBJS += oss-fuzz/fuzz-commit-graph.o
 FUZZ_OBJS += oss-fuzz/fuzz-config.o
+FUZZ_OBJS += oss-fuzz/fuzz-credential-from-url-gently.o
 FUZZ_OBJS += oss-fuzz/fuzz-date.o
 FUZZ_OBJS += oss-fuzz/fuzz-pack-headers.o
 FUZZ_OBJS += oss-fuzz/fuzz-pack-idx.o

ci/run-build-and-minimal-fuzzers.sh +10 -1

index af8065f349..631796ab8b 100755
@@ -13,7 +13,16 @@ group "Build fuzzers" make \
 	LIB_FUZZING_ENGINE="-fsanitize=fuzzer,address" \
 	fuzz-all
 
-for fuzzer in commit-graph config date pack-headers pack-idx ; do
+fuzzers="
+commit-graph
+config
+credential-from-url-gently
+date
+pack-headers
+pack-idx
+"
+
+for fuzzer in $fuzzers; do
 	begin_group "fuzz-$fuzzer"
 	./oss-fuzz/fuzz-$fuzzer -verbosity=0 -runs=1 || exit 1
 	end_group "fuzz-$fuzzer"

oss-fuzz/.gitignore +1 -0

index a877c11f42..2cfc845b20 100644
@@ -1,5 +1,6 @@
 fuzz-commit-graph
 fuzz-config
+fuzz-credential-from-url-gently
 fuzz-date
 fuzz-pack-headers
 fuzz-pack-idx

oss-fuzz/fuzz-credential-from-url-gently.c new file +32 -0

new file mode 100644
index 0000000000..c872f9ad2d
@@ -0,0 +1,32 @@
+#include "git-compat-util.h"
+#include <stddef.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <string.h>
+#include <stdio.h>
+#include "credential.h"
+
+int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);
+
+int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
+{
+	struct credential c;
+	char *buf;
+
+	buf = malloc(size + 1);
+	if (!buf)
+		return 0;
+
+	memcpy(buf, data, size);
+	buf[size] = 0;
+
+	// start fuzzing
+	credential_init(&c);
+	credential_from_url_gently(&c, buf, 1);
+
+	// cleanup
+	credential_clear(&c);
+	free(buf);
+
+	return 0;
+}

Signature · 1 line

2.47.0.rc1.288.g06298d1525-goog

Josh Steadmon

author

2024-10-14 21:04 UTC #11 lore raw

PATCH v2 2/3 fuzz: port fuzz-parse-attr-line from OSS-Fuzz

In reply to Josh Steadmon

From: Eric Sesterhenn <[email protected]>

Git's fuzz tests are run continuously as part of OSS-Fuzz [1]. Several additional fuzz tests have been contributed directly to OSS-Fuzz; however, these tests are vulnerable to bitrot because they are not built during Git's CI runs, and thus breaking changes are much less likely to be noticed by Git contributors.

Port one of these tests back to the Git project:
fuzz-parse-attr-line

This test was originally written by Eric Sesterhenn as part of a security audit of Git [2]. It was then contributed to the OSS-Fuzz repo in commit c58ac4492 (Git fuzzing: uncomment the existing and add new targets. (#11486), 2024-02-21) by Jaroslav Lobačevski. I (Josh Steadmon) have verified with both Eric and Jaroslav that they're OK with moving this test to the Git project.

[1] https://github.com/google/oss-fuzz
[2] https://ostif.org/wp-content/uploads/2023/01/X41-OSTIF-Gitlab-Git-Security-Audit-20230117-public.pdf

Co-authored-by Jaroslav Lobačevski <[email protected]>

Co-authored-by Josh Steadmon <[email protected]>

Signed-off-by Josh Steadmon <[email protected]>

---
 Makefile                            |  1 +
 attr.c                              | 40 ++-------------------------
 attr.h                              | 43 +++++++++++++++++++++++++++++
 ci/run-build-and-minimal-fuzzers.sh |  1 +
 oss-fuzz/.gitignore                 |  1 +
 oss-fuzz/fuzz-parse-attr-line.c     | 39 ++++++++++++++++++++++++++
 6 files changed, 87 insertions(+), 38 deletions(-)
 create mode 100644 oss-fuzz/fuzz-parse-attr-line.c

Makefile +1 -0

index 3ce391062f..141e194bf5 100644
@@ -2382,6 +2382,7 @@ FUZZ_OBJS += oss-fuzz/fuzz-credential-from-url-gently.o
 FUZZ_OBJS += oss-fuzz/fuzz-date.o
 FUZZ_OBJS += oss-fuzz/fuzz-pack-headers.o
 FUZZ_OBJS += oss-fuzz/fuzz-pack-idx.o
+FUZZ_OBJS += oss-fuzz/fuzz-parse-attr-line.o
 .PHONY: fuzz-objs
 fuzz-objs: $(FUZZ_OBJS)

attr.c +2 ~~-38~~

index 06b5b5e55e..b0073c5277 100644
@@ -259,42 +259,6 @@ const struct git_attr *git_attr(const char *name)
 	return git_attr_internal(name, strlen(name));
 }
 
-/* What does a matched pattern decide? */
-struct attr_state {
-	const struct git_attr *attr;
-	const char *setto;
-};
-
-struct pattern {
-	const char *pattern;
-	int patternlen;
-	int nowildcardlen;
-	unsigned flags;		/* PATTERN_FLAG_* */
-};
-
-/*
- * One rule, as from a .gitattributes file.
- *
- * If is_macro is true, then u.attr is a pointer to the git_attr being
- * defined.
- *
- * If is_macro is false, then u.pat is the filename pattern to which the
- * rule applies.
- *
- * In either case, num_attr is the number of attributes affected by
- * this rule, and state is an array listing them.  The attributes are
- * listed as they appear in the file (macros unexpanded).
- */
-struct match_attr {
-	union {
-		struct pattern pat;
-		const struct git_attr *attr;
-	} u;
-	char is_macro;
-	size_t num_attr;
-	struct attr_state state[FLEX_ARRAY];
-};
-
 static const char blank[] = " \t\r\n";
 
 /* Flags usable in read_attr() and parse_attr_line() family of functions. */

@@ -353,8 +317,8 @@ static const char *parse_attr(const char *src, int lineno, const char *cp,
 	return ep + strspn(ep, blank);
 }
 
-static struct match_attr *parse_attr_line(const char *line, const char *src,
-					  int lineno, unsigned flags)
+struct match_attr *parse_attr_line(const char *line, const char *src,
+				   int lineno, unsigned flags)
 {
 	size_t namelen, num_attr, i;
 	const char *cp, *name, *states;

attr.h +43 -0

47 lines not on this page · raw message

ci/run-build-and-minimal-fuzzers.sh +1 -0

index 631796ab8b..13c85320d6 100755
@@ -20,6 +20,7 @@ credential-from-url-gently
 date
 pack-headers
 pack-idx
+parse-attr-line
 "
 
 for fuzzer in $fuzzers; do

oss-fuzz/.gitignore +1 -0

index 2cfc845b20..ec185f061c 100644
@@ -4,3 +4,4 @@ fuzz-credential-from-url-gently
 fuzz-date
 fuzz-pack-headers
 fuzz-pack-idx
+fuzz-parse-attr-line

oss-fuzz/fuzz-parse-attr-line.c new file +39 -0

new file mode 100644
index 0000000000..45a4c4e53c
@@ -0,0 +1,39 @@
+#include "git-compat-util.h"
+#include <stddef.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <string.h>
+#include "attr.h"
+
+int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size);
+
+int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size)
+{
+	struct match_attr *res;
+	char *buf;
+
+	buf = malloc(size + 1);
+	if (!buf)
+		return 0;
+
+	memcpy(buf, data, size);
+	buf[size] = 0;
+
+	res = parse_attr_line(buf, "dummy", 0, 0);
+
+	if (res) {
+		int j;
+		for (j = 0; j < res->num_attr; j++) {
+			const char *setto = res->state[j].setto;
+			if (ATTR_TRUE(setto) || ATTR_FALSE(setto) ||
+				ATTR_UNSET(setto))
+				;
+			else
+				free((char *)setto);
+		}
+		free(res);
+	}
+	free(buf);
+
+	return 0;
+}

Signature · 1 line

2.47.0.rc1.288.g06298d1525-goog

Josh Steadmon

author

2024-10-14 21:04 UTC #12 lore raw

PATCH v2 3/3 fuzz: port fuzz-url-decode-mem from OSS-Fuzz

In reply to Josh Steadmon

From: Eric Sesterhenn <[email protected]>

Git's fuzz tests are run continuously as part of OSS-Fuzz [1]. Several additional fuzz tests have been contributed directly to OSS-Fuzz; however, these tests are vulnerable to bitrot because they are not built during Git's CI runs, and thus breaking changes are much less likely to be noticed by Git contributors.

Port one of these tests back to the Git project:
fuzz-url-decode-mem

This test was originally written by Eric Sesterhenn as part of a security audit of Git [2]. It was then contributed to the OSS-Fuzz repo in commit c58ac4492 (Git fuzzing: uncomment the existing and add new targets. (#11486), 2024-02-21) by Jaroslav Lobačevski. I (Josh Steadmon) have verified with both Eric and Jaroslav that they're OK with moving this test to the Git project.

[1] https://github.com/google/oss-fuzz
[2] https://ostif.org/wp-content/uploads/2023/01/X41-OSTIF-Gitlab-Git-Security-Audit-20230117-public.pdf

Co-authored-by Jaroslav Lobačevski <[email protected]>

Co-authored-by Josh Steadmon <[email protected]>

Signed-off-by Josh Steadmon <[email protected]>

---
 Makefile                            |  1 +
 ci/run-build-and-minimal-fuzzers.sh |  1 +
 oss-fuzz/.gitignore                 |  1 +
 oss-fuzz/fuzz-url-decode-mem.c      | 43 +++++++++++++++++++++++++++++
 4 files changed, 46 insertions(+)
 create mode 100644 oss-fuzz/fuzz-url-decode-mem.c

Makefile +1 -0

index 141e194bf5..2cd7a6e003 100644
@@ -2383,6 +2383,7 @@ FUZZ_OBJS += oss-fuzz/fuzz-date.o
 FUZZ_OBJS += oss-fuzz/fuzz-pack-headers.o
 FUZZ_OBJS += oss-fuzz/fuzz-pack-idx.o
 FUZZ_OBJS += oss-fuzz/fuzz-parse-attr-line.o
+FUZZ_OBJS += oss-fuzz/fuzz-url-decode-mem.o
 .PHONY: fuzz-objs
 fuzz-objs: $(FUZZ_OBJS)

ci/run-build-and-minimal-fuzzers.sh +1 -0

index 13c85320d6..e7b97952e7 100755
@@ -21,6 +21,7 @@ date
 pack-headers
 pack-idx
 parse-attr-line
+url-decode-mem
 "
 
 for fuzzer in $fuzzers; do

oss-fuzz/.gitignore +1 -0

index ec185f061c..f2d74de457 100644
@@ -5,3 +5,4 @@ fuzz-date
 fuzz-pack-headers
 fuzz-pack-idx
 fuzz-parse-attr-line
+fuzz-url-decode-mem

oss-fuzz/fuzz-url-decode-mem.c new file +43 -0

43 lines not on this page · raw message

Signature · 1 line

2.47.0.rc1.288.g06298d1525-goog