line-log: protect inner strbuf from free

---
line-log: protect inner strbuf from free

This fixes a regression introduced in 2.46.0.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1806%2Fderrickstolee%2Fline-log-use-after-free-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1806/derrickstolee/line-log-use-after-free-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1806

 line-log.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)

line-log.c +4 -6

index 67c80b39a0d..29cf66bdd10 100644
@@ -897,13 +897,13 @@ static void print_line(const char *prefix, char first,
 		fputs("\\ No newline at end of file\n", file);
 }
 
-static char *output_prefix(struct diff_options *opt)
+static const char *output_prefix(struct diff_options *opt)
 {
 	if (opt->output_prefix) {
 		struct strbuf *sb = opt->output_prefix(opt, opt->output_prefix_data);
 		return sb->buf;
 	} else {
-		return xstrdup("");
+		return "";
 	}
 }

@@ -916,7 +916,7 @@ static void dump_diff_hacky_one(struct rev_info *rev, struct line_log_data *rang
 	struct diff_ranges *diff = &range->diff;
 
 	struct diff_options *opt = &rev->diffopt;
-	char *prefix = output_prefix(opt);
+	const char *prefix = output_prefix(opt);
 	const char *c_reset = diff_get_color(opt->use_color, DIFF_RESET);
 	const char *c_frag = diff_get_color(opt->use_color, DIFF_FRAGINFO);
 	const char *c_meta = diff_get_color(opt->use_color, DIFF_METAINFO);

@@ -1003,7 +1003,6 @@ static void dump_diff_hacky_one(struct rev_info *rev, struct line_log_data *rang
 out:
 	free(p_ends);
 	free(t_ends);
-	free(prefix);
 }
 
 /*

@@ -1012,10 +1011,9 @@ out:
  */
 static void dump_diff_hacky(struct rev_info *rev, struct line_log_data *range)
 {
-	char *prefix = output_prefix(&rev->diffopt);
+	const char *prefix = output_prefix(&rev->diffopt);
 
 	fprintf(rev->diffopt.file, "%s\n", prefix);
-	free(prefix);
 
 	while (range) {
 		dump_diff_hacky_one(rev, range);

base-commit: e9356ba3ea2a6754281ff7697b3e5a1697b21e24

Signature · 1 line

gitgitgadget

Jeff King

2024-10-02 23:56 UTC #1 lore raw

Quoted 22 lines · Derrick Stolee via GitGitGadget

The output_prefix() method in line-log.c may call a function pointer via the diff_options struct. This function pointer returns a strbuf struct and then its buffer is passed back. However, that implies that the consumer is responsible to free the string. This is especially true because the default behavior is to duplicate the empty string.
The existing functions used in the output_prefix pointer include:
1. idiff_prefix_cb() in diff-lib.c. This returns the data pointer, so the value exists across multiple calls.
2. diff_output_prefix_callback() in graph.c. This uses a static strbuf struct, so it reuses buffers across calls. These should not be freed.
3. output_prefix_cb() in range-diff.c. This is similar to the diff-lib.c case.
In each case, we should not be freeing this buffer. We can convert the output_prefix() function to return a const char pointer and stop freeing the result.

Quoted 5 lines

This choice is essentially the opposite of what was done in 394affd46d (line-log: always allocate the output prefix, 2024-06-07).
This was discovered via 'valgrind' while investigating a public report of a bug in 'git log --graph -L' [1].

Good catch. Your analysis looks correct, though I had two questions I managed to answer myself:

1. This seems like an obvious use-after-free problem. Why didn't we catch it sooner? I think the answer is that most uses of the output_prefix callback happen via diff_line_prefix(), which was not broken by 394affd46d. It's only line-log that was affected.

Building with ASan and running:

./git log --graph -L :diff_line_prefix:diff.c

shows the problem immediately (and that your patch fixes it). Should we include a new test in this patch?

2. If the caller isn't freeing it, then who does? Should diffopt cleanup do so? The answer is "no", the pointer is not owned by that struct. In your cases (1) and (3), the caller does so after finishing with the diffopt struct. In case (2) it is effectively "leaked", but still reachable by the static strbuf. That's not great, and is a problem for eventual libification. But for now, we can ignore it as it won't trigger the leak-checker.

It does make me wonder what leak Patrick saw that caused him to write 394affd46d, and whether we're now leaking in some case that I'm missing.

I do think this would have been a harder mistake to make if the callback simply returned a "const char *" pointer. We'd lose the ability to show prefixes with embedded NULs, but I'm not sure that's a big deal. It would also help for line-log to use the existing helper rather than inventing its own. So together on top something like this (which I'd probably turn into two patches if this seems to others like it's valuable and not just churn):

 diff-lib.c   |  4 ++--
 diff.c       |  9 +++------
 diff.h       |  2 +-
 graph.c      |  4 ++--
 line-log.c   | 14 ++------------
 log-tree.c   |  7 +++----
 range-diff.c |  4 ++--
 7 files changed, 15 insertions(+), 29 deletions(-)

diff-lib.c +2 -2

index a680768ee7..6b14b95962 100644
@@ -701,7 +701,7 @@ int index_differs_from(struct repository *r,
 	return (has_changes != 0);
 }
 
-static struct strbuf *idiff_prefix_cb(struct diff_options *opt UNUSED, void *data)
+static const char *idiff_prefix_cb(struct diff_options *opt UNUSED, void *data)
 {
 	return data;
 }

@@ -716,7 +716,7 @@ void show_interdiff(const struct object_id *oid1, const struct object_id *oid2,
 	opts.output_format = DIFF_FORMAT_PATCH;
 	opts.output_prefix = idiff_prefix_cb;
 	strbuf_addchars(&prefix, ' ', indent);
-	opts.output_prefix_data = &prefix;
+	opts.output_prefix_data = prefix.buf;
 	diff_setup_done(&opts);
 
 	diff_tree_oid(oid1, oid2, "", &opts);

diff.c +3 -6

index 173cbe2bed..efde65feef 100644
@@ -2317,12 +2317,9 @@ const char *diff_get_color(int diff_use_color, enum color_diff ix)
 
 const char *diff_line_prefix(struct diff_options *opt)
 {
-	struct strbuf *msgbuf;
-	if (!opt->output_prefix)
-		return "";
-
-	msgbuf = opt->output_prefix(opt, opt->output_prefix_data);
-	return msgbuf->buf;
+	return opt->output_prefix ?
+		opt->output_prefix(opt, opt->output_prefix_data) :
+		"";
 }
 
 static unsigned long sane_truncate_line(char *line, unsigned long len)

diff.h +1 -1

index 0cde3b34e2..2a9c9191c1 100644
@@ -94,7 +94,7 @@ typedef void (*add_remove_fn_t)(struct diff_options *options,
 typedef void (*diff_format_fn_t)(struct diff_queue_struct *q,
 		struct diff_options *options, void *data);
 
-typedef struct strbuf *(*diff_prefix_fn_t)(struct diff_options *opt, void *data);
+typedef const char *(*diff_prefix_fn_t)(struct diff_options *opt, void *data);
 
 #define DIFF_FORMAT_RAW		0x0001
 #define DIFF_FORMAT_DIFFSTAT	0x0002

graph.c +2 -2

index 091c14cf4f..ebb7d1e66f 100644
@@ -314,7 +314,7 @@ struct git_graph {
 	unsigned short default_column_color;
 };
 
-static struct strbuf *diff_output_prefix_callback(struct diff_options *opt, void *data)
+static const char *diff_output_prefix_callback(struct diff_options *opt, void *data)
 {
 	struct git_graph *graph = data;
 	static struct strbuf msgbuf = STRBUF_INIT;

@@ -327,7 +327,7 @@ static struct strbuf *diff_output_prefix_callback(struct diff_options *opt, void
 			   opt->line_prefix_length);
 	if (graph)
 		graph_padding_line(graph, &msgbuf);
-	return &msgbuf;
+	return msgbuf.buf;
 }
 
 static const struct diff_options *default_diffopt;

line-log.c +2 ~~-12~~

index 29cf66bdd1..63945c4729 100644
@@ -897,16 +897,6 @@ static void print_line(const char *prefix, char first,
 		fputs("\\ No newline at end of file\n", file);
 }
 
-static const char *output_prefix(struct diff_options *opt)
-{
-	if (opt->output_prefix) {
-		struct strbuf *sb = opt->output_prefix(opt, opt->output_prefix_data);
-		return sb->buf;
-	} else {
-		return "";
-	}
-}
-
 static void dump_diff_hacky_one(struct rev_info *rev, struct line_log_data *range)
 {
 	unsigned int i, j = 0;

@@ -916,7 +906,7 @@ static void dump_diff_hacky_one(struct rev_info *rev, struct line_log_data *rang
 	struct diff_ranges *diff = &range->diff;
 
 	struct diff_options *opt = &rev->diffopt;
-	const char *prefix = output_prefix(opt);
+	const char *prefix = diff_line_prefix(opt);
 	const char *c_reset = diff_get_color(opt->use_color, DIFF_RESET);
 	const char *c_frag = diff_get_color(opt->use_color, DIFF_FRAGINFO);
 	const char *c_meta = diff_get_color(opt->use_color, DIFF_METAINFO);

@@ -1011,7 +1001,7 @@ static void dump_diff_hacky_one(struct rev_info *rev, struct line_log_data *rang
  */
 static void dump_diff_hacky(struct rev_info *rev, struct line_log_data *range)
 {
-	const char *prefix = output_prefix(&rev->diffopt);
+	const char *prefix = diff_line_prefix(&rev->diffopt);
 
 	fprintf(rev->diffopt.file, "%s\n", prefix);

log-tree.c +3 -4

index 3758e0d3b8..1b80abe7d5 100644
@@ -923,10 +923,9 @@ int log_tree_diff_flush(struct rev_info *opt)
 			 */
 			int pch = DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_PATCH;
 			if (opt->diffopt.output_prefix) {
-				struct strbuf *msg = NULL;
-				msg = opt->diffopt.output_prefix(&opt->diffopt,
-					opt->diffopt.output_prefix_data);
-				fwrite(msg->buf, msg->len, 1, opt->diffopt.file);
+				fputs(opt->diffopt.output_prefix(&opt->diffopt,
+								 opt->diffopt.output_prefix_data),
+				      opt->diffopt.file);
 			}
 
 			/*

range-diff.c +2 -2

index bbb0952264..10885ba301 100644
@@ -480,7 +480,7 @@ static void patch_diff(const char *a, const char *b,
 	diff_flush(diffopt);
 }
 
-static struct strbuf *output_prefix_cb(struct diff_options *opt UNUSED, void *data)
+static const char *output_prefix_cb(struct diff_options *opt UNUSED, void *data)
 {
 	return data;
 }

@@ -508,7 +508,7 @@ static void output(struct string_list *a, struct string_list *b,
 	opts.flags.suppress_hunk_header_line_count = 1;
 	opts.output_prefix = output_prefix_cb;
 	strbuf_addstr(&indent, "    ");
-	opts.output_prefix_data = &indent;
+	opts.output_prefix_data = indent.buf;
 	diff_setup_done(&opts);
 
 	/*

Jeff King

2024-10-03 00:19 UTC #2 lore raw

Quoted 8 lines · Jeff King

Good catch. Your analysis looks correct, though I had two questions I managed to answer myself:
1. This seems like an obvious use-after-free problem. Why didn't we catch it sooner? I think the answer is that most uses of the output_prefix callback happen via diff_line_prefix(), which was not broken by 394affd46d. It's only line-log that was affected.

Quoted 6 lines

Building with ASan and running:
./git log --graph -L :diff_line_prefix:diff.c
shows the problem immediately (and that your patch fixes it). Should we include a new test in this patch?

I think this would do it if you'd like to squash it in. Note that showing the actual patch is necessary to trigger the problem (since that's where we free the prefix buf). Unfortunately it makes the expected output a lot uglier. :(

t/t4211-line-log.sh +28 -0

index 02d76dca28..950451cf6a 100755
@@ -337,4 +337,32 @@ test_expect_success 'zero-width regex .* matches any function name' '
 	test_cmp expect actual
 '
 
+test_expect_success 'show line-log with graph' '
+	qz_to_tab_space >expect <<-EOF &&
+	* $head_oid Modify func2() in file.c
+	|Z
+	| diff --git a/file.c b/file.c
+	| --- a/file.c
+	| +++ b/file.c
+	| @@ -6,4 +6,4 @@
+	|  int func2()
+	|  {
+	| -    return F2;
+	| +    return F2 + 2;
+	|  }
+	* $root_oid Add func1() and func2() in file.c
+	ZZ
+	  diff --git a/file.c b/file.c
+	  --- /dev/null
+	  +++ b/file.c
+	  @@ -0,0 +6,4 @@
+	  +int func2()
+	  +{
+	  +    return F2;
+	  +}
+	EOF
+	git log --graph --oneline -L:func2:file.c >actual &&
+	test_cmp expect actual
+'
+
 test_done

-Peff

Derrick Stolee

2024-10-03 02:36 UTC #3 lore raw

Quoted 1 line · Jeff King

On Wed, Oct 02, 2024 at 04:07:04PM +0000, Derrick Stolee via GitGitGadget wrote:

...

Quoted 8 lines

Good catch. Your analysis looks correct, though I had two questions I managed to answer myself:
1. This seems like an obvious use-after-free problem. Why didn't we catch it sooner? I think the answer is that most uses of the output_prefix callback happen via diff_line_prefix(), which was not broken by 394affd46d. It's only line-log that was affected.

Quoted 6 lines

Building with ASan and running:
./git log --graph -L :diff_line_prefix:diff.c
shows the problem immediately (and that your patch fixes it). Should we include a new test in this patch?

Thank you for sending a test in your second reply. I will include it in v2.

Quoted 7 lines

2. If the caller isn't freeing it, then who does? Should diffopt cleanup do so? The answer is "no", the pointer is not owned by that struct. In your cases (1) and (3), the caller does so after finishing with the diffopt struct. In case (2) it is effectively "leaked", but still reachable by the static strbuf. That's not great, and is a problem for eventual libification. But for now, we can ignore it as it won't trigger the leak-checker.

I agree with this, including the potential for cleaning up the static strbuf and using the 'data' parameter like the other functions.

Quoted 3 lines

It does make me wonder what leak Patrick saw that caused him to write 394affd46d, and whether we're now leaking in some case that I'm missing.

Looking at the change, I can only guess that it was the previous use of

	char *prefix = "";

that signaled that an unfreeable string was being assigned to a non-const pointer. This signals that _something_ is wrong with the function, but the way that the buffer is returned by the function pointer is suspicious, too.

Quoted 7 lines

I do think this would have been a harder mistake to make if the callback simply returned a "const char *" pointer. We'd lose the ability to show prefixes with embedded NULs, but I'm not sure that's a big deal. It would also help for line-log to use the existing helper rather than inventing its own. So together on top something like this (which I'd probably turn into two patches if this seems to others like it's valuable and not just churn):

I do agree that changing the return type will make this easier to prevent and the code should be easier to read as well.

Your diffed patch looks pretty good. I made an attempt at guessing where you would have split them (first remove the duplicate method, then change the method prototype and callers).

I even took some time to attempt to remove the static strbuf from diff_output_prefix_callback() in favor of using the 'data' member of the diff_options struct, but it was not incredibly obvious how to communicate ownership of the struct which would need to store both the graph struct and the strbuf. Perhaps this would be good for #leftoverbits.

Thanks,
-Stolee

Jeff King

2024-10-03 06:11 UTC #4 lore raw

Quoted 4 lines · Derrick Stolee

Quoted 3 lines
It does make me wonder what leak Patrick saw that caused him to write 394affd46d, and whether we're now leaking in some case that I'm missing.

Quoted 8 lines

Looking at the change, I can only guess that it was the previous use of
	char *prefix = "";
that signaled that an unfreeable string was being assigned to a non-const pointer. This signals that _something_ is wrong with the function, but the way that the buffer is returned by the function pointer is suspicious, too.

Ah, of course. I saw Patrick's name and just assumed it was part of leak-checking fixes. But of course he also fixed -Wwrite-strings issues.

Quoted 8 lines

Quoted 7 lines
I do think this would have been a harder mistake to make if the callback simply returned a "const char *" pointer. We'd lose the ability to show prefixes with embedded NULs, but I'm not sure that's a big deal. It would also help for line-log to use the existing helper rather than inventing its own. So together on top something like this (which I'd probably turn into two patches if this seems to others like it's valuable and not just churn):

Quoted 6 lines

I do agree that changing the return type will make this easier to prevent and the code should be easier to read as well.
Your diffed patch looks pretty good. I made an attempt at guessing where you would have split them (first remove the duplicate method, then change the method prototype and callers).

Yep, exactly. I actually ended up with a third patch which is a nearby cleanup. I'll hold them back for now, though. Your patch is a regression fix which we should prioritize (though it sounds like it is in 2.46, not the upcoming 2.47?). I'll post my on top as a separate series.

Quoted 5 lines

I even took some time to attempt to remove the static strbuf from diff_output_prefix_callback() in favor of using the 'data' member of the diff_options struct, but it was not incredibly obvious how to communicate ownership of the struct which would need to store both the graph struct and the strbuf. Perhaps this would be good for #leftoverbits.

Yeah, I think probably "struct git_graph" would need to own the buffer, initialize it in graph_init(), and then discard it in graph_clear(). But that gets weird because apparently you can set the callback without a git_graph? Looks like that is triggered by "--line-prefix" without "--graph". Yuck.

But in that case we are just showing the line_prefix string, so we could return that directly. Something like the patch below.

The whole thing feels a bit over-engineered with the callback. The graph code is the only one that needs anything beyond a static string. And the way --line-prefix interacts with it is odd, since some callers override the callback (e.g., "range-diff --line-prefix=foo" is accepted, but doesn't do anything). I don't think there's a bug anybody cares about, but, well...it's not how I would have written it. ;)

graph.c +15 -6

index c046f6285d..bf000fdbe1 100644
@@ -309,21 +309,28 @@ struct git_graph {
 	 * stored as an index into the array column_colors.
 	 */
 	unsigned short default_column_color;
+
+	/*
+	 * Scratch buffer for generating prefixes to be used with
+	 * diff_output_prefix_callback().
+	 */
+	struct strbuf prefix_buf;
 };
 
 static const char *diff_output_prefix_callback(struct diff_options *opt, void *data)
 {
 	struct git_graph *graph = data;
-	static struct strbuf msgbuf = STRBUF_INIT;
 
 	assert(opt);
 
-	strbuf_reset(&msgbuf);
+	if (!graph)
+		return opt->line_prefix;
+
+	strbuf_reset(&graph->prefix_buf);
 	if (opt->line_prefix)
-		strbuf_addstr(&msgbuf, opt->line_prefix);
-	if (graph)
-		graph_padding_line(graph, &msgbuf);
-	return msgbuf.buf;
+		strbuf_addstr(&graph->prefix_buf, opt->line_prefix);
+	graph_padding_line(graph, &graph->prefix_buf);
+	return graph->prefix_buf.buf;
 }
 
 static const struct diff_options *default_diffopt;

@@ -393,6 +400,7 @@ struct git_graph *graph_init(struct rev_info *opt)
 	 * The diff output prefix callback, with this we can make
 	 * all the diff output to align with the graph lines.
 	 */
+	strbuf_init(&graph->prefix_buf, 0);
 	opt->diffopt.output_prefix = diff_output_prefix_callback;
 	opt->diffopt.output_prefix_data = graph;

@@ -408,6 +416,7 @@ void graph_clear(struct git_graph *graph)
 	free(graph->new_columns);
 	free(graph->mapping);
 	free(graph->old_mapping);
+	strbuf_release(&graph->prefix_buf);
 	free(graph);
 }

-Peff

Derrick Stolee

2024-10-03 12:23 UTC #5 lore raw

Quoted 37 lines · Jeff King

Quoted 8 lines · Derrick Stolee
Quoted 7 lines
I do think this would have been a harder mistake to make if the callback simply returned a "const char *" pointer. We'd lose the ability to show prefixes with embedded NULs, but I'm not sure that's a big deal. It would also help for line-log to use the existing helper rather than inventing its own. So together on top something like this (which I'd probably turn into two patches if this seems to others like it's valuable and not just churn):
Quoted 6 lines
I do agree that changing the return type will make this easier to prevent and the code should be easier to read as well.
Your diffed patch looks pretty good. I made an attempt at guessing where you would have split them (first remove the duplicate method, then change the method prototype and callers).
Yep, exactly. I actually ended up with a third patch which is a nearby cleanup. I'll hold them back for now, though. Your patch is a regression fix which we should prioritize (though it sounds like it is in 2.46, not the upcoming 2.47?). I'll post my on top as a separate series.
Quoted 5 lines
I even took some time to attempt to remove the static strbuf from diff_output_prefix_callback() in favor of using the 'data' member of the diff_options struct, but it was not incredibly obvious how to communicate ownership of the struct which would need to store both the graph struct and the strbuf. Perhaps this would be good for #leftoverbits.
Yeah, I think probably "struct git_graph" would need to own the buffer, initialize it in graph_init(), and then discard it in graph_clear(). But that gets weird because apparently you can set the callback without a git_graph? Looks like that is triggered by "--line-prefix" without "--graph". Yuck.
But in that case we are just showing the line_prefix string, so we could return that directly. Something like the patch below.

Quoted 6 lines

The whole thing feels a bit over-engineered with the callback. The graph code is the only one that needs anything beyond a static string. And the way --line-prefix interacts with it is odd, since some callers override the callback (e.g., "range-diff --line-prefix=foo" is accepted, but doesn't do anything). I don't think there's a bug anybody cares about, but, well...it's not how I would have written it. ;)

Thanks for exploring this with the diff you sent. It's subtle, but you did a good job of recognizing that when the callback is used without a 'graph' value, it returns opt->line_prefix.

graph.c struct git_graph { · line 327

Quoted 2 lines

graph.c +2 -0

+	if (!graph)
+		return opt->line_prefix;

I was wondering why there was no need to edit graph_setup_line_prefix() in your diff, and this explains why. Your change is simple enough to handle that change while being robust to a future assignment of the callback data.

For what it's worth, I attempted creating a custom callback with a BUG() statement in it and the only test failure was t4013-diff-various.sh. That test is very non-standard, and it did not recognize the process failure, only the output difference. #leftoverbits could make that test fail on process failure.

Thanks,
-Stolee

Jeff King

2024-10-03 21:03 UTC #6 lore raw

Quoted 11 lines · Derrick Stolee

Quoted 6 lines
The whole thing feels a bit over-engineered with the callback. The graph code is the only one that needs anything beyond a static string. And the way --line-prefix interacts with it is odd, since some callers override the callback (e.g., "range-diff --line-prefix=foo" is accepted, but doesn't do anything). I don't think there's a bug anybody cares about, but, well...it's not how I would have written it. ;)
Thanks for exploring this with the diff you sent. It's subtle, but you did a good job of recognizing that when the callback is used without a 'graph' value, it returns opt->line_prefix.

Quoted 7 lines

graph.c struct git_graph { · line 327
Quoted 2 lines
graph.c +2 -0
+	if (!graph)
+		return opt->line_prefix;
I was wondering why there was no need to edit graph_setup_line_prefix() in your diff, and this explains why. Your change is simple enough to handle that change while being robust to a future assignment of the callback data.

Yeah. I actually ended up pulling that into its own patch because it's sufficiently subtle. I'll send that along in a minute.

-Peff

Derrick Stolee via GitGitGadget

author

2024-10-03 11:58 UTC #7 lore raw

v2 cover line-log: protect inner strbuf from free

[1] https://github.com/git-for-windows/git/issues/5185

This fixes a regression introduced in 2.46.0.

The change made in 394affd46d (line-log: always allocate the output prefix, 2024-06-07) made the method more consistent in that it did not return a static empty string that would fail to free(), but it does lead to double-frees when a strbuf buffer is freed multiple times.

In v2, I add Peff's test to the first patch. I also split his diff into two more follow-up patches because the additional clarity seems valuable to me. I have forged his sign-off in all three patches.

Note to the maintainer: feel free to take only the first patch, as Peff replied that he may work on the remaining cleanup independently (but I had already prepared patches 2 & 3).

Thanks, -Stolee

Derrick Stolee (1):
line-log: protect inner strbuf from free

Jeff King (2):
line-log: remove output_prefix()
diff: modify output_prefix function pointer

 diff-lib.c          |  4 ++--
 diff.c              |  8 +++-----
 diff.h              |  2 +-
 graph.c             |  4 ++--
 line-log.c          | 16 ++--------------
 log-tree.c          |  4 ++--
 range-diff.c        |  4 ++--
 t/t4211-line-log.sh | 28 ++++++++++++++++++++++++++++
 8 files changed, 42 insertions(+), 28 deletions(-)

base-commit: e9356ba3ea2a6754281ff7697b3e5a1697b21e24
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1806%2Fderrickstolee%2Fline-log-use-after-free-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1806/derrickstolee/line-log-use-after-free-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/1806

Range-diff vs v1:

1: 5d341e42d83 ! 1: 05c21616c35 line-log: protect inner strbuf from free

     @@ Commit message
      
          [1] https://github.com/git-for-windows/git/issues/5185
      
     +    This issue would have been caught by the new test, when Git is compiled
     +    with ASan to catch these double frees.
     +
     +    Co-authored-by: Jeff King <[email protected]>
     +    Signed-off-by: Jeff King <[email protected]>
          Signed-off-by: Derrick Stolee <[email protected]>
      
       ## line-log.c ##
     @@ line-log.c: out:
       
       	while (range) {
       		dump_diff_hacky_one(rev, range);
     +
     + ## t/t4211-line-log.sh ##
     +@@ t/t4211-line-log.sh: test_expect_success 'zero-width regex .* matches any function name' '
     + 	test_cmp expect actual
     + '
     + 
     ++test_expect_success 'show line-log with graph' '
     ++	qz_to_tab_space >expect <<-EOF &&
     ++	* $head_oid Modify func2() in file.c
     ++	|Z
     ++	| diff --git a/file.c b/file.c
     ++	| --- a/file.c
     ++	| +++ b/file.c
     ++	| @@ -6,4 +6,4 @@
     ++	|  int func2()
     ++	|  {
     ++	| -    return F2;
     ++	| +    return F2 + 2;
     ++	|  }
     ++	* $root_oid Add func1() and func2() in file.c
     ++	ZZ
     ++	  diff --git a/file.c b/file.c
     ++	  --- /dev/null
     ++	  +++ b/file.c
     ++	  @@ -0,0 +6,4 @@
     ++	  +int func2()
     ++	  +{
     ++	  +    return F2;
     ++	  +}
     ++	EOF
     ++	git log --graph --oneline -L:func2:file.c >actual &&
     ++	test_cmp expect actual
     ++'
     ++
     + test_done
 -:  ----------- > 2:  94d2c034b4a line-log: remove output_prefix()
 -:  ----------- > 3:  e1d825ad212 diff: modify output_prefix function pointer

Signature · 1 line

gitgitgadget

Derrick Stolee via GitGitGadget

author

2024-10-03 11:58 UTC #8 lore raw

PATCH v2 1/3 line-log: protect inner strbuf from free

From: Derrick Stolee <[email protected]>

The existing functions used in the output_prefix pointer include:

1. idiff_prefix_cb() in diff-lib.c. This returns the data pointer, so the value exists across multiple calls.

2. diff_output_prefix_callback() in graph.c. This uses a static strbuf struct, so it reuses buffers across calls. These should not be freed.

3. output_prefix_cb() in range-diff.c. This is similar to the diff-lib.c case.

In each case, we should not be freeing this buffer. We can convert the output_prefix() function to return a const char pointer and stop freeing the result.

This choice is essentially the opposite of what was done in 394affd46d (line-log: always allocate the output prefix, 2024-06-07).

This was discovered via 'valgrind' while investigating a public report of a bug in 'git log --graph -L' [1].

This issue would have been caught by the new test, when Git is compiled with ASan to catch these double frees.

Co-authored-by Jeff King <[email protected]>

Signed-off-by Jeff King <[email protected]>

---
 line-log.c          | 10 ++++------
 t/t4211-line-log.sh | 28 ++++++++++++++++++++++++++++
 2 files changed, 32 insertions(+), 6 deletions(-)

line-log.c +4 -6

index 67c80b39a0d..29cf66bdd10 100644
@@ -897,13 +897,13 @@ static void print_line(const char *prefix, char first,
 		fputs("\\ No newline at end of file\n", file);
 }
 
-static char *output_prefix(struct diff_options *opt)
+static const char *output_prefix(struct diff_options *opt)
 {
 	if (opt->output_prefix) {
 		struct strbuf *sb = opt->output_prefix(opt, opt->output_prefix_data);
 		return sb->buf;
 	} else {
-		return xstrdup("");
+		return "";
 	}
 }

@@ -916,7 +916,7 @@ static void dump_diff_hacky_one(struct rev_info *rev, struct line_log_data *rang
 	struct diff_ranges *diff = &range->diff;
 
 	struct diff_options *opt = &rev->diffopt;
-	char *prefix = output_prefix(opt);
+	const char *prefix = output_prefix(opt);
 	const char *c_reset = diff_get_color(opt->use_color, DIFF_RESET);
 	const char *c_frag = diff_get_color(opt->use_color, DIFF_FRAGINFO);
 	const char *c_meta = diff_get_color(opt->use_color, DIFF_METAINFO);

@@ -1003,7 +1003,6 @@ static void dump_diff_hacky_one(struct rev_info *rev, struct line_log_data *rang
 out:
 	free(p_ends);
 	free(t_ends);
-	free(prefix);
 }
 
 /*

@@ -1012,10 +1011,9 @@ out:
  */
 static void dump_diff_hacky(struct rev_info *rev, struct line_log_data *range)
 {
-	char *prefix = output_prefix(&rev->diffopt);
+	const char *prefix = output_prefix(&rev->diffopt);
 
 	fprintf(rev->diffopt.file, "%s\n", prefix);
-	free(prefix);
 
 	while (range) {
 		dump_diff_hacky_one(rev, range);

t/t4211-line-log.sh +28 -0

index 02d76dca284..950451cf6a6 100755
@@ -337,4 +337,32 @@ test_expect_success 'zero-width regex .* matches any function name' '
 	test_cmp expect actual
 '
 
+test_expect_success 'show line-log with graph' '
+	qz_to_tab_space >expect <<-EOF &&
+	* $head_oid Modify func2() in file.c
+	|Z
+	| diff --git a/file.c b/file.c
+	| --- a/file.c
+	| +++ b/file.c
+	| @@ -6,4 +6,4 @@
+	|  int func2()
+	|  {
+	| -    return F2;
+	| +    return F2 + 2;
+	|  }
+	* $root_oid Add func1() and func2() in file.c
+	ZZ
+	  diff --git a/file.c b/file.c
+	  --- /dev/null
+	  +++ b/file.c
+	  @@ -0,0 +6,4 @@
+	  +int func2()
+	  +{
+	  +    return F2;
+	  +}
+	EOF
+	git log --graph --oneline -L:func2:file.c >actual &&
+	test_cmp expect actual
+'
+
 test_done

Signature · 1 line

gitgitgadget

Patrick Steinhardt

2024-10-04 04:32 UTC #9 lore raw

Quoted 27 lines · Derrick Stolee via GitGitGadget

From: Derrick Stolee <[email protected]>
The output_prefix() method in line-log.c may call a function pointer via the diff_options struct. This function pointer returns a strbuf struct and then its buffer is passed back. However, that implies that the consumer is responsible to free the string. This is especially true because the default behavior is to duplicate the empty string.
The existing functions used in the output_prefix pointer include:
1. idiff_prefix_cb() in diff-lib.c. This returns the data pointer, so the value exists across multiple calls.
2. diff_output_prefix_callback() in graph.c. This uses a static strbuf struct, so it reuses buffers across calls. These should not be freed.
3. output_prefix_cb() in range-diff.c. This is similar to the diff-lib.c case.
In each case, we should not be freeing this buffer. We can convert the output_prefix() function to return a const char pointer and stop freeing the result.
This choice is essentially the opposite of what was done in 394affd46d (line-log: always allocate the output prefix, 2024-06-07).

Quoted 7 lines

This was discovered via 'valgrind' while investigating a public report of a bug in 'git log --graph -L' [1].
[1] https://github.com/git-for-windows/git/issues/5185
This issue would have been caught by the new test, when Git is compiled with ASan to catch these double frees.

Thanks a bunch for fixing this! The change looks good to me.

Patrick

Jeff King via GitGitGadget

author

2024-10-03 11:58 UTC #10 lore raw

PATCH v2 2/3 line-log: remove output_prefix()

From: Jeff King <[email protected]>

Now that output_prefix() returns a const char * type, it matches the behavior of diff_line_prefix() and no longer needs to exist on its own.

Signed-off-by Jeff King <[email protected]>

---
 line-log.c | 14 ++------------
 1 file changed, 2 insertions(+), 12 deletions(-)

line-log.c +2 ~~-12~~

index 29cf66bdd10..63945c47299 100644
@@ -897,16 +897,6 @@ static void print_line(const char *prefix, char first,
 		fputs("\\ No newline at end of file\n", file);
 }
 
-static const char *output_prefix(struct diff_options *opt)
-{
-	if (opt->output_prefix) {
-		struct strbuf *sb = opt->output_prefix(opt, opt->output_prefix_data);
-		return sb->buf;
-	} else {
-		return "";
-	}
-}
-
 static void dump_diff_hacky_one(struct rev_info *rev, struct line_log_data *range)
 {
 	unsigned int i, j = 0;

@@ -916,7 +906,7 @@ static void dump_diff_hacky_one(struct rev_info *rev, struct line_log_data *rang
 	struct diff_ranges *diff = &range->diff;
 
 	struct diff_options *opt = &rev->diffopt;
-	const char *prefix = output_prefix(opt);
+	const char *prefix = diff_line_prefix(opt);
 	const char *c_reset = diff_get_color(opt->use_color, DIFF_RESET);
 	const char *c_frag = diff_get_color(opt->use_color, DIFF_FRAGINFO);
 	const char *c_meta = diff_get_color(opt->use_color, DIFF_METAINFO);

@@ -1011,7 +1001,7 @@ out:
  */
 static void dump_diff_hacky(struct rev_info *rev, struct line_log_data *range)
 {
-	const char *prefix = output_prefix(&rev->diffopt);
+	const char *prefix = diff_line_prefix(&rev->diffopt);
 
 	fprintf(rev->diffopt.file, "%s\n", prefix);

Signature · 1 line

gitgitgadget

Jeff King via GitGitGadget

author

2024-10-03 11:58 UTC #11 lore raw

PATCH v2 3/3 diff: modify output_prefix function pointer

From: Jeff King <[email protected]>

The uses of the output_prefix function pointer in the diff_options struct is currently difficult to work with by returning a pointer to a strbuf. There is only one use that cares about the length of the string, which appears to be the only justification of the return type.

We already noticed confusing memory issues around this return type, so use a const char * return type to make it clear that the caller does not own this string buffer.

Signed-off-by Jeff King <[email protected]>

---
 diff-lib.c   | 4 ++--
 diff.c       | 8 +++-----
 diff.h       | 2 +-
 graph.c      | 4 ++--
 log-tree.c   | 4 ++--
 range-diff.c | 4 ++--
 6 files changed, 12 insertions(+), 14 deletions(-)

diff-lib.c +2 -2

index a680768ee75..6b14b959629 100644
@@ -701,7 +701,7 @@ int index_differs_from(struct repository *r,
 	return (has_changes != 0);
 }
 
-static struct strbuf *idiff_prefix_cb(struct diff_options *opt UNUSED, void *data)
+static const char *idiff_prefix_cb(struct diff_options *opt UNUSED, void *data)
 {
 	return data;
 }

@@ -716,7 +716,7 @@ void show_interdiff(const struct object_id *oid1, const struct object_id *oid2,
 	opts.output_format = DIFF_FORMAT_PATCH;
 	opts.output_prefix = idiff_prefix_cb;
 	strbuf_addchars(&prefix, ' ', indent);
-	opts.output_prefix_data = &prefix;
+	opts.output_prefix_data = prefix.buf;
 	diff_setup_done(&opts);
 
 	diff_tree_oid(oid1, oid2, "", &opts);

diff.c +3 -5

index 84a6bb08681..82a9545afbd 100644
@@ -2315,12 +2315,10 @@ const char *diff_get_color(int diff_use_color, enum color_diff ix)
 
 const char *diff_line_prefix(struct diff_options *opt)
 {
-	struct strbuf *msgbuf;
-	if (!opt->output_prefix)
-		return "";
+	if (opt->output_prefix)
+		return opt->output_prefix(opt, opt->output_prefix_data);
 
-	msgbuf = opt->output_prefix(opt, opt->output_prefix_data);
-	return msgbuf->buf;
+	return "";
 }
 
 static unsigned long sane_truncate_line(char *line, unsigned long len)

diff.h +1 -1

index fb40c6e6d60..978450e20ac 100644
@@ -94,7 +94,7 @@ typedef void (*add_remove_fn_t)(struct diff_options *options,
 typedef void (*diff_format_fn_t)(struct diff_queue_struct *q,
 		struct diff_options *options, void *data);
 
-typedef struct strbuf *(*diff_prefix_fn_t)(struct diff_options *opt, void *data);
+typedef const char *(*diff_prefix_fn_t)(struct diff_options *opt, void *data);
 
 #define DIFF_FORMAT_RAW		0x0001
 #define DIFF_FORMAT_DIFFSTAT	0x0002

graph.c +2 -2

index 091c14cf4fb..ebb7d1e66f4 100644
@@ -314,7 +314,7 @@ struct git_graph {
 	unsigned short default_column_color;
 };
 
-static struct strbuf *diff_output_prefix_callback(struct diff_options *opt, void *data)
+static const char *diff_output_prefix_callback(struct diff_options *opt, void *data)
 {
 	struct git_graph *graph = data;
 	static struct strbuf msgbuf = STRBUF_INIT;

@@ -327,7 +327,7 @@ static struct strbuf *diff_output_prefix_callback(struct diff_options *opt, void
 			   opt->line_prefix_length);
 	if (graph)
 		graph_padding_line(graph, &msgbuf);
-	return &msgbuf;
+	return msgbuf.buf;
 }
 
 static const struct diff_options *default_diffopt;

log-tree.c +2 -2

index 3758e0d3b8e..3af34b91a59 100644
@@ -923,10 +923,10 @@ int log_tree_diff_flush(struct rev_info *opt)
 			 */
 			int pch = DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_PATCH;
 			if (opt->diffopt.output_prefix) {
-				struct strbuf *msg = NULL;
+				const char *msg;
 				msg = opt->diffopt.output_prefix(&opt->diffopt,
 					opt->diffopt.output_prefix_data);
-				fwrite(msg->buf, msg->len, 1, opt->diffopt.file);
+				fwrite(msg, strlen(msg), 1, opt->diffopt.file);
 			}
 
 			/*

range-diff.c +2 -2

index bbb0952264b..10885ba3013 100644
@@ -480,7 +480,7 @@ static void patch_diff(const char *a, const char *b,
 	diff_flush(diffopt);
 }
 
-static struct strbuf *output_prefix_cb(struct diff_options *opt UNUSED, void *data)
+static const char *output_prefix_cb(struct diff_options *opt UNUSED, void *data)
 {
 	return data;
 }

@@ -508,7 +508,7 @@ static void output(struct string_list *a, struct string_list *b,
 	opts.flags.suppress_hunk_header_line_count = 1;
 	opts.output_prefix = output_prefix_cb;
 	strbuf_addstr(&indent, "    ");
-	opts.output_prefix_data = &indent;
+	opts.output_prefix_data = indent.buf;
 	diff_setup_done(&opts);
 
 	/*

Signature · 1 line

gitgitgadget

Junio C Hamano

2024-10-03 16:26 UTC #12 lore raw

Re: [PATCH v2 3/3] diff: modify output_prefix function pointer

Quoted 13 lines · Jeff King via GitGitGadget

From: Jeff King <[email protected]>
The uses of the output_prefix function pointer in the diff_options struct is currently difficult to work with by returning a pointer to a strbuf. There is only one use that cares about the length of the string, which appears to be the only justification of the return type.
We already noticed confusing memory issues around this return type, so use a const char * return type to make it clear that the caller does not own this string buffer.
Signed-off-by Jeff King <[email protected]>
Signed-off-by Derrick Stolee <[email protected]>

Quoted 8 lines

---
 diff-lib.c   | 4 ++--
 diff.c       | 8 +++-----
 diff.h       | 2 +-
 graph.c      | 4 ++--
 log-tree.c   | 4 ++--
 range-diff.c | 4 ++--
 6 files changed, 12 insertions(+), 14 deletions(-)

Very nice.

log-tree.c int log_tree_diff_flush(struct rev_info *opt) · line 930

Quoted 8 lines

log-tree.c +2 -2

 			if (opt->diffopt.output_prefix) {
-				struct strbuf *msg = NULL;
+				const char *msg;
 				msg = opt->diffopt.output_prefix(&opt->diffopt,
 					opt->diffopt.output_prefix_data);
-				fwrite(msg->buf, msg->len, 1, opt->diffopt.file);
+				fwrite(msg, strlen(msg), 1, opt->diffopt.file);
 			}

OK. We are not relying on the strbuf being able to have embedded NUL in the buffer, and this looks very sensible.

Thanks.

Jeff King

2024-10-03 21:05 UTC #13 lore raw

cover diff output_prefix cleanups

Quoted 7 lines · Derrick Stolee via GitGitGadget

This fixes a regression introduced in 2.46.0.
The change made in 394affd46d (line-log: always allocate the output prefix, 2024-06-07) made the method more consistent in that it did not return a static empty string that would fail to free(), but it does lead to double-frees when a strbuf buffer is freed multiple times.

Quoted 7 lines

In v2, I add Peff's test to the first patch. I also split his diff into two more follow-up patches because the additional clarity seems valuable to me. I have forged his sign-off in all three patches.
Note to the maintainer: feel free to take only the first patch, as Peff replied that he may work on the remaining cleanup independently (but I had already prepared patches 2 & 3).

Oh, I wasn't expecting you to go to that trouble, and had already polished them up myself. :)

So certainly your patch 1 looks good to me now. Here's what I would put on top (but I would suggest making it a separate branch, since yours is a fairly urgent fix and mine is all cleanup).

[1/5]: line-log: use diff_line_prefix() instead of custom helper [2/5]: diff: drop line_prefix_length field
[3/5]: diff: return const char from output_prefix callback [4/5]: diff: return line_prefix directly when possible [5/5]: diff: store graph prefix buf in git_graph struct

 diff-lib.c   |  4 ++--
 diff.c       | 10 +++-------
 diff.h       |  3 +--
 graph.c      | 29 +++++++++++++++++------------
 line-log.c   | 14 ++------------
 log-tree.c   |  7 +------
 range-diff.c |  4 ++--
 7 files changed, 28 insertions(+), 43 deletions(-)

-Peff

Jeff King

2024-10-03 21:06 UTC #14 lore raw

PATCH 1/5 line-log: use diff_line_prefix() instead of custom helper

Our local output_prefix() is exactly the same as the public diff_line_prefix() function. Let's just use that one, saving us a little bit of code.

Signed-off-by Jeff King <[email protected]>

---
 line-log.c | 14 ++------------
 1 file changed, 2 insertions(+), 12 deletions(-)

line-log.c +2 ~~-12~~

index 29cf66bdd1..63945c4729 100644
@@ -897,16 +897,6 @@ static void print_line(const char *prefix, char first,
 		fputs("\\ No newline at end of file\n", file);
 }
 
-static const char *output_prefix(struct diff_options *opt)
-{
-	if (opt->output_prefix) {
-		struct strbuf *sb = opt->output_prefix(opt, opt->output_prefix_data);
-		return sb->buf;
-	} else {
-		return "";
-	}
-}
-
 static void dump_diff_hacky_one(struct rev_info *rev, struct line_log_data *range)
 {
 	unsigned int i, j = 0;

@@ -916,7 +906,7 @@ static void dump_diff_hacky_one(struct rev_info *rev, struct line_log_data *rang
 	struct diff_ranges *diff = &range->diff;
 
 	struct diff_options *opt = &rev->diffopt;
-	const char *prefix = output_prefix(opt);
+	const char *prefix = diff_line_prefix(opt);
 	const char *c_reset = diff_get_color(opt->use_color, DIFF_RESET);
 	const char *c_frag = diff_get_color(opt->use_color, DIFF_FRAGINFO);
 	const char *c_meta = diff_get_color(opt->use_color, DIFF_METAINFO);

@@ -1011,7 +1001,7 @@ static void dump_diff_hacky_one(struct rev_info *rev, struct line_log_data *rang
  */
 static void dump_diff_hacky(struct rev_info *rev, struct line_log_data *range)
 {
-	const char *prefix = output_prefix(&rev->diffopt);
+	const char *prefix = diff_line_prefix(&rev->diffopt);
 
 	fprintf(rev->diffopt.file, "%s\n", prefix);

Signature · 1 line

2.47.0.rc1.384.g9f398d04fd

Jeff King

2024-10-03 21:06 UTC #15 lore raw

PATCH 2/5 diff: drop line_prefix_length field

The diff_options structure holds a line_prefix string and an associated length. But the length is always just the strlen() of the NUL-terminated string. Let's simplify the code by just storing the string pointer and assuming it is NUL-terminated when we use it.

This will cause us to compute the string length in a few extra spots, but I don't think any of these are particularly hot code paths.

Signed-off-by Jeff King <[email protected]>

---
 diff.c  | 1 -
 diff.h  | 1 -
 graph.c | 8 ++------
 3 files changed, 2 insertions(+), 8 deletions(-)

diff.c +0 -1

index 173cbe2bed..858e001993 100644
@@ -5400,7 +5400,6 @@ static int diff_opt_line_prefix(const struct option *opt,
 
 	BUG_ON_OPT_NEG(unset);
 	options->line_prefix = optarg;
-	options->line_prefix_length = strlen(options->line_prefix);
 	graph_setup_line_prefix(options);
 	return 0;
 }

diff.h +0 -1

index 0cde3b34e2..ea3634106d 100644
@@ -274,7 +274,6 @@ struct diff_options {
 	const char *single_follow;
 	const char *a_prefix, *b_prefix;
 	const char *line_prefix;
-	size_t line_prefix_length;
 
 	/**
 	 * collection of boolean options that affects the operation, but some do

graph.c +2 -6

index 091c14cf4f..2cee47294f 100644
@@ -76,10 +76,7 @@ static void graph_show_line_prefix(const struct diff_options *diffopt)
 	if (!diffopt || !diffopt->line_prefix)
 		return;
 
-	fwrite(diffopt->line_prefix,
-	       sizeof(char),
-	       diffopt->line_prefix_length,
-	       diffopt->file);
+	fputs(diffopt->line_prefix, diffopt->file);
 }
 
 static const char **column_colors;

@@ -323,8 +320,7 @@ static struct strbuf *diff_output_prefix_callback(struct diff_options *opt, void
 
 	strbuf_reset(&msgbuf);
 	if (opt->line_prefix)
-		strbuf_add(&msgbuf, opt->line_prefix,
-			   opt->line_prefix_length);
+		strbuf_addstr(&msgbuf, opt->line_prefix);
 	if (graph)
 		graph_padding_line(graph, &msgbuf);
 	return &msgbuf;

Signature · 1 line

2.47.0.rc1.384.g9f398d04fd

Jeff King

2024-10-03 21:09 UTC #16 lore raw

PATCH 3/5 diff: return const char from output_prefix callback

The diff_options structure has an output_prefix callback for returning a prefix string, but it does so by returning a pointer to a strbuf.

This makes the interface awkward. There's no reason the callback should need to use a strbuf, and it creates questions about whether the ownership of the resulting buffer should be transferred to the caller (it should not be, but a recent attempt to clean up this code led to a double-free in some cases).

The one advantage we get is that the strbuf contains a ptr/len pair, so we could in theory have a prefix with embedded NULs. But we can observe that none of the existing callbacks would ever produce such a NUL (they are usually just indentation or graph symbols, and even the "--line-prefix" option takes a NUL-terminated string).

And anyway, only one caller (the one in log_tree_diff_flush) actually looks at the strbuf length. In every other case we use a helper function which discards the length and just returns the NUL-terminated string.

So let's just have the callback return a "const char *" pointer. It's up to the callbacks themselves if they want to use a strbuf under the hood. And now the caller in log_tree_diff_flush() can just use the helper function along with everybody else. That lets us even simplify out the function pointer check, since the helper returns an empty string (technically this does mean we'll sometimes issue an empty fputs() call, but I don't think this code path is hot enough to care about that).

Signed-off-by Jeff King <[email protected]>

---
 diff-lib.c   | 4 ++--
 diff.c       | 9 +++------
 diff.h       | 2 +-
 graph.c      | 4 ++--
 log-tree.c   | 7 +------
 range-diff.c | 4 ++--
 6 files changed, 11 insertions(+), 19 deletions(-)

diff-lib.c +2 -2

index a680768ee7..6b14b95962 100644
@@ -701,7 +701,7 @@ int index_differs_from(struct repository *r,
 	return (has_changes != 0);
 }
 
-static struct strbuf *idiff_prefix_cb(struct diff_options *opt UNUSED, void *data)
+static const char *idiff_prefix_cb(struct diff_options *opt UNUSED, void *data)
 {
 	return data;
 }

@@ -716,7 +716,7 @@ void show_interdiff(const struct object_id *oid1, const struct object_id *oid2,
 	opts.output_format = DIFF_FORMAT_PATCH;
 	opts.output_prefix = idiff_prefix_cb;
 	strbuf_addchars(&prefix, ' ', indent);
-	opts.output_prefix_data = &prefix;
+	opts.output_prefix_data = prefix.buf;
 	diff_setup_done(&opts);
 
 	diff_tree_oid(oid1, oid2, "", &opts);

diff.c +3 -6

index 858e001993..6c96154fed 100644
@@ -2317,12 +2317,9 @@ const char *diff_get_color(int diff_use_color, enum color_diff ix)
 
 const char *diff_line_prefix(struct diff_options *opt)
 {
-	struct strbuf *msgbuf;
-	if (!opt->output_prefix)
-		return "";
-
-	msgbuf = opt->output_prefix(opt, opt->output_prefix_data);
-	return msgbuf->buf;
+	return opt->output_prefix ?
+		opt->output_prefix(opt, opt->output_prefix_data) :
+		"";
 }
 
 static unsigned long sane_truncate_line(char *line, unsigned long len)

diff.h +1 -1

index ea3634106d..5c8de79535 100644
@@ -94,7 +94,7 @@ typedef void (*add_remove_fn_t)(struct diff_options *options,
 typedef void (*diff_format_fn_t)(struct diff_queue_struct *q,
 		struct diff_options *options, void *data);
 
-typedef struct strbuf *(*diff_prefix_fn_t)(struct diff_options *opt, void *data);
+typedef const char *(*diff_prefix_fn_t)(struct diff_options *opt, void *data);
 
 #define DIFF_FORMAT_RAW		0x0001
 #define DIFF_FORMAT_DIFFSTAT	0x0002

graph.c +2 -2

index 2cee47294f..c046f6285d 100644
@@ -311,7 +311,7 @@ struct git_graph {
 	unsigned short default_column_color;
 };
 
-static struct strbuf *diff_output_prefix_callback(struct diff_options *opt, void *data)
+static const char *diff_output_prefix_callback(struct diff_options *opt, void *data)
 {
 	struct git_graph *graph = data;
 	static struct strbuf msgbuf = STRBUF_INIT;

@@ -323,7 +323,7 @@ static struct strbuf *diff_output_prefix_callback(struct diff_options *opt, void
 		strbuf_addstr(&msgbuf, opt->line_prefix);
 	if (graph)
 		graph_padding_line(graph, &msgbuf);
-	return &msgbuf;
+	return msgbuf.buf;
 }
 
 static const struct diff_options *default_diffopt;

log-tree.c +1 -6

index 3758e0d3b8..33eb96b50c 100644
@@ -922,12 +922,7 @@ int log_tree_diff_flush(struct rev_info *opt)
 			 * diff/diffstat output for readability.
 			 */
 			int pch = DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_PATCH;
-			if (opt->diffopt.output_prefix) {
-				struct strbuf *msg = NULL;
-				msg = opt->diffopt.output_prefix(&opt->diffopt,
-					opt->diffopt.output_prefix_data);
-				fwrite(msg->buf, msg->len, 1, opt->diffopt.file);
-			}
+			fputs(diff_line_prefix(&opt->diffopt), opt->diffopt.file);
 
 			/*
 			 * We may have shown three-dashes line early

range-diff.c +2 -2

index bbb0952264..10885ba301 100644
@@ -480,7 +480,7 @@ static void patch_diff(const char *a, const char *b,
 	diff_flush(diffopt);
 }
 
-static struct strbuf *output_prefix_cb(struct diff_options *opt UNUSED, void *data)
+static const char *output_prefix_cb(struct diff_options *opt UNUSED, void *data)
 {
 	return data;
 }

@@ -508,7 +508,7 @@ static void output(struct string_list *a, struct string_list *b,
 	opts.flags.suppress_hunk_header_line_count = 1;
 	opts.output_prefix = output_prefix_cb;
 	strbuf_addstr(&indent, "    ");
-	opts.output_prefix_data = &indent;
+	opts.output_prefix_data = indent.buf;
 	diff_setup_done(&opts);
 
 	/*

Signature · 1 line

2.47.0.rc1.384.g9f398d04fd

Jeff King

2024-10-03 21:11 UTC #17 lore raw

PATCH 4/5 diff: return line_prefix directly when possible

We may point our output_prefix callback to diff_output_prefix_callback() in any of these cases:

1. we have a user-provided line_prefix

2. we have a graph prefix to show

3. both (1) and (2)

The function combines the available elements into a strbuf and returns its pointer.

In the case that we just have the line_prefix, though, there is no need for the strbuf. We can return the string directly.

This is a minor optimization by itself, but also will allow us to clean up some memory ownership issues on top.

Signed-off-by Jeff King <[email protected]>

---
The old code did handle a case "0" where neither is set. But in that case we would never set up the callback in the first place. So I didn't think it was worth being defensive there.

 graph.c | 3 +++
 1 file changed, 3 insertions(+)

graph.c +3 -0

index c046f6285d..0d200ca77f 100644
@@ -318,6 +318,9 @@ static const char *diff_output_prefix_callback(struct diff_options *opt, void *d
 
 	assert(opt);
 
+	if (!graph)
+		return opt->line_prefix;
+
 	strbuf_reset(&msgbuf);
 	if (opt->line_prefix)
 		strbuf_addstr(&msgbuf, opt->line_prefix);

Signature · 1 line

2.47.0.rc1.384.g9f398d04fd

Jeff King

2024-10-03 21:13 UTC #18 lore raw

PATCH 5/5 diff: store graph prefix buf in git_graph struct

The diffopt output_prefix interface makes it the callback's job to handle ownership of the memory it returns, keeping it valid while callers are using it and then eventually freeing it when we are done diffing.

In diff_output_prefix_callback() we handle this with a static strbuf, effectively "leaking" it when the diff is done (but not triggering any leak detectors because it's technically still reachable). This has not been a big problem in practice, but it is a problem for libification:
two diffs running in the same process could stomp on each other's prefix buffers.

Since we only need the strbuf when we are formatting graph padding, we can give ownership of the strbuf to the git_graph struct, letting us free it when that struct is no longer in use.

Signed-off-by Jeff King <[email protected]>

---
 graph.c | 18 ++++++++++++------
 1 file changed, 12 insertions(+), 6 deletions(-)

graph.c +12 -6

index 0d200ca77f..bf000fdbe1 100644
@@ -309,24 +309,28 @@ struct git_graph {
 	 * stored as an index into the array column_colors.
 	 */
 	unsigned short default_column_color;
+
+	/*
+	 * Scratch buffer for generating prefixes to be used with
+	 * diff_output_prefix_callback().
+	 */
+	struct strbuf prefix_buf;
 };
 
 static const char *diff_output_prefix_callback(struct diff_options *opt, void *data)
 {
 	struct git_graph *graph = data;
-	static struct strbuf msgbuf = STRBUF_INIT;
 
 	assert(opt);
 
 	if (!graph)
 		return opt->line_prefix;
 
-	strbuf_reset(&msgbuf);
+	strbuf_reset(&graph->prefix_buf);
 	if (opt->line_prefix)
-		strbuf_addstr(&msgbuf, opt->line_prefix);
-	if (graph)
-		graph_padding_line(graph, &msgbuf);
-	return msgbuf.buf;
+		strbuf_addstr(&graph->prefix_buf, opt->line_prefix);
+	graph_padding_line(graph, &graph->prefix_buf);
+	return graph->prefix_buf.buf;
 }
 
 static const struct diff_options *default_diffopt;

@@ -396,6 +400,7 @@ struct git_graph *graph_init(struct rev_info *opt)
 	 * The diff output prefix callback, with this we can make
 	 * all the diff output to align with the graph lines.
 	 */
+	strbuf_init(&graph->prefix_buf, 0);
 	opt->diffopt.output_prefix = diff_output_prefix_callback;
 	opt->diffopt.output_prefix_data = graph;

@@ -411,6 +416,7 @@ void graph_clear(struct git_graph *graph)
 	free(graph->new_columns);
 	free(graph->mapping);
 	free(graph->old_mapping);
+	strbuf_release(&graph->prefix_buf);
 	free(graph);
 }

Signature · 1 line

2.47.0.rc1.384.g9f398d04fd

Junio C Hamano

2024-10-03 23:43 UTC #19 lore raw

Re: [PATCH 5/5] diff: store graph prefix buf in git_graph struct

Quoted 6 lines · Jeff King

Since we only need the strbuf when we are formatting graph padding, we can give ownership of the strbuf to the git_graph struct, letting us free it when that struct is no longer in use.
static const char *diff_output_prefix_callback(struct diff_options *opt, void *data) {

graph.c struct git_graph { · line 324

Quoted 9 lines

graph.c +0 -2

 	struct git_graph *graph = data;
-	static struct strbuf msgbuf = STRBUF_INIT;
 
 	assert(opt);
 
 	if (!graph)
 		return opt->line_prefix;
 
-	strbuf_reset(&msgbuf);

Oooh, I love this change. The fewer file scope statics (or global states in general) we have, the better ;-).

Patrick Steinhardt

2024-10-04 04:32 UTC #20 lore raw

Re: [PATCH 5/5] diff: store graph prefix buf in git_graph struct

Quoted 13 lines · Junio C Hamano

Quoted 11 lines · Jeff King
Since we only need the strbuf when we are formatting graph padding, we can give ownership of the strbuf to the git_graph struct, letting us free it when that struct is no longer in use.
static const char *diff_output_prefix_callback(struct diff_options *opt, void *data) {
graph.c +0 -1
 	struct git_graph *graph = data;
-	static struct strbuf msgbuf = STRBUF_INIT;
 
 	assert(opt);

Quoted 7 lines

graph.c struct git_graph { · line 324
Quoted 4 lines
graph.c +0 -1
 	if (!graph)
 		return opt->line_prefix;
 
-	strbuf_reset(&msgbuf);
Oooh, I love this change. The fewer file scope statics (or global states in general) we have, the better ;-).

True, thanks for going the extra mile here! The other patches look good to me, as well.

Patrick

Derrick Stolee

2024-10-04 19:27 UTC #21 lore raw

Re: [PATCH 0/5] diff output_prefix cleanups

Quoted 8 lines · Jeff King

Quoted 3 lines · Derrick Stolee via GitGitGadget
Note to the maintainer: feel free to take only the first patch, as Peff replied that he may work on the remaining cleanup independently (but I had already prepared patches 2 & 3).
Oh, I wasn't expecting you to go to that trouble, and had already polished them up myself. :)

It's perfectly fine that we were attempting to save each other work.

Quoted 3 lines

So certainly your patch 1 looks good to me now. Here's what I would put on top (but I would suggest making it a separate branch, since yours is a fairly urgent fix and mine is all cleanup).

I approve of this plan. Please only consider my first patch and drop the others.

Thanks,
-Stolee

Junio C Hamano

2024-10-04 19:31 UTC #22 lore raw

Re: [PATCH 0/5] diff output_prefix cleanups

Quoted 9 lines · Derrick Stolee

Quoted 7 lines · Jeff King
Quoted 3 lines · Derrick Stolee via GitGitGadget
Note to the maintainer: feel free to take only the first patch, as Peff replied that he may work on the remaining cleanup independently (but I had already prepared patches 2 & 3).
Oh, I wasn't expecting you to go to that trouble, and had already polished them up myself. :)

Quoted 8 lines

It's perfectly fine that we were attempting to save each other work.
Quoted 3 lines
So certainly your patch 1 looks good to me now. Here's what I would put on top (but I would suggest making it a separate branch, since yours is a fairly urgent fix and mine is all cleanup).
I approve of this plan. Please only consider my first patch and drop the others.

Yup. Thanks, both. The result looked very sensible.

Derrick Stolee

2024-10-04 19:33 UTC #23 lore raw

Re: [PATCH 0/5] diff output_prefix cleanups