Memory leak fixes (pt.9)
cover Memory leak fixes (pt.9)
Hi,
here's another round of memory leak fixes. With this series we're down to 10 test suites which are failing with the leak sanitizer. There are two patch series in flight [1][2] that fix three more test suites, so in total we're down to 7 test suites that we'll still have to fix up. So:
we're almost done!
Patrick
[1]: <[email protected]>
[2]: <[email protected]>
Patrick Steinhardt (21):
builtin/ls-remote: plug leaking server options t/helper: fix leaks in "reach" test tool
grep: fix leak in grep_splice_or()
builtin/grep: fix leak with --max-count=0
revision: fix leaking bloom filters
diff-lib: fix leaking diffopts in do_diff_cache()
pretty: clear signature check
upload-pack: fix leaking URI protocols builtin/commit: fix leaking change data contents
trailer: fix leaking trailer values
builtin/commit: fix leaking cleanup config
transport-helper: fix leaking import/export marks builtin/tag: fix leaking key ID on failure to sign
combine-diff: fix leaking lost lines
dir: release untracked cache data
sparse-index: correctly free EWAH contents
t/helper: stop re-initialization of the_repository t/helper: fix leaking buffer in "dump-untracked-cache"
dir: fix leak when parsing "status.showUntrackedFiles" builtin/merge: release outbut buffer after performing merge list-objects-filter-options: work around reported leak on error
builtin/commit.c | 26 +++++++++++++++++------ builtin/grep.c | 13 +++++++++--- builtin/ls-remote.c | 1 + builtin/merge.c | 1 + builtin/tag.c | 2 +- combine-diff.c | 2 +- diff-lib.c | 1 + dir.c | 12 +++++++++-- grep.c | 1 + list-objects-filter-options.c | 17 ++++++--------- pretty.c | 1 + revision.c | 5 +++++ sparse-index.c | 7 ++++-- t/helper/test-dump-untracked-cache.c | 2 ++ t/helper/test-reach.c | 10 +++++++++ t/helper/test-read-cache.c | 2 -- t/t4038-diff-combined.sh | 1 + t/t4202-log.sh | 1 + t/t4216-log-bloom.sh | 1 + t/t5702-protocol-v2.sh | 1 + t/t5801-remote-helpers.sh | 1 + t/t6112-rev-list-filters-objects.sh | 1 + t/t6424-merge-unrelated-index-changes.sh | 1 + t/t6600-test-reach.sh | 1 + t/t7004-tag.sh | 1 + t/t7031-verify-tag-signed-ssh.sh | 1 + t/t7063-status-untracked-cache.sh | 1 + t/t7500-commit-template-squash-signoff.sh | 1 + t/t7502-commit-porcelain.sh | 1 + t/t7510-signed-commit.sh | 1 + t/t7513-interpret-trailers.sh | 1 + t/t7519-status-fsmonitor.sh | 1 + t/t7528-signed-commit-ssh.sh | 1 + t/t7610-mergetool.sh | 1 + t/t7810-grep.sh | 1 + trailer.c | 18 +++++++++++----- transport-helper.c | 2 ++ upload-pack.c | 1 + 38 files changed, 111 insertions(+), 32 deletions(-)
Signature · 1 line
2.47.0.dirty
PATCH 1/21 builtin/ls-remote: plug leaking server options
The server options populated via OPT_STRING_LIST() is never cleared, causing a memory leak. Plug it.
This leak is exposed by t5702, but plugging it alone does not make the whole test suite pass.
--- builtin/ls-remote.c | 1 + 1 file changed, 1 insertion(+)
builtin/ls-remote.c +1 -0
@@ -166,6 +166,7 @@ int cmd_ls_remote(int argc, status =0 ;/* we found something */ } +string_list_clear (&server_options,0 );ref_sorting_release (sorting);ref_array_clear (&ref_array);if (transport_disconnect (transport))
Signature · 1 line
2.47.0.dirty
PATCH 2/21 t/helper: fix leaks in "reach" test tool
In reply to Patrick SteinhardtThe "reach" test tool doesn't bother to clean up any of its allocated resources, causing various leaks. Plug them.
--- t/helper/test-reach.c | 10 ++++++++++ t/t6600-test-reach.sh | 1 + 2 files changed, 11 insertions(+)
t/helper/test-reach.c +10 -0
@@ -127,10 +127,12 @@ int cmd__reach(int ac, const char **av)exit (128 );printf ("%s(A,X):\n" , av[1 ]);print_sorted_commit_ids (list); +free_commit_list (list); }else if (!strcmp (av[1 ],"reduce_heads" )) {struct commit_list *list =reduce_heads (X );printf ("%s(X):\n" , av[1 ]);print_sorted_commit_ids (list); +free_commit_list (list); }else if (!strcmp (av[1 ],"can_all_from_reach" )) {printf ("%s(X,Y):%d\n" , av[1 ],can_all_from_reach (X ,Y ,1 )); }else if (!strcmp (av[1 ],"can_all_from_reach_with_flag" )) {
@@ -153,6 +155,7 @@ int cmd__reach(int ac, const char **av) filter.with_commit_tag_algo =0 ;printf ("%s(_,A,X,_):%d\n" , av[1 ],commit_contains (&filter,A ,X , &cache)); +clear_contains_cache (&cache); }else if (!strcmp (av[1 ],"get_reachable_subset" )) {const int reachable_flag =1 ;int i, count =0 ;
@@ -176,7 +179,14 @@ int cmd__reach(int ac, const char **av)die (_ ("too many commits marked reachable" ));print_sorted_commit_ids (list); +free_commit_list (list); } +object_array_clear (&X_obj); +strbuf_release (&buf); +free_commit_list (X ); +free_commit_list (Y ); +free (X_array); +free (Y_array);return 0 ; }
t/t6600-test-reach.sh +1 -0
@@ -2,6 +2,7 @@ test_description='basic commit reachability tests' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Construct a grid-like commit graph with points (x,y)
Signature · 1 line
2.47.0.dirty
PATCH 3/21 grep: fix leak in `grep_splice_or()`
In reply to Patrick SteinhardtIn grep_splice_or() we search for the next TRUE node in our tree of grep exrpessions and replace it with the given new expression. But we don't free the old node, which causes a memory leak. Plug it.
This leak is exposed by t7810, but plugging it alone isn't sufficient to make the test suite pass.
--- grep.c | 1 + 1 file changed, 1 insertion(+)
grep.c +1 -0
@@ -756,6 +756,7 @@ static struct grep_expr *grep_splice_or(struct grep_expr *x, struct grep_expr *yassert (x->node ==GREP_NODE_OR );if (x->u.binary.right && x->u.binary.right->node ==GREP_NODE_TRUE ) { +free (x->u.binary.right); x->u.binary.right = y;break ; }
Signature · 1 line
2.47.0.dirty
PATCH 4/21 builtin/grep: fix leak with `--max-count=0`
In reply to Patrick SteinhardtWhen executing with --max-count=0 we'll return early from git-grep(1) without performing any cleanup, which causes memory leaks. Plug these.
--- builtin/grep.c | 13 ++++++++++--- t/t7810-grep.sh | 1 + 2 files changed, 11 insertions(+), 3 deletions(-)
builtin/grep.c +10 -3
@@ -906,6 +906,7 @@ int cmd_grep(int argc,int dummy;int use_index =1 ;int allow_revs; +int ret;struct option options[] = { OPT_BOOL(0, "cached", &cached,
@@ -1172,8 +1173,10 @@ int cmd_grep(int argc,* Optimize out the case where the amount of matches is limited to zero. * We do this to keep results consistent with GNU grep(1). */ -if (opt.max_count ==0 ) -return 1 ; +if (opt.max_count ==0 ) { + ret =1 ; + goto out; + }if (show_in_pager) {if (num_threads >1 )
@@ -1267,10 +1270,14 @@ int cmd_grep(int argc, hit |=wait_all ();if (hit && show_in_pager)run_pager (&opt, prefix); + + ret = !hit; + +out:clear_pathspec (&pathspec);string_list_clear (&path_list,0 );free_grep_patterns (&opt);object_array_clear (&list);free_repos (); -return !hit; +return ret; }
t/t7810-grep.sh +1 -0
@@ -9,6 +9,7 @@ test_description='git grep various. GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_invalid_grep_expression() {
Signature · 1 line
2.47.0.dirty
PATCH 5/21 revision: fix leaking bloom filters
In reply to Patrick SteinhardtThe memory allocated by prepare_to_use_bloom_filter() is not released by release_revisions(), causing a memory leak. Plug it.
--- revision.c | 5 +++++ t/t4216-log-bloom.sh | 1 + 2 files changed, 6 insertions(+)
revision.c +5 -0
@@ -3227,6 +3227,11 @@ void release_revisions(struct rev_info *revs)clear_decoration (&revs->treesame, free);line_log_free (revs);oidset_clear (&revs->missing_commits); + +for (int i =0 ; i < revs->bloom_keys_nr; i++) +clear_bloom_key (&revs->bloom_keys[i]); +FREE_AND_NULL (revs->bloom_keys); + revs->bloom_keys_nr =0 ; }static void add_child (struct rev_info *revs,struct commit *parent,struct commit *child)
t/t4216-log-bloom.sh +1 -0
@@ -4,6 +4,7 @@ test_description='git log for a path with Bloom filters' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-chunk.sh
Signature · 1 line
2.47.0.dirty
PATCH 6/21 diff-lib: fix leaking diffopts in `do_diff_cache()`
In reply to Patrick SteinhardtIn do_diff_cache() we initialize a new rev_info and then overwrite its diffopt with a user-provided set of options. This can leak memory because repo_init_revisions() may end up allocating memory for the diffopt itself depending on the configuration. And as that field is overwritten we won't ever free that.
Plug the memory leak by releasing the diffopts before we overwrite them.
--- diff-lib.c | 1 + t/t7610-mergetool.sh | 1 + 2 files changed, 2 insertions(+)
diff-lib.c +1 -0
@@ -661,6 +661,7 @@ int do_diff_cache(const struct object_id *tree_oid, struct diff_options *opt)repo_init_revisions (opt->repo, &revs,NULL );copy_pathspec (&revs.prune_data, &opt->pathspec); +diff_free (&revs.diffopt); revs.diffopt = *opt; revs.diffopt.no_free =1 ;
t/t7610-mergetool.sh +1 -0
@@ -10,6 +10,7 @@ Testing basic merge tool invocation' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # All the mergetool test work by checking out a temporary branch based
Signature · 1 line
2.47.0.dirty
Re: [PATCH 06/21] diff-lib: fix leaking diffopts in `do_diff_cache()`
Quoted 4 lines · Patrick Steinhardt
In
do_diff_cache()we initialize a newrev_infoand then overwrite itsdiffoptwith a user-provided set of options. This can leak memory becauserepo_init_revisions()may end up allocating memory for thediffoptitself depending on the configuration. And as that field is
s/as that/since that/
“since” communicates causality better.
Quoted 1 line
overwritten we won't ever free that.
s/free that/free it/
Signature · 1 line
Kristoffer Haugsbakk
PATCH 7/21 pretty: clear signature check
In reply to Patrick SteinhardtThe signature check in of the formatting context is never getting released. Fix this to plug the resulting memory leak.
--- pretty.c | 1 + t/t4202-log.sh | 1 + t/t7031-verify-tag-signed-ssh.sh | 1 + t/t7510-signed-commit.sh | 1 + t/t7528-signed-commit-ssh.sh | 1 + 5 files changed, 5 insertions(+)
pretty.c +1 -0
@@ -2032,6 +2032,7 @@ void repo_format_commit_message(struct repository *r,free (context.commit_encoding);repo_unuse_commit_buffer (r, commit, context.message); +signature_check_clear (&context.signature_check); } static void pp_header(struct pretty_print_context *pp,
t/t4202-log.sh +1 -0
@@ -5,6 +5,7 @@ test_description='git log' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY/lib-gpg.sh" . "$TEST_DIRECTORY/lib-terminal.sh"
t/t7031-verify-tag-signed-ssh.sh +1 -0
@@ -4,6 +4,7 @@ test_description='signed tag tests' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY/lib-gpg.sh"
t/t7510-signed-commit.sh +1 -0
@@ -4,6 +4,7 @@ test_description='signed commit tests' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh GNUPGHOME_NOT_USED=$GNUPGHOME . "$TEST_DIRECTORY/lib-gpg.sh"
t/t7528-signed-commit-ssh.sh +1 -0
@@ -4,6 +4,7 @@ test_description='ssh signed commit tests' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh GNUPGHOME_NOT_USED=$GNUPGHOME . "$TEST_DIRECTORY/lib-gpg.sh"
Signature · 1 line
2.47.0.dirty
Re: [PATCH 07/21] pretty: clear signature check
Quoted 17 lines · Patrick Steinhardt
The signature check in of the formatting context is never getting released. Fix this to plug the resulting memory leak.
Signed-off-by Patrick Steinhardt <[email protected]>--- pretty.c | 1 + t/t4202-log.sh | 1 + t/t7031-verify-tag-signed-ssh.sh | 1 + t/t7510-signed-commit.sh | 1 + t/t7528-signed-commit-ssh.sh | 1 + 5 files changed, 5 insertions(+)pretty.c +0
-0
pretty.c void repo_format_commit_message(struct repository *r, · line 2035
Quoted 4 lines
pretty.c +1
-0@@ -2032,6 +2032,7 @@ void repo_format_commit_message(struct repository *r,free (context.commit_encoding);repo_unuse_commit_buffer (r, commit, context.message); +signature_check_clear (&context.signature_check);
I was having a very hard time finding where this gets allocated, and to be honest, I still don't know for sure. I think in check_commit_signature() which is called by format_commit_one(). In "[PATCH 20/21] builtin/merge: release outbut buffer after performing merge" you mention it's not obvious to the caller they need know about memory they need to clean up, isn't that case here as well?
--
Toon
Re: [PATCH 07/21] pretty: clear signature check
Quoted 20 lines · Toon Claes
Quoted 18 lines · Patrick Steinhardt
The signature check in of the formatting context is never getting released. Fix this to plug the resulting memory leak.
Signed-off-by Patrick Steinhardt <[email protected]>--- pretty.c | 1 + t/t4202-log.sh | 1 + t/t7031-verify-tag-signed-ssh.sh | 1 + t/t7510-signed-commit.sh | 1 + t/t7528-signed-commit-ssh.sh | 1 + 5 files changed, 5 insertions(+)pretty.c +0
-0@@ -2032,6 +2032,7 @@ void repo_format_commit_message(struct repository *r,
Quoted 10 lines
pretty.c void repo_format_commit_message(struct repository *r, · line 2035
Quoted 3 lines
pretty.c +1
-0free (context.commit_encoding);repo_unuse_commit_buffer (r, commit, context.message); +signature_check_clear (&context.signature_check);I was having a very hard time finding where this gets allocated, and to be honest, I still don't know for sure. I think in
check_commit_signature()which is called byformat_commit_one(). In "[PATCH 20/21] builtin/merge: release outbut buffer after performing merge" you mention it's not obvious to the caller they need know about memory they need to clean up, isn't that case here as well?
Well, I think it's clearer in this context, but hidden by the fact that we pass around a wrapper-structure. But that's not really the problem of the struct signature_check subsystem, but rather of "pretty.c".
In any case, the callchain is:
- format_commit_one()
- check_commit_signature()
- check_signature()
- verify_signed_buffer(), which is a function pointer depending on whether we verify a GPG, x509 or SSH signature.
From my point of view, that callchain isn't all that important in this context. All we need to know is that we allocate the signature check struct on our stack, and as it may get populated we have to clean it up, as well.
Patrick
PATCH 8/21 upload-pack: fix leaking URI protocols
In reply to Patrick SteinhardtWe don't clear struct upload_pack::uri_protocols, which causes a memory leak. Fix this.
--- t/t5702-protocol-v2.sh | 1 + upload-pack.c | 1 + 2 files changed, 2 insertions(+)
t/t5702-protocol-v2.sh +1 -0
@@ -7,6 +7,7 @@ TEST_NO_CREATE_REPO=1 GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Test protocol v2 with 'git://' transport
upload-pack.c +1 -0
@@ -166,6 +166,7 @@ static void upload_pack_data_clear(struct upload_pack_data *data)object_array_clear (&data->extra_edge_obj);list_objects_filter_release (&data->filter_options);string_list_clear (&data->allowed_filters,0 ); +string_list_clear (&data->uri_protocols,0 );free ((char *)data->pack_objects_hook); }
Signature · 1 line
2.47.0.dirty
PATCH 9/21 builtin/commit: fix leaking change data contents
In reply to Patrick SteinhardtWhile we free the worktree change data, we never free its contents. Fix this.
--- builtin/commit.c | 9 ++++++++- t/t7500-commit-template-squash-signoff.sh | 1 + 2 files changed, 9 insertions(+), 1 deletion(-)
builtin/commit.c +8 -1
@@ -728,6 +728,13 @@ static void prepare_amend_commit(struct commit *commit, struct strbuf *sb,repo_unuse_commit_buffer (the_repository, commit, buffer); } +static void change_data_free(void *util, const char *str UNUSED) +{ +struct wt_status_change_data *d = util; +free (d->rename_source); +free (d); +} + static int prepare_to_commit(const char *index_file, const char *prefix, struct commit *current_head, struct wt_status *s,
@@ -991,7 +998,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix, s->use_color =0 ; committable =run_status (s->fp, index_file, prefix,1 , s); s->use_color = saved_color_setting; -string_list_clear (&s->change,1 ); +string_list_clear_func (&s->change, change_data_free); }else {struct object_id oid;const char *parent ="HEAD" ;
t/t7500-commit-template-squash-signoff.sh +1 -0
@@ -7,6 +7,7 @@ test_description='git commit Tests for template, signoff, squash and -F functions.' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-rebase.sh
Signature · 1 line
2.47.0.dirty
PATCH 10/21 trailer: fix leaking trailer values
In reply to Patrick SteinhardtFix leaking trailer values when replacing the value with a command or when the token value is empty.
--- t/t7513-interpret-trailers.sh | 1 + trailer.c | 18 +++++++++++++----- 2 files changed, 14 insertions(+), 5 deletions(-)
t/t7513-interpret-trailers.sh +1 -0
@@ -5,6 +5,7 @@ test_description='git interpret-trailers' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # When we want one trailing space at the end of each line, let's use sed
trailer.c +13 -5
@@ -249,17 +249,23 @@ static char *apply_command(struct conf_info *conf, const char *arg)static void apply_item_command (struct trailer_item *in_tok,struct arg_item *arg_tok) {if (arg_tok->conf.command || arg_tok->conf.cmd) { -const char *arg; +char *value_to_free =NULL ; +char *arg; +if (arg_tok->value && arg_tok->value[0 ]) { - arg = arg_tok->value; + arg = (char *)arg_tok->value; }else {if (in_tok && in_tok->value) arg =xstrdup (in_tok->value);else arg =xstrdup ("" ); + value_to_free = arg_tok->value; } + arg_tok->value =apply_command (&arg_tok->conf, arg); -free ((char *)arg); + +free (value_to_free); +free (arg); } }
@@ -1114,6 +1120,7 @@ void format_trailers(const struct process_trailer_options *opts,if (item->token) {struct strbuf tok =STRBUF_INIT ;struct strbuf val =STRBUF_INIT ; +strbuf_addstr (&tok, item->token);strbuf_addstr (&val, item->value);
@@ -1124,7 +1131,7 @@ void format_trailers(const struct process_trailer_options *opts,* corresponding value). */ if (opts->trim_empty && !strlen (item->value)) -continue ; + goto next;if (!opts->filter || opts->filter (&tok, opts->filter_data)) {if (opts->separator && out->len != origlen)
@@ -1145,9 +1152,10 @@ void format_trailers(const struct process_trailer_options *opts,if (!opts->separator)strbuf_addch (out,'\n' ); } + +next:strbuf_release (&tok);strbuf_release (&val); - }else if (!opts->only_trailers) {if (opts->separator && out->len != origlen) {strbuf_addbuf (out, opts->separator);
Signature · 1 line
2.47.0.dirty
Re: [PATCH 10/21] trailer: fix leaking trailer values
Quoted 11 lines · Patrick Steinhardt
Fix leaking trailer values when replacing the value with a command or when the token value is empty.
Signed-off-by Patrick Steinhardt <[email protected]>---
trailer.c +0
-0
trailer.c void format_trailers(const struct process_trailer_options *opts, · line 1134
Quoted 5 lines
trailer.c +1
-1@@ -1124,7 +1131,7 @@ void format_trailers(const struct process_trailer_options *opts,* corresponding value). */ if (opts->trim_empty && !strlen (item->value)) -continue ; + goto next;
While this is technically correct, I found it rather hard to understand what's happening. What do you think about instead turning the if below in an else if ?
Quoted 17 lines
trailer.c +2
-1if (!opts->filter || opts->filter (&tok, opts->filter_data)) {if (opts->separator && out->len != origlen)@@ -1145,9 +1152,10 @@ void format_trailers(const struct process_trailer_options *opts,if (!opts->separator)strbuf_addch (out,'\n' ); } + +next:strbuf_release (&tok);strbuf_release (&val); - }else if (!opts->only_trailers) {if (opts->separator && out->len != origlen) {strbuf_addbuf (out, opts->separator);Signature · 1 line
2.47.0.dirty
--
Toon
Re: [PATCH 10/21] trailer: fix leaking trailer values
Quoted 8 lines · Toon Claes
Quoted 6 lines · Patrick Steinhardt
Fix leaking trailer values when replacing the value with a command or when the token value is empty.
Signed-off-by Patrick Steinhardt <[email protected]>---
Quoted 14 lines
trailer.c void format_trailers(const struct process_trailer_options *opts, · line 1134
Quoted 10 lines
trailer.c +1
-1@@ -1124,7 +1131,7 @@ void format_trailers(const struct process_trailer_options *opts,* corresponding value). */ if (opts->trim_empty && !strlen (item->value)) -continue ; + goto next;While this is technically correct, I found it rather hard to understand what's happening. What do you think about instead turning the
ifbelow in anelse if?
An even better idea is to lift the buffers outside of the loop. Like this we don't have to reallocate them on every iteration and can easily release them once at the end of the function.
Patrick
PATCH 11/21 builtin/commit: fix leaking cleanup config
In reply to Patrick SteinhardtThe cleanup string set by the config is leaking when it is being overridden by an option. Fix this by tracking these via two separate variables such that we can free the old value.
--- builtin/commit.c | 17 ++++++++++++----- t/t7502-commit-porcelain.sh | 1 + 2 files changed, 13 insertions(+), 5 deletions(-)
builtin/commit.c +12 -5
@@ -135,7 +135,7 @@ static struct strvec trailer_args = STRVEC_INIT;* is specified explicitly. */ static enum commit_msg_cleanup_mode cleanup_mode; -static char *cleanup_arg; +static char *cleanup_config;static enum commit_whence whence;static int use_editor =1 , include_status =1 ;
@@ -1387,8 +1387,6 @@ static int parse_and_validate_options(int argc, const char *argv[],if (0 <= edit_flag) use_editor = edit_flag; - cleanup_mode =get_cleanup_mode (cleanup_arg, use_editor); -handle_untracked_files_arg (s);if (all && argc >0 )
@@ -1636,8 +1634,10 @@ static int git_commit_config(const char *k, const char *v, include_status =git_config_bool (k, v);return 0 ; } -if (!strcmp (k,"commit.cleanup" )) -return git_config_string (&cleanup_arg, k, v); +if (!strcmp (k,"commit.cleanup" )) { +FREE_AND_NULL (cleanup_config); +return git_config_string (&cleanup_config, k, v); + }if (!strcmp (k,"commit.gpgsign" )) { sign_commit =git_config_bool (k, v) ?"" :NULL ;return 0 ;
@@ -1658,6 +1658,7 @@ int cmd_commit(int argc, struct repository *repo UNUSED) {static struct wt_status s; +static const char *cleanup_arg =NULL ;static struct option builtin_commit_options[] = {OPT__QUIET (&quiet,N_ ("suppress summary after successful commit" )),OPT__VERBOSE (&verbose,N_ ("show diff in commit message template" )),
@@ -1757,6 +1758,12 @@ int cmd_commit(int argc,if (verbose ==-1 ) verbose = (config_commit_verbose <0 ) ?0 : config_commit_verbose; +if (cleanup_arg) { +free (cleanup_config); + cleanup_config =xstrdup (cleanup_arg); + } + cleanup_mode =get_cleanup_mode (cleanup_config, use_editor); +if (dry_run)return dry_run_commit (argv, prefix, current_head, &s); index_file =prepare_index (argv, prefix, current_head,0 );
t/t7502-commit-porcelain.sh +1 -0
@@ -5,6 +5,7 @@ test_description='git commit porcelain-ish' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh commit_msg_is () {
Signature · 1 line
2.47.0.dirty
PATCH 12/21 transport-helper: fix leaking import/export marks
In reply to Patrick SteinhardtFix leaking import and export marks for transport helpers.
--- t/t5801-remote-helpers.sh | 1 + transport-helper.c | 2 ++ 2 files changed, 3 insertions(+)
t/t5801-remote-helpers.sh +1 -0
@@ -8,6 +8,7 @@ test_description='Test remote-helper import and export commands' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-gpg.sh
transport-helper.c +2 -0
@@ -399,6 +399,8 @@ static int release_helper(struct transport *transport)int res =0 ;struct helper_data *data = transport->data;refspec_clear (&data->rs); +free (data->import_marks); +free (data->export_marks); res =disconnect_helper (transport);free (transport->data);return res;
Signature · 1 line
2.47.0.dirty
PATCH 13/21 builtin/tag: fix leaking key ID on failure to sign
In reply to Patrick SteinhardtWe do not free the key ID when signing a tag fails. Do so by using the common exit path.
--- builtin/tag.c | 2 +- t/t7004-tag.sh | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)
builtin/tag.c +1 -1
@@ -164,7 +164,7 @@ static int do_sign(struct strbuf *buffer, struct object_id **compat_oid,int ret =-1 ;if (sign_buffer (buffer, &sig, keyid)) -return -1 ; + goto out;if (compat) {const struct git_hash_algo *algo = the_repository->hash_algo;
t/t7004-tag.sh +1 -0
@@ -10,6 +10,7 @@ Tests for operations with tags.' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-gpg.sh . "$TEST_DIRECTORY"/lib-terminal.sh
Signature · 1 line
2.47.0.dirty
PATCH 14/21 combine-diff: fix leaking lost lines
In reply to Patrick SteinhardtThe cnt variable tracks the number of lines in a patch diff. It can happen though that there are no newlines, in which case we'd still end up allocating our array of slines. In fact, we always allocate it with cnt + 2 entries. But when we loop through the array to clear it at the end of this function we only loop until lno < cnt, and thus we may not end up releasing whatever the two extra slines contain.
Plug this memory leak.
--- combine-diff.c | 2 +- t/t4038-diff-combined.sh | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)
combine-diff.c +1 -1
@@ -1220,7 +1220,7 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent, }free (result); -for (lno =0 ; lno < cnt; lno++) { +for (lno =0 ; lno < cnt +2 ; lno++) {if (sline[lno].lost) {struct lline *ll = sline[lno].lost;while (ll) {
t/t4038-diff-combined.sh +1 -0
@@ -5,6 +5,7 @@ test_description='combined diff' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-diff.sh
Signature · 1 line
2.47.0.dirty
PATCH 15/21 dir: release untracked cache data
In reply to Patrick SteinhardtThere are several cases where we invalidate untracked cache directory entries where we do not free the underlying data, but reset the number of entries. This causes us to leak memory because free_untracked() will not iterate over any potential entries which we still had in the array.
Fix this issue by freeing old entries. The leak is exposed by t7519, but plugging it alone does not make the whole test suite pass.
--- dir.c | 8 ++++++++ 1 file changed, 8 insertions(+)
dir.c +8 -0
@@ -1056,6 +1056,8 @@ static void do_invalidate_gitignore(struct untracked_cache_dir *dir) {int i; dir->valid =0 ; +for (size_t i =0 ; i < dir->untracked_nr; i++) +free (dir->untracked[i]); dir->untracked_nr =0 ;for (i =0 ; i < dir->dirs_nr; i++)do_invalidate_gitignore (dir->dirs[i]);
@@ -1083,6 +1085,8 @@ static void invalidate_directory(struct untracked_cache *uc, uc->dir_invalidated++; dir->valid =0 ; +for (size_t i =0 ; i < dir->untracked_nr; i++) +free (dir->untracked[i]); dir->untracked_nr =0 ;for (i =0 ; i < dir->dirs_nr; i++) dir->dirs[i]->recurse =0 ;
@@ -3573,6 +3577,8 @@ static void write_one_dir(struct untracked_cache_dir *untracked,* for safety.. */ if (!untracked->valid) { +for (size_t i =0 ; i < untracked->untracked_nr; i++) +free (untracked->untracked[i]); untracked->untracked_nr =0 ; untracked->check_only =0 ; }
@@ -3905,6 +3911,8 @@ static void invalidate_one_directory(struct untracked_cache *uc, { uc->dir_invalidated++; ucd->valid =0 ; +for (size_t i =0 ; i < ucd->untracked_nr; i++) +free (ucd->untracked[i]); ucd->untracked_nr =0 ; }
Signature · 1 line
2.47.0.dirty
PATCH 16/21 sparse-index: correctly free EWAH contents
In reply to Patrick SteinhardtWhile we free the fsmonitor_dirty member of struct index_state, we do not free the contents of that EWAH. Do so by using ewah_free() instead of FREE_AND_NULL().
This leak is exposed by t7519, but plugging it alone does not make the test suite pass.
--- sparse-index.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
sparse-index.c +5 -2
@@ -2,6 +2,7 @@#include "git-compat-util.h" #include "environment.h" +#include "ewah/ewok.h" #include "gettext.h" #include "name-hash.h" #include "read-cache-ll.h"
@@ -242,7 +243,8 @@ int convert_to_sparse(struct index_state *istate, int flags)cache_tree_update (istate,0 ); istate->fsmonitor_has_run_once =0 ; -FREE_AND_NULL (istate->fsmonitor_dirty); +ewah_free (istate->fsmonitor_dirty); + istate->fsmonitor_dirty =NULL ;FREE_AND_NULL (istate->fsmonitor_last_update); istate->sparse_index =INDEX_COLLAPSED ;
@@ -438,7 +440,8 @@ void expand_index(struct index_state *istate, struct pattern_list *pl) istate->cache_nr = full->cache_nr; istate->cache_alloc = full->cache_alloc; istate->fsmonitor_has_run_once =0 ; -FREE_AND_NULL (istate->fsmonitor_dirty); +ewah_free (istate->fsmonitor_dirty); + istate->fsmonitor_dirty =NULL ;FREE_AND_NULL (istate->fsmonitor_last_update);strbuf_release (&base);
Signature · 1 line
2.47.0.dirty
PATCH 17/21 t/helper: stop re-initialization of `the_repository`
In reply to Patrick SteinhardtWhile "common-main.c" already initializes the_repository for us, we do so a second time in the "read-cache" test helper. This causes a memory leak because the old repository's contents isn't released.
Stop calling initialize_repository() to plug this leak.
--- t/helper/test-read-cache.c | 2 -- t/t7519-status-fsmonitor.sh | 1 + 2 files changed, 1 insertion(+), 2 deletions(-)
t/helper/test-read-cache.c +0 -2
@@ -11,8 +11,6 @@ int cmd__read_cache(int argc, const char **argv)int i, cnt =1 ;const char *name =NULL ; -initialize_repository (the_repository); -if (argc >1 &&skip_prefix (argv[1 ],"--print-and-refresh=" , &name)) { argc--; argv++;
t/t7519-status-fsmonitor.sh +1 -0
@@ -2,6 +2,7 @@ test_description='git status with file system watcher' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Note, after "git reset --hard HEAD" no extensions exist other than 'TREE'
Signature · 1 line
2.47.0.dirty
PATCH 18/21 t/helper: fix leaking buffer in "dump-untracked-cache"
In reply to Patrick SteinhardtWe never release the local struct strbuf base buffer, thus leaking memory. Fix this leak.
This leak is exposed by t7063, but plugging it alone does not make the whole test suite pass.
--- t/helper/test-dump-untracked-cache.c | 2 ++ 1 file changed, 2 insertions(+)
t/helper/test-dump-untracked-cache.c +2 -0
@@ -68,5 +68,7 @@ int cmd__dump_untracked_cache(int ac UNUSED, const char **av UNUSED)printf ("flags %08x\n" , uc->dir_flags);if (uc->root)dump (uc->root, &base); + +strbuf_release (&base);return 0 ; }
Signature · 1 line
2.47.0.dirty
PATCH 19/21 dir: fix leak when parsing "status.showUntrackedFiles"
In reply to Patrick SteinhardtWe use repo_config_get_string() to read "status.showUntrackedFiles" from the config subsystem. This function allocates the result, but we never free the result after parsing it.
The value never leaves the scope of the calling function, so refactor it to instead use repo_config_get_string_tmp(), which does not hand over ownership to the caller.
--- dir.c | 4 ++-- t/t7063-status-untracked-cache.sh | 1 + 2 files changed, 3 insertions(+), 2 deletions(-)
dir.c +2 -2
@@ -2872,14 +2872,14 @@ static void set_untracked_ident(struct untracked_cache *uc)static unsigned new_untracked_cache_flags (struct index_state *istate) {struct repository *repo = istate->repo; -char *val; +const char *val;/* * This logic is coordinated with the setting of these flags in * wt-status.c#wt_status_collect_untracked(), and the evaluation * of the config setting in commit.c#git_status_config() */ -if (!repo_config_get_string (repo,"status.showuntrackedfiles" , &val) && +if (!repo_config_get_string_tmp (repo,"status.showuntrackedfiles" , &val) && !strcmp (val,"all" ))return 0 ;
t/t7063-status-untracked-cache.sh +1 -0
@@ -5,6 +5,7 @@ test_description='test untracked cache' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # On some filesystems (e.g. FreeBSD's ext2 and ufs) directory mtime
Signature · 1 line
2.47.0.dirty
PATCH 20/21 builtin/merge: release outbut buffer after performing merge
In reply to Patrick SteinhardtThe obuf member of struct merge_options is used to buffer output in some cases. In order to not discard its allocated memory we only release its contents in merge_finalize() when we're not currently recursing into a subtree.
This results in some situations where we seemingly do not release the buffer reliably. We thus have calls to strbuf_release() for this buffer scattered across the codebase. But we're missing one callsite in git-merge(1), which causes a memory leak.
We should ideally refactor this interface so that callers don't have to know about any such internals. But for now, paper over the issue by adding one more strbuf_release() call.
--- builtin/merge.c | 1 + t/t6424-merge-unrelated-index-changes.sh | 1 + 2 files changed, 2 insertions(+)
builtin/merge.c +1 -0
@@ -754,6 +754,7 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common, clean =merge_recursive (&o, head, remoteheads->item, reversed, &result);free_commit_list (reversed); +strbuf_release (&o.obuf);if (clean <0 ) {rollback_lock_file (&lock);
t/t6424-merge-unrelated-index-changes.sh +1 -0
@@ -2,6 +2,7 @@ test_description="merges with unrelated index changes" +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Testcase for some simple merges
Signature · 1 line
2.47.0.dirty
Re: [PATCH 20/21] builtin/merge: release outbut buffer after performing merge
Quoted 5 lines · Patrick Steinhardt
The
obufmember ofstruct merge_optionsis used to buffer output in some cases. In order to not discard its allocated memory we only release its contents inmerge_finalize()when we're not currently recursing into a subtree.
Quoted 8 lines
This results in some situations where we seemingly do not release the buffer reliably. We thus have calls to
strbuf_release()for this buffer scattered across the codebase. But we're missing one callsite in git-merge(1), which causes a memory leak.We should ideally refactor this interface so that callers don't have to know about any such internals. But for now, paper over the issue by adding one more
strbuf_release()call.
Shall we mark this as #leftoverbits?
--
Toon
Re: [PATCH 20/21] builtin/merge: release outbut buffer after performing merge
Quoted 12 lines · Toon Claes
Quoted 10 lines · Patrick Steinhardt
The
obufmember ofstruct merge_optionsis used to buffer output in some cases. In order to not discard its allocated memory we only release its contents inmerge_finalize()when we're not currently recursing into a subtree.This results in some situations where we seemingly do not release the buffer reliably. We thus have calls to
strbuf_release()for this buffer scattered across the codebase. But we're missing one callsite in git-merge(1), which causes a memory leak.
Quoted 5 lines
Quoted 3 lines
We should ideally refactor this interface so that callers don't have to know about any such internals. But for now, paper over the issue by adding one more
strbuf_release()call.Shall we mark this as #leftoverbits?
I guess it is now marked as such :)
Patrick
PATCH 21/21 list-objects-filter-options: work around reported leak on error
In reply to Patrick SteinhardtThis one is a little bit more curious. In t6112, we have a test that exercises the git rev-list --filter option with invalid filters. We execute git-rev-list(1) via test_must_fail, which means that we check for leaks even though Git exits with an error code. This causes the following leak:
Direct leak of 27 byte(s) in 1 object(s) allocated from:
#0 0x5555555e6946 in realloc.part.0 lsan_interceptors.cpp.o
#1 0x5555558fb4b6 in xrealloc wrapper.c:137:8
#2 0x5555558b6e06 in strbuf_grow strbuf.c:112:2
#3 0x5555558b7550 in strbuf_add strbuf.c:311:2
#4 0x5555557c1a88 in strbuf_addstr strbuf.h:310:2
#5 0x5555557c1d4c in parse_list_objects_filter list-objects-filter-options.c:261:3
#6 0x555555885ead in handle_revision_pseudo_opt revision.c:2899:3
#7 0x555555884e20 in setup_revisions revision.c:3014:11
#8 0x5555556c4b42 in cmd_rev_list builtin/rev-list.c:588:9
#9 0x5555555ec5e3 in run_builtin git.c:483:11
#10 0x5555555eb1e4 in handle_builtin git.c:749:13
#11 0x5555555ec001 in run_argv git.c:819:4
#12 0x5555555eaf94 in cmd_main git.c:954:19
#13 0x5555556fd569 in main common-main.c:64:11
#14 0x7ffff7ca714d in __libc_start_call_main (.../lib/libc.so.6+0x2a14d)
#15 0x7ffff7ca7208 in __libc_start_main@GLIBC_2.2.5 (.../libc.so.6+0x2a208)
#16 0x5555555ad064 in _start (git+0x59064)This leak is valid, as we call die() and do not clean up the memory at all. But what's curious is that this is the only leak reported, because we don't clean up any other allocated memory, either, and I have no idea why the leak sanitizer treats this buffer specially.
In any case, we can work around the leak by shuffling things around a bit. Instead of calling gently_parse_list_objects_filter() and dying after we have modified the filter spec, we simply do so beforehand. Like this we don't allocate the buffer in the error case, which makes the reported leak go away.
It's not pretty, but it manages to make t6112 leak free.
--- list-objects-filter-options.c | 17 +++++++---------- t/t6112-rev-list-filters-objects.sh | 1 + 2 files changed, 8 insertions(+), 10 deletions(-)
list-objects-filter-options.c +7 -10
@@ -252,16 +252,14 @@ void parse_list_objects_filter( const char *arg) {struct strbuf errbuf =STRBUF_INIT ; -int parse_error;if (!filter_options->filter_spec.buf)BUG ("filter_options not properly initialized" );if (!filter_options->choice) { +if (gently_parse_list_objects_filter (filter_options, arg, &errbuf)) +die ("%s" , errbuf.buf);strbuf_addstr (&filter_options->filter_spec, arg); - - parse_error =gently_parse_list_objects_filter ( - filter_options, arg, &errbuf); }else {struct list_objects_filter_options *sub;
@@ -271,18 +269,17 @@ void parse_list_objects_filter(*/ transform_to_combine_type (filter_options); -strbuf_addch (&filter_options->filter_spec,'+' ); -filter_spec_append_urlencode (filter_options, arg);ALLOC_GROW_BY (filter_options->sub, filter_options->sub_nr,1 , filter_options->sub_alloc); sub = &filter_options->sub[filter_options->sub_nr -1 ];list_objects_filter_init (sub); - parse_error =gently_parse_list_objects_filter (sub, arg, - &errbuf); +if (gently_parse_list_objects_filter (sub, arg, &errbuf)) +die ("%s" , errbuf.buf); + +strbuf_addch (&filter_options->filter_spec,'+' ); +filter_spec_append_urlencode (filter_options, arg); } -if (parse_error) -die ("%s" , errbuf.buf); } int opt_parse_list_objects_filter(const struct option *opt,
t/t6112-rev-list-filters-objects.sh +1 -0
@@ -5,6 +5,7 @@ test_description='git rev-list using object filtering' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Test the blob:none filter.
Signature · 1 line
2.47.0.dirty
Re: [PATCH 21/21] list-objects-filter-options: work around reported leak on error
Quoted 72 lines · Patrick Steinhardt
This one is a little bit more curious. In t6112, we have a test that exercises the
git rev-list --filteroption with invalid filters. We execute git-rev-list(1) viatest_must_fail, which means that we check for leaks even though Git exits with an error code. This causes the following leak:Direct leak of 27 byte(s) in 1 object(s) allocated from: #0 0x5555555e6946 in realloc.part.0 lsan_interceptors.cpp.o #1 0x5555558fb4b6 in xrealloc wrapper.c:137:8 #2 0x5555558b6e06 in strbuf_grow strbuf.c:112:2 #3 0x5555558b7550 in strbuf_add strbuf.c:311:2 #4 0x5555557c1a88 in strbuf_addstr strbuf.h:310:2 #5 0x5555557c1d4c in parse_list_objects_filter list-objects-filter-options.c:261:3 #6 0x555555885ead in handle_revision_pseudo_opt revision.c:2899:3 #7 0x555555884e20 in setup_revisions revision.c:3014:11 #8 0x5555556c4b42 in cmd_rev_list builtin/rev-list.c:588:9 #9 0x5555555ec5e3 in run_builtin git.c:483:11 #10 0x5555555eb1e4 in handle_builtin git.c:749:13 #11 0x5555555ec001 in run_argv git.c:819:4 #12 0x5555555eaf94 in cmd_main git.c:954:19 #13 0x5555556fd569 in main common-main.c:64:11 #14 0x7ffff7ca714d in __libc_start_call_main (.../lib/libc.so.6+0x2a14d) #15 0x7ffff7ca7208 in __libc_start_main@GLIBC_2.2.5 (.../libc.so.6+0x2a208) #16 0x5555555ad064 in _start (git+0x59064)This leak is valid, as we call
die()and do not clean up the memory at all. But what's curious is that this is the only leak reported, because we don't clean up any other allocated memory, either, and I have no idea why the leak sanitizer treats this buffer specially.In any case, we can work around the leak by shuffling things around a bit. Instead of calling
gently_parse_list_objects_filter()and dying after we have modified the filter spec, we simply do so beforehand. Like this we don't allocate the buffer in the error case, which makes the reported leak go away.It's not pretty, but it manages to make t6112 leak free.
Signed-off-by Patrick Steinhardt <[email protected]>--- list-objects-filter-options.c | 17 +++++++---------- t/t6112-rev-list-filters-objects.sh | 1 + 2 files changed, 8 insertions(+), 10 deletions(-)list-objects-filter-options.c +2
-5@@ -252,16 +252,14 @@ void parse_list_objects_filter( const char *arg) {struct strbuf errbuf =STRBUF_INIT ; -int parse_error;if (!filter_options->filter_spec.buf)BUG ("filter_options not properly initialized" );if (!filter_options->choice) { +if (gently_parse_list_objects_filter (filter_options, arg, &errbuf)) +die ("%s" , errbuf.buf);strbuf_addstr (&filter_options->filter_spec, arg); - - parse_error =gently_parse_list_objects_filter ( - filter_options, arg, &errbuf); }else {struct list_objects_filter_options *sub;@@ -271,18 +269,17 @@ void parse_list_objects_filter(*/ transform_to_combine_type (filter_options); -strbuf_addch (&filter_options->filter_spec,'+' );
list-objects-filter-options.c void parse_list_objects_filter( · line 278
Quoted 10 lines
list-objects-filter-options.c +2
-3-filter_spec_append_urlencode (filter_options, arg);ALLOC_GROW_BY (filter_options->sub, filter_options->sub_nr,1 , filter_options->sub_alloc); sub = &filter_options->sub[filter_options->sub_nr -1 ];list_objects_filter_init (sub); - parse_error =gently_parse_list_objects_filter (sub, arg, - &errbuf); +if (gently_parse_list_objects_filter (sub, arg, &errbuf)) +die ("%s" , errbuf.buf);
Do we actually have a test hitting this code path? I wanted to figure out why filter_options->sub wasn't leaky (I assume that's what you're talking about in your commit message), but I wasn't able to reproduce a scenario where we should die here.
--
Toon
Re: [PATCH 21/21] list-objects-filter-options: work around reported leak on error
Quoted 34 lines · Toon Claes
Quoted 33 lines · Patrick Steinhardt
list-objects-filter-options.c +2
-6@@ -252,16 +252,14 @@ void parse_list_objects_filter( const char *arg) {struct strbuf errbuf =STRBUF_INIT ; -int parse_error;if (!filter_options->filter_spec.buf)BUG ("filter_options not properly initialized" );if (!filter_options->choice) { +if (gently_parse_list_objects_filter (filter_options, arg, &errbuf)) +die ("%s" , errbuf.buf);strbuf_addstr (&filter_options->filter_spec, arg); - - parse_error =gently_parse_list_objects_filter ( - filter_options, arg, &errbuf); }else {struct list_objects_filter_options *sub;@@ -271,18 +269,17 @@ void parse_list_objects_filter(*/ transform_to_combine_type (filter_options); -strbuf_addch (&filter_options->filter_spec,'+' ); -filter_spec_append_urlencode (filter_options, arg);ALLOC_GROW_BY (filter_options->sub, filter_options->sub_nr,1 , filter_options->sub_alloc); sub = &filter_options->sub[filter_options->sub_nr -1 ];
Quoted 10 lines
list-objects-filter-options.c void parse_list_objects_filter( · line 278
Quoted 5 lines
list-objects-filter-options.c +2
-2list_objects_filter_init (sub); - parse_error =gently_parse_list_objects_filter (sub, arg, - &errbuf); +if (gently_parse_list_objects_filter (sub, arg, &errbuf)) +die ("%s" , errbuf.buf);Do we actually have a test hitting this code path? I wanted to figure out why
filter_options->subwasn't leaky (I assume that's what you're talking about in your commit message), but I wasn't able to reproduce a scenario where we should die here.
You only refer to the second hunk, right? I couldn't find any code paths hitting this, so this is more of a "Let's massage this code in the same way" change. I don't want the next person to go through the same rabbit hole as I had to go through :)
Patrick
Quoted 7 lines · Patrick Steinhardt
Hi,
here's another round of memory leak fixes. With this series we're down to 10 test suites which are failing with the leak sanitizer. There are two patch series in flight [1][2] that fix three more test suites, so in total we're down to 7 test suites that we'll still have to fix up. So:
we're almost done!
I was just looking through Toon's responses lower down in the thread, and realized that I never picked this one up.
I think that makes sense... since this came on 11 October, which is when Junio signed off for vacation. I think that we each assumed that the other would (or had) pick it up.
In any case, this is now in my tree as 'ps/leakfixes-part-9'. Sorry for the wait.
Thanks,
Taylor
Quoted 12 lines · Taylor Blau
Quoted 7 lines · Patrick Steinhardt
Hi,
here's another round of memory leak fixes. With this series we're down to 10 test suites which are failing with the leak sanitizer. There are two patch series in flight [1][2] that fix three more test suites, so in total we're down to 7 test suites that we'll still have to fix up. So:
we're almost done!I was just looking through Toon's responses lower down in the thread, and realized that I never picked this one up.
Quoted 6 lines
I think that makes sense... since this came on 11 October, which is when Junio signed off for vacation. I think that we each assumed that the other would (or had) pick it up.
In any case, this is now in my tree as 'ps/leakfixes-part-9'. Sorry for the wait.
No worries, I'm happy that you keep things moving in the first place. Thanks!
Patrick
v2 cover Memory leak fixes (pt.9)
In reply to Patrick SteinhardtHi,
this is the second version of my 9th series of memory leak fixes.
Changes compared to v1:
- Split up the trailer fixes into two separate patches so that we can explain them standalone.
- Adapt the second trailer memory leak fix to instead pull up the strbufs out of the loop such that we can reuse them. This makes the code flow more naturally and optimizes the loop.
Thanks!
Patrick
Patrick Steinhardt (22):
builtin/ls-remote: plug leaking server options t/helper: fix leaks in "reach" test tool
grep: fix leak in grep_splice_or()
builtin/grep: fix leak with --max-count=0
revision: fix leaking bloom filters
diff-lib: fix leaking diffopts in do_diff_cache()
pretty: clear signature check
upload-pack: fix leaking URI protocols builtin/commit: fix leaking change data contents
trailer: fix leaking trailer values
trailer: fix leaking strbufs when formatting trailers builtin/commit: fix leaking cleanup config
transport-helper: fix leaking import/export marks builtin/tag: fix leaking key ID on failure to sign
combine-diff: fix leaking lost lines
dir: release untracked cache data
sparse-index: correctly free EWAH contents
t/helper: stop re-initialization of the_repository t/helper: fix leaking buffer in "dump-untracked-cache"
dir: fix leak when parsing "status.showUntrackedFiles" builtin/merge: release outbut buffer after performing merge list-objects-filter-options: work around reported leak on error
builtin/commit.c | 26 +++++++++++++++++------ builtin/grep.c | 13 +++++++++--- builtin/ls-remote.c | 1 + builtin/merge.c | 1 + builtin/tag.c | 2 +- combine-diff.c | 2 +- diff-lib.c | 1 + dir.c | 12 +++++++++-- grep.c | 1 + list-objects-filter-options.c | 17 ++++++--------- pretty.c | 1 + revision.c | 5 +++++ sparse-index.c | 7 ++++-- t/helper/test-dump-untracked-cache.c | 2 ++ t/helper/test-reach.c | 10 +++++++++ t/helper/test-read-cache.c | 2 -- t/t4038-diff-combined.sh | 1 + t/t4202-log.sh | 1 + t/t4216-log-bloom.sh | 1 + t/t5702-protocol-v2.sh | 1 + t/t5801-remote-helpers.sh | 1 + t/t6112-rev-list-filters-objects.sh | 1 + t/t6424-merge-unrelated-index-changes.sh | 1 + t/t6600-test-reach.sh | 1 + t/t7004-tag.sh | 1 + t/t7031-verify-tag-signed-ssh.sh | 1 + t/t7063-status-untracked-cache.sh | 1 + t/t7500-commit-template-squash-signoff.sh | 1 + t/t7502-commit-porcelain.sh | 1 + t/t7510-signed-commit.sh | 1 + t/t7513-interpret-trailers.sh | 1 + t/t7519-status-fsmonitor.sh | 1 + t/t7528-signed-commit-ssh.sh | 1 + t/t7610-mergetool.sh | 1 + t/t7810-grep.sh | 1 + trailer.c | 25 +++++++++++++++------- transport-helper.c | 2 ++ upload-pack.c | 1 + 38 files changed, 115 insertions(+), 35 deletions(-)
Range-diff against v1:
1: 89b66411354 = 1: 89b66411354 builtin/ls-remote: plug leaking server options 2: 1c42e194b20 = 2: 1c42e194b20 t/helper: fix leaks in "reach" test tool 3: cb4eee37b40 = 3: cb4eee37b40 grep: fix leak in `grep_splice_or()` 4: 6b2c8842ef5 = 4: 6b2c8842ef5 builtin/grep: fix leak with `--max-count=0` 5: 7527b31a28f = 5: 7527b31a28f revision: fix leaking bloom filters 6: 60af98cb2c7 = 6: 60af98cb2c7 diff-lib: fix leaking diffopts in `do_diff_cache()` 7: 5d5f6867f91 = 7: 5d5f6867f91 pretty: clear signature check 8: 0d503e40194 = 8: 0d503e40194 upload-pack: fix leaking URI protocols 9: 9f967dfe5d5 = 9: 9f967dfe5d5 builtin/commit: fix leaking change data contents -: ----------- > 10: e3ffd59123f trailer: fix leaking trailer values 10: ca5370d572d ! 11: 5b851453bce trailer: fix leaking trailer values
@@ Metadata
Author: Patrick Steinhardt <[email protected]>
## Commit message ##
- trailer: fix leaking trailer values
+ trailer: fix leaking strbufs when formatting trailers
- Fix leaking trailer values when replacing the value with a command or
- when the token value is empty.
+ We are populating, but never releasing two string buffers in
+ format_trailers(), causing a memory leak. Plug this leak by lifting
+ those buffers outside of the loop and releasing them on function return.
+ This fixes the memory leaks, but also optimizes the loop as we don't
+ have to reallocate the buffers on every single iteration.
@@ t/t7513-interpret-trailers.sh
# When we want one trailing space at the end of each line, let's use sed
## trailer.c ##
-@@ trailer.c: static char *apply_command(struct conf_info *conf, const char *arg)
- static void apply_item_command(struct trailer_item *in_tok, struct arg_item *arg_tok) +@@ trailer.c: void format_trailers(const struct process_trailer_options *opts,
+ struct list_head *trailers,
+ struct strbuf *out)
{
- if (arg_tok->conf.command || arg_tok->conf.cmd) {
-- const char *arg;
-+ char *value_to_free = NULL;
-+ char *arg;
-+
- if (arg_tok->value && arg_tok->value[0]) {
-- arg = arg_tok->value;
-+ arg = (char *)arg_tok->value;
- } else {
- if (in_tok && in_tok->value)
- arg = xstrdup(in_tok->value);
- else
- arg = xstrdup("");
-+ value_to_free = arg_tok->value;
- }
-+
- arg_tok->value = apply_command(&arg_tok->conf, arg);
-- free((char *)arg);
-+
-+ free(value_to_free);
-+ free(arg);
- }
- }
++ struct strbuf tok = STRBUF_INIT;
++ struct strbuf val = STRBUF_INIT;
+ size_t origlen = out->len;
+ struct list_head *pos;
+ struct trailer_item *item;
-@@ trailer.c: void format_trailers(const struct process_trailer_options *opts,
- if (item->token) {
- struct strbuf tok = STRBUF_INIT;
- struct strbuf val = STRBUF_INIT;
+
+ list_for_each(pos, trailers) {
+ item = list_entry(pos, struct trailer_item, list);
+ if (item->token) {
+- struct strbuf tok = STRBUF_INIT;
+- struct strbuf val = STRBUF_INIT;
++ strbuf_reset(&tok);
strbuf_addstr(&tok, item->token);
++ strbuf_reset(&val);
strbuf_addstr(&val, item->value);
-@@ trailer.c: void format_trailers(const struct process_trailer_options *opts,
- * corresponding value).
- */
- if (opts->trim_empty && !strlen(item->value))
-- continue;
-+ goto next;
-
- if (!opts->filter || opts->filter(&tok, opts->filter_data)) {
- if (opts->separator && out->len != origlen)
+ /*
@@ trailer.c: void format_trailers(const struct process_trailer_options *opts,
if (!opts->separator)
strbuf_addch(out, '\n');
}
-+
-+next:
- strbuf_release(&tok);
- strbuf_release(&val);
+- strbuf_release(&tok);
+- strbuf_release(&val);
-
} else if (!opts->only_trailers) {
if (opts->separator && out->len != origlen) {
strbuf_addbuf(out, opts->separator);
+@@ trailer.c: void format_trailers(const struct process_trailer_options *opts,
+ strbuf_addch(out, '\n');
+ }
+ }
++
++ strbuf_release(&tok);
++ strbuf_release(&val);
+ }
+
+ void format_trailers_from_commit(const struct process_trailer_options *opts,
11: 8ca4344ed86 = 12: 60c3f6146f3 builtin/commit: fix leaking cleanup config 12: 16d969ed7b1 = 13: ea81cd8db86 transport-helper: fix leaking import/export marks 13: 9e4bc5bf73f = 14: b700ab9079f builtin/tag: fix leaking key ID on failure to sign 14: 8d305d9b1c8 = 15: 76bbcb3fe30 combine-diff: fix leaking lost lines 15: f977a033cf4 = 16: d6b96a4012d dir: release untracked cache data 16: 170cc61edaa = 17: 3aa6bac5079 sparse-index: correctly free EWAH contents 17: 8e10ee844c6 = 18: 132fe750906 t/helper: stop re-initialization of `the_repository` 18: 71fd1c76b8a = 19: b8b702eeb28 t/helper: fix leaking buffer in "dump-untracked-cache" 19: 8ccc246432d = 20: ad309ac1b37 dir: fix leak when parsing "status.showUntrackedFiles" 20: bc2206aa47e = 21: 5e243f9ee53 builtin/merge: release outbut buffer after performing merge 21: 6a2baf0d3e5 = 22: b75376e3725 list-objects-filter-options: work around reported leak on error
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
PATCH v2 1/22 builtin/ls-remote: plug leaking server options
The server options populated via OPT_STRING_LIST() is never cleared, causing a memory leak. Plug it.
This leak is exposed by t5702, but plugging it alone does not make the whole test suite pass.
--- builtin/ls-remote.c | 1 + 1 file changed, 1 insertion(+)
builtin/ls-remote.c +1 -0
@@ -166,6 +166,7 @@ int cmd_ls_remote(int argc, status =0 ;/* we found something */ } +string_list_clear (&server_options,0 );ref_sorting_release (sorting);ref_array_clear (&ref_array);if (transport_disconnect (transport))
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
Re: [PATCH v2 01/22] builtin/ls-remote: plug leaking server options
Quoted 1 line · Patrick Steinhardt
The server options populated via
OPT_STRING_LIST()is never cleared,
s/is/are/
Quoted 6 lines
causing a memory leak. Plug it.
This leak is exposed by t5702, but plugging it alone does not make the whole test suite pass.
Signed-off-by Patrick Steinhardt <[email protected]>
Re: [PATCH v2 01/22] builtin/ls-remote: plug leaking server options
Quoted 4 lines · Justin Tobler
Quoted 1 line · Patrick Steinhardt
The server options populated via
OPT_STRING_LIST()is never cleared,s/is/are/
I guess “is” was chosen because “the list is”.
The [list] populated via OPT_STRING_LIST() is never cleared,
Signature · 1 line
Kristoffer Haugsbakk
PATCH v2 2/22 t/helper: fix leaks in "reach" test tool
In reply to Patrick SteinhardtThe "reach" test tool doesn't bother to clean up any of its allocated resources, causing various leaks. Plug them.
--- t/helper/test-reach.c | 10 ++++++++++ t/t6600-test-reach.sh | 1 + 2 files changed, 11 insertions(+)
t/helper/test-reach.c +10 -0
@@ -127,10 +127,12 @@ int cmd__reach(int ac, const char **av)exit (128 );printf ("%s(A,X):\n" , av[1 ]);print_sorted_commit_ids (list); +free_commit_list (list); }else if (!strcmp (av[1 ],"reduce_heads" )) {struct commit_list *list =reduce_heads (X );printf ("%s(X):\n" , av[1 ]);print_sorted_commit_ids (list); +free_commit_list (list); }else if (!strcmp (av[1 ],"can_all_from_reach" )) {printf ("%s(X,Y):%d\n" , av[1 ],can_all_from_reach (X ,Y ,1 )); }else if (!strcmp (av[1 ],"can_all_from_reach_with_flag" )) {
@@ -153,6 +155,7 @@ int cmd__reach(int ac, const char **av) filter.with_commit_tag_algo =0 ;printf ("%s(_,A,X,_):%d\n" , av[1 ],commit_contains (&filter,A ,X , &cache)); +clear_contains_cache (&cache); }else if (!strcmp (av[1 ],"get_reachable_subset" )) {const int reachable_flag =1 ;int i, count =0 ;
@@ -176,7 +179,14 @@ int cmd__reach(int ac, const char **av)die (_ ("too many commits marked reachable" ));print_sorted_commit_ids (list); +free_commit_list (list); } +object_array_clear (&X_obj); +strbuf_release (&buf); +free_commit_list (X ); +free_commit_list (Y ); +free (X_array); +free (Y_array);return 0 ; }
t/t6600-test-reach.sh +1 -0
@@ -2,6 +2,7 @@ test_description='basic commit reachability tests' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Construct a grid-like commit graph with points (x,y)
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
PATCH v2 3/22 grep: fix leak in `grep_splice_or()`
In reply to Patrick SteinhardtIn grep_splice_or() we search for the next TRUE node in our tree of grep exrpessions and replace it with the given new expression. But we don't free the old node, which causes a memory leak. Plug it.
This leak is exposed by t7810, but plugging it alone isn't sufficient to make the test suite pass.
--- grep.c | 1 + 1 file changed, 1 insertion(+)
grep.c +1 -0
@@ -756,6 +756,7 @@ static struct grep_expr *grep_splice_or(struct grep_expr *x, struct grep_expr *yassert (x->node ==GREP_NODE_OR );if (x->u.binary.right && x->u.binary.right->node ==GREP_NODE_TRUE ) { +free (x->u.binary.right); x->u.binary.right = y;break ; }
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
Re: [PATCH v2 03/22] grep: fix leak in `grep_splice_or()`
Quoted 2 lines · Patrick Steinhardt
In
grep_splice_or()we search for the nextTRUEnode in our tree of grep exrpessions and replace it with the given new expression. But we
s/exrpessions/expressions/
PATCH v2 4/22 builtin/grep: fix leak with `--max-count=0`
In reply to Patrick SteinhardtWhen executing with --max-count=0 we'll return early from git-grep(1) without performing any cleanup, which causes memory leaks. Plug these.
--- builtin/grep.c | 13 ++++++++++--- t/t7810-grep.sh | 1 + 2 files changed, 11 insertions(+), 3 deletions(-)
builtin/grep.c +10 -3
@@ -906,6 +906,7 @@ int cmd_grep(int argc,int dummy;int use_index =1 ;int allow_revs; +int ret;struct option options[] = { OPT_BOOL(0, "cached", &cached,
@@ -1172,8 +1173,10 @@ int cmd_grep(int argc,* Optimize out the case where the amount of matches is limited to zero. * We do this to keep results consistent with GNU grep(1). */ -if (opt.max_count ==0 ) -return 1 ; +if (opt.max_count ==0 ) { + ret =1 ; + goto out; + }if (show_in_pager) {if (num_threads >1 )
@@ -1267,10 +1270,14 @@ int cmd_grep(int argc, hit |=wait_all ();if (hit && show_in_pager)run_pager (&opt, prefix); + + ret = !hit; + +out:clear_pathspec (&pathspec);string_list_clear (&path_list,0 );free_grep_patterns (&opt);object_array_clear (&list);free_repos (); -return !hit; +return ret; }
t/t7810-grep.sh +1 -0
@@ -9,6 +9,7 @@ test_description='git grep various. GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_invalid_grep_expression() {
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
PATCH v2 5/22 revision: fix leaking bloom filters
In reply to Patrick SteinhardtThe memory allocated by prepare_to_use_bloom_filter() is not released by release_revisions(), causing a memory leak. Plug it.
--- revision.c | 5 +++++ t/t4216-log-bloom.sh | 1 + 2 files changed, 6 insertions(+)
revision.c +5 -0
@@ -3227,6 +3227,11 @@ void release_revisions(struct rev_info *revs)clear_decoration (&revs->treesame, free);line_log_free (revs);oidset_clear (&revs->missing_commits); + +for (int i =0 ; i < revs->bloom_keys_nr; i++) +clear_bloom_key (&revs->bloom_keys[i]); +FREE_AND_NULL (revs->bloom_keys); + revs->bloom_keys_nr =0 ; }static void add_child (struct rev_info *revs,struct commit *parent,struct commit *child)
t/t4216-log-bloom.sh +1 -0
@@ -4,6 +4,7 @@ test_description='git log for a path with Bloom filters' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-chunk.sh
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
PATCH v2 6/22 diff-lib: fix leaking diffopts in `do_diff_cache()`
In reply to Patrick SteinhardtIn do_diff_cache() we initialize a new rev_info and then overwrite its diffopt with a user-provided set of options. This can leak memory because repo_init_revisions() may end up allocating memory for the diffopt itself depending on the configuration. And as that field is overwritten we won't ever free that.
Plug the memory leak by releasing the diffopts before we overwrite them.
--- diff-lib.c | 1 + t/t7610-mergetool.sh | 1 + 2 files changed, 2 insertions(+)
diff-lib.c +1 -0
@@ -661,6 +661,7 @@ int do_diff_cache(const struct object_id *tree_oid, struct diff_options *opt)repo_init_revisions (opt->repo, &revs,NULL );copy_pathspec (&revs.prune_data, &opt->pathspec); +diff_free (&revs.diffopt); revs.diffopt = *opt; revs.diffopt.no_free =1 ;
t/t7610-mergetool.sh +1 -0
@@ -10,6 +10,7 @@ Testing basic merge tool invocation' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # All the mergetool test work by checking out a temporary branch based
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
PATCH v2 7/22 pretty: clear signature check
In reply to Patrick SteinhardtThe signature check in of the formatting context is never getting released. Fix this to plug the resulting memory leak.
--- pretty.c | 1 + t/t4202-log.sh | 1 + t/t7031-verify-tag-signed-ssh.sh | 1 + t/t7510-signed-commit.sh | 1 + t/t7528-signed-commit-ssh.sh | 1 + 5 files changed, 5 insertions(+)
pretty.c +1 -0
@@ -2032,6 +2032,7 @@ void repo_format_commit_message(struct repository *r,free (context.commit_encoding);repo_unuse_commit_buffer (r, commit, context.message); +signature_check_clear (&context.signature_check); } static void pp_header(struct pretty_print_context *pp,
t/t4202-log.sh +1 -0
@@ -5,6 +5,7 @@ test_description='git log' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY/lib-gpg.sh" . "$TEST_DIRECTORY/lib-terminal.sh"
t/t7031-verify-tag-signed-ssh.sh +1 -0
@@ -4,6 +4,7 @@ test_description='signed tag tests' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY/lib-gpg.sh"
t/t7510-signed-commit.sh +1 -0
@@ -4,6 +4,7 @@ test_description='signed commit tests' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh GNUPGHOME_NOT_USED=$GNUPGHOME . "$TEST_DIRECTORY/lib-gpg.sh"
t/t7528-signed-commit-ssh.sh +1 -0
@@ -4,6 +4,7 @@ test_description='ssh signed commit tests' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh GNUPGHOME_NOT_USED=$GNUPGHOME . "$TEST_DIRECTORY/lib-gpg.sh"
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
Re: [PATCH v2 07/22] pretty: clear signature check
Quoted 1 line · Patrick Steinhardt
The signature check in of the formatting context is never getting
s/in of/in/
Quoted 16 lines
released. Fix this to plug the resulting memory leak.
Signed-off-by Patrick Steinhardt <[email protected]>--- pretty.c | 1 + t/t4202-log.sh | 1 + t/t7031-verify-tag-signed-ssh.sh | 1 + t/t7510-signed-commit.sh | 1 + t/t7528-signed-commit-ssh.sh | 1 + 5 files changed, 5 insertions(+)pretty.c +0
-0
pretty.c void repo_format_commit_message(struct repository *r, · line 2035
Quoted 4 lines
pretty.c +1
-0@@ -2032,6 +2032,7 @@ void repo_format_commit_message(struct repository *r,free (context.commit_encoding);repo_unuse_commit_buffer (r, commit, context.message); +signature_check_clear (&context.signature_check);
Initially I was thinking it might be nice to see the format_commit_context cleanup extracted to its own function, but being that it only has a single use internal to "pretty.c" it probably isn't worth it.
Quoted 3 lines
pretty.c +0
-0} static void pp_header(struct pretty_print_context *pp,
PATCH v2 8/22 upload-pack: fix leaking URI protocols
In reply to Patrick SteinhardtWe don't clear struct upload_pack::uri_protocols, which causes a memory leak. Fix this.
--- t/t5702-protocol-v2.sh | 1 + upload-pack.c | 1 + 2 files changed, 2 insertions(+)
t/t5702-protocol-v2.sh +1 -0
@@ -7,6 +7,7 @@ TEST_NO_CREATE_REPO=1 GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Test protocol v2 with 'git://' transport
upload-pack.c +1 -0
@@ -166,6 +166,7 @@ static void upload_pack_data_clear(struct upload_pack_data *data)object_array_clear (&data->extra_edge_obj);list_objects_filter_release (&data->filter_options);string_list_clear (&data->allowed_filters,0 ); +string_list_clear (&data->uri_protocols,0 );free ((char *)data->pack_objects_hook); }
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
PATCH v2 9/22 builtin/commit: fix leaking change data contents
In reply to Patrick SteinhardtWhile we free the worktree change data, we never free its contents. Fix this.
--- builtin/commit.c | 9 ++++++++- t/t7500-commit-template-squash-signoff.sh | 1 + 2 files changed, 9 insertions(+), 1 deletion(-)
builtin/commit.c +8 -1
@@ -728,6 +728,13 @@ static void prepare_amend_commit(struct commit *commit, struct strbuf *sb,repo_unuse_commit_buffer (the_repository, commit, buffer); } +static void change_data_free(void *util, const char *str UNUSED) +{ +struct wt_status_change_data *d = util; +free (d->rename_source); +free (d); +} + static int prepare_to_commit(const char *index_file, const char *prefix, struct commit *current_head, struct wt_status *s,
@@ -991,7 +998,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix, s->use_color =0 ; committable =run_status (s->fp, index_file, prefix,1 , s); s->use_color = saved_color_setting; -string_list_clear (&s->change,1 ); +string_list_clear_func (&s->change, change_data_free); }else {struct object_id oid;const char *parent ="HEAD" ;
t/t7500-commit-template-squash-signoff.sh +1 -0
@@ -7,6 +7,7 @@ test_description='git commit Tests for template, signoff, squash and -F functions.' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-rebase.sh
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
Re: [PATCH v2 09/22] builtin/commit: fix leaking change data contents
Quoted 2 lines · Patrick Steinhardt
While we free the worktree change data, we never free its contents. Fix this.
Ah, so this worktree change data was part of the string list and was being freed, but it also had memory allocations itself that were not. Makes sense :)
Quoted 50 lines
Signed-off-by Patrick Steinhardt <[email protected]>--- builtin/commit.c | 9 ++++++++- t/t7500-commit-template-squash-signoff.sh | 1 + 2 files changed, 9 insertions(+), 1 deletion(-)builtin/commit.c +8
-1@@ -728,6 +728,13 @@ static void prepare_amend_commit(struct commit *commit, struct strbuf *sb,repo_unuse_commit_buffer (the_repository, commit, buffer); } +static void change_data_free(void *util, const char *str UNUSED) +{ +struct wt_status_change_data *d = util; +free (d->rename_source); +free (d); +} + static int prepare_to_commit(const char *index_file, const char *prefix, struct commit *current_head, struct wt_status *s,@@ -991,7 +998,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix, s->use_color =0 ; committable =run_status (s->fp, index_file, prefix,1 , s); s->use_color = saved_color_setting; -string_list_clear (&s->change,1 ); +string_list_clear_func (&s->change, change_data_free); }else {struct object_id oid;const char *parent ="HEAD" ;t/t7500-commit-template-squash-signoff.sh +1
-0@@ -7,6 +7,7 @@ test_description='git commit Tests for template, signoff, squash and -F functions.' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-rebase.shSignature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
PATCH v2 10/22 trailer: fix leaking trailer values
In reply to Patrick SteinhardtFix leaking trailer values when replacing the value with a command or when the token value is empty.
This leak is exposed by t7513, but plugging it does not make the whole test suite pass.
--- trailer.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)
trailer.c +9 -3
@@ -249,17 +249,23 @@ static char *apply_command(struct conf_info *conf, const char *arg)static void apply_item_command (struct trailer_item *in_tok,struct arg_item *arg_tok) {if (arg_tok->conf.command || arg_tok->conf.cmd) { -const char *arg; +char *value_to_free =NULL ; +char *arg; +if (arg_tok->value && arg_tok->value[0 ]) { - arg = arg_tok->value; + arg = (char *)arg_tok->value; }else {if (in_tok && in_tok->value) arg =xstrdup (in_tok->value);else arg =xstrdup ("" ); + value_to_free = arg_tok->value; } + arg_tok->value =apply_command (&arg_tok->conf, arg); -free ((char *)arg); + +free (value_to_free); +free (arg); } }
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
Re: [PATCH v2 10/22] trailer: fix leaking trailer values
Quoted 16 lines · Patrick Steinhardt
Fix leaking trailer values when replacing the value with a command or when the token value is empty.
This leak is exposed by t7513, but plugging it does not make the whole test suite pass.
Signed-off-by Patrick Steinhardt <[email protected]>--- trailer.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)trailer.c +0
-0
trailer.c static char *apply_command(struct conf_info *conf, const char *arg) · line 256
Quoted 10 lines
trailer.c +4
-2@@ -249,17 +249,23 @@ static char *apply_command(struct conf_info *conf, const char *arg)static void apply_item_command (struct trailer_item *in_tok,struct arg_item *arg_tok) {if (arg_tok->conf.command || arg_tok->conf.cmd) { -const char *arg; +char *value_to_free =NULL ; +char *arg; +if (arg_tok->value && arg_tok->value[0 ]) { - arg = arg_tok->value; + arg = (char *)arg_tok->value;
Naive question, is this cast not redundant? From looking at struct arg_item, value already looks to be this type.
Quoted 20 lines
trailer.c +5
-1}else {if (in_tok && in_tok->value) arg =xstrdup (in_tok->value);else arg =xstrdup ("" ); + value_to_free = arg_tok->value; } + arg_tok->value =apply_command (&arg_tok->conf, arg); -free ((char *)arg); + +free (value_to_free); +free (arg); } }Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
Re: [PATCH v2 10/22] trailer: fix leaking trailer values
Quoted 22 lines · Justin Tobler
Quoted 21 lines · Patrick Steinhardt
Fix leaking trailer values when replacing the value with a command or when the token value is empty.
This leak is exposed by t7513, but plugging it does not make the whole test suite pass.
Signed-off-by Patrick Steinhardt <[email protected]>--- trailer.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)trailer.c +1
-1@@ -249,17 +249,23 @@ static char *apply_command(struct conf_info *conf, const char *arg)static void apply_item_command (struct trailer_item *in_tok,struct arg_item *arg_tok) {if (arg_tok->conf.command || arg_tok->conf.cmd) { -const char *arg; +char *value_to_free =NULL ;
Quoted 8 lines
trailer.c static char *apply_command(struct conf_info *conf, const char *arg) · line 256
Quoted 5 lines
trailer.c +3
-1+char *arg; +if (arg_tok->value && arg_tok->value[0 ]) { - arg = arg_tok->value; + arg = (char *)arg_tok->value;Naive question, is this cast not redundant? From looking at
struct arg_item,valuealready looks to be this type.
Indeed it is, good catch!
Patrick
PATCH v2 11/22 trailer: fix leaking strbufs when formatting trailers
In reply to Patrick SteinhardtWe are populating, but never releasing two string buffers in format_trailers(), causing a memory leak. Plug this leak by lifting those buffers outside of the loop and releasing them on function return. This fixes the memory leaks, but also optimizes the loop as we don't have to reallocate the buffers on every single iteration.
--- t/t7513-interpret-trailers.sh | 1 + trailer.c | 13 ++++++++----- 2 files changed, 9 insertions(+), 5 deletions(-)
t/t7513-interpret-trailers.sh +1 -0
@@ -5,6 +5,7 @@ test_description='git interpret-trailers' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # When we want one trailing space at the end of each line, let's use sed
trailer.c +8 -5
@@ -1111,16 +1111,19 @@ void format_trailers(const struct process_trailer_options *opts,struct list_head *trailers, struct strbuf *out) { + struct strbuf tok = STRBUF_INIT; +struct strbuf val =STRBUF_INIT ;size_t origlen = out->len;struct list_head *pos;struct trailer_item *item; +list_for_each (pos, trailers) { item = list_entry(pos, struct trailer_item, list);if (item->token) { -struct strbuf tok =STRBUF_INIT ; -struct strbuf val =STRBUF_INIT ; +strbuf_reset (&tok);strbuf_addstr (&tok, item->token); +strbuf_reset (&val);strbuf_addstr (&val, item->value);/*
@@ -1151,9 +1154,6 @@ void format_trailers(const struct process_trailer_options *opts,if (!opts->separator)strbuf_addch (out,'\n' ); } -strbuf_release (&tok); -strbuf_release (&val); - }else if (!opts->only_trailers) {if (opts->separator && out->len != origlen) {strbuf_addbuf (out, opts->separator);
@@ -1165,6 +1165,9 @@ void format_trailers(const struct process_trailer_options *opts,strbuf_addch (out,'\n' ); } } + +strbuf_release (&tok); +strbuf_release (&val); } void format_trailers_from_commit(const struct process_trailer_options *opts,
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
Re: [PATCH v2 11/22] trailer: fix leaking strbufs when formatting trailers
Quoted 38 lines · Patrick Steinhardt
We are populating, but never releasing two string buffers in
format_trailers(), causing a memory leak. Plug this leak by lifting those buffers outside of the loop and releasing them on function return. This fixes the memory leaks, but also optimizes the loop as we don't have to reallocate the buffers on every single iteration.Signed-off-by Patrick Steinhardt <[email protected]>--- t/t7513-interpret-trailers.sh | 1 + trailer.c | 13 ++++++++----- 2 files changed, 9 insertions(+), 5 deletions(-)t/t7513-interpret-trailers.sh +1
-0@@ -5,6 +5,7 @@ test_description='git interpret-trailers' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # When we want one trailing space at the end of each line, let's use sedtrailer.c +2
-0@@ -1111,16 +1111,19 @@ void format_trailers(const struct process_trailer_options *opts,struct list_head *trailers, struct strbuf *out) { + struct strbuf tok = STRBUF_INIT; +struct strbuf val =STRBUF_INIT ;size_t origlen = out->len;struct list_head *pos;struct trailer_item *item;
trailer.c void format_trailers(const struct process_trailer_options *opts, · line 1127
Quoted 10 lines
trailer.c +3
-2+list_for_each (pos, trailers) { item = list_entry(pos, struct trailer_item, list);if (item->token) { -struct strbuf tok =STRBUF_INIT ; -struct strbuf val =STRBUF_INIT ; +strbuf_reset (&tok);strbuf_addstr (&tok, item->token); +strbuf_reset (&val);strbuf_addstr (&val, item->value);
I have a vague preference towards writing this as:
strbuf_reset(&tok);
strbuf_reset(&val);
strbuf_addstr(&tok, item->token);
strbuf_addstr(&val, item->value);to make clear(er) to the reader that, OK, we are resetting both of these buffers before using them at the beginning of the loop. To me it reads a little clearer seeing both strbuf_reset() calls one right after the other.
I don't feel all that strongly about it, but I also see that you have a couple of small changes you're sitting on from Kristoffer's review, so I figured I'd throw it out there anyway as we are expecting a new round to address those.
Thanks,
Taylor
Re: [PATCH v2 11/22] trailer: fix leaking strbufs when formatting trailers
In reply to Patrick SteinhardtQuoted 5 lines · Patrick Steinhardt
We are populating, but never releasing two string buffers in
format_trailers(), causing a memory leak. Plug this leak by lifting those buffers outside of the loop and releasing them on function return. This fixes the memory leaks, but also optimizes the loop as we don't have to reallocate the buffers on every single iteration.
I see that we were previously calling strbuf_release() inside of the loop. In practice were these never hit? Or just sometimes skipped my the "continue" in the loop? The commit message makes it sound like the former.
Re: [PATCH v2 11/22] trailer: fix leaking strbufs when formatting trailers
Quoted 11 lines · Justin Tobler
Quoted 5 lines · Patrick Steinhardt
We are populating, but never releasing two string buffers in
format_trailers(), causing a memory leak. Plug this leak by lifting those buffers outside of the loop and releasing them on function return. This fixes the memory leaks, but also optimizes the loop as we don't have to reallocate the buffers on every single iteration.I see that we were previously calling
strbuf_release()inside of the loop. In practice were these never hit? Or just sometimes skipped my the "continue" in the loop? The commit message makes it sound like the former.
True, "never" is definitely too strong of a wording here. Will adapt, thanks!
Patrick
PATCH v2 12/22 builtin/commit: fix leaking cleanup config
In reply to Patrick SteinhardtThe cleanup string set by the config is leaking when it is being overridden by an option. Fix this by tracking these via two separate variables such that we can free the old value.
--- builtin/commit.c | 17 ++++++++++++----- t/t7502-commit-porcelain.sh | 1 + 2 files changed, 13 insertions(+), 5 deletions(-)
builtin/commit.c +12 -5
@@ -135,7 +135,7 @@ static struct strvec trailer_args = STRVEC_INIT;* is specified explicitly. */ static enum commit_msg_cleanup_mode cleanup_mode; -static char *cleanup_arg; +static char *cleanup_config;static enum commit_whence whence;static int use_editor =1 , include_status =1 ;
@@ -1387,8 +1387,6 @@ static int parse_and_validate_options(int argc, const char *argv[],if (0 <= edit_flag) use_editor = edit_flag; - cleanup_mode =get_cleanup_mode (cleanup_arg, use_editor); -handle_untracked_files_arg (s);if (all && argc >0 )
@@ -1636,8 +1634,10 @@ static int git_commit_config(const char *k, const char *v, include_status =git_config_bool (k, v);return 0 ; } -if (!strcmp (k,"commit.cleanup" )) -return git_config_string (&cleanup_arg, k, v); +if (!strcmp (k,"commit.cleanup" )) { +FREE_AND_NULL (cleanup_config); +return git_config_string (&cleanup_config, k, v); + }if (!strcmp (k,"commit.gpgsign" )) { sign_commit =git_config_bool (k, v) ?"" :NULL ;return 0 ;
@@ -1658,6 +1658,7 @@ int cmd_commit(int argc, struct repository *repo UNUSED) {static struct wt_status s; +static const char *cleanup_arg =NULL ;static struct option builtin_commit_options[] = {OPT__QUIET (&quiet,N_ ("suppress summary after successful commit" )),OPT__VERBOSE (&verbose,N_ ("show diff in commit message template" )),
@@ -1757,6 +1758,12 @@ int cmd_commit(int argc,if (verbose ==-1 ) verbose = (config_commit_verbose <0 ) ?0 : config_commit_verbose; +if (cleanup_arg) { +free (cleanup_config); + cleanup_config =xstrdup (cleanup_arg); + } + cleanup_mode =get_cleanup_mode (cleanup_config, use_editor); +if (dry_run)return dry_run_commit (argv, prefix, current_head, &s); index_file =prepare_index (argv, prefix, current_head,0 );
t/t7502-commit-porcelain.sh +1 -0
@@ -5,6 +5,7 @@ test_description='git commit porcelain-ish' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh commit_msg_is () {
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
PATCH v2 13/22 transport-helper: fix leaking import/export marks
In reply to Patrick SteinhardtFix leaking import and export marks for transport helpers.
--- t/t5801-remote-helpers.sh | 1 + transport-helper.c | 2 ++ 2 files changed, 3 insertions(+)
t/t5801-remote-helpers.sh +1 -0
@@ -8,6 +8,7 @@ test_description='Test remote-helper import and export commands' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-gpg.sh
transport-helper.c +2 -0
@@ -399,6 +399,8 @@ static int release_helper(struct transport *transport)int res =0 ;struct helper_data *data = transport->data;refspec_clear (&data->rs); +free (data->import_marks); +free (data->export_marks); res =disconnect_helper (transport);free (transport->data);return res;
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
PATCH v2 14/22 builtin/tag: fix leaking key ID on failure to sign
In reply to Patrick SteinhardtWe do not free the key ID when signing a tag fails. Do so by using the common exit path.
--- builtin/tag.c | 2 +- t/t7004-tag.sh | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)
builtin/tag.c +1 -1
@@ -164,7 +164,7 @@ static int do_sign(struct strbuf *buffer, struct object_id **compat_oid,int ret =-1 ;if (sign_buffer (buffer, &sig, keyid)) -return -1 ; + goto out;if (compat) {const struct git_hash_algo *algo = the_repository->hash_algo;
t/t7004-tag.sh +1 -0
@@ -10,6 +10,7 @@ Tests for operations with tags.' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-gpg.sh . "$TEST_DIRECTORY"/lib-terminal.sh
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
PATCH v2 15/22 combine-diff: fix leaking lost lines
In reply to Patrick SteinhardtThe cnt variable tracks the number of lines in a patch diff. It can happen though that there are no newlines, in which case we'd still end up allocating our array of slines. In fact, we always allocate it with cnt + 2 entries. But when we loop through the array to clear it at the end of this function we only loop until lno < cnt, and thus we may not end up releasing whatever the two extra slines contain.
Plug this memory leak.
--- combine-diff.c | 2 +- t/t4038-diff-combined.sh | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)
combine-diff.c +1 -1
@@ -1220,7 +1220,7 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent, }free (result); -for (lno =0 ; lno < cnt; lno++) { +for (lno =0 ; lno < cnt +2 ; lno++) {if (sline[lno].lost) {struct lline *ll = sline[lno].lost;while (ll) {
t/t4038-diff-combined.sh +1 -0
@@ -5,6 +5,7 @@ test_description='combined diff' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-diff.sh
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
Re: [PATCH v2 15/22] combine-diff: fix leaking lost lines
Quoted 20 lines · Patrick Steinhardt
The
cntvariable tracks the number of lines in a patch diff. It can happen though that there are no newlines, in which case we'd still end up allocating our array ofslines. In fact, we always allocate it withcnt + 2entries. But when we loop through the array to clear it at the end of this function we only loop untillno < cnt, and thus we may not end up releasing whatever the two extraslines contain.Plug this memory leak.
Signed-off-by Patrick Steinhardt <[email protected]>--- combine-diff.c | 2 +- t/t4038-diff-combined.sh | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)combine-diff.c +0
-0
combine-diff.c static void show_patch_diff(struct combine_diff_path *elem, int num_parent, · line 1223
Quoted 5 lines
combine-diff.c +1
-1@@ -1220,7 +1220,7 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent, }free (result); -for (lno =0 ; lno < cnt; lno++) { +for (lno =0 ; lno < cnt +2 ; lno++) {
From looking only at the code, its not very obvious to me where the "+2" comes from. Not really worth a reroll, but it might be nice to leave a note with some context.
Quoted 19 lines
combine-diff.c +0
-0if (sline[lno].lost) {struct lline *ll = sline[lno].lost;while (ll) {t/t4038-diff-combined.sh +1
-0@@ -5,6 +5,7 @@ test_description='combined diff' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-diff.shSignature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
Re: [PATCH v2 15/22] combine-diff: fix leaking lost lines
Quoted 24 lines · Justin Tobler
Quoted 23 lines · Patrick Steinhardt
The
cntvariable tracks the number of lines in a patch diff. It can happen though that there are no newlines, in which case we'd still end up allocating our array ofslines. In fact, we always allocate it withcnt + 2entries. But when we loop through the array to clear it at the end of this function we only loop untillno < cnt, and thus we may not end up releasing whatever the two extraslines contain.Plug this memory leak.
Signed-off-by Patrick Steinhardt <[email protected]>--- combine-diff.c | 2 +- t/t4038-diff-combined.sh | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)combine-diff.c +0
-0@@ -1220,7 +1220,7 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent, }free (result);
Quoted 6 lines
combine-diff.c static void show_patch_diff(struct combine_diff_path *elem, int num_parent, · line 1223
Quoted 2 lines
combine-diff.c +1
-1-for (lno =0 ; lno < cnt; lno++) { +for (lno =0 ; lno < cnt +2 ; lno++) {From looking only at the code, its not very obvious to me where the "+2" comes from. Not really worth a reroll, but it might be nice to leave a note with some context.
The code is quite convoluted and hard to read, agreed. In any case, we call CALLOC_ARRAY(sline, st_add(cnt, 2)), and as a comment further up mentions:
/* Even p_lno[cnt+1] is valid -- that is for the end line number for
* deletion hunk at the end.
*/This explains the +1. The second +1 seems to never be populated and acts more like a sentinel value.
I don't really think it makes a ton of sense to explain why the sline array is overallocated when freeing it, and we already do explain it. But I'll touch up the commit message a bit and sneak in a fix of the above comment to be properly formatted, which also gives the necessary context when reading the diff, only.
Patrick
PATCH v2 16/22 dir: release untracked cache data
In reply to Patrick SteinhardtThere are several cases where we invalidate untracked cache directory entries where we do not free the underlying data, but reset the number of entries. This causes us to leak memory because free_untracked() will not iterate over any potential entries which we still had in the array.
Fix this issue by freeing old entries. The leak is exposed by t7519, but plugging it alone does not make the whole test suite pass.
--- dir.c | 8 ++++++++ 1 file changed, 8 insertions(+)
dir.c +8 -0
@@ -1056,6 +1056,8 @@ static void do_invalidate_gitignore(struct untracked_cache_dir *dir) {int i; dir->valid =0 ; +for (size_t i =0 ; i < dir->untracked_nr; i++) +free (dir->untracked[i]); dir->untracked_nr =0 ;for (i =0 ; i < dir->dirs_nr; i++)do_invalidate_gitignore (dir->dirs[i]);
@@ -1083,6 +1085,8 @@ static void invalidate_directory(struct untracked_cache *uc, uc->dir_invalidated++; dir->valid =0 ; +for (size_t i =0 ; i < dir->untracked_nr; i++) +free (dir->untracked[i]); dir->untracked_nr =0 ;for (i =0 ; i < dir->dirs_nr; i++) dir->dirs[i]->recurse =0 ;
@@ -3573,6 +3577,8 @@ static void write_one_dir(struct untracked_cache_dir *untracked,* for safety.. */ if (!untracked->valid) { +for (size_t i =0 ; i < untracked->untracked_nr; i++) +free (untracked->untracked[i]); untracked->untracked_nr =0 ; untracked->check_only =0 ; }
@@ -3905,6 +3911,8 @@ static void invalidate_one_directory(struct untracked_cache *uc, { uc->dir_invalidated++; ucd->valid =0 ; +for (size_t i =0 ; i < ucd->untracked_nr; i++) +free (ucd->untracked[i]); ucd->untracked_nr =0 ; }
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
PATCH v2 17/22 sparse-index: correctly free EWAH contents
In reply to Patrick SteinhardtWhile we free the fsmonitor_dirty member of struct index_state, we do not free the contents of that EWAH. Do so by using ewah_free() instead of FREE_AND_NULL().
This leak is exposed by t7519, but plugging it alone does not make the test suite pass.
--- sparse-index.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
sparse-index.c +5 -2
@@ -2,6 +2,7 @@#include "git-compat-util.h" #include "environment.h" +#include "ewah/ewok.h" #include "gettext.h" #include "name-hash.h" #include "read-cache-ll.h"
@@ -242,7 +243,8 @@ int convert_to_sparse(struct index_state *istate, int flags)cache_tree_update (istate,0 ); istate->fsmonitor_has_run_once =0 ; -FREE_AND_NULL (istate->fsmonitor_dirty); +ewah_free (istate->fsmonitor_dirty); + istate->fsmonitor_dirty =NULL ;FREE_AND_NULL (istate->fsmonitor_last_update); istate->sparse_index =INDEX_COLLAPSED ;
@@ -438,7 +440,8 @@ void expand_index(struct index_state *istate, struct pattern_list *pl) istate->cache_nr = full->cache_nr; istate->cache_alloc = full->cache_alloc; istate->fsmonitor_has_run_once =0 ; -FREE_AND_NULL (istate->fsmonitor_dirty); +ewah_free (istate->fsmonitor_dirty); + istate->fsmonitor_dirty =NULL ;FREE_AND_NULL (istate->fsmonitor_last_update);strbuf_release (&base);
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
PATCH v2 18/22 t/helper: stop re-initialization of `the_repository`
In reply to Patrick SteinhardtWhile "common-main.c" already initializes the_repository for us, we do so a second time in the "read-cache" test helper. This causes a memory leak because the old repository's contents isn't released.
Stop calling initialize_repository() to plug this leak.
--- t/helper/test-read-cache.c | 2 -- t/t7519-status-fsmonitor.sh | 1 + 2 files changed, 1 insertion(+), 2 deletions(-)
t/helper/test-read-cache.c +0 -2
@@ -11,8 +11,6 @@ int cmd__read_cache(int argc, const char **argv)int i, cnt =1 ;const char *name =NULL ; -initialize_repository (the_repository); -if (argc >1 &&skip_prefix (argv[1 ],"--print-and-refresh=" , &name)) { argc--; argv++;
t/t7519-status-fsmonitor.sh +1 -0
@@ -2,6 +2,7 @@ test_description='git status with file system watcher' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Note, after "git reset --hard HEAD" no extensions exist other than 'TREE'
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
PATCH v2 19/22 t/helper: fix leaking buffer in "dump-untracked-cache"
In reply to Patrick SteinhardtWe never release the local struct strbuf base buffer, thus leaking memory. Fix this leak.
This leak is exposed by t7063, but plugging it alone does not make the whole test suite pass.
--- t/helper/test-dump-untracked-cache.c | 2 ++ 1 file changed, 2 insertions(+)
t/helper/test-dump-untracked-cache.c +2 -0
@@ -68,5 +68,7 @@ int cmd__dump_untracked_cache(int ac UNUSED, const char **av UNUSED)printf ("flags %08x\n" , uc->dir_flags);if (uc->root)dump (uc->root, &base); + +strbuf_release (&base);return 0 ; }
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
PATCH v2 20/22 dir: fix leak when parsing "status.showUntrackedFiles"
In reply to Patrick SteinhardtWe use repo_config_get_string() to read "status.showUntrackedFiles" from the config subsystem. This function allocates the result, but we never free the result after parsing it.
The value never leaves the scope of the calling function, so refactor it to instead use repo_config_get_string_tmp(), which does not hand over ownership to the caller.
--- dir.c | 4 ++-- t/t7063-status-untracked-cache.sh | 1 + 2 files changed, 3 insertions(+), 2 deletions(-)
dir.c +2 -2
@@ -2872,14 +2872,14 @@ static void set_untracked_ident(struct untracked_cache *uc)static unsigned new_untracked_cache_flags (struct index_state *istate) {struct repository *repo = istate->repo; -char *val; +const char *val;/* * This logic is coordinated with the setting of these flags in * wt-status.c#wt_status_collect_untracked(), and the evaluation * of the config setting in commit.c#git_status_config() */ -if (!repo_config_get_string (repo,"status.showuntrackedfiles" , &val) && +if (!repo_config_get_string_tmp (repo,"status.showuntrackedfiles" , &val) && !strcmp (val,"all" ))return 0 ;
t/t7063-status-untracked-cache.sh +1 -0
@@ -5,6 +5,7 @@ test_description='test untracked cache' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # On some filesystems (e.g. FreeBSD's ext2 and ufs) directory mtime
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
PATCH v2 21/22 builtin/merge: release outbut buffer after performing merge
In reply to Patrick SteinhardtThe obuf member of struct merge_options is used to buffer output in some cases. In order to not discard its allocated memory we only release its contents in merge_finalize() when we're not currently recursing into a subtree.
This results in some situations where we seemingly do not release the buffer reliably. We thus have calls to strbuf_release() for this buffer scattered across the codebase. But we're missing one callsite in git-merge(1), which causes a memory leak.
We should ideally refactor this interface so that callers don't have to know about any such internals. But for now, paper over the issue by adding one more strbuf_release() call.
--- builtin/merge.c | 1 + t/t6424-merge-unrelated-index-changes.sh | 1 + 2 files changed, 2 insertions(+)
builtin/merge.c +1 -0
@@ -754,6 +754,7 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common, clean =merge_recursive (&o, head, remoteheads->item, reversed, &result);free_commit_list (reversed); +strbuf_release (&o.obuf);if (clean <0 ) {rollback_lock_file (&lock);
t/t6424-merge-unrelated-index-changes.sh +1 -0
@@ -2,6 +2,7 @@ test_description="merges with unrelated index changes" +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Testcase for some simple merges
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
Re: [PATCH v2 21/22] builtin/merge: release outbut buffer after performing merge
Quoted 1 line
Re: [PATCH v2 21/22] builtin/merge: release outbut buffer after performing merge
s/outbut/output/
PATCH v2 22/22 list-objects-filter-options: work around reported leak on error
In reply to Patrick SteinhardtThis one is a little bit more curious. In t6112, we have a test that exercises the git rev-list --filter option with invalid filters. We execute git-rev-list(1) via test_must_fail, which means that we check for leaks even though Git exits with an error code. This causes the following leak:
Direct leak of 27 byte(s) in 1 object(s) allocated from:
#0 0x5555555e6946 in realloc.part.0 lsan_interceptors.cpp.o
#1 0x5555558fb4b6 in xrealloc wrapper.c:137:8
#2 0x5555558b6e06 in strbuf_grow strbuf.c:112:2
#3 0x5555558b7550 in strbuf_add strbuf.c:311:2
#4 0x5555557c1a88 in strbuf_addstr strbuf.h:310:2
#5 0x5555557c1d4c in parse_list_objects_filter list-objects-filter-options.c:261:3
#6 0x555555885ead in handle_revision_pseudo_opt revision.c:2899:3
#7 0x555555884e20 in setup_revisions revision.c:3014:11
#8 0x5555556c4b42 in cmd_rev_list builtin/rev-list.c:588:9
#9 0x5555555ec5e3 in run_builtin git.c:483:11
#10 0x5555555eb1e4 in handle_builtin git.c:749:13
#11 0x5555555ec001 in run_argv git.c:819:4
#12 0x5555555eaf94 in cmd_main git.c:954:19
#13 0x5555556fd569 in main common-main.c:64:11
#14 0x7ffff7ca714d in __libc_start_call_main (.../lib/libc.so.6+0x2a14d)
#15 0x7ffff7ca7208 in __libc_start_main@GLIBC_2.2.5 (.../libc.so.6+0x2a208)
#16 0x5555555ad064 in _start (git+0x59064)This leak is valid, as we call die() and do not clean up the memory at all. But what's curious is that this is the only leak reported, because we don't clean up any other allocated memory, either, and I have no idea why the leak sanitizer treats this buffer specially.
In any case, we can work around the leak by shuffling things around a bit. Instead of calling gently_parse_list_objects_filter() and dying after we have modified the filter spec, we simply do so beforehand. Like this we don't allocate the buffer in the error case, which makes the reported leak go away.
It's not pretty, but it manages to make t6112 leak free.
--- list-objects-filter-options.c | 17 +++++++---------- t/t6112-rev-list-filters-objects.sh | 1 + 2 files changed, 8 insertions(+), 10 deletions(-)
list-objects-filter-options.c +7 -10
@@ -252,16 +252,14 @@ void parse_list_objects_filter( const char *arg) {struct strbuf errbuf =STRBUF_INIT ; -int parse_error;if (!filter_options->filter_spec.buf)BUG ("filter_options not properly initialized" );if (!filter_options->choice) { +if (gently_parse_list_objects_filter (filter_options, arg, &errbuf)) +die ("%s" , errbuf.buf);strbuf_addstr (&filter_options->filter_spec, arg); - - parse_error =gently_parse_list_objects_filter ( - filter_options, arg, &errbuf); }else {struct list_objects_filter_options *sub;
@@ -271,18 +269,17 @@ void parse_list_objects_filter(*/ transform_to_combine_type (filter_options); -strbuf_addch (&filter_options->filter_spec,'+' ); -filter_spec_append_urlencode (filter_options, arg);ALLOC_GROW_BY (filter_options->sub, filter_options->sub_nr,1 , filter_options->sub_alloc); sub = &filter_options->sub[filter_options->sub_nr -1 ];list_objects_filter_init (sub); - parse_error =gently_parse_list_objects_filter (sub, arg, - &errbuf); +if (gently_parse_list_objects_filter (sub, arg, &errbuf)) +die ("%s" , errbuf.buf); + +strbuf_addch (&filter_options->filter_spec,'+' ); +filter_spec_append_urlencode (filter_options, arg); } -if (parse_error) -die ("%s" , errbuf.buf); } int opt_parse_list_objects_filter(const struct option *opt,
t/t6112-rev-list-filters-objects.sh +1 -0
@@ -5,6 +5,7 @@ test_description='git rev-list using object filtering' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Test the blob:none filter.
Signature · 1 line
2.47.0.72.gef8ce8f3d4.dirty
Quoted 3 lines · Patrick Steinhardt
Hi,
this is the second version of my 9th series of memory leak fixes.
I can’t even imagine the amount of effort it takes to plug all these leaks in parallel with the stuff that everyone else is doing in the code. Nice work. :)
Of course all the comments that I left just now are nitpicks.
Signature · 1 line
Kristoffer Haugsbakk
Quoted 10 lines · Kristoffer Haugsbakk
Quoted 3 lines · Patrick Steinhardt
Hi,
this is the second version of my 9th series of memory leak fixes.
I can’t even imagine the amount of effort it takes to plug all these leaks in parallel with the stuff that everyone else is doing in the code. Nice work. :)
Of course all the comments that I left just now are nitpicks.
Thanks! I've queued all fixes locally, but will wait a bit before sending them out so that other reviewers have a chance to chime in first.
Patrick
Quoted 3 lines · Patrick Steinhardt
Thanks! I've queued all fixes locally, but will wait a bit before sending them out so that other reviewers have a chance to chime in first.
I've got nothing more to add.
--
Toon
Quoted 9 lines · Patrick Steinhardt
Hi,
this is the second version of my 9th series of memory leak fixes.
Changes compared to v1:
- Split up the trailer fixes into two separate patches so that we can explain them standalone.
Quoted 7 lines
- Adapt the second trailer memory leak fix to instead pull up the strbufs out of the loop such that we can reuse them. This makes the code flow more naturally and optimizes the loop.
Thanks!
Patrick
I've reviewed all the patches in this version and left a few non-blocking thoughts. Overall, this version looks good to me though. :)
-Justin
Quoted 14 lines · Justin Tobler
Quoted 13 lines · Patrick Steinhardt
Hi,
this is the second version of my 9th series of memory leak fixes.
Changes compared to v1:
- Split up the trailer fixes into two separate patches so that we can explain them standalone.
- Adapt the second trailer memory leak fix to instead pull up the strbufs out of the loop such that we can reuse them. This makes the code flow more naturally and optimizes the loop.
Quoted 6 lines
Quoted 3 lines
Thanks!
Patrick
I've reviewed all the patches in this version and left a few non-blocking thoughts. Overall, this version looks good to me though. :)
Thanks for your review!
Patrick
v3 cover Memory leak fixes (pt.9)
In reply to Patrick SteinhardtHi,
this is the third revision of my 9th series of memory leak fixes. Changes compared to v2:
- Remove an unnecessary cast.
- Fix a duplicate newline.
- Polish a couple of commit messages.
Thanks!
Patrick
Patrick Steinhardt (22):
builtin/ls-remote: plug leaking server options t/helper: fix leaks in "reach" test tool
grep: fix leak in grep_splice_or()
builtin/grep: fix leak with --max-count=0
revision: fix leaking bloom filters
diff-lib: fix leaking diffopts in do_diff_cache()
pretty: clear signature check
upload-pack: fix leaking URI protocols builtin/commit: fix leaking change data contents
trailer: fix leaking trailer values
trailer: fix leaking strbufs when formatting trailers builtin/commit: fix leaking cleanup config
transport-helper: fix leaking import/export marks builtin/tag: fix leaking key ID on failure to sign
combine-diff: fix leaking lost lines
dir: release untracked cache data
sparse-index: correctly free EWAH contents
t/helper: stop re-initialization of the_repository t/helper: fix leaking buffer in "dump-untracked-cache"
dir: fix leak when parsing "status.showUntrackedFiles" builtin/merge: release output buffer after performing merge list-objects-filter-options: work around reported leak on error
builtin/commit.c | 26 +++++++++++++++++------ builtin/grep.c | 13 +++++++++--- builtin/ls-remote.c | 1 + builtin/merge.c | 1 + builtin/tag.c | 2 +- combine-diff.c | 5 +++-- diff-lib.c | 1 + dir.c | 12 +++++++++-- grep.c | 1 + list-objects-filter-options.c | 17 ++++++--------- pretty.c | 1 + revision.c | 5 +++++ sparse-index.c | 7 ++++-- t/helper/test-dump-untracked-cache.c | 2 ++ t/helper/test-reach.c | 10 +++++++++ t/helper/test-read-cache.c | 2 -- t/t4038-diff-combined.sh | 1 + t/t4202-log.sh | 1 + t/t4216-log-bloom.sh | 1 + t/t5702-protocol-v2.sh | 1 + t/t5801-remote-helpers.sh | 1 + t/t6112-rev-list-filters-objects.sh | 1 + t/t6424-merge-unrelated-index-changes.sh | 1 + t/t6600-test-reach.sh | 1 + t/t7004-tag.sh | 1 + t/t7031-verify-tag-signed-ssh.sh | 1 + t/t7063-status-untracked-cache.sh | 1 + t/t7500-commit-template-squash-signoff.sh | 1 + t/t7502-commit-porcelain.sh | 1 + t/t7510-signed-commit.sh | 1 + t/t7513-interpret-trailers.sh | 1 + t/t7519-status-fsmonitor.sh | 1 + t/t7528-signed-commit-ssh.sh | 1 + t/t7610-mergetool.sh | 1 + t/t7810-grep.sh | 1 + trailer.c | 22 +++++++++++++------ transport-helper.c | 2 ++ upload-pack.c | 1 + 38 files changed, 115 insertions(+), 35 deletions(-)
Range-diff against v2:
1: 89b66411354 ! 1: fbb9e68e2f2 builtin/ls-remote: plug leaking server options
@@ Metadata
## Commit message ##
builtin/ls-remote: plug leaking server options
- The server options populated via OPT_STRING_LIST() is never cleared,
- causing a memory leak. Plug it.
+ The list of server options populated via OPT_STRING_LIST() is never
+ cleared, causing a memory leak. Plug it.
This leak is exposed by t5702, but plugging it alone does not make the whole test suite pass.
2: 1c42e194b20 = 2: 17136f4bdb3 t/helper: fix leaks in "reach" test tool 3: cb4eee37b40 ! 3: 74b21470194 grep: fix leak in `grep_splice_or()`
@@ Commit message
grep: fix leak in grep_splice_or()
In grep_splice_or() we search for the next TRUE node in our tree of
- grep exrpessions and replace it with the given new expression. But we
+ grep expressions and replace it with the given new expression. But we
don't free the old node, which causes a memory leak. Plug it.
This leak is exposed by t7810, but plugging it alone isn't sufficient to
4: 6b2c8842ef5 = 4: d716f93169a builtin/grep: fix leak with `--max-count=0` 5: 7527b31a28f = 5: aeb8a19d28d revision: fix leaking bloom filters 6: 60af98cb2c7 ! 6: 24d9d9b1358 diff-lib: fix leaking diffopts in `do_diff_cache()`
@@ Commit message
In do_diff_cache() we initialize a new rev_info and then overwrite its diffopt with a user-provided set of options. This can leak memory because repo_init_revisions() may end up allocating memory for the
- diffopt itself depending on the configuration. And as that field is
- overwritten we won't ever free that.
+ diffopt itself depending on the configuration. And since that field is
+ overwritten we won't ever free it.
Plug the memory leak by releasing the diffopts before we overwrite them.
7: 5d5f6867f91 ! 7: 58ebef7e757 pretty: clear signature check
@@ Metadata
## Commit message ##
pretty: clear signature check
- The signature check in of the formatting context is never getting
- released. Fix this to plug the resulting memory leak.
+ The signature check in the formatting context is never getting released.
+ Fix this to plug the resulting memory leak.
8: 0d503e40194 = 8: c5813079d44 upload-pack: fix leaking URI protocols
9: 9f967dfe5d5 = 9: f66fa4e0e25 builtin/commit: fix leaking change data contents
10: e3ffd59123f ! 10: dac63bae39e trailer: fix leaking trailer values
@@ trailer.c: static char *apply_command(struct conf_info *conf, const char *arg)
+ char *arg;
+
if (arg_tok->value && arg_tok->value[0]) {
-- arg = arg_tok->value;
-+ arg = (char *)arg_tok->value;
+ arg = arg_tok->value;
} else {
- if (in_tok && in_tok->value)
+@@ trailer.c: static void apply_item_command(struct trailer_item *in_tok, struct arg_item *arg
arg = xstrdup(in_tok->value);
else
arg = xstrdup("");11: 5b851453bce ! 11: 82269e5d5be trailer: fix leaking strbufs when formatting trailers
@@ Metadata
## Commit message ##
trailer: fix leaking strbufs when formatting trailers
- We are populating, but never releasing two string buffers in
- format_trailers(), causing a memory leak. Plug this leak by lifting
- those buffers outside of the loop and releasing them on function return.
- This fixes the memory leaks, but also optimizes the loop as we don't
- have to reallocate the buffers on every single iteration.
+ When formatting trailer lines we iterate through each of the trailers
+ and munge their respective token/value pairs according to the trailer
+ options. When formatting a trailer that has its item->token pointer
+ set we perform the munging in two local buffers. In the case where we
+ figure out that the value is empty and trim_empty is set we just skip
+ over the trailer item. But the buffers are local to the loop and we
+ don't release their contents, leading to a memory leak.
+
+ Plug this leak by lifting the buffers outside of the loop and releasing
+ them on function return. This fixes the memory leaks, but also optimizes
+ the loop as we don't have to reallocate the buffers on every single
+ iteration.
@@ trailer.c: void format_trailers(const struct process_trailer_options *opts,
size_t origlen = out->len;
struct list_head *pos;
struct trailer_item *item;
-
-+
+@@ trailer.c: void format_trailers(const struct process_trailer_options *opts,
list_for_each(pos, trailers) {
item = list_entry(pos, struct trailer_item, list);
if (item->token) {
12: 60c3f6146f3 = 12: a627e03702e builtin/commit: fix leaking cleanup config 13: ea81cd8db86 = 13: 40e0c2a2cac transport-helper: fix leaking import/export marks 14: b700ab9079f = 14: ffa5d9eae7e builtin/tag: fix leaking key ID on failure to sign 15: 76bbcb3fe30 ! 15: 70dd0cb6933 combine-diff: fix leaking lost lines
@@ Commit message
The cnt variable tracks the number of lines in a patch diff. It can happen though that there are no newlines, in which case we'd still end up allocating our array of slines. In fact, we always allocate it with
- cnt + 2 entries. But when we loop through the array to clear it at the
- end of this function we only loop until lno < cnt, and thus we may not
- end up releasing whatever the two extra slines contain.
+ cnt + 2 entries: one extra entry for the deletion hunk at the end, and
+ another entry that we don't seem to ever populate at all but acts as a
+ kind of sentinel value.
- Plug this memory leak.
+ When we loop through the array to clear it at the end of this function
+ we only loop until lno < cnt, and thus we may not end up releasing
+ whatever the two extra slines contain. While that shouldn't matter for
+ the sentinel value, it does matter for the extra deletion hunk sline.
+ Regardless of that, plug this memory leak by releasing both extra
+ entries, which makes the logic a bit easier to reason about.
+
+ While at it, fix the formatting of a local comment, which incidentally
+ also provides the necessary context for why we overallocate the sline
+ array.
## combine-diff.c ##
+@@ combine-diff.c: static void show_patch_diff(struct combine_diff_path *elem, int num_parent,
+ result_file.ptr = result;
+ result_file.size = result_size;
+
+- /* Even p_lno[cnt+1] is valid -- that is for the end line number
++ /*
++ * Even p_lno[cnt+1] is valid -- that is for the end line number
+ * for deletion hunk at the end.
+ */
+ CALLOC_ARRAY(sline[0].p_lno, st_mult(st_add(cnt, 2), num_parent));
@@ combine-diff.c: static void show_patch_diff(struct combine_diff_path *elem, int num_parent,
}
free(result);
16: d6b96a4012d = 16: b248f266a91 dir: release untracked cache data 17: 3aa6bac5079 = 17: 76e9a6d5792 sparse-index: correctly free EWAH contents 18: 132fe750906 = 18: 70f16486d77 t/helper: stop re-initialization of `the_repository` 19: b8b702eeb28 = 19: f056d31ca50 t/helper: fix leaking buffer in "dump-untracked-cache" 20: ad309ac1b37 = 20: 714c9286e7a dir: fix leak when parsing "status.showUntrackedFiles" 21: 5e243f9ee53 ! 21: 0ff65c1213b builtin/merge: release outbut buffer after performing merge
@@ Metadata
Author: Patrick Steinhardt <[email protected]>
## Commit message ##
- builtin/merge: release outbut buffer after performing merge
+ builtin/merge: release output buffer after performing merge
The obuf member of struct merge_options is used to buffer output in some cases. In order to not discard its allocated memory we only release
22: b75376e3725 = 22: d9e122bb5db list-objects-filter-options: work around reported leak on error
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 1/22 builtin/ls-remote: plug leaking server options
The list of server options populated via OPT_STRING_LIST() is never cleared, causing a memory leak. Plug it.
This leak is exposed by t5702, but plugging it alone does not make the whole test suite pass.
--- builtin/ls-remote.c | 1 + 1 file changed, 1 insertion(+)
builtin/ls-remote.c +1 -0
@@ -166,6 +166,7 @@ int cmd_ls_remote(int argc, status =0 ;/* we found something */ } +string_list_clear (&server_options,0 );ref_sorting_release (sorting);ref_array_clear (&ref_array);if (transport_disconnect (transport))
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 2/22 t/helper: fix leaks in "reach" test tool
In reply to Patrick SteinhardtThe "reach" test tool doesn't bother to clean up any of its allocated resources, causing various leaks. Plug them.
--- t/helper/test-reach.c | 10 ++++++++++ t/t6600-test-reach.sh | 1 + 2 files changed, 11 insertions(+)
t/helper/test-reach.c +10 -0
@@ -127,10 +127,12 @@ int cmd__reach(int ac, const char **av)exit (128 );printf ("%s(A,X):\n" , av[1 ]);print_sorted_commit_ids (list); +free_commit_list (list); }else if (!strcmp (av[1 ],"reduce_heads" )) {struct commit_list *list =reduce_heads (X );printf ("%s(X):\n" , av[1 ]);print_sorted_commit_ids (list); +free_commit_list (list); }else if (!strcmp (av[1 ],"can_all_from_reach" )) {printf ("%s(X,Y):%d\n" , av[1 ],can_all_from_reach (X ,Y ,1 )); }else if (!strcmp (av[1 ],"can_all_from_reach_with_flag" )) {
@@ -153,6 +155,7 @@ int cmd__reach(int ac, const char **av) filter.with_commit_tag_algo =0 ;printf ("%s(_,A,X,_):%d\n" , av[1 ],commit_contains (&filter,A ,X , &cache)); +clear_contains_cache (&cache); }else if (!strcmp (av[1 ],"get_reachable_subset" )) {const int reachable_flag =1 ;int i, count =0 ;
@@ -176,7 +179,14 @@ int cmd__reach(int ac, const char **av)die (_ ("too many commits marked reachable" ));print_sorted_commit_ids (list); +free_commit_list (list); } +object_array_clear (&X_obj); +strbuf_release (&buf); +free_commit_list (X ); +free_commit_list (Y ); +free (X_array); +free (Y_array);return 0 ; }
t/t6600-test-reach.sh +1 -0
@@ -2,6 +2,7 @@ test_description='basic commit reachability tests' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Construct a grid-like commit graph with points (x,y)
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 3/22 grep: fix leak in `grep_splice_or()`
In reply to Patrick SteinhardtIn grep_splice_or() we search for the next TRUE node in our tree of grep expressions and replace it with the given new expression. But we don't free the old node, which causes a memory leak. Plug it.
This leak is exposed by t7810, but plugging it alone isn't sufficient to make the test suite pass.
--- grep.c | 1 + 1 file changed, 1 insertion(+)
grep.c +1 -0
@@ -756,6 +756,7 @@ static struct grep_expr *grep_splice_or(struct grep_expr *x, struct grep_expr *yassert (x->node ==GREP_NODE_OR );if (x->u.binary.right && x->u.binary.right->node ==GREP_NODE_TRUE ) { +free (x->u.binary.right); x->u.binary.right = y;break ; }
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 4/22 builtin/grep: fix leak with `--max-count=0`
In reply to Patrick SteinhardtWhen executing with --max-count=0 we'll return early from git-grep(1) without performing any cleanup, which causes memory leaks. Plug these.
--- builtin/grep.c | 13 ++++++++++--- t/t7810-grep.sh | 1 + 2 files changed, 11 insertions(+), 3 deletions(-)
builtin/grep.c +10 -3
@@ -906,6 +906,7 @@ int cmd_grep(int argc,int dummy;int use_index =1 ;int allow_revs; +int ret;struct option options[] = { OPT_BOOL(0, "cached", &cached,
@@ -1172,8 +1173,10 @@ int cmd_grep(int argc,* Optimize out the case where the amount of matches is limited to zero. * We do this to keep results consistent with GNU grep(1). */ -if (opt.max_count ==0 ) -return 1 ; +if (opt.max_count ==0 ) { + ret =1 ; + goto out; + }if (show_in_pager) {if (num_threads >1 )
@@ -1267,10 +1270,14 @@ int cmd_grep(int argc, hit |=wait_all ();if (hit && show_in_pager)run_pager (&opt, prefix); + + ret = !hit; + +out:clear_pathspec (&pathspec);string_list_clear (&path_list,0 );free_grep_patterns (&opt);object_array_clear (&list);free_repos (); -return !hit; +return ret; }
t/t7810-grep.sh +1 -0
@@ -9,6 +9,7 @@ test_description='git grep various. GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_invalid_grep_expression() {
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 5/22 revision: fix leaking bloom filters
In reply to Patrick SteinhardtThe memory allocated by prepare_to_use_bloom_filter() is not released by release_revisions(), causing a memory leak. Plug it.
--- revision.c | 5 +++++ t/t4216-log-bloom.sh | 1 + 2 files changed, 6 insertions(+)
revision.c +5 -0
@@ -3227,6 +3227,11 @@ void release_revisions(struct rev_info *revs)clear_decoration (&revs->treesame, free);line_log_free (revs);oidset_clear (&revs->missing_commits); + +for (int i =0 ; i < revs->bloom_keys_nr; i++) +clear_bloom_key (&revs->bloom_keys[i]); +FREE_AND_NULL (revs->bloom_keys); + revs->bloom_keys_nr =0 ; }static void add_child (struct rev_info *revs,struct commit *parent,struct commit *child)
t/t4216-log-bloom.sh +1 -0
@@ -4,6 +4,7 @@ test_description='git log for a path with Bloom filters' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-chunk.sh
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 6/22 diff-lib: fix leaking diffopts in `do_diff_cache()`
In reply to Patrick SteinhardtIn do_diff_cache() we initialize a new rev_info and then overwrite its diffopt with a user-provided set of options. This can leak memory because repo_init_revisions() may end up allocating memory for the diffopt itself depending on the configuration. And since that field is overwritten we won't ever free it.
Plug the memory leak by releasing the diffopts before we overwrite them.
--- diff-lib.c | 1 + t/t7610-mergetool.sh | 1 + 2 files changed, 2 insertions(+)
diff-lib.c +1 -0
@@ -661,6 +661,7 @@ int do_diff_cache(const struct object_id *tree_oid, struct diff_options *opt)repo_init_revisions (opt->repo, &revs,NULL );copy_pathspec (&revs.prune_data, &opt->pathspec); +diff_free (&revs.diffopt); revs.diffopt = *opt; revs.diffopt.no_free =1 ;
t/t7610-mergetool.sh +1 -0
@@ -10,6 +10,7 @@ Testing basic merge tool invocation' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # All the mergetool test work by checking out a temporary branch based
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 7/22 pretty: clear signature check
In reply to Patrick SteinhardtThe signature check in the formatting context is never getting released. Fix this to plug the resulting memory leak.
--- pretty.c | 1 + t/t4202-log.sh | 1 + t/t7031-verify-tag-signed-ssh.sh | 1 + t/t7510-signed-commit.sh | 1 + t/t7528-signed-commit-ssh.sh | 1 + 5 files changed, 5 insertions(+)
pretty.c +1 -0
@@ -2032,6 +2032,7 @@ void repo_format_commit_message(struct repository *r,free (context.commit_encoding);repo_unuse_commit_buffer (r, commit, context.message); +signature_check_clear (&context.signature_check); } static void pp_header(struct pretty_print_context *pp,
t/t4202-log.sh +1 -0
@@ -5,6 +5,7 @@ test_description='git log' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY/lib-gpg.sh" . "$TEST_DIRECTORY/lib-terminal.sh"
t/t7031-verify-tag-signed-ssh.sh +1 -0
@@ -4,6 +4,7 @@ test_description='signed tag tests' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY/lib-gpg.sh"
t/t7510-signed-commit.sh +1 -0
@@ -4,6 +4,7 @@ test_description='signed commit tests' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh GNUPGHOME_NOT_USED=$GNUPGHOME . "$TEST_DIRECTORY/lib-gpg.sh"
t/t7528-signed-commit-ssh.sh +1 -0
@@ -4,6 +4,7 @@ test_description='ssh signed commit tests' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh GNUPGHOME_NOT_USED=$GNUPGHOME . "$TEST_DIRECTORY/lib-gpg.sh"
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 8/22 upload-pack: fix leaking URI protocols
In reply to Patrick SteinhardtWe don't clear struct upload_pack::uri_protocols, which causes a memory leak. Fix this.
--- t/t5702-protocol-v2.sh | 1 + upload-pack.c | 1 + 2 files changed, 2 insertions(+)
t/t5702-protocol-v2.sh +1 -0
@@ -7,6 +7,7 @@ TEST_NO_CREATE_REPO=1 GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Test protocol v2 with 'git://' transport
upload-pack.c +1 -0
@@ -166,6 +166,7 @@ static void upload_pack_data_clear(struct upload_pack_data *data)object_array_clear (&data->extra_edge_obj);list_objects_filter_release (&data->filter_options);string_list_clear (&data->allowed_filters,0 ); +string_list_clear (&data->uri_protocols,0 );free ((char *)data->pack_objects_hook); }
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 9/22 builtin/commit: fix leaking change data contents
In reply to Patrick SteinhardtWhile we free the worktree change data, we never free its contents. Fix this.
--- builtin/commit.c | 9 ++++++++- t/t7500-commit-template-squash-signoff.sh | 1 + 2 files changed, 9 insertions(+), 1 deletion(-)
builtin/commit.c +8 -1
@@ -728,6 +728,13 @@ static void prepare_amend_commit(struct commit *commit, struct strbuf *sb,repo_unuse_commit_buffer (the_repository, commit, buffer); } +static void change_data_free(void *util, const char *str UNUSED) +{ +struct wt_status_change_data *d = util; +free (d->rename_source); +free (d); +} + static int prepare_to_commit(const char *index_file, const char *prefix, struct commit *current_head, struct wt_status *s,
@@ -991,7 +998,7 @@ static int prepare_to_commit(const char *index_file, const char *prefix, s->use_color =0 ; committable =run_status (s->fp, index_file, prefix,1 , s); s->use_color = saved_color_setting; -string_list_clear (&s->change,1 ); +string_list_clear_func (&s->change, change_data_free); }else {struct object_id oid;const char *parent ="HEAD" ;
t/t7500-commit-template-squash-signoff.sh +1 -0
@@ -7,6 +7,7 @@ test_description='git commit Tests for template, signoff, squash and -F functions.' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-rebase.sh
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 10/22 trailer: fix leaking trailer values
In reply to Patrick SteinhardtFix leaking trailer values when replacing the value with a command or when the token value is empty.
This leak is exposed by t7513, but plugging it does not make the whole test suite pass.
--- trailer.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
trailer.c +8 -2
@@ -249,7 +249,9 @@ static char *apply_command(struct conf_info *conf, const char *arg)static void apply_item_command (struct trailer_item *in_tok,struct arg_item *arg_tok) {if (arg_tok->conf.command || arg_tok->conf.cmd) { -const char *arg; +char *value_to_free =NULL ; +char *arg; +if (arg_tok->value && arg_tok->value[0 ]) { arg = arg_tok->value; }else {
@@ -257,9 +259,13 @@ static void apply_item_command(struct trailer_item *in_tok, struct arg_item *arg arg =xstrdup (in_tok->value); else arg =xstrdup ("" ); + value_to_free = arg_tok->value; } + arg_tok->value =apply_command (&arg_tok->conf, arg); -free ((char *)arg); + +free (value_to_free); +free (arg); } }
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 11/22 trailer: fix leaking strbufs when formatting trailers
In reply to Patrick SteinhardtWhen formatting trailer lines we iterate through each of the trailers and munge their respective token/value pairs according to the trailer options. When formatting a trailer that has its item->token pointer set we perform the munging in two local buffers. In the case where we figure out that the value is empty and trim_empty is set we just skip over the trailer item. But the buffers are local to the loop and we don't release their contents, leading to a memory leak.
Plug this leak by lifting the buffers outside of the loop and releasing them on function return. This fixes the memory leaks, but also optimizes the loop as we don't have to reallocate the buffers on every single iteration.
--- t/t7513-interpret-trailers.sh | 1 + trailer.c | 12 +++++++----- 2 files changed, 8 insertions(+), 5 deletions(-)
t/t7513-interpret-trailers.sh +1 -0
@@ -5,6 +5,7 @@ test_description='git interpret-trailers' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # When we want one trailing space at the end of each line, let's use sed
trailer.c +7 -5
@@ -1111,6 +1111,8 @@ void format_trailers(const struct process_trailer_options *opts,struct list_head *trailers, struct strbuf *out) { + struct strbuf tok = STRBUF_INIT; +struct strbuf val =STRBUF_INIT ;size_t origlen = out->len;struct list_head *pos;struct trailer_item *item;
@@ -1118,9 +1120,9 @@ void format_trailers(const struct process_trailer_options *opts,list_for_each (pos, trailers) { item = list_entry(pos, struct trailer_item, list);if (item->token) { -struct strbuf tok =STRBUF_INIT ; -struct strbuf val =STRBUF_INIT ; +strbuf_reset (&tok);strbuf_addstr (&tok, item->token); +strbuf_reset (&val);strbuf_addstr (&val, item->value);/*
@@ -1151,9 +1153,6 @@ void format_trailers(const struct process_trailer_options *opts,if (!opts->separator)strbuf_addch (out,'\n' ); } -strbuf_release (&tok); -strbuf_release (&val); - }else if (!opts->only_trailers) {if (opts->separator && out->len != origlen) {strbuf_addbuf (out, opts->separator);
@@ -1165,6 +1164,9 @@ void format_trailers(const struct process_trailer_options *opts,strbuf_addch (out,'\n' ); } } + +strbuf_release (&tok); +strbuf_release (&val); } void format_trailers_from_commit(const struct process_trailer_options *opts,
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 12/22 builtin/commit: fix leaking cleanup config
In reply to Patrick SteinhardtThe cleanup string set by the config is leaking when it is being overridden by an option. Fix this by tracking these via two separate variables such that we can free the old value.
--- builtin/commit.c | 17 ++++++++++++----- t/t7502-commit-porcelain.sh | 1 + 2 files changed, 13 insertions(+), 5 deletions(-)
builtin/commit.c +12 -5
@@ -135,7 +135,7 @@ static struct strvec trailer_args = STRVEC_INIT;* is specified explicitly. */ static enum commit_msg_cleanup_mode cleanup_mode; -static char *cleanup_arg; +static char *cleanup_config;static enum commit_whence whence;static int use_editor =1 , include_status =1 ;
@@ -1387,8 +1387,6 @@ static int parse_and_validate_options(int argc, const char *argv[],if (0 <= edit_flag) use_editor = edit_flag; - cleanup_mode =get_cleanup_mode (cleanup_arg, use_editor); -handle_untracked_files_arg (s);if (all && argc >0 )
@@ -1636,8 +1634,10 @@ static int git_commit_config(const char *k, const char *v, include_status =git_config_bool (k, v);return 0 ; } -if (!strcmp (k,"commit.cleanup" )) -return git_config_string (&cleanup_arg, k, v); +if (!strcmp (k,"commit.cleanup" )) { +FREE_AND_NULL (cleanup_config); +return git_config_string (&cleanup_config, k, v); + }if (!strcmp (k,"commit.gpgsign" )) { sign_commit =git_config_bool (k, v) ?"" :NULL ;return 0 ;
@@ -1658,6 +1658,7 @@ int cmd_commit(int argc, struct repository *repo UNUSED) {static struct wt_status s; +static const char *cleanup_arg =NULL ;static struct option builtin_commit_options[] = {OPT__QUIET (&quiet,N_ ("suppress summary after successful commit" )),OPT__VERBOSE (&verbose,N_ ("show diff in commit message template" )),
@@ -1757,6 +1758,12 @@ int cmd_commit(int argc,if (verbose ==-1 ) verbose = (config_commit_verbose <0 ) ?0 : config_commit_verbose; +if (cleanup_arg) { +free (cleanup_config); + cleanup_config =xstrdup (cleanup_arg); + } + cleanup_mode =get_cleanup_mode (cleanup_config, use_editor); +if (dry_run)return dry_run_commit (argv, prefix, current_head, &s); index_file =prepare_index (argv, prefix, current_head,0 );
t/t7502-commit-porcelain.sh +1 -0
@@ -5,6 +5,7 @@ test_description='git commit porcelain-ish' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh commit_msg_is () {
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 13/22 transport-helper: fix leaking import/export marks
In reply to Patrick SteinhardtFix leaking import and export marks for transport helpers.
--- t/t5801-remote-helpers.sh | 1 + transport-helper.c | 2 ++ 2 files changed, 3 insertions(+)
t/t5801-remote-helpers.sh +1 -0
@@ -8,6 +8,7 @@ test_description='Test remote-helper import and export commands' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-gpg.sh
transport-helper.c +2 -0
@@ -399,6 +399,8 @@ static int release_helper(struct transport *transport)int res =0 ;struct helper_data *data = transport->data;refspec_clear (&data->rs); +free (data->import_marks); +free (data->export_marks); res =disconnect_helper (transport);free (transport->data);return res;
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 14/22 builtin/tag: fix leaking key ID on failure to sign
In reply to Patrick SteinhardtWe do not free the key ID when signing a tag fails. Do so by using the common exit path.
--- builtin/tag.c | 2 +- t/t7004-tag.sh | 1 + 2 files changed, 2 insertions(+), 1 deletion(-)
builtin/tag.c +1 -1
@@ -164,7 +164,7 @@ static int do_sign(struct strbuf *buffer, struct object_id **compat_oid,int ret =-1 ;if (sign_buffer (buffer, &sig, keyid)) -return -1 ; + goto out;if (compat) {const struct git_hash_algo *algo = the_repository->hash_algo;
t/t7004-tag.sh +1 -0
@@ -10,6 +10,7 @@ Tests for operations with tags.' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-gpg.sh . "$TEST_DIRECTORY"/lib-terminal.sh
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 15/22 combine-diff: fix leaking lost lines
In reply to Patrick SteinhardtThe cnt variable tracks the number of lines in a patch diff. It can happen though that there are no newlines, in which case we'd still end up allocating our array of slines. In fact, we always allocate it with cnt + 2 entries: one extra entry for the deletion hunk at the end, and another entry that we don't seem to ever populate at all but acts as a kind of sentinel value.
When we loop through the array to clear it at the end of this function we only loop until lno < cnt, and thus we may not end up releasing whatever the two extra slines contain. While that shouldn't matter for the sentinel value, it does matter for the extra deletion hunk sline. Regardless of that, plug this memory leak by releasing both extra entries, which makes the logic a bit easier to reason about.
While at it, fix the formatting of a local comment, which incidentally also provides the necessary context for why we overallocate the sline array.
--- combine-diff.c | 5 +++-- t/t4038-diff-combined.sh | 1 + 2 files changed, 4 insertions(+), 2 deletions(-)
combine-diff.c +3 -2
@@ -1185,7 +1185,8 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent, result_file.ptr = result; result_file.size = result_size; -/* Even p_lno[cnt+1] is valid -- that is for the end line number +/* +* Even p_lno[cnt+1] is valid -- that is for the end line number * for deletion hunk at the end. */ CALLOC_ARRAY (sline[0 ].p_lno,st_mult (st_add (cnt,2 ), num_parent));
@@ -1220,7 +1221,7 @@ static void show_patch_diff(struct combine_diff_path *elem, int num_parent, }free (result); -for (lno =0 ; lno < cnt; lno++) { +for (lno =0 ; lno < cnt +2 ; lno++) {if (sline[lno].lost) {struct lline *ll = sline[lno].lost;while (ll) {
t/t4038-diff-combined.sh +1 -0
@@ -5,6 +5,7 @@ test_description='combined diff' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-diff.sh
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 16/22 dir: release untracked cache data
In reply to Patrick SteinhardtThere are several cases where we invalidate untracked cache directory entries where we do not free the underlying data, but reset the number of entries. This causes us to leak memory because free_untracked() will not iterate over any potential entries which we still had in the array.
Fix this issue by freeing old entries. The leak is exposed by t7519, but plugging it alone does not make the whole test suite pass.
--- dir.c | 8 ++++++++ 1 file changed, 8 insertions(+)
dir.c +8 -0
@@ -1056,6 +1056,8 @@ static void do_invalidate_gitignore(struct untracked_cache_dir *dir) {int i; dir->valid =0 ; +for (size_t i =0 ; i < dir->untracked_nr; i++) +free (dir->untracked[i]); dir->untracked_nr =0 ;for (i =0 ; i < dir->dirs_nr; i++)do_invalidate_gitignore (dir->dirs[i]);
@@ -1083,6 +1085,8 @@ static void invalidate_directory(struct untracked_cache *uc, uc->dir_invalidated++; dir->valid =0 ; +for (size_t i =0 ; i < dir->untracked_nr; i++) +free (dir->untracked[i]); dir->untracked_nr =0 ;for (i =0 ; i < dir->dirs_nr; i++) dir->dirs[i]->recurse =0 ;
@@ -3573,6 +3577,8 @@ static void write_one_dir(struct untracked_cache_dir *untracked,* for safety.. */ if (!untracked->valid) { +for (size_t i =0 ; i < untracked->untracked_nr; i++) +free (untracked->untracked[i]); untracked->untracked_nr =0 ; untracked->check_only =0 ; }
@@ -3905,6 +3911,8 @@ static void invalidate_one_directory(struct untracked_cache *uc, { uc->dir_invalidated++; ucd->valid =0 ; +for (size_t i =0 ; i < ucd->untracked_nr; i++) +free (ucd->untracked[i]); ucd->untracked_nr =0 ; }
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 17/22 sparse-index: correctly free EWAH contents
In reply to Patrick SteinhardtWhile we free the fsmonitor_dirty member of struct index_state, we do not free the contents of that EWAH. Do so by using ewah_free() instead of FREE_AND_NULL().
This leak is exposed by t7519, but plugging it alone does not make the test suite pass.
--- sparse-index.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
sparse-index.c +5 -2
@@ -2,6 +2,7 @@#include "git-compat-util.h" #include "environment.h" +#include "ewah/ewok.h" #include "gettext.h" #include "name-hash.h" #include "read-cache-ll.h"
@@ -242,7 +243,8 @@ int convert_to_sparse(struct index_state *istate, int flags)cache_tree_update (istate,0 ); istate->fsmonitor_has_run_once =0 ; -FREE_AND_NULL (istate->fsmonitor_dirty); +ewah_free (istate->fsmonitor_dirty); + istate->fsmonitor_dirty =NULL ;FREE_AND_NULL (istate->fsmonitor_last_update); istate->sparse_index =INDEX_COLLAPSED ;
@@ -438,7 +440,8 @@ void expand_index(struct index_state *istate, struct pattern_list *pl) istate->cache_nr = full->cache_nr; istate->cache_alloc = full->cache_alloc; istate->fsmonitor_has_run_once =0 ; -FREE_AND_NULL (istate->fsmonitor_dirty); +ewah_free (istate->fsmonitor_dirty); + istate->fsmonitor_dirty =NULL ;FREE_AND_NULL (istate->fsmonitor_last_update);strbuf_release (&base);
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 18/22 t/helper: stop re-initialization of `the_repository`
In reply to Patrick SteinhardtWhile "common-main.c" already initializes the_repository for us, we do so a second time in the "read-cache" test helper. This causes a memory leak because the old repository's contents isn't released.
Stop calling initialize_repository() to plug this leak.
--- t/helper/test-read-cache.c | 2 -- t/t7519-status-fsmonitor.sh | 1 + 2 files changed, 1 insertion(+), 2 deletions(-)
t/helper/test-read-cache.c +0 -2
@@ -11,8 +11,6 @@ int cmd__read_cache(int argc, const char **argv)int i, cnt =1 ;const char *name =NULL ; -initialize_repository (the_repository); -if (argc >1 &&skip_prefix (argv[1 ],"--print-and-refresh=" , &name)) { argc--; argv++;
t/t7519-status-fsmonitor.sh +1 -0
@@ -2,6 +2,7 @@ test_description='git status with file system watcher' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Note, after "git reset --hard HEAD" no extensions exist other than 'TREE'
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 19/22 t/helper: fix leaking buffer in "dump-untracked-cache"
In reply to Patrick SteinhardtWe never release the local struct strbuf base buffer, thus leaking memory. Fix this leak.
This leak is exposed by t7063, but plugging it alone does not make the whole test suite pass.
--- t/helper/test-dump-untracked-cache.c | 2 ++ 1 file changed, 2 insertions(+)
t/helper/test-dump-untracked-cache.c +2 -0
@@ -68,5 +68,7 @@ int cmd__dump_untracked_cache(int ac UNUSED, const char **av UNUSED)printf ("flags %08x\n" , uc->dir_flags);if (uc->root)dump (uc->root, &base); + +strbuf_release (&base);return 0 ; }
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 20/22 dir: fix leak when parsing "status.showUntrackedFiles"
In reply to Patrick SteinhardtWe use repo_config_get_string() to read "status.showUntrackedFiles" from the config subsystem. This function allocates the result, but we never free the result after parsing it.
The value never leaves the scope of the calling function, so refactor it to instead use repo_config_get_string_tmp(), which does not hand over ownership to the caller.
--- dir.c | 4 ++-- t/t7063-status-untracked-cache.sh | 1 + 2 files changed, 3 insertions(+), 2 deletions(-)
dir.c +2 -2
@@ -2872,14 +2872,14 @@ static void set_untracked_ident(struct untracked_cache *uc)static unsigned new_untracked_cache_flags (struct index_state *istate) {struct repository *repo = istate->repo; -char *val; +const char *val;/* * This logic is coordinated with the setting of these flags in * wt-status.c#wt_status_collect_untracked(), and the evaluation * of the config setting in commit.c#git_status_config() */ -if (!repo_config_get_string (repo,"status.showuntrackedfiles" , &val) && +if (!repo_config_get_string_tmp (repo,"status.showuntrackedfiles" , &val) && !strcmp (val,"all" ))return 0 ;
t/t7063-status-untracked-cache.sh +1 -0
@@ -5,6 +5,7 @@ test_description='test untracked cache' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # On some filesystems (e.g. FreeBSD's ext2 and ufs) directory mtime
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 21/22 builtin/merge: release output buffer after performing merge
In reply to Patrick SteinhardtThe obuf member of struct merge_options is used to buffer output in some cases. In order to not discard its allocated memory we only release its contents in merge_finalize() when we're not currently recursing into a subtree.
This results in some situations where we seemingly do not release the buffer reliably. We thus have calls to strbuf_release() for this buffer scattered across the codebase. But we're missing one callsite in git-merge(1), which causes a memory leak.
We should ideally refactor this interface so that callers don't have to know about any such internals. But for now, paper over the issue by adding one more strbuf_release() call.
--- builtin/merge.c | 1 + t/t6424-merge-unrelated-index-changes.sh | 1 + 2 files changed, 2 insertions(+)
builtin/merge.c +1 -0
@@ -754,6 +754,7 @@ static int try_merge_strategy(const char *strategy, struct commit_list *common, clean =merge_recursive (&o, head, remoteheads->item, reversed, &result);free_commit_list (reversed); +strbuf_release (&o.obuf);if (clean <0 ) {rollback_lock_file (&lock);
t/t6424-merge-unrelated-index-changes.sh +1 -0
@@ -2,6 +2,7 @@ test_description="merges with unrelated index changes" +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Testcase for some simple merges
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
PATCH v3 22/22 list-objects-filter-options: work around reported leak on error
In reply to Patrick SteinhardtThis one is a little bit more curious. In t6112, we have a test that exercises the git rev-list --filter option with invalid filters. We execute git-rev-list(1) via test_must_fail, which means that we check for leaks even though Git exits with an error code. This causes the following leak:
Direct leak of 27 byte(s) in 1 object(s) allocated from:
#0 0x5555555e6946 in realloc.part.0 lsan_interceptors.cpp.o
#1 0x5555558fb4b6 in xrealloc wrapper.c:137:8
#2 0x5555558b6e06 in strbuf_grow strbuf.c:112:2
#3 0x5555558b7550 in strbuf_add strbuf.c:311:2
#4 0x5555557c1a88 in strbuf_addstr strbuf.h:310:2
#5 0x5555557c1d4c in parse_list_objects_filter list-objects-filter-options.c:261:3
#6 0x555555885ead in handle_revision_pseudo_opt revision.c:2899:3
#7 0x555555884e20 in setup_revisions revision.c:3014:11
#8 0x5555556c4b42 in cmd_rev_list builtin/rev-list.c:588:9
#9 0x5555555ec5e3 in run_builtin git.c:483:11
#10 0x5555555eb1e4 in handle_builtin git.c:749:13
#11 0x5555555ec001 in run_argv git.c:819:4
#12 0x5555555eaf94 in cmd_main git.c:954:19
#13 0x5555556fd569 in main common-main.c:64:11
#14 0x7ffff7ca714d in __libc_start_call_main (.../lib/libc.so.6+0x2a14d)
#15 0x7ffff7ca7208 in __libc_start_main@GLIBC_2.2.5 (.../libc.so.6+0x2a208)
#16 0x5555555ad064 in _start (git+0x59064)This leak is valid, as we call die() and do not clean up the memory at all. But what's curious is that this is the only leak reported, because we don't clean up any other allocated memory, either, and I have no idea why the leak sanitizer treats this buffer specially.
In any case, we can work around the leak by shuffling things around a bit. Instead of calling gently_parse_list_objects_filter() and dying after we have modified the filter spec, we simply do so beforehand. Like this we don't allocate the buffer in the error case, which makes the reported leak go away.
It's not pretty, but it manages to make t6112 leak free.
--- list-objects-filter-options.c | 17 +++++++---------- t/t6112-rev-list-filters-objects.sh | 1 + 2 files changed, 8 insertions(+), 10 deletions(-)
list-objects-filter-options.c +7 -10
@@ -252,16 +252,14 @@ void parse_list_objects_filter( const char *arg) {struct strbuf errbuf =STRBUF_INIT ; -int parse_error;if (!filter_options->filter_spec.buf)BUG ("filter_options not properly initialized" );if (!filter_options->choice) { +if (gently_parse_list_objects_filter (filter_options, arg, &errbuf)) +die ("%s" , errbuf.buf);strbuf_addstr (&filter_options->filter_spec, arg); - - parse_error =gently_parse_list_objects_filter ( - filter_options, arg, &errbuf); }else {struct list_objects_filter_options *sub;
@@ -271,18 +269,17 @@ void parse_list_objects_filter(*/ transform_to_combine_type (filter_options); -strbuf_addch (&filter_options->filter_spec,'+' ); -filter_spec_append_urlencode (filter_options, arg);ALLOC_GROW_BY (filter_options->sub, filter_options->sub_nr,1 , filter_options->sub_alloc); sub = &filter_options->sub[filter_options->sub_nr -1 ];list_objects_filter_init (sub); - parse_error =gently_parse_list_objects_filter (sub, arg, - &errbuf); +if (gently_parse_list_objects_filter (sub, arg, &errbuf)) +die ("%s" , errbuf.buf); + +strbuf_addch (&filter_options->filter_spec,'+' ); +filter_spec_append_urlencode (filter_options, arg); } -if (parse_error) -die ("%s" , errbuf.buf); } int opt_parse_list_objects_filter(const struct option *opt,
t/t6112-rev-list-filters-objects.sh +1 -0
@@ -5,6 +5,7 @@ test_description='git rev-list using object filtering' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Test the blob:none filter.
Signature · 1 line
2.47.0.229.g8f8d6eee53.dirty
Quoted 9 lines · Patrick Steinhardt
Changes compared to v2:
- Remove an unnecessary cast.
- Fix a duplicate newline.
- Polish a couple of commit messages.
Thanks!
I spot checked the ones that did not change from v2 and the ones I checked at all looked sensible. Perhaps this is now ready for 'next'?
Thanks.
Quoted 8 lines · Junio C Hamano
Quoted 6 lines · Patrick Steinhardt
Changes compared to v2:
- Remove an unnecessary cast.
- Fix a duplicate newline.
Quoted 7 lines
Quoted 3 lines
- Polish a couple of commit messages.
Thanks!
I spot checked the ones that did not change from v2 and the ones I checked at all looked sensible. Perhaps this is now ready for 'next'?
From my point of view it should be ready, yes.
Patrick
Quoted 11 lines · Patrick Steinhardt
Quoted 10 lines · Junio C Hamano
Quoted 8 lines · Patrick Steinhardt
Changes compared to v2:
- Remove an unnecessary cast.
- Fix a duplicate newline.
- Polish a couple of commit messages.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
Thanks!
I spot checked the ones that did not change from v2 and the ones I checked at all looked sensible. Perhaps this is now ready for 'next'?
From my point of view it should be ready, yes.
From looking at the range-diff, this version looks good to me. :)
-Justin