Memory leak fixes (pt.5)
cover Memory leak fixes (pt.5)
Hi,
another week, another set of memory leak fixes. With this series we're down to ~128 leaking test suites, down from 359 before starting with this effort. So we're about 2/3rds done. Naturally, part 6 is also almost ready.
This series is built on top of bb9c16bd4f (The sixth batch, 2024-08-19) with Junio's ps/leakfixes-part-4 at 77d4b3dd73 (builtin/diff: free symmetric diff members, 2024-08-14) merged into it.
Thanks!
Patrick
Patrick Steinhardt (20):
mailinfo: fix leaking header data
convert: fix leaks when resetting attributes
pretty: fix memory leaks when parsing pretty formats
pretty: fix leaking key/value separator buffer builtin/merge-tree: fix leaking -X strategy options builtin/upload-archive: fix leaking args passed to write_archive() builtin/archive: fix leaking OPT_FILENAME() value
midx-write: fix leaking hashfile on error cases builtin/repack: fix leaks when computing packs to repack t/helper: fix leaking multi-pack-indices in "read-midx"
transport: fix leaking OID arrays in git:// transport data builtin/send-pack: fix leaking refspecs
sideband: fix leaks when configuring sideband colors builtin/fetch-pack: fix leaking refs
remote: fix leaking config strings
remote: fix leaks when matching refspecs
remote: fix leaking peer ref when expanding refmap builtin/fetch: fix leaking transaction with --atomic
transport: fix leaking arguments when fetching from bundle
transport: fix leaking negotiation tips
archive.c | 10 ++++++ builtin/archive.c | 7 ++-- builtin/fetch-pack.c | 20 ++++++----- builtin/fetch.c | 8 ++--- builtin/merge-tree.c | 13 +++++-- builtin/repack.c | 36 ++++++++++++++----- builtin/send-pack.c | 1 + builtin/upload-archive.c | 8 +++-- convert.c | 3 ++ mailinfo.c | 17 +++++++-- midx-write.c | 24 ++++++------- pretty.c | 13 +++++-- remote.c | 55 +++++++++++++++++++++-------- sideband.c | 8 +++-- t/helper/test-read-midx.c | 8 ++++- t/t4150-am.sh | 1 + t/t4205-log-pretty-formats.sh | 2 ++ t/t4301-merge-tree-write-tree.sh | 1 + t/t5000-tar-tree.sh | 1 + t/t5003-archive-zip.sh | 1 + t/t5100-mailinfo.sh | 1 + t/t5319-multi-pack-index.sh | 2 ++ t/t5400-send-pack.sh | 1 + t/t5401-update-hooks.sh | 2 ++ t/t5408-send-pack-stdin.sh | 2 ++ t/t5409-colorize-remote-messages.sh | 1 + t/t5501-fetch-push-alternates.sh | 1 + t/t5505-remote.sh | 1 + t/t5510-fetch.sh | 1 + t/t5519-push-alternates.sh | 1 + t/t5536-fetch-conflicts.sh | 1 + t/t5548-push-porcelain.sh | 1 + t/t5553-set-upstream.sh | 1 + t/t5574-fetch-output.sh | 1 + t/t5703-upload-pack-ref-in-want.sh | 1 + t/t5812-proto-disable-http.sh | 2 ++ t/t6050-replace.sh | 1 + t/t7704-repack-cruft.sh | 1 + transport.c | 8 +++++ 39 files changed, 203 insertions(+), 64 deletions(-)
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH 1/20 mailinfo: fix leaking header data
We populate the mailinfo arrays p_hdr_data and s_hdr_data with data parsed from the mail headers. These arrays may end up being only partially populated with gaps in case some of the headers do not parse properly. This causes memory leaks because strbuf_list_free() will stop iterating once it hits the first NULL pointer in the backing array.
Fix this by open-coding a variant of strbuf_list_free() that knows to iterate through all headers.
--- mailinfo.c | 17 +++++++++++++++-- t/t5100-mailinfo.sh | 1 + 2 files changed, 16 insertions(+), 2 deletions(-)
mailinfo.c +15 -2
@@ -1290,8 +1290,21 @@ void clear_mailinfo(struct mailinfo *mi)strbuf_release (&mi->inbody_header_accum);free (mi->message_id); -strbuf_list_free (mi->p_hdr_data); -strbuf_list_free (mi->s_hdr_data); +for (size_t i =0 ; header[i]; i++) { +if (!mi->p_hdr_data[i]) +continue ; +strbuf_release (mi->p_hdr_data[i]); +free (mi->p_hdr_data[i]); + } +free (mi->p_hdr_data); + +for (size_t i =0 ; header[i]; i++) { +if (!mi->s_hdr_data[i]) +continue ; +strbuf_release (mi->s_hdr_data[i]); +free (mi->s_hdr_data[i]); + } +free (mi->s_hdr_data);while (mi->content < mi->content_top) {free (*(mi->content_top));
t/t5100-mailinfo.sh +1 -0
@@ -5,6 +5,7 @@ test_description='git mailinfo and git mailsplit test' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh DATA="$TEST_DIRECTORY/t5100"
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
Re: [PATCH 01/20] mailinfo: fix leaking header data
Quoted 9 lines · Patrick Steinhardt
We populate the
mailinfoarraysp_hdr_dataands_hdr_datawith data parsed from the mail headers. These arrays may end up being only partially populated with gaps in case some of the headers do not parse properly. This causes memory leaks becausestrbuf_list_free()will stop iterating once it hits the firstNULLpointer in the backing array.Fix this by open-coding a variant of
strbuf_list_free()that knows to iterate through all headers.
Well spotted. An array of strbuf is often a wrong data structure for anything, but luckily we use it and suffer from a bug like this only rarely.
The fix makes sense. Will queue.
Quoted 47 lines
Signed-off-by Patrick Steinhardt <[email protected]>--- mailinfo.c | 17 +++++++++++++++-- t/t5100-mailinfo.sh | 1 + 2 files changed, 16 insertions(+), 2 deletions(-)mailinfo.c +15
-2@@ -1290,8 +1290,21 @@ void clear_mailinfo(struct mailinfo *mi)strbuf_release (&mi->inbody_header_accum);free (mi->message_id); -strbuf_list_free (mi->p_hdr_data); -strbuf_list_free (mi->s_hdr_data); +for (size_t i =0 ; header[i]; i++) { +if (!mi->p_hdr_data[i]) +continue ; +strbuf_release (mi->p_hdr_data[i]); +free (mi->p_hdr_data[i]); + } +free (mi->p_hdr_data); + +for (size_t i =0 ; header[i]; i++) { +if (!mi->s_hdr_data[i]) +continue ; +strbuf_release (mi->s_hdr_data[i]); +free (mi->s_hdr_data[i]); + } +free (mi->s_hdr_data);while (mi->content < mi->content_top) {free (*(mi->content_top));t/t5100-mailinfo.sh +1
-0@@ -5,6 +5,7 @@ test_description='git mailinfo and git mailsplit test' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh DATA="$TEST_DIRECTORY/t5100"
PATCH 2/20 convert: fix leaks when resetting attributes
In reply to Patrick SteinhardtWhen resetting parsed gitattributes, we free the list of convert drivers parsed from the config. We only free some of the drivers' fields though and thus have memory leaks.
Fix this by freeing all allocated convert driver fields to plug these memory leaks.
--- convert.c | 3 +++ t/t4150-am.sh | 1 + 2 files changed, 4 insertions(+)
convert.c +3 -0
@@ -1371,6 +1371,9 @@ void reset_parsed_attributes(void)for (drv = user_convert; drv; drv = next) { next = drv->next;free ((void *)drv->name); +free ((void *)drv->smudge); +free ((void *)drv->clean); +free ((void *)drv->process);free (drv); } user_convert =NULL ;
t/t4150-am.sh +1 -0
@@ -5,6 +5,7 @@ test_description='git am running' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_expect_success 'setup: messages' '
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
Re: [PATCH 02/20] convert: fix leaks when resetting attributes
Quoted 9 lines · Patrick Steinhardt
When resetting parsed gitattributes, we free the list of convert drivers parsed from the config. We only free some of the drivers' fields though and thus have memory leaks.
Fix this by freeing all allocated convert driver fields to plug these memory leaks.
Signed-off-by Patrick Steinhardt <[email protected]>---
The helper has only one caller, which makes me wonder why we do not have to call it more often. If "git checkout" or "log -p" notice an attribute file is updated, I wonder they should call it, for example. But of course this is completely out of scope of this topic.
This has been leaking even before 9642479a (convert: fix leaking config strings, 2024-08-01) straightened out the ownership rules of the value obtained from git_config_string().
The fix makes sense. Will queue. Thanks.
PATCH 3/20 pretty: fix memory leaks when parsing pretty formats
In reply to Patrick SteinhardtWhen parsing pretty formats from the config we leak the name and user format whenever these are set multiple times. This is because we do not free any already-set value in case there is one.
Plugging this leak for the name is trivial. For the user format we need to be a bit more careful, because we may end up assigning a pointer into the allocated region when the string is prefixed with either "format" or "tformat:". In order to make it safe to unconditionally free the user format we thus strdup the stripped string into the field instead of a pointer into the string.
--- pretty.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)
pretty.c +9 -3
@@ -63,7 +63,7 @@ static int git_pretty_formats_config(const char *var, const char *value, void *cb UNUSED) {struct cmt_fmt_map *commit_format =NULL ; -const char *name; +const char *name, *stripped;char *fmt;int i;
@@ -90,15 +90,21 @@ static int git_pretty_formats_config(const char *var, const char *value, commit_formats_len++; } +free ((char *) commit_format->name); commit_format->name =xstrdup (name); commit_format->format =CMIT_FMT_USERFORMAT ;if (git_config_string (&fmt, var, value))return -1 ; -if (skip_prefix (fmt,"format:" , &commit_format->user_format)) { +free ((char *) commit_format->user_format); +if (skip_prefix (fmt,"format:" , &stripped)) { commit_format->is_tformat =0 ; - }else if (skip_prefix (fmt,"tformat:" , &commit_format->user_format)) { + commit_format->user_format =xstrdup (stripped); +free (fmt); + }else if (skip_prefix (fmt,"tformat:" , &stripped)) { commit_format->is_tformat =1 ; + commit_format->user_format =xstrdup (stripped); +free (fmt); }else if (strchr (fmt,'%' )) { commit_format->is_tformat =1 ; commit_format->user_format = fmt;
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
Re: [PATCH 03/20] pretty: fix memory leaks when parsing pretty formats
Quoted 10 lines · Patrick Steinhardt
When parsing pretty formats from the config we leak the name and user format whenever these are set multiple times. This is because we do not free any already-set value in case there is one.
Plugging this leak for the name is trivial. For the user format we need to be a bit more careful, because we may end up assigning a pointer into the allocated region when the string is prefixed with either "format" or "tformat:". In order to make it safe to unconditionally free the user format we thus strdup the stripped string into the field instead of a pointer into the string.
Yup, the stripped one is trickier, but the change looks correct.
Will queue.
By the way, we tend to prefer no spaces after (cast) in our codebase, but I just noticed that it is not spelled out in the coding guidelines. Comparing
$ git grep -E -e '\((int|char \*)\) ' \*.c ':!compat/' ':!contrib/'
$ git grep -E -e '\((int|char \*)\)[^ ]' \*.c ':!compat/' ':!contrib/'tells me that the extra space after the (cast) is found mostly in borrowed or imported sources and majority of culprits are found in reftable library X-<.
Thanks.
--- >8 ---
Subject: CodingGuidelines: spaces around C operators
As we have operated with "write like how your surrounding code is written" for too long, after a huge code drop from another project, we'll end up being inconsistent before such an imported code is cleaned up. We have many uses of cast operator with a space before its operand, mostly in the reftable code.
Spell the convention out before it spreads to other places.
--- Documentation/CodingGuidelines | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
Documentation/CodingGuidelines +10 -1
@@ -303,7 +303,9 @@ For C programs: v12.01, 2022-03-28). - Variables have to be declared at the beginning of the block, before - the first statement (i.e. -Wdeclaration-after-statement). + the first statement (i.e. -Wdeclaration-after-statement). It is + encouraged to have a blank line between the end of the declarations + and the first statement in the block. - NULL pointers shall be written as NULL, not as 0.
@@ -323,6 +325,13 @@ For C programs: while( condition ) func (bar+1); + - A binary operator (other than ",") and ternary conditional "?:" + have a space on each side of the operator to separate it from its + operands. E.g. "A + 1", not "A+1". + + - A unary operator (other than "." and "->") have no space between it + and its operand. E.g. "(char *)ptr", not "(char *) ptr". + - Do not explicitly compare an integral value with constant 0 or '\0', or a pointer value with constant NULL. For instance, to validate that counted array <ptr, cnt> is initialized but has no elements, write:
Re: [PATCH 03/20] pretty: fix memory leaks when parsing pretty formats
Quoted 21 lines · Junio C Hamano
Quoted 10 lines · Patrick Steinhardt
When parsing pretty formats from the config we leak the name and user format whenever these are set multiple times. This is because we do not free any already-set value in case there is one.
Plugging this leak for the name is trivial. For the user format we need to be a bit more careful, because we may end up assigning a pointer into the allocated region when the string is prefixed with either "format" or "tformat:". In order to make it safe to unconditionally free the user format we thus strdup the stripped string into the field instead of a pointer into the string.
Yup, the stripped one is trickier, but the change looks correct.
Will queue.
By the way, we tend to prefer no spaces after (cast) in our codebase, but I just noticed that it is not spelled out in the coding guidelines. Comparing
Quoted 6 lines
$ git grep -E -e '\((int|char \*)\) ' \*.c ':!compat/' ':!contrib/' $ git grep -E -e '\((int|char \*)\)[^ ]' \*.c ':!compat/' ':!contrib/'tells me that the extra space after the (cast) is found mostly in borrowed or imported sources and majority of culprits are found in reftable library X-<.
Not entirely surprising. I myself have typically favored adding a space after the cast, and I remember wondering about the coding style several times here. I never wondered enough to actually check though.
So thanks for clarifying and thanks for updating the coding guidelines around this.
Patrick
PATCH 4/20 pretty: fix leaking key/value separator buffer
In reply to Patrick SteinhardtThe format_set_trailers_options() function is responsible for parsing a custom pretty format for trailers. It puts the parsed options into a struct process_trailer_options structure, while the allocated memory required for this will be put into separate caller-provided arguments. It is thus the caller's responsibility to free the memory not via the options structure, but via the other parameters.
While we do this alright for the separator and filter keys, we do not free the memory associated with the key/value separator. Fix this to plug this memory leak.
--- pretty.c | 1 + t/t4205-log-pretty-formats.sh | 2 ++ 2 files changed, 3 insertions(+)
pretty.c +1 -0
@@ -1776,6 +1776,7 @@ static size_t format_commit_one(struct strbuf *sb, /* in UTF-8 */ } trailer_out:string_list_clear (&filter_list,0 ); +strbuf_release (&kvsepbuf);strbuf_release (&sepbuf);return ret; }
t/t4205-log-pretty-formats.sh +2 -0
@@ -5,6 +5,8 @@ # test_description='Test pretty formats' + +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Tested non-UTF-8 encoding
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH 5/20 builtin/merge-tree: fix leaking `-X` strategy options
In reply to Patrick SteinhardtThe -X switch for git-merge-tree(1) will push each option into a local xopts vector that we then end up parsing. The vector never gets freed though, causing a memory leak. Plug it.
--- builtin/merge-tree.c | 13 ++++++++++--- t/t4301-merge-tree-write-tree.sh | 1 + 2 files changed, 11 insertions(+), 3 deletions(-)
builtin/merge-tree.c +10 -3
@@ -533,6 +533,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)int expected_remaining_argc;int original_argc;const char *merge_base =NULL ; +int ret;const char *const merge_tree_usage[] = {N_ ("git merge-tree [--write-tree] [<options>] <branch1> <branch2>" ),
@@ -625,7 +626,9 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)strbuf_list_free (split); }strbuf_release (&buf); -return 0 ; + + ret =0 ; + goto out; }/* Figure out which mode to use */
@@ -664,7 +667,11 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)/* Do the relevant type of merge */ if (o.mode ==MODE_REAL ) -return real_merge (&o, merge_base, argv[0 ], argv[1 ], prefix); + ret =real_merge (&o, merge_base, argv[0 ], argv[1 ], prefix);else -return trivial_merge (argv[0 ], argv[1 ], argv[2 ]); + ret =trivial_merge (argv[0 ], argv[1 ], argv[2 ]); + +out: +strvec_clear (&xopts); +return ret; }
t/t4301-merge-tree-write-tree.sh +1 -0
@@ -2,6 +2,7 @@ test_description='git merge-tree --write-tree' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # This test is ort-specific
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
Re: [PATCH 05/20] builtin/merge-tree: fix leaking `-X` strategy options
Quoted 3 lines · Patrick Steinhardt
The
-Xswitch for git-merge-tree(1) will push each option into a localxoptsvector that we then end up parsing. The vector never gets freed though, causing a memory leak. Plug it.
It is unfortunate that the ownership rules of the second parameter to parse_merge_opt() lets the string borrowed by the merge_options struct. Otherwise we could have just cleared xopts immediately after we are done with calling it.
The clean-up looks correct. Will queue.
Thanks.
Quoted 56 lines
Signed-off-by Patrick Steinhardt <[email protected]>--- builtin/merge-tree.c | 13 ++++++++++--- t/t4301-merge-tree-write-tree.sh | 1 + 2 files changed, 11 insertions(+), 3 deletions(-)builtin/merge-tree.c +10
-3@@ -533,6 +533,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)int expected_remaining_argc;int original_argc;const char *merge_base =NULL ; +int ret;const char *const merge_tree_usage[] = {N_ ("git merge-tree [--write-tree] [<options>] <branch1> <branch2>" ),@@ -625,7 +626,9 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)strbuf_list_free (split); }strbuf_release (&buf); -return 0 ; + + ret =0 ; + goto out; }/* Figure out which mode to use */ @@ -664,7 +667,11 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)/* Do the relevant type of merge */ if (o.mode ==MODE_REAL ) -return real_merge (&o, merge_base, argv[0 ], argv[1 ], prefix); + ret =real_merge (&o, merge_base, argv[0 ], argv[1 ], prefix);else -return trivial_merge (argv[0 ], argv[1 ], argv[2 ]); + ret =trivial_merge (argv[0 ], argv[1 ], argv[2 ]); + +out: +strvec_clear (&xopts); +return ret; }t/t4301-merge-tree-write-tree.sh +1
-0@@ -2,6 +2,7 @@ test_description='git merge-tree --write-tree' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # This test is ort-specific
PATCH 6/20 builtin/upload-archive: fix leaking args passed to `write_archive()`
In reply to Patrick SteinhardtIn git-upload-archive(1), we pass an array of arguments to write_archive() to tell it what exactly to do. We don't ever clear the vector though, causing a memory leak. Furthermore though, the call to write_archive() may cause contents of the array to be modified, which would cause us to leak memory to allocated strings held by it.
Fix the issue by having write_archive() create a shallow copy of argv before parsing the arguments. Like this, we won't modify the caller's array and can easily strvec_clear() it to plug these memory leaks.
--- archive.c | 10 ++++++++++ builtin/upload-archive.c | 8 ++++++-- 2 files changed, 16 insertions(+), 2 deletions(-)
archive.c +10 -0
@@ -736,6 +736,7 @@ int write_archive(int argc, const char **argv, const char *prefix,struct pretty_print_describe_status describe_status = {0 };struct pretty_print_context ctx = {0 };struct archiver_args args; +const char **argv_copy;int rc;git_config_get_bool ("uploadarchive.allowunreachable" , &remote_allow_unreachable);
@@ -749,6 +750,14 @@ int write_archive(int argc, const char **argv, const char *prefix, args.repo = repo; args.prefix = prefix;string_list_init_dup (&args.extra_files); + +/* +* `parse_archive_args()` modifies contents of `argv`, which is what we +* want. Our callers may not want it though, so we create a copy here. +*/ +DUP_ARRAY (argv_copy, argv, argc); + argv = argv_copy; + argc =parse_archive_args (argc, argv, &ar, &args, name_hint, remote);if (!startup_info->have_repository) {/*
@@ -767,6 +776,7 @@ int write_archive(int argc, const char **argv, const char *prefix,string_list_clear_func (&args.extra_files, extra_file_info_clear);free (args.refname);clear_pathspec (&args.pathspec); +free (argv_copy);return rc; }
builtin/upload-archive.c +6 -2
@@ -22,6 +22,7 @@ int cmd_upload_archive_writer(int argc, const char **argv, const char *prefix) {struct strvec sent_argv =STRVEC_INIT ;const char *arg_cmd ="argument " ; +int ret;if (argc !=2 || !strcmp (argv[1 ],"-h" ))usage (upload_archive_usage);
@@ -46,8 +47,11 @@ int cmd_upload_archive_writer(int argc, const char **argv, const char *prefix) }/* parse all options sent by the client */ -return write_archive (sent_argv.nr, sent_argv.v, prefix, - the_repository,NULL ,1 ); + ret =write_archive (sent_argv.nr, sent_argv.v, prefix, + the_repository,NULL ,1 ); + +strvec_clear (&sent_argv); +return ret; } __attribute__((format (printf, 1, 2)))
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH 7/20 builtin/archive: fix leaking `OPT_FILENAME()` value
In reply to Patrick SteinhardtThe "--output" switch is an OPT_FILENAME() option, which allocates memory when specified by the user. But while we free the string when executed without the "--remote" switch, we don't otherwise because we return via a separate exit path that doesn't know to free it.
Fix this by creating a common exit path.
--- builtin/archive.c | 7 +++++-- t/t5000-tar-tree.sh | 1 + t/t5003-archive-zip.sh | 1 + 3 files changed, 7 insertions(+), 2 deletions(-)
builtin/archive.c +5 -2
@@ -100,13 +100,16 @@ int cmd_archive(int argc, const char **argv, const char *prefix)if (output)create_output_file (output); -if (remote) -return run_remote_archiver (argc, argv, remote, exec, output); +if (remote) { + ret =run_remote_archiver (argc, argv, remote, exec, output); + goto out; + }setvbuf (stderr,NULL , _IOLBF,BUFSIZ ); ret =write_archive (argc, argv, prefix, the_repository, output,0 ); +out:free (output);return ret; }
t/t5000-tar-tree.sh +1 -0
@@ -25,6 +25,7 @@ commit id embedding: ' TEST_CREATE_REPO_NO_TEMPLATE=1 +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh SUBSTFORMAT=%H%n
t/t5003-archive-zip.sh +1 -0
@@ -3,6 +3,7 @@ test_description='git archive --format=zip test' TEST_CREATE_REPO_NO_TEMPLATE=1 +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh SUBSTFORMAT=%H%n
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH 8/20 midx-write: fix leaking hashfile on error cases
In reply to Patrick SteinhardtWhen writing the MIDX file we first create the struct hashfile used to write the trailer hash, and then afterwards we verify whether we can actually write the MIDX in the first place. When we decide that we can't, this leads to a memory leak because we never free the hash file contents.
We could fix this by freeing the hashfile on the exit path. There is a better option though: we can simply move the checks for the error condition earlier. As there is no early exit between creating the hashfile and finalizing it anymore this is sufficient to fix the memory leak.
--- midx-write.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-)
midx-write.c +12 -12
@@ -1308,6 +1308,18 @@ static int write_midx_internal(const char *object_dir, pack_name_concat_len +=MIDX_CHUNK_ALIGNMENT - (pack_name_concat_len %MIDX_CHUNK_ALIGNMENT ); +if (ctx.nr - dropped_packs ==0 ) { +error (_ ("no pack files to index." )); + result =1 ; + goto cleanup; + } + +if (!ctx.entries_nr) { +if (flags &MIDX_WRITE_BITMAP ) +warning (_ ("refusing to write multi-pack .bitmap without any objects" )); + flags &= ~(MIDX_WRITE_REV_INDEX |MIDX_WRITE_BITMAP ); + } +if (ctx.incremental) {struct strbuf lock_name =STRBUF_INIT ;
@@ -1333,18 +1345,6 @@ static int write_midx_internal(const char *object_dir, f =hashfd (get_lock_file_fd (&lk),get_lock_file_path (&lk)); } -if (ctx.nr - dropped_packs ==0 ) { -error (_ ("no pack files to index." )); - result =1 ; - goto cleanup; - } - -if (!ctx.entries_nr) { -if (flags &MIDX_WRITE_BITMAP ) -warning (_ ("refusing to write multi-pack .bitmap without any objects" )); - flags &= ~(MIDX_WRITE_REV_INDEX |MIDX_WRITE_BITMAP ); - } - cf =init_chunkfile (f); add_chunk(cf, MIDX_CHUNKID_PACKNAMES, pack_name_concat_len,
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
Re: [PATCH 08/20] midx-write: fix leaking hashfile on error cases
Quoted 11 lines · Patrick Steinhardt
When writing the MIDX file we first create the
struct hashfileused to write the trailer hash, and then afterwards we verify whether we can actually write the MIDX in the first place. When we decide that we can't, this leads to a memory leak because we never free the hash file contents.We could fix this by freeing the hashfile on the exit path. There is a better option though: we can simply move the checks for the error condition earlier. As there is no early exit between creating the hashfile and finalizing it anymore this is sufficient to fix the memory leak.
The above is a good explanation why "are we dropping everything" block was moved up, but does not explain why the other "if there is no objects" block has to move (it is however easy to see that the latter block can be moved without any bad side effect).
In any case, the struct hashfile hashfd() gives us is now associated with the struct chunkfile cf immediately after it is instanciated, and there is no early exit while the chunkfile is in use, which is great.
Re: [PATCH 08/20] midx-write: fix leaking hashfile on error cases
Quoted 19 lines · Junio C Hamano
Quoted 11 lines · Patrick Steinhardt
When writing the MIDX file we first create the
struct hashfileused to write the trailer hash, and then afterwards we verify whether we can actually write the MIDX in the first place. When we decide that we can't, this leads to a memory leak because we never free the hash file contents.We could fix this by freeing the hashfile on the exit path. There is a better option though: we can simply move the checks for the error condition earlier. As there is no early exit between creating the hashfile and finalizing it anymore this is sufficient to fix the memory leak.
The above is a good explanation why "are we dropping everything" block was moved up, but does not explain why the other "if there is no objects" block has to move (it is however easy to see that the latter block can be moved without any bad side effect).
Quoted 4 lines
In any case, the struct hashfile
hashfd()gives us is now associated with the struct chunkfile cf immediately after it is instanciated, and there is no early exit while the chunkfile is in use, which is great.
There basically is no reason why the other block needs to move. It just felt more natural to keep massaging of parameters closly together before we go off and actually do "the thing". I'll document this accordingly.
Patrick
PATCH 9/20 builtin/repack: fix leaks when computing packs to repack
In reply to Patrick SteinhardtWhen writing an MIDX in git-repack(1) we first collect all the pack names that we want to add to it in a string list. This list is marked as NODUP, which indicates that it will neither duplicate nor own strings added to it. In write_midx_included_packs() we then insert() strings via xstrdup() or strbuf_detach(), but the resulting strings will not be owned by anything and thus leak.
Fix this issue by marking the list as DUP and using a local buffer to compute the pack names.
This leak is hit in t5319, but plugging it is not sufficient to make the whole test suite pass.
--- builtin/repack.c | 36 +++++++++++++++++++++++++++--------- 1 file changed, 27 insertions(+), 9 deletions(-)
builtin/repack.c +27 -9
@@ -732,14 +732,23 @@ static void midx_included_packs(struct string_list *include, struct pack_geometry *geometry) {struct string_list_item *item; +struct strbuf buf =STRBUF_INIT ; + +for_each_string_list_item (item, &existing->kept_packs) { +strbuf_reset (&buf); +strbuf_addf (&buf,"%s.idx" , item->string); +string_list_insert (include, buf.buf); + } + +for_each_string_list_item (item, names) { +strbuf_reset (&buf); +strbuf_addf (&buf,"pack-%s.idx" , item->string); +string_list_insert (include, buf.buf); + } -for_each_string_list_item (item, &existing->kept_packs) -string_list_insert (include,xstrfmt ("%s.idx" , item->string)); -for_each_string_list_item (item, names) -string_list_insert (include,xstrfmt ("pack-%s.idx" , item->string));if (geometry->split_factor) { -struct strbuf buf =STRBUF_INIT ;uint32_t i; +for (i = geometry->split; i < geometry->pack_nr; i++) {struct packed_git *p = geometry->pack[i];
@@ -754,17 +763,21 @@ static void midx_included_packs(struct string_list *include,if (!p->pack_local)continue ; +strbuf_reset (&buf);strbuf_addstr (&buf,pack_basename (p));strbuf_strip_suffix (&buf,".pack" );strbuf_addstr (&buf,".idx" ); -string_list_insert (include,strbuf_detach (&buf,NULL )); +string_list_insert (include, buf.buf); } }else {for_each_string_list_item (item, &existing->non_kept_packs) {if (pack_is_marked_for_deletion (item))continue ; -string_list_insert (include,xstrfmt ("%s.idx" , item->string)); + +strbuf_reset (&buf); +strbuf_addf (&buf,"%s.idx" , item->string); +string_list_insert (include, buf.buf); } }
@@ -784,8 +797,13 @@ static void midx_included_packs(struct string_list *include,*/ if (pack_is_marked_for_deletion (item))continue ; -string_list_insert (include,xstrfmt ("%s.idx" , item->string)); + +strbuf_reset (&buf); +strbuf_addf (&buf,"%s.idx" , item->string); +string_list_insert (include, buf.buf); } + +strbuf_release (&buf); } static int write_midx_included_packs(struct string_list *include,
@@ -1476,7 +1494,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix)mark_packs_for_deletion (&existing, &names);if (write_midx) { -struct string_list include =STRING_LIST_INIT_NODUP ; +struct string_list include =STRING_LIST_INIT_DUP ;midx_included_packs (&include, &existing, &names, &geometry); ret = write_midx_included_packs(&include, &geometry, &names,
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH 10/20 t/helper: fix leaking multi-pack-indices in "read-midx"
In reply to Patrick SteinhardtSeveral of the subcommands of test-helper read-midx do not close the MIDX that they have opened, leading to memory leaks. Fix those.
--- t/helper/test-read-midx.c | 8 +++++++- t/t5319-multi-pack-index.sh | 2 ++ t/t7704-repack-cruft.sh | 1 + 3 files changed, 10 insertions(+), 1 deletion(-)
t/helper/test-read-midx.c +7 -1
@@ -86,6 +86,8 @@ static int read_midx_checksum(const char *object_dir)if (!m)return 1 ;printf ("%s\n" ,hash_to_hex (get_midx_checksum (m))); + +close_midx (m);return 0 ; }
@@ -102,10 +104,12 @@ static int read_midx_preferred_pack(const char *object_dir)if (midx_preferred_pack (midx, &preferred_pack) <0 ) {warning (_ ("could not determine MIDX preferred pack" )); +close_midx (midx);return 1 ; }printf ("%s\n" , midx->pack_names[preferred_pack]); +close_midx (midx);return 0 ; }
@@ -122,8 +126,10 @@ static int read_midx_bitmapped_packs(const char *object_dir)return 1 ;for (i =0 ; i < midx->num_packs + midx->num_packs_in_base; i++) { -if (nth_bitmapped_pack (the_repository, midx, &pack, i) <0 ) +if (nth_bitmapped_pack (the_repository, midx, &pack, i) <0 ) { +close_midx (midx);return 1 ; + }printf ("%s\n" ,pack_basename (pack.p));printf (" bitmap_pos: %" PRIuMAX"\n" , (uintmax_t )pack.bitmap_pos);
t/t5319-multi-pack-index.sh +2 -0
@@ -1,6 +1,8 @@ #!/bin/sh test_description='multi-pack-indexes' + +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-chunk.sh . "$TEST_DIRECTORY"/lib-midx.sh
t/t7704-repack-cruft.sh +1 -0
@@ -2,6 +2,7 @@ test_description='git repack works correctly' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh objdir=.git/objects
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH 11/20 transport: fix leaking OID arrays in git:// transport data
In reply to Patrick SteinhardtThe transport data for the "git://" protocol contains two OID arrays that we never free, creating a memory leak. Plug them.
--- t/t5501-fetch-push-alternates.sh | 1 + t/t5519-push-alternates.sh | 1 + transport.c | 2 ++ 3 files changed, 4 insertions(+)
t/t5501-fetch-push-alternates.sh +1 -0
@@ -4,6 +4,7 @@ test_description='fetch/push involving alternates' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh count_objects () {
t/t5519-push-alternates.sh +1 -0
@@ -5,6 +5,7 @@ test_description='push to a repository that borrows from elsewhere' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_expect_success setup '
transport.c +2 -0
@@ -946,6 +946,8 @@ static int disconnect_git(struct transport *transport) }list_objects_filter_release (&data->options.filter_options); +oid_array_clear (&data->extra_have); +oid_array_clear (&data->shallow);free (data);return 0 ; }
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH 12/20 builtin/send-pack: fix leaking refspecs
In reply to Patrick SteinhardtWe never free data associated with the assembled refspec in git-send-pack(1), causing a memory leak. Fix this.
--- builtin/send-pack.c | 1 + t/t5400-send-pack.sh | 1 + t/t5401-update-hooks.sh | 2 ++ t/t5408-send-pack-stdin.sh | 2 ++ t/t5548-push-porcelain.sh | 1 + t/t5812-proto-disable-http.sh | 2 ++ 6 files changed, 9 insertions(+)
builtin/send-pack.c +1 -0
@@ -338,5 +338,6 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix)free_refs (remote_refs);free_refs (local_refs); +refspec_clear (&rs);return ret; }
t/t5400-send-pack.sh +1 -0
@@ -9,6 +9,7 @@ test_description='See why rewinding head breaks send-pack GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh cnt=64
t/t5401-update-hooks.sh +2 -0
@@ -4,6 +4,8 @@ # test_description='Test the update hook infrastructure.' + +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_expect_success setup '
t/t5408-send-pack-stdin.sh +2 -0
@@ -1,6 +1,8 @@ #!/bin/sh test_description='send-pack --stdin tests' + +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh create_ref () {
t/t5548-push-porcelain.sh +1 -0
@@ -4,6 +4,7 @@ # test_description='Test git push porcelain output' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Create commits in <repo> and assign each commit's oid to shell variables
t/t5812-proto-disable-http.sh +2 -0
@@ -1,6 +1,8 @@ #!/bin/sh test_description='test disabling of git-over-http in clone/fetch' + +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY/lib-proto-disable.sh" . "$TEST_DIRECTORY/lib-httpd.sh"
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH 13/20 sideband: fix leaks when configuring sideband colors
In reply to Patrick SteinhardtWe read a bunch of configs in use_sideband_colors() to configure the colors that Git should use. We never free the strings read from the config though, causing memory leaks. Fix those.
--- sideband.c | 8 +++++--- t/t5409-colorize-remote-messages.sh | 1 + 2 files changed, 6 insertions(+), 3 deletions(-)
sideband.c +5 -3
@@ -30,7 +30,7 @@ static int use_sideband_colors(void)const char *key ="color.remote" ;struct strbuf sb =STRBUF_INIT ; -char *value; +char *value =NULL ;int i;if (use_sideband_colors_cached >=0 )
@@ -43,15 +43,17 @@ static int use_sideband_colors(void) }else { use_sideband_colors_cached =GIT_COLOR_AUTO ; } +FREE_AND_NULL (value);for (i =0 ; i <ARRAY_SIZE (keywords); i++) {strbuf_reset (&sb);strbuf_addf (&sb,"%s.%s" , key, keywords[i].keyword);if (git_config_get_string (sb.buf, &value))continue ; -if (color_parse (value, keywords[i].color)) -continue ; +color_parse (value, keywords[i].color); +FREE_AND_NULL (value); } +strbuf_release (&sb);return use_sideband_colors_cached; }
t/t5409-colorize-remote-messages.sh +1 -0
@@ -2,6 +2,7 @@ test_description='remote messages are colorized on the client' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_expect_success 'setup' '
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
Re: [PATCH 13/20] sideband: fix leaks when configuring sideband colors
Quoted 15 lines · Patrick Steinhardt
We read a bunch of configs in
use_sideband_colors()to configure the colors that Git should use. We never free the strings read from the config though, causing memory leaks. Fix those.Signed-off-by Patrick Steinhardt <[email protected]>--- sideband.c | 8 +++++--- t/t5409-colorize-remote-messages.sh | 1 + 2 files changed, 6 insertions(+), 3 deletions(-)sideband.c +0
-0
sideband.c static int use_sideband_colors(void) · line 33
Quoted 5 lines
sideband.c +1
-1@@ -30,7 +30,7 @@ static int use_sideband_colors(void)const char *key ="color.remote" ;struct strbuf sb =STRBUF_INIT ; -char *value; +char *value =NULL ;
Hmph, this is a bit unfortunate. If there is no configuration variable, on the first call to this function, we come to the end of if/else cascade and we need to free.
Another possibility to convey the intention better may be to assign NULL to value after the "we already know the value, so return early" took place.
sideband.c static int use_sideband_colors(void) · line 46
Quoted 5 lines
sideband.c +1
-0@@ -43,15 +43,17 @@ static int use_sideband_colors(void) }else { use_sideband_colors_cached =GIT_COLOR_AUTO ; } +FREE_AND_NULL (value);
This can be a simple "free(value)"; I do not see the need to clear the variable at this point.
sideband.c static int use_sideband_colors(void) · line 54
Quoted 9 lines
sideband.c +2
-2for (i =0 ; i <ARRAY_SIZE (keywords); i++) {strbuf_reset (&sb);strbuf_addf (&sb,"%s.%s" , key, keywords[i].keyword);if (git_config_get_string (sb.buf, &value))continue ; -if (color_parse (value, keywords[i].color)) -continue ; +color_parse (value, keywords[i].color); +FREE_AND_NULL (value);
Likewise. I do not see the need to clear. We only come here after we got something from gti_config_get_string(). That value may or may not fail color_parse(), but when we reach this point, value always has something we need to free, not some leftover value from the previous iteration.
The patch is _not_ wrong per-se, though.
Thanks.
Re: [PATCH 13/20] sideband: fix leaks when configuring sideband colors
Quoted 27 lines · Junio C Hamano
Quoted 15 lines · Patrick Steinhardt
We read a bunch of configs in
use_sideband_colors()to configure the colors that Git should use. We never free the strings read from the config though, causing memory leaks. Fix those.Signed-off-by Patrick Steinhardt <[email protected]>--- sideband.c | 8 +++++--- t/t5409-colorize-remote-messages.sh | 1 + 2 files changed, 6 insertions(+), 3 deletions(-)sideband.c +0
-0sideband.c static int use_sideband_colors(void) · line 33
Quoted 5 lines
sideband.c +1
-1@@ -30,7 +30,7 @@ static int use_sideband_colors(void)const char *key ="color.remote" ;struct strbuf sb =STRBUF_INIT ; -char *value; +char *value =NULL ;Hmph, this is a bit unfortunate. If there is no configuration variable, on the first call to this function, we come to the end of if/else cascade and we need to free.
Quoted 3 lines
Another possibility to convey the intention better may be to assign NULL to value after the "we already know the value, so return early" took place.
There is an even better option: just use git_config_get_string_tmp(). Then we don't have to worry about freeing the strings at all.
Patrick
PATCH 14/20 builtin/fetch-pack: fix leaking refs
In reply to Patrick SteinhardtWe build several ref lists in git-fetch-pack(1), but never free them. Fix those leaks.
--- builtin/fetch-pack.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-)
builtin/fetch-pack.c +12 -8
@@ -46,7 +46,7 @@ static void add_sought_entry(struct ref ***sought, int *nr, int *alloc, int cmd_fetch_pack(int argc, const char **argv, const char *prefix UNUSED) {int i, ret; -struct ref *ref =NULL ; +struct ref *fetched_refs =NULL , *remote_refs =NULL ;const char *dest =NULL ;struct ref **sought =NULL ;int nr_sought =0 , alloc_sought =0 ;
@@ -228,19 +228,20 @@ int cmd_fetch_pack(int argc, const char **argv, const char *prefix UNUSED) version =discover_version (&reader);switch (version) {case protocol_v2: -get_remote_refs (fd[1 ], &reader, &ref,0 ,NULL ,NULL , +get_remote_refs (fd[1 ], &reader, &remote_refs,0 ,NULL ,NULL , args.stateless_rpc);break ;case protocol_v1:case protocol_v0: -get_remote_heads (&reader, &ref,0 ,NULL , &shallow); +get_remote_heads (&reader, &remote_refs,0 ,NULL , &shallow);break ;case protocol_unknown_version:BUG ("unknown protocol version" ); } - ref =fetch_pack (&args, fd, ref, sought, nr_sought, + fetched_refs =fetch_pack (&args, fd, remote_refs, sought, nr_sought, &shallow, pack_lockfiles_ptr, version); +if (pack_lockfiles.nr) {int i;
@@ -260,7 +261,7 @@ int cmd_fetch_pack(int argc, const char **argv, const char *prefix UNUSED)if (finish_connect (conn))return 1 ; - ret = !ref; + ret = !fetched_refs;/* * If the heads to pull were given, we should have consumed
@@ -270,11 +271,14 @@ int cmd_fetch_pack(int argc, const char **argv, const char *prefix UNUSED)*/ ret |=report_unmatched_refs (sought, nr_sought); -while (ref) { +for (struct ref *ref = fetched_refs; ref; ref = ref->next)printf ("%s %s\n" ,oid_to_hex (&ref->old_oid), ref->name); - ref = ref->next; - } +for (size_t i =0 ; i < nr_sought; i++) +free_one_ref (sought[i]); +free (sought); +free_refs (fetched_refs); +free_refs (remote_refs);return ret; }
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
Re: [PATCH 14/20] builtin/fetch-pack: fix leaking refs
Quoted 7 lines · Patrick Steinhardt
We build several ref lists in git-fetch-pack(1), but never free them. Fix those leaks.
Signed-off-by Patrick Steinhardt <[email protected]>--- builtin/fetch-pack.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-)
Quite straight-forward. In addition to plugging leaks, the resulting code is much easier to reason about by avoiding the reuse of the variable.
PATCH 15/20 remote: fix leaking config strings
In reply to Patrick SteinhardtWe're leaking several config strings when assembling remotes, either because we do not free preceding values in case a config was set multiple times, or because we do not free them when releasing the remote state. Plug those leaks.
--- remote.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
remote.c +9 -1
@@ -373,8 +373,10 @@ static int handle_config(const char *key, const char *value,return -1 ; branch =make_branch (remote_state, name, namelen);if (!strcmp (subkey,"remote" )) { +FREE_AND_NULL (branch->remote_name);return git_config_string (&branch->remote_name, key, value); }else if (!strcmp (subkey,"pushremote" )) { +FREE_AND_NULL (branch->pushremote_name);return git_config_string (&branch->pushremote_name, key, value); }else if (!strcmp (subkey,"merge" )) {if (!value)
@@ -406,9 +408,11 @@ static int handle_config(const char *key, const char *value,return 0 ;/* Handle remote.* variables */ -if (!name && !strcmp (subkey,"pushdefault" )) +if (!name && !strcmp (subkey,"pushdefault" )) { +FREE_AND_NULL (remote_state->pushremote_name);return git_config_string (&remote_state->pushremote_name, key, value); + }if (!name)return 0 ;
@@ -475,12 +479,15 @@ static int handle_config(const char *key, const char *value,else if (!strcmp (value,"--tags" )) remote->fetch_tags =2 ; }else if (!strcmp (subkey,"proxy" )) { +FREE_AND_NULL (remote->http_proxy);return git_config_string (&remote->http_proxy, key, value); }else if (!strcmp (subkey,"proxyauthmethod" )) { +FREE_AND_NULL (remote->http_proxy_authmethod);return git_config_string (&remote->http_proxy_authmethod, key, value); }else if (!strcmp (subkey,"vcs" )) { +FREE_AND_NULL (remote->foreign_vcs);return git_config_string (&remote->foreign_vcs, key, value); }return 0 ;
@@ -2802,6 +2809,7 @@ void remote_state_clear(struct remote_state *remote_state)for (i =0 ; i < remote_state->remotes_nr; i++)remote_clear (remote_state->remotes[i]);FREE_AND_NULL (remote_state->remotes); +FREE_AND_NULL (remote_state->pushremote_name); remote_state->remotes_alloc =0 ; remote_state->remotes_nr =0 ;
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
Re: [PATCH 15/20] remote: fix leaking config strings
remote.c void remote_state_clear(struct remote_state *remote_state) · line 2814
Quoted 7 lines · Patrick Steinhardt
remote.c +1
-0@@ -2802,6 +2809,7 @@ void remote_state_clear(struct remote_state *remote_state)for (i =0 ; i < remote_state->remotes_nr; i++)remote_clear (remote_state->remotes[i]);FREE_AND_NULL (remote_state->remotes); +FREE_AND_NULL (remote_state->pushremote_name); remote_state->remotes_alloc =0 ; remote_state->remotes_nr =0 ;
As remote_state has two extra structures embedded in it, I wonder if we should be clearing them in this function, but possibly it is cleared elsewhere or perhaps in a later series?
As the focus of this step is about strings that we obtained from the config API, it is totally outside the scope of this topic, even if it turns out to be needed to clear them.
Looking good. Thanks.
Re: [PATCH 15/20] remote: fix leaking config strings
Quoted 13 lines · Junio C Hamano
remote.c void remote_state_clear(struct remote_state *remote_state) · line 2814
Quoted 7 lines · Patrick Steinhardt
remote.c +1
-0@@ -2802,6 +2809,7 @@ void remote_state_clear(struct remote_state *remote_state)for (i =0 ; i < remote_state->remotes_nr; i++)remote_clear (remote_state->remotes[i]);FREE_AND_NULL (remote_state->remotes); +FREE_AND_NULL (remote_state->pushremote_name); remote_state->remotes_alloc =0 ; remote_state->remotes_nr =0 ;As
remote_statehas two extra structures embedded in it, I wonder if we should be clearing them in this function, but possibly it is cleared elsewhere or perhaps in a later series?
It is not yet part of any subsequent patch series, mostly because I didn't happen to stumble over such leaks yet. Both of the rewrites very much are leaky though, and would be hit when we use "insteadOf" or "pushInsteadOf" configs.
The struct branch also needs handling and is being populated via "branch" configs.
Quoted 3 lines
As the focus of this step is about strings that we obtained from the config API, it is totally outside the scope of this topic, even if it turns out to be needed to clear them.
Well, these are being populated via config strings. So I'd rather fix them in this commit, as well.
Patrick
Re: [PATCH 15/20] remote: fix leaking config strings
Quoted 15 lines · Patrick Steinhardt
Quoted 13 lines · Junio C Hamano
remote.c void remote_state_clear(struct remote_state *remote_state) · line 2814
Quoted 7 lines · Patrick Steinhardt
remote.c +1
-0@@ -2802,6 +2809,7 @@ void remote_state_clear(struct remote_state *remote_state)for (i =0 ; i < remote_state->remotes_nr; i++)remote_clear (remote_state->remotes[i]);FREE_AND_NULL (remote_state->remotes); +FREE_AND_NULL (remote_state->pushremote_name); remote_state->remotes_alloc =0 ; remote_state->remotes_nr =0 ;As
remote_statehas two extra structures embedded in it, I wonder if we should be clearing them in this function, but possibly it is cleared elsewhere or perhaps in a later series?
Quoted 4 lines
It is not yet part of any subsequent patch series, mostly because I didn't happen to stumble over such leaks yet. Both of the rewrites very much are leaky though, and would be hit when we use "insteadOf" or "pushInsteadOf" configs.
Yes, I was wondering if our test coverage for the feature is lacking. In a sense, leaks from these insteadOf configuration variables are tiny and uninteresting, compared to how much it bothers me to find us not testing these often used features.
PATCH 16/20 remote: fix leaks when matching refspecs
In reply to Patrick SteinhardtIn match_explicit(), we try to match a source ref with a destination ref according to a refspec item. This matching sometimes requires us to allocate a new source spec so that it looks like we expect. And while we in some end up assigning this allocated ref as peer_ref, which hands over ownership of it to the caller, in other cases we don't. We neither free it though, causing a memory leak.
Fix the leak by creating a common exit path where we can easily free the source ref in case it is allocated and hasn't been handed over to the caller.
--- remote.c | 43 +++++++++++++++++++++++++++++-------------- t/t5505-remote.sh | 1 + 2 files changed, 30 insertions(+), 14 deletions(-)
remote.c +29 -14
@@ -1325,18 +1325,21 @@ static int match_explicit(struct ref *src, struct ref *dst,struct ref ***dst_tail, struct refspec_item *rs) { - struct ref *matched_src, *matched_dst; -int allocated_src; + struct ref *matched_src = NULL, *matched_dst = NULL; +int allocated_src =0 , ret;const char *dst_value = rs->dst;char *dst_guess; -if (rs->pattern || rs->matching || rs->negative) -return 0 ; +if (rs->pattern || rs->matching || rs->negative) { + ret =0 ; + goto out; + } - matched_src = matched_dst =NULL ; -if (match_explicit_lhs (src, rs, &matched_src, &allocated_src) <0 ) -return -1 ; +if (match_explicit_lhs (src, rs, &matched_src, &allocated_src) <0 ) { + ret =-1 ; + goto out; + }if (!dst_value) {int flag;
@@ -1375,18 +1378,30 @@ static int match_explicit(struct ref *src, struct ref *dst, dst_value);break ; } -if (!matched_dst) -return -1 ; -if (matched_dst->peer_ref) -return error (_ ("dst ref %s receives from more than one src" ), - matched_dst->name); -else { + +if (!matched_dst) { + ret =-1 ; + goto out; + } + +if (matched_dst->peer_ref) { + ret =error (_ ("dst ref %s receives from more than one src" ), + matched_dst->name); + goto out; + }else { matched_dst->peer_ref = allocated_src ? matched_src :copy_ref (matched_src); matched_dst->force = rs->force; + matched_src =NULL ; } -return 0 ; + + ret =0 ; + +out: +if (allocated_src) +free_one_ref (matched_src); +return ret; } static int match_explicit_refs(struct ref *src, struct ref *dst,
t/t5505-remote.sh +1 -0
@@ -2,6 +2,7 @@ test_description='git remote porcelain-ish' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh setup_repository () {
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH 17/20 remote: fix leaking peer ref when expanding refmap
In reply to Patrick SteinhardtWhen expanding remote refs via the refspec in get_expanded_map(), we first copy the remote ref and then override its peer ref with the expanded name. This may cause a memory leak though in case the peer ref is already set, as this field is being copied by copy_ref(), as well.
Fix the leak by freeing the peer ref before we re-assign the field.
--- remote.c | 2 ++ t/t5536-fetch-conflicts.sh | 1 + t/t5553-set-upstream.sh | 1 + t/t5703-upload-pack-ref-in-want.sh | 1 + t/t6050-replace.sh | 1 + 5 files changed, 6 insertions(+)
remote.c +2 -0
@@ -2062,6 +2062,8 @@ static struct ref *get_expanded_map(const struct ref *remote_refs, !ignore_symref_update(expn_name, &scratch)) {struct ref *cpy =copy_ref (ref); +if (cpy->peer_ref) +free_one_ref (cpy->peer_ref); cpy->peer_ref =alloc_ref (expn_name);if (refspec->force) cpy->peer_ref->force =1 ;
t/t5536-fetch-conflicts.sh +1 -0
@@ -2,6 +2,7 @@ test_description='fetch handles conflicting refspecs correctly' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh D=$(pwd)
t/t5553-set-upstream.sh +1 -0
@@ -4,6 +4,7 @@ test_description='"git fetch/pull --set-upstream" basic tests.' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh check_config () {
t/t5703-upload-pack-ref-in-want.sh +1 -0
@@ -2,6 +2,7 @@ test_description='upload-pack ref-in-want' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh get_actual_refs () {
t/t6050-replace.sh +1 -0
@@ -7,6 +7,7 @@ test_description='Tests replace refs functionality' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY/lib-gpg.sh"
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH 18/20 builtin/fetch: fix leaking transaction with `--atomic`
In reply to Patrick SteinhardtWith the --atomic flag, we use a single ref transaction to commit all ref updates in git-fetch(1). The lifetime of transactions is somewhat
weird: while ref_transaction_abort() will free the transaction, a call to ref_transaction_commit() won't. We thus have to manually free the transaction in the successful case.
Adapt the code to free the transaction in the exit path to plug the resulting memory leak. As ref_transaction_abort() already freed the transaction for us, we have to unset the transaction when we hit that code path to not cause a double free.
--- builtin/fetch.c | 8 ++++---- t/t5574-fetch-output.sh | 1 + 2 files changed, 5 insertions(+), 4 deletions(-)
builtin/fetch.c +4 -4
@@ -1731,11 +1731,8 @@ static int do_fetch(struct transport *transport, goto cleanup; retcode =ref_transaction_commit (transaction, &err); -if (retcode) { -ref_transaction_free (transaction); - transaction =NULL ; +if (retcode) goto cleanup; - } }commit_fetch_head (&fetch_head);
@@ -1803,8 +1800,11 @@ static int do_fetch(struct transport *transport,if (transaction &&ref_transaction_abort (transaction, &err) && err.len)error ("%s" , err.buf); + transaction =NULL ; } +if (transaction) +ref_transaction_free (transaction);display_state_release (&display_state);close_fetch_head (&fetch_head);strbuf_release (&err);
t/t5574-fetch-output.sh +1 -0
@@ -5,6 +5,7 @@ test_description='git fetch output format' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_expect_success 'fetch with invalid output format configuration' '
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH 19/20 transport: fix leaking arguments when fetching from bundle
In reply to Patrick SteinhardtIn fetch_refs_from_bundle() we assemble a vector of arguments to pass to unbundle(), but never free it. Fix this leak.
This memory leak gets hit in t5510, but fixing it isn't sufficient to make the whole test suite pass.
--- transport.c | 2 ++ 1 file changed, 2 insertions(+)
transport.c +2 -0
@@ -189,6 +189,8 @@ static int fetch_refs_from_bundle(struct transport *transport, &extra_index_pack_args, fetch_pack_fsck_objects() ? VERIFY_BUNDLE_FSCK : 0); transport->hash_algo = data->header.hash_algo; + +strvec_clear (&extra_index_pack_args);return ret; }
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
Re: [PATCH 19/20] transport: fix leaking arguments when fetching from bundle
Quoted 16 lines · Patrick Steinhardt
In
fetch_refs_from_bundle()we assemble a vector of arguments to pass tounbundle(), but never free it. Fix this leak.This memory leak gets hit in t5510, but fixing it isn't sufficient to make the whole test suite pass.
Signed-off-by Patrick Steinhardt <[email protected]>--- transport.c | 2 ++ 1 file changed, 2 insertions(+)transport.c +0
-0
transport.c static int fetch_refs_from_bundle(struct transport *transport, · line 195
Quoted 7 lines
transport.c +2
-0@@ -189,6 +189,8 @@ static int fetch_refs_from_bundle(struct transport *transport, &extra_index_pack_args, fetch_pack_fsck_objects() ? VERIFY_BUNDLE_FSCK : 0); transport->hash_algo = data->header.hash_algo; + +strvec_clear (&extra_index_pack_args);return ret; }
Hmph. unbundle() has this:
if (extra_index_pack_args) {
strvec_pushv(&ip.args, extra_index_pack_args->v);
strvec_clear(extra_index_pack_args);
}so while I think this patch would not hurt at all, do we need this patch?
The other caller of unbundle() that passes the extra_index_pack_args is cmd_bundle_unbundle() and it does not do anything after it is done.
Re: [PATCH 19/20] transport: fix leaking arguments when fetching from bundle
Quoted 28 lines · Junio C Hamano
Quoted 16 lines · Patrick Steinhardt
In
fetch_refs_from_bundle()we assemble a vector of arguments to pass tounbundle(), but never free it. Fix this leak.This memory leak gets hit in t5510, but fixing it isn't sufficient to make the whole test suite pass.
Signed-off-by Patrick Steinhardt <[email protected]>--- transport.c | 2 ++ 1 file changed, 2 insertions(+)transport.c +0
-0transport.c static int fetch_refs_from_bundle(struct transport *transport, · line 195
Quoted 7 lines
transport.c +2
-0@@ -189,6 +189,8 @@ static int fetch_refs_from_bundle(struct transport *transport, &extra_index_pack_args, fetch_pack_fsck_objects() ? VERIFY_BUNDLE_FSCK : 0); transport->hash_algo = data->header.hash_algo; + +strvec_clear (&extra_index_pack_args);return ret; }Hmph.
unbundle()has this:
Quoted 7 lines
if (extra_index_pack_args) { strvec_pushv(&ip.args, extra_index_pack_args->v); strvec_clear(extra_index_pack_args); }so while I think this patch would not hurt at all, do we need this patch?
Yes we do, because there is an early return in case verify_bundle() fails. I didn't notice that we have it in unbundle() though.
Quoted 3 lines
The other caller of
unbundle()that passes theextra_index_pack_argsiscmd_bundle_unbundle()and it does not do anything after it is done.
I'd argue it's bad practice to have unbundle() clear the caller provided args for us and somewhat surprising. While one way to fix this would be to have a common exit path where we always free them. But I'd much rather make it the responsibility of the caller itself to free them.
I'll adapt the code in this spirit.
Patrick
Re: [PATCH 19/20] transport: fix leaking arguments when fetching from bundle
Quoted 2 lines · Patrick Steinhardt
Yes we do, because there is an early return in case
verify_bundle()fails. I didn't notice that we have it inunbundle()though.
Ah, I missed that one.
Quoted 2 lines
I'd argue it's bad practice to have
unbundle()clear the caller provided args for us and somewhat surprising.
I was surprised so it makes it at least two of us. In some call patterns, having the callee _consume_ what the caller fed it may be natural, but not in this code path.
Thanks.
PATCH 20/20 transport: fix leaking negotiation tips
In reply to Patrick SteinhardtWe do not free negotiation tips in the transport's smart options. Fix this by freeing them on disconnect.
--- t/t5510-fetch.sh | 1 + transport.c | 4 ++++ 2 files changed, 5 insertions(+)
t/t5510-fetch.sh +1 -0
@@ -5,6 +5,7 @@ test_description='Per branch config variables affects "git fetch". ' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-bundle.sh
transport.c +4 -0
@@ -947,6 +947,10 @@ static int disconnect_git(struct transport *transport)finish_connect (data->conn); } +if (data->options.negotiation_tips) { +oid_array_clear (data->options.negotiation_tips); +free (data->options.negotiation_tips); + }list_objects_filter_release (&data->options.filter_options);oid_array_clear (&data->extra_have);oid_array_clear (&data->shallow);
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
v2 cover Memory leak fixes (pt.5)
In reply to Patrick SteinhardtHi,
this is version 2 of this patch series that fixes another set of memory leaks.
Changes compared to v1:
- Remove spaces between cast and variable.
- Clarify why we move the ctx.entries_nr block.
- Adapt sideband colors to use git_config_get_string_tmp() such that we do not have to allocate the strings in the first place.
- Fix more memory leaks for config values in the remote state.
- Refactor unbundle() to not free extra args passed by the caller anymore. Instead, this is now always done by the caler.
Thanks!
Patrick
Patrick Steinhardt (20):
mailinfo: fix leaking header data
convert: fix leaks when resetting attributes
pretty: fix memory leaks when parsing pretty formats
pretty: fix leaking key/value separator buffer builtin/merge-tree: fix leaking -X strategy options builtin/upload-archive: fix leaking args passed to write_archive() builtin/archive: fix leaking OPT_FILENAME() value
midx-write: fix leaking hashfile on error cases builtin/repack: fix leaks when computing packs to repack t/helper: fix leaking multi-pack-indices in "read-midx"
transport: fix leaking OID arrays in git:// transport data builtin/send-pack: fix leaking refspecs
sideband: fix leaks when configuring sideband colors builtin/fetch-pack: fix leaking refs
remote: fix leaking config strings
remote: fix leaks when matching refspecs
remote: fix leaking peer ref when expanding refmap builtin/fetch: fix leaking transaction with --atomic
transport: fix leaking arguments when fetching from bundle
transport: fix leaking negotiation tips
archive.c | 10 ++++ builtin/archive.c | 7 ++- builtin/bundle.c | 2 + builtin/fetch-pack.c | 20 ++++--- builtin/fetch.c | 8 +-- builtin/merge-tree.c | 13 ++++- builtin/repack.c | 36 +++++++++--- builtin/send-pack.c | 1 + builtin/upload-archive.c | 8 ++- bundle.c | 4 +- convert.c | 3 + mailinfo.c | 17 +++++- midx-write.c | 24 ++++---- pretty.c | 13 ++++- remote.c | 85 +++++++++++++++++++++++------ sideband.c | 15 +++-- t/helper/test-read-midx.c | 8 ++- t/t4150-am.sh | 1 + t/t4205-log-pretty-formats.sh | 2 + t/t4301-merge-tree-write-tree.sh | 1 + t/t5000-tar-tree.sh | 1 + t/t5003-archive-zip.sh | 1 + t/t5100-mailinfo.sh | 1 + t/t5319-multi-pack-index.sh | 2 + t/t5400-send-pack.sh | 1 + t/t5401-update-hooks.sh | 2 + t/t5408-send-pack-stdin.sh | 2 + t/t5409-colorize-remote-messages.sh | 1 + t/t5501-fetch-push-alternates.sh | 1 + t/t5505-remote.sh | 1 + t/t5510-fetch.sh | 1 + t/t5519-push-alternates.sh | 1 + t/t5536-fetch-conflicts.sh | 1 + t/t5548-push-porcelain.sh | 1 + t/t5553-set-upstream.sh | 1 + t/t5574-fetch-output.sh | 1 + t/t5703-upload-pack-ref-in-want.sh | 1 + t/t5812-proto-disable-http.sh | 2 + t/t6050-replace.sh | 1 + t/t7704-repack-cruft.sh | 1 + transport.c | 8 +++ 41 files changed, 237 insertions(+), 73 deletions(-)
Range-diff against v1:
1: 69e30ea5179 = 1: 69e30ea5179 mailinfo: fix leaking header data
2: ed0f01bf92c = 2: ed0f01bf92c convert: fix leaks when resetting attributes
3: 82f3908f962 ! 3: 8a1963685e7 pretty: fix memory leaks when parsing pretty formats
@@ pretty.c: static int git_pretty_formats_config(const char *var, const char *valu
commit_formats_len++;
}
-+ free((char *) commit_format->name);
++ free((char *)commit_format->name);
commit_format->name = xstrdup(name);
commit_format->format = CMIT_FMT_USERFORMAT;
if (git_config_string(&fmt, var, value))
return -1;
- if (skip_prefix(fmt, "format:", &commit_format->user_format)) {
-+ free((char *) commit_format->user_format);
++ free((char *)commit_format->user_format);
+ if (skip_prefix(fmt, "format:", &stripped)) {
commit_format->is_tformat = 0;
- } else if (skip_prefix(fmt, "tformat:", &commit_format->user_format)) {
4: 696467780e6 = 4: 1c368a4489a pretty: fix leaking key/value separator buffer
5: 53db2fc7206 = 5: c62c5e9604e builtin/merge-tree: fix leaking `-X` strategy options
6: 5b05a325218 = 6: afdd7f90b13 builtin/upload-archive: fix leaking args passed to `write_archive()`
7: bad575df126 = 7: 38487f3f65b builtin/archive: fix leaking `OPT_FILENAME()` value
8: 5f042ce5098 ! 8: 693c93ddbf7 midx-write: fix leaking hashfile on error cases @@ Commit message
hashfile and finalizing it anymore this is sufficient to fix the memory leak.
+ While at it, also move around the block checking for ctx.entries_nr.
+ This change is not required to fix the memory leak, but it feels natural
+ to move together all massaging of parameters before we go with them and
+ execute the actual logic.
+
## midx-write.c ##
9: 5c820da9761 = 9: 6bca72e5c57 builtin/repack: fix leaks when computing packs to repack
10: 9caf5eeea93 = 10: 48b60279d18 t/helper: fix leaking multi-pack-indices in "read-midx"
11: 8e12c55536d = 11: 0cb440ef648 transport: fix leaking OID arrays in git:// transport data
12: 5d8e0a3d8b4 = 12: f4300c9326b builtin/send-pack: fix leaking refspecs
13: 5d09959b642 ! 13: 28805c15a42 sideband: fix leaks when configuring sideband colors
@@ Commit message
We read a bunch of configs in `use_sideband_colors()` to configure the
colors that Git should use. We never free the strings read from the
- config though, causing memory leaks. Fix those.
+ config though, causing memory leaks.
+
+ Refactor the code to use `git_config_get_string_tmp()` instead, which
+ does not allocate memory. As we throw the strings away after parsing
+ them anyway there is no need to use allocated strings.
Signed-off-by: Patrick Steinhardt <[email protected]>
@@ sideband.c: static int use_sideband_colors(void)
const char *key = "color.remote";
struct strbuf sb = STRBUF_INIT;
- char *value;
-+ char *value = NULL;
++ const char *value;
int i;
if (use_sideband_colors_cached >= 0)
-@@ sideband.c: static int use_sideband_colors(void)
- } else {
+ return use_sideband_colors_cached;
+
+- if (!git_config_get_string(key, &value)) {
++ if (!git_config_get_string_tmp(key, &value))
+ use_sideband_colors_cached = git_config_colorbool(key, value);
+- } else if (!git_config_get_string("color.ui", &value)) {
++ else if (!git_config_get_string_tmp("color.ui", &value))
+ use_sideband_colors_cached = git_config_colorbool("color.ui", value);
+- } else {
++ else
use_sideband_colors_cached = GIT_COLOR_AUTO;
- }
-+ FREE_AND_NULL(value);
+- }
for (i = 0; i < ARRAY_SIZE(keywords); i++) {
strbuf_reset(&sb);
strbuf_addf(&sb, "%s.%s", key, keywords[i].keyword);
- if (git_config_get_string(sb.buf, &value))
- continue;
-- if (color_parse(value, keywords[i].color))
+- if (git_config_get_string(sb.buf, &value))
- continue;
+- if (color_parse(value, keywords[i].color))
++ if (git_config_get_string_tmp(sb.buf, &value))
+ continue;
+ color_parse(value, keywords[i].color);
-+ FREE_AND_NULL(value);
}
+
strbuf_release(&sb);
14: 1c94195488d = 14: aac84c5a2b7 builtin/fetch-pack: fix leaking refs
15: 97346d6f944 ! 15: 532328b7814 remote: fix leaking config strings
@@ Commit message
We're leaking several config strings when assembling remotes, either
because we do not free preceding values in case a config was set
multiple times, or because we do not free them when releasing the remote
- state. Plug those leaks.
+ state. This includes config strings for "branch" sections, "insteadOf",
+ "pushInsteadOf", and "pushDefault".
+
+ Plug those leaks.
Signed-off-by: Patrick Steinhardt <[email protected]>
## remote.c ##
+@@ remote.c: static struct branch *make_branch(struct remote_state *remote_state,
+ return ret;
+ }
+
++static void branch_release(struct branch *branch)
++{
++ free((char *)branch->name);
++ free((char *)branch->refname);
++ free(branch->remote_name);
++ free(branch->pushremote_name);
++ for (int i = 0; i < branch->merge_nr; i++)
++ refspec_item_clear(branch->merge[i]);
++ free(branch->merge);
++}
++
+ static struct rewrite *make_rewrite(struct rewrites *r,
+ const char *base, size_t len)
+ {
+@@ remote.c: static struct rewrite *make_rewrite(struct rewrites *r,
+ return ret;
+ }
+
++static void rewrites_release(struct rewrites *r)
++{
++ for (int i = 0; i < r->rewrite_nr; i++)
++ free((char *)r->rewrite[i]->base);
++ free(r->rewrite);
++ memset(r, 0, sizeof(*r));
++}
++
+ static void add_instead_of(struct rewrite *rewrite, const char *instead_of)
+ {
+ ALLOC_GROW(rewrite->instead_of, rewrite->instead_of_nr + 1, rewrite->instead_of_alloc);
@@ remote.c: static int handle_config(const char *key, const char *value,
return -1;
branch = make_branch(remote_state, name, namelen);
@@ remote.c: static int handle_config(const char *key, const char *value,
return git_config_string(&remote->foreign_vcs, key, value);
}
return 0;
-@@ remote.c: void remote_state_clear(struct remote_state *remote_state)
+@@ remote.c: struct remote_state *remote_state_new(void)
+
+ void remote_state_clear(struct remote_state *remote_state)
+ {
++ struct hashmap_iter iter;
++ struct branch *b;
+ int i;
+
for (i = 0; i < remote_state->remotes_nr; i++)
remote_clear(remote_state->remotes[i]);
FREE_AND_NULL(remote_state->remotes);
@@ remote.c: void remote_state_clear(struct remote_state *remote_state)
remote_state->remotes_alloc = 0;
remote_state->remotes_nr = 0;
++ rewrites_release(&remote_state->rewrites);
++ rewrites_release(&remote_state->rewrites_push);
++
+ hashmap_clear_and_free(&remote_state->remotes_hash, struct remote, ent);
+- hashmap_clear_and_free(&remote_state->branches_hash, struct remote, ent);
++ hashmap_for_each_entry(&remote_state->branches_hash, &iter, b, ent) {
++ branch_release(b);
++ free(b);
++ }
++ hashmap_clear(&remote_state->branches_hash);
+ }
+
+ /*
16: e1d0be37636 = 16: 440b3d99372 remote: fix leaks when matching refspecs
17: 773fe580d75 = 17: 662ec4e6484 remote: fix leaking peer ref when expanding refmap
18: 9ede792550e = 18: dbd8eaa2cb1 builtin/fetch: fix leaking transaction with `--atomic`
19: b72f7d1ee39 ! 19: 4c5740afe43 transport: fix leaking arguments when fetching from bundle @@ Commit message
transport: fix leaking arguments when fetching from bundle
In fetch_refs_from_bundle() we assemble a vector of arguments to pass
- to unbundle(), but never free it. Fix this leak.
+ to unbundle(), but never free it. And in theory we wouldn't have to
+ because unbundle() already knows to free the vector for us. But it
+ fails to do so when it exits early due to verify_bundle() failing.
+
+ The calling convention that the arguments are freed by the callee and
+ not the caller feels somewhat weird. Refactor the code such that it is
+ instead the responsibility of the caller to free the vector, adapting
+ the only two callsites where we pass extra arguments. This also fixes
+ the memory leak.
This memory leak gets hit in t5510, but fixing it isn't sufficient to make the whole test suite pass.
+ ## builtin/bundle.c ##
+@@ builtin/bundle.c: static int cmd_bundle_unbundle(int argc, const char **argv, const char *prefix)
+ &extra_index_pack_args, 0) ||
+ list_bundle_refs(&header, argc, argv);
+ bundle_header_release(&header);
++
+ cleanup:
++ strvec_clear(&extra_index_pack_args);
+ free(bundle_file);
+ return ret;
+ }
+
+ ## bundle.c ##
+@@ bundle.c: int unbundle(struct repository *r, struct bundle_header *header,
+ if (flags & VERIFY_BUNDLE_FSCK)
+ strvec_push(&ip.args, "--fsck-objects");
+
+- if (extra_index_pack_args) {
++ if (extra_index_pack_args)
+ strvec_pushv(&ip.args, extra_index_pack_args->v);
+- strvec_clear(extra_index_pack_args);
+- }
+
+ ip.in = bundle_fd;
+ ip.no_stdout = 1;
+
## transport.c ##
@@ transport.c: static int fetch_refs_from_bundle(struct transport *transport,
&extra_index_pack_args,
20: e8f479deeb2 = 20: c3d2b035761 transport: fix leaking negotiation tips
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 1/20 mailinfo: fix leaking header data
We populate the mailinfo arrays p_hdr_data and s_hdr_data with data parsed from the mail headers. These arrays may end up being only partially populated with gaps in case some of the headers do not parse properly. This causes memory leaks because strbuf_list_free() will stop iterating once it hits the first NULL pointer in the backing array.
Fix this by open-coding a variant of strbuf_list_free() that knows to iterate through all headers.
--- mailinfo.c | 17 +++++++++++++++-- t/t5100-mailinfo.sh | 1 + 2 files changed, 16 insertions(+), 2 deletions(-)
mailinfo.c +15 -2
@@ -1290,8 +1290,21 @@ void clear_mailinfo(struct mailinfo *mi)strbuf_release (&mi->inbody_header_accum);free (mi->message_id); -strbuf_list_free (mi->p_hdr_data); -strbuf_list_free (mi->s_hdr_data); +for (size_t i =0 ; header[i]; i++) { +if (!mi->p_hdr_data[i]) +continue ; +strbuf_release (mi->p_hdr_data[i]); +free (mi->p_hdr_data[i]); + } +free (mi->p_hdr_data); + +for (size_t i =0 ; header[i]; i++) { +if (!mi->s_hdr_data[i]) +continue ; +strbuf_release (mi->s_hdr_data[i]); +free (mi->s_hdr_data[i]); + } +free (mi->s_hdr_data);while (mi->content < mi->content_top) {free (*(mi->content_top));
t/t5100-mailinfo.sh +1 -0
@@ -5,6 +5,7 @@ test_description='git mailinfo and git mailsplit test' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh DATA="$TEST_DIRECTORY/t5100"
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 2/20 convert: fix leaks when resetting attributes
In reply to Patrick SteinhardtWhen resetting parsed gitattributes, we free the list of convert drivers parsed from the config. We only free some of the drivers' fields though and thus have memory leaks.
Fix this by freeing all allocated convert driver fields to plug these memory leaks.
--- convert.c | 3 +++ t/t4150-am.sh | 1 + 2 files changed, 4 insertions(+)
convert.c +3 -0
@@ -1371,6 +1371,9 @@ void reset_parsed_attributes(void)for (drv = user_convert; drv; drv = next) { next = drv->next;free ((void *)drv->name); +free ((void *)drv->smudge); +free ((void *)drv->clean); +free ((void *)drv->process);free (drv); } user_convert =NULL ;
t/t4150-am.sh +1 -0
@@ -5,6 +5,7 @@ test_description='git am running' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_expect_success 'setup: messages' '
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 3/20 pretty: fix memory leaks when parsing pretty formats
In reply to Patrick SteinhardtWhen parsing pretty formats from the config we leak the name and user format whenever these are set multiple times. This is because we do not free any already-set value in case there is one.
Plugging this leak for the name is trivial. For the user format we need to be a bit more careful, because we may end up assigning a pointer into the allocated region when the string is prefixed with either "format" or "tformat:". In order to make it safe to unconditionally free the user format we thus strdup the stripped string into the field instead of a pointer into the string.
--- pretty.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)
pretty.c +9 -3
@@ -63,7 +63,7 @@ static int git_pretty_formats_config(const char *var, const char *value, void *cb UNUSED) {struct cmt_fmt_map *commit_format =NULL ; -const char *name; +const char *name, *stripped;char *fmt;int i;
@@ -90,15 +90,21 @@ static int git_pretty_formats_config(const char *var, const char *value, commit_formats_len++; } +free ((char *)commit_format->name); commit_format->name =xstrdup (name); commit_format->format =CMIT_FMT_USERFORMAT ;if (git_config_string (&fmt, var, value))return -1 ; -if (skip_prefix (fmt,"format:" , &commit_format->user_format)) { +free ((char *)commit_format->user_format); +if (skip_prefix (fmt,"format:" , &stripped)) { commit_format->is_tformat =0 ; - }else if (skip_prefix (fmt,"tformat:" , &commit_format->user_format)) { + commit_format->user_format =xstrdup (stripped); +free (fmt); + }else if (skip_prefix (fmt,"tformat:" , &stripped)) { commit_format->is_tformat =1 ; + commit_format->user_format =xstrdup (stripped); +free (fmt); }else if (strchr (fmt,'%' )) { commit_format->is_tformat =1 ; commit_format->user_format = fmt;
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 4/20 pretty: fix leaking key/value separator buffer
In reply to Patrick SteinhardtThe format_set_trailers_options() function is responsible for parsing a custom pretty format for trailers. It puts the parsed options into a struct process_trailer_options structure, while the allocated memory required for this will be put into separate caller-provided arguments. It is thus the caller's responsibility to free the memory not via the options structure, but via the other parameters.
While we do this alright for the separator and filter keys, we do not free the memory associated with the key/value separator. Fix this to plug this memory leak.
--- pretty.c | 1 + t/t4205-log-pretty-formats.sh | 2 ++ 2 files changed, 3 insertions(+)
pretty.c +1 -0
@@ -1776,6 +1776,7 @@ static size_t format_commit_one(struct strbuf *sb, /* in UTF-8 */ } trailer_out:string_list_clear (&filter_list,0 ); +strbuf_release (&kvsepbuf);strbuf_release (&sepbuf);return ret; }
t/t4205-log-pretty-formats.sh +2 -0
@@ -5,6 +5,8 @@ # test_description='Test pretty formats' + +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Tested non-UTF-8 encoding
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 5/20 builtin/merge-tree: fix leaking `-X` strategy options
In reply to Patrick SteinhardtThe -X switch for git-merge-tree(1) will push each option into a local xopts vector that we then end up parsing. The vector never gets freed though, causing a memory leak. Plug it.
--- builtin/merge-tree.c | 13 ++++++++++--- t/t4301-merge-tree-write-tree.sh | 1 + 2 files changed, 11 insertions(+), 3 deletions(-)
builtin/merge-tree.c +10 -3
@@ -533,6 +533,7 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)int expected_remaining_argc;int original_argc;const char *merge_base =NULL ; +int ret;const char *const merge_tree_usage[] = {N_ ("git merge-tree [--write-tree] [<options>] <branch1> <branch2>" ),
@@ -625,7 +626,9 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)strbuf_list_free (split); }strbuf_release (&buf); -return 0 ; + + ret =0 ; + goto out; }/* Figure out which mode to use */
@@ -664,7 +667,11 @@ int cmd_merge_tree(int argc, const char **argv, const char *prefix)/* Do the relevant type of merge */ if (o.mode ==MODE_REAL ) -return real_merge (&o, merge_base, argv[0 ], argv[1 ], prefix); + ret =real_merge (&o, merge_base, argv[0 ], argv[1 ], prefix);else -return trivial_merge (argv[0 ], argv[1 ], argv[2 ]); + ret =trivial_merge (argv[0 ], argv[1 ], argv[2 ]); + +out: +strvec_clear (&xopts); +return ret; }
t/t4301-merge-tree-write-tree.sh +1 -0
@@ -2,6 +2,7 @@ test_description='git merge-tree --write-tree' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # This test is ort-specific
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 6/20 builtin/upload-archive: fix leaking args passed to `write_archive()`
In reply to Patrick SteinhardtIn git-upload-archive(1), we pass an array of arguments to write_archive() to tell it what exactly to do. We don't ever clear the vector though, causing a memory leak. Furthermore though, the call to write_archive() may cause contents of the array to be modified, which would cause us to leak memory to allocated strings held by it.
Fix the issue by having write_archive() create a shallow copy of argv before parsing the arguments. Like this, we won't modify the caller's array and can easily strvec_clear() it to plug these memory leaks.
--- archive.c | 10 ++++++++++ builtin/upload-archive.c | 8 ++++++-- 2 files changed, 16 insertions(+), 2 deletions(-)
archive.c +10 -0
@@ -736,6 +736,7 @@ int write_archive(int argc, const char **argv, const char *prefix,struct pretty_print_describe_status describe_status = {0 };struct pretty_print_context ctx = {0 };struct archiver_args args; +const char **argv_copy;int rc;git_config_get_bool ("uploadarchive.allowunreachable" , &remote_allow_unreachable);
@@ -749,6 +750,14 @@ int write_archive(int argc, const char **argv, const char *prefix, args.repo = repo; args.prefix = prefix;string_list_init_dup (&args.extra_files); + +/* +* `parse_archive_args()` modifies contents of `argv`, which is what we +* want. Our callers may not want it though, so we create a copy here. +*/ +DUP_ARRAY (argv_copy, argv, argc); + argv = argv_copy; + argc =parse_archive_args (argc, argv, &ar, &args, name_hint, remote);if (!startup_info->have_repository) {/*
@@ -767,6 +776,7 @@ int write_archive(int argc, const char **argv, const char *prefix,string_list_clear_func (&args.extra_files, extra_file_info_clear);free (args.refname);clear_pathspec (&args.pathspec); +free (argv_copy);return rc; }
builtin/upload-archive.c +6 -2
@@ -22,6 +22,7 @@ int cmd_upload_archive_writer(int argc, const char **argv, const char *prefix) {struct strvec sent_argv =STRVEC_INIT ;const char *arg_cmd ="argument " ; +int ret;if (argc !=2 || !strcmp (argv[1 ],"-h" ))usage (upload_archive_usage);
@@ -46,8 +47,11 @@ int cmd_upload_archive_writer(int argc, const char **argv, const char *prefix) }/* parse all options sent by the client */ -return write_archive (sent_argv.nr, sent_argv.v, prefix, - the_repository,NULL ,1 ); + ret =write_archive (sent_argv.nr, sent_argv.v, prefix, + the_repository,NULL ,1 ); + +strvec_clear (&sent_argv); +return ret; } __attribute__((format (printf, 1, 2)))
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 7/20 builtin/archive: fix leaking `OPT_FILENAME()` value
In reply to Patrick SteinhardtThe "--output" switch is an OPT_FILENAME() option, which allocates memory when specified by the user. But while we free the string when executed without the "--remote" switch, we don't otherwise because we return via a separate exit path that doesn't know to free it.
Fix this by creating a common exit path.
--- builtin/archive.c | 7 +++++-- t/t5000-tar-tree.sh | 1 + t/t5003-archive-zip.sh | 1 + 3 files changed, 7 insertions(+), 2 deletions(-)
builtin/archive.c +5 -2
@@ -100,13 +100,16 @@ int cmd_archive(int argc, const char **argv, const char *prefix)if (output)create_output_file (output); -if (remote) -return run_remote_archiver (argc, argv, remote, exec, output); +if (remote) { + ret =run_remote_archiver (argc, argv, remote, exec, output); + goto out; + }setvbuf (stderr,NULL , _IOLBF,BUFSIZ ); ret =write_archive (argc, argv, prefix, the_repository, output,0 ); +out:free (output);return ret; }
t/t5000-tar-tree.sh +1 -0
@@ -25,6 +25,7 @@ commit id embedding: ' TEST_CREATE_REPO_NO_TEMPLATE=1 +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh SUBSTFORMAT=%H%n
t/t5003-archive-zip.sh +1 -0
@@ -3,6 +3,7 @@ test_description='git archive --format=zip test' TEST_CREATE_REPO_NO_TEMPLATE=1 +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh SUBSTFORMAT=%H%n
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 8/20 midx-write: fix leaking hashfile on error cases
In reply to Patrick SteinhardtWhen writing the MIDX file we first create the struct hashfile used to write the trailer hash, and then afterwards we verify whether we can actually write the MIDX in the first place. When we decide that we can't, this leads to a memory leak because we never free the hash file contents.
We could fix this by freeing the hashfile on the exit path. There is a better option though: we can simply move the checks for the error condition earlier. As there is no early exit between creating the hashfile and finalizing it anymore this is sufficient to fix the memory leak.
While at it, also move around the block checking for ctx.entries_nr. This change is not required to fix the memory leak, but it feels natural to move together all massaging of parameters before we go with them and execute the actual logic.
--- midx-write.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-)
midx-write.c +12 -12
@@ -1308,6 +1308,18 @@ static int write_midx_internal(const char *object_dir, pack_name_concat_len +=MIDX_CHUNK_ALIGNMENT - (pack_name_concat_len %MIDX_CHUNK_ALIGNMENT ); +if (ctx.nr - dropped_packs ==0 ) { +error (_ ("no pack files to index." )); + result =1 ; + goto cleanup; + } + +if (!ctx.entries_nr) { +if (flags &MIDX_WRITE_BITMAP ) +warning (_ ("refusing to write multi-pack .bitmap without any objects" )); + flags &= ~(MIDX_WRITE_REV_INDEX |MIDX_WRITE_BITMAP ); + } +if (ctx.incremental) {struct strbuf lock_name =STRBUF_INIT ;
@@ -1333,18 +1345,6 @@ static int write_midx_internal(const char *object_dir, f =hashfd (get_lock_file_fd (&lk),get_lock_file_path (&lk)); } -if (ctx.nr - dropped_packs ==0 ) { -error (_ ("no pack files to index." )); - result =1 ; - goto cleanup; - } - -if (!ctx.entries_nr) { -if (flags &MIDX_WRITE_BITMAP ) -warning (_ ("refusing to write multi-pack .bitmap without any objects" )); - flags &= ~(MIDX_WRITE_REV_INDEX |MIDX_WRITE_BITMAP ); - } - cf =init_chunkfile (f); add_chunk(cf, MIDX_CHUNKID_PACKNAMES, pack_name_concat_len,
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 9/20 builtin/repack: fix leaks when computing packs to repack
In reply to Patrick SteinhardtWhen writing an MIDX in git-repack(1) we first collect all the pack names that we want to add to it in a string list. This list is marked as NODUP, which indicates that it will neither duplicate nor own strings added to it. In write_midx_included_packs() we then insert() strings via xstrdup() or strbuf_detach(), but the resulting strings will not be owned by anything and thus leak.
Fix this issue by marking the list as DUP and using a local buffer to compute the pack names.
This leak is hit in t5319, but plugging it is not sufficient to make the whole test suite pass.
--- builtin/repack.c | 36 +++++++++++++++++++++++++++--------- 1 file changed, 27 insertions(+), 9 deletions(-)
builtin/repack.c +27 -9
@@ -732,14 +732,23 @@ static void midx_included_packs(struct string_list *include, struct pack_geometry *geometry) {struct string_list_item *item; +struct strbuf buf =STRBUF_INIT ; + +for_each_string_list_item (item, &existing->kept_packs) { +strbuf_reset (&buf); +strbuf_addf (&buf,"%s.idx" , item->string); +string_list_insert (include, buf.buf); + } + +for_each_string_list_item (item, names) { +strbuf_reset (&buf); +strbuf_addf (&buf,"pack-%s.idx" , item->string); +string_list_insert (include, buf.buf); + } -for_each_string_list_item (item, &existing->kept_packs) -string_list_insert (include,xstrfmt ("%s.idx" , item->string)); -for_each_string_list_item (item, names) -string_list_insert (include,xstrfmt ("pack-%s.idx" , item->string));if (geometry->split_factor) { -struct strbuf buf =STRBUF_INIT ;uint32_t i; +for (i = geometry->split; i < geometry->pack_nr; i++) {struct packed_git *p = geometry->pack[i];
@@ -754,17 +763,21 @@ static void midx_included_packs(struct string_list *include,if (!p->pack_local)continue ; +strbuf_reset (&buf);strbuf_addstr (&buf,pack_basename (p));strbuf_strip_suffix (&buf,".pack" );strbuf_addstr (&buf,".idx" ); -string_list_insert (include,strbuf_detach (&buf,NULL )); +string_list_insert (include, buf.buf); } }else {for_each_string_list_item (item, &existing->non_kept_packs) {if (pack_is_marked_for_deletion (item))continue ; -string_list_insert (include,xstrfmt ("%s.idx" , item->string)); + +strbuf_reset (&buf); +strbuf_addf (&buf,"%s.idx" , item->string); +string_list_insert (include, buf.buf); } }
@@ -784,8 +797,13 @@ static void midx_included_packs(struct string_list *include,*/ if (pack_is_marked_for_deletion (item))continue ; -string_list_insert (include,xstrfmt ("%s.idx" , item->string)); + +strbuf_reset (&buf); +strbuf_addf (&buf,"%s.idx" , item->string); +string_list_insert (include, buf.buf); } + +strbuf_release (&buf); } static int write_midx_included_packs(struct string_list *include,
@@ -1476,7 +1494,7 @@ int cmd_repack(int argc, const char **argv, const char *prefix)mark_packs_for_deletion (&existing, &names);if (write_midx) { -struct string_list include =STRING_LIST_INIT_NODUP ; +struct string_list include =STRING_LIST_INIT_DUP ;midx_included_packs (&include, &existing, &names, &geometry); ret = write_midx_included_packs(&include, &geometry, &names,
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 10/20 t/helper: fix leaking multi-pack-indices in "read-midx"
In reply to Patrick SteinhardtSeveral of the subcommands of test-helper read-midx do not close the MIDX that they have opened, leading to memory leaks. Fix those.
--- t/helper/test-read-midx.c | 8 +++++++- t/t5319-multi-pack-index.sh | 2 ++ t/t7704-repack-cruft.sh | 1 + 3 files changed, 10 insertions(+), 1 deletion(-)
t/helper/test-read-midx.c +7 -1
@@ -86,6 +86,8 @@ static int read_midx_checksum(const char *object_dir)if (!m)return 1 ;printf ("%s\n" ,hash_to_hex (get_midx_checksum (m))); + +close_midx (m);return 0 ; }
@@ -102,10 +104,12 @@ static int read_midx_preferred_pack(const char *object_dir)if (midx_preferred_pack (midx, &preferred_pack) <0 ) {warning (_ ("could not determine MIDX preferred pack" )); +close_midx (midx);return 1 ; }printf ("%s\n" , midx->pack_names[preferred_pack]); +close_midx (midx);return 0 ; }
@@ -122,8 +126,10 @@ static int read_midx_bitmapped_packs(const char *object_dir)return 1 ;for (i =0 ; i < midx->num_packs + midx->num_packs_in_base; i++) { -if (nth_bitmapped_pack (the_repository, midx, &pack, i) <0 ) +if (nth_bitmapped_pack (the_repository, midx, &pack, i) <0 ) { +close_midx (midx);return 1 ; + }printf ("%s\n" ,pack_basename (pack.p));printf (" bitmap_pos: %" PRIuMAX"\n" , (uintmax_t )pack.bitmap_pos);
t/t5319-multi-pack-index.sh +2 -0
@@ -1,6 +1,8 @@ #!/bin/sh test_description='multi-pack-indexes' + +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-chunk.sh . "$TEST_DIRECTORY"/lib-midx.sh
t/t7704-repack-cruft.sh +1 -0
@@ -2,6 +2,7 @@ test_description='git repack works correctly' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh objdir=.git/objects
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 11/20 transport: fix leaking OID arrays in git:// transport data
In reply to Patrick SteinhardtThe transport data for the "git://" protocol contains two OID arrays that we never free, creating a memory leak. Plug them.
--- t/t5501-fetch-push-alternates.sh | 1 + t/t5519-push-alternates.sh | 1 + transport.c | 2 ++ 3 files changed, 4 insertions(+)
t/t5501-fetch-push-alternates.sh +1 -0
@@ -4,6 +4,7 @@ test_description='fetch/push involving alternates' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh count_objects () {
t/t5519-push-alternates.sh +1 -0
@@ -5,6 +5,7 @@ test_description='push to a repository that borrows from elsewhere' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_expect_success setup '
transport.c +2 -0
@@ -946,6 +946,8 @@ static int disconnect_git(struct transport *transport) }list_objects_filter_release (&data->options.filter_options); +oid_array_clear (&data->extra_have); +oid_array_clear (&data->shallow);free (data);return 0 ; }
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 12/20 builtin/send-pack: fix leaking refspecs
In reply to Patrick SteinhardtWe never free data associated with the assembled refspec in git-send-pack(1), causing a memory leak. Fix this.
--- builtin/send-pack.c | 1 + t/t5400-send-pack.sh | 1 + t/t5401-update-hooks.sh | 2 ++ t/t5408-send-pack-stdin.sh | 2 ++ t/t5548-push-porcelain.sh | 1 + t/t5812-proto-disable-http.sh | 2 ++ 6 files changed, 9 insertions(+)
builtin/send-pack.c +1 -0
@@ -338,5 +338,6 @@ int cmd_send_pack(int argc, const char **argv, const char *prefix)free_refs (remote_refs);free_refs (local_refs); +refspec_clear (&rs);return ret; }
t/t5400-send-pack.sh +1 -0
@@ -9,6 +9,7 @@ test_description='See why rewinding head breaks send-pack GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh cnt=64
t/t5401-update-hooks.sh +2 -0
@@ -4,6 +4,8 @@ # test_description='Test the update hook infrastructure.' + +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_expect_success setup '
t/t5408-send-pack-stdin.sh +2 -0
@@ -1,6 +1,8 @@ #!/bin/sh test_description='send-pack --stdin tests' + +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh create_ref () {
t/t5548-push-porcelain.sh +1 -0
@@ -4,6 +4,7 @@ # test_description='Test git push porcelain output' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh # Create commits in <repo> and assign each commit's oid to shell variables
t/t5812-proto-disable-http.sh +2 -0
@@ -1,6 +1,8 @@ #!/bin/sh test_description='test disabling of git-over-http in clone/fetch' + +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY/lib-proto-disable.sh" . "$TEST_DIRECTORY/lib-httpd.sh"
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 13/20 sideband: fix leaks when configuring sideband colors
In reply to Patrick SteinhardtWe read a bunch of configs in use_sideband_colors() to configure the colors that Git should use. We never free the strings read from the config though, causing memory leaks.
Refactor the code to use git_config_get_string_tmp() instead, which does not allocate memory. As we throw the strings away after parsing them anyway there is no need to use allocated strings.
--- sideband.c | 15 +++++++-------- t/t5409-colorize-remote-messages.sh | 1 + 2 files changed, 8 insertions(+), 8 deletions(-)
sideband.c +7 -8
@@ -30,28 +30,27 @@ static int use_sideband_colors(void)const char *key ="color.remote" ;struct strbuf sb =STRBUF_INIT ; -char *value; +const char *value;int i;if (use_sideband_colors_cached >=0 )return use_sideband_colors_cached; -if (!git_config_get_string (key, &value)) { +if (!git_config_get_string_tmp (key, &value)) use_sideband_colors_cached =git_config_colorbool (key, value); - }else if (!git_config_get_string ("color.ui" , &value)) { +else if (!git_config_get_string_tmp ("color.ui" , &value)) use_sideband_colors_cached =git_config_colorbool ("color.ui" , value); - }else { +else use_sideband_colors_cached =GIT_COLOR_AUTO ; - }for (i =0 ; i <ARRAY_SIZE (keywords); i++) {strbuf_reset (&sb);strbuf_addf (&sb,"%s.%s" , key, keywords[i].keyword); -if (git_config_get_string (sb.buf, &value)) -continue ; -if (color_parse (value, keywords[i].color)) +if (git_config_get_string_tmp (sb.buf, &value))continue ; +color_parse (value, keywords[i].color); } +strbuf_release (&sb);return use_sideband_colors_cached; }
t/t5409-colorize-remote-messages.sh +1 -0
@@ -2,6 +2,7 @@ test_description='remote messages are colorized on the client' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_expect_success 'setup' '
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 14/20 builtin/fetch-pack: fix leaking refs
In reply to Patrick SteinhardtWe build several ref lists in git-fetch-pack(1), but never free them. Fix those leaks.
--- builtin/fetch-pack.c | 20 ++++++++++++-------- 1 file changed, 12 insertions(+), 8 deletions(-)
builtin/fetch-pack.c +12 -8
@@ -46,7 +46,7 @@ static void add_sought_entry(struct ref ***sought, int *nr, int *alloc, int cmd_fetch_pack(int argc, const char **argv, const char *prefix UNUSED) {int i, ret; -struct ref *ref =NULL ; +struct ref *fetched_refs =NULL , *remote_refs =NULL ;const char *dest =NULL ;struct ref **sought =NULL ;int nr_sought =0 , alloc_sought =0 ;
@@ -228,19 +228,20 @@ int cmd_fetch_pack(int argc, const char **argv, const char *prefix UNUSED) version =discover_version (&reader);switch (version) {case protocol_v2: -get_remote_refs (fd[1 ], &reader, &ref,0 ,NULL ,NULL , +get_remote_refs (fd[1 ], &reader, &remote_refs,0 ,NULL ,NULL , args.stateless_rpc);break ;case protocol_v1:case protocol_v0: -get_remote_heads (&reader, &ref,0 ,NULL , &shallow); +get_remote_heads (&reader, &remote_refs,0 ,NULL , &shallow);break ;case protocol_unknown_version:BUG ("unknown protocol version" ); } - ref =fetch_pack (&args, fd, ref, sought, nr_sought, + fetched_refs =fetch_pack (&args, fd, remote_refs, sought, nr_sought, &shallow, pack_lockfiles_ptr, version); +if (pack_lockfiles.nr) {int i;
@@ -260,7 +261,7 @@ int cmd_fetch_pack(int argc, const char **argv, const char *prefix UNUSED)if (finish_connect (conn))return 1 ; - ret = !ref; + ret = !fetched_refs;/* * If the heads to pull were given, we should have consumed
@@ -270,11 +271,14 @@ int cmd_fetch_pack(int argc, const char **argv, const char *prefix UNUSED)*/ ret |=report_unmatched_refs (sought, nr_sought); -while (ref) { +for (struct ref *ref = fetched_refs; ref; ref = ref->next)printf ("%s %s\n" ,oid_to_hex (&ref->old_oid), ref->name); - ref = ref->next; - } +for (size_t i =0 ; i < nr_sought; i++) +free_one_ref (sought[i]); +free (sought); +free_refs (fetched_refs); +free_refs (remote_refs);return ret; }
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 15/20 remote: fix leaking config strings
In reply to Patrick SteinhardtWe're leaking several config strings when assembling remotes, either because we do not free preceding values in case a config was set multiple times, or because we do not free them when releasing the remote state. This includes config strings for "branch" sections, "insteadOf", "pushInsteadOf", and "pushDefault".
Plug those leaks.
--- remote.c | 40 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 38 insertions(+), 2 deletions(-)
remote.c +38 -2
@@ -243,6 +243,17 @@ static struct branch *make_branch(struct remote_state *remote_state,return ret; } +static void branch_release (struct branch *branch) +{ +free ((char *)branch->name); +free ((char *)branch->refname); +free (branch->remote_name); +free (branch->pushremote_name); +for (int i =0 ; i < branch->merge_nr; i++) +refspec_item_clear (branch->merge[i]); +free (branch->merge); +} +static struct rewrite *make_rewrite (struct rewrites *r,const char *base,size_t len) {
@@ -263,6 +274,14 @@ static struct rewrite *make_rewrite(struct rewrites *r,return ret; } +static void rewrites_release (struct rewrites *r) +{ +for (int i =0 ; i < r->rewrite_nr; i++) +free ((char *)r->rewrite[i]->base); +free (r->rewrite); +memset (r,0 ,sizeof (*r)); +} +static void add_instead_of (struct rewrite *rewrite,const char *instead_of) {ALLOC_GROW (rewrite->instead_of, rewrite->instead_of_nr +1 , rewrite->instead_of_alloc);
@@ -373,8 +392,10 @@ static int handle_config(const char *key, const char *value,return -1 ; branch =make_branch (remote_state, name, namelen);if (!strcmp (subkey,"remote" )) { +FREE_AND_NULL (branch->remote_name);return git_config_string (&branch->remote_name, key, value); }else if (!strcmp (subkey,"pushremote" )) { +FREE_AND_NULL (branch->pushremote_name);return git_config_string (&branch->pushremote_name, key, value); }else if (!strcmp (subkey,"merge" )) {if (!value)
@@ -406,9 +427,11 @@ static int handle_config(const char *key, const char *value,return 0 ;/* Handle remote.* variables */ -if (!name && !strcmp (subkey,"pushdefault" )) +if (!name && !strcmp (subkey,"pushdefault" )) { +FREE_AND_NULL (remote_state->pushremote_name);return git_config_string (&remote_state->pushremote_name, key, value); + }if (!name)return 0 ;
@@ -475,12 +498,15 @@ static int handle_config(const char *key, const char *value,else if (!strcmp (value,"--tags" )) remote->fetch_tags =2 ; }else if (!strcmp (subkey,"proxy" )) { +FREE_AND_NULL (remote->http_proxy);return git_config_string (&remote->http_proxy, key, value); }else if (!strcmp (subkey,"proxyauthmethod" )) { +FREE_AND_NULL (remote->http_proxy_authmethod);return git_config_string (&remote->http_proxy_authmethod, key, value); }else if (!strcmp (subkey,"vcs" )) { +FREE_AND_NULL (remote->foreign_vcs);return git_config_string (&remote->foreign_vcs, key, value); }return 0 ;
@@ -2797,16 +2823,26 @@ struct remote_state *remote_state_new(void)void remote_state_clear (struct remote_state *remote_state) { +struct hashmap_iter iter; +struct branch *b;int i;for (i =0 ; i < remote_state->remotes_nr; i++)remote_clear (remote_state->remotes[i]);FREE_AND_NULL (remote_state->remotes); +FREE_AND_NULL (remote_state->pushremote_name); remote_state->remotes_alloc =0 ; remote_state->remotes_nr =0 ; +rewrites_release (&remote_state->rewrites); +rewrites_release (&remote_state->rewrites_push); + hashmap_clear_and_free(&remote_state->remotes_hash, struct remote, ent); - hashmap_clear_and_free(&remote_state->branches_hash, struct remote, ent); +hashmap_for_each_entry (&remote_state->branches_hash, &iter, b, ent) { +branch_release (b); +free (b); + } +hashmap_clear (&remote_state->branches_hash); }/*
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 16/20 remote: fix leaks when matching refspecs
In reply to Patrick SteinhardtIn match_explicit(), we try to match a source ref with a destination ref according to a refspec item. This matching sometimes requires us to allocate a new source spec so that it looks like we expect. And while we in some end up assigning this allocated ref as peer_ref, which hands over ownership of it to the caller, in other cases we don't. We neither free it though, causing a memory leak.
Fix the leak by creating a common exit path where we can easily free the source ref in case it is allocated and hasn't been handed over to the caller.
--- remote.c | 43 +++++++++++++++++++++++++++++-------------- t/t5505-remote.sh | 1 + 2 files changed, 30 insertions(+), 14 deletions(-)
remote.c +29 -14
@@ -1344,18 +1344,21 @@ static int match_explicit(struct ref *src, struct ref *dst,struct ref ***dst_tail, struct refspec_item *rs) { - struct ref *matched_src, *matched_dst; -int allocated_src; + struct ref *matched_src = NULL, *matched_dst = NULL; +int allocated_src =0 , ret;const char *dst_value = rs->dst;char *dst_guess; -if (rs->pattern || rs->matching || rs->negative) -return 0 ; +if (rs->pattern || rs->matching || rs->negative) { + ret =0 ; + goto out; + } - matched_src = matched_dst =NULL ; -if (match_explicit_lhs (src, rs, &matched_src, &allocated_src) <0 ) -return -1 ; +if (match_explicit_lhs (src, rs, &matched_src, &allocated_src) <0 ) { + ret =-1 ; + goto out; + }if (!dst_value) {int flag;
@@ -1394,18 +1397,30 @@ static int match_explicit(struct ref *src, struct ref *dst, dst_value);break ; } -if (!matched_dst) -return -1 ; -if (matched_dst->peer_ref) -return error (_ ("dst ref %s receives from more than one src" ), - matched_dst->name); -else { + +if (!matched_dst) { + ret =-1 ; + goto out; + } + +if (matched_dst->peer_ref) { + ret =error (_ ("dst ref %s receives from more than one src" ), + matched_dst->name); + goto out; + }else { matched_dst->peer_ref = allocated_src ? matched_src :copy_ref (matched_src); matched_dst->force = rs->force; + matched_src =NULL ; } -return 0 ; + + ret =0 ; + +out: +if (allocated_src) +free_one_ref (matched_src); +return ret; } static int match_explicit_refs(struct ref *src, struct ref *dst,
t/t5505-remote.sh +1 -0
@@ -2,6 +2,7 @@ test_description='git remote porcelain-ish' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh setup_repository () {
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 17/20 remote: fix leaking peer ref when expanding refmap
In reply to Patrick SteinhardtWhen expanding remote refs via the refspec in get_expanded_map(), we first copy the remote ref and then override its peer ref with the expanded name. This may cause a memory leak though in case the peer ref is already set, as this field is being copied by copy_ref(), as well.
Fix the leak by freeing the peer ref before we re-assign the field.
--- remote.c | 2 ++ t/t5536-fetch-conflicts.sh | 1 + t/t5553-set-upstream.sh | 1 + t/t5703-upload-pack-ref-in-want.sh | 1 + t/t6050-replace.sh | 1 + 5 files changed, 6 insertions(+)
remote.c +2 -0
@@ -2081,6 +2081,8 @@ static struct ref *get_expanded_map(const struct ref *remote_refs, !ignore_symref_update(expn_name, &scratch)) {struct ref *cpy =copy_ref (ref); +if (cpy->peer_ref) +free_one_ref (cpy->peer_ref); cpy->peer_ref =alloc_ref (expn_name);if (refspec->force) cpy->peer_ref->force =1 ;
t/t5536-fetch-conflicts.sh +1 -0
@@ -2,6 +2,7 @@ test_description='fetch handles conflicting refspecs correctly' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh D=$(pwd)
t/t5553-set-upstream.sh +1 -0
@@ -4,6 +4,7 @@ test_description='"git fetch/pull --set-upstream" basic tests.' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh check_config () {
t/t5703-upload-pack-ref-in-want.sh +1 -0
@@ -2,6 +2,7 @@ test_description='upload-pack ref-in-want' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh get_actual_refs () {
t/t6050-replace.sh +1 -0
@@ -7,6 +7,7 @@ test_description='Tests replace refs functionality' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY/lib-gpg.sh"
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 18/20 builtin/fetch: fix leaking transaction with `--atomic`
In reply to Patrick SteinhardtWith the --atomic flag, we use a single ref transaction to commit all ref updates in git-fetch(1). The lifetime of transactions is somewhat
weird: while ref_transaction_abort() will free the transaction, a call to ref_transaction_commit() won't. We thus have to manually free the transaction in the successful case.
Adapt the code to free the transaction in the exit path to plug the resulting memory leak. As ref_transaction_abort() already freed the transaction for us, we have to unset the transaction when we hit that code path to not cause a double free.
--- builtin/fetch.c | 8 ++++---- t/t5574-fetch-output.sh | 1 + 2 files changed, 5 insertions(+), 4 deletions(-)
builtin/fetch.c +4 -4
@@ -1731,11 +1731,8 @@ static int do_fetch(struct transport *transport, goto cleanup; retcode =ref_transaction_commit (transaction, &err); -if (retcode) { -ref_transaction_free (transaction); - transaction =NULL ; +if (retcode) goto cleanup; - } }commit_fetch_head (&fetch_head);
@@ -1803,8 +1800,11 @@ static int do_fetch(struct transport *transport,if (transaction &&ref_transaction_abort (transaction, &err) && err.len)error ("%s" , err.buf); + transaction =NULL ; } +if (transaction) +ref_transaction_free (transaction);display_state_release (&display_state);close_fetch_head (&fetch_head);strbuf_release (&err);
t/t5574-fetch-output.sh +1 -0
@@ -5,6 +5,7 @@ test_description='git fetch output format' GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh test_expect_success 'fetch with invalid output format configuration' '
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
PATCH v2 19/20 transport: fix leaking arguments when fetching from bundle
In reply to Patrick SteinhardtIn fetch_refs_from_bundle() we assemble a vector of arguments to pass to unbundle(), but never free it. And in theory we wouldn't have to because unbundle() already knows to free the vector for us. But it fails to do so when it exits early due to verify_bundle() failing.
The calling convention that the arguments are freed by the callee and not the caller feels somewhat weird. Refactor the code such that it is instead the responsibility of the caller to free the vector, adapting the only two callsites where we pass extra arguments. This also fixes the memory leak.
This memory leak gets hit in t5510, but fixing it isn't sufficient to make the whole test suite pass.
--- builtin/bundle.c | 2 ++ bundle.c | 4 +--- transport.c | 2 ++ 3 files changed, 5 insertions(+), 3 deletions(-)
builtin/bundle.c +2 -0
@@ -220,7 +220,9 @@ static int cmd_bundle_unbundle(int argc, const char **argv, const char *prefix) &extra_index_pack_args, 0) ||list_bundle_refs (&header, argc, argv);bundle_header_release (&header); + cleanup: +strvec_clear (&extra_index_pack_args);free (bundle_file);return ret; }
bundle.c +1 -3
@@ -639,10 +639,8 @@ int unbundle(struct repository *r, struct bundle_header *header,if (flags &VERIFY_BUNDLE_FSCK )strvec_push (&ip.args,"--fsck-objects" ); -if (extra_index_pack_args) { +if (extra_index_pack_args)strvec_pushv (&ip.args, extra_index_pack_args->v); -strvec_clear (extra_index_pack_args); - } ip.in = bundle_fd; ip.no_stdout =1 ;
transport.c +2 -0
@@ -189,6 +189,8 @@ static int fetch_refs_from_bundle(struct transport *transport, &extra_index_pack_args, fetch_pack_fsck_objects() ? VERIFY_BUNDLE_FSCK : 0); transport->hash_algo = data->header.hash_algo; + +strvec_clear (&extra_index_pack_args);return ret; }
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
Re: [PATCH v2 19/20] transport: fix leaking arguments when fetching from bundle
Quoted 56 lines · Patrick Steinhardt
In
fetch_refs_from_bundle()we assemble a vector of arguments to pass tounbundle(), but never free it. And in theory we wouldn't have to becauseunbundle()already knows to free the vector for us. But it fails to do so when it exits early due toverify_bundle()failing.The calling convention that the arguments are freed by the callee and not the caller feels somewhat weird. Refactor the code such that it is instead the responsibility of the caller to free the vector, adapting the only two callsites where we pass extra arguments. This also fixes the memory leak.
This memory leak gets hit in t5510, but fixing it isn't sufficient to make the whole test suite pass.
Signed-off-by Patrick Steinhardt <[email protected]>--- builtin/bundle.c | 2 ++ bundle.c | 4 +--- transport.c | 2 ++ 3 files changed, 5 insertions(+), 3 deletions(-)builtin/bundle.c +2
-0@@ -220,7 +220,9 @@ static int cmd_bundle_unbundle(int argc, const char **argv, const char *prefix) &extra_index_pack_args, 0) ||list_bundle_refs (&header, argc, argv);bundle_header_release (&header); + cleanup: +strvec_clear (&extra_index_pack_args);free (bundle_file);return ret; }bundle.c +1
-3@@ -639,10 +639,8 @@ int unbundle(struct repository *r, struct bundle_header *header,if (flags &VERIFY_BUNDLE_FSCK )strvec_push (&ip.args,"--fsck-objects" ); -if (extra_index_pack_args) { +if (extra_index_pack_args)strvec_pushv (&ip.args, extra_index_pack_args->v); -strvec_clear (extra_index_pack_args); - } ip.in = bundle_fd; ip.no_stdout =1 ;transport.c +0
-0
transport.c static int fetch_refs_from_bundle(struct transport *transport, · line 195
Quoted 7 lines
transport.c +2
-0@@ -189,6 +189,8 @@ static int fetch_refs_from_bundle(struct transport *transport, &extra_index_pack_args, fetch_pack_fsck_objects() ? VERIFY_BUNDLE_FSCK : 0); transport->hash_algo = data->header.hash_algo; + +strvec_clear (&extra_index_pack_args);return ret; }
Looks much better. Will queue. Thanks.
PATCH v2 20/20 transport: fix leaking negotiation tips
In reply to Patrick SteinhardtWe do not free negotiation tips in the transport's smart options. Fix this by freeing them on disconnect.
--- t/t5510-fetch.sh | 1 + transport.c | 4 ++++ 2 files changed, 5 insertions(+)
t/t5510-fetch.sh +1 -0
@@ -5,6 +5,7 @@ test_description='Per branch config variables affects "git fetch". ' +TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh . "$TEST_DIRECTORY"/lib-bundle.sh
transport.c +4 -0
@@ -947,6 +947,10 @@ static int disconnect_git(struct transport *transport)finish_connect (data->conn); } +if (data->options.negotiation_tips) { +oid_array_clear (data->options.negotiation_tips); +free (data->options.negotiation_tips); + }list_objects_filter_release (&data->options.filter_options);oid_array_clear (&data->extra_have);oid_array_clear (&data->shallow);
Signature · 1 line
2.46.0.164.g477ce5ccd6.dirty
Quoted 9 lines · Patrick Steinhardt
this is version 2 of this patch series that fixes another set of memory leaks.
Changes compared to v1:
- Remove spaces between cast and variable.
- Clarify why we move the
ctx.entries_nrblock.
Quoted 7 lines
- Adapt sideband colors to use
git_config_get_string_tmp()such that we do not have to allocate the strings in the first place.- Fix more memory leaks for config values in the remote state.
- Refactor
unbundle()to not free extra args passed by the caller anymore. Instead, this is now always done by the caler.
All these changes look sensible to me.
Let me wait for a few days for others' comments, and then mark the topic for 'next'.
Thanks.