pull: avoid crash of invalid merge head

From: Jiri Kuncar <[email protected]>

Adds NULL guards for lookup_commit_reference() to avoid segfaults.

Those invalid references are possibly caused by parallel fetches or gc racing on the same repository.

This effectively treats failed lookup as "not up to date" so caller falls to a normal merge, which reports the broken object instead of crashing.

---
pull: avoid crash of invalid merge head

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2223%2Fjirikuncar%2Fjk%2Fpull-null-merge-head-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2223/jirikuncar/jk/pull-null-merge-head-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/2223

 builtin/pull.c  | 10 +++++++++-
 t/t5520-pull.sh | 26 ++++++++++++++++++++++++++
 2 files changed, 35 insertions(+), 1 deletion(-)

index db3ee0aab3..80e79daeb9 100644
@@ -800,8 +800,12 @@ static int get_can_ff(struct object_id *orig_head,
 
 	orig_merge_head = &merge_heads->oid[0];
 	head = lookup_commit_reference(the_repository, orig_head);
-	commit_list_insert(head, &list);
+	if (!head)
+		return 0;
 	merge_head = lookup_commit_reference(the_repository, orig_merge_head);
+	if (!merge_head)
+		return 0;
+	commit_list_insert(head, &list);
 	ret = repo_is_descendant_of(the_repository, merge_head, list);
 	commit_list_free(list);
 	if (ret < 0)

@@ -820,12 +824,16 @@ static int already_up_to_date(struct object_id *orig_head,
 	struct commit *ours;
 
 	ours = lookup_commit_reference(the_repository, orig_head);
+	if (!ours)
+		return 0;
 	for (size_t i = 0; i < merge_heads->nr; i++) {
 		struct commit_list *list = NULL;
 		struct commit *theirs;
 		int ok;
 
 		theirs = lookup_commit_reference(the_repository, &merge_heads->oid[i]);
+		if (!theirs)
+			return 0;
 		commit_list_insert(theirs, &list);
 		ok = repo_is_descendant_of(the_repository, ours, list);
 		commit_list_free(list);

index 27f38ab3c8..7a3eadddd3 100755
@@ -888,4 +888,30 @@ test_expect_success 'git pull --rebase against local branch' '
 	test_cmp expect file2
 '
 
+test_expect_success 'pull does not crash when a merge head does not resolve' '
+	test_when_finished "rm -rf up dn" &&
+	git init up &&
+	(
+		cd up &&
+		test_commit base &&
+		git switch -c sideA &&
+		test_commit a &&
+		git switch -c sideB base &&
+		test_commit b
+	) &&
+	git clone up dn &&
+	(
+		cd dn &&
+		git -c fetch.unpackLimit=1000 fetch origin \
+			"+refs/heads/*:refs/remotes/origin/*" &&
+		git commit-graph write --reachable &&
+		oid=$(git rev-parse refs/remotes/origin/sideA) &&
+		obj=.git/objects/$(test_oid_to_path "$oid") &&
+		test -f "$obj" &&
+		chmod u+w "$obj" &&
+		>"$obj" &&
+		test_must_fail git pull --no-rebase origin sideA sideB
+	)
+'
+
 test_done

base-commit: fa7f9290efe2bd22dd736689597b474b93798e11

gitgitgadget

The log message sounds a bit unusual from our norm (see Documentation/SubmittingPatches).

It is of course good to deal with a corrupt state more gracefully rather than crashing. From a cursory look, the particular solution chosen, to drive the caller to perform a merge and have it fail, may smell a bit like cheating, in that we could diagnose the breakage better by reporting what was broken at each place, but it probably is a good choice.

If we really want to improve the situation for 'orig_head', for example, we would probably want to turn it into a commit object instance a lot earlier and pass the commit object instance around in the call chain. Passing around many struct object_id instances instead of object instances is an unnatural consequence of how this program evolved. It was originally written as a shell script, and of course passing hexadecimal object names was the only way the script could drive 'git merge-base' and other programs to see if the commit recorded as the current 'HEAD' will fast-forward to the commit that is fetched from the remote to be merged in, for example. Once we go that route to resolve object names early to object instances, we will not have multiple lookup_commit_reference() calls on the same object name (which require us to watch out for failures) to begin with.

The above is a long-winded way to say that it is a good improvement that does not do more than it needs to do and we will not have to spend too much effort to undo when we revamp the internals to do "the right thing" later.

From: Jiri Kuncar <[email protected]>

get_can_ff() and already_up_to_date() pass the result of lookup_commit_reference() straight to commit_list_insert() and repo_is_descendant_of() without checking it. When the object behind HEAD or one of the merge heads cannot be parsed, e.g. because a loose object was left truncated by a fetch or gc racing on the same repository, lookup_commit_reference() returns NULL and "git pull" segfaults instead of reporting the corruption.

Treat a failed lookup as "cannot fast-forward" and "not up to date", so that the caller falls through to the normal merge path, which already diagnoses the broken object and fails cleanly.

An alternative would be to report the breakage at each lookup site, which could give a more precise diagnosis. The minimal guards are preferred because they do no more than is needed to avoid the crash, and will be easy to drop once "git pull" is reworked to resolve object names into commit objects early and pass those around, at which point there will not be multiple lookups of the same object name to guard in the first place.

The test corrupts the loose object in place rather than removing
it: a missing object that is still recorded in the commit-graph is caught by the consistency check in fetch-pack before "git pull" reaches the fast-forward check, so removing it would not exercise the crash.

---
pull: avoid segfault when commit lookup fails

Changes since v1:

* Rewrite the commit message per SubmittingPatches (imperative mood, present-tense problem statement, alternatives considered), as pointed out by Junio.
* Drop the "test -f"/"chmod"/truncate steps from the test in favour of "rm -f && echo garbage >", the idiom already used in t1450. Plain "rm -f" alone does not reproduce the crash: a missing object that is still in the commit-graph is caught by fetch-pack's consistency check before "git pull" reaches get_can_ff(), so the object has to remain present but unparseable. Documented this in a test comment and in the log message.
* Drop the redundant "git fetch" in the test setup; "git clone" already populates refs/remotes/origin/*.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2223%2Fjirikuncar%2Fjk%2Fpull-null-merge-head-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2223/jirikuncar/jk/pull-null-merge-head-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/2223

Range-diff vs v1:

1: db6ecf62ec ! 1: c00ae9d699 pull: avoid crash of invalid merge head
@@ Metadata
Author: Jiri Kuncar <[email protected]>

## Commit message ##
- pull: avoid crash of invalid merge head
+ pull: avoid segfault when commit lookup fails

- Adds NULL guards for lookup_commit_reference() to avoid segfaults.
+ get_can_ff() and already_up_to_date() pass the result of
+ lookup_commit_reference() straight to commit_list_insert() and
+ repo_is_descendant_of() without checking it. When the object
+ behind HEAD or one of the merge heads cannot be parsed, e.g. because
+ a loose object was left truncated by a fetch or gc racing on the
+ same repository, lookup_commit_reference() returns NULL and
+ "git pull" segfaults instead of reporting the corruption.

- Those invalid references are possibly caused by parallel fetches or
- gc racing on the same repository.
+ Treat a failed lookup as "cannot fast-forward" and "not up to date",
+ so that the caller falls through to the normal merge path, which
+ already diagnoses the broken object and fails cleanly.

- This effectively treats failed lookup as "not up to date" so caller
- falls to a normal merge, which reports the broken object instead of
- crashing.
+ An alternative would be to report the breakage at each lookup site,
+ which could give a more precise diagnosis. The minimal guards are
+ preferred because they do no more than is needed to avoid the
+ crash, and will be easy to drop once "git pull" is reworked to
+ resolve object names into commit objects early and pass those
+ around, at which point there will not be multiple lookups of the
+ same object name to guard in the first place.
+
+ The test corrupts the loose object in place rather than removing
+ it: a missing object that is still recorded in the commit-graph is
+ caught by the consistency check in fetch-pack before "git pull"
+ reaches the fast-forward check, so removing it would not exercise
+ the crash.

@@ t/t5520-pull.sh: test_expect_success 'git pull --rebase against local branch' '
+ git clone up dn &&
+ (
+ cd dn &&
-+ git -c fetch.unpackLimit=1000 fetch origin \
-+ "+refs/heads/*:refs/remotes/origin/*" &&
+ git commit-graph write --reachable &&
+ oid=$(git rev-parse refs/remotes/origin/sideA) &&
+ obj=.git/objects/$(test_oid_to_path "$oid") &&
-+ test -f "$obj" &&
-+ chmod u+w "$obj" &&
-+ >"$obj" &&
++
++ # Corrupt the object instead of removing it: a missing
++ # object that is still in the commit-graph is caught by
++ # fetch before pull ever reaches the fast-forward check.
++ rm -f "$obj" &&
++ echo garbage >"$obj" &&
+ test_must_fail git pull --no-rebase origin sideA sideB
+ )
+'

 builtin/pull.c  | 10 +++++++++-
 t/t5520-pull.sh | 27 +++++++++++++++++++++++++++
 2 files changed, 36 insertions(+), 1 deletion(-)

index db3ee0aab3..80e79daeb9 100644
@@ -800,8 +800,12 @@ static int get_can_ff(struct object_id *orig_head,
 
 	orig_merge_head = &merge_heads->oid[0];
 	head = lookup_commit_reference(the_repository, orig_head);
-	commit_list_insert(head, &list);
+	if (!head)
+		return 0;
 	merge_head = lookup_commit_reference(the_repository, orig_merge_head);
+	if (!merge_head)
+		return 0;
+	commit_list_insert(head, &list);
 	ret = repo_is_descendant_of(the_repository, merge_head, list);
 	commit_list_free(list);
 	if (ret < 0)

@@ -820,12 +824,16 @@ static int already_up_to_date(struct object_id *orig_head,
 	struct commit *ours;
 
 	ours = lookup_commit_reference(the_repository, orig_head);
+	if (!ours)
+		return 0;
 	for (size_t i = 0; i < merge_heads->nr; i++) {
 		struct commit_list *list = NULL;
 		struct commit *theirs;
 		int ok;
 
 		theirs = lookup_commit_reference(the_repository, &merge_heads->oid[i]);
+		if (!theirs)
+			return 0;
 		commit_list_insert(theirs, &list);
 		ok = repo_is_descendant_of(the_repository, ours, list);
 		commit_list_free(list);

index 27f38ab3c8..b3ab8f4c94 100755
@@ -888,4 +888,31 @@ test_expect_success 'git pull --rebase against local branch' '
 	test_cmp expect file2
 '
 
+test_expect_success 'pull does not crash when a merge head does not resolve' '
+	test_when_finished "rm -rf up dn" &&
+	git init up &&
+	(
+		cd up &&
+		test_commit base &&
+		git switch -c sideA &&
+		test_commit a &&
+		git switch -c sideB base &&
+		test_commit b
+	) &&
+	git clone up dn &&
+	(
+		cd dn &&
+		git commit-graph write --reachable &&
+		oid=$(git rev-parse refs/remotes/origin/sideA) &&
+		obj=.git/objects/$(test_oid_to_path "$oid") &&
+
+		# Corrupt the object instead of removing it: a missing
+		# object that is still in the commit-graph is caught by
+		# fetch before pull ever reaches the fast-forward check.
+		rm -f "$obj" &&
+		echo garbage >"$obj" &&
+		test_must_fail git pull --no-rebase origin sideA sideB
+	)
+'
+
 test_done

base-commit: fa7f9290efe2bd22dd736689597b474b93798e11

gitgitgadget