reflog expire --stale-fix: be generous about missing objects

From: Johannes Schindelin <[email protected]>

Whenever a user runs git reflog expire --stale-fix, the most likely reason is that their repository is at least _somewhat_ corrupt. Which means that it is more than just possible that some objects are missing.

If that is the case, that can currently let the command abort through the phase where it tries to mark all reachable objects.

Instead of adding insult to injury, let's be gentle and continue as best as we can in such a scenario, simply by ignoring the missing objects and moving on.

---
Make git reflog expire --stale-fix a lot more useful

Yesterday, I tried to run a quick test to find out whether master's version of git repack prevents .bitmap files from being deleted by still having them mmap()ed. Since I do not have a build of master lying around just like that, I checked it out, built the thing, and then ran

./bin-wrappers/git -c alias.c='!(cd /path/to/directory && ./test-whether-bitmaps-are-mmaped-during-repack.sh) c

Do NOT try this at home! The problem with this invocation is that the alias will still have GIT_DIR set, therefore the git init in that script will not create a new Git directory, and the git repack -ad in that script will remove all kinds of precious objects from the Git checkout. Even though I interrupted the run as quickly as I realized that things were going wrong, my repository was corrupted in a major way, and it took me many hours to get back to a healthy state.

It made matters worse that git reflog expire --stale-fix was less helpful than it could have been, and this patch is the result of my directed emotional energy.

Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-873%2Fdscho%2Freflog-expire-with-missing-objects-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-873/dscho/reflog-expire-with-missing-objects-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/873

 builtin/reflog.c  |  3 +++
 t/t1410-reflog.sh | 26 ++++++++++++++++++++++++++
 2 files changed, 29 insertions(+)

index ca1d8079f320..09541d1c8048 100644
@@ -602,6 +602,9 @@ static int cmd_reflog_expire(int argc, const char **argv, const char *prefix)
 	 */
 	if (cb.cmd.stalefix) {
 		repo_init_revisions(the_repository, &cb.cmd.revs, prefix);
+		cb.cmd.revs.do_not_die_on_missing_tree = 1;
+		cb.cmd.revs.ignore_missing = 1;
+		cb.cmd.revs.ignore_missing_links = 1;
 		if (flags & EXPIRE_REFLOGS_VERBOSE)
 			printf(_("Marking reachable objects..."));
 		mark_reachable_objects(&cb.cmd.revs, 0, 0, NULL);

index ecccaa063453..27b9080251a9 100755
@@ -158,6 +158,32 @@ test_expect_success 'reflog expire' '
 	check_fsck "dangling commit $K"
 '
 
+test_expect_success '--stale-fix handles missing objects generously' '
+	git -c core.logAllRefUpdates=false fast-import --date-format=now <<-EOS &&
+	commit refs/heads/stale-fix
+	mark :1
+	committer Author <[email protected]> now
+	data <<EOF
+	start stale fix
+	EOF
+	M 100644 inline file
+	data <<EOF
+	contents
+	EOF
+	commit refs/heads/stale-fix
+	committer Author <[email protected]> now
+	data <<EOF
+	stale fix branch tip
+	EOF
+	from :1
+	EOS
+
+	parent_oid=$(git rev-parse stale-fix^) &&
+	test_when_finished "recover $parent_oid" &&
+	corrupt $parent_oid &&
+	git reflog expire --stale-fix
+'
+
 test_expect_success 'prune and fsck' '
 
 	git prune &&

base-commit: 1d4f2316c5b767ccbf20cc3d55c98d1f92e6e1ce

gitgitgadget

"Johannes Schindelin via GitGitGadget" <[email protected]>

I am of two minds.

I appreciate an effort of making it looser and less likely to get in trouble when running in a corrupt repository, but --stale-fix is a bit special and should probably be removed.

The only reason the option was added was because we needed to have an easy way to recover from a specific kind of reflog corruption that would have resulted by a (then known) bug in "git reflog" in the released version of Git that came immediately before the version of Git that added the "fix" option, while the root cause of the corruption got fixed.

Back when 1389d9dd (reflog expire --fix-stale, 2007-01-06) was written, it was very useful to have a way to recover from the corruption likely to have happened with the version of Git that came before it. But it no longer is relevant after this many years. There may be other ways to break the reflog entries, but --stale-fix was never designed to deal with anything but a specific way the reflog gets corrupted (namely, by the old version of Git that corrupted reflog in a specific way), so keeping it would not be very useful.

FWIW, I have used --stale-fix for cases where a repo's reflog was "corrupted" by its alternate pruning objects.

E.g., if you do something like this:

git clone -s orig.git new.git

you're now at risk of "git gc" in orig.git corrupting new.git, because its reachability check doesn't know anything about those refs. You can mitigate that by keeping a copy of new.git's refs in orig.git. Something like:

git -C orig.git fetch ../new.git refs/*:refs/preserve/* git -C orig.git gc

But that doesn't know about reflogs at all! It will still corrupt them (but only sometimes; depending how often you do that fetch, you might catch the same values in orig.git's reflog).

Possibly the correct answer here is "turn off reflogs in new.git", but they are sometimes useful, and things _mostly_ work (for history that doesn't rewind, or where the rewound bits are specific to new.git). So it's useful to be able to run something like "reflog expire --stale-fix" to clear out the occasional problem.

(A careful reader will note that objects mentioned only in the index have a similar problem; but again, those tend to be local to new.git, and don't exist at all in a server context).

-Peff

Hi,

Thank you for the original historical context.

I want to add the experience with that half year during which git gc with worktrees was broken, as it would happily ignore the reflogs of the worktree-specific HEADs, all except the one from the worktree from which git gc was run. That was some fun time, and --stale-fix was a life saver.

With this _additional_ historical context, I would deem it wise to keep --stale-fix around.

Ciao,
Dscho

The option was invented for a specific case, but if its "fix" applies for breakages caused by more recent bugs and user induced actions, I would agree that that it gives us a good reason to keep it around.

I have to wonder if the explanation in the documentation for the option needs to be made less specific to the originally motivating case, though.

Thanks.