Fix 2.26.0 rebase regression and documentation shortcoming

This two commit series addresses two points raised by Peff about rebase backend issues, in reverse order of what he reported. The first is a fix to a regression in 2.26.0 (when 2.26.0-rc0 added the dropping of commits which become empty, if the last commit in the series was the one that became empty the rebase would complete without cleaning out state files), and the other is just a documentation update about a backend difference that we were previously unaware of.

The first patch is a one-line fix to the merge backend. The second patch could probably do with a fix to the apply backend but I instead only updated the documentation to note this issue in order to minimize the risk with the 2.26.0 release.

Elijah Newren (2):
sequencer: clear CHERRY_PICK_HEAD upon dropping a become-empty commit git-rebase.txt: highlight backend differences with commit rewording

 Documentation/git-rebase.txt | 10 ++++++++++
 sequencer.c                  |  1 +
 2 files changed, 11 insertions(+)

base-commit: b4374e96c84ed9394fed363973eb540da308ed4f
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-722%2Fnewren%2Frebase-fixups-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-722/newren/rebase-fixups-v1
Pull-Request: https://github.com/git/git/pull/722

gitgitgadget

From: Elijah Newren <[email protected]>

In commit e98c4269c8 ("rebase (interactive-backend): fix handling of commits that become empty", 2020-02-15), the merge backend was changed to drop commits that did not start empty but became so after being applied (because their changes were a subset of what was already upstream). This new code path did not need to go through the process of creating a commit, since we were dropping the commit instead. Unfortunately, this also means we bypassed the clearing of the CHERRY_PICK_HEAD file, which if there were no further commits to cherry-pick would mean that the rebase would end but assume there was still an operation in progress. Ensure that we clear CHERRY_PICK_HEAD when we decide to drop the commit.

---
 sequencer.c | 1 +
 1 file changed, 1 insertion(+)

index 7477b15422a..8b4e0200c5f 100644
@@ -1957,6 +1957,7 @@ static int do_pick_commit(struct repository *r,
 		flags |= ALLOW_EMPTY;
 	} else if (allow == 2) {
 		drop_commit = 1;
+		unlink(git_path_cherry_pick_head(r));
 		fprintf(stderr,
 			_("dropping %s %s -- patch contents already upstream\n"),
 			oid_to_hex(&commit->object.oid), msg.subject);

gitgitgadget

Hi Elijah

Thanks for fixing this, it needs to clean up MERGE_MSG as well though

Best Wishes

Phillip

When this fails for whatever reason, do we need to do something special? The same question for MERGE_MSG Phillip mentioned.

Thanks, both.

I don't think there's much we _can_ do. The other call-sites seem to just ignore failures. I suspect turning those into unlink_or_warn() would be a good idea (and should be safe; it treats ENOENT as a noop, so anything it reports should be an actual error worthy of the user's attention).

But probably it would make sense to do that consistently for all of those calls, and that doesn't need to be part of the 2.26 fixup.

-Peff

Yes, I think that is a sensible way forward.

Thanks, all.

From: Elijah Newren <[email protected]>

As noted by Junio:
Back when "git am" was written, it was not considered a bug that the "git am --resolved" option did not offer the user a chance to update the log message to match the adjustment of the code the user made, but honestly, I'd have to say that it is a bug in "git am" in that over time it wasn't adjusted to the new world order where we encourage users to describe what they did when the automation hiccuped by opening an editor. These days, even when automation worked well (e.g. a clean auto-merge with "git merge"), we open an editor. The world has changed, and so should the expectations.

Junio also suggested providing a workaround such as allowing --no-edit together with git rebase --continue, but that should probably be done in a patch after the git-2.26.0 release. For now, just document the known difference in the Behavioral Differences section.

---
 Documentation/git-rebase.txt | 10 ++++++++++
 1 file changed, 10 insertions(+)

index 8c1f4b82680..f7a6033607f 100644
@@ -699,6 +699,16 @@ suffer from the same shortcoming.  (See
 https://lore.kernel.org/git/[email protected]/ for
 details.)
 
+Commit Rewording
+~~~~~~~~~~~~~~~~
+
+When a conflict occurs while rebasing, rebase stops and asks the user
+to resolve.  Since the user may need to make notable changes while
+resolving conflicts, after conflicts are resolved and the user has run
+`git rebase --continue`, the rebase should open an editor and ask the
+user to update the commit message.  The merge backend does this, while
+the apply backend blindly applies the original commit message.
+
 Miscellaneous differences
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 

gitgitgadget

This two commit series addresses two points raised by Peff about rebase backend issues. The first is a two-line fix to a regression in 2.26.0 (when "the eighth batch for 2.26.0" added the dropping of commits which become empty, if the last commit in the series was the one that became empty the rebase would complete without cleaning out state files), and the other is just a documentation update about a backend difference that we were previously unaware of.

Changes since v1:

* Clean out any MERGE_MSG file in addition to CHERRY_PICK_HEAD, and add a test

Elijah Newren (2):
sequencer: clear state upon dropping a become-empty commit git-rebase.txt: highlight backend differences with commit rewording

 Documentation/git-rebase.txt | 10 ++++++++++
 sequencer.c                  |  2 ++
 t/t3424-rebase-empty.sh      |  8 ++++++++
 3 files changed, 20 insertions(+)

base-commit: b4374e96c84ed9394fed363973eb540da308ed4f
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-722%2Fnewren%2Frebase-fixups-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-722/newren/rebase-fixups-v2
Pull-Request: https://github.com/git/git/pull/722

Range-diff vs v1:

1: 132f769adb0 ! 1: 84b89d78435 sequencer: clear CHERRY_PICK_HEAD upon dropping a become-empty commit
@@ -1,6 +1,6 @@
Author: Elijah Newren <[email protected]>

- sequencer: clear CHERRY_PICK_HEAD upon dropping a become-empty commit
+ sequencer: clear state upon dropping a become-empty commit

In commit e98c4269c8 ("rebase (interactive-backend): fix handling of commits that become empty", 2020-02-15), the merge backend was changed
@@ -9,10 +9,10 @@
upstream). This new code path did not need to go through the process of creating a commit, since we were dropping the commit instead. Unfortunately, this also means we bypassed the clearing of the
- CHERRY_PICK_HEAD file, which if there were no further commits to
- cherry-pick would mean that the rebase would end but assume there was
- still an operation in progress. Ensure that we clear CHERRY_PICK_HEAD
- when we decide to drop the commit.
+ CHERRY_PICK_HEAD and MERGE_MSG files, which if there were no further
+ commits to cherry-pick would mean that the rebase would end but assume
+ there was still an operation in progress. Ensure that we clear such
+ state files when we decide to drop the commit.

@@ -24,6 +24,24 @@
} else if (allow == 2) {
drop_commit = 1;
+ unlink(git_path_cherry_pick_head(r));
++ unlink(git_path_merge_msg(r));
fprintf(stderr,
_("dropping %s %s -- patch contents already upstream\n"), oid_to_hex(&commit->object.oid), msg.subject);
+
+ diff --git a/t/t3424-rebase-empty.sh b/t/t3424-rebase-empty.sh
+ --- a/t/t3424-rebase-empty.sh
+ +++ b/t/t3424-rebase-empty.sh
+@@
+ test_cmp expect actual
+ '
+
++test_expect_success 'rebase --merge does not leave state laying around' '
++ git checkout -B testing localmods~2 &&
++ git rebase --merge upstream &&
++
++ test_path_is_missing .git/CHERRY_PICK_HEAD &&
++ test_path_is_missing .git/MERGE_MSG
++'
++
+ test_done
2: 12932b847ef = 2: 6d51cff41d9 git-rebase.txt: highlight backend differences with commit rewording

gitgitgadget

From: Elijah Newren <[email protected]>

In commit e98c4269c8 ("rebase (interactive-backend): fix handling of commits that become empty", 2020-02-15), the merge backend was changed to drop commits that did not start empty but became so after being applied (because their changes were a subset of what was already upstream). This new code path did not need to go through the process of creating a commit, since we were dropping the commit instead. Unfortunately, this also means we bypassed the clearing of the CHERRY_PICK_HEAD and MERGE_MSG files, which if there were no further commits to cherry-pick would mean that the rebase would end but assume there was still an operation in progress. Ensure that we clear such state files when we decide to drop the commit.

---
 sequencer.c             | 2 ++
 t/t3424-rebase-empty.sh | 8 ++++++++
 2 files changed, 10 insertions(+)

index 7477b15422a..e528225e787 100644
@@ -1957,6 +1957,8 @@ static int do_pick_commit(struct repository *r,
 		flags |= ALLOW_EMPTY;
 	} else if (allow == 2) {
 		drop_commit = 1;
+		unlink(git_path_cherry_pick_head(r));
+		unlink(git_path_merge_msg(r));
 		fprintf(stderr,
 			_("dropping %s %s -- patch contents already upstream\n"),
 			oid_to_hex(&commit->object.oid), msg.subject);

index 98fc2a558a0..e1e30517ea6 100755
@@ -123,4 +123,12 @@ test_expect_success 'rebase --interactive uses default of --empty=ask' '
 	test_cmp expect actual
 '
 
+test_expect_success 'rebase --merge does not leave state laying around' '
+	git checkout -B testing localmods~2 &&
+	git rebase --merge upstream &&
+
+	test_path_is_missing .git/CHERRY_PICK_HEAD &&
+	test_path_is_missing .git/MERGE_MSG
+'
+
 test_done

gitgitgadget

Thanks, I can confirm this fixes my case (which is not surprising, as it is the same as your new test). The patch looks good. Two minor comments below, but I doubt there is anything to act on.

Yeah, I was looking for something like that but instead found the unlink() directives for cleaning up various state files scattered throughout the code. I think sequencer.c is in need of some cleaning up; the slow transition from "do what shell does, now work both with an external shell and some pieces built in, now move slightly more towards being built-in" seems to have left a lot of artifacts around and made it a bit unwieldy.

As another anecdote along these lines, I really wanted to do my demo
of an in-memory rebase with the existing builtin/rebase.c and
sequencer.c but it was too much effort even for just a demo to rip out
the unwanted parts, so I did my in-memory rebase demo in a completely different file (https://github.com/newren/git/blob/git-merge-2020-demo/builtin/fast-rebase.c)

I'm not sure deferring to my expertise with sequencer.c makes sense, since you have about twice as many commits to sequencer.c as me. But I was deferring to Phillip and he commented on my v1 and seemed happy (other than my missing handling of MERGE_MSG).

Yeah, it's not clear to me what's best either. When I was testing my changes locally I was checking status output. However, after creating the fix and deciding to add a regression test, I switched to checking for the existence of those files basically for the reasons you mention, despite knowing I'm only testing for certain state files rather than testing that git in general doesn't think it's still in the middle of some operation.

OK. As long as you looked and didn't find anything obvious that should be used, I'm content to leave it for a later cleanup (I also looked briefly and didn't find anything useful).

Heh, all memories of sequencer.c have been wiped from my memory. I thought you might have looked at it more recently because of this rebase backend work, but I guess that didn't involve poking at the sequencer internals much.

I did just double check that "git status" has no way to produce a machine-readable version of the data. That might be worth addressing in general[1], but I think what you have here is a good test for now.

-Peff

[1] In particular, I think that git-prompt.sh reimplements some of this logic, and I would be surprised if there wasn't some weird corner case where they differ. The prompt code does try to avoid invoking subprocesses for efficiency, but I imagine we're running git-status already to get the dirty state.

From: Elijah Newren <[email protected]>

As noted by Junio:
Back when "git am" was written, it was not considered a bug that the "git am --resolved" option did not offer the user a chance to update the log message to match the adjustment of the code the user made, but honestly, I'd have to say that it is a bug in "git am" in that over time it wasn't adjusted to the new world order where we encourage users to describe what they did when the automation hiccuped by opening an editor. These days, even when automation worked well (e.g. a clean auto-merge with "git merge"), we open an editor. The world has changed, and so should the expectations.

Junio also suggested providing a workaround such as allowing --no-edit together with git rebase --continue, but that should probably be done in a patch after the git-2.26.0 release. For now, just document the known difference in the Behavioral Differences section.

---
 Documentation/git-rebase.txt | 10 ++++++++++
 1 file changed, 10 insertions(+)

index 8c1f4b82680..f7a6033607f 100644
@@ -699,6 +699,16 @@ suffer from the same shortcoming.  (See
 https://lore.kernel.org/git/[email protected]/ for
 details.)
 
+Commit Rewording
+~~~~~~~~~~~~~~~~
+
+When a conflict occurs while rebasing, rebase stops and asks the user
+to resolve.  Since the user may need to make notable changes while
+resolving conflicts, after conflicts are resolved and the user has run
+`git rebase --continue`, the rebase should open an editor and ask the
+user to update the commit message.  The merge backend does this, while
+the apply backend blindly applies the original commit message.
+
 Miscellaneous differences
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 

gitgitgadget

Thanks. With the earlier bug fix and this documentation change, I'm OK with keeping the merge-backend transition in v2.26.0. This change _might_ cause problems in somebody's script, but that will be true whether it comes in 2.26 or 2.27, and I think it's clear this is the end state we want to get to eventually.

-Peff

I am OK either way, so let's take these two, plus the "don't i18n the literals that must be given as option values" patch from Jiang, and discard the "let's delay the inevitable to buy some time" patch.

I merged the last one locally to 'next' already but I haven't pushed the result out, so it is still possible to back out ;-)