Skip to contentSkip to conversation

Interactive rebase with submodules

DISCUSSION John Keeping 2 participants 5 messages Jun 15, 2016

lore
Share & tools
lore
#0 lore raw

I've encountered a scenario where git rebase --interactive drops a commit which contains a modification to a submodule but no other changes.

This occurs when there is a conflict when applying the commit (for example if the submodule's history has been rewritten and you are rewriting the parent repository to match the new version of the submodule).

To clarify:

git rebase -i
# Edit a commit, switching submodule to an unrelated commit git rebase --continue
# Conflict in submodule, checkout the correct submodule commit git add path/to/submodule
# Only change in index is updated submodule
git rebase --continue
# No commit is created for the submodule change

This appears to be because the git-rebase--interactive script inspects whether there is anything to commit when rebase --continue is invoked by running:

    git diff-index --cached --quiet --ignore-submodules HEAD --

Is there a reason for the --ignore-submodules in this command? Removing that option results in the expected behaviour.

I can understand not updating submodules while running the rebase, but I expected that having resolved a conflict and added my change to the index it would be applied by git rebase --continue, as indeed it is if there happen to be other (non-submodule) changes in the same commit.

Signature · 1 line
John
#1 lore raw

Am 17.01.2012 19:47, schrieb John Keeping:

Quoted 16 lines

I've encountered a scenario where git rebase --interactive drops a commit which contains a modification to a submodule but no other changes.

This occurs when there is a conflict when applying the commit (for example if the submodule's history has been rewritten and you are rewriting the parent repository to match the new version of the submodule).

To clarify:

git rebase -i
# Edit a commit, switching submodule to an unrelated commit git rebase --continue
# Conflict in submodule, checkout the correct submodule commit git add path/to/submodule
# Only change in index is updated submodule
git rebase --continue
# No commit is created for the submodule change

Quoted 3 lines

This appears to be because the git-rebase--interactive script inspects whether there is anything to commit when rebase --continue is invoked by running:

    git diff-index --cached --quiet --ignore-submodules HEAD --

Thanks for pinning that down.

Quoted 1 line

Is there a reason for the --ignore-submodules in this command? Removing that option results in the expected behaviour.

Yes, removing it will help your use case but break others. The reason for that is that because submodules are not updated during a rebase it doesn't make sense to compare their HEAD to what is recorded in the superproject, as that might have been changed by an earlier commit. And as the submodules HEAD hasn't been updated back then, it is stale and will always show up as modified (even if it wasn't).

Quoted 1 line

I can understand not updating submodules while running the rebase, but I expected that having resolved a conflict and added my change to the index it would be applied by git rebase --continue, as indeed it is if there happen to be other (non-submodule) changes in the same commit.

The irony is that you would have to update submodules (or at least their HEAD and use "--ignore-submodules=dirty") while running rebase to make that work in all cases ;-)

But just updating the HEAD would be dangerous as you would have to be very careful to restore the submodules HEAD after the rebase, or the submodule's work tree will be out of sync.

I suspect in the long run a rebase should, e.g. when invoked with --recurse-submodules, update the submodules too and won't use the --ignore-submodule option for diff anymore ... then everything should Just Work. But until that happens, I don't see a solution for your problem.

#2 lore raw
Quoted 5 lines · Jens Lehmann

Am 17.01.2012 19:47, schrieb John Keeping:

Quoted 3 lines

This appears to be because the git-rebase--interactive script inspects whether there is anything to commit when rebase --continue is invoked by running:

     git diff-index --cached --quiet --ignore-submodules HEAD --
Quoted 8 lines
Quoted 1 line

Is there a reason for the --ignore-submodules in this command? Removing that option results in the expected behaviour.

Yes, removing it will help your use case but break others. The reason for that is that because submodules are not updated during a rebase it doesn't make sense to compare their HEAD to what is recorded in the superproject, as that might have been changed by an earlier commit. And as the submodules HEAD hasn't been updated back then, it is stale and will always show up as modified (even if it wasn't).

Is this worse than the current behaviour? If I perform a rebase where there is a (non-submodule) conflict in a commit where a submodule changes I can see something like:

# Changes to be committed:
# modified: path/to/submodule
#
# Unmerged paths:
# both modified: path/to/file
#
# Changes not staged for commit:
# modified: path/to/submodule (new commits)

This occurs if a later commit in the rebase will modify the submodule. In this case, rebase --continue correctly recreates the commit once I have resolved the conflict in the file, ignoring the unstaged submodule changes.

Quoted 5 lines
Quoted 1 line

I can understand not updating submodules while running the rebase, but I expected that having resolved a conflict and added my change to the index it would be applied by git rebase --continue, as indeed it is if there happen to be other (non-submodule) changes in the same commit.

The irony is that you would have to update submodules (or at least their HEAD and use "--ignore-submodules=dirty") while running rebase to make that work in all cases ;-)

I don't this this is the case, since diff-tree is being invoked with --cached won't it ignore changes in the work tree anyway?

Quoted 3 lines

But just updating the HEAD would be dangerous as you would have to be very careful to restore the submodules HEAD after the rebase, or the submodule's work tree will be out of sync.

Just updating HEAD in the submodule without touching its work tree doesn't seem like a good idea. I think it will cause a lot more confusion when running git status which will show unexpected modified content for the submodule.

Since I did not expect rebase to perform a submodule update, I was not surprised to see unstaged submodule changes when rebasing, but I did expect rebase to commit anything I had added to the index.

Signature · 1 line
John
#3 lore raw

Am 18.01.2012 12:21, schrieb John Keeping:

Quoted 8 lines · Jens Lehmann
Quoted 7 lines

Am 17.01.2012 19:47, schrieb John Keeping:

Quoted 5 lines

This appears to be because the git-rebase--interactive script inspects whether there is anything to commit when rebase --continue is invoked by running:

     git diff-index --cached --quiet --ignore-submodules HEAD --

Is there a reason for the --ignore-submodules in this command? Removing that option results in the expected behaviour.

Quoted 8 lines
Quoted 6 lines

Yes, removing it will help your use case but break others. The reason for that is that because submodules are not updated during a rebase it doesn't make sense to compare their HEAD to what is recorded in the superproject, as that might have been changed by an earlier commit. And as the submodules HEAD hasn't been updated back then, it is stale and will always show up as modified (even if it wasn't).

Is this worse than the current behaviour?

If we break established behavior that would be worse unless we have a *very* good reason to do so. But please see below ...

Quoted 11 lines

If I perform a rebase where there is a (non-submodule) conflict in a commit where a submodule changes I can see something like:

# Changes to be committed:
# modified: path/to/submodule
#
# Unmerged paths:
# both modified: path/to/file
#
# Changes not staged for commit:
# modified: path/to/submodule (new commits)

Quoted 1 line

This occurs if a later commit in the rebase will modify the submodule. In this case, rebase --continue correctly recreates the commit once I have resolved the conflict in the file, ignoring the unstaged submodule changes.

Yeah, as rebase doesn't touch the submodules, that is expected.

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

I can understand not updating submodules while running the rebase, but I expected that having resolved a conflict and added my change to the index it would be applied by git rebase --continue, as indeed it is if there happen to be other (non-submodule) changes in the same commit.

The irony is that you would have to update submodules (or at least their HEAD and use "--ignore-submodules=dirty") while running rebase to make that work in all cases ;-)

I don't this this is the case, since diff-tree is being invoked with --cached won't it ignore changes in the work tree anyway?

Right, thanks for nudging me with the clue bat ...

I missed the "--cached" option and did not question if the code does what the commit message of 6848d58c6 (where the --ignore-submodules option was introduced) said:

Ignore dirty submodule states during rebase and stash

When rebasing or stashing, chances are that you do not care about dirty submodules, since they are not updated by those actions anyway. So ignore the submodules' states.

Note: the submodule states -- as committed in the superproject -- will still be stashed and rebased, it is _just_ the state of the submodule in the working tree which is ignored.

I think this logic misses the case when only submodule changes are left in a commit.

Quoted 5 lines
Quoted 3 lines

But just updating the HEAD would be dangerous as you would have to be very careful to restore the submodules HEAD after the rebase, or the submodule's work tree will be out of sync.

Just updating HEAD in the submodule without touching its work tree doesn't seem like a good idea. I think it will cause a lot more confusion when running git status which will show unexpected modified content for the submodule.

Yes, we agree here.

Quoted 1 line

Since I did not expect rebase to perform a submodule update, I was not surprised to see unstaged submodule changes when rebasing, but I did expect rebase to commit anything I had added to the index.

Right.

I'll have to add some tests for that case, but I doubt I'll manage that today. Until I can provide a complete patch, this diff should fix your problem (no, I did not test if that change is enough to fix the problem, but at least it does not break the test suite ;-):

---------------8<--------------

git-rebase--interactive.sh +1 -1
index 5812222..4546749 100644
@@ -672,7 +672,7 @@ rearrange_squash () {
 case "$action" in
 continue)
        # do we have anything to commit?
-       if git diff-index --cached --quiet --ignore-submodules HEAD --
+       if git diff-index --cached --quiet HEAD --
        then
                : Nothing to commit -- skip this
        else
#4 lore raw
Quoted 23 lines · Jens Lehmann

Am 18.01.2012 12:21, schrieb John Keeping:

Quoted 9 lines · Jens Lehmann
Quoted 6 lines

Am 17.01.2012 19:47, schrieb John Keeping:

Quoted 1 line

I can understand not updating submodules while running the rebase, but I expected that having resolved a conflict and added my change to the index it would be applied by git rebase --continue, as indeed it is if there happen to be other (non-submodule) changes in the same commit.

The irony is that you would have to update submodules (or at least their HEAD and use "--ignore-submodules=dirty") while running rebase to make that work in all cases ;-)

I don't this this is the case, since diff-tree is being invoked with --cached won't it ignore changes in the work tree anyway?

Right, thanks for nudging me with the clue bat ...

I missed the "--cached" option and did not question if the code does what the commit message of 6848d58c6 (where the --ignore-submodules option was introduced) said:

Ignore dirty submodule states during rebase and stash

When rebasing or stashing, chances are that you do not care about dirty submodules, since they are not updated by those actions anyway. So ignore the submodules' states.

Quoted 6 lines

Note: the submodule states -- as committed in the superproject -- will still be stashed and rebased, it is _just_ the state of the submodule in the working tree which is ignored.

I think this logic misses the case when only submodule changes are left in a commit.

Yes, this is precisely the case that doesn't work as I expect.

Quoted 23 lines
Quoted 5 lines
Quoted 3 lines

But just updating the HEAD would be dangerous as you would have to be very careful to restore the submodules HEAD after the rebase, or the submodule's work tree will be out of sync.

Just updating HEAD in the submodule without touching its work tree doesn't seem like a good idea. I think it will cause a lot more confusion when running git status which will show unexpected modified content for the submodule.

Yes, we agree here.

Quoted 1 line

Since I did not expect rebase to perform a submodule update, I was not surprised to see unstaged submodule changes when rebasing, but I did expect rebase to commit anything I had added to the index.

Right.

I'll have to add some tests for that case, but I doubt I'll manage that today. Until I can provide a complete patch, this diff should fix your problem (no, I did not test if that change is enough to fix the problem, but at least it does not break the test suite ;-):

---------------8<--------------

git-rebase--interactive.sh +0 -0
index 5812222..4546749 100644

git-rebase--interactive.sh rearrange_squash () { · line 678

Quoted 8 lines
git-rebase--interactive.sh +1 -1
@@ -672,7 +672,7 @@ rearrange_squash () {
  case "$action" in
  continue)
         # do we have anything to commit?
-       if git diff-index --cached --quiet --ignore-submodules HEAD --
+       if git diff-index --cached --quiet HEAD --
         then
                 : Nothing to commit -- skip this
         else

This patch does indeed fix the problem I was seeing.

Thanks,

Signature · 1 line
John