Interactive rebase with submodules
I've encountered a scenario where git rebase --interactive drops a commit which contains a modification to a submodule but no other changes.
This occurs when there is a conflict when applying the commit (for example if the submodule's history has been rewritten and you are rewriting the parent repository to match the new version of the submodule).
To clarify:
git rebase -i
# Edit a commit, switching submodule to an unrelated commit git rebase --continue
# Conflict in submodule, checkout the correct submodule commit git add path/to/submodule
# Only change in index is updated submodule
git rebase --continue
# No commit is created for the submodule change
This appears to be because the git-rebase--interactive script inspects whether there is anything to commit when rebase --continue is invoked by running:
git diff-index --cached --quiet --ignore-submodules HEAD --
Is there a reason for the --ignore-submodules in this command? Removing that option results in the expected behaviour.
I can understand not updating submodules while running the rebase, but I expected that having resolved a conflict and added my change to the index it would be applied by git rebase --continue, as indeed it is if there happen to be other (non-submodule) changes in the same commit.
Signature · 1 line
John
Am 17.01.2012 19:47, schrieb John Keeping:
Quoted 16 lines
I've encountered a scenario where git rebase --interactive drops a commit which contains a modification to a submodule but no other changes.
This occurs when there is a conflict when applying the commit (for example if the submodule's history has been rewritten and you are rewriting the parent repository to match the new version of the submodule).
To clarify:
git rebase -i
# Edit a commit, switching submodule to an unrelated commit git rebase --continue
# Conflict in submodule, checkout the correct submodule commit git add path/to/submodule
# Only change in index is updated submodule
git rebase --continue
# No commit is created for the submodule change
Quoted 3 lines
This appears to be because the git-rebase--interactive script inspects whether there is anything to commit when
rebase --continueis invoked by running:git diff-index --cached --quiet --ignore-submodules HEAD --
Thanks for pinning that down.
Quoted 1 line
Is there a reason for the
--ignore-submodulesin this command? Removing that option results in the expected behaviour.
Yes, removing it will help your use case but break others. The reason for that is that because submodules are not updated during a rebase it doesn't make sense to compare their HEAD to what is recorded in the superproject, as that might have been changed by an earlier commit. And as the submodules HEAD hasn't been updated back then, it is stale and will always show up as modified (even if it wasn't).
Quoted 1 line
I can understand not updating submodules while running the rebase, but I expected that having resolved a conflict and added my change to the index it would be applied by
git rebase --continue, as indeed it is if there happen to be other (non-submodule) changes in the same commit.
The irony is that you would have to update submodules (or at least their HEAD and use "--ignore-submodules=dirty") while running rebase to make that work in all cases ;-)
But just updating the HEAD would be dangerous as you would have to be very careful to restore the submodules HEAD after the rebase, or the submodule's work tree will be out of sync.
I suspect in the long run a rebase should, e.g. when invoked with --recurse-submodules, update the submodules too and won't use the --ignore-submodule option for diff anymore ... then everything should Just Work. But until that happens, I don't see a solution for your problem.
Quoted 5 lines · Jens Lehmann
Am 17.01.2012 19:47, schrieb John Keeping:
Quoted 3 lines
This appears to be because the git-rebase--interactive script inspects whether there is anything to commit when
rebase --continueis invoked by running:git diff-index --cached --quiet --ignore-submodules HEAD --
Quoted 8 lines
Quoted 1 line
Is there a reason for the
--ignore-submodulesin this command? Removing that option results in the expected behaviour.Yes, removing it will help your use case but break others. The reason for that is that because submodules are not updated during a rebase it doesn't make sense to compare their HEAD to what is recorded in the superproject, as that might have been changed by an earlier commit. And as the submodules HEAD hasn't been updated back then, it is stale and will always show up as modified (even if it wasn't).
Is this worse than the current behaviour? If I perform a rebase where there is a (non-submodule) conflict in a commit where a submodule changes I can see something like:
# Changes to be committed:
# modified: path/to/submodule
#
# Unmerged paths:
# both modified: path/to/file
#
# Changes not staged for commit:
# modified: path/to/submodule (new commits)
This occurs if a later commit in the rebase will modify the submodule. In this case, rebase --continue correctly recreates the commit once I have resolved the conflict in the file, ignoring the unstaged submodule changes.
Quoted 5 lines
Quoted 1 line
I can understand not updating submodules while running the rebase, but I expected that having resolved a conflict and added my change to the index it would be applied by
git rebase --continue, as indeed it is if there happen to be other (non-submodule) changes in the same commit.The irony is that you would have to update submodules (or at least their HEAD and use "--ignore-submodules=dirty") while running rebase to make that work in all cases ;-)
I don't this this is the case, since diff-tree is being invoked with --cached won't it ignore changes in the work tree anyway?
Quoted 3 lines
But just updating the HEAD would be dangerous as you would have to be very careful to restore the submodules HEAD after the rebase, or the submodule's work tree will be out of sync.
Just updating HEAD in the submodule without touching its work tree doesn't seem like a good idea. I think it will cause a lot more confusion when running git status which will show unexpected modified content for the submodule.
Since I did not expect rebase to perform a submodule update, I was not surprised to see unstaged submodule changes when rebasing, but I did expect rebase to commit anything I had added to the index.
Signature · 1 line
John
Am 18.01.2012 12:21, schrieb John Keeping:
Quoted 8 lines · Jens Lehmann
Quoted 7 lines
Am 17.01.2012 19:47, schrieb John Keeping:
Quoted 5 lines
This appears to be because the git-rebase--interactive script inspects whether there is anything to commit when
rebase --continueis invoked by running:git diff-index --cached --quiet --ignore-submodules HEAD --Is there a reason for the
--ignore-submodulesin this command? Removing that option results in the expected behaviour.
Quoted 8 lines
Quoted 6 lines
Yes, removing it will help your use case but break others. The reason for that is that because submodules are not updated during a rebase it doesn't make sense to compare their HEAD to what is recorded in the superproject, as that might have been changed by an earlier commit. And as the submodules HEAD hasn't been updated back then, it is stale and will always show up as modified (even if it wasn't).
Is this worse than the current behaviour?
If we break established behavior that would be worse unless we have a *very* good reason to do so. But please see below ...
Quoted 11 lines
If I perform a rebase where there is a (non-submodule) conflict in a commit where a submodule changes I can see something like:
# Changes to be committed:
# modified: path/to/submodule
#
# Unmerged paths:
# both modified: path/to/file
#
# Changes not staged for commit:
# modified: path/to/submodule (new commits)
Quoted 1 line
This occurs if a later commit in the rebase will modify the submodule. In this case,
rebase --continuecorrectly recreates the commit once I have resolved the conflict in the file, ignoring the unstaged submodule changes.
Yeah, as rebase doesn't touch the submodules, that is expected.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
I can understand not updating submodules while running the rebase, but I expected that having resolved a conflict and added my change to the index it would be applied by
git rebase --continue, as indeed it is if there happen to be other (non-submodule) changes in the same commit.The irony is that you would have to update submodules (or at least their HEAD and use "--ignore-submodules=dirty") while running rebase to make that work in all cases ;-)
I don't this this is the case, since diff-tree is being invoked with --cached won't it ignore changes in the work tree anyway?
Right, thanks for nudging me with the clue bat ...
I missed the "--cached" option and did not question if the code does what the commit message of 6848d58c6 (where the --ignore-submodules option was introduced) said:
Ignore dirty submodule states during rebase and stash
When rebasing or stashing, chances are that you do not care about dirty submodules, since they are not updated by those actions anyway. So ignore the submodules' states.
Note: the submodule states -- as committed in the superproject -- will still be stashed and rebased, it is _just_ the state of the submodule in the working tree which is ignored.
I think this logic misses the case when only submodule changes are left in a commit.
Quoted 5 lines
Quoted 3 lines
But just updating the HEAD would be dangerous as you would have to be very careful to restore the submodules HEAD after the rebase, or the submodule's work tree will be out of sync.
Just updating HEAD in the submodule without touching its work tree doesn't seem like a good idea. I think it will cause a lot more confusion when running
git statuswhich will show unexpected modified content for the submodule.
Yes, we agree here.
Quoted 1 line
Since I did not expect rebase to perform a submodule update, I was not surprised to see unstaged submodule changes when rebasing, but I did expect rebase to commit anything I had added to the index.
Right.
I'll have to add some tests for that case, but I doubt I'll manage that today. Until I can provide a complete patch, this diff should fix your problem (no, I did not test if that change is enough to fix the problem, but at least it does not break the test suite ;-):
---------------8<--------------
git-rebase--interactive.sh +1 -1
@@ -672,7 +672,7 @@ rearrange_squash () { case "$action" in continue) # do we have anything to commit? - if git diff-index --cached --quiet --ignore-submodules HEAD -- + if git diff-index --cached --quiet HEAD -- then : Nothing to commit -- skip this else
Quoted 23 lines · Jens Lehmann
Am 18.01.2012 12:21, schrieb John Keeping:
Quoted 9 lines · Jens Lehmann
Quoted 6 lines
Am 17.01.2012 19:47, schrieb John Keeping:
Quoted 1 line
I can understand not updating submodules while running the rebase, but I expected that having resolved a conflict and added my change to the index it would be applied by
git rebase --continue, as indeed it is if there happen to be other (non-submodule) changes in the same commit.The irony is that you would have to update submodules (or at least their HEAD and use "--ignore-submodules=dirty") while running rebase to make that work in all cases ;-)
I don't this this is the case, since diff-tree is being invoked with --cached won't it ignore changes in the work tree anyway?
Right, thanks for nudging me with the clue bat ...
I missed the "--cached" option and did not question if the code does what the commit message of
6848d58c6(where the --ignore-submodules option was introduced) said:Ignore dirty submodule states during rebase and stash
When rebasing or stashing, chances are that you do not care about dirty submodules, since they are not updated by those actions anyway. So ignore the submodules' states.
Quoted 6 lines
Note: the submodule states -- as committed in the superproject -- will still be stashed and rebased, it is _just_ the state of the submodule in the working tree which is ignored.
I think this logic misses the case when only submodule changes are left in a commit.
Yes, this is precisely the case that doesn't work as I expect.
Quoted 23 lines
Quoted 5 lines
Quoted 3 lines
But just updating the HEAD would be dangerous as you would have to be very careful to restore the submodules HEAD after the rebase, or the submodule's work tree will be out of sync.
Just updating HEAD in the submodule without touching its work tree doesn't seem like a good idea. I think it will cause a lot more confusion when running
git statuswhich will show unexpected modified content for the submodule.Yes, we agree here.
Quoted 1 line
Since I did not expect rebase to perform a submodule update, I was not surprised to see unstaged submodule changes when rebasing, but I did expect rebase to commit anything I had added to the index.
Right.
I'll have to add some tests for that case, but I doubt I'll manage that today. Until I can provide a complete patch, this diff should fix your problem (no, I did not test if that change is enough to fix the problem, but at least it does not break the test suite ;-):
---------------8<--------------
git-rebase--interactive.sh +0
-0
git-rebase--interactive.sh rearrange_squash () { · line 678
Quoted 8 lines
git-rebase--interactive.sh +1
-1@@ -672,7 +672,7 @@ rearrange_squash () { case "$action" in continue) # do we have anything to commit? - if git diff-index --cached --quiet --ignore-submodules HEAD -- + if git diff-index --cached --quiet HEAD -- then : Nothing to commit -- skip this else
This patch does indeed fix the problem I was seeing.
Thanks,
Signature · 1 line
John