Working copy as a pseudo-head

I was trying to explain to someone today why "git pull" fails if you have working-copy edits of one of the files that changed in the other branch, and it occurred to me that there may be a way to model the world so it doesn't fail. I am still relatively new to git so I hope this is not a totally idiotic idea.

Right now there is a hard distinction between what's in the working copy and the revision history of the tree. Merge is an operation on the revision history, and the working copy doesn't enter into it aside from the sanity check in git-merge to prevent you from inadvertently overwriting local edits.

What if instead, we had a notion of a temporary commit, call it WORK, that is ahead of what we currently think of as HEAD? That is, where now we have

A----B----C----D(HEAD)

we instead allowed

A----B----C----D(HEAD)----E(WORK)

When you do some operation that currently would complain about working-copy edits, it instead commits a WORK revision but does not adjust HEAD. Then it performs the operation against HEAD (advancing HEAD as needed) and rebases WORK under the newly merged HEAD, updating the working copy as a normal rebase would. Then, with one exception I'll get to below, it removes the WORK revision from the version history but leaves the contents of the working copy alone.

For example, say you have a pristine working copy of a branch with some history:

A----B----C(HEAD)

and you make some edits to the working copy. Now you want to pull down the latest changes from upstream, revision "X". So the system silently commits your work in progress to a temporary revision, which (while the merge is running) changes your history to:

A----B----C(HEAD)----D(WORK)

Then it merges the new changes with HEAD to form a new "Y" revision:

           /--D(WORK)
          /

A----B----C----Y(HEAD)
\ /
\-------X--/

Then it rebases WORK:

A----B----C----Y(HEAD)----D(WORK)
\ /
\-------X--/

And finally it removes WORK from the version history.

A----B----C----Y(HEAD) + working-copy edits
\ /
\-------X--/

I believe something like the above is currently how Cogito does its "update from the origin without losing working copy edits" thing, though it has to stash the working-copy edits in a temporary branch. Having the underlying functionality supported in the core git tools would be far preferable to having to hack around the lack of it in a frontend.

The one exception to the above flow is when you want to switch branches. Here, we create a WORK temporary commit on the current branch and move over to the new branch as normal, blowing away the WORK commit on the *new* branch, if any, and leaving its edits in the working copy. That allows you to freely switch back and forth between branches without worrying about whether you need to save your changes in progress, and without cluttering your repository with intermediate copies of files you're editing.

This is a behavior change! But I think the proposed behavior makes more sense. It treats the working copy as an extension of whatever branch you're on, rather than a separate entity that can flit back and forth between branches if there aren't conflicts or prevent you from changing branches at all if there are. It has always seemed confusing and inconsistent to me that, in some sense, doing a commit makes my changes LESS sticky -- that is,

echo foo >> testfile
git checkout other-branch
tail -1 testfile

says "foo", but

echo foo >> testfile
git commit -a
git checkout other-branch
tail -1 testfile

doesn't. I know (or I think I do) why it's doing that, but from a user experience point of view it's just weird, because it makes it look like "commit" reduces the importance of my change. And it also makes it a lot less convenient to pop over to another branch for a moment to check out the old version of some code I'm in the middle of rewriting.

Also, I realize I can use StGIT to get something like the above. But again, I think it'd be cleaner to support this in the underlying tool; "pull down some updates while I'm still working" is a very common usage model that's handled fine by other SCMs (Subversion, etc.) and right now with Git it requires a bunch of awkward extra commands. And preserving working-copy edits when switching branches seems like a huge convenience win to me; I have a hard time imagining how the current behavior is a time-saver for anyone (though I'm sure I'll hear all about it now!)

Comments? Is this just nuts?

It is not "nuts", but we do not do it currently because it is a bit too combersome to make it recoverable when things go wrong while keeping the cost of making the necessary back-up for recovery low. When switching branches with local changes, "git-checkout -m" needs to deal with a similar issue, but that case only deal with two trees and a working tree. To do this for three-way merge, the conflicts you need to deal with become more complex.

What needs to be done is very simple and straightforward. You first stash away the working tree state in a "virtual" tree, then perform the usual 3-way merge using the common ancestor, your HEAD and the other head, and come up with the merge result without any of your changes. Then you run another three-way merge between the merge result and your previous working tree state using the HEAD before the merge as the common ancestor.

       o---o---X (other head)
      /

---o---o---H (your head)
\
W (your working tree)

==>

       o---o---X (other head)
      /         \

---o---o---H---M (merge between the base trees)
\
W (your working tree)

==>

       o---o---X (other head)
      /         \

---o---o---H---M (merge between the base trees)
\ \
W---W'(your working tree, updated for the merge result)

The design goal here is that you do not want to get any of your local changes (i.e. "diff H W") to be included when you record the result of the base merge 'M', and you would want the diff between H and W to be forward-ported to the diff between M and W'.

The first goal is already quite cumbersome when the merge between X and H involve conflicing merges. We use the working tree to perform the file level merge (i.e. you edit them with your favorite editor, and tell git when you are done), so while that is happening we temporarily need to remove what was between H and W. If the merge is too complex and the user decides to revert the whole thing, we would need to rewind the index and working tree to the state at W (that means we would at least need to stash the diff between H and index, and index and working tree before starting the merge).

After the base merge is done, we would need to update the HEAD to point at M. Then we would perform the second three-way merge between M and W using the original HEAD as their common ancestor to come up with W'. When this conflicts, there is no easy way for the user to recover, other than going back to the original state (that is, HEAD points at H and working tree is W), even after having spent effort to merge between X and H to produce M.

In common usage (call that "best current practice" if you may), you do not have local changes in the working tree that would conflict with the merge, so the current behaviour feels cumbersome only when the safety valve kicks in. After seeing it fail due to the safety valve, you can do the same as what the above pictures depict using existing tools.

* preserve the local changes (i.e. come up with the virtual "W"):

git commit -a -m WIP
git tag -f WIP
git reset --hard HEAD^

* perform the merge in the now-clean tree:

git pull ;# again from the same place
edit edit
test test
git commit -a ;# resolve conflicts, test and commit

* recover what you have preserved, and discard the virtual W:

git pull --no-commit . WIP ;# merge in
git reset ;# revert index to HEAD aka base merge result. git tag -d WIP