git rebase behaviour changed?

Hi,

git-rebase was a very useful tool for me to help organize my patches.

Recently, it seems the behaviour of git-rebase changed. It used to take the commits I'd made to my "master" branch and reapply them to a new "master" branch on top of "origin". Where rebase from git-0.99.x used to work, git-1.1.3 now does nothing and gives me the following message:

git-rebase origin
-> Current branch refs/heads/master is up to date.

However, I can do the "rebase" manually with:

git branch master-20060117
git reset --hard origin

git-format-patch -k --stdout --full-index origin master-20060117 | \
	git am --binary -3 -k

Is this broken, or am I meant to be doing something different now?

thanks,

Mike

What does "git-merge-base master-20060117 origin" give you? If it is the same as "origin", then the master-20060117 has been merged with origin, and rebase does not run in this case.

Here is the simplest example:

                  1---2---3---4 master
                 /
        origin  0

Of course, you _could_ extract patches #1, #2, #3, and #4 between origin and master, and apply them on top of #0 to reconstruct "master" as you found out, but there is not much point doing so.

Rebase changes the "master" branch when the development track between you (master) and upstream (origin) have forked:

                  1---2---3---4 master
                 /
        origin' 0---5---6 origin

In this case, things are rearranged by rebase:

                        1'--2'--3'--4' master
                       /
        origin' 0--5--6 origin

End of on-topic answers.

BTW, what this means is that it would not rearrange something like this:

                    2---3
                   /     \
                  1---4---5---6 master
                 / 
        origin  0

But a structure like this could be rebased:

                    2---3
                   /     \
                  1---4---5---6 master
                 / 
        origin' 0---7---8 origin

to produce something like this:

                          1'--2'--3'--4'--6' master
                         / 
        origin' 0---7---8 origin

The ordering of patches may turn out to be wrong; #4 might conflict with already applied #2 and #3. In general, rebasing such a merged structure is highly discouraged. I think there was a discussion on this topic on the list recently, and a short summary was: "if you do a merge, do not rebase; if you are going to rebase, do not merge". The thread is this one:

	http://thread.gmane.org/gmane.comp.version-control.git/14308

Especially please look at a couple of message from Linus:

	http://article.gmane.org/gmane.linux.kernel/365410
        http://article.gmane.org/gmane.linux.kernel/365409
        http://article.gmane.org/gmane.linux.kernel/365501

I guess we could decompose the commit ancestry chain in such a case, and reproduce something like this:

                            2'--3'
                           /     \
                          1'--4'--5'--6' master
                         / 
        origin' 0---7---8 origin

Rebase has never done this, though. It is left as an exercise for the reader ;-).

Well, I thought I was in the above situation, but it seems that "origin" has been merged into "master" :/

The "pull, rebase, commit, commit, send patches, pull, ..." strategy used to work for me, but now it doesn't.

I want to do rebases. So is it that behaviour of "git pull" that has been changed to do merges, and I should be using "fetch" instead of "pull" or something similar?

Mike

btw. I'm not the only person having this problem. Others using the same commands, and upgrading GIT have run into it too, so something has changed...

Now, I have realised that a simple mistake (merging from origin in you scenario) would lead git-rebase to discard earlier patches during the rebase. If you had a single commit *after* the merge, git-rebase would have rebased that single patch, and dropped earlier patches.

git-rebase should refuse to run in the above scenario. Is there a straightforward way to ask if the merge base is "shared"?

<thinking>
If the commit following right after the merge base on "our" side is a merge commit, there's a good chance we're about to fuck up. To make double sure, we can walk up that commit to the other parent (the one that is not the merge base for the current merge) and get what merge base between that commit and the current merge base. If it returns anything interesting, we bail out if we are conservative -- or walk up the history again if we take a more adventurous approach.

If it returns empty, it's a splice merge and get outta here -- you are not supposed to rebase a splice merge. And if the merge after our original merge base was an octopus, die too. Those are not rebasable either.
</thinking>

So, the conservative (and easy) approach would be to make rebase bail out when it finds any merge commit.

cheers,

martin

It may not necessarily be a mistake.

Sorry I am always slow but I am a bit slower than I usually am tonight, and do not understand this part without an illustration:

        master    1---2---3---4---5---A
                 /           /
	origin  0---6---7---B


                A = master head
                B = origin head == merge base

	rev-list B..A = 1 2 3 4 5
        rev-list A..B = 6 7

The first rev-list is "what we have but they do not". They are the candidates to be fed upstream. The latter is "what they have but we do not". Potentially some of them are commit that represent patches we submitted earlier upstream.

Among the first list of commit, there is #4 which is a merge. So we reject. Is that what you meant? Which makes sense in this picture (but I am a bit tired and maybe this may not apply in a different picture).

By the way, the longer I think about this, the more I agree with the conclusion of the earlier thread: "if you rebase, do not merge; if you merge, do not rebase". It is really about picking the right workflow.

Let's say you submitted #1, #2, #3 earlier, and #3 was accepted upstream and came back as #7, and let's further assume that we are lucky enough that patch-id gives the same answer for "diff-tree #2" and "diff-tree #7". So the set of commits left are #1, #3, and #5 (#4 is just a merge so we will not re-apply).

Now, what is the shape of the final "rebased" ancestry graph we would want?

        master                1'--3'--5'--A'
                             /
	origin  0---6---7---B

If this is what we want, why did we make #4 merge in the first place, I wonder. If the workflow is based on rebase [*1*], instead of making a merge at #4, the developer *should* have done fetch and rebase, not merge:

        master    1---2---3
                 /
	origin  0---6---7---B

        ==>

        master                1'--3'
                             /
	origin  0---6---7---B

This would have been easier to manage at the point we discovered #6, #7, and #B, than creating #4 merge only to discard it later.

And #5 and #A can be built on top of #3'.

        master                1'--3'--5---A
                             /
	origin  0---6---7---B

[Footnote]

*1* That is certainly easier to manage for an individual developer than a merge based workflow. I know it because I operated that way for a long time back when Linus was managing git)

My fault. I did have a few bits of paper here on my lap, but gmail's textbox sucks at ascii art...

Yep, exacly the example I was thinking about.

(...)

Exactly. We refuse to reset the head and begin the rebase operation, because it looks like operator error.

Definitely. But errors and misunderstandings are frequent, and people who haven't thought the process through or aren't that familiar with the internals are very likely to try it.

Refusing to rebase, with a good error msg gives the user a chance to evaluate what to do with the commits. Right now, if I read the situation right, there's a good chance commits 1,2 and 3 will be "lost" once the rebase is complete.

GIT won't literally lose them, someone could run git-fsck and fish out the dangling heads from the repo, and after a bit of spelunking recover them, but it's suddenly a really tricky operation.

cheers,

martin

"git-lost-found".

You are right. We will lose #1 and #2, (although the "already up to date" might catch some cases) and this _is_ dangerous. I need to do something about this soon.

Thanks for the discussion.

[Footnote]

*1* ... or #1 and #3 --- sorry, my "one of them picked up by upstream" scenario description was inconsistent in the previous message.

Actually, I think we are OK; I do not think we would lose any commits. git-format-patch (actually, git-cherry called from there) does the right thing. It does not use the merge base done in git-rebase in any way.

In any case, we _do_ need an explanation and error-out upon finding a merge, as we discussed. If somebody really wants to rebase a merge, he can do that by hand, as Mike easily demonstrated.

Sorry, I accidentally removed the part from my message where I said "Yes it was changed on Nov 28 after discussion on the list regarding rebase".

	$ git whatchanged -S'is up to date' git-rebase.sh

would show that commit.