git-log --cherry-pick gives different results when using tag or tag^{}

Hello,

In mykernel repository, I'm having 2 different behaviours with git-log but I don't understand why:

Doing:

    $ git log --oneline --cherry-pick --left-right v3.4.71-1^{}...next

and

    $ git log --oneline --cherry-pick --left-right v3.4.71-1...next

give something different (where v3.4.71-1 is a tag).

The command using ^{} looks the one that gives correct result I think.

Could anybody enlight me ?

Thanks.

[+cc Junio, as the bug blames to him]

Yeah, this looks like a bug. Here's a simple reproduction recipe:

  commit() {
    echo content >$1 &&
    git add $1 &&
    git commit -m $1
  }

git init repo && cd repo &&
commit one &&
commit two &&
sleep 1 &&
git tag -m foo mytag &&
git checkout -b side HEAD^ &&
git cherry-pick mytag &&
commit three

The sleep seems to be necessary, to give the commit and its cherry-picked version different commit times (presumably because it impacts the order in which we visit them during the traversal).

Running:

  git log --oneline --decorate --cherry-pick --left-right mytag^{}...HEAD

produces the expected:

but running it with the tag, as:

  git log --oneline --decorate --cherry-pick --left-right mytag...HEAD

yields:

Not only do we get the cherry-pick wrong (we should omit both "twos"), but we seem to erroneously count the tagged "two" as being on the right-hand side, which it clearly is not (and which is probably why we don't find the match via --cherry-pick).

This worked in v1.8.4, but is broken in v1.8.5. It bisects to Junio's 895c5ba (revision: do not peel tags used in range notation, 2013-09-19), which sounds promising.

I think what is happening is that we used to apply the SYMMETRIC_LEFT flag directly to the commit. Now we apply it to the tag, and it does not seem to get propagated. The patch below fixes it for me, but I have no idea if we actually need to be setting the other flags, or just SYMMETRIC_LEFT. I also wonder if the non-symmetric two-dot case needs to access any pointed-to commit and propagate flags in a similar way.

index 7010aff..1d99bfc 100644
@@ -1197,6 +1197,8 @@ int handle_revision_arg(const char *arg_, struct rev_info *revs, int flags, unsi
 				free_commit_list(exclude);
 
 				a_flags = flags | SYMMETRIC_LEFT;
+				a->object.flags |= a_flags;
+				b->object.flags |= flags;
 			}
 
 			a_obj->flags |= a_flags;

-Peff

Thanks a lot Jeff for your good analyze.

Thanks.

Where do we pass down other flags from tags to commits? For example, if we do this:

	$ git log ^v1.8.5 master

we mark v1.8.5 tag as UNINTERESTING, and throw that tag (not commit v1.8.5^0) into revs->pending.objects[]. We do the same for 'master', which is a commit.

Later, in prepare_revision_walk(), we call handle_commit() on them, and unwrap the tag v1.8.5 to get v1.8.5^0, and then handles that commit object with flags obtained from the tag object. This code only cares about UNINTERESTING and manually propagates it.

Perhaps that code needs to propagate at least SYMMETRIC_LEFT down to the commit object as well, no? With your patch, the topmost level of tag object and the eventual commit object are marked with the flag, but if we were dealing with a tag that points at another tag that in turn points at a commit, the intermediate tag will not be marked with SYMMETRIC_LEFT (nor UNINTERESTING for that matter), which may not affect the final outcome, but it somewhat feels wrong.

How about doing it this way instead (totally untested, though)?

 revision.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index a68fde6..def070e 100644
@@ -276,6 +276,7 @@ static struct commit *handle_commit(struct rev_info *revs,
 				return NULL;
 			die("bad object %s", sha1_to_hex(tag->tagged->sha1));
 		}
+		object->flags |= flags;
 	}
 
 	/*

@@ -287,7 +288,6 @@ static struct commit *handle_commit(struct rev_info *revs,
 		if (parse_commit(commit) < 0)
 			die("unable to parse commit %s", name);
 		if (flags & UNINTERESTING) {
-			commit->object.flags |= UNINTERESTING;
 			mark_parents_uninteresting(commit);
 			revs->limited = 1;
 		}

Thanks for picking up this line of thought. I had some notion that the right solution would be in propagating the flags later from the pending tags to the commits, but I didn't quite know where to look. Knowing that we explicitly propagate UNINTERESTING but nothing else makes what I was seeing make a lot more sense.

Agreed. I think the lack of flags on intermediate tags has always been that way, even before 895c5ba, and I do not know of any case where it currently matters. But it seems like the obvious right thing to mark those intermediate tags.

Makes sense. It also means we will propagate flags down to any pointed-to trees and blobs. I can't think of a case where that will matter either (and they cannot be SYMMETRIC_LEFT, as that only makes sense for commit objects).

I do notice that when we have a tree, we explicitly propagate UNINTERESTING to the rest of the tree. Should we be propagating all flags instead? Again, I can't think of a reason to do so (and if it is not UNINTERESTING, it is a non-trivial amount of time to mark all paths in the tree).

    object->flags, and we already have it set

With the previous fix 895c5ba3 (revision: do not peel tags used in range notation, 2013-09-19), handle_revision_arg() that processes command line arguments for the "git log" family of commands no longer directly places the object pointed by the tag in the pending object array when it sees a tag object. We used to place pointee there after copying the flag bits like UNINTERESTING and SYMMETRIC_LEFT.

This change meant that any flag that is relevant to later history traversal must now be propagated to the pointed objects (most often these are commits) while starting the traversal, which is partly done by handle_commit() that is called from prepare_revision_walk(). We did propagate UNINTERESTING, but did not do so for others, most notably SYMMETRIC_LEFT. This caused "git log --left-right v1.0..." (where "v1.0" is a tag) to start losing the "leftness" from the commit the tag points at.

---

* Comes directly on top of the faulty commit, so that we could backport it to 1.8.4.x series.

 revision.c               |  2 +-
 t/t6000-rev-list-misc.sh | 11 +++++++++++
 2 files changed, 12 insertions(+), 1 deletion(-)

index 7010aff..6d1c8f9 100644
@@ -265,6 +265,7 @@ static struct commit *handle_commit(struct rev_info *revs, struct object *object
 				return NULL;
 			die("bad object %s", sha1_to_hex(tag->tagged->sha1));
 		}
+		object->flags |= flags;
 	}
 
 	/*

@@ -276,7 +277,6 @@ static struct commit *handle_commit(struct rev_info *revs, struct object *object
 		if (parse_commit(commit) < 0)
 			die("unable to parse commit %s", name);
 		if (flags & UNINTERESTING) {
-			commit->object.flags |= UNINTERESTING;
 			mark_parents_uninteresting(commit);
 			revs->limited = 1;
 		}

index 15e3d64..b84d6b0 100755
@@ -56,4 +56,15 @@ test_expect_success 'rev-list A..B and rev-list ^A B are the same' '
 	test_cmp expect actual
 '
 
+test_expect_success 'symleft flag bit is propagated down from tag' '
+	git log --format="%m %s" --left-right v1.0...master >actual &&
+	cat >expect <<-\EOF &&
+	> two
+	> one
+	< another
+	< that
+	EOF
+	test_cmp expect actual
+'
+
 test_done

Looks good to me. As per my previous mail, I _think_ you could squash in:

index f786b51..2db906c 100644
@@ -316,13 +316,10 @@ static struct commit *handle_commit(struct rev_info *revs,
 	 * Blob object? You know the drill by now..
 	 */
 	if (object->type == OBJ_BLOB) {
-		struct blob *blob = (struct blob *)object;
 		if (!revs->blob_objects)
 			return NULL;
-		if (flags & UNINTERESTING) {
-			mark_blob_uninteresting(blob);
+		if (flags & UNINTERESTING)
 			return NULL;
-		}
 		add_pending_object(revs, object, "");
 		return NULL;
 	}

but that is not very much code reduction (and mark_blob_uninteresting is very cheap). So it may not be worth the risk that my analysis is wrong. :)

-Peff

Your analysis is correct, but I think the pros-and-cons of the your squashable change boils down to the choice between:

- leaving it in will keep similarity between tree and blob codepaths (both have mark_X_uninteresting(); and

- reducing cycles by taking advantage of the explicit knowledge that mark_X_uninteresting() recurses for a tree while it does not for a blob.

But I have a suspicion that my patch may break if any codepath looks at the current flag on the object and decides "ah, it already is marked" and punts.

It indeed looks like mark_tree_uninteresting() does have that property. When an uninteresting tag directly points at a tree, if we propagate the UNINTERESTING bit to the pointee while peeling, wouldn't we end up calling mark_tree_uninteresting() on a tree, whose flags already have UNINTERESTING bit set, causing it not to recurse?

Extending that line of thought further, what should this do?

    git rev-list --objects ^HEAD^^{tree} HEAD^{tree} |
    git pack-object --stdin pack

It says "I am interested in the objects that is used in the tree of HEAD, but I do not need those that already appear in HEAD^".

With the current code (with or without the fix under discussion, or even without the faulty "do not peel tags used in range notation"), the tree of the HEAD^ is marked in handle_revision_arg() as UNINTERESTING when it is placed in revs->pending.objects[], and the handle_commit() --- we should rename it to handle_pending_object() or something, by the way --- will call mark_tree_uninteresting() on that tree, which then would say "It is already uninteresting" and return without marking the objects common to these two trees uninteresting, no?

I think that is a related but separate bug that dates back to prehistoric times, and the asymmetry between handle_commit() deals with commits and trees should have been a clear clue that tells us something is fishy. It calls "mark PARENTS uninteresting", leaving the responsibility of marking the commit itself to the caller, but it calls mark_tree_uninteresting() whose caller is not supposed to mark the tree itself.

Which suggest me that a right fix for this separate bug would be to introduce mark_tree_contents_uninteresting() or something, which has the parallel semantics to mark_parents_uninteresting(). Then mark_blob_uninteresting() call in the function can clearly go. Such a change will make it clear that handle_commit() is responsible for handling the flags for the given object, and any helper functions called by it should not peek and stop the flag of the object itself when deciding to recurse into the objects linked to it.