git fetch with refspec does not include tags?

DISCUSSION Carlsson, Magnus 4 participants 12 messages Aug 17, 2017 – Aug 20, 2017 1 message not in the archive

Share & tools

Carlsson, Magnus

author

2017-08-17 09:03 UTC #1 lore raw

In reply to a message not in the archive

In the git fetch documentation it states that by default you will fetch all tags that point into the history to the branches fetched.

"By default, any tag that points into the histories being fetched is also fetched; the effect is to fetch tags that point at branches that you are interested in. This default behavior can be changed by using the --tags or --no-tags options or by configuring remote.<name>.tagOpt. By using a refspec that fetches tags explicitly, you can fetch tags that do not point into branches you are interested in as well."

But for me I get tags if I do "git fetch" or "git fetch origin" but if I do "git fetch origin master" I don't get tags related to the master branch.

I understand that this might be due to me specifying a refspec and then it will only get that exact refspec, but for me it's not that clear from the documentation what I should expect. I read it as when I fetch something all related tags will come along.

Using: git version 2.13.0.rc1.15.gd2bbb7c

-- Magnus

MAGNUS CARLSSON
Staff Software Engineer
ARRIS

o: +46 13 36 75 92
e: [email protected]
w: www.arris.com

ARRIS: Legal entity: Arris Sweden AB - Registered Office: Teknikringen 2, 583 30 Linkoping, Sweden - Reg No:556518-5831 - VAT No:SE 556518-583

This electronic transmission (and any attached document) is for the sole use of the individual or entity to whom it is addressed. It is confidential and may be attorney-client privileged. In any event the Sender reserves, to the fullest extent, any "legal advice privilege". Any further distribution or copying of this message is strictly prohibited. If you received this message in error, please notify the Sender immediately and destroy the attached message (and all attached documents).

Jeff King

2017-08-17 09:29 UTC #2 lore raw

Quoted 11 lines · Magnus

In the git fetch documentation it states that by default you will fetch all tags that point into the history to the branches fetched.
"By default, any tag that points into the histories being fetched is also fetched; the effect is to fetch tags that point at branches that you are interested in. This default behavior can be changed by using the --tags or --no-tags options or by configuring remote.<name>.tagOpt. By using a refspec that fetches tags explicitly, you can fetch tags that do not point into branches you are interested in as well."

Quoted 8 lines

But for me I get tags if I do "git fetch" or "git fetch origin" but if I do "git fetch origin master" I don't get tags related to the master branch.
I understand that this might be due to me specifying a refspec and then it will only get that exact refspec, but for me it's not that clear from the documentation what I should expect. I read it as when I fetch something all related tags will come along.

I'll admit that our tag-autofollow behavior has often confused me. So I'll try to untangle what's happening at least if not the reasoning. :)

I think the problem is not that you have a refspec, but that your refspec has no destination. Looking at the fetch code, we seem to turn on autotags only when the destination is a "real" ref and not just the default FETCH_HEAD. Which sort-of makes sense. If you're doing a one-off into FETCH_HEAD, you probably don't want to create tags, even if you have the objects they point to.

But this is further complicated by the opportunistic tracking-ref updates. You can see some interesting behavior with a setup like this:

git init parent
git -C parent commit --allow-empty -m one &&
git -C parent tag -m foo mytag

git init child
cd child
git remote add origin ../parent

and then:

  # no tags, we just populate FETCH_HEAD because of the bare URL
  git fetch ../parent

  # this does fetch tags, because we're storing the result according to
  # the configured refspec ("refs/heads/*:refs/remotes/origin/*").
  git fetch origin

  # this doesn't fetch tags, as the main command is "just" populating
  # FETCH_HEAD. But then our logic for "hey, we fetched the ref for
  # refs/remotes/origin/master, so let's update it on the side" kicks
  # in. And we end up updating FETCH_HEAD _and_ the tracking branch, but
  # not the tags. Weird.
  git fetch origin master

  # and this one does fetch tags, because we have a real destination.
  git fetch origin master:foo

So what I'd say is:

1. Definitely these defaults are under-documented. I couldn't find them anywhere in git-fetch(1).

2. If we continue to follow the "are we storing any refs" rule for the default, possibly it should expand to "did we store anything, including opportunistic tracking-ref updates".

-Peff

Carlsson, Magnus

author

2017-08-17 11:30 UTC #3 lore raw

Thanks for your quick answer!

Quoted 2 lines

1. Definitely these defaults are under-documented. I couldn't find them anywhere in git-fetch(1).

Yes, some sort of explanation would be good, especially since one of the first sentences state that you always get relevant tags.

Quoted 3 lines

2. If we continue to follow the "are we storing any refs" rule for the default, possibly it should expand to "did we store anything, including opportunistic tracking-ref updates".

Personally I think that I would prefer to always fetch relevant tags. If I don't want tags I can simply use the "--no-tags"

-- Magnus

MAGNUS CARLSSON
Staff Software Engineer
ARRIS

o: +46 13 36 75 92
e: [email protected]
w: www.arris.com

ARRIS: Legal entity: Arris Sweden AB - Registered Office: Teknikringen 2, 583 30 Linkoping, Sweden - Reg No:556518-5831 - VAT No:SE 556518-583

________________________________________
From: Jeff King <[email protected]>
Sent: Thursday, August 17, 2017 11:28
To: Carlsson, Magnus
Cc: [email protected]
Subject: Re: git fetch with refspec does not include tags?

Quoted 11 lines · Magnus

In the git fetch documentation it states that by default you will fetch all tags that point into the history to the branches fetched.
"By default, any tag that points into the histories being fetched is also fetched; the effect is to fetch tags that point at branches that you are interested in. This default behavior can be changed by using the --tags or --no-tags options or by configuring remote.<name>.tagOpt. By using a refspec that fetches tags explicitly, you can fetch tags that do not point into branches you are interested in as well."

Quoted 8 lines

But for me I get tags if I do "git fetch" or "git fetch origin" but if I do "git fetch origin master" I don't get tags related to the master branch.
I understand that this might be due to me specifying a refspec and then it will only get that exact refspec, but for me it's not that clear from the documentation what I should expect. I read it as when I fetch something all related tags will come along.

I'll admit that our tag-autofollow behavior has often confused me. So I'll try to untangle what's happening at least if not the reasoning. :)

But this is further complicated by the opportunistic tracking-ref updates. You can see some interesting behavior with a setup like this:

git init parent
git -C parent commit --allow-empty -m one &&
git -C parent tag -m foo mytag

git init child
cd child
git remote add origin ../parent

and then:

  # no tags, we just populate FETCH_HEAD because of the bare URL
  git fetch ../parent

  # this does fetch tags, because we're storing the result according to
  # the configured refspec ("refs/heads/*:refs/remotes/origin/*").
  git fetch origin

  # this doesn't fetch tags, as the main command is "just" populating
  # FETCH_HEAD. But then our logic for "hey, we fetched the ref for
  # refs/remotes/origin/master, so let's update it on the side" kicks
  # in. And we end up updating FETCH_HEAD _and_ the tracking branch, but
  # not the tags. Weird.
  git fetch origin master

  # and this one does fetch tags, because we have a real destination.
  git fetch origin master:foo

So what I'd say is:

1. Definitely these defaults are under-documented. I couldn't find them anywhere in git-fetch(1).

2. If we continue to follow the "are we storing any refs" rule for the default, possibly it should expand to "did we store anything, including opportunistic tracking-ref updates".

-Peff

Jeff King

2017-08-17 14:22 UTC #4 lore raw

Quoted 6 lines · Magnus

Quoted 3 lines
2. If we continue to follow the "are we storing any refs" rule for the default, possibly it should expand to "did we store anything, including opportunistic tracking-ref updates".
Personally I think that I would prefer to always fetch relevant tags. If I don't want tags I can simply use the "--no-tags"

I think it's a bit more complex because "git pull" uses "git fetch" under the hood. In fact, your "git fetch origin master" is exactly what gets run when you do:

git pull origin master

That's maybe OK. But I think one-off pulls like:

git pull https://example.com/repo.git master

probably wouldn't want it. I'd have to give it some thought.

-Peff

Junio C Hamano

2017-08-17 19:39 UTC #6 lore raw

In reply to Jeff King

Quoted 6 lines · Jeff King

  # no tags, we just populate FETCH_HEAD because of the bare URL
  git fetch ../parent

  # this does fetch tags, because we're storing the result according to
  # the configured refspec ("refs/heads/*:refs/remotes/origin/*").
  git fetch origin

The above two look good.

Quoted 6 lines

  # this doesn't fetch tags, as the main command is "just" populating
  # FETCH_HEAD. But then our logic for "hey, we fetched the ref for
  # refs/remotes/origin/master, so let's update it on the side" kicks
  # in. And we end up updating FETCH_HEAD _and_ the tracking branch, but
  # not the tags. Weird.
  git fetch origin master

Yes, it looks weird, but I suspect that it is probably more correct not to fetch tags in this case. I wonder if it would be a solution not to do the "on the side" thing---after all the user didn't tell us to update refs/remotes/origin/master with this command line.

Then not following tags will be in line with all of the above reasoning above.

Quoted 2 lines

  # and this one does fetch tags, because we have a real destination.
  git fetch origin master:foo

Yup, that is expected.

Quoted 4 lines

So what I'd say is:
1. Definitely these defaults are under-documented. I couldn't find them anywhere in git-fetch(1).

Yup.

Quoted 3 lines

2. If we continue to follow the "are we storing any refs" rule for the default, possibly it should expand to "did we store anything, including opportunistic tracking-ref updates".

That also is a workable way to make things consistent.

Junio C Hamano

2017-08-17 19:41 UTC #5 lore raw

In reply to Jeff King

Quoted 11 lines · Jeff King

I think it's a bit more complex because "git pull" uses "git fetch" under the hood. In fact, your "git fetch origin master" is exactly what gets run when you do:
git pull origin master
That's maybe OK. But I think one-off pulls like:
git pull https://example.com/repo.git master
probably wouldn't want it. I'd have to give it some thought.

I agree with both. If you have named remote, you presumably are keeping copies of their branches as remote-tracking branches, and it may be fine to follow tags. An explicit URL used for one-off should not grab anything but the named thing, I would think.

Kevin Daudt

2017-08-17 20:23 UTC #7 lore raw

In reply to Junio C Hamano

Quoted 18 lines · Junio C Hamano

Quoted 6 lines · Jeff King

  # no tags, we just populate FETCH_HEAD because of the bare URL
  git fetch ../parent

  # this does fetch tags, because we're storing the result according to
  # the configured refspec ("refs/heads/*:refs/remotes/origin/*").
  git fetch origin

The above two look good.

Quoted 6 lines

  # this doesn't fetch tags, as the main command is "just" populating
  # FETCH_HEAD. But then our logic for "hey, we fetched the ref for
  # refs/remotes/origin/master, so let's update it on the side" kicks
  # in. And we end up updating FETCH_HEAD _and_ the tracking branch, but
  # not the tags. Weird.
  git fetch origin master

Quoted 4 lines

Yes, it looks weird, but I suspect that it is probably more correct not to fetch tags in this case. I wonder if it would be a solution not to do the "on the side" thing---after all the user didn't tell us to update refs/remotes/origin/master with this command line.

Isn't that how git fetch used to behave, or am I misunderstanding what you mean? It used to be that git fetch <remote> <branch> would not update any remote tracking branches.

From the 1.8.4 release notes:

Quoted 7 lines

"git fetch origin master" unlike "git fetch origin" or "git fetch" did not update "refs/remotes/origin/master"; this was an early design decision to keep the update of remote tracking branches predictable, but in practice it turns out that people find it more convenient to opportunistically update them whenever we have a chance, and we have been updating them when we run "git push" which already breaks the original "predictability" anyway.

Junio C Hamano

2017-08-17 20:38 UTC #8 lore raw

Quoted 24 lines · Kevin Daudt

Quoted 18 lines · Junio C Hamano
Quoted 6 lines · Jeff King
  # no tags, we just populate FETCH_HEAD because of the bare URL
  git fetch ../parent
  # this does fetch tags, because we're storing the result according to
  # the configured refspec ("refs/heads/*:refs/remotes/origin/*").
  git fetch origin
The above two look good.
Quoted 6 lines
  # this doesn't fetch tags, as the main command is "just" populating
  # FETCH_HEAD. But then our logic for "hey, we fetched the ref for
  # refs/remotes/origin/master, so let's update it on the side" kicks
  # in. And we end up updating FETCH_HEAD _and_ the tracking branch, but
  # not the tags. Weird.
  git fetch origin master
Quoted 4 lines
Yes, it looks weird, but I suspect that it is probably more correct not to fetch tags in this case. I wonder if it would be a solution not to do the "on the side" thing---after all the user didn't tell us to update refs/remotes/origin/master with this command line.

Quoted 13 lines

Isn't that how git fetch used to behave, or am I misunderstanding what you mean? It used to be that git fetch <remote> <branch> would not update any remote tracking branches.
From the 1.8.4 release notes:
Quoted 7 lines
"git fetch origin master" unlike "git fetch origin" or "git fetch" did not update "refs/remotes/origin/master"; this was an early design decision to keep the update of remote tracking branches predictable, but in practice it turns out that people find it more convenient to opportunistically update them whenever we have a chance, and we have been updating them when we run "git push" which already breaks the original "predictability" anyway.

No, you are not misunderstanding anything. The "pretend that we immediately turned around and fetched" done by "git push" was already breaking the predictability, but the change in 1.8.4 made it even worse. I am saying that going back to the old behaviour may be one option to address the issue being discussed in this thread.

Kevin Daudt

2017-08-17 20:44 UTC #9 lore raw

Quoted 40 lines · Junio C Hamano

Quoted 24 lines · Kevin Daudt
Quoted 18 lines · Junio C Hamano
Quoted 6 lines · Jeff King
  # no tags, we just populate FETCH_HEAD because of the bare URL
  git fetch ../parent
  # this does fetch tags, because we're storing the result according to
  # the configured refspec ("refs/heads/*:refs/remotes/origin/*").
  git fetch origin
The above two look good.
Quoted 6 lines
  # this doesn't fetch tags, as the main command is "just" populating
  # FETCH_HEAD. But then our logic for "hey, we fetched the ref for
  # refs/remotes/origin/master, so let's update it on the side" kicks
  # in. And we end up updating FETCH_HEAD _and_ the tracking branch, but
  # not the tags. Weird.
  git fetch origin master
Quoted 4 lines
Yes, it looks weird, but I suspect that it is probably more correct not to fetch tags in this case. I wonder if it would be a solution not to do the "on the side" thing---after all the user didn't tell us to update refs/remotes/origin/master with this command line.
Quoted 13 lines
Isn't that how git fetch used to behave, or am I misunderstanding what you mean? It used to be that git fetch <remote> <branch> would not update any remote tracking branches.
From the 1.8.4 release notes:
Quoted 7 lines
"git fetch origin master" unlike "git fetch origin" or "git fetch" did not update "refs/remotes/origin/master"; this was an early design decision to keep the update of remote tracking branches predictable, but in practice it turns out that people find it more convenient to opportunistically update them whenever we have a chance, and we have been updating them when we run "git push" which already breaks the original "predictability" anyway.

Quoted 6 lines

No, you are not misunderstanding anything. The "pretend that we immediately turned around and fetched" done by "git push" was already breaking the predictability, but the change in 1.8.4 made it even worse. I am saying that going back to the old behaviour may be one option to address the issue being discussed in this thread.

Ok. The reason I'm bring this up is because we often had to tell users in the irc channel that git fetch <remote> <branch> did not update the remote tracking branches, which confused them, so reverting back might reintroduce this confusion again.

Jeff King

2017-08-20 07:47 UTC #10 lore raw

Quoted 10 lines · Kevin Daudt

Quoted 5 lines
No, you are not misunderstanding anything. The "pretend that we immediately turned around and fetched" done by "git push" was already breaking the predictability, but the change in 1.8.4 made it even worse. I am saying that going back to the old behaviour may be one option to address the issue being discussed in this thread.
Ok. The reason I'm bring this up is because we often had to tell users in the irc channel that git fetch <remote> <branch> did not update the remote tracking branches, which confused them, so reverting back might reintroduce this confusion again.

Yeah, I don't think we want to go back to the original behavior. I agree that it is partially to blame for the inconsistency that started this thread, but I think on balance it has saved much more confusion than it has started. And we can address that inconsistency with better tag rules (like the "autofollow if we wrote any real refs" thing).

I don't have a patch for that yet, so if anybody feels like taking a look, it would be much appreciated.

-Peff

Jeff King

2017-08-20 07:50 UTC #11 lore raw

Quoted 8 lines · Jeff King

Yeah, I don't think we want to go back to the original behavior. I agree that it is partially to blame for the inconsistency that started this thread, but I think on balance it has saved much more confusion than it has started. And we can address that inconsistency with better tag rules (like the "autofollow if we wrote any real refs" thing).
I don't have a patch for that yet, so if anybody feels like taking a look, it would be much appreciated.

Also: I don't think we've seen a patch yet for documenting the current auto-follow behavior. Even if we don't make a behavior change, let's not forget to improve that, which should be much less work. :)

-Peff

Junio C Hamano

2017-08-20 15:52 UTC #12 lore raw

Quoted 8 lines · Jeff King

Quoted 6 lines · Jeff King
Yeah, I don't think we want to go back to the original behavior. I agree that it is partially to blame for the inconsistency that started this thread, but I think on balance it has saved much more confusion than it has started. And we can address that inconsistency with better tag rules (like the "autofollow if we wrote any real refs" thing).

Quoted 6 lines

Quoted 2 lines
I don't have a patch for that yet, so if anybody feels like taking a look, it would be much appreciated.
Also: I don't think we've seen a patch yet for documenting the current auto-follow behavior. Even if we don't make a behavior change, let's not forget to improve that, which should be much less work. :)

Thanks for two thoughtful follow-ups.