git remote prune removes local tags, depending on fetch config
git remote prune <name> should "delete all stale remote-tracking branches under <name>". I was surprised to discover, after some troubleshooting, that it also deletes *all* local tags that don't exist on the remote, if the following refspec is included in the remote's fetch config:
+refs/tags/*:refs/tags/*
So, if remote.origin.fetch is configured to fetch all tags from the remote, any tags I create locally will be deleted when running git remote prune origin. This is not intuitive [1], nor is is it explained in the docs [2]. Is this behavior obvious to someone with a better understanding of Git internals?
I did find a better way to automatically fetch tags (using tagopt instead of adding the fetch refspec). However, the refspec doesn't seem "wrong" in itself; in particular, git fetch --tags used to be considered equivalent to specifying the refspec "refs/tags/*:refs/tags/*" -- implying that this is a sensible refspec [3]. So I wouldn't expect it to "break" the behavior of another command.
[1] https://stackoverflow.com/q/34687657/1327867
[2] https://git-scm.com/docs/git-remote.html#git-remote-empruneem
[3] https://github.com/git/git/commit/c5a84e92a2fe9e8748e32341c344d7a6c0f52a50
On Mon, Jan 15 2018, Michael Giuffrida jotted:
Quoted 22 lines
git remote prune <name>should "delete all stale remote-tracking branches under <name>". I was surprised to discover, after some troubleshooting, that it also deletes *all* local tags that don't exist on the remote, if the following refspec is included in the remote's fetch config:+refs/tags/*:refs/tags/*So, if
remote.origin.fetchis configured to fetch all tags from the remote, any tags I create locally will be deleted when runninggit remote prune origin. This is not intuitive [1], nor is is it explained in the docs [2]. Is this behavior obvious to someone with a better understanding of Git internals?I did find a better way to automatically fetch tags (using tagopt instead of adding the fetch refspec). However, the refspec doesn't seem "wrong" in itself; in particular,
git fetch --tagsused to be considered equivalent to specifying the refspec "refs/tags/*:refs/tags/*" -- implying that this is a sensible refspec [3]. So I wouldn't expect it to "break" the behavior of another command.
Quoted 3 lines
[1] https://stackoverflow.com/q/34687657/1327867
[2] https://git-scm.com/docs/git-remote.html#git-remote-empruneem
[3] https://github.com/git/git/commit/c5a84e92a2fe9e8748e32341c344d7a6c0f52a50
These docs are really confusing, but it is working as intended, and really should be re-documented.
The git remote prune subcommand just ends up piggy-backing on git-fetch, whose behavior is explained here:
https://git-scm.com/docs/git-fetch.html#git-fetch---prune
It's worked this way since at least v1.8.5.6, maybe at some distant point in the past it only did this for branches when invoked via git-remote as the documentation says.
RELATED:
I've actually had the reverse problem with this. I want some way to turn this behavior on without explicitly hacking the refspec, so I can do it globally in /etc/gitconfig or in ~/.gitconfig without screwing with the config of each checkout on certain machines.
You can set fetch.prune=true, but that only prunes the branches, you need to inject remote.origin.fetch into each checkout, unless I've missed some way of doing this.
I wanted to add fetch.pruneTags that would make it as if you had refs/tags/*:refs/tags/* in the fetch spec, but I haven't hacked that up yet, if anyone can see any inherent issue with that plan I'd like to know about it.
Just to confirm, you're talking about the behavior of removing *all* tags that aren't found on the remote? (Not just tags that used to be on some remote, but have since been removed from that remote.) So, with your proposed workflow, you would never create your own tags locally, without pushing them to the remote before running git fetch -- otherwise they would simply be deleted.
It doesn't seem like a useful feature -- you wouldn't expect git fetch --prune to remove your local branches that you were developing on, right?
Quoted 55 lines · Ævar Arnfjörð Bjarmason
On Mon, Jan 15 2018, Michael Giuffrida jotted:
Quoted 22 lines
git remote prune <name>should "delete all stale remote-tracking branches under <name>". I was surprised to discover, after some troubleshooting, that it also deletes *all* local tags that don't exist on the remote, if the following refspec is included in the remote's fetch config:+refs/tags/*:refs/tags/*So, if
remote.origin.fetchis configured to fetch all tags from the remote, any tags I create locally will be deleted when runninggit remote prune origin. This is not intuitive [1], nor is is it explained in the docs [2]. Is this behavior obvious to someone with a better understanding of Git internals?I did find a better way to automatically fetch tags (using tagopt instead of adding the fetch refspec). However, the refspec doesn't seem "wrong" in itself; in particular,
git fetch --tagsused to be considered equivalent to specifying the refspec "refs/tags/*:refs/tags/*" -- implying that this is a sensible refspec [3]. So I wouldn't expect it to "break" the behavior of another command.Quoted 3 lines
[1] https://stackoverflow.com/q/34687657/1327867
[2] https://git-scm.com/docs/git-remote.html#git-remote-empruneem
[3] https://github.com/git/git/commit/c5a84e92a2fe9e8748e32341c344d7a6c0f52a50These docs are really confusing, but it is working as intended, and really should be re-documented.
The
git remote prunesubcommand just ends up piggy-backing on git-fetch, whose behavior is explained here:
https://git-scm.com/docs/git-fetch.html#git-fetch---pruneIt's worked this way since at least v1.8.5.6, maybe at some distant point in the past it only did this for branches when invoked via git-remote as the documentation says.
RELATED:
I've actually had the reverse problem with this. I want some way to turn this behavior on without explicitly hacking the refspec, so I can do it globally in
/etc/gitconfigor in ~/.gitconfig without screwing with the config of each checkout on certain machines.You can set fetch.prune=true, but that only prunes the branches, you need to inject remote.origin.fetch into each checkout, unless I've missed some way of doing this.
I wanted to add fetch.pruneTags that would make it as if you had refs/tags/*:refs/tags/* in the fetch spec, but I haven't hacked that up yet, if anyone can see any inherent issue with that plan I'd like to know about it.
On Tue, Jan 16 2018, Michael Giuffrida jotted:
Quoted 10 lines
Just to confirm, you're talking about the behavior of removing *all* tags that aren't found on the remote? (Not just tags that used to be on some remote, but have since been removed from that remote.) So, with your proposed workflow, you would never create your own tags locally, without pushing them to the remote before running
git fetch-- otherwise they would simply be deleted.It doesn't seem like a useful feature -- you wouldn't expect
git fetch --pruneto remove your local branches that you were developing on, right?
Yes, I realize this isn't everyone's cup of tea, but it's very useful in our in-house use-case for git.
We have repos that have tags pushed to them on every deployment, thus the cumulative set of tags is in the hundreds of thousands.
There's a pruning script that prunes both branches and tags from the server (and archives them to an archive repo that has every ref ever).
Then we need to push these deletions back to client checkouts. For branches that's easy, just set fetch.prune=true. For tags I need to chase down checkouts and do:
git -C {} config --replace-all remote.origin.fetch "refs/tags/*:refs/tags/*" "^refs/tags/*:refs/tags/*$"It would be easier to just set fetch.{prune,pruneTags}=true in /etc/gitconfig, this is on e.g. shared development staging boxes where you might have a manually cloned checkout in your ~.
Of course it'll wipe away any manually created tag you have, *ideally* we'd have something like the refs/remotes/ namespace for tags, but that ship has sailed.
But meanwhile tag pruning is an OK compromise, 99% of our users will never even think to create a tag (and if they created one manually they couldn't push it), the advanced users that would know what sort of repo they're editing and can out-out of the behavior by editing their local config.
Quoted 57 lines
Quoted 55 lines · Ævar Arnfjörð Bjarmason
On Mon, Jan 15 2018, Michael Giuffrida jotted:
Quoted 22 lines
git remote prune <name>should "delete all stale remote-tracking branches under <name>". I was surprised to discover, after some troubleshooting, that it also deletes *all* local tags that don't exist on the remote, if the following refspec is included in the remote's fetch config:+refs/tags/*:refs/tags/*So, if
remote.origin.fetchis configured to fetch all tags from the remote, any tags I create locally will be deleted when runninggit remote prune origin. This is not intuitive [1], nor is is it explained in the docs [2]. Is this behavior obvious to someone with a better understanding of Git internals?I did find a better way to automatically fetch tags (using tagopt instead of adding the fetch refspec). However, the refspec doesn't seem "wrong" in itself; in particular,
git fetch --tagsused to be considered equivalent to specifying the refspec "refs/tags/*:refs/tags/*" -- implying that this is a sensible refspec [3]. So I wouldn't expect it to "break" the behavior of another command.Quoted 3 lines
[1] https://stackoverflow.com/q/34687657/1327867
[2] https://git-scm.com/docs/git-remote.html#git-remote-empruneem
[3] https://github.com/git/git/commit/c5a84e92a2fe9e8748e32341c344d7a6c0f52a50These docs are really confusing, but it is working as intended, and really should be re-documented.
The
git remote prunesubcommand just ends up piggy-backing on git-fetch, whose behavior is explained here:
https://git-scm.com/docs/git-fetch.html#git-fetch---pruneIt's worked this way since at least v1.8.5.6, maybe at some distant point in the past it only did this for branches when invoked via git-remote as the documentation says.
RELATED:
I've actually had the reverse problem with this. I want some way to turn this behavior on without explicitly hacking the refspec, so I can do it globally in
/etc/gitconfigor in ~/.gitconfig without screwing with the config of each checkout on certain machines.You can set fetch.prune=true, but that only prunes the branches, you need to inject remote.origin.fetch into each checkout, unless I've missed some way of doing this.
I wanted to add fetch.pruneTags that would make it as if you had refs/tags/*:refs/tags/* in the fetch spec, but I haven't hacked that up yet, if anyone can see any inherent issue with that plan I'd like to know about it.
cover document & test fetch pruning + WIP fetch.pruneTags
Michael Giuffrida noted that the git-remote docs were very confusing, and upthread I said I wanted this shiny related thing in 11/11.
Along the way I fixed up fetch tests & documentation to hopefully be a lot less confusing.
I think 1-10/11 of this makes sense for inclusion as-is (pending review etc.), 11/11 is broken currently, but review / comments on it welcome, particularly the CLI / config interface / docs etc.
The bug causing it not to work is less of a "I can't figure this out" and more of a "I won't have time again for hacking in the next couple of days, and wanted to see what people thought", but if someone wants to see what I'm screwing up there and do my homework for me that's also most welcome.
Ævar Arnfjörð Bjarmason (11):
fetch tests: refactor in preparation for testing tag pruning fetch tests: arrange arguments for future readability fetch tests: add a tag to be deleted to the pruning tests fetch tests: double quote a variable for interpolation fetch tests: test --prune and refspec interaction
git fetch doc: add a new section to explain the ins & outs of pruning git remote doc: correct dangerous lies about what prune does git-fetch & config doc: link to the new PRUNING section
fetch: don't redundantly null something calloc() gave us fetch tests: add scaffolding for the new fetch.pruneTags WIP fetch: add a --fetch-prune option and fetch.pruneTags config
Documentation/config.txt | 21 ++++++- Documentation/fetch-options.txt | 18 +++++- Documentation/git-fetch.txt | 49 ++++++++++++++++ Documentation/git-remote.txt | 14 +++-- builtin/fetch.c | 38 ++++++++++-- remote.c | 2 + remote.h | 1 + t/t5510-fetch.sh | 125 +++++++++++++++++++++++++++------------- 8 files changed, 216 insertions(+), 52 deletions(-)
Signature · 1 line
2.15.1.424.g9478a66081
PATCH 1/11 fetch tests: refactor in preparation for testing tag pruning
In a subsequent commit this function will learn to test for tag pruning, prepare for that by making space for more variables, and making it clear that "expected" here refers to branches.
--- t/t5510-fetch.sh | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
t/t5510-fetch.sh +6 -3
@@ -549,9 +549,12 @@ set_config_tristate () { } test_configured_prune () { - fetch_prune=$1 remote_origin_prune=$2 cmdline=$3 expected=$4 + fetch_prune=$1 + remote_origin_prune=$2 + cmdline=$3 + expected_branch=$4 - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; $4" ' + test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && (
@@ -572,7 +575,7 @@ test_configured_prune () { set_config_tristate remote.origin.prune $remote_origin_prune && git fetch $cmdline && - case "$expected" in + case "$expected_branch" in pruned) test_must_fail git rev-parse --verify refs/remotes/origin/newbranch ;;
Signature · 1 line
2.15.1.424.g9478a66081
PATCH 2/11 fetch tests: arrange arguments for future readability
In reply to Ævar Arnfjörð BjarmasonRe-arrange the arguments to the test_configured_prune() function used in this test to pass the arguments to --fetch last. A subsequent change will test for more elaborate fetch arguments, including long refspecs. It'll be more readable to be able to wrap those on a new line of their own.
--- t/t5510-fetch.sh | 82 ++++++++++++++++++++++++++++++-------------------------- 1 file changed, 44 insertions(+), 38 deletions(-)
t/t5510-fetch.sh +44 -38
@@ -551,10 +551,10 @@ set_config_tristate () { test_configured_prune () { fetch_prune=$1 remote_origin_prune=$2 - cmdline=$3 - expected_branch=$4 + expected_branch=$3 + cmdline=$4 - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" ' + test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && (
@@ -587,41 +587,47 @@ test_configured_prune () { ' } -test_configured_prune unset unset "" kept -test_configured_prune unset unset "--no-prune" kept -test_configured_prune unset unset "--prune" pruned - -test_configured_prune false unset "" kept -test_configured_prune false unset "--no-prune" kept -test_configured_prune false unset "--prune" pruned - -test_configured_prune true unset "" pruned -test_configured_prune true unset "--prune" pruned -test_configured_prune true unset "--no-prune" kept - -test_configured_prune unset false "" kept -test_configured_prune unset false "--no-prune" kept -test_configured_prune unset false "--prune" pruned - -test_configured_prune false false "" kept -test_configured_prune false false "--no-prune" kept -test_configured_prune false false "--prune" pruned - -test_configured_prune true false "" kept -test_configured_prune true false "--prune" pruned -test_configured_prune true false "--no-prune" kept - -test_configured_prune unset true "" pruned -test_configured_prune unset true "--no-prune" kept -test_configured_prune unset true "--prune" pruned - -test_configured_prune false true "" pruned -test_configured_prune false true "--no-prune" kept -test_configured_prune false true "--prune" pruned - -test_configured_prune true true "" pruned -test_configured_prune true true "--prune" pruned -test_configured_prune true true "--no-prune" kept +# $1 config: fetch.prune +# $2 config: remote.<name>.prune +# $3 expect: branch to be pruned? +# $4 git-fetch $cmdline: +# +# $1 $2 $3 $4 +test_configured_prune unset unset kept "" +test_configured_prune unset unset kept "--no-prune" +test_configured_prune unset unset pruned "--prune" + +test_configured_prune false unset kept "" +test_configured_prune false unset kept "--no-prune" +test_configured_prune false unset pruned "--prune" + +test_configured_prune true unset pruned "" +test_configured_prune true unset pruned "--prune" +test_configured_prune true unset kept "--no-prune" + +test_configured_prune unset false kept "" +test_configured_prune unset false kept "--no-prune" +test_configured_prune unset false pruned "--prune" + +test_configured_prune false false kept "" +test_configured_prune false false kept "--no-prune" +test_configured_prune false false pruned "--prune" + +test_configured_prune true false kept "" +test_configured_prune true false pruned "--prune" +test_configured_prune true false kept "--no-prune" + +test_configured_prune unset true pruned "" +test_configured_prune unset true kept "--no-prune" +test_configured_prune unset true pruned "--prune" + +test_configured_prune false true pruned "" +test_configured_prune false true kept "--no-prune" +test_configured_prune false true pruned "--prune" + +test_configured_prune true true pruned "" +test_configured_prune true true pruned "--prune" +test_configured_prune true true kept "--no-prune" test_expect_success 'all boundary commits are excluded' ' test_commit base &&
Signature · 1 line
2.15.1.424.g9478a66081
PATCH 3/11 fetch tests: add a tag to be deleted to the pruning tests
In reply to Ævar Arnfjörð BjarmasonAdd a tag to be deleted to the fetch --prune tests. The tag is always kept for now, which is the expected behavior, but now I can add a test for tag pruning in a later commit.
--- t/t5510-fetch.sh | 93 ++++++++++++++++++++++++++++++++------------------------ 1 file changed, 53 insertions(+), 40 deletions(-)
t/t5510-fetch.sh +53 -40
@@ -552,21 +552,25 @@ test_configured_prune () { fetch_prune=$1 remote_origin_prune=$2 expected_branch=$3 - cmdline=$4 + expected_tag=$4 + cmdline=$5 - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" ' + test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && + git tag -f newtag && ( cd one && test_unconfig fetch.prune && test_unconfig remote.origin.prune && git fetch && - git rev-parse --verify refs/remotes/origin/newbranch + git rev-parse --verify refs/remotes/origin/newbranch && + git rev-parse --verify refs/tags/newtag ) && # now remove it git branch -d newbranch && + git tag -d newtag && # then test (
@@ -582,6 +586,14 @@ test_configured_prune () { kept) git rev-parse --verify refs/remotes/origin/newbranch ;; + esac && + case "$expected_tag" in + pruned) + test_must_fail git rev-parse --verify refs/tags/newtag + ;; + kept) + git rev-parse --verify refs/tags/newtag + ;; esac ) '
@@ -590,44 +602,45 @@ test_configured_prune () { # $1 config: fetch.prune # $2 config: remote.<name>.prune # $3 expect: branch to be pruned? -# $4 git-fetch $cmdline: +# $4 expect: tag to be pruned? +# $5 git-fetch $cmdline: # -# $1 $2 $3 $4 -test_configured_prune unset unset kept "" -test_configured_prune unset unset kept "--no-prune" -test_configured_prune unset unset pruned "--prune" - -test_configured_prune false unset kept "" -test_configured_prune false unset kept "--no-prune" -test_configured_prune false unset pruned "--prune" - -test_configured_prune true unset pruned "" -test_configured_prune true unset pruned "--prune" -test_configured_prune true unset kept "--no-prune" - -test_configured_prune unset false kept "" -test_configured_prune unset false kept "--no-prune" -test_configured_prune unset false pruned "--prune" - -test_configured_prune false false kept "" -test_configured_prune false false kept "--no-prune" -test_configured_prune false false pruned "--prune" - -test_configured_prune true false kept "" -test_configured_prune true false pruned "--prune" -test_configured_prune true false kept "--no-prune" - -test_configured_prune unset true pruned "" -test_configured_prune unset true kept "--no-prune" -test_configured_prune unset true pruned "--prune" - -test_configured_prune false true pruned "" -test_configured_prune false true kept "--no-prune" -test_configured_prune false true pruned "--prune" - -test_configured_prune true true pruned "" -test_configured_prune true true pruned "--prune" -test_configured_prune true true kept "--no-prune" +# $1 $2 $3 $4 $5 +test_configured_prune unset unset kept kept "" +test_configured_prune unset unset kept kept "--no-prune" +test_configured_prune unset unset pruned kept "--prune" + +test_configured_prune false unset kept kept "" +test_configured_prune false unset kept kept "--no-prune" +test_configured_prune false unset pruned kept "--prune" + +test_configured_prune true unset pruned kept "" +test_configured_prune true unset pruned kept "--prune" +test_configured_prune true unset kept kept "--no-prune" + +test_configured_prune unset false kept kept "" +test_configured_prune unset false kept kept "--no-prune" +test_configured_prune unset false pruned kept "--prune" + +test_configured_prune false false kept kept "" +test_configured_prune false false kept kept "--no-prune" +test_configured_prune false false pruned kept "--prune" + +test_configured_prune true false kept kept "" +test_configured_prune true false pruned kept "--prune" +test_configured_prune true false kept kept "--no-prune" + +test_configured_prune unset true pruned kept "" +test_configured_prune unset true kept kept "--no-prune" +test_configured_prune unset true pruned kept "--prune" + +test_configured_prune false true pruned kept "" +test_configured_prune false true kept kept "--no-prune" +test_configured_prune false true pruned kept "--prune" + +test_configured_prune true true pruned kept "" +test_configured_prune true true pruned kept "--prune" +test_configured_prune true true kept kept "--no-prune" test_expect_success 'all boundary commits are excluded' ' test_commit base &&
Signature · 1 line
2.15.1.424.g9478a66081
PATCH 4/11 fetch tests: double quote a variable for interpolation
In reply to Ævar Arnfjörð BjarmasonIf the $cmdline variable contains multiple arguments they won't be interpolated correctly since the body of the test is single quoted. I don't know what part of test-lib.sh is expanding variables within single-quoted strings, but interpolating this inline is the desired behavior here.
This will be used in a subsequent commit to pass more than one variable to git-fetch.
--- t/t5510-fetch.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
t/t5510-fetch.sh +1 -1
@@ -578,7 +578,7 @@ test_configured_prune () { set_config_tristate fetch.prune $fetch_prune && set_config_tristate remote.origin.prune $remote_origin_prune && - git fetch $cmdline && + git fetch '"$cmdline"' && case "$expected_branch" in pruned) test_must_fail git rev-parse --verify refs/remotes/origin/newbranch
Signature · 1 line
2.15.1.424.g9478a66081
PATCH 5/11 fetch tests: test --prune and refspec interaction
In reply to Ævar Arnfjörð BjarmasonAdd a test for the interaction between explicitly provided refspecs and fetch.prune.
There's no point in adding this boilerplate to every combination of unset/false/true, it's instructive and sufficient to show that no matter if the variable is unset, false or true the refspec on the command-line overrides any configuration variable.
--- t/t5510-fetch.sh | 12 ++++++++++++ 1 file changed, 12 insertions(+)
t/t5510-fetch.sh +12 -0
@@ -609,6 +609,10 @@ test_configured_prune () { test_configured_prune unset unset kept kept "" test_configured_prune unset unset kept kept "--no-prune" test_configured_prune unset unset pruned kept "--prune" +test_configured_prune unset unset kept pruned \ + "--prune origin 'refs/tags/*:refs/tags/*'" +test_configured_prune unset unset pruned pruned \ + "--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'" test_configured_prune false unset kept kept "" test_configured_prune false unset kept kept "--no-prune"
@@ -625,6 +629,10 @@ test_configured_prune unset false pruned kept "--prune" test_configured_prune false false kept kept "" test_configured_prune false false kept kept "--no-prune" test_configured_prune false false pruned kept "--prune" +test_configured_prune false false kept pruned \ + "--prune origin 'refs/tags/*:refs/tags/*'" +test_configured_prune false false pruned pruned \ + "--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'" test_configured_prune true false kept kept "" test_configured_prune true false pruned kept "--prune"
@@ -641,6 +649,10 @@ test_configured_prune false true pruned kept "--prune" test_configured_prune true true pruned kept "" test_configured_prune true true pruned kept "--prune" test_configured_prune true true kept kept "--no-prune" +test_configured_prune true true kept pruned \ + "--prune origin 'refs/tags/*:refs/tags/*'" +test_configured_prune true true pruned pruned \ + "--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'" test_expect_success 'all boundary commits are excluded' ' test_commit base &&
Signature · 1 line
2.15.1.424.g9478a66081
PATCH 6/11 git fetch doc: add a new section to explain the ins & outs of pruning
In reply to Ævar Arnfjörð BjarmasonAdd a new section to canonically explain how remote reference pruning works, and how users should be careful about using it in conjunction with tag refspecs in particular.
A subsequent commit will update the git-remote documentation to refer to this section, and details the motivation for writing this in the first place.
--- Documentation/git-fetch.txt | 49 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+)
Documentation/git-fetch.txt +49 -0
@@ -99,6 +99,55 @@ The latter use of the `remote.<repository>.fetch` values can be overridden by giving the `--refmap=<refspec>` parameter(s) on the command line. +PRUNING +------- + +Git has a default disposition to keeping data unless it's explicitly +thrown away, this extends to keeping a hold of local references to +branches on remotes that have themselves deleted those branches. + +If left to accumulate these stale references might make performance +worse on big and busy repos that have a lot of branch churn, and +e.g. make the output of commands like `git branch -a --contains +<commit>` needlessly verbose, as well as impacting anything else +that'll work with the complete set of known references. + +These remote tracking references can be deleted as a one-off with +either of: + +------------------------------------------------ +# While fetching +$ git fetch --prune <name> + +# Only prune, don't fetch +$ git remote <name> +------------------------------------------------ + +To prune references on a remote as part of your normal workflow +without needing to remember to run that set `fetch.prune` globally, or +`remote.<name>.prune` per-remote in the config. See +linkgit:git-config[1]. + +Here's where things get tricky and more specific. The pruning feature +doesn't actually care about branches, instead it'll prune local <-> +remote references as a function of the refspec of the remote (see +`<refspec>` and <<CRTB,CONFIGURED REMOTE-TRACKING BRANCHES>> above). + +Therefore if the refspec for the remote includes +e.g. `refs/tags/*:refs/tags/*`, or you manually run e.g. `git fetch +--prune <name> "refs/tags/*:refs/tags/*"` it won't be stale remote +tracking branches that are deleted, but any local tag that doesn't +exist on the remote. + +This might not be what you expect, i.e. you want to prune remote +`<name>`, but also explicitly fetch tags from it, so when you fetch +from it you delete all your local tags, most of which may not have +come from the `<name>` remote in the first place. + +So be careful when using this with a refspec like +`refs/tags/*:refs/tags/*`, or any other refspec which might map +references from multiple remotes to the same local namespace. + OUTPUT ------
Signature · 1 line
2.15.1.424.g9478a66081
Re: [PATCH 06/11] git fetch doc: add a new section to explain the ins & outs of pruning
Quoted 12 lines · Ævar Arnfjörð Bjarmason
Add a new section to canonically explain how remote reference pruning works, and how users should be careful about using it in conjunction with tag refspecs in particular.
A subsequent commit will update the git-remote documentation to refer to this section, and details the motivation for writing this in the first place.
Signed-off-by Ævar Arnfjörð Bjarmason <[email protected]>---
Documentation/git-fetch.txt The latter use of the `remote.<repository>.fetch` values can be · line 102
Quoted 4 lines
Documentation/git-fetch.txt +4
-0@@ -99,6 +99,55 @@ The latter use of the `remote.<repository>.fetch` values can be +PRUNING +------- + +Git has a default disposition to keeping data unless it's explicitly
s/keeping/keep/ or s/to keeping/of keeping/
Documentation/git-fetch.txt The latter use of the `remote.<repository>.fetch` values can be · line 106
Quoted 1 line
Documentation/git-fetch.txt +1
-0+thrown away, this extends to keeping a hold of local references to
s/away,/away;/
s/to keeping a hold of/holding onto/
Documentation/git-fetch.txt The latter use of the `remote.<repository>.fetch` values can be · line 109
Quoted 3 lines
Documentation/git-fetch.txt +3
-0+branches on remotes that have themselves deleted those branches. + +If left to accumulate these stale references might make performance
s/accumulate/&,/
Quoted 5 lines
Documentation/git-fetch.txt +5
-0+worse on big and busy repos that have a lot of branch churn, and +e.g. make the output of commands like `git branch -a --contains +<commit>` needlessly verbose, as well as impacting anything else +that'll work with the complete set of known references. +
Documentation/git-fetch.txt The latter use of the `remote.<repository>.fetch` values can be · line 124
Quoted 10 lines
Documentation/git-fetch.txt +10
-0+These remote tracking references can be deleted as a one-off with +either of: + +------------------------------------------------ +# While fetching +$ git fetch --prune <name> + +# Only prune, don't fetch +$ git remote <name> +------------------------------------------------
Did you mean "git remote prune <name>"?
Documentation/git-fetch.txt The latter use of the `remote.<repository>.fetch` values can be · line 126
Quoted 1 line
Documentation/git-fetch.txt +1
-0+To prune references on a remote as part of your normal workflow
This reads as if it's possible to prune something on the remote machine itself. Maybe just say:
To prune references as part of your normal workflow...
Quoted 3 lines
Documentation/git-fetch.txt +3
-0+without needing to remember to run that set `fetch.prune` globally, or +`remote.<name>.prune` per-remote in the config. See +linkgit:git-config[1].
PATCH 7/11 git remote doc: correct dangerous lies about what prune does
In reply to Ævar Arnfjörð BjarmasonThe "git remote prune <name>" command uses the same machinery as "git fetch <name> --prune", and shares all the same caveats, but its documentation has suggested that it'll just "delete stale remote-tracking branches under <name>".
This isn't true, and hasn't been true since at least v1.8.5.6 (the oldest version I could be bothered to test).
E.g. if "+refs/tags/*:refs/tags/*" is explicitly set in the refspec of the remote it'll delete all local tags <name> doesn't know about.
Instead, briefly give the reader just enough of a hint that this option might constitute a shotgun aimed at their foot, and point them to the new PRUNING section in the git-fetch documentation which explains all the nuances of what this facility does.
See "[BUG] git remote prune removes local tags, depending on fetch config" (CACi5S_39wNrbfjLfn0xhCY+uewtFN2YmnAcRc86z6pjUTjWPHQ@mail.gmail.com) by Michael Giuffrida for the initial report.
--- Documentation/git-remote.txt | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
Documentation/git-remote.txt +9 -5
@@ -172,10 +172,14 @@ With `-n` option, the remote heads are not queried first with 'prune':: -Deletes all stale remote-tracking branches under <name>. -These stale branches have already been removed from the remote repository -referenced by <name>, but are still locally available in -"remotes/<name>". +Deletes stale references associated with <name>. By default stale +remote-tracking branches under <name>, but depending on global +configuration and the configuration of the remote we might even prune +local tags. Equivalent to `git fetch <name> --prune`, except that no +new references will be fetched. ++ +See the PRUNING section of linkgit:git-fetch[1] for what it'll prune +depending on various configuration. + With `--dry-run` option, report what branches will be pruned, but do not actually prune them.
@@ -189,7 +193,7 @@ remotes.default is not defined, all remotes which do not have the configuration parameter remote.<name>.skipDefaultUpdate set to true will be updated. (See linkgit:git-config[1]). + -With `--prune` option, prune all the remotes that are updated. +With `--prune` option, run pruning against all the remotes that are updated. DISCUSSION
Signature · 1 line
2.15.1.424.g9478a66081
PATCH 8/11 git-fetch & config doc: link to the new PRUNING section
In reply to Ævar Arnfjörð BjarmasonAmend the documentation for fetch.prune, fetch.<name>.prune and --prune to link to the recently added PRUNING section.
I'd have liked to link directly to it with "<<PRUNING>>" from fetch-options.txt, since it's included in git-fetch.txt (git-pull.txt also includes it, but doesn't include that option). However making a reference across files yields this error:
[...]/Documentation/git-fetch.xml:226: element xref: validity error : IDREF attribute linkend references an unknown ID "PRUNING"
--- Documentation/config.txt | 6 +++++- Documentation/fetch-options.txt | 3 +++ 2 files changed, 8 insertions(+), 1 deletion(-)
Documentation/config.txt +5 -1
@@ -1398,7 +1398,8 @@ fetch.unpackLimit:: fetch.prune:: If true, fetch will automatically behave as if the `--prune` - option was given on the command line. See also `remote.<name>.prune`. + option was given on the command line. See also `remote.<name>.prune` + and the PRUNING section of linkgit:git-fetch[1]. fetch.output:: Control how ref update status is printed. Valid values are
@@ -2944,6 +2945,9 @@ remote.<name>.prune:: remove any remote-tracking references that no longer exist on the remote (as if the `--prune` option was given on the command line). Overrides `fetch.prune` settings, if any. ++ +See also `remote.<name>.prune` and the PRUNING section of +linkgit:git-fetch[1]. remotes.<group>:: The list of remotes which are fetched by "git remote update
Documentation/fetch-options.txt +3 -0
@@ -74,6 +74,9 @@ ifndef::git-pull[] line or in the remote configuration, for example if the remote was cloned with the --mirror option), then they are also subject to pruning. ++ +See the PRUNING section below for more details. + endif::git-pull[] ifndef::git-pull[]
Signature · 1 line
2.15.1.424.g9478a66081
PATCH 9/11 fetch: don't redundantly null something calloc() gave us
In reply to Ævar Arnfjörð BjarmasonStop redundantly NULL-ing the last element of the refs structure, which was retrieved via calloc() and is thus guaranteed to be pre-NULL'd.
This code dates back to b888d61c83 ("Make fetch a builtin", 2007-09-10), where wasn't any reason to do this back then either, it's just something left over from when git-fetch was initially introduced.
The initial motivation for this change was to make a subsequent change which'll also modify the refs variable smaller, since it won't have to copy this redundant "NULL the last + 1 item" pattern.
--- builtin/fetch.c | 1 - 1 file changed, 1 deletion(-)
builtin/fetch.c +0 -1
@@ -1302,7 +1302,6 @@ static int fetch_one(struct remote *remote, int argc, const char **argv) }else refs[j++] = argv[i]; } - refs[j] =NULL ; ref_nr = j; }
Signature · 1 line
2.15.1.424.g9478a66081
PATCH 10/11 fetch tests: add scaffolding for the new fetch.pruneTags
In reply to Ævar Arnfjörð BjarmasonThe fetch.pruneTags configuration doesn't exist yet, but will be added in a subsequent commit. Since testing for it requires adding new parameters to the test_configured_prune function it's easier to review this patch first to assert that no functional changes are introduced yet.
--- t/t5510-fetch.sh | 90 ++++++++++++++++++++++++++++++-------------------------- 1 file changed, 49 insertions(+), 41 deletions(-)
t/t5510-fetch.sh +49 -41
@@ -551,18 +551,22 @@ set_config_tristate () { test_configured_prune () { fetch_prune=$1 remote_origin_prune=$2 - expected_branch=$3 - expected_tag=$4 - cmdline=$5 + fetch_prune_tags=$3 + remote_origin_prune_tags=$4 + expected_branch=$5 + expected_tag=$6 + cmdline=$7 - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" ' + test_expect_success "prune fetch.prune=$1 fetch.pruneTags=$2 remote.origin.prune=$3 remote.origin.prune=$4${7:+ $7}; branch:$5 tag:$6" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && git tag -f newtag && ( cd one && test_unconfig fetch.prune && + test_unconfig fetch.pruneTags && test_unconfig remote.origin.prune && + test_unconfig remote.origin.pruneTags && git fetch && git rev-parse --verify refs/remotes/origin/newbranch && git rev-parse --verify refs/tags/newtag
@@ -576,7 +580,9 @@ test_configured_prune () { ( cd one && set_config_tristate fetch.prune $fetch_prune && + set_config_tristate fetch.pruneTags $fetch_prune_tags && set_config_tristate remote.origin.prune $remote_origin_prune && + set_config_tristate remote.origin.pruneTags $remote_origin_prune_tags && git fetch '"$cmdline"' && case "$expected_branch" in
@@ -601,57 +607,59 @@ test_configured_prune () { # $1 config: fetch.prune # $2 config: remote.<name>.prune -# $3 expect: branch to be pruned? -# $4 expect: tag to be pruned? -# $5 git-fetch $cmdline: +# $3 config: fetch.pruneTags +# $4 config: remote.<name>.pruneTags +# $5 expect: branch to be pruned? +# $6 expect: tag to be pruned? +# $7 git-fetch $cmdline: # -# $1 $2 $3 $4 $5 -test_configured_prune unset unset kept kept "" -test_configured_prune unset unset kept kept "--no-prune" -test_configured_prune unset unset pruned kept "--prune" -test_configured_prune unset unset kept pruned \ +# $1 $2 $3 $4 $5 $6 $7 +test_configured_prune unset unset unset unset kept kept "" +test_configured_prune unset unset unset unset kept kept "--no-prune" +test_configured_prune unset unset unset unset pruned kept "--prune" +test_configured_prune unset unset unset unset kept pruned \ "--prune origin 'refs/tags/*:refs/tags/*'" -test_configured_prune unset unset pruned pruned \ +test_configured_prune unset unset unset unset pruned pruned \ "--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'" -test_configured_prune false unset kept kept "" -test_configured_prune false unset kept kept "--no-prune" -test_configured_prune false unset pruned kept "--prune" +test_configured_prune false unset unset unset kept kept "" +test_configured_prune false unset unset unset kept kept "--no-prune" +test_configured_prune false unset unset unset pruned kept "--prune" -test_configured_prune true unset pruned kept "" -test_configured_prune true unset pruned kept "--prune" -test_configured_prune true unset kept kept "--no-prune" +test_configured_prune true unset unset unset pruned kept "" +test_configured_prune true unset unset unset pruned kept "--prune" +test_configured_prune true unset unset unset kept kept "--no-prune" -test_configured_prune unset false kept kept "" -test_configured_prune unset false kept kept "--no-prune" -test_configured_prune unset false pruned kept "--prune" +test_configured_prune unset false unset unset kept kept "" +test_configured_prune unset false unset unset kept kept "--no-prune" +test_configured_prune unset false unset unset pruned kept "--prune" -test_configured_prune false false kept kept "" -test_configured_prune false false kept kept "--no-prune" -test_configured_prune false false pruned kept "--prune" -test_configured_prune false false kept pruned \ +test_configured_prune false false unset unset kept kept "" +test_configured_prune false false unset unset kept kept "--no-prune" +test_configured_prune false false unset unset pruned kept "--prune" +test_configured_prune false false unset unset kept pruned \ "--prune origin 'refs/tags/*:refs/tags/*'" -test_configured_prune false false pruned pruned \ +test_configured_prune false false unset unset pruned pruned \ "--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'" -test_configured_prune true false kept kept "" -test_configured_prune true false pruned kept "--prune" -test_configured_prune true false kept kept "--no-prune" +test_configured_prune true false unset unset kept kept "" +test_configured_prune true false unset unset pruned kept "--prune" +test_configured_prune true false unset unset kept kept "--no-prune" -test_configured_prune unset true pruned kept "" -test_configured_prune unset true kept kept "--no-prune" -test_configured_prune unset true pruned kept "--prune" +test_configured_prune unset true unset unset pruned kept "" +test_configured_prune unset true unset unset kept kept "--no-prune" +test_configured_prune unset true unset unset pruned kept "--prune" -test_configured_prune false true pruned kept "" -test_configured_prune false true kept kept "--no-prune" -test_configured_prune false true pruned kept "--prune" +test_configured_prune false true unset unset pruned kept "" +test_configured_prune false true unset unset kept kept "--no-prune" +test_configured_prune false true unset unset pruned kept "--prune" -test_configured_prune true true pruned kept "" -test_configured_prune true true pruned kept "--prune" -test_configured_prune true true kept kept "--no-prune" -test_configured_prune true true kept pruned \ +test_configured_prune true true unset unset pruned kept "" +test_configured_prune true true unset unset pruned kept "--prune" +test_configured_prune true true unset unset kept kept "--no-prune" +test_configured_prune true true unset unset kept pruned \ "--prune origin 'refs/tags/*:refs/tags/*'" -test_configured_prune true true pruned pruned \ +test_configured_prune true true unset unset pruned pruned \ "--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'" test_expect_success 'all boundary commits are excluded' '
Signature · 1 line
2.15.1.424.g9478a66081
PATCH 11/11 WIP fetch: add a --fetch-prune option and fetch.pruneTags config
In reply to Ævar Arnfjörð Bjarmason[WIP: This doesn't (yet) work as advertised, see further WIP note below]
Add a --fetch-prune option to git-fetch along with fetch.pruneTags config option. This allows for doing:
git fetch origin -p -P
Or simply:
git config fetch.prune true &&
git config fetch.pruneTags true &&
git fetch originInstead of the much more verbose:
git fetch --prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'
A use-case of git which is a pain to support now is frequent pulling from a repo which is having both its branches *and* tags deleted regularly. At work we create deployment tags in the repo for each rollout, and there's *lots* of those, so they're archived within weeks for performance reasons.
Without this change it's hard to centrally configure such repos in /etc/gitconfig (on servers that are only used for working with them). You need to set fetch.prune=true globally, and then for each repo:
git -C {} config --replace-all remote.origin.fetch "refs/tags/*:refs/tags/*" "^refs/tags/*:refs/tags/*$"Now I can simply set fetch.pruneTags=true in /etc/gitconfig as well, and users running "git pull" will automatically get the pruning semantics we want.
See my "Re: [BUG] git remote prune removes local tags, depending on fetch config" ([email protected];
https://public-inbox.org/git/[email protected]/) for more details.
[WIP: This doesn't actually work, since the implementation is to just pretend that 'refs/tags/*:refs/tags/*' was supplied on the command-line, we'll consequently stop fetching any branches, since we don't use the default refspec found in .git/config anymore
I'm not familiar enough with this code and the refspec machinery to know if this argument injection is crazy, and how to tell it to *also* do whatever it's doing by default]
--- Documentation/config.txt | 15 +++++++++++++++ Documentation/fetch-options.txt | 15 ++++++++++++++- builtin/fetch.c | 37 ++++++++++++++++++++++++++++++++++--- remote.c | 2 ++ remote.h | 1 + t/t5510-fetch.sh | 1 + 6 files changed, 67 insertions(+), 4 deletions(-)
Documentation/config.txt +15 -0
@@ -1401,6 +1401,14 @@ fetch.prune:: option was given on the command line. See also `remote.<name>.prune` and the PRUNING section of linkgit:git-fetch[1]. +fetch.pruneTags:: + If true, fetch will automatically behave as if the + `refs/tags/*:refs/tags/*` refspec was provided when pruning, + if not set already. This allows for setting both this option + and `fetch.prune` to maintain a 1=1 mapping to upstrem + refs. See also `remote.<name>.pruneTags` and the PRUNING + section of linkgit:git-fetch[1]. + fetch.output:: Control how ref update status is printed. Valid values are `full` and `compact`. Default value is `full`. See section
@@ -2945,6 +2953,13 @@ remote.<name>.prune:: remove any remote-tracking references that no longer exist on the remote (as if the `--prune` option was given on the command line). Overrides `fetch.prune` settings, if any. + +remote.<name>.pruneTags:: + When set to true, fetching from this remote by default will also + remove any local tags that no longer exist on the remote (as + if the `--prune` option was given on the command line in + conjunction with the `refs/tags/*:refs/tags/*` refspec). + Overrides `fetch.prune` settings, if any. + See also `remote.<name>.prune` and the PRUNING section of linkgit:git-fetch[1].
Documentation/fetch-options.txt +14 -1
@@ -73,7 +73,20 @@ ifndef::git-pull[] are fetched due to an explicit refspec (either on the command line or in the remote configuration, for example if the remote was cloned with the --mirror option), then they are also - subject to pruning. + subject to pruning. Supplying `--prune-tags` is a shorthand for + providing the tag refspec. ++ +See the PRUNING section below for more details. + +-P:: +--prune-tags:: + Before fetching, remove any local tags that no longer exist on + the remote if `--prune` is enabled. This option should be used + more carefully, unlike `--prune` it will remove any local + references (local tags) that have been created. This option is + merely a shorthand for providing the explicit tag refspec + along with `--prune`, see the discussion about that in its + documentation. + See the PRUNING section below for more details.
builtin/fetch.c +34 -3
@@ -38,6 +38,10 @@ static int fetch_prune_config = -1; /* unspecified */static int prune =-1 ;/* unspecified */ #define PRUNE_BY_DEFAULT 0/* do we prune by default? */ +static int fetch_prune_tags_config =-1 ;/* unspecified */ +static int prune_tags =-1 ;/* unspecified */ +#define PRUNE_TAGS_BY_DEFAULT 0/* do we prune tags by default? */ +static int all, append, dry_run, force, keep, multiple, update_head_ok, verbosity, deepen_relative;static int progress =-1 ;static int tags =TAGS_DEFAULT , unshallow, update_shallow, deepen;
@@ -64,6 +68,11 @@ static int git_fetch_config(const char *k, const char *v, void *cb)return 0 ; } +if (!strcmp (k,"fetch.prunetags" )) { + fetch_prune_tags_config =git_config_bool (k, v); +return 0 ; + } +if (!strcmp (k,"submodule.recurse" )) {int r =git_config_bool (k, v) ?RECURSE_SUBMODULES_ON :RECURSE_SUBMODULES_OFF ;
@@ -126,6 +135,8 @@ static struct option builtin_fetch_options[] = { N_("number of submodules fetched in parallel")),OPT_BOOL ('p' ,"prune" , &prune,N_ ("prune remote-tracking branches no longer on remote" )), +OPT_BOOL ('P' ,"prune-tags" , &prune_tags, +N_ ("prune local tags not found on remote" )), { OPTION_CALLBACK, 0, "recurse-submodules", &recurse_submodules, N_("on-demand"),N_ ("control recursive fetching of submodules" ), PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules },
@@ -1212,6 +1223,8 @@ static void add_options_to_argv(struct argv_array *argv)argv_array_push (argv,"--dry-run" );if (prune !=-1 )argv_array_push (argv, prune ?"--prune" :"--no-prune" ); +if (prune_tags !=-1 ) +argv_array_push (argv, prune_tags ?"--prune-tags" :"--no-prune-tags" );if (update_head_ok)argv_array_push (argv,"--update-head-ok" );if (force)
@@ -1270,6 +1283,7 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)static const char **refs =NULL ;struct refspec *refspec;int ref_nr =0 ; +int j =0 ;int exit_code;if (!remote)
@@ -1288,10 +1302,27 @@ static int fetch_one(struct remote *remote, int argc, const char **argv) prune =PRUNE_BY_DEFAULT ; } +if (prune_tags <0 ) { +/* no command line request */ +if (0 <= gtransport->remote->prune_tags) + prune_tags = gtransport->remote->prune_tags; +else if (0 <= fetch_prune_tags_config) + prune_tags = fetch_prune_tags_config; +else + prune_tags =PRUNE_TAGS_BY_DEFAULT ; + } + +if (argc >0 || prune_tags) { +size_t nr_alloc =st_add3 (argc, prune_tags,1 ); + refs =xcalloc (nr_alloc,sizeof (const char *)); +if (prune_tags) { + refs[j++] =xstrfmt ("refs/tags/*:refs/tags/*" ); + ref_nr++; + } + } +if (argc >0 ) { -int j =0 ;int i; - refs =xcalloc (st_add (argc,1 ),sizeof (const char *));for (i =0 ; i < argc; i++) {if (!strcmp (argv[i],"tag" )) { i++;
@@ -1301,8 +1332,8 @@ static int fetch_one(struct remote *remote, int argc, const char **argv) argv[i], argv[i]); }else refs[j++] = argv[i]; + ref_nr++; } - ref_nr = j; }sigchain_push_common (unlock_pack_on_signal);
remote.c +2 -0
@@ -391,6 +391,8 @@ static int handle_config(const char *key, const char *value, void *cb) remote->skip_default_update =git_config_bool (key, value); else if (!strcmp(subkey, "prune")) remote->prune =git_config_bool (key, value); + else if (!strcmp(subkey, "prunetags")) + remote->prune_tags =git_config_bool (key, value); else if (!strcmp(subkey, "url")) {const char *v;if (git_config_string (&v, key, value))
remote.h +1 -0
@@ -47,6 +47,7 @@ struct remote {int skip_default_update;int mirror;int prune; +int prune_tags;const char *receivepack;const char *uploadpack;
t/t5510-fetch.sh +1 -0
@@ -617,6 +617,7 @@ test_configured_prune () { test_configured_prune unset unset unset unset kept kept "" test_configured_prune unset unset unset unset kept kept "--no-prune" test_configured_prune unset unset unset unset pruned kept "--prune" +test_configured_prune unset unset unset unset pruned pruned "--prune --prune-tags" # WIP: broken, see commit msg test_configured_prune unset unset unset unset kept pruned \ "--prune origin 'refs/tags/*:refs/tags/*'" test_configured_prune unset unset unset unset pruned pruned \
Signature · 1 line
2.15.1.424.g9478a66081
v2 cover document & test fetch pruning & add fetch.pruneTags
In reply to Ævar Arnfjörð BjarmasonNow v2 and fully non-RFC, changes:
Ævar Arnfjörð Bjarmason (12):
fetch tests: refactor in preparation for testing tag pruning fetch tests: arrange arguments for future readability fetch tests: add a tag to be deleted to the pruning tests fetch tests: double quote a variable for interpolation fetch tests: test --prune and refspec interaction
No changes.
git fetch doc: add a new section to explain the ins & outs of pruning
Grammar etc. fixes from Eric. Thanks!
git remote doc: correct dangerous lies about what prune does git-fetch & config doc: link to the new PRUNING section
No changes.
fetch: don't redundantly NULL something calloc() gave us
Minor rephrasing of the commit message.
fetch: stop accessing "remote" variable indirectly
NEW: Amends some existing confusing code, whose pattern will be used by 12/12.
fetch tests: add scaffolding for the new fetch.pruneTags
I screwed up positional arguments in the test description, fixed.
fetch: add a --fetch-prune option and fetch.pruneTags config
Now actually works, and with a very different implementation which involves making the previously private add_fetch_refspec() function in remote.c part of the API.
Documentation/config.txt | 20 +++++- Documentation/fetch-options.txt | 18 ++++- Documentation/git-fetch.txt | 73 +++++++++++++++++++ Documentation/git-remote.txt | 14 ++-- builtin/fetch.c | 37 +++++++++- remote.c | 5 +- remote.h | 3 + t/t5510-fetch.sh | 154 +++++++++++++++++++++++++++++----------- 8 files changed, 272 insertions(+), 52 deletions(-)
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 1/12 fetch tests: refactor in preparation for testing tag pruning
In a subsequent commit this function will learn to test for tag pruning, prepare for that by making space for more variables, and making it clear that "expected" here refers to branches.
--- t/t5510-fetch.sh | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
t/t5510-fetch.sh +6 -3
@@ -549,9 +549,12 @@ set_config_tristate () { } test_configured_prune () { - fetch_prune=$1 remote_origin_prune=$2 cmdline=$3 expected=$4 + fetch_prune=$1 + remote_origin_prune=$2 + cmdline=$3 + expected_branch=$4 - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; $4" ' + test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && (
@@ -572,7 +575,7 @@ test_configured_prune () { set_config_tristate remote.origin.prune $remote_origin_prune && git fetch $cmdline && - case "$expected" in + case "$expected_branch" in pruned) test_must_fail git rev-parse --verify refs/remotes/origin/newbranch ;;
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 2/12 fetch tests: arrange arguments for future readability
In reply to Ævar Arnfjörð BjarmasonRe-arrange the arguments to the test_configured_prune() function used in this test to pass the arguments to --fetch last. A subsequent change will test for more elaborate fetch arguments, including long refspecs. It'll be more readable to be able to wrap those on a new line of their own.
--- t/t5510-fetch.sh | 82 ++++++++++++++++++++++++++++++-------------------------- 1 file changed, 44 insertions(+), 38 deletions(-)
t/t5510-fetch.sh +44 -38
@@ -551,10 +551,10 @@ set_config_tristate () { test_configured_prune () { fetch_prune=$1 remote_origin_prune=$2 - cmdline=$3 - expected_branch=$4 + expected_branch=$3 + cmdline=$4 - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" ' + test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && (
@@ -587,41 +587,47 @@ test_configured_prune () { ' } -test_configured_prune unset unset "" kept -test_configured_prune unset unset "--no-prune" kept -test_configured_prune unset unset "--prune" pruned - -test_configured_prune false unset "" kept -test_configured_prune false unset "--no-prune" kept -test_configured_prune false unset "--prune" pruned - -test_configured_prune true unset "" pruned -test_configured_prune true unset "--prune" pruned -test_configured_prune true unset "--no-prune" kept - -test_configured_prune unset false "" kept -test_configured_prune unset false "--no-prune" kept -test_configured_prune unset false "--prune" pruned - -test_configured_prune false false "" kept -test_configured_prune false false "--no-prune" kept -test_configured_prune false false "--prune" pruned - -test_configured_prune true false "" kept -test_configured_prune true false "--prune" pruned -test_configured_prune true false "--no-prune" kept - -test_configured_prune unset true "" pruned -test_configured_prune unset true "--no-prune" kept -test_configured_prune unset true "--prune" pruned - -test_configured_prune false true "" pruned -test_configured_prune false true "--no-prune" kept -test_configured_prune false true "--prune" pruned - -test_configured_prune true true "" pruned -test_configured_prune true true "--prune" pruned -test_configured_prune true true "--no-prune" kept +# $1 config: fetch.prune +# $2 config: remote.<name>.prune +# $3 expect: branch to be pruned? +# $4 git-fetch $cmdline: +# +# $1 $2 $3 $4 +test_configured_prune unset unset kept "" +test_configured_prune unset unset kept "--no-prune" +test_configured_prune unset unset pruned "--prune" + +test_configured_prune false unset kept "" +test_configured_prune false unset kept "--no-prune" +test_configured_prune false unset pruned "--prune" + +test_configured_prune true unset pruned "" +test_configured_prune true unset pruned "--prune" +test_configured_prune true unset kept "--no-prune" + +test_configured_prune unset false kept "" +test_configured_prune unset false kept "--no-prune" +test_configured_prune unset false pruned "--prune" + +test_configured_prune false false kept "" +test_configured_prune false false kept "--no-prune" +test_configured_prune false false pruned "--prune" + +test_configured_prune true false kept "" +test_configured_prune true false pruned "--prune" +test_configured_prune true false kept "--no-prune" + +test_configured_prune unset true pruned "" +test_configured_prune unset true kept "--no-prune" +test_configured_prune unset true pruned "--prune" + +test_configured_prune false true pruned "" +test_configured_prune false true kept "--no-prune" +test_configured_prune false true pruned "--prune" + +test_configured_prune true true pruned "" +test_configured_prune true true pruned "--prune" +test_configured_prune true true kept "--no-prune" test_expect_success 'all boundary commits are excluded' ' test_commit base &&
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 3/12 fetch tests: add a tag to be deleted to the pruning tests
In reply to Ævar Arnfjörð BjarmasonAdd a tag to be deleted to the fetch --prune tests. The tag is always kept for now, which is the expected behavior, but now I can add a test for tag pruning in a later commit.
--- t/t5510-fetch.sh | 93 ++++++++++++++++++++++++++++++++------------------------ 1 file changed, 53 insertions(+), 40 deletions(-)
t/t5510-fetch.sh +53 -40
@@ -552,21 +552,25 @@ test_configured_prune () { fetch_prune=$1 remote_origin_prune=$2 expected_branch=$3 - cmdline=$4 + expected_tag=$4 + cmdline=$5 - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" ' + test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && + git tag -f newtag && ( cd one && test_unconfig fetch.prune && test_unconfig remote.origin.prune && git fetch && - git rev-parse --verify refs/remotes/origin/newbranch + git rev-parse --verify refs/remotes/origin/newbranch && + git rev-parse --verify refs/tags/newtag ) && # now remove it git branch -d newbranch && + git tag -d newtag && # then test (
@@ -582,6 +586,14 @@ test_configured_prune () { kept) git rev-parse --verify refs/remotes/origin/newbranch ;; + esac && + case "$expected_tag" in + pruned) + test_must_fail git rev-parse --verify refs/tags/newtag + ;; + kept) + git rev-parse --verify refs/tags/newtag + ;; esac ) '
@@ -590,44 +602,45 @@ test_configured_prune () { # $1 config: fetch.prune # $2 config: remote.<name>.prune # $3 expect: branch to be pruned? -# $4 git-fetch $cmdline: +# $4 expect: tag to be pruned? +# $5 git-fetch $cmdline: # -# $1 $2 $3 $4 -test_configured_prune unset unset kept "" -test_configured_prune unset unset kept "--no-prune" -test_configured_prune unset unset pruned "--prune" - -test_configured_prune false unset kept "" -test_configured_prune false unset kept "--no-prune" -test_configured_prune false unset pruned "--prune" - -test_configured_prune true unset pruned "" -test_configured_prune true unset pruned "--prune" -test_configured_prune true unset kept "--no-prune" - -test_configured_prune unset false kept "" -test_configured_prune unset false kept "--no-prune" -test_configured_prune unset false pruned "--prune" - -test_configured_prune false false kept "" -test_configured_prune false false kept "--no-prune" -test_configured_prune false false pruned "--prune" - -test_configured_prune true false kept "" -test_configured_prune true false pruned "--prune" -test_configured_prune true false kept "--no-prune" - -test_configured_prune unset true pruned "" -test_configured_prune unset true kept "--no-prune" -test_configured_prune unset true pruned "--prune" - -test_configured_prune false true pruned "" -test_configured_prune false true kept "--no-prune" -test_configured_prune false true pruned "--prune" - -test_configured_prune true true pruned "" -test_configured_prune true true pruned "--prune" -test_configured_prune true true kept "--no-prune" +# $1 $2 $3 $4 $5 +test_configured_prune unset unset kept kept "" +test_configured_prune unset unset kept kept "--no-prune" +test_configured_prune unset unset pruned kept "--prune" + +test_configured_prune false unset kept kept "" +test_configured_prune false unset kept kept "--no-prune" +test_configured_prune false unset pruned kept "--prune" + +test_configured_prune true unset pruned kept "" +test_configured_prune true unset pruned kept "--prune" +test_configured_prune true unset kept kept "--no-prune" + +test_configured_prune unset false kept kept "" +test_configured_prune unset false kept kept "--no-prune" +test_configured_prune unset false pruned kept "--prune" + +test_configured_prune false false kept kept "" +test_configured_prune false false kept kept "--no-prune" +test_configured_prune false false pruned kept "--prune" + +test_configured_prune true false kept kept "" +test_configured_prune true false pruned kept "--prune" +test_configured_prune true false kept kept "--no-prune" + +test_configured_prune unset true pruned kept "" +test_configured_prune unset true kept kept "--no-prune" +test_configured_prune unset true pruned kept "--prune" + +test_configured_prune false true pruned kept "" +test_configured_prune false true kept kept "--no-prune" +test_configured_prune false true pruned kept "--prune" + +test_configured_prune true true pruned kept "" +test_configured_prune true true pruned kept "--prune" +test_configured_prune true true kept kept "--no-prune" test_expect_success 'all boundary commits are excluded' ' test_commit base &&
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 4/12 fetch tests: double quote a variable for interpolation
In reply to Ævar Arnfjörð BjarmasonIf the $cmdline variable contains multiple arguments they won't be interpolated correctly since the body of the test is single quoted. I don't know what part of test-lib.sh is expanding variables within single-quoted strings, but interpolating this inline is the desired behavior here.
This will be used in a subsequent commit to pass more than one variable to git-fetch.
--- t/t5510-fetch.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
t/t5510-fetch.sh +1 -1
@@ -578,7 +578,7 @@ test_configured_prune () { set_config_tristate fetch.prune $fetch_prune && set_config_tristate remote.origin.prune $remote_origin_prune && - git fetch $cmdline && + git fetch '"$cmdline"' && case "$expected_branch" in pruned) test_must_fail git rev-parse --verify refs/remotes/origin/newbranch
Signature · 1 line
2.15.1.424.g9478a66081
Re: [PATCH v2 04/12] fetch tests: double quote a variable for interpolation
Quoted 4 lines · Ævar Arnfjörð Bjarmason
If the $cmdline variable contains multiple arguments they won't be interpolated correctly since the body of the test is single quoted. I don't know what part of
test-lib.shis expanding variables within single-quoted strings,...
dothis='echo whatever $IFS separated strings'
test_expect_success label '
$dothis
'
works because test_expect_success ends up beint a glorified 'eval' and it sees the value of $dothis.
Quoted 2 lines
but interpolating this inline is the desired
behavior here.
I am not sure what you meant by this, though.
t/t5510-fetch.sh test_configured_prune () { · line 581
Quoted 2 lines
t/t5510-fetch.sh +1
-1- git fetch $cmdline && + git fetch '"$cmdline"' &&
Would this work with cmdline that needs to be quoted for the resulting shell script to be syntactically correct (e.g. cmdline with a single dq in it)? By stepping out of sq pair, you are allowing/asking the shell that forms test_expect_success command line arguments to interpolate cmdline, instead of asking the shell that evals test_expect_success with its command line argument strings.
In other words, I suspect that the caller of test_configured_prune now must sq_quote the cmdline arguments it passes to this helper, i.e.
cmdline="$(git rev-parse --sq-quote arg1 'arg"2' arg3)" test_configured_prune ... "$cmdline" ...
for this patch to be correct.
Re: [PATCH v2 04/12] fetch tests: double quote a variable for interpolation
On Mon, Jan 22 2018, Junio C. Hamano jotted:
Quoted 12 lines
Quoted 4 lines · Ævar Arnfjörð Bjarmason
If the $cmdline variable contains multiple arguments they won't be interpolated correctly since the body of the test is single quoted. I don't know what part of
test-lib.shis expanding variables within single-quoted strings,...dothis='echo whatever $IFS separated strings'
test_expect_successlabel '
$dothis
'
Quoted 7 lines
works because
test_expect_successends up beint a glorified 'eval' and it sees the value of $dothis.Quoted 2 lines
but interpolating this inline is the desired
behavior here.I am not sure what you meant by this, though.
Looking into this again:
myvar="a b 'c' \"d\" \"\\\"e\\\"\""
test_expect_success 'blah' '
/usr/bin/perl -MData::Dumper -wE say\ Dumper\ \\@ARGV -- $myvar &&
/usr/bin/perl -MData::Dumper -wE say\ Dumper\ \\@ARGV -- '"$myvar"' &&
echo $myvar &&
false
'Produces:
Initialized empty Git repository in /home/avar/g/git/t/trash directory.t5510-fetch/.git/
expecting success:
/usr/bin/perl -MData::Dumper -wE say\ Dumper\ \\@ARGV -- $myvar &&
/usr/bin/perl -MData::Dumper -wE say\ Dumper\ \\@ARGV -- a b 'c' "d" "\"e\"" &&
echo $myvar &&
false
+ /usr/bin/perl -MData::Dumper -wE say Dumper \@ARGV -- a b 'c' "d" "\"e\""
$VAR1 = [
'a',
'b',
'\'c\'',
'"d"',
'"\\"e\\""'
];
+ /usr/bin/perl -MData::Dumper -wE say Dumper \@ARGV -- a b c d "e"
$VAR1 = [
'a',
'b',
'c',
'd',
'"e"'
];
+ echo a b 'c' "d" "\"e\""
a b 'c' "d" "\"e\""
+ false
error: last command exited with $?=1
not ok 1 - blah
#
# /usr/bin/perl -MData::Dumper -wE say\ Dumper\ \\@ARGV -- $myvar &&
# /usr/bin/perl -MData::Dumper -wE say\ Dumper\ \\@ARGV -- a b 'c' "d" "\"e\"" &&
# echo $myvar &&
# false
#I.e. the desired effect is to get variables like the refspec like/this not 'like/this'. I could also just apply this on top, which gives the same end result, but now I wonder if starting some args with e.g. unescaped + and with : and * in the string is portable:
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 48f49b613a..2d311059e9 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -587 +587 @@ test_configured_prune () {
- git fetch '"$cmdline"' &&
+ git fetch $cmdline &&
@@ -621 +621 @@ test_configured_prune unset unset unset unset kept pruned \
- "--prune origin 'refs/tags/*:refs/tags/*'"
+ "--prune origin refs/tags/*:refs/tags/*"
@@ -623 +623 @@ test_configured_prune unset unset unset unset pruned pruned \
- "--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
+ "--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
@@ -641 +641 @@ test_configured_prune false false unset unset kept pruned \
- "--prune origin 'refs/tags/*:refs/tags/*'"
+ "--prune origin refs/tags/*:refs/tags/*"
@@ -643 +643 @@ test_configured_prune false false unset unset pruned pruned \
- "--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
+ "--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
@@ -661 +661 @@ test_configured_prune true true unset unset kept pruned \
- "--prune origin 'refs/tags/*:refs/tags/*'"
+ "--prune origin refs/tags/*:refs/tags/*"
@@ -663 +663 @@ test_configured_prune true true unset unset pruned pruned \
- "--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'"
+ "--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*"
@@ -684 +684 @@ test_configured_prune true false true false kept kept ""
-# When --prune-tags is supplied it's ignored if an explict refspec is
+# When --prune-tags is supplied its ignored if an explict refspec is
@@ -687 +687 @@ test_configured_prune unset unset unset unset pruned kept \
- "--prune --prune-tags origin '+refs/heads/*:refs/remotes/origin/*'"
+ "--prune --prune-tags origin +refs/heads/*:refs/remotes/origin/*"
@@ -689 +689 @@ test_configured_prune unset unset true unset pruned kept \
- "--prune --prune-tags origin '+refs/heads/*:refs/remotes/origin/*'"
+ "--prune --prune-tags origin +refs/heads/*:refs/remotes/origin/*"
@@ -691 +691 @@ test_configured_prune unset unset unset true pruned kept \
- "--prune --prune-tags origin '+refs/heads/*:refs/remotes/origin/*'"
+ "--prune --prune-tags origin +refs/heads/*:refs/remotes/origin/*"Quoted 11 lines
t/t5510-fetch.sh test_configured_prune () { · line 581
Quoted 2 lines
t/t5510-fetch.sh +1
-1- git fetch $cmdline && + git fetch '"$cmdline"' &&Would this work with cmdline that needs to be quoted for the resulting shell script to be syntactically correct (e.g. cmdline with a single dq in it)? By stepping out of sq pair, you are allowing/asking the shell that forms
test_expect_successcommand line arguments to interpolate cmdline, instead of asking the shell that evalstest_expect_successwith its command line argument strings.
Quoted 8 lines
In other words, I suspect that the caller of
test_configured_prunenow mustsq_quotethe cmdline arguments it passes to this helper, i.e.cmdline="$(git rev-parse --sq-quote arg1 'arg"2' arg3)" test_configured_prune ... "$cmdline" ...for this patch to be correct.
Sorry at this point I'm confused about this whole thing. This doesn't work and overquotes:
@@ -559,2 +569,3 @@ test_configured_prune () {
+ cmdline_q="$(git rev-parse --sq-quote $cmdline)"
test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2 fetch.pruneTags=$3 remote.origin.pruneTags=$4${7:+ $7}; branch:$5 tag:$6" '
@@ -586,3 +597,4 @@ test_configured_prune () {
- git fetch '"$cmdline"' &&
+ /usr/bin/perl -MData::Dumper -wE say\ Dumper\ \\@ARGV -- '"$cmdline_q"' &&
+ git fetch '"$cmdline_q"' &&
case "$expected_branch" inSo does just using $cmdline_q without making the calling shell interpolate it. Is there an idiom I should be following that I'm missing here?
PATCH v2 5/12 fetch tests: test --prune and refspec interaction
In reply to Ævar Arnfjörð BjarmasonAdd a test for the interaction between explicitly provided refspecs and fetch.prune.
There's no point in adding this boilerplate to every combination of unset/false/true, it's instructive and sufficient to show that no matter if the variable is unset, false or true the refspec on the command-line overrides any configuration variable.
--- t/t5510-fetch.sh | 12 ++++++++++++ 1 file changed, 12 insertions(+)
t/t5510-fetch.sh +12 -0
@@ -609,6 +609,10 @@ test_configured_prune () { test_configured_prune unset unset kept kept "" test_configured_prune unset unset kept kept "--no-prune" test_configured_prune unset unset pruned kept "--prune" +test_configured_prune unset unset kept pruned \ + "--prune origin 'refs/tags/*:refs/tags/*'" +test_configured_prune unset unset pruned pruned \ + "--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'" test_configured_prune false unset kept kept "" test_configured_prune false unset kept kept "--no-prune"
@@ -625,6 +629,10 @@ test_configured_prune unset false pruned kept "--prune" test_configured_prune false false kept kept "" test_configured_prune false false kept kept "--no-prune" test_configured_prune false false pruned kept "--prune" +test_configured_prune false false kept pruned \ + "--prune origin 'refs/tags/*:refs/tags/*'" +test_configured_prune false false pruned pruned \ + "--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'" test_configured_prune true false kept kept "" test_configured_prune true false pruned kept "--prune"
@@ -641,6 +649,10 @@ test_configured_prune false true pruned kept "--prune" test_configured_prune true true pruned kept "" test_configured_prune true true pruned kept "--prune" test_configured_prune true true kept kept "--no-prune" +test_configured_prune true true kept pruned \ + "--prune origin 'refs/tags/*:refs/tags/*'" +test_configured_prune true true pruned pruned \ + "--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'" test_expect_success 'all boundary commits are excluded' ' test_commit base &&
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 6/12 git fetch doc: add a new section to explain the ins & outs of pruning
In reply to Ævar Arnfjörð BjarmasonAdd a new section to canonically explain how remote reference pruning works, and how users should be careful about using it in conjunction with tag refspecs in particular.
A subsequent commit will update the git-remote documentation to refer to this section, and details the motivation for writing this in the first place.
--- Documentation/git-fetch.txt | 49 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+)
Documentation/git-fetch.txt +49 -0
@@ -99,6 +99,55 @@ The latter use of the `remote.<repository>.fetch` values can be overridden by giving the `--refmap=<refspec>` parameter(s) on the command line. +PRUNING +------- + +Git has a default disposition of keeping data unless it's explicitly +thrown away; this extends to holding onto local references to branches +on remotes that have themselves deleted those branches. + +If left to accumulate, these stale references might make performance +worse on big and busy repos that have a lot of branch churn, and +e.g. make the output of commands like `git branch -a --contains +<commit>` needlessly verbose, as well as impacting anything else +that'll work with the complete set of known references. + +These remote tracking references can be deleted as a one-off with +either of: + +------------------------------------------------ +# While fetching +$ git fetch --prune <name> + +# Only prune, don't fetch +$ git remote prune <name> +------------------------------------------------ + +To prune references as part of your normal workflow without needing to +remember to run that set `fetch.prune` globally, or +`remote.<name>.prune` per-remote in the config. See +linkgit:git-config[1]. + +Here's where things get tricky and more specific. The pruning feature +doesn't actually care about branches, instead it'll prune local <-> +remote references as a function of the refspec of the remote (see +`<refspec>` and <<CRTB,CONFIGURED REMOTE-TRACKING BRANCHES>> above). + +Therefore if the refspec for the remote includes +e.g. `refs/tags/*:refs/tags/*`, or you manually run e.g. `git fetch +--prune <name> "refs/tags/*:refs/tags/*"` it won't be stale remote +tracking branches that are deleted, but any local tag that doesn't +exist on the remote. + +This might not be what you expect, i.e. you want to prune remote +`<name>`, but also explicitly fetch tags from it, so when you fetch +from it you delete all your local tags, most of which may not have +come from the `<name>` remote in the first place. + +So be careful when using this with a refspec like +`refs/tags/*:refs/tags/*`, or any other refspec which might map +references from multiple remotes to the same local namespace. + OUTPUT ------
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 7/12 git remote doc: correct dangerous lies about what prune does
In reply to Ævar Arnfjörð BjarmasonThe "git remote prune <name>" command uses the same machinery as "git fetch <name> --prune", and shares all the same caveats, but its documentation has suggested that it'll just "delete stale remote-tracking branches under <name>".
This isn't true, and hasn't been true since at least v1.8.5.6 (the oldest version I could be bothered to test).
E.g. if "+refs/tags/*:refs/tags/*" is explicitly set in the refspec of the remote it'll delete all local tags <name> doesn't know about.
Instead, briefly give the reader just enough of a hint that this option might constitute a shotgun aimed at their foot, and point them to the new PRUNING section in the git-fetch documentation which explains all the nuances of what this facility does.
See "[BUG] git remote prune removes local tags, depending on fetch config" (CACi5S_39wNrbfjLfn0xhCY+uewtFN2YmnAcRc86z6pjUTjWPHQ@mail.gmail.com) by Michael Giuffrida for the initial report.
--- Documentation/git-remote.txt | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
Documentation/git-remote.txt +9 -5
@@ -172,10 +172,14 @@ With `-n` option, the remote heads are not queried first with 'prune':: -Deletes all stale remote-tracking branches under <name>. -These stale branches have already been removed from the remote repository -referenced by <name>, but are still locally available in -"remotes/<name>". +Deletes stale references associated with <name>. By default stale +remote-tracking branches under <name>, but depending on global +configuration and the configuration of the remote we might even prune +local tags. Equivalent to `git fetch <name> --prune`, except that no +new references will be fetched. ++ +See the PRUNING section of linkgit:git-fetch[1] for what it'll prune +depending on various configuration. + With `--dry-run` option, report what branches will be pruned, but do not actually prune them.
@@ -189,7 +193,7 @@ remotes.default is not defined, all remotes which do not have the configuration parameter remote.<name>.skipDefaultUpdate set to true will be updated. (See linkgit:git-config[1]). + -With `--prune` option, prune all the remotes that are updated. +With `--prune` option, run pruning against all the remotes that are updated. DISCUSSION
Signature · 1 line
2.15.1.424.g9478a66081
Re: [PATCH v2 07/12] git remote doc: correct dangerous lies about what prune does
Documentation/git-remote.txt With `-n` option, the remote heads are not queried first with
Quoted 4 lines · Ævar Arnfjörð Bjarmason
Documentation/git-remote.txt +4
-0+Deletes stale references associated with <name>. By default stale +remote-tracking branches under <name>, but depending on global +configuration and the configuration of the remote we might even prune +local tags....
An optional clarification
s/prune local tags/& that haven't been pushed there/
Quoted 2 lines
+... Equivalent to
git fetch <name> --prune, except that no +new references will be fetched.
git fetch --prune <name> (dashed options first and then non options), just like you wrote in the previous step.
Re: [PATCH v2 07/12] git remote doc: correct dangerous lies about what prune does
On Mon, Jan 22 2018, Junio C. Hamano jotted:
Quoted 9 lines
Documentation/git-remote.txt With `-n` option, the remote heads are not queried first with
Quoted 4 lines · Ævar Arnfjörð Bjarmason
Documentation/git-remote.txt +4
-0+Deletes stale references associated with <name>. By default stale +remote-tracking branches under <name>, but depending on global +configuration and the configuration of the remote we might even prune +local tags....An optional clarification
Quoted 7 lines
s/prune local tags/& that haven't been pushed there/
Quoted 2 lines
+... Equivalent to
git fetch <name> --prune, except that no +new references will be fetched.
git fetch --prune <name>(dashed options first and then non options), just like you wrote in the previous step.
Thanks. Will fix both.
PATCH v2 8/12 git-fetch & config doc: link to the new PRUNING section
In reply to Ævar Arnfjörð BjarmasonAmend the documentation for fetch.prune, fetch.<name>.prune and --prune to link to the recently added PRUNING section.
I'd have liked to link directly to it with "<<PRUNING>>" from fetch-options.txt, since it's included in git-fetch.txt (git-pull.txt also includes it, but doesn't include that option). However making a reference across files yields this error:
[...]/Documentation/git-fetch.xml:226: element xref: validity error : IDREF attribute linkend references an unknown ID "PRUNING"
--- Documentation/config.txt | 6 +++++- Documentation/fetch-options.txt | 3 +++ 2 files changed, 8 insertions(+), 1 deletion(-)
Documentation/config.txt +5 -1
@@ -1398,7 +1398,8 @@ fetch.unpackLimit:: fetch.prune:: If true, fetch will automatically behave as if the `--prune` - option was given on the command line. See also `remote.<name>.prune`. + option was given on the command line. See also `remote.<name>.prune` + and the PRUNING section of linkgit:git-fetch[1]. fetch.output:: Control how ref update status is printed. Valid values are
@@ -2944,6 +2945,9 @@ remote.<name>.prune:: remove any remote-tracking references that no longer exist on the remote (as if the `--prune` option was given on the command line). Overrides `fetch.prune` settings, if any. ++ +See also `remote.<name>.prune` and the PRUNING section of +linkgit:git-fetch[1]. remotes.<group>:: The list of remotes which are fetched by "git remote update
Documentation/fetch-options.txt +3 -0
@@ -74,6 +74,9 @@ ifndef::git-pull[] line or in the remote configuration, for example if the remote was cloned with the --mirror option), then they are also subject to pruning. ++ +See the PRUNING section below for more details. + endif::git-pull[] ifndef::git-pull[]
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 9/12 fetch: don't redundantly NULL something calloc() gave us
In reply to Ævar Arnfjörð BjarmasonStop redundantly NULL-ing the last element of the refs structure, which was retrieved via calloc(), and is thus guaranteed to be pre-NULL'd.
This code dates back to b888d61c83 ("Make fetch a builtin", 2007-09-10), where wasn't any reason to do this back then either, it's just something left over from when git-fetch was initially introduced.
The initial motivation for this change was to make a subsequent change which'll also modify the refs variable smaller, since it won't have to copy this redundant "NULL the last + 1 item" pattern.
--- builtin/fetch.c | 1 - 1 file changed, 1 deletion(-)
builtin/fetch.c +0 -1
@@ -1302,7 +1302,6 @@ static int fetch_one(struct remote *remote, int argc, const char **argv) }else refs[j++] = argv[i]; } - refs[j] =NULL ; ref_nr = j; }
Signature · 1 line
2.15.1.424.g9478a66081
Re: [PATCH v2 09/12] fetch: don't redundantly NULL something calloc() gave us
On Sun, Jan 21 2018, Ævar Arnfjörð Bjarmason jotted:
Quoted 1 line
2007-09-10), where wasn't any reason to do this back then either, it's
s/where wasn't/where there wasn't/
Quoted 3 lines
The initial motivation for this change was to make a subsequent change which'll also modify the refs variable smaller, since it won't have to copy this redundant "NULL the last + 1 item" pattern.
Which, FWIW is not happening later in this series anymore, so this is just a "cleanup while I'm at it" patch at this point. Will note that in a v3 commit message pending more comments.
PATCH v2 10/12 fetch: stop accessing "remote" variable indirectly
In reply to Ævar Arnfjörð BjarmasonAccess the "remote" variable passed to the fetch_one() directly rather than through the gtransport wrapper struct constructed in this function for other purposes.
This makes the code more readable, as it's now obvious that the remote struct doesn't somehow get munged by the prepare_transport() function above, which takes the "remote" struct as an argument and constructs the "gtransport" struct, containing among other things the "remote" struct.
This pattern of accessing the container struct was added in 737c5a9cde ("fetch: make --prune configurable", 2013-07-13) when this code was initially introduced.
--- builtin/fetch.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
builtin/fetch.c +2 -2
@@ -1280,8 +1280,8 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)if (prune <0 ) {/* no command line request */ -if (0 <= gtransport->remote->prune) - prune = gtransport->remote->prune; +if (0 <= remote->prune) + prune = remote->prune;else if (0 <= fetch_prune_config) prune = fetch_prune_config;else
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 11/12 fetch tests: add scaffolding for the new fetch.pruneTags
In reply to Ævar Arnfjörð BjarmasonThe fetch.pruneTags configuration doesn't exist yet, but will be added in a subsequent commit. Since testing for it requires adding new parameters to the test_configured_prune function it's easier to review this patch first to assert that no functional changes are introduced yet.
--- t/t5510-fetch.sh | 90 ++++++++++++++++++++++++++++++-------------------------- 1 file changed, 49 insertions(+), 41 deletions(-)
t/t5510-fetch.sh +49 -41
@@ -551,18 +551,22 @@ set_config_tristate () { test_configured_prune () { fetch_prune=$1 remote_origin_prune=$2 - expected_branch=$3 - expected_tag=$4 - cmdline=$5 + fetch_prune_tags=$3 + remote_origin_prune_tags=$4 + expected_branch=$5 + expected_tag=$6 + cmdline=$7 - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" ' + test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2 fetch.pruneTags=$3 remote.origin.pruneTags=$4${7:+ $7}; branch:$5 tag:$6" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && git tag -f newtag && ( cd one && test_unconfig fetch.prune && + test_unconfig fetch.pruneTags && test_unconfig remote.origin.prune && + test_unconfig remote.origin.pruneTags && git fetch && git rev-parse --verify refs/remotes/origin/newbranch && git rev-parse --verify refs/tags/newtag
@@ -576,7 +580,9 @@ test_configured_prune () { ( cd one && set_config_tristate fetch.prune $fetch_prune && + set_config_tristate fetch.pruneTags $fetch_prune_tags && set_config_tristate remote.origin.prune $remote_origin_prune && + set_config_tristate remote.origin.pruneTags $remote_origin_prune_tags && git fetch '"$cmdline"' && case "$expected_branch" in
@@ -601,57 +607,59 @@ test_configured_prune () { # $1 config: fetch.prune # $2 config: remote.<name>.prune -# $3 expect: branch to be pruned? -# $4 expect: tag to be pruned? -# $5 git-fetch $cmdline: +# $3 config: fetch.pruneTags +# $4 config: remote.<name>.pruneTags +# $5 expect: branch to be pruned? +# $6 expect: tag to be pruned? +# $7 git-fetch $cmdline: # -# $1 $2 $3 $4 $5 -test_configured_prune unset unset kept kept "" -test_configured_prune unset unset kept kept "--no-prune" -test_configured_prune unset unset pruned kept "--prune" -test_configured_prune unset unset kept pruned \ +# $1 $2 $3 $4 $5 $6 $7 +test_configured_prune unset unset unset unset kept kept "" +test_configured_prune unset unset unset unset kept kept "--no-prune" +test_configured_prune unset unset unset unset pruned kept "--prune" +test_configured_prune unset unset unset unset kept pruned \ "--prune origin 'refs/tags/*:refs/tags/*'" -test_configured_prune unset unset pruned pruned \ +test_configured_prune unset unset unset unset pruned pruned \ "--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'" -test_configured_prune false unset kept kept "" -test_configured_prune false unset kept kept "--no-prune" -test_configured_prune false unset pruned kept "--prune" +test_configured_prune false unset unset unset kept kept "" +test_configured_prune false unset unset unset kept kept "--no-prune" +test_configured_prune false unset unset unset pruned kept "--prune" -test_configured_prune true unset pruned kept "" -test_configured_prune true unset pruned kept "--prune" -test_configured_prune true unset kept kept "--no-prune" +test_configured_prune true unset unset unset pruned kept "" +test_configured_prune true unset unset unset pruned kept "--prune" +test_configured_prune true unset unset unset kept kept "--no-prune" -test_configured_prune unset false kept kept "" -test_configured_prune unset false kept kept "--no-prune" -test_configured_prune unset false pruned kept "--prune" +test_configured_prune unset false unset unset kept kept "" +test_configured_prune unset false unset unset kept kept "--no-prune" +test_configured_prune unset false unset unset pruned kept "--prune" -test_configured_prune false false kept kept "" -test_configured_prune false false kept kept "--no-prune" -test_configured_prune false false pruned kept "--prune" -test_configured_prune false false kept pruned \ +test_configured_prune false false unset unset kept kept "" +test_configured_prune false false unset unset kept kept "--no-prune" +test_configured_prune false false unset unset pruned kept "--prune" +test_configured_prune false false unset unset kept pruned \ "--prune origin 'refs/tags/*:refs/tags/*'" -test_configured_prune false false pruned pruned \ +test_configured_prune false false unset unset pruned pruned \ "--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'" -test_configured_prune true false kept kept "" -test_configured_prune true false pruned kept "--prune" -test_configured_prune true false kept kept "--no-prune" +test_configured_prune true false unset unset kept kept "" +test_configured_prune true false unset unset pruned kept "--prune" +test_configured_prune true false unset unset kept kept "--no-prune" -test_configured_prune unset true pruned kept "" -test_configured_prune unset true kept kept "--no-prune" -test_configured_prune unset true pruned kept "--prune" +test_configured_prune unset true unset unset pruned kept "" +test_configured_prune unset true unset unset kept kept "--no-prune" +test_configured_prune unset true unset unset pruned kept "--prune" -test_configured_prune false true pruned kept "" -test_configured_prune false true kept kept "--no-prune" -test_configured_prune false true pruned kept "--prune" +test_configured_prune false true unset unset pruned kept "" +test_configured_prune false true unset unset kept kept "--no-prune" +test_configured_prune false true unset unset pruned kept "--prune" -test_configured_prune true true pruned kept "" -test_configured_prune true true pruned kept "--prune" -test_configured_prune true true kept kept "--no-prune" -test_configured_prune true true kept pruned \ +test_configured_prune true true unset unset pruned kept "" +test_configured_prune true true unset unset pruned kept "--prune" +test_configured_prune true true unset unset kept kept "--no-prune" +test_configured_prune true true unset unset kept pruned \ "--prune origin 'refs/tags/*:refs/tags/*'" -test_configured_prune true true pruned pruned \ +test_configured_prune true true unset unset pruned pruned \ "--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'" test_expect_success 'all boundary commits are excluded' '
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 12/12 fetch: add a --fetch-prune option and fetch.pruneTags config
In reply to Ævar Arnfjörð BjarmasonAdd a --fetch-prune option to git-fetch, along with fetch.pruneTags config option. This allows for doing any of:
git fetch -p -P
git fetch --prune --prune-tags
git fetch -p -P origin
git fetch --prune --prune-tags origin
Or simply:
git config fetch.prune true &&
git config fetch.pruneTags true &&
git fetchInstead of the much more verbose:
git fetch --prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'
Before this feature it was painful to support the use-case of pulling from a repo which is having both its branches *and* tags deleted regularly, and wanting our local references to reflect upstream.
At work we create deployment tags in the repo for each rollout, and there's *lots* of those, so they're archived within weeks for performance reasons.
Without this change it's hard to centrally configure such repos in /etc/gitconfig (on servers that are only used for working with them). You need to set fetch.prune=true globally, and then for each repo:
git -C {} config --replace-all remote.origin.fetch "refs/tags/*:refs/tags/*" "^refs/tags/*:refs/tags/*$"Now I can simply set fetch.pruneTags=true in /etc/gitconfig as well, and users running "git pull" will automatically get the pruning semantics we want.
See my "Re: [BUG] git remote prune removes local tags, depending on fetch config" ([email protected];
https://public-inbox.org/git/[email protected]/) for more background info.
--- Documentation/config.txt | 14 ++++++++++++++ Documentation/fetch-options.txt | 15 ++++++++++++++- Documentation/git-fetch.txt | 24 ++++++++++++++++++++++++ builtin/fetch.c | 32 ++++++++++++++++++++++++++++++++ remote.c | 5 ++++- remote.h | 3 +++ t/t5510-fetch.sh | 30 ++++++++++++++++++++++++++++++ 7 files changed, 121 insertions(+), 2 deletions(-)
Documentation/config.txt +14 -0
@@ -1401,6 +1401,14 @@ fetch.prune:: option was given on the command line. See also `remote.<name>.prune` and the PRUNING section of linkgit:git-fetch[1]. +fetch.pruneTags:: + If true, fetch will automatically behave as if the + `refs/tags/*:refs/tags/*` refspec was provided when pruning, + if not set already. This allows for setting both this option + and `fetch.prune` to maintain a 1=1 mapping to upstrem + refs. See also `remote.<name>.pruneTags` and the PRUNING + section of linkgit:git-fetch[1]. + fetch.output:: Control how ref update status is printed. Valid values are `full` and `compact`. Default value is `full`. See section
@@ -2945,6 +2953,12 @@ remote.<name>.prune:: remove any remote-tracking references that no longer exist on the remote (as if the `--prune` option was given on the command line). Overrides `fetch.prune` settings, if any. + +remote.<name>.pruneTags:: + When set to true, fetching from this remote by default will also + remove any local tags that no longer exist on the remote if pruning + is activated in general via `remote.<name>.prune`, `fetch.prune` or + `--prune`. Overrides `fetch.pruneTags` settings, if any. + See also `remote.<name>.prune` and the PRUNING section of linkgit:git-fetch[1].
Documentation/fetch-options.txt +14 -1
@@ -73,7 +73,20 @@ ifndef::git-pull[] are fetched due to an explicit refspec (either on the command line or in the remote configuration, for example if the remote was cloned with the --mirror option), then they are also - subject to pruning. + subject to pruning. Supplying `--prune-tags` is a shorthand for + providing the tag refspec. ++ +See the PRUNING section below for more details. + +-P:: +--prune-tags:: + Before fetching, remove any local tags that no longer exist on + the remote if `--prune` is enabled. This option should be used + more carefully, unlike `--prune` it will remove any local + references (local tags) that have been created. This option is + merely a shorthand for providing the explicit tag refspec + along with `--prune`, see the discussion about that in its + documentation. + See the PRUNING section below for more details.
Documentation/git-fetch.txt +24 -0
@@ -148,6 +148,30 @@ So be careful when using this with a refspec like `refs/tags/*:refs/tags/*`, or any other refspec which might map references from multiple remotes to the same local namespace. +Since keeping up-to-date with both branches and tags on the remote is +a common use-case the `--prune-tags` option can be supplied along with +`--prune` to prune local tags that don't exist on the remote. Tag +pruning can also be enabled with `fetch.pruneTags` or +`remote.<name>.pruneTags` in the config. See linkgit:git-config[1]. + +The `--prune-tags` option is equivalent to having +`refs/tags/*:refs/tags/*` configured in the refspecs for the +remote. This can lead to some seemingly strange interactions: + +------------------------------------------------ +# These both fetch tags +$ git fetch --no-tags origin 'refs/tags/*:refs/tags/*' +$ git fetch --no-tags --prune-tags origin +------------------------------------------------ + +The reason it doesn't error out when provided without `--prune` or its +config versions is for flexibility of the configured versions, and to +maintain a 1=1 mapping between what the command line flags do, and +what the configuration versions do. It's reasonable to e.g. configure +`fetch.pruneTags=true` in `~/.gitconfig` to have tags pruned whenever +`git fetch --prune` is run, without making every invocation of `git +fetch` without `--prune` an error. + OUTPUT ------
builtin/fetch.c +32 -0
@@ -38,6 +38,10 @@ static int fetch_prune_config = -1; /* unspecified */static int prune =-1 ;/* unspecified */ #define PRUNE_BY_DEFAULT 0/* do we prune by default? */ +static int fetch_prune_tags_config =-1 ;/* unspecified */ +static int prune_tags =-1 ;/* unspecified */ +#define PRUNE_TAGS_BY_DEFAULT 0/* do we prune tags by default? */ +static int all, append, dry_run, force, keep, multiple, update_head_ok, verbosity, deepen_relative;static int progress =-1 ;static int tags =TAGS_DEFAULT , unshallow, update_shallow, deepen;
@@ -64,6 +68,11 @@ static int git_fetch_config(const char *k, const char *v, void *cb)return 0 ; } +if (!strcmp (k,"fetch.prunetags" )) { + fetch_prune_tags_config =git_config_bool (k, v); +return 0 ; + } +if (!strcmp (k,"submodule.recurse" )) {int r =git_config_bool (k, v) ?RECURSE_SUBMODULES_ON :RECURSE_SUBMODULES_OFF ;
@@ -126,6 +135,8 @@ static struct option builtin_fetch_options[] = { N_("number of submodules fetched in parallel")),OPT_BOOL ('p' ,"prune" , &prune,N_ ("prune remote-tracking branches no longer on remote" )), +OPT_BOOL ('P' ,"prune-tags" , &prune_tags, +N_ ("prune local tags not found on remote" )), { OPTION_CALLBACK, 0, "recurse-submodules", &recurse_submodules, N_("on-demand"),N_ ("control recursive fetching of submodules" ), PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules },
@@ -1212,6 +1223,8 @@ static void add_options_to_argv(struct argv_array *argv)argv_array_push (argv,"--dry-run" );if (prune !=-1 )argv_array_push (argv, prune ?"--prune" :"--no-prune" ); +if (prune_tags !=-1 ) +argv_array_push (argv, prune_tags ?"--prune-tags" :"--no-prune-tags" );if (update_head_ok)argv_array_push (argv,"--update-head-ok" );if (force)
@@ -1315,6 +1328,22 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)return exit_code; } +static void maybe_add_prune_tags_refspec (struct remote *remote) +{ +if (prune_tags <0 ) { +/* no command line request */ +if (0 <= remote->prune_tags) + prune_tags = remote->prune_tags; +else if (0 <= fetch_prune_tags_config) + prune_tags = fetch_prune_tags_config; +else + prune_tags =PRUNE_TAGS_BY_DEFAULT ; + } + +if (prune_tags) +add_fetch_refspec (remote,xstrdup ("refs/tags/*:refs/tags/*" )); +} +int cmd_fetch (int argc,const char **argv,const char *prefix) {int i;
@@ -1368,6 +1397,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) }else if (argc ==0 ) {/* No arguments -- use default remote */ remote =remote_get (NULL ); +maybe_add_prune_tags_refspec (remote); result =fetch_one (remote, argc, argv); }else if (multiple) {/* All arguments are assumed to be remotes or groups */
@@ -1386,6 +1416,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) }else {/* Zero or one remotes */ remote =remote_get (argv[0 ]); +if (argc ==1 ) +maybe_add_prune_tags_refspec (remote); result =fetch_one (remote, argc-1 , argv+1 ); } }
remote.c +4 -1
@@ -95,7 +95,7 @@ static void add_push_refspec(struct remote *remote, const char *ref) remote->push_refspec[remote->push_refspec_nr++] = ref; } -static void add_fetch_refspec (struct remote *remote,const char *ref) +void add_fetch_refspec (struct remote *remote,const char *ref) { ALLOC_GROW(remote->fetch_refspec, remote->fetch_refspec_nr + 1,
@@ -173,6 +173,7 @@ static struct remote *make_remote(const char *name, int len) ret =xcalloc (1 ,sizeof (struct remote )); ret->prune =-1 ;/* unspecified */ + ret->prune_tags =-1 ;/* unspecified */ ALLOC_GROW (remotes, remotes_nr +1 , remotes_alloc); remotes[remotes_nr++] = ret; ret->name =xstrndup (name, len);
@@ -391,6 +392,8 @@ static int handle_config(const char *key, const char *value, void *cb) remote->skip_default_update =git_config_bool (key, value); else if (!strcmp(subkey, "prune")) remote->prune =git_config_bool (key, value); + else if (!strcmp(subkey, "prunetags")) + remote->prune_tags =git_config_bool (key, value); else if (!strcmp(subkey, "url")) {const char *v;if (git_config_string (&v, key, value))
remote.h +3 -0
@@ -47,6 +47,7 @@ struct remote {int skip_default_update;int mirror;int prune; +int prune_tags;const char *receivepack;const char *uploadpack;
@@ -133,6 +134,8 @@ struct ref {#define REF_HEADS (1u << 1)#define REF_TAGS (1u << 2) +void add_fetch_refspec (struct remote *remote,const char *ref); +extern struct ref *find_ref_by_name (const struct ref *list,const char *name);struct ref *alloc_ref (const char *name);
t/t5510-fetch.sh +30 -0
@@ -662,6 +662,36 @@ test_configured_prune true true unset unset kept pruned \ test_configured_prune true true unset unset pruned pruned \ "--prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'" + +# --prune-tags on its own does nothing, needs --prune as well, same +# for for fetch.pruneTags without fetch.prune +test_configured_prune unset unset unset unset kept kept "--prune-tags" +test_configured_prune unset unset unset unset kept kept "origin --prune-tags" +test_configured_prune unset unset true unset kept kept "" +test_configured_prune unset unset unset true kept kept "" + +# These will prune the tags +test_configured_prune unset unset unset unset pruned pruned "--prune --prune-tags" +test_configured_prune unset unset unset unset pruned pruned "origin --prune --prune-tags" +test_configured_prune true unset true unset pruned pruned "" +test_configured_prune unset true unset true pruned pruned "" + +# Check that remote.<name>.pruneTags overrides fetch.pruneTags as with +# remote.<name>.prune and fetch.prune +test_configured_prune false true false true pruned pruned "" +test_configured_prune true false true false kept kept "" + +# When --prune-tags is supplied it's ignored if an explict refspec is +# given, same for the configuration options. +test_configured_prune unset unset unset unset pruned kept \ + "--prune --prune-tags origin '+refs/heads/*:refs/remotes/origin/*'" +test_configured_prune unset unset true unset pruned kept \ + "--prune --prune-tags origin '+refs/heads/*:refs/remotes/origin/*'" +test_configured_prune unset unset unset true pruned kept \ + "--prune --prune-tags origin '+refs/heads/*:refs/remotes/origin/*'" + +test_configured_prune true unset true unset pruned pruned "" + test_expect_success 'all boundary commits are excluded' ' test_commit base && test_commit oneside &&
Signature · 1 line
2.15.1.424.g9478a66081
Re: [PATCH v2 12/12] fetch: add a --fetch-prune option and fetch.pruneTags config
Quoted 26 lines · Ævar Arnfjörð Bjarmason
Add a --fetch-prune option to git-fetch, along with fetch.pruneTags config option. This allows for doing any of:
git fetch -p -P
git fetch --prune --prune-tags
git fetch -p -P origin
git fetch --prune --prune-tags originOr simply:
git config fetch.prune true && git config fetch.pruneTags true && git fetchInstead of the much more verbose:
git fetch --prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'Before this feature it was painful to support the use-case of pulling from a repo which is having both its branches *and* tags deleted regularly, and wanting our local references to reflect upstream.
At work we create deployment tags in the repo for each rollout, and there's *lots* of those, so they're archived within weeks for performance reasons.
Quoted 6 lines
Without this change it's hard to centrally configure such repos in
/etc/gitconfig(on servers that are only used for working with them). You need to set fetch.prune=true globally, and then for each repo:git -C {} config --replace-all remote.origin.fetch "refs/tags/*:refs/tags/*" "^refs/tags/*:refs/tags/*$"
I think the last one is supposed to be a regular expression on existing values. Shouldn't the asterisks be quoted?
Otherwise, it would appears as if "refs/tags:refs/tags///" are replaced with "refs/tags/*:refs/tags/*", but it certainly is not what you are doing. I also wonder why the existing one does not expect a leading '+', which I think is what we place by default when you clone.
Documentation/fetch-options.txt ifndef::git-pull[] · line 89
Quoted 6 lines
+-P:: +--prune-tags::Documentation/fetch-options.txt +4
-0+ .... This option is + merely a shorthand for providing the explicit tag refspec + along with `--prune`, see the discussion about that in its + documentation.
So would "git fetch -P origin" be like "git fetch --prune --tags origin"?
Quoted 16 lines
+
See the PRUNING section below for more details.Documentation/git-fetch.txt +5
-0@@ -148,6 +148,30 @@ So be careful when using this with a refspec like `refs/tags/*:refs/tags/*`, or any other refspec which might map references from multiple remotes to the same local namespace. +Since keeping up-to-date with both branches and tags on the remote is +a common use-case the `--prune-tags` option can be supplied along with +`--prune` to prune local tags that don't exist on the remote. Tag +pruning can also be enabled with `fetch.pruneTags` or +`remote.<name>.pruneTags` in the config. See linkgit:git-config[1].
Documentation/git-fetch.txt So be careful when using this with a refspec like · line 165
Quoted 10 lines
Documentation/git-fetch.txt +10
-0+ +The `--prune-tags` option is equivalent to having +`refs/tags/*:refs/tags/*` configured in the refspecs for the +remote. This can lead to some seemingly strange interactions: + +------------------------------------------------ +# These both fetch tags +$ git fetch --no-tags origin 'refs/tags/*:refs/tags/*' +$ git fetch --no-tags --prune-tags origin +------------------------------------------------
This description is too confusing. First you say "having ... configured in the refspecs for the remote", but configured refspecs for the remote (I presume that you are missing 'fetch' from that description and are talking about the "remote.$name.fetch" configuration variable) are overridden when you give explicit refspecs from the command line, so the above two are not even equivalent. The first one gives a refspec explicitly from the command line, so other configured refspecs like
[remote "origin"] fetch = +refs/heads/*:refs/remotes/origin/*
should be ignored, while the second one, if --prune-tags tells Git to behave as if
[remote "origin"] fetch = refs/tags/*:refs/tags/*
also exists in the config, would not cause other ones for the same remote from getting ignored. So...
Re: [PATCH v2 12/12] fetch: add a --fetch-prune option and fetch.pruneTags config
On Mon, Jan 22 2018, Junio C. Hamano jotted:
Quoted 33 lines
Quoted 31 lines · Ævar Arnfjörð Bjarmason
Add a --fetch-prune option to git-fetch, along with fetch.pruneTags config option. This allows for doing any of:
git fetch -p -P
git fetch --prune --prune-tags
git fetch -p -P origin
git fetch --prune --prune-tags originOr simply:
git config fetch.prune true && git config fetch.pruneTags true && git fetchInstead of the much more verbose:
git fetch --prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'Before this feature it was painful to support the use-case of pulling from a repo which is having both its branches *and* tags deleted regularly, and wanting our local references to reflect upstream.
At work we create deployment tags in the repo for each rollout, and there's *lots* of those, so they're archived within weeks for performance reasons.
Without this change it's hard to centrally configure such repos in
/etc/gitconfig(on servers that are only used for working with them). You need to set fetch.prune=true globally, and then for each repo:
Quoted 8 lines
Quoted 1 line
git -C {} config --replace-all remote.origin.fetch "refs/tags/*:refs/tags/*" "^refs/tags/*:refs/tags/*$"I think the last one is supposed to be a regular expression on existing values. Shouldn't the asterisks be quoted?
Otherwise, it would appears as if "refs/tags:refs/tags///" are replaced with "refs/tags/*:refs/tags/*", but it certainly is not what you are doing.
Yes, well spotted. I copied this from an escaped version and forgot to update that escaping. Will fix.
Quoted 2 lines
I also wonder why the existing one does not expect a leading '+', which I think is what we place by default when you clone.
I didn't raise this because I didn't want to get side-tracked with yet another thing, but it appears to me that a + prefix in tags refspecs does absolutely nothing. Consider on a fresh git.git clobbering the new v2.16.1 tag locally:
$ git tag -a -m"fake" -f v2.16.1 v2.16.0
Updated tag 'v2.16.1' (was eb5fcb24f)This should clobber it, and does:
$ git fetch --prune --dry-run origin '+refs/tags/*:refs/tags/*'
From github.com:git/git
t [tag update] v2.16.1 -> v2.16.1But so will this without +, which seems like a bug to me:
$ git fetch --prune --dry-run origin 'refs/tags/*:refs/tags/*'
From github.com:git/git
t [tag update] v2.16.1 -> v2.16.1But maybe I'm missing something.
Quoted 9 lines
Documentation/fetch-options.txt ifndef::git-pull[] · line 89
Quoted 6 lines
+-P:: +--prune-tags::Documentation/fetch-options.txt +4
-0+ .... This option is + merely a shorthand for providing the explicit tag refspec + along with `--prune`, see the discussion about that in its + documentation.So would "git fetch -P origin" be like "git fetch --prune --tags origin"?
No, as I understand it --tags just has the effect of considering remote tags that aren't tags of the branches you're fetching, so e.g. on a fresh git.git:
$ git tag -a -m"fake" -f v2.16.2 v2.16.0
$ git fetch --prune --tags --dry-run
$I.e. it does nothing. Whereas this will prune the new fake tag:
$ git fetch --prune origin 'refs/tags/*:refs/tags/*'
From github.com:git/git
- [deleted] (none) -> v2.16.2And so does the --prune-tags option:
$ ~/g/git/git-fetch --prune --prune-tags origin
From github.com:git/git
- [deleted] (none) -> v2.16.2Quoted 38 lines
Quoted 16 lines
+
See the PRUNING section below for more details.Documentation/git-fetch.txt +5
-0@@ -148,6 +148,30 @@ So be careful when using this with a refspec like `refs/tags/*:refs/tags/*`, or any other refspec which might map references from multiple remotes to the same local namespace. +Since keeping up-to-date with both branches and tags on the remote is +a common use-case the `--prune-tags` option can be supplied along with +`--prune` to prune local tags that don't exist on the remote. Tag +pruning can also be enabled with `fetch.pruneTags` or +`remote.<name>.pruneTags` in the config. See linkgit:git-config[1].Documentation/git-fetch.txt So be careful when using this with a refspec like · line 165
Quoted 10 lines
Documentation/git-fetch.txt +10
-0+ +The `--prune-tags` option is equivalent to having +`refs/tags/*:refs/tags/*` configured in the refspecs for the +remote. This can lead to some seemingly strange interactions: + +------------------------------------------------ +# These both fetch tags +$ git fetch --no-tags origin 'refs/tags/*:refs/tags/*' +$ git fetch --no-tags --prune-tags origin +------------------------------------------------This description is too confusing. First you say "having ... configured in the refspecs for the remote", but configured refspecs for the remote (I presume that you are missing 'fetch' from that description and are talking about the "remote.$name.fetch" configuration variable) are overridden when you give explicit refspecs from the command line, so the above two are not even equivalent. The first one gives a refspec explicitly from the command line, so other configured refspecs like
[remote "origin"] fetch = +refs/heads/*:refs/remotes/origin/*
Quoted 7 lines
should be ignored, while the second one, if --prune-tags tells Git to behave as if
[remote "origin"] fetch = refs/tags/*:refs/tags/*
also exists in the config, would not cause other ones for the same remote from getting ignored. So...
Indeed, I'll reword this. FWIW I meant something closer to "declared in the refspecs of the remote" not "configured ... remote".
v3 cover document & test fetch pruning & add fetch.pruneTags
In reply to Ævar Arnfjörð BjarmasonAddresses Junio's comments on v2 + more fixes. The
Ævar Arnfjörð Bjarmason (11):
fetch: don't redundantly NULL something calloc() gave us
fetch: stop accessing "remote" variable indirectly
Moved these patches to the beginning. No changes except a note to the commit message saying how these trivial changes relate (or not) to subsequent changes.
fetch tests: refactor in preparation for testing tag pruning
No changes.
fetch tests: re-arrange arguments for future readability
s/arrange/re-arrange/ in commit message subject.
fetch tests: add a tag to be deleted to the pruning tests
No changes.
fetch tests: test --prune and refspec interaction
I'm now just skipping quoting things like +refs/... on the command-line, which as grepping the rest of the test suite shows is fine, this eliminated the need for "fetch tests: double quote a variable for interpolation" so I've ejected it.
git fetch doc: add a new section to explain the ins & outs of pruning
No changes.
git remote doc: correct dangerous lies about what prune does
Rewording of doc per Junio's suggestion.
git-fetch & config doc: link to the new PRUNING section
No changes.
fetch tests: add scaffolding for the new fetch.pruneTags
No changes except continuing remove refspec quoting changes noted above & using +refs/tags/... instead of refs/tags/... for consistency with the last patch...
fetch: add a --fetch-prune option and fetch.pruneTags config
We now consistently use a + prefixed refspec for tag pruning, even though it yields the same result. See the discussion in [email protected], I think it's less confusing.
Fix regex in --replace-all command in the commit message.
Rewording to address Junio's comments.
The short help for --prune-tags is now:
- N_("prune local tags not found on remote")),
+ N_("prune local tags no longer on remote and clobber changed tags")),
Add --prune-tags to the bash completion.
Documentation/config.txt | 20 ++++- Documentation/fetch-options.txt | 18 +++- Documentation/git-fetch.txt | 76 +++++++++++++++++ Documentation/git-remote.txt | 14 +-- builtin/fetch.c | 37 +++++++- contrib/completion/git-completion.bash | 2 +- remote.c | 5 +- remote.h | 3 + t/t5510-fetch.sh | 152 ++++++++++++++++++++++++--------- 9 files changed, 275 insertions(+), 52 deletions(-)
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v3 1/11 fetch: don't redundantly NULL something calloc() gave us
Stop redundantly NULL-ing the last element of the refs structure, which was retrieved via calloc(), and is thus guaranteed to be pre-NULL'd.
This code dates back to b888d61c83 ("Make fetch a builtin", 2007-09-10), where wasn't any reason to do this back then either, it's just something left over from when git-fetch was initially introduced.
The initial motivation for this change was to make a subsequent change which would also modify the refs variable smaller, since it won't have to copy this redundant "NULL the last + 1 item" pattern.
As it turns out that change was bad, and better done differently, but as this pattern is still pointless let's fix it.
--- builtin/fetch.c | 1 - 1 file changed, 1 deletion(-)
builtin/fetch.c +0 -1
@@ -1302,7 +1302,6 @@ static int fetch_one(struct remote *remote, int argc, const char **argv) }else refs[j++] = argv[i]; } - refs[j] =NULL ; ref_nr = j; }
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v3 2/11 fetch: stop accessing "remote" variable indirectly
In reply to Ævar Arnfjörð BjarmasonAccess the "remote" variable passed to the fetch_one() directly rather than through the gtransport wrapper struct constructed in this function for other purposes.
This makes the code more readable, as it's now obvious that the remote struct doesn't somehow get munged by the prepare_transport() function above, which takes the "remote" struct as an argument and constructs the "gtransport" struct, containing among other things the "remote" struct.
A subsequent change will copy this pattern to access a new remote->prune-tags field, but without hte use of the gtransport variable. It's useful once that change lands to see that the two pieces of code behave exactly the same.
This pattern of accessing the container struct was added in 737c5a9cde ("fetch: make --prune configurable", 2013-07-13) when this code was initially introduced.
--- builtin/fetch.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
builtin/fetch.c +2 -2
@@ -1280,8 +1280,8 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)if (prune <0 ) {/* no command line request */ -if (0 <= gtransport->remote->prune) - prune = gtransport->remote->prune; +if (0 <= remote->prune) + prune = remote->prune;else if (0 <= fetch_prune_config) prune = fetch_prune_config;else
Signature · 1 line
2.15.1.424.g9478a66081
Re: [PATCH v3 02/11] fetch: stop accessing "remote" variable indirectly
Quoted 1 line · Ævar Arnfjörð Bjarmason
remote->prune-tags field, but without hte use of the gtransport
s/-tags/_tags/; s/hte/the/;
No need to resend for fixing the above, as I've done so locally before queuing.
Thanks.
PATCH v3 3/11 fetch tests: refactor in preparation for testing tag pruning
In reply to Ævar Arnfjörð BjarmasonIn a subsequent commit this function will learn to test for tag pruning, prepare for that by making space for more variables, and making it clear that "expected" here refers to branches.
--- t/t5510-fetch.sh | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
t/t5510-fetch.sh +6 -3
@@ -549,9 +549,12 @@ set_config_tristate () { } test_configured_prune () { - fetch_prune=$1 remote_origin_prune=$2 cmdline=$3 expected=$4 + fetch_prune=$1 + remote_origin_prune=$2 + cmdline=$3 + expected_branch=$4 - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; $4" ' + test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && (
@@ -572,7 +575,7 @@ test_configured_prune () { set_config_tristate remote.origin.prune $remote_origin_prune && git fetch $cmdline && - case "$expected" in + case "$expected_branch" in pruned) test_must_fail git rev-parse --verify refs/remotes/origin/newbranch ;;
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v3 4/11 fetch tests: re-arrange arguments for future readability
In reply to Ævar Arnfjörð BjarmasonRe-arrange the arguments to the test_configured_prune() function used in this test to pass the arguments to --fetch last. A subsequent change will test for more elaborate fetch arguments, including long refspecs. It'll be more readable to be able to wrap those on a new line of their own.
--- t/t5510-fetch.sh | 82 ++++++++++++++++++++++++++++++-------------------------- 1 file changed, 44 insertions(+), 38 deletions(-)
t/t5510-fetch.sh +44 -38
@@ -551,10 +551,10 @@ set_config_tristate () { test_configured_prune () { fetch_prune=$1 remote_origin_prune=$2 - cmdline=$3 - expected_branch=$4 + expected_branch=$3 + cmdline=$4 - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" ' + test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && (
@@ -587,41 +587,47 @@ test_configured_prune () { ' } -test_configured_prune unset unset "" kept -test_configured_prune unset unset "--no-prune" kept -test_configured_prune unset unset "--prune" pruned - -test_configured_prune false unset "" kept -test_configured_prune false unset "--no-prune" kept -test_configured_prune false unset "--prune" pruned - -test_configured_prune true unset "" pruned -test_configured_prune true unset "--prune" pruned -test_configured_prune true unset "--no-prune" kept - -test_configured_prune unset false "" kept -test_configured_prune unset false "--no-prune" kept -test_configured_prune unset false "--prune" pruned - -test_configured_prune false false "" kept -test_configured_prune false false "--no-prune" kept -test_configured_prune false false "--prune" pruned - -test_configured_prune true false "" kept -test_configured_prune true false "--prune" pruned -test_configured_prune true false "--no-prune" kept - -test_configured_prune unset true "" pruned -test_configured_prune unset true "--no-prune" kept -test_configured_prune unset true "--prune" pruned - -test_configured_prune false true "" pruned -test_configured_prune false true "--no-prune" kept -test_configured_prune false true "--prune" pruned - -test_configured_prune true true "" pruned -test_configured_prune true true "--prune" pruned -test_configured_prune true true "--no-prune" kept +# $1 config: fetch.prune +# $2 config: remote.<name>.prune +# $3 expect: branch to be pruned? +# $4 git-fetch $cmdline: +# +# $1 $2 $3 $4 +test_configured_prune unset unset kept "" +test_configured_prune unset unset kept "--no-prune" +test_configured_prune unset unset pruned "--prune" + +test_configured_prune false unset kept "" +test_configured_prune false unset kept "--no-prune" +test_configured_prune false unset pruned "--prune" + +test_configured_prune true unset pruned "" +test_configured_prune true unset pruned "--prune" +test_configured_prune true unset kept "--no-prune" + +test_configured_prune unset false kept "" +test_configured_prune unset false kept "--no-prune" +test_configured_prune unset false pruned "--prune" + +test_configured_prune false false kept "" +test_configured_prune false false kept "--no-prune" +test_configured_prune false false pruned "--prune" + +test_configured_prune true false kept "" +test_configured_prune true false pruned "--prune" +test_configured_prune true false kept "--no-prune" + +test_configured_prune unset true pruned "" +test_configured_prune unset true kept "--no-prune" +test_configured_prune unset true pruned "--prune" + +test_configured_prune false true pruned "" +test_configured_prune false true kept "--no-prune" +test_configured_prune false true pruned "--prune" + +test_configured_prune true true pruned "" +test_configured_prune true true pruned "--prune" +test_configured_prune true true kept "--no-prune" test_expect_success 'all boundary commits are excluded' ' test_commit base &&
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v3 5/11 fetch tests: add a tag to be deleted to the pruning tests
In reply to Ævar Arnfjörð BjarmasonAdd a tag to be deleted to the fetch --prune tests. The tag is always kept for now, which is the expected behavior, but now I can add a test for tag pruning in a later commit.
--- t/t5510-fetch.sh | 93 ++++++++++++++++++++++++++++++++------------------------ 1 file changed, 53 insertions(+), 40 deletions(-)
t/t5510-fetch.sh +53 -40
@@ -552,21 +552,25 @@ test_configured_prune () { fetch_prune=$1 remote_origin_prune=$2 expected_branch=$3 - cmdline=$4 + expected_tag=$4 + cmdline=$5 - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" ' + test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && + git tag -f newtag && ( cd one && test_unconfig fetch.prune && test_unconfig remote.origin.prune && git fetch && - git rev-parse --verify refs/remotes/origin/newbranch + git rev-parse --verify refs/remotes/origin/newbranch && + git rev-parse --verify refs/tags/newtag ) && # now remove it git branch -d newbranch && + git tag -d newtag && # then test (
@@ -582,6 +586,14 @@ test_configured_prune () { kept) git rev-parse --verify refs/remotes/origin/newbranch ;; + esac && + case "$expected_tag" in + pruned) + test_must_fail git rev-parse --verify refs/tags/newtag + ;; + kept) + git rev-parse --verify refs/tags/newtag + ;; esac ) '
@@ -590,44 +602,45 @@ test_configured_prune () { # $1 config: fetch.prune # $2 config: remote.<name>.prune # $3 expect: branch to be pruned? -# $4 git-fetch $cmdline: +# $4 expect: tag to be pruned? +# $5 git-fetch $cmdline: # -# $1 $2 $3 $4 -test_configured_prune unset unset kept "" -test_configured_prune unset unset kept "--no-prune" -test_configured_prune unset unset pruned "--prune" - -test_configured_prune false unset kept "" -test_configured_prune false unset kept "--no-prune" -test_configured_prune false unset pruned "--prune" - -test_configured_prune true unset pruned "" -test_configured_prune true unset pruned "--prune" -test_configured_prune true unset kept "--no-prune" - -test_configured_prune unset false kept "" -test_configured_prune unset false kept "--no-prune" -test_configured_prune unset false pruned "--prune" - -test_configured_prune false false kept "" -test_configured_prune false false kept "--no-prune" -test_configured_prune false false pruned "--prune" - -test_configured_prune true false kept "" -test_configured_prune true false pruned "--prune" -test_configured_prune true false kept "--no-prune" - -test_configured_prune unset true pruned "" -test_configured_prune unset true kept "--no-prune" -test_configured_prune unset true pruned "--prune" - -test_configured_prune false true pruned "" -test_configured_prune false true kept "--no-prune" -test_configured_prune false true pruned "--prune" - -test_configured_prune true true pruned "" -test_configured_prune true true pruned "--prune" -test_configured_prune true true kept "--no-prune" +# $1 $2 $3 $4 $5 +test_configured_prune unset unset kept kept "" +test_configured_prune unset unset kept kept "--no-prune" +test_configured_prune unset unset pruned kept "--prune" + +test_configured_prune false unset kept kept "" +test_configured_prune false unset kept kept "--no-prune" +test_configured_prune false unset pruned kept "--prune" + +test_configured_prune true unset pruned kept "" +test_configured_prune true unset pruned kept "--prune" +test_configured_prune true unset kept kept "--no-prune" + +test_configured_prune unset false kept kept "" +test_configured_prune unset false kept kept "--no-prune" +test_configured_prune unset false pruned kept "--prune" + +test_configured_prune false false kept kept "" +test_configured_prune false false kept kept "--no-prune" +test_configured_prune false false pruned kept "--prune" + +test_configured_prune true false kept kept "" +test_configured_prune true false pruned kept "--prune" +test_configured_prune true false kept kept "--no-prune" + +test_configured_prune unset true pruned kept "" +test_configured_prune unset true kept kept "--no-prune" +test_configured_prune unset true pruned kept "--prune" + +test_configured_prune false true pruned kept "" +test_configured_prune false true kept kept "--no-prune" +test_configured_prune false true pruned kept "--prune" + +test_configured_prune true true pruned kept "" +test_configured_prune true true pruned kept "--prune" +test_configured_prune true true kept kept "--no-prune" test_expect_success 'all boundary commits are excluded' ' test_commit base &&
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v3 6/11 fetch tests: test --prune and refspec interaction
In reply to Ævar Arnfjörð BjarmasonAdd a test for the interaction between explicitly provided refspecs and fetch.prune.
There's no point in adding this boilerplate to every combination of unset/false/true, it's instructive and sufficient to show that no matter if the variable is unset, false or true the refspec on the command-line overrides any configuration variable.
--- t/t5510-fetch.sh | 12 ++++++++++++ 1 file changed, 12 insertions(+)
t/t5510-fetch.sh +12 -0
@@ -609,6 +609,10 @@ test_configured_prune () { test_configured_prune unset unset kept kept "" test_configured_prune unset unset kept kept "--no-prune" test_configured_prune unset unset pruned kept "--prune" +test_configured_prune unset unset kept pruned \ + "--prune origin +refs/tags/*:refs/tags/*" +test_configured_prune unset unset pruned pruned \ + "--prune origin +refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" test_configured_prune false unset kept kept "" test_configured_prune false unset kept kept "--no-prune"
@@ -625,6 +629,10 @@ test_configured_prune unset false pruned kept "--prune" test_configured_prune false false kept kept "" test_configured_prune false false kept kept "--no-prune" test_configured_prune false false pruned kept "--prune" +test_configured_prune false false kept pruned \ + "--prune origin +refs/tags/*:refs/tags/*" +test_configured_prune false false pruned pruned \ + "--prune origin +refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" test_configured_prune true false kept kept "" test_configured_prune true false pruned kept "--prune"
@@ -641,6 +649,10 @@ test_configured_prune false true pruned kept "--prune" test_configured_prune true true pruned kept "" test_configured_prune true true pruned kept "--prune" test_configured_prune true true kept kept "--no-prune" +test_configured_prune true true kept pruned \ + "--prune origin +refs/tags/*:refs/tags/*" +test_configured_prune true true pruned pruned \ + "--prune origin +refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" test_expect_success 'all boundary commits are excluded' ' test_commit base &&
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v3 7/11 git fetch doc: add a new section to explain the ins & outs of pruning
In reply to Ævar Arnfjörð BjarmasonAdd a new section to canonically explain how remote reference pruning works, and how users should be careful about using it in conjunction with tag refspecs in particular.
A subsequent commit will update the git-remote documentation to refer to this section, and details the motivation for writing this in the first place.
--- Documentation/git-fetch.txt | 49 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+)
Documentation/git-fetch.txt +49 -0
@@ -99,6 +99,55 @@ The latter use of the `remote.<repository>.fetch` values can be overridden by giving the `--refmap=<refspec>` parameter(s) on the command line. +PRUNING +------- + +Git has a default disposition of keeping data unless it's explicitly +thrown away; this extends to holding onto local references to branches +on remotes that have themselves deleted those branches. + +If left to accumulate, these stale references might make performance +worse on big and busy repos that have a lot of branch churn, and +e.g. make the output of commands like `git branch -a --contains +<commit>` needlessly verbose, as well as impacting anything else +that'll work with the complete set of known references. + +These remote tracking references can be deleted as a one-off with +either of: + +------------------------------------------------ +# While fetching +$ git fetch --prune <name> + +# Only prune, don't fetch +$ git remote prune <name> +------------------------------------------------ + +To prune references as part of your normal workflow without needing to +remember to run that set `fetch.prune` globally, or +`remote.<name>.prune` per-remote in the config. See +linkgit:git-config[1]. + +Here's where things get tricky and more specific. The pruning feature +doesn't actually care about branches, instead it'll prune local <-> +remote references as a function of the refspec of the remote (see +`<refspec>` and <<CRTB,CONFIGURED REMOTE-TRACKING BRANCHES>> above). + +Therefore if the refspec for the remote includes +e.g. `refs/tags/*:refs/tags/*`, or you manually run e.g. `git fetch +--prune <name> "refs/tags/*:refs/tags/*"` it won't be stale remote +tracking branches that are deleted, but any local tag that doesn't +exist on the remote. + +This might not be what you expect, i.e. you want to prune remote +`<name>`, but also explicitly fetch tags from it, so when you fetch +from it you delete all your local tags, most of which may not have +come from the `<name>` remote in the first place. + +So be careful when using this with a refspec like +`refs/tags/*:refs/tags/*`, or any other refspec which might map +references from multiple remotes to the same local namespace. + OUTPUT ------
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v3 8/11 git remote doc: correct dangerous lies about what prune does
In reply to Ævar Arnfjörð BjarmasonThe "git remote prune <name>" command uses the same machinery as "git fetch <name> --prune", and shares all the same caveats, but its documentation has suggested that it'll just "delete stale remote-tracking branches under <name>".
This isn't true, and hasn't been true since at least v1.8.5.6 (the oldest version I could be bothered to test).
E.g. if "+refs/tags/*:refs/tags/*" is explicitly set in the refspec of the remote it'll delete all local tags <name> doesn't know about.
Instead, briefly give the reader just enough of a hint that this option might constitute a shotgun aimed at their foot, and point them to the new PRUNING section in the git-fetch documentation which explains all the nuances of what this facility does.
See "[BUG] git remote prune removes local tags, depending on fetch config" (CACi5S_39wNrbfjLfn0xhCY+uewtFN2YmnAcRc86z6pjUTjWPHQ@mail.gmail.com) by Michael Giuffrida for the initial report.
--- Documentation/git-remote.txt | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
Documentation/git-remote.txt +9 -5
@@ -172,10 +172,14 @@ With `-n` option, the remote heads are not queried first with 'prune':: -Deletes all stale remote-tracking branches under <name>. -These stale branches have already been removed from the remote repository -referenced by <name>, but are still locally available in -"remotes/<name>". +Deletes stale references associated with <name>. By default stale +remote-tracking branches under <name>, but depending on global +configuration and the configuration of the remote we might even prune +local tags that haven't been pushed there. Equivalent to `git fetch +--prune <name>`, except that no new references will be fetched. ++ +See the PRUNING section of linkgit:git-fetch[1] for what it'll prune +depending on various configuration. + With `--dry-run` option, report what branches will be pruned, but do not actually prune them.
@@ -189,7 +193,7 @@ remotes.default is not defined, all remotes which do not have the configuration parameter remote.<name>.skipDefaultUpdate set to true will be updated. (See linkgit:git-config[1]). + -With `--prune` option, prune all the remotes that are updated. +With `--prune` option, run pruning against all the remotes that are updated. DISCUSSION
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v3 9/11 git-fetch & config doc: link to the new PRUNING section
In reply to Ævar Arnfjörð BjarmasonAmend the documentation for fetch.prune, fetch.<name>.prune and --prune to link to the recently added PRUNING section.
I'd have liked to link directly to it with "<<PRUNING>>" from fetch-options.txt, since it's included in git-fetch.txt (git-pull.txt also includes it, but doesn't include that option). However making a reference across files yields this error:
[...]/Documentation/git-fetch.xml:226: element xref: validity error : IDREF attribute linkend references an unknown ID "PRUNING"
--- Documentation/config.txt | 6 +++++- Documentation/fetch-options.txt | 3 +++ 2 files changed, 8 insertions(+), 1 deletion(-)
Documentation/config.txt +5 -1
@@ -1398,7 +1398,8 @@ fetch.unpackLimit:: fetch.prune:: If true, fetch will automatically behave as if the `--prune` - option was given on the command line. See also `remote.<name>.prune`. + option was given on the command line. See also `remote.<name>.prune` + and the PRUNING section of linkgit:git-fetch[1]. fetch.output:: Control how ref update status is printed. Valid values are
@@ -2944,6 +2945,9 @@ remote.<name>.prune:: remove any remote-tracking references that no longer exist on the remote (as if the `--prune` option was given on the command line). Overrides `fetch.prune` settings, if any. ++ +See also `remote.<name>.prune` and the PRUNING section of +linkgit:git-fetch[1]. remotes.<group>:: The list of remotes which are fetched by "git remote update
Documentation/fetch-options.txt +3 -0
@@ -74,6 +74,9 @@ ifndef::git-pull[] line or in the remote configuration, for example if the remote was cloned with the --mirror option), then they are also subject to pruning. ++ +See the PRUNING section below for more details. + endif::git-pull[] ifndef::git-pull[]
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v3 10/11 fetch tests: add scaffolding for the new fetch.pruneTags
In reply to Ævar Arnfjörð BjarmasonThe fetch.pruneTags configuration doesn't exist yet, but will be added in a subsequent commit. Since testing for it requires adding new parameters to the test_configured_prune function it's easier to review this patch first to assert that no functional changes are introduced yet.
--- t/t5510-fetch.sh | 90 ++++++++++++++++++++++++++++++-------------------------- 1 file changed, 49 insertions(+), 41 deletions(-)
t/t5510-fetch.sh +49 -41
@@ -551,18 +551,22 @@ set_config_tristate () { test_configured_prune () { fetch_prune=$1 remote_origin_prune=$2 - expected_branch=$3 - expected_tag=$4 - cmdline=$5 + fetch_prune_tags=$3 + remote_origin_prune_tags=$4 + expected_branch=$5 + expected_tag=$6 + cmdline=$7 - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" ' + test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2 fetch.pruneTags=$3 remote.origin.pruneTags=$4${7:+ $7}; branch:$5 tag:$6" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && git tag -f newtag && ( cd one && test_unconfig fetch.prune && + test_unconfig fetch.pruneTags && test_unconfig remote.origin.prune && + test_unconfig remote.origin.pruneTags && git fetch && git rev-parse --verify refs/remotes/origin/newbranch && git rev-parse --verify refs/tags/newtag
@@ -576,7 +580,9 @@ test_configured_prune () { ( cd one && set_config_tristate fetch.prune $fetch_prune && + set_config_tristate fetch.pruneTags $fetch_prune_tags && set_config_tristate remote.origin.prune $remote_origin_prune && + set_config_tristate remote.origin.pruneTags $remote_origin_prune_tags && git fetch $cmdline && case "$expected_branch" in
@@ -601,57 +607,59 @@ test_configured_prune () { # $1 config: fetch.prune # $2 config: remote.<name>.prune -# $3 expect: branch to be pruned? -# $4 expect: tag to be pruned? -# $5 git-fetch $cmdline: +# $3 config: fetch.pruneTags +# $4 config: remote.<name>.pruneTags +# $5 expect: branch to be pruned? +# $6 expect: tag to be pruned? +# $7 git-fetch $cmdline: # -# $1 $2 $3 $4 $5 -test_configured_prune unset unset kept kept "" -test_configured_prune unset unset kept kept "--no-prune" -test_configured_prune unset unset pruned kept "--prune" -test_configured_prune unset unset kept pruned \ +# $1 $2 $3 $4 $5 $6 $7 +test_configured_prune unset unset unset unset kept kept "" +test_configured_prune unset unset unset unset kept kept "--no-prune" +test_configured_prune unset unset unset unset pruned kept "--prune" +test_configured_prune unset unset unset unset kept pruned \ "--prune origin +refs/tags/*:refs/tags/*" -test_configured_prune unset unset pruned pruned \ +test_configured_prune unset unset unset unset pruned pruned \ "--prune origin +refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" -test_configured_prune false unset kept kept "" -test_configured_prune false unset kept kept "--no-prune" -test_configured_prune false unset pruned kept "--prune" +test_configured_prune false unset unset unset kept kept "" +test_configured_prune false unset unset unset kept kept "--no-prune" +test_configured_prune false unset unset unset pruned kept "--prune" -test_configured_prune true unset pruned kept "" -test_configured_prune true unset pruned kept "--prune" -test_configured_prune true unset kept kept "--no-prune" +test_configured_prune true unset unset unset pruned kept "" +test_configured_prune true unset unset unset pruned kept "--prune" +test_configured_prune true unset unset unset kept kept "--no-prune" -test_configured_prune unset false kept kept "" -test_configured_prune unset false kept kept "--no-prune" -test_configured_prune unset false pruned kept "--prune" +test_configured_prune unset false unset unset kept kept "" +test_configured_prune unset false unset unset kept kept "--no-prune" +test_configured_prune unset false unset unset pruned kept "--prune" -test_configured_prune false false kept kept "" -test_configured_prune false false kept kept "--no-prune" -test_configured_prune false false pruned kept "--prune" -test_configured_prune false false kept pruned \ +test_configured_prune false false unset unset kept kept "" +test_configured_prune false false unset unset kept kept "--no-prune" +test_configured_prune false false unset unset pruned kept "--prune" +test_configured_prune false false unset unset kept pruned \ "--prune origin +refs/tags/*:refs/tags/*" -test_configured_prune false false pruned pruned \ +test_configured_prune false false unset unset pruned pruned \ "--prune origin +refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" -test_configured_prune true false kept kept "" -test_configured_prune true false pruned kept "--prune" -test_configured_prune true false kept kept "--no-prune" +test_configured_prune true false unset unset kept kept "" +test_configured_prune true false unset unset pruned kept "--prune" +test_configured_prune true false unset unset kept kept "--no-prune" -test_configured_prune unset true pruned kept "" -test_configured_prune unset true kept kept "--no-prune" -test_configured_prune unset true pruned kept "--prune" +test_configured_prune unset true unset unset pruned kept "" +test_configured_prune unset true unset unset kept kept "--no-prune" +test_configured_prune unset true unset unset pruned kept "--prune" -test_configured_prune false true pruned kept "" -test_configured_prune false true kept kept "--no-prune" -test_configured_prune false true pruned kept "--prune" +test_configured_prune false true unset unset pruned kept "" +test_configured_prune false true unset unset kept kept "--no-prune" +test_configured_prune false true unset unset pruned kept "--prune" -test_configured_prune true true pruned kept "" -test_configured_prune true true pruned kept "--prune" -test_configured_prune true true kept kept "--no-prune" -test_configured_prune true true kept pruned \ +test_configured_prune true true unset unset pruned kept "" +test_configured_prune true true unset unset pruned kept "--prune" +test_configured_prune true true unset unset kept kept "--no-prune" +test_configured_prune true true unset unset kept pruned \ "--prune origin +refs/tags/*:refs/tags/*" -test_configured_prune true true pruned pruned \ +test_configured_prune true true unset unset pruned pruned \ "--prune origin +refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" test_expect_success 'all boundary commits are excluded' '
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v3 11/11 fetch: add a --fetch-prune option and fetch.pruneTags config
In reply to Ævar Arnfjörð BjarmasonAdd a --fetch-prune option to git-fetch, along with fetch.pruneTags config option. This allows for doing any of:
git fetch -p -P
git fetch --prune --prune-tags
git fetch -p -P origin
git fetch --prune --prune-tags origin
Or simply:
git config fetch.prune true &&
git config fetch.pruneTags true &&
git fetchInstead of the much more verbose:
git fetch --prune origin '+refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'
Before this feature it was painful to support the use-case of pulling from a repo which is having both its branches *and* tags deleted regularly, and have our local references to reflect upstream.
At work we create deployment tags in the repo for each rollout, and there's *lots* of those, so they're archived within weeks for performance reasons.
Without this change it's hard to centrally configure such repos in /etc/gitconfig (on servers that are only used for working with them). You need to set fetch.prune=true globally, and then for each repo:
git -C {} config --replace-all remote.origin.fetch "+refs/tags/*:refs/tags/*" "^\+*refs/tags/\*:refs/tags/\*$"Now I can simply set fetch.pruneTags=true in /etc/gitconfig as well, and users running "git pull" will automatically get the pruning semantics I want.
Even though "git remote" has corresponding "prune" and "update --prune" subcommands I'm intentionally not adding a corresponding prune-tags or "update --prune --prune-tags" mode to that command.
It's advertised (as noted in my recent "git remote doc: correct dangerous lies about what prune does") as only modifying remote tracking references, whereas any --prune-tags option is always going to modify what from the user's perspective is a local copy of the tag, since there's no such thing as a remote tracking tag.
See my "Re: [BUG] git remote prune removes local tags, depending on fetch config" ([email protected];
https://public-inbox.org/git/[email protected]/) for more background info.
--- Documentation/config.txt | 14 ++++++++++++++ Documentation/fetch-options.txt | 15 ++++++++++++++- Documentation/git-fetch.txt | 27 +++++++++++++++++++++++++++ builtin/fetch.c | 32 ++++++++++++++++++++++++++++++++ contrib/completion/git-completion.bash | 2 +- remote.c | 5 ++++- remote.h | 3 +++ t/t5510-fetch.sh | 30 ++++++++++++++++++++++++++++++ 8 files changed, 125 insertions(+), 3 deletions(-)
Documentation/config.txt +14 -0
@@ -1401,6 +1401,14 @@ fetch.prune:: option was given on the command line. See also `remote.<name>.prune` and the PRUNING section of linkgit:git-fetch[1]. +fetch.pruneTags:: + If true, fetch will automatically behave as if the + `refs/tags/*:refs/tags/*` refspec was provided when pruning, + if not set already. This allows for setting both this option + and `fetch.prune` to maintain a 1=1 mapping to upstrem + refs. See also `remote.<name>.pruneTags` and the PRUNING + section of linkgit:git-fetch[1]. + fetch.output:: Control how ref update status is printed. Valid values are `full` and `compact`. Default value is `full`. See section
@@ -2945,6 +2953,12 @@ remote.<name>.prune:: remove any remote-tracking references that no longer exist on the remote (as if the `--prune` option was given on the command line). Overrides `fetch.prune` settings, if any. + +remote.<name>.pruneTags:: + When set to true, fetching from this remote by default will also + remove any local tags that no longer exist on the remote if pruning + is activated in general via `remote.<name>.prune`, `fetch.prune` or + `--prune`. Overrides `fetch.pruneTags` settings, if any. + See also `remote.<name>.prune` and the PRUNING section of linkgit:git-fetch[1].
Documentation/fetch-options.txt +14 -1
@@ -73,7 +73,20 @@ ifndef::git-pull[] are fetched due to an explicit refspec (either on the command line or in the remote configuration, for example if the remote was cloned with the --mirror option), then they are also - subject to pruning. + subject to pruning. Supplying `--prune-tags` is a shorthand for + providing the tag refspec. ++ +See the PRUNING section below for more details. + +-P:: +--prune-tags:: + Before fetching, remove any local tags that no longer exist on + the remote if `--prune` is enabled. This option should be used + more carefully, unlike `--prune` it will remove any local + references (local tags) that have been created. This option is + merely a shorthand for providing the explicit tag refspec + along with `--prune`, see the discussion about that in its + documentation. + See the PRUNING section below for more details.
Documentation/git-fetch.txt +27 -0
@@ -148,6 +148,33 @@ So be careful when using this with a refspec like `refs/tags/*:refs/tags/*`, or any other refspec which might map references from multiple remotes to the same local namespace. +Since keeping up-to-date with both branches and tags on the remote is +a common use-case the `--prune-tags` option can be supplied along with +`--prune` to prune local tags that don't exist on the remote, and +force-update those tags that differ. Tag pruning can also be enabled +with `fetch.pruneTags` or `remote.<name>.pruneTags` in the config. See +linkgit:git-config[1]. + +The `--prune-tags` option is equivalent to having +`+refs/tags/*:refs/tags/*` declared in the refspecs of the +remote. This can lead to some seemingly strange interactions: + +------------------------------------------------ +# These both fetch tags +$ git fetch --no-tags origin '+refs/tags/*:refs/tags/*' +$ git fetch --no-tags --prune-tags origin +------------------------------------------------ + +The reason it doesn't error out when provided without `--prune` or its +config versions is for flexibility of the configured versions, and to +maintain a 1=1 mapping between what the command line flags do, and +what the configuration versions do. + +It's reasonable to e.g. configure `fetch.pruneTags=true` in +`~/.gitconfig` to have tags pruned whenever `git fetch --prune` is +run, without making every invocation of `git fetch` without `--prune` +an error. + OUTPUT ------
builtin/fetch.c +32 -0
@@ -38,6 +38,10 @@ static int fetch_prune_config = -1; /* unspecified */static int prune =-1 ;/* unspecified */ #define PRUNE_BY_DEFAULT 0/* do we prune by default? */ +static int fetch_prune_tags_config =-1 ;/* unspecified */ +static int prune_tags =-1 ;/* unspecified */ +#define PRUNE_TAGS_BY_DEFAULT 0/* do we prune tags by default? */ +static int all, append, dry_run, force, keep, multiple, update_head_ok, verbosity, deepen_relative;static int progress =-1 ;static int tags =TAGS_DEFAULT , unshallow, update_shallow, deepen;
@@ -64,6 +68,11 @@ static int git_fetch_config(const char *k, const char *v, void *cb)return 0 ; } +if (!strcmp (k,"fetch.prunetags" )) { + fetch_prune_tags_config =git_config_bool (k, v); +return 0 ; + } +if (!strcmp (k,"submodule.recurse" )) {int r =git_config_bool (k, v) ?RECURSE_SUBMODULES_ON :RECURSE_SUBMODULES_OFF ;
@@ -126,6 +135,8 @@ static struct option builtin_fetch_options[] = { N_("number of submodules fetched in parallel")),OPT_BOOL ('p' ,"prune" , &prune,N_ ("prune remote-tracking branches no longer on remote" )), +OPT_BOOL ('P' ,"prune-tags" , &prune_tags, +N_ ("prune local tags no longer on remote and clobber changed tags" )), { OPTION_CALLBACK, 0, "recurse-submodules", &recurse_submodules, N_("on-demand"),N_ ("control recursive fetching of submodules" ), PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules },
@@ -1212,6 +1223,8 @@ static void add_options_to_argv(struct argv_array *argv)argv_array_push (argv,"--dry-run" );if (prune !=-1 )argv_array_push (argv, prune ?"--prune" :"--no-prune" ); +if (prune_tags !=-1 ) +argv_array_push (argv, prune_tags ?"--prune-tags" :"--no-prune-tags" );if (update_head_ok)argv_array_push (argv,"--update-head-ok" );if (force)
@@ -1315,6 +1328,22 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)return exit_code; } +static void maybe_add_prune_tags_refspec (struct remote *remote) +{ +if (prune_tags <0 ) { +/* no command line request */ +if (0 <= remote->prune_tags) + prune_tags = remote->prune_tags; +else if (0 <= fetch_prune_tags_config) + prune_tags = fetch_prune_tags_config; +else + prune_tags =PRUNE_TAGS_BY_DEFAULT ; + } + +if (prune_tags) +add_fetch_refspec (remote,xstrdup ("+refs/tags/*:refs/tags/*" )); +} +int cmd_fetch (int argc,const char **argv,const char *prefix) {int i;
@@ -1368,6 +1397,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) }else if (argc ==0 ) {/* No arguments -- use default remote */ remote =remote_get (NULL ); +maybe_add_prune_tags_refspec (remote); result =fetch_one (remote, argc, argv); }else if (multiple) {/* All arguments are assumed to be remotes or groups */
@@ -1386,6 +1416,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) }else {/* Zero or one remotes */ remote =remote_get (argv[0 ]); +if (argc ==1 ) +maybe_add_prune_tags_refspec (remote); result =fetch_one (remote, argc-1 , argv+1 ); } }
contrib/completion/git-completion.bash +1 -1
@@ -1468,7 +1468,7 @@ __git_fetch_recurse_submodules="yes on-demand no" __git_fetch_options=" --quiet --verbose --append --upload-pack --force --keep --depth= --tags --no-tags --all --prune --dry-run --recurse-submodules= - --unshallow --update-shallow + --unshallow --update-shallow --prune-tags " _git_fetch ()
remote.c +4 -1
@@ -95,7 +95,7 @@ static void add_push_refspec(struct remote *remote, const char *ref) remote->push_refspec[remote->push_refspec_nr++] = ref; } -static void add_fetch_refspec (struct remote *remote,const char *ref) +void add_fetch_refspec (struct remote *remote,const char *ref) { ALLOC_GROW(remote->fetch_refspec, remote->fetch_refspec_nr + 1,
@@ -173,6 +173,7 @@ static struct remote *make_remote(const char *name, int len) ret =xcalloc (1 ,sizeof (struct remote )); ret->prune =-1 ;/* unspecified */ + ret->prune_tags =-1 ;/* unspecified */ ALLOC_GROW (remotes, remotes_nr +1 , remotes_alloc); remotes[remotes_nr++] = ret; ret->name =xstrndup (name, len);
@@ -391,6 +392,8 @@ static int handle_config(const char *key, const char *value, void *cb) remote->skip_default_update =git_config_bool (key, value); else if (!strcmp(subkey, "prune")) remote->prune =git_config_bool (key, value); + else if (!strcmp(subkey, "prunetags")) + remote->prune_tags =git_config_bool (key, value); else if (!strcmp(subkey, "url")) {const char *v;if (git_config_string (&v, key, value))
remote.h +3 -0
@@ -47,6 +47,7 @@ struct remote {int skip_default_update;int mirror;int prune; +int prune_tags;const char *receivepack;const char *uploadpack;
@@ -133,6 +134,8 @@ struct ref {#define REF_HEADS (1u << 1)#define REF_TAGS (1u << 2) +void add_fetch_refspec (struct remote *remote,const char *ref); +extern struct ref *find_ref_by_name (const struct ref *list,const char *name);struct ref *alloc_ref (const char *name);
t/t5510-fetch.sh +30 -0
@@ -662,6 +662,36 @@ test_configured_prune true true unset unset kept pruned \ test_configured_prune true true unset unset pruned pruned \ "--prune origin +refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" + +# --prune-tags on its own does nothing, needs --prune as well, same +# for for fetch.pruneTags without fetch.prune +test_configured_prune unset unset unset unset kept kept "--prune-tags" +test_configured_prune unset unset unset unset kept kept "origin --prune-tags" +test_configured_prune unset unset true unset kept kept "" +test_configured_prune unset unset unset true kept kept "" + +# These will prune the tags +test_configured_prune unset unset unset unset pruned pruned "--prune --prune-tags" +test_configured_prune unset unset unset unset pruned pruned "origin --prune --prune-tags" +test_configured_prune true unset true unset pruned pruned "" +test_configured_prune unset true unset true pruned pruned "" + +# Check that remote.<name>.pruneTags overrides fetch.pruneTags as with +# remote.<name>.prune and fetch.prune +test_configured_prune false true false true pruned pruned "" +test_configured_prune true false true false kept kept "" + +# When --prune-tags is supplied it's ignored if an explict refspec is +# given, same for the configuration options. +test_configured_prune unset unset unset unset pruned kept \ + "--prune --prune-tags origin +refs/heads/*:refs/remotes/origin/*" +test_configured_prune unset unset true unset pruned kept \ + "--prune --prune-tags origin +refs/heads/*:refs/remotes/origin/*" +test_configured_prune unset unset unset true pruned kept \ + "--prune --prune-tags origin +refs/heads/*:refs/remotes/origin/*" + +test_configured_prune true unset true unset pruned pruned "" + test_expect_success 'all boundary commits are excluded' ' test_commit base && test_commit oneside &&
Signature · 1 line
2.15.1.424.g9478a66081
Re: [PATCH v3 11/11] fetch: add a --fetch-prune option and fetch.pruneTags config
Quoted 1 line · Ævar Arnfjörð Bjarmason
git -C {} config --replace-all remote.origin.fetch "+refs/tags/*:refs/tags/*" "^\+*refs/tags/\*:refs/tags/\*$"
Shouldn't the last arg be
'^+\*refs/tags/\*:refs/tags/\*$'
instead?
Re: [PATCH v3 11/11] fetch: add a --fetch-prune option and fetch.pruneTags config
On Wed, Jan 24 2018, Junio C. Hamano jotted:
Quoted 9 lines
Quoted 1 line · Ævar Arnfjörð Bjarmason
git -C {} config --replace-all remote.origin.fetch "+refs/tags/*:refs/tags/*" "^\+*refs/tags/\*:refs/tags/\*$"
Shouldn't the last arg be
'^+\*refs/tags/\*:refs/tags/\*$'
instead?
^+\* isn't a valid pattern.
This invocation is to replace both +refs/tags/[...] and refs/tags/[...] with +refs/tags/[...]. This would be more specific, i.e. ^\+?
git config --replace-all remote.origin.fetch "+refs/tags/*:refs/tags/*" "^\+?refs/tags/\*:refs/tags/\*$"
But I couldn't remeber offhand whether all regcomp() we use supports X?, whereas they definitely do support X*.
Re: [PATCH v3 11/11] fetch: add a --fetch-prune option and fetch.pruneTags config
Quoted 6 lines · Ævar Arnfjörð Bjarmason
On Wed, Jan 24 2018, Junio C. Hamano jotted:
Quoted 4 lines
Quoted 1 line · Ævar Arnfjörð Bjarmason
git -C {} config --replace-all remote.origin.fetch "+refs/tags/*:refs/tags/*" "^\+*refs/tags/\*:refs/tags/\*$"
Quoted 7 lines
Quoted 5 lines
Shouldn't the last arg be
'^+\*refs/tags/\*:refs/tags/\*$'
instead?
^+\* isn't a valid pattern.
Yikes, sorry for the noise. You did mean "there might be a plus sign at the beginning, but there may not be".
Re: [PATCH v3 00/11] document & test fetch pruning & add fetch.pruneTags
In reply to Ævar Arnfjörð BjarmasonQuoted 10 lines · Ævar Arnfjörð Bjarmason
Documentation/config.txt | 20 ++++- Documentation/fetch-options.txt | 18 +++- Documentation/git-fetch.txt | 76 +++++++++++++++++ Documentation/git-remote.txt | 14 +-- builtin/fetch.c | 37 +++++++- contrib/completion/git-completion.bash | 2 +- remote.c | 5 +- remote.h | 3 + t/t5510-fetch.sh | 152 ++++++++++++++++++++++++--------- 9 files changed, 275 insertions(+), 52 deletions(-)
I've queued this round but saw a few minor conflicts with another topi in flight, so please double check the result when I push the result out shortly.
Thanks.
Re: [PATCH v3 00/11] document & test fetch pruning & add fetch.pruneTags
On Wed, Jan 24 2018, Junio C. Hamano jotted:
Quoted 13 lines
Quoted 10 lines · Ævar Arnfjörð Bjarmason
Documentation/config.txt | 20 ++++- Documentation/fetch-options.txt | 18 +++- Documentation/git-fetch.txt | 76 +++++++++++++++++ Documentation/git-remote.txt | 14 +-- builtin/fetch.c | 37 +++++++- contrib/completion/git-completion.bash | 2 +- remote.c | 5 +- remote.h | 3 + t/t5510-fetch.sh | 152 ++++++++++++++++++++++++--------- 9 files changed, 275 insertions(+), 52 deletions(-)
Quoted 3 lines
I've queued this round but saw a few minor conflicts with another topi in flight, so please double check the result when I push the result out shortly.
Yikes. fetch.c is busy these days. Looked it over, looks good to me. In particular the potential logic error introduced by mis-merging "fetch:
refactor calculation of remote list" has a test which would fail.
Re: [PATCH v3 00/11] document & test fetch pruning & add fetch.pruneTags
In reply to Ævar Arnfjörð BjarmasonOn Tue, Jan 23 2018, Ævar Arnfjörð Bjarmason jotted:
Quoted 4 lines
I'm now just skipping quoting things like +refs/... on the command-line, which as grepping the rest of the test suite shows is fine, this eliminated the need for "fetch tests: double quote a variable for interpolation" so I've ejected it.
There's a segfault bug in 11/11, which wasn't found because the test suite doesn't test git fetch <url> just git fetch <named> and this is handled differently.
I'll send a fix soon, but don't merge this down from pu for now.
In order to test for that I brought that cmdline quoting patch back, I can't find a better way to do that, and in addition I have this similar WIP patch:
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 2f5bd966be..8fe4f3c13b 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -549,13 +549,39 @@ set_config_tristate () {
}
test_configured_prune () {
+ test_configured_prune_guts "$@" "name"
+ test_configured_prune_guts "$@" "link"
+}
+
+test_configured_prune_guts () {
fetch_prune=$1
remote_origin_prune=$2
expected_branch=$3
expected_tag=$4
cmdline=$5
-
- test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
+ mode=$6
+
+ if ! test -e prune-guts-setup
+ then
+ test_expect_success 'prune_guts setup' '
+ git -C one config remote.origin.url >one.remote-url &&
+ git -C one config remote.origin.fetch >one.remote-fetch &&
+ touch prune-guts-setup
+ '
+ fi
+
+ if test "$mode" = 'link'
+ then
+ remote_url="file://$(cat one.remote-url)"
+ remote_fetch="$(cat one.remote-fetch)"
+ cmdline_setup="\"$remote_url\" \"$remote_fetch\""
+ if test "$cmdline" != ""
+ then
+ cmdline=$(printf "%s" "$cmdline" | sed -e 's! origin! "'"$remote_url"'"!g')
+ fi
+ fi
+
+ test_expect_success "$mode prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
# make sure a newbranch is there in . and also in one
git branch -f newbranch &&
git tag -f newtag &&
@@ -563,7 +589,7 @@ test_configured_prune () {
cd one &&
test_unconfig fetch.prune &&
test_unconfig remote.origin.prune &&
- git fetch &&
+ git fetch '"$cmdline_setup"' &&
git rev-parse --verify refs/remotes/origin/newbranch &&
git rev-parse --verify refs/tags/newtag
) &&It'll be amended a bit more, but the general idea is there, because of how this whole quoting mess looks like I have to resort to the above hack outside of the test setup.
v2 cover document & test fetch pruning & add fetch.pruneTags
In reply to Ævar Arnfjörð BjarmasonAs noted in my [email protected] there was a bug I noticed in v3 where it would segfault on some git-fetch invocations, but there were not tests anywhere that caught that.
So in addition to fixing that issue, this fleshens out the testing being set up as part of this series so we'll test those sorts of invocations. It would segfault on some git fetch <url>, not git fetch <name>.
Ævar Arnfjörð Bjarmason (17):
fetch: don't redundantly NULL something calloc() gave us
Rephrased commit message.
fetch: trivially refactor assignment to ref_nr
New, makes a subsequent change smaller.
fetch: stop accessing "remote" variable indirectly
Typo fix in commit message noted by Junio.
remote: add a macro for "refs/tags/*:refs/tags/*"
New, makes a subsequent change smaller.
fetch tests: refactor in preparation for testing tag pruning fetch tests: re-arrange arguments for future readability fetch tests: add a tag to be deleted to the pruning tests
No changes.
fetch tests: test --prune and refspec interaction
Changed +refs/tags/*:refs/tags/ to refs/tags/*:refs/tags/. No functional difference, since git doesn't care. Just to be consistent with the macro added earlier & doing the same in commit messages & tests later in the series.
fetch tests: double quote a variable for interpolation
Now back from an earlier version, needed for a later change.
fetch tests: expand case/esac for later change
New, makes the next patch smaller / easier to review.
fetch tests: fetch <url> <spec> as well as fetch [<remote>]
For all git fetch <name> we now run another version of the test where we test an equivalent git fetch <url>. This sort of exhaustive testing was missing in our whole test suite, and would have caught the segfault in v3.
git fetch doc: add a new section to explain the ins & outs of pruning git remote doc: correct dangerous lies about what prune does git-fetch & config doc: link to the new PRUNING section
No changes except omitting the "+" in front of refs/tags/[...] as noted above.
fetch tests: add scaffolding for the new fetch.pruneTags
Ditto "+" change + minor changes carried over from previous patches.
fetch: add a --fetch-prune option and fetch.pruneTags config
The bug in v3 was that the remote->fetch variable needs to chaned in lockstep with remote->fetch_refspec, but only the latter was changed. Codepaths that fetched by URL would under --prune-tags expect as many items in both, and segfault on the access to remote->fetch.
As explained in the amended commit message the API is not amenable to ALLOC_GROW, so there's now a add_prune_tags_to_fetch_refspec() function in remote.c which adds the new element to remote->fetch via xrealloc() + memcpy().
Careful review of that most welcome.
There's lots more tests that catch the case where it segfaulted.
fetch: make the --fetch-prune work with <url>
The previous patch was changed to document that this wouldn't work:
git fetch <url of origin> --prune --prune-tag
This makes it work, at the cost of some complexity in fetch_one(). I think it makes sense to keep this, I just wanted to split it off from the previous patch to clearly show the hoops we need to jump through for that one case.
Documentation/config.txt | 20 ++- Documentation/fetch-options.txt | 17 ++- Documentation/git-fetch.txt | 87 ++++++++++++ Documentation/git-remote.txt | 14 +- builtin/fetch.c | 54 ++++++-- contrib/completion/git-completion.bash | 2 +- remote.c | 15 ++ remote.h | 5 + t/t5510-fetch.sh | 242 +++++++++++++++++++++++++++------ 9 files changed, 395 insertions(+), 61 deletions(-)
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 1/17 fetch: don't redundantly NULL something calloc() gave us
Stop redundantly NULL-ing the last element of the refs structure, which was retrieved via calloc(), and is thus guaranteed to be pre-NULL'd.
This code dates back to b888d61c83 ("Make fetch a builtin", 2007-09-10), where wasn't any reason to do this back then either, it's just boilerplate left over from when git-fetch was initially introduced.
The motivation for this change was to make a subsequent change which would also modify the refs variable smaller, since it won't have to copy this redundant "NULL the last + 1 item" pattern.
We may not end up keeping that change, but as this pattern is still pointless, so let's fix it.
--- builtin/fetch.c | 1 - 1 file changed, 1 deletion(-)
builtin/fetch.c +0 -1
@@ -1302,7 +1302,6 @@ static int fetch_one(struct remote *remote, int argc, const char **argv) }else refs[j++] = argv[i]; } - refs[j] =NULL ; ref_nr = j; }
Signature · 1 line
2.15.1.424.g9478a66081
Re: [PATCH v2 01/17] fetch: don't redundantly NULL something calloc() gave us
Quoted 8 lines · Ævar Arnfjörð Bjarmason
Stop redundantly NULL-ing the last element of the refs structure, which was retrieved via
calloc(), and is thus guaranteed to be pre-NULL'd.[...]Signed-off-by Ævar Arnfjörð Bjarmason <[email protected]>---
builtin/fetch.c static int fetch_one(struct remote *remote, int argc, const char **argv) · line 1306
Quoted 6 lines
builtin/fetch.c +0
-1@@ -1302,7 +1302,6 @@ static int fetch_one(struct remote *remote, int argc, const char **argv) }else refs[j++] = argv[i]; } - refs[j] =NULL ; ref_nr = j; }
This is purely subjective, and I neglected to mention it as early as v1, but I find that this change hurts readability. Specifically, as I'm scanning or reading code, explicit termination conditions, like this NULL assignment, are things I'm expecting to see; they're part of the idiom of the language. When they're missing, I have to stop, go back, and study the code more carefully to see if the "missing bit" is a bug or is intentional. And, it's easy to misread xcalloc() as xmalloc(), meaning that it's likely that one studying the code would conclude that the missing NULL assignment is a bug.
If anything, if this patch wants to eliminate redundancy, I'd expect it to change xcalloc() to xmalloc() and keep the NULL assignment, thus leaving the idiom intact.
Re: [PATCH v2 01/17] fetch: don't redundantly NULL something calloc() gave us
On Fri, Feb 09 2018, Eric Sunshine jotted:
Quoted 27 lines
Quoted 8 lines · Ævar Arnfjörð Bjarmason
Stop redundantly NULL-ing the last element of the refs structure, which was retrieved via
calloc(), and is thus guaranteed to be pre-NULL'd.[...]Signed-off-by Ævar Arnfjörð Bjarmason <[email protected]>---
builtin/fetch.c static int fetch_one(struct remote *remote, int argc, const char **argv) · line 1306
Quoted 6 lines
builtin/fetch.c +0
-1@@ -1302,7 +1302,6 @@ static int fetch_one(struct remote *remote, int argc, const char **argv) }else refs[j++] = argv[i]; } - refs[j] =NULL ; ref_nr = j; }This is purely subjective, and I neglected to mention it as early as v1, but I find that this change hurts readability. Specifically, as I'm scanning or reading code, explicit termination conditions, like this NULL assignment, are things I'm expecting to see; they're part of the idiom of the language. When they're missing, I have to stop, go back, and study the code more carefully to see if the "missing bit" is a bug or is intentional. And, it's easy to misread
xcalloc()asxmalloc(), meaning that it's likely that one studying the code would conclude that the missing NULL assignment is a bug.
Quoted 3 lines
If anything, if this patch wants to eliminate redundancy, I'd expect it to change
xcalloc()toxmalloc()and keep the NULL assignment, thus leaving the idiom intact.
Thanks for all your review, really appreciate it.
I'm going to keep this as-is, reasons:
* With calloc it's easier to look at the values in a debugger, you get NULLs instead of some random garbage for e.g. the ref src/dst. It makes it clear it's unset / the tail value.
* Ditto fewer things to step through / inpect in a debugger. E.g. I have a dump of variables before/after in the debugger, with assignments like this it's just adding verbosity & something to eyeball for something that's never going to change.
* If there's a bug in the code using calloc is likely to reveal it sooner, since you'll be deref-ing NULL instead of some stray (possibly still valid) pointer you got from malloc.
* It looks more in line with established idioms in the codebase. I wasn't able to find other cases where we were double-NULL ing calloc'd data, but rather stuff like this:
git grep -W '\bpattern\b' -- '*/ls-remote.c'
PATCH v2 2/17 fetch: trivially refactor assignment to ref_nr
In reply to Ævar Arnfjörð BjarmasonTrivially refactor an assignment to make a subsequent patch smaller. The "ref_nr" variable is initialized to 0 earlier, just as "j" is, and "j" is only incremented in that loop, so this change isn't a logic error.
This change makes a subsequent change which splits the incrementing of "ref_nr" into two blocks.
--- builtin/fetch.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
builtin/fetch.c +1 -1
@@ -1301,8 +1301,8 @@ static int fetch_one(struct remote *remote, int argc, const char **argv) argv[i], argv[i]); }else refs[j++] = argv[i]; + ref_nr++; } - ref_nr = j; }sigchain_push_common (unlock_pack_on_signal);
Signature · 1 line
2.15.1.424.g9478a66081
Re: [PATCH v2 02/17] fetch: trivially refactor assignment to ref_nr
Quoted 7 lines · Ævar Arnfjörð Bjarmason
Trivially refactor an assignment to make a subsequent patch smaller. The "
ref_nr" variable is initialized to 0 earlier, just as "j" is, and "j" is only incremented in that loop, so this change isn't a logic error.This change makes a subsequent change which splits the incrementing of "
ref_nr" into two blocks.
"This change _simplifies_ a subsequent..."
or
"This change make a subsequent change _easier_..."
Quoted 1 line
Signed-off-by Ævar Arnfjörð Bjarmason <[email protected]>
PATCH v2 3/17 fetch: stop accessing "remote" variable indirectly
In reply to Ævar Arnfjörð BjarmasonAccess the "remote" variable passed to the fetch_one() directly rather than through the gtransport wrapper struct constructed in this function for other purposes.
This makes the code more readable, as it's now obvious that the remote struct doesn't somehow get munged by the prepare_transport() function above, which takes the "remote" struct as an argument and constructs the "gtransport" struct, containing among other things the "remote" struct.
A subsequent change will copy this pattern to access a new remote->prune_tags field, but without the use of the gtransport variable. It's useful once that change lands to see that the two pieces of code behave exactly the same.
This pattern of accessing the container struct was added in 737c5a9cde ("fetch: make --prune configurable", 2013-07-13) when this code was initially introduced.
--- builtin/fetch.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
builtin/fetch.c +2 -2
@@ -1280,8 +1280,8 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)if (prune <0 ) {/* no command line request */ -if (0 <= gtransport->remote->prune) - prune = gtransport->remote->prune; +if (0 <= remote->prune) + prune = remote->prune;else if (0 <= fetch_prune_config) prune = fetch_prune_config;else
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 4/17 remote: add a macro for "refs/tags/*:refs/tags/*"
In reply to Ævar Arnfjörð BjarmasonAdd a macro with the refspec string "refs/tags/*:refs/tags/*". There's been a pre-defined struct version of this since e0aaa29ff3 ("Have a constant extern refspec for "--tags"", 2008-04-17), but nothing that could be passed to e.g. add_fetch_refspec().
This will be used in subsequent commits to avoid hardcoding this string in multiple places.
--- remote.c | 1 + remote.h | 2 ++ 2 files changed, 3 insertions(+)
remote.c +1 -0
@@ -22,6 +22,7 @@ static struct refspec s_tag_refspec = {"refs/tags/*" }; +/* See TAG_REFSPEC for the string version */ const struct refspec *tag_refspec = &s_tag_refspec;struct counted_string {
remote.h +2 -0
@@ -297,4 +297,6 @@ extern int parseopt_push_cas_option(const struct option *, const char *arg, intextern int is_empty_cas (const struct push_cas_option *);void apply_push_cas (struct push_cas_option *,struct remote *,struct ref *); +#define TAG_REFSPEC "refs/tags/*:refs/tags/*" +#endif
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 5/17 fetch tests: refactor in preparation for testing tag pruning
In reply to Ævar Arnfjörð BjarmasonIn a subsequent commit this function will learn to test for tag pruning, prepare for that by making space for more variables, and making it clear that "expected" here refers to branches.
--- t/t5510-fetch.sh | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
t/t5510-fetch.sh +6 -3
@@ -549,9 +549,12 @@ set_config_tristate () { } test_configured_prune () { - fetch_prune=$1 remote_origin_prune=$2 cmdline=$3 expected=$4 + fetch_prune=$1 + remote_origin_prune=$2 + cmdline=$3 + expected_branch=$4 - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; $4" ' + test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && (
@@ -572,7 +575,7 @@ test_configured_prune () { set_config_tristate remote.origin.prune $remote_origin_prune && git fetch $cmdline && - case "$expected" in + case "$expected_branch" in pruned) test_must_fail git rev-parse --verify refs/remotes/origin/newbranch ;;
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 6/17 fetch tests: re-arrange arguments for future readability
In reply to Ævar Arnfjörð BjarmasonRe-arrange the arguments to the test_configured_prune() function used in this test to pass the arguments to --fetch last. A subsequent change will test for more elaborate fetch arguments, including long refspecs. It'll be more readable to be able to wrap those on a new line of their own.
--- t/t5510-fetch.sh | 82 ++++++++++++++++++++++++++++++-------------------------- 1 file changed, 44 insertions(+), 38 deletions(-)
t/t5510-fetch.sh +44 -38
@@ -551,10 +551,10 @@ set_config_tristate () { test_configured_prune () { fetch_prune=$1 remote_origin_prune=$2 - cmdline=$3 - expected_branch=$4 + expected_branch=$3 + cmdline=$4 - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" ' + test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && (
@@ -587,41 +587,47 @@ test_configured_prune () { ' } -test_configured_prune unset unset "" kept -test_configured_prune unset unset "--no-prune" kept -test_configured_prune unset unset "--prune" pruned - -test_configured_prune false unset "" kept -test_configured_prune false unset "--no-prune" kept -test_configured_prune false unset "--prune" pruned - -test_configured_prune true unset "" pruned -test_configured_prune true unset "--prune" pruned -test_configured_prune true unset "--no-prune" kept - -test_configured_prune unset false "" kept -test_configured_prune unset false "--no-prune" kept -test_configured_prune unset false "--prune" pruned - -test_configured_prune false false "" kept -test_configured_prune false false "--no-prune" kept -test_configured_prune false false "--prune" pruned - -test_configured_prune true false "" kept -test_configured_prune true false "--prune" pruned -test_configured_prune true false "--no-prune" kept - -test_configured_prune unset true "" pruned -test_configured_prune unset true "--no-prune" kept -test_configured_prune unset true "--prune" pruned - -test_configured_prune false true "" pruned -test_configured_prune false true "--no-prune" kept -test_configured_prune false true "--prune" pruned - -test_configured_prune true true "" pruned -test_configured_prune true true "--prune" pruned -test_configured_prune true true "--no-prune" kept +# $1 config: fetch.prune +# $2 config: remote.<name>.prune +# $3 expect: branch to be pruned? +# $4 git-fetch $cmdline: +# +# $1 $2 $3 $4 +test_configured_prune unset unset kept "" +test_configured_prune unset unset kept "--no-prune" +test_configured_prune unset unset pruned "--prune" + +test_configured_prune false unset kept "" +test_configured_prune false unset kept "--no-prune" +test_configured_prune false unset pruned "--prune" + +test_configured_prune true unset pruned "" +test_configured_prune true unset pruned "--prune" +test_configured_prune true unset kept "--no-prune" + +test_configured_prune unset false kept "" +test_configured_prune unset false kept "--no-prune" +test_configured_prune unset false pruned "--prune" + +test_configured_prune false false kept "" +test_configured_prune false false kept "--no-prune" +test_configured_prune false false pruned "--prune" + +test_configured_prune true false kept "" +test_configured_prune true false pruned "--prune" +test_configured_prune true false kept "--no-prune" + +test_configured_prune unset true pruned "" +test_configured_prune unset true kept "--no-prune" +test_configured_prune unset true pruned "--prune" + +test_configured_prune false true pruned "" +test_configured_prune false true kept "--no-prune" +test_configured_prune false true pruned "--prune" + +test_configured_prune true true pruned "" +test_configured_prune true true pruned "--prune" +test_configured_prune true true kept "--no-prune" test_expect_success 'all boundary commits are excluded' ' test_commit base &&
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 7/17 fetch tests: add a tag to be deleted to the pruning tests
In reply to Ævar Arnfjörð BjarmasonAdd a tag to be deleted to the fetch --prune tests. The tag is always kept for now, which is the expected behavior, but now I can add a test for tag pruning in a later commit.
--- t/t5510-fetch.sh | 93 ++++++++++++++++++++++++++++++++------------------------ 1 file changed, 53 insertions(+), 40 deletions(-)
t/t5510-fetch.sh +53 -40
@@ -552,21 +552,25 @@ test_configured_prune () { fetch_prune=$1 remote_origin_prune=$2 expected_branch=$3 - cmdline=$4 + expected_tag=$4 + cmdline=$5 - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" ' + test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && + git tag -f newtag && ( cd one && test_unconfig fetch.prune && test_unconfig remote.origin.prune && git fetch && - git rev-parse --verify refs/remotes/origin/newbranch + git rev-parse --verify refs/remotes/origin/newbranch && + git rev-parse --verify refs/tags/newtag ) && # now remove it git branch -d newbranch && + git tag -d newtag && # then test (
@@ -582,6 +586,14 @@ test_configured_prune () { kept) git rev-parse --verify refs/remotes/origin/newbranch ;; + esac && + case "$expected_tag" in + pruned) + test_must_fail git rev-parse --verify refs/tags/newtag + ;; + kept) + git rev-parse --verify refs/tags/newtag + ;; esac ) '
@@ -590,44 +602,45 @@ test_configured_prune () { # $1 config: fetch.prune # $2 config: remote.<name>.prune # $3 expect: branch to be pruned? -# $4 git-fetch $cmdline: +# $4 expect: tag to be pruned? +# $5 git-fetch $cmdline: # -# $1 $2 $3 $4 -test_configured_prune unset unset kept "" -test_configured_prune unset unset kept "--no-prune" -test_configured_prune unset unset pruned "--prune" - -test_configured_prune false unset kept "" -test_configured_prune false unset kept "--no-prune" -test_configured_prune false unset pruned "--prune" - -test_configured_prune true unset pruned "" -test_configured_prune true unset pruned "--prune" -test_configured_prune true unset kept "--no-prune" - -test_configured_prune unset false kept "" -test_configured_prune unset false kept "--no-prune" -test_configured_prune unset false pruned "--prune" - -test_configured_prune false false kept "" -test_configured_prune false false kept "--no-prune" -test_configured_prune false false pruned "--prune" - -test_configured_prune true false kept "" -test_configured_prune true false pruned "--prune" -test_configured_prune true false kept "--no-prune" - -test_configured_prune unset true pruned "" -test_configured_prune unset true kept "--no-prune" -test_configured_prune unset true pruned "--prune" - -test_configured_prune false true pruned "" -test_configured_prune false true kept "--no-prune" -test_configured_prune false true pruned "--prune" - -test_configured_prune true true pruned "" -test_configured_prune true true pruned "--prune" -test_configured_prune true true kept "--no-prune" +# $1 $2 $3 $4 $5 +test_configured_prune unset unset kept kept "" +test_configured_prune unset unset kept kept "--no-prune" +test_configured_prune unset unset pruned kept "--prune" + +test_configured_prune false unset kept kept "" +test_configured_prune false unset kept kept "--no-prune" +test_configured_prune false unset pruned kept "--prune" + +test_configured_prune true unset pruned kept "" +test_configured_prune true unset pruned kept "--prune" +test_configured_prune true unset kept kept "--no-prune" + +test_configured_prune unset false kept kept "" +test_configured_prune unset false kept kept "--no-prune" +test_configured_prune unset false pruned kept "--prune" + +test_configured_prune false false kept kept "" +test_configured_prune false false kept kept "--no-prune" +test_configured_prune false false pruned kept "--prune" + +test_configured_prune true false kept kept "" +test_configured_prune true false pruned kept "--prune" +test_configured_prune true false kept kept "--no-prune" + +test_configured_prune unset true pruned kept "" +test_configured_prune unset true kept kept "--no-prune" +test_configured_prune unset true pruned kept "--prune" + +test_configured_prune false true pruned kept "" +test_configured_prune false true kept kept "--no-prune" +test_configured_prune false true pruned kept "--prune" + +test_configured_prune true true pruned kept "" +test_configured_prune true true pruned kept "--prune" +test_configured_prune true true kept kept "--no-prune" test_expect_success 'all boundary commits are excluded' ' test_commit base &&
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 8/17 fetch tests: test --prune and refspec interaction
In reply to Ævar Arnfjörð BjarmasonAdd a test for the interaction between explicitly provided refspecs and fetch.prune.
There's no point in adding this boilerplate to every combination of unset/false/true, it's instructive and sufficient to show that no matter if the variable is unset, false or true the refspec on the command-line overrides any configuration variable.
--- t/t5510-fetch.sh | 12 ++++++++++++ 1 file changed, 12 insertions(+)
t/t5510-fetch.sh +12 -0
@@ -609,6 +609,10 @@ test_configured_prune () { test_configured_prune unset unset kept kept "" test_configured_prune unset unset kept kept "--no-prune" test_configured_prune unset unset pruned kept "--prune" +test_configured_prune unset unset kept pruned \ + "--prune origin refs/tags/*:refs/tags/*" +test_configured_prune unset unset pruned pruned \ + "--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" test_configured_prune false unset kept kept "" test_configured_prune false unset kept kept "--no-prune"
@@ -625,6 +629,10 @@ test_configured_prune unset false pruned kept "--prune" test_configured_prune false false kept kept "" test_configured_prune false false kept kept "--no-prune" test_configured_prune false false pruned kept "--prune" +test_configured_prune false false kept pruned \ + "--prune origin refs/tags/*:refs/tags/*" +test_configured_prune false false pruned pruned \ + "--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" test_configured_prune true false kept kept "" test_configured_prune true false pruned kept "--prune"
@@ -641,6 +649,10 @@ test_configured_prune false true pruned kept "--prune" test_configured_prune true true pruned kept "" test_configured_prune true true pruned kept "--prune" test_configured_prune true true kept kept "--no-prune" +test_configured_prune true true kept pruned \ + "--prune origin refs/tags/*:refs/tags/*" +test_configured_prune true true pruned pruned \ + "--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" test_expect_success 'all boundary commits are excluded' ' test_commit base &&
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 9/17 fetch tests: double quote a variable for interpolation
In reply to Ævar Arnfjörð BjarmasonIf the $cmdline variable contains arguments with spaces they won't be interpolated correctly, since the body of the test is single quoted, and because test-lib.sh does its own eval().
This will be used in a subsequent commit to pass arguments that need to be quoted to git-fetch, i.e. a file:// path to fetch, which will have a space in it.
--- t/t5510-fetch.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
t/t5510-fetch.sh +1 -1
@@ -578,7 +578,7 @@ test_configured_prune () { set_config_tristate fetch.prune $fetch_prune && set_config_tristate remote.origin.prune $remote_origin_prune && - git fetch $cmdline && + git fetch '"$cmdline"' && case "$expected_branch" in pruned) test_must_fail git rev-parse --verify refs/remotes/origin/newbranch
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 10/17 fetch tests: expand case/esac for later change
In reply to Ævar Arnfjörð BjarmasonExpand a compact case/esac statement for a later change that'll add more logic to the body of the "*" case. This is a whitespace-only change.
--- t/t5510-fetch.sh | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
t/t5510-fetch.sh +6 -2
@@ -543,8 +543,12 @@ test_expect_success "should be able to fetch with duplicate refspecs" ' set_config_tristate () { # var=$1 val=$2 case "$2" in - unset) test_unconfig "$1" ;; - *) git config "$1" "$2" ;; + unset) + test_unconfig "$1" + ;; + *) + git config "$1" "$2" + ;; esac }
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 11/17 fetch tests: fetch <url> <spec> as well as fetch [<remote>]
In reply to Ævar Arnfjörð BjarmasonWhen a remote URL is supplied on the command-line the internals of the fetch are different, in particular the code in get_ref_map(). An earlier version of the subsequent fetch.pruneTags patch hid a segfault because the difference wasn't tested for.
Now all the tests are run as both of the variants of:
git fetch
git -c [...] fetch $(git config remote.origin.url) $(git config remote.origin.fetch)I'm using -c because while the [fetch] config just set by set_config_tristate will be picked up, the remote.origin.* config won't override it as intended.
Work around that and turn this into a purely command-line test by always setting the variables on the command-line, and translate any setting of remote.origin.X into fetch.X.
The reason for choosing the names "name" and "link" as opposed to e.g. "named" and "url" is because they're the same length, which makes the test output easier to read as it will be aligned.
Due to shellscript quoting madness it's not worthwhile to do all of this within a test_expect_success, but do the parts that can easily be done there, including the one-time setting of variables that don't change between runs to be used by subsequent runs in the 'prune_type setup' test.
--- t/t5510-fetch.sh | 47 +++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 43 insertions(+), 4 deletions(-)
t/t5510-fetch.sh +43 -4
@@ -548,18 +548,52 @@ set_config_tristate () { ;; *) git config "$1" "$2" + key=$(echo $1 | sed -e 's/^remote\.origin/fetch/') + git_fetch_c="$git_fetch_c -c $key=$2" ;; esac } test_configured_prune () { + test_configured_prune_type "$@" "name" + test_configured_prune_type "$@" "link" +} + +test_configured_prune_type () { fetch_prune=$1 remote_origin_prune=$2 expected_branch=$3 expected_tag=$4 cmdline=$5 - - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" ' + mode=$6 + + if ! test -e prune-type-setup-done + then + test_expect_success 'prune_type setup' ' + git -C one config remote.origin.url >one.remote-url && + git -C one config remote.origin.fetch >one.remote-fetch && + remote_url="file://$(cat one.remote-url)" && + remote_fetch="$(cat one.remote-fetch)" && + cmdline_setup="\"$remote_url\" \"$remote_fetch\"" && + touch prune-type-setup-done + ' + fi + + if test "$mode" = 'link' + then + new_cmdline="" + + if test "$cmdline" = "" + then + new_cmdline=$cmdline_setup + else + new_cmdline=$(printf "%s" "$cmdline" | perl -pe 's[origin(?!/)]["'"$remote_url"'"]g') + fi + + cmdline="$new_cmdline" + fi + + test_expect_success "$mode prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && git tag -f newtag &&
@@ -567,7 +601,7 @@ test_configured_prune () { cd one && test_unconfig fetch.prune && test_unconfig remote.origin.prune && - git fetch && + git fetch '"$cmdline_setup"' && git rev-parse --verify refs/remotes/origin/newbranch && git rev-parse --verify refs/tags/newtag ) &&
@@ -579,10 +613,15 @@ test_configured_prune () { # then test ( cd one && + git_fetch_c="" && set_config_tristate fetch.prune $fetch_prune && set_config_tristate remote.origin.prune $remote_origin_prune && - git fetch '"$cmdline"' && + if test "$mode" != "link" + then + git_fetch_c="" + fi && + git$git_fetch_c fetch '"$cmdline"' && case "$expected_branch" in pruned) test_must_fail git rev-parse --verify refs/remotes/origin/newbranch
Signature · 1 line
2.15.1.424.g9478a66081
Re: [PATCH v2 11/17] fetch tests: fetch <url> <spec> as well as fetch [<remote>]
Quoted 22 lines · Ævar Arnfjörð Bjarmason
When a remote URL is supplied on the command-line the internals of the fetch are different, in particular the code in
get_ref_map(). An earlier version of the subsequent fetch.pruneTags patch hid a segfault because the difference wasn't tested for.Now all the tests are run as both of the variants of:
git fetch git -c [...] fetch $(git config remote.origin.url) $(git config remote.origin.fetch)I'm using -c because while the [fetch] config just set by
set_config_tristatewill be picked up, the remote.origin.* config won't override it as intended.Work around that and turn this into a purely command-line test by always setting the variables on the command-line, and translate any setting of remote.origin.X into fetch.X.
[...]Signed-off-by Ævar Arnfjörð Bjarmason <[email protected]>---
t/t5510-fetch.sh set_config_tristate () { · line 550
Quoted 3 lines
t/t5510-fetch.sh +1
-0@@ -548,18 +548,52 @@ set_config_tristate () { *) git config "$1" "$2" + key=$(echo $1 | sed -e 's/^remote\.origin/fetch/')
Faster (thus more Windows-friendly) assuming that $1 always starts with "remote.origin":
key=fetch${u#remote.origin}Quoted 12 lines
t/t5510-fetch.sh +2
-1+ git_fetch_c="$git_fetch_c -c $key=$2" ;; esac } +test_configured_prune_type () { fetch_prune=$1 remote_origin_prune=$2 expected_branch=$3 expected_tag=$4 cmdline=$5 -
t/t5510-fetch.sh set_config_tristate () { · line 576
Quoted 10 lines
t/t5510-fetch.sh +9
-1- test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" ' + mode=$6 + + if ! test -e prune-type-setup-done + then + test_expect_success 'prune_type setup' ' + git -C one config remote.origin.url >one.remote-url && + git -C one config remote.origin.fetch >one.remote-fetch && + remote_url="file://$(cat one.remote-url)" && + remote_fetch="$(cat one.remote-fetch)" &&
Is there a reason that these values need to be captured to files (which are otherwise not used) before being assigned to variables? That is, wouldn't this work?
remote_url="file://$(git -C one config remote.origin.url)" &&
remote_fetch="$(git -C one config remote.origin.fetch)" &&t/t5510-fetch.sh set_config_tristate () { · line 578
Quoted 2 lines
t/t5510-fetch.sh +2
-0+ cmdline_setup="\"$remote_url\" \"$remote_fetch\"" && + touch prune-type-setup-done
Why does "prune-type-setup-done" need to be a file rather than a simple shell variable (which is global by default even when assigned inside test_expect_success)?
Also, since the purpose of this code seems to compute 'cmdline_setup' just once, can't you do away with 'prune-type-setup-done' altogether and change:
if ! test -e prune-type-setup-done
to:
if test -z "$cmdline_setup"
Quoted 2 lines
t/t5510-fetch.sh +2
-0+ ' + fi
Re: [PATCH v2 11/17] fetch tests: fetch <url> <spec> as well as fetch [<remote>]
On Fri, Feb 09 2018, Eric Sunshine jotted:
Quoted 31 lines
Quoted 22 lines · Ævar Arnfjörð Bjarmason
When a remote URL is supplied on the command-line the internals of the fetch are different, in particular the code in
get_ref_map(). An earlier version of the subsequent fetch.pruneTags patch hid a segfault because the difference wasn't tested for.Now all the tests are run as both of the variants of:
git fetch git -c [...] fetch $(git config remote.origin.url) $(git config remote.origin.fetch)I'm using -c because while the [fetch] config just set by
set_config_tristatewill be picked up, the remote.origin.* config won't override it as intended.Work around that and turn this into a purely command-line test by always setting the variables on the command-line, and translate any setting of remote.origin.X into fetch.X.
[...]Signed-off-by Ævar Arnfjörð Bjarmason <[email protected]>---
t/t5510-fetch.sh set_config_tristate () { · line 550
Quoted 3 lines
t/t5510-fetch.sh +1
-0@@ -548,18 +548,52 @@ set_config_tristate () { *) git config "$1" "$2" + key=$(echo $1 | sed -e 's/^remote\.origin/fetch/')Faster (thus more Windows-friendly) assuming that $1 always starts with "remote.origin":
Quoted 1 line
key=fetch${u#remote.origin}Tests fail with this and I'm not excited to be the first user in git's test suite to use some novel shell feature, no existing uses of ${u[...].
I also think stuff like this is on the wrong side of cleverness v.s. benefit. I can't find any reference to this syntax in bash or dash manpages (forward-search "${u"), but echo | sed is obvious, and it's not going to make a big difference for Windows.
Quoted 27 lines
Quoted 12 lines
t/t5510-fetch.sh +2
-1+ git_fetch_c="$git_fetch_c -c $key=$2" ;; esac } +test_configured_prune_type () { fetch_prune=$1 remote_origin_prune=$2 expected_branch=$3 expected_tag=$4 cmdline=$5 -t/t5510-fetch.sh set_config_tristate () { · line 576
Quoted 10 lines
t/t5510-fetch.sh +9
-1- test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" ' + mode=$6 + + if ! test -e prune-type-setup-done + then + test_expect_success 'prune_type setup' ' + git -C one config remote.origin.url >one.remote-url && + git -C one config remote.origin.fetch >one.remote-fetch && + remote_url="file://$(cat one.remote-url)" && + remote_fetch="$(cat one.remote-fetch)" &&Is there a reason that these values need to be captured to files (which are otherwise not used) before being assigned to variables? That is, wouldn't this work?
Quoted 2 lines
remote_url="file://$(git -C one config remote.origin.url)" &&
remote_fetch="$(git -C one config remote.origin.fetch)" &&Nope, I'll just do that. This was cruft left over from an earlier version which I didn't clean up.
Quoted 11 lines
t/t5510-fetch.sh set_config_tristate () { · line 578
Quoted 2 lines
t/t5510-fetch.sh +2
-0+ cmdline_setup="\"$remote_url\" \"$remote_fetch\"" && + touch prune-type-setup-doneWhy does "prune-type-setup-done" need to be a file rather than a simple shell variable (which is global by default even when assigned inside
test_expect_success)?Also, since the purpose of this code seems to compute '
cmdline_setup' just once, can't you do away with 'prune-type-setup-done' altogether and change:
Quoted 8 lines
if ! test -e prune-type-setup-done
to:
if test -z "$cmdline_setup"
Quoted 2 lines
t/t5510-fetch.sh +2
-0+ ' + fi
Yup, that's much simpler. Thanks.
Re: [PATCH v2 11/17] fetch tests: fetch <url> <spec> as well as fetch [<remote>]
Quoted 11 lines · Ævar Arnfjörð Bjarmason
Quoted 10 lines
t/t5510-fetch.sh set_config_tristate () { · line 550
Quoted 5 lines
t/t5510-fetch.sh +1
-0@@ -548,18 +548,52 @@ set_config_tristate () { *) git config "$1" "$2" + key=$(echo $1 | sed -e 's/^remote\.origin/fetch/')Faster (thus more Windows-friendly) assuming that $1 always starts with "remote.origin":
key=fetch${u#remote.origin}
Quoted 8 lines
Tests fail with this and I'm not excited to be the first user in git's test suite to use some novel shell feature, no existing uses of ${u[...].
I also think stuff like this is on the wrong side of cleverness v.s. benefit. I can't find any reference to this syntax in bash or dash manpages (forward-search "${u"), but echo | sed is obvious, and it's not going to make a big difference for Windows.
The "u" isn't the magic, it's the "#". I.e.:
key=fetch${1#remote.origin}and it's used all over the place in our scripts. I'm not sure why Eric wrote "u". :)
-Peff
Re: [PATCH v2 11/17] fetch tests: fetch <url> <spec> as well as fetch [<remote>]
Quoted 17 lines · Jeff King
Quoted 7 lines · Ævar Arnfjörð Bjarmason
Quoted 6 lines
Quoted 1 line
+ key=$(echo $1 | sed -e 's/^remote\.origin/fetch/')
Faster (thus more Windows-friendly) assuming that $1 always starts with "remote.origin":
key=fetch${u#remote.origin}Quoted 8 lines
Tests fail with this and I'm not excited to be the first user in git's test suite to use some novel shell feature, no existing uses of ${u[...].
I also think stuff like this is on the wrong side of cleverness v.s. benefit. I can't find any reference to this syntax in bash or dash manpages (forward-search "${u"), but echo | sed is obvious, and it's not going to make a big difference for Windows.
Quoted 6 lines
The "u" isn't the magic, it's the "#". I.e.:
key=fetch${1#remote.origin}and it's used all over the place in our scripts. I'm not sure why Eric wrote "u". :)
Because I was testing the expression interactively in the shell and assigned to a variable arbitrarily named "u". When I copy/pasted the working expression from the shell session into the email, I stupidly forgot to change the "u" to "1".
Re: [PATCH v2 11/17] fetch tests: fetch <url> <spec> as well as fetch [<remote>]
In reply to Ævar Arnfjörð BjarmasonQuoted 10 lines · Ævar Arnfjörð Bjarmason
Quoted 6 lines
Quoted 1 line
+ key=$(echo $1 | sed -e 's/^remote\.origin/fetch/')
Faster (thus more Windows-friendly) assuming that $1 always starts with "remote.origin":
key=fetch${u#remote.origin}Tests fail with this and I'm not excited to be the first user in git's test suite to use some novel shell feature, no existing uses of ${u[...].
s/u/1/, I think.
Re: [PATCH v2 11/17] fetch tests: fetch <url> <spec> as well as fetch [<remote>]
Quoted 7 lines · Junio C Hamano
Quoted 5 lines · Ævar Arnfjörð Bjarmason
Quoted 5 lines
Quoted 1 line
+ key=$(echo $1 | sed -e 's/^remote\.origin/fetch/')
Faster (thus more Windows-friendly) assuming that $1 always starts with "remote.origin":
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
key=fetch${u#remote.origin}Tests fail with this and I'm not excited to be the first user in git's test suite to use some novel shell feature, no existing uses of ${u[...].
s/u/1/, I think.
Ah, that's been already pointed out. Sorry for the noise.
PATCH v2 12/17 git fetch doc: add a new section to explain the ins & outs of pruning
In reply to Ævar Arnfjörð BjarmasonAdd a new section to canonically explain how remote reference pruning works, and how users should be careful about using it in conjunction with tag refspecs in particular.
A subsequent commit will update the git-remote documentation to refer to this section, and details the motivation for writing this in the first place.
--- Documentation/git-fetch.txt | 49 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+)
Documentation/git-fetch.txt +49 -0
@@ -99,6 +99,55 @@ The latter use of the `remote.<repository>.fetch` values can be overridden by giving the `--refmap=<refspec>` parameter(s) on the command line. +PRUNING +------- + +Git has a default disposition of keeping data unless it's explicitly +thrown away; this extends to holding onto local references to branches +on remotes that have themselves deleted those branches. + +If left to accumulate, these stale references might make performance +worse on big and busy repos that have a lot of branch churn, and +e.g. make the output of commands like `git branch -a --contains +<commit>` needlessly verbose, as well as impacting anything else +that'll work with the complete set of known references. + +These remote tracking references can be deleted as a one-off with +either of: + +------------------------------------------------ +# While fetching +$ git fetch --prune <name> + +# Only prune, don't fetch +$ git remote prune <name> +------------------------------------------------ + +To prune references as part of your normal workflow without needing to +remember to run that set `fetch.prune` globally, or +`remote.<name>.prune` per-remote in the config. See +linkgit:git-config[1]. + +Here's where things get tricky and more specific. The pruning feature +doesn't actually care about branches, instead it'll prune local <-> +remote references as a function of the refspec of the remote (see +`<refspec>` and <<CRTB,CONFIGURED REMOTE-TRACKING BRANCHES>> above). + +Therefore if the refspec for the remote includes +e.g. `refs/tags/*:refs/tags/*`, or you manually run e.g. `git fetch +--prune <name> "refs/tags/*:refs/tags/*"` it won't be stale remote +tracking branches that are deleted, but any local tag that doesn't +exist on the remote. + +This might not be what you expect, i.e. you want to prune remote +`<name>`, but also explicitly fetch tags from it, so when you fetch +from it you delete all your local tags, most of which may not have +come from the `<name>` remote in the first place. + +So be careful when using this with a refspec like +`refs/tags/*:refs/tags/*`, or any other refspec which might map +references from multiple remotes to the same local namespace. + OUTPUT ------
Signature · 1 line
2.15.1.424.g9478a66081
Re: [PATCH v2 12/17] git fetch doc: add a new section to explain the ins & outs of pruning
Quoted 8 lines · Ævar Arnfjörð Bjarmason
Add a new section to canonically explain how remote reference pruning works, and how users should be careful about using it in conjunction with tag refspecs in particular.
[...]Signed-off-by Ævar Arnfjörð Bjarmason <[email protected]>---
Documentation/git-fetch.txt The latter use of the `remote.<repository>.fetch` values can be · line 108
Quoted 10 lines
Documentation/git-fetch.txt +10
-0@@ -99,6 +99,55 @@ The latter use of the `remote.<repository>.fetch` values can be +PRUNING +------- +[...] +If left to accumulate, these stale references might make performance +worse on big and busy repos that have a lot of branch churn, and +e.g. make the output of commands like `git branch -a --contains +<commit>` needlessly verbose, as well as impacting anything else +that'll work with the complete set of known references. + +These remote tracking references can be deleted as a one-off with
I think we call these "remote-tracking" (note the hyphen), which are local but track something remote, rather than "remote tracking" (no hyphen) which would themselves be remote.
Documentation/git-fetch.txt The latter use of the `remote.<repository>.fetch` values can be · line 127
Quoted 12 lines
Documentation/git-fetch.txt +12
-0+either of: + +------------------------------------------------ +# While fetching +$ git fetch --prune <name> + +# Only prune, don't fetch +$ git remote prune <name> +------------------------------------------------ + +To prune references as part of your normal workflow without needing to +remember to run that set `fetch.prune` globally, or
s/that/&,/
Quoted 2 lines
Documentation/git-fetch.txt +2
-0+`remote.<name>.prune` per-remote in the config. See +linkgit:git-config[1].
PATCH v2 13/17 git remote doc: correct dangerous lies about what prune does
In reply to Ævar Arnfjörð BjarmasonThe "git remote prune <name>" command uses the same machinery as "git fetch <name> --prune", and shares all the same caveats, but its documentation has suggested that it'll just "delete stale remote-tracking branches under <name>".
This isn't true, and hasn't been true since at least v1.8.5.6 (the oldest version I could be bothered to test).
E.g. if "refs/tags/*:refs/tags/*" is explicitly set in the refspec of the remote it'll delete all local tags <name> doesn't know about.
Instead, briefly give the reader just enough of a hint that this option might constitute a shotgun aimed at their foot, and point them to the new PRUNING section in the git-fetch documentation which explains all the nuances of what this facility does.
See "[BUG] git remote prune removes local tags, depending on fetch config" (CACi5S_39wNrbfjLfn0xhCY+uewtFN2YmnAcRc86z6pjUTjWPHQ@mail.gmail.com) by Michael Giuffrida for the initial report.
--- Documentation/git-remote.txt | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
Documentation/git-remote.txt +9 -5
@@ -172,10 +172,14 @@ With `-n` option, the remote heads are not queried first with 'prune':: -Deletes all stale remote-tracking branches under <name>. -These stale branches have already been removed from the remote repository -referenced by <name>, but are still locally available in -"remotes/<name>". +Deletes stale references associated with <name>. By default stale +remote-tracking branches under <name>, but depending on global +configuration and the configuration of the remote we might even prune +local tags that haven't been pushed there. Equivalent to `git fetch +--prune <name>`, except that no new references will be fetched. ++ +See the PRUNING section of linkgit:git-fetch[1] for what it'll prune +depending on various configuration. + With `--dry-run` option, report what branches will be pruned, but do not actually prune them.
@@ -189,7 +193,7 @@ remotes.default is not defined, all remotes which do not have the configuration parameter remote.<name>.skipDefaultUpdate set to true will be updated. (See linkgit:git-config[1]). + -With `--prune` option, prune all the remotes that are updated. +With `--prune` option, run pruning against all the remotes that are updated. DISCUSSION
Signature · 1 line
2.15.1.424.g9478a66081
Re: [PATCH v2 13/17] git remote doc: correct dangerous lies about what prune does
Quoted 10 lines · Ævar Arnfjörð Bjarmason
The "git remote prune <name>" command uses the same machinery as "git fetch <name> --prune", and shares all the same caveats, but its documentation has suggested that it'll just "delete stale remote-tracking branches under <name>".
This isn't true, and hasn't been true since at least v1.8.5.6 (the oldest version I could be bothered to test).
E.g. if "refs/tags/*:refs/tags/*" is explicitly set in the refspec of the remote it'll delete all local tags <name> doesn't know about.
Not worth a re-roll as this is only the commit message: s/remote/&,/
More below...
Quoted 9 lines
Instead, briefly give the reader just enough of a hint that this option might constitute a shotgun aimed at their foot, and point them to the new PRUNING section in the git-fetch documentation which explains all the nuances of what this facility does.
[...]Signed-off-by Ævar Arnfjörð Bjarmason <[email protected]>---
Documentation/git-remote.txt With `-n` option, the remote heads are not queried first with · line 172
Quoted 5 lines
Documentation/git-remote.txt +1
-4@@ -172,10 +172,14 @@ With `-n` option, the remote heads are not queried first with -Deletes all stale remote-tracking branches under <name>. -These stale branches have already been removed from the remote repository -referenced by <name>, but are still locally available in -"remotes/<name>". +Deletes stale references associated with <name>. By default stale
Nit: s/default/&,/
Documentation/git-remote.txt With `-n` option, the remote heads are not queried first with
Quoted 3 lines
Documentation/git-remote.txt +3
-0+remote-tracking branches under <name>, but depending on global +configuration and the configuration of the remote we might even prune +local tags that haven't been pushed there.
This is a bit difficult to digest grammatically due to the phrase being incomplete. It could say either:
By default, stale ... under <name> _are deleted_, but depending...
or:
Deletes stale ... <name>; by default, stale...
PATCH v2 14/17 git-fetch & config doc: link to the new PRUNING section
In reply to Ævar Arnfjörð BjarmasonAmend the documentation for fetch.prune, fetch.<name>.prune and --prune to link to the recently added PRUNING section.
I'd have liked to link directly to it with "<<PRUNING>>" from fetch-options.txt, since it's included in git-fetch.txt (git-pull.txt also includes it, but doesn't include that option). However making a reference across files yields this error:
[...]/Documentation/git-fetch.xml:226: element xref: validity error : IDREF attribute linkend references an unknown ID "PRUNING"
--- Documentation/config.txt | 6 +++++- Documentation/fetch-options.txt | 3 +++ 2 files changed, 8 insertions(+), 1 deletion(-)
Documentation/config.txt +5 -1
@@ -1398,7 +1398,8 @@ fetch.unpackLimit:: fetch.prune:: If true, fetch will automatically behave as if the `--prune` - option was given on the command line. See also `remote.<name>.prune`. + option was given on the command line. See also `remote.<name>.prune` + and the PRUNING section of linkgit:git-fetch[1]. fetch.output:: Control how ref update status is printed. Valid values are
@@ -2944,6 +2945,9 @@ remote.<name>.prune:: remove any remote-tracking references that no longer exist on the remote (as if the `--prune` option was given on the command line). Overrides `fetch.prune` settings, if any. ++ +See also `remote.<name>.prune` and the PRUNING section of +linkgit:git-fetch[1]. remotes.<group>:: The list of remotes which are fetched by "git remote update
Documentation/fetch-options.txt +3 -0
@@ -74,6 +74,9 @@ ifndef::git-pull[] line or in the remote configuration, for example if the remote was cloned with the --mirror option), then they are also subject to pruning. ++ +See the PRUNING section below for more details. + endif::git-pull[] ifndef::git-pull[]
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 15/17 fetch tests: add scaffolding for the new fetch.pruneTags
In reply to Ævar Arnfjörð BjarmasonThe fetch.pruneTags configuration doesn't exist yet, but will be added in a subsequent commit. Since testing for it requires adding new parameters to the test_configured_prune function it's easier to review this patch first to assert that no functional changes are introduced yet.
--- t/t5510-fetch.sh | 92 ++++++++++++++++++++++++++++++-------------------------- 1 file changed, 50 insertions(+), 42 deletions(-)
t/t5510-fetch.sh +50 -42
@@ -562,10 +562,12 @@ test_configured_prune () { test_configured_prune_type () { fetch_prune=$1 remote_origin_prune=$2 - expected_branch=$3 - expected_tag=$4 - cmdline=$5 - mode=$6 + fetch_prune_tags=$3 + remote_origin_prune_tags=$4 + expected_branch=$5 + expected_tag=$6 + cmdline=$7 + mode=$8 if ! test -e prune-type-setup-done then
@@ -593,14 +595,16 @@ test_configured_prune_type () { cmdline="$new_cmdline" fi - test_expect_success "$mode prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" ' + test_expect_success "$mode prune fetch.prune=$1 remote.origin.prune=$2 fetch.pruneTags=$3 remote.origin.pruneTags=$4${7:+ $7}; branch:$5 tag:$6" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && git tag -f newtag && ( cd one && test_unconfig fetch.prune && + test_unconfig fetch.pruneTags && test_unconfig remote.origin.prune && + test_unconfig remote.origin.pruneTags && git fetch '"$cmdline_setup"' && git rev-parse --verify refs/remotes/origin/newbranch && git rev-parse --verify refs/tags/newtag
@@ -615,7 +619,9 @@ test_configured_prune_type () { cd one && git_fetch_c="" && set_config_tristate fetch.prune $fetch_prune && + set_config_tristate fetch.pruneTags $fetch_prune_tags && set_config_tristate remote.origin.prune $remote_origin_prune && + set_config_tristate remote.origin.pruneTags $remote_origin_prune_tags && if test "$mode" != "link" then
@@ -644,57 +650,59 @@ test_configured_prune_type () { # $1 config: fetch.prune # $2 config: remote.<name>.prune -# $3 expect: branch to be pruned? -# $4 expect: tag to be pruned? -# $5 git-fetch $cmdline: +# $3 config: fetch.pruneTags +# $4 config: remote.<name>.pruneTags +# $5 expect: branch to be pruned? +# $6 expect: tag to be pruned? +# $7 git-fetch $cmdline: # -# $1 $2 $3 $4 $5 -test_configured_prune unset unset kept kept "" -test_configured_prune unset unset kept kept "--no-prune" -test_configured_prune unset unset pruned kept "--prune" -test_configured_prune unset unset kept pruned \ +# $1 $2 $3 $4 $5 $6 $7 +test_configured_prune unset unset unset unset kept kept "" +test_configured_prune unset unset unset unset kept kept "--no-prune" +test_configured_prune unset unset unset unset pruned kept "--prune" +test_configured_prune unset unset unset unset kept pruned \ "--prune origin refs/tags/*:refs/tags/*" -test_configured_prune unset unset pruned pruned \ +test_configured_prune unset unset unset unset pruned pruned \ "--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" -test_configured_prune false unset kept kept "" -test_configured_prune false unset kept kept "--no-prune" -test_configured_prune false unset pruned kept "--prune" +test_configured_prune false unset unset unset kept kept "" +test_configured_prune false unset unset unset kept kept "--no-prune" +test_configured_prune false unset unset unset pruned kept "--prune" -test_configured_prune true unset pruned kept "" -test_configured_prune true unset pruned kept "--prune" -test_configured_prune true unset kept kept "--no-prune" +test_configured_prune true unset unset unset pruned kept "" +test_configured_prune true unset unset unset pruned kept "--prune" +test_configured_prune true unset unset unset kept kept "--no-prune" -test_configured_prune unset false kept kept "" -test_configured_prune unset false kept kept "--no-prune" -test_configured_prune unset false pruned kept "--prune" +test_configured_prune unset false unset unset kept kept "" +test_configured_prune unset false unset unset kept kept "--no-prune" +test_configured_prune unset false unset unset pruned kept "--prune" -test_configured_prune false false kept kept "" -test_configured_prune false false kept kept "--no-prune" -test_configured_prune false false pruned kept "--prune" -test_configured_prune false false kept pruned \ +test_configured_prune false false unset unset kept kept "" +test_configured_prune false false unset unset kept kept "--no-prune" +test_configured_prune false false unset unset pruned kept "--prune" +test_configured_prune false false unset unset kept pruned \ "--prune origin refs/tags/*:refs/tags/*" -test_configured_prune false false pruned pruned \ +test_configured_prune false false unset unset pruned pruned \ "--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" -test_configured_prune true false kept kept "" -test_configured_prune true false pruned kept "--prune" -test_configured_prune true false kept kept "--no-prune" +test_configured_prune true false unset unset kept kept "" +test_configured_prune true false unset unset pruned kept "--prune" +test_configured_prune true false unset unset kept kept "--no-prune" -test_configured_prune unset true pruned kept "" -test_configured_prune unset true kept kept "--no-prune" -test_configured_prune unset true pruned kept "--prune" +test_configured_prune unset true unset unset pruned kept "" +test_configured_prune unset true unset unset kept kept "--no-prune" +test_configured_prune unset true unset unset pruned kept "--prune" -test_configured_prune false true pruned kept "" -test_configured_prune false true kept kept "--no-prune" -test_configured_prune false true pruned kept "--prune" +test_configured_prune false true unset unset pruned kept "" +test_configured_prune false true unset unset kept kept "--no-prune" +test_configured_prune false true unset unset pruned kept "--prune" -test_configured_prune true true pruned kept "" -test_configured_prune true true pruned kept "--prune" -test_configured_prune true true kept kept "--no-prune" -test_configured_prune true true kept pruned \ +test_configured_prune true true unset unset pruned kept "" +test_configured_prune true true unset unset pruned kept "--prune" +test_configured_prune true true unset unset kept kept "--no-prune" +test_configured_prune true true unset unset kept pruned \ "--prune origin refs/tags/*:refs/tags/*" -test_configured_prune true true pruned pruned \ +test_configured_prune true true unset unset pruned pruned \ "--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" test_expect_success 'all boundary commits are excluded' '
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v2 16/17 fetch: add a --fetch-prune option and fetch.pruneTags config
In reply to Ævar Arnfjörð BjarmasonAdd a --fetch-prune option to git-fetch, along with fetch.pruneTags config option. This allows for doing any of:
git fetch -p -P
git fetch --prune --prune-tags
git fetch -p -P origin
git fetch --prune --prune-tags origin
Or simply:
git config fetch.prune true &&
git config fetch.pruneTags true &&
git fetchInstead of the much more verbose:
git fetch --prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'
Before this feature it was painful to support the use-case of pulling from a repo which is having both its branches *and* tags deleted regularly, and have our local references to reflect upstream.
At work we create deployment tags in the repo for each rollout, and there's *lots* of those, so they're archived within weeks for performance reasons.
Without this change it's hard to centrally configure such repos in /etc/gitconfig (on servers that are only used for working with them). You need to set fetch.prune=true globally, and then for each repo:
git -C {} config --replace-all remote.origin.fetch "refs/tags/*:refs/tags/*" "^\+*refs/tags/\*:refs/tags/\*$"Now I can simply set fetch.pruneTags=true in /etc/gitconfig as well, and users running "git pull" will automatically get the pruning semantics I want.
Even though "git remote" has corresponding "prune" and "update --prune" subcommands I'm intentionally not adding a corresponding prune-tags or "update --prune --prune-tags" mode to that command.
It's advertised (as noted in my recent "git remote doc: correct dangerous lies about what prune does") as only modifying remote tracking references, whereas any --prune-tags option is always going to modify what from the user's perspective is a local copy of the tag, since there's no such thing as a remote tracking tag.
Ideally add_prune_tags_to_fetch_refspec() would be something that would use ALLOC_GROW() to grow the 'fetch` member of the 'remote' struct. Instead I'm realloc-ing remote->fetch and adding the tag_refspec to the end.
The reason is that parse_{fetch,push}_refspec which allocate the refspec (ultimately remote->fetch) struct are called many places that don't have access to a 'remote' struct. It would be hard to change all their callsites to be amenable to carry around the bookkeeping variables required for dynamic allocation.
All the other callers of the API first incrementally construct the string version of the refspec in remote->fetch_refspec via add_fetch_refspec(), before finally calling parse_fetch_refspec() via some variation of remote_get().
It's less of a pain to deal with the one special case that needs to modify already constructed refspecs than to chase down and change all the other callsites. The API I'm adding is intentionally not generalized because if we add more of these we'd probably want to re-visit how this is done.
See my "Re: [BUG] git remote prune removes local tags, depending on fetch config" ([email protected];
https://public-inbox.org/git/[email protected]/) for more background info.
--- Documentation/config.txt | 14 +++++++ Documentation/fetch-options.txt | 14 ++++++- Documentation/git-fetch.txt | 47 +++++++++++++++++++++++ builtin/fetch.c | 32 ++++++++++++++-- contrib/completion/git-completion.bash | 2 +- remote.c | 14 +++++++ remote.h | 3 ++ t/t5510-fetch.sh | 69 ++++++++++++++++++++++++++++++++++ 8 files changed, 190 insertions(+), 5 deletions(-)
Documentation/config.txt +14 -0
@@ -1401,6 +1401,14 @@ fetch.prune:: option was given on the command line. See also `remote.<name>.prune` and the PRUNING section of linkgit:git-fetch[1]. +fetch.pruneTags:: + If true, fetch will automatically behave as if the + `refs/tags/*:refs/tags/*` refspec was provided when pruning, + if not set already. This allows for setting both this option + and `fetch.prune` to maintain a 1=1 mapping to upstream + refs. See also `remote.<name>.pruneTags` and the PRUNING + section of linkgit:git-fetch[1]. + fetch.output:: Control how ref update status is printed. Valid values are `full` and `compact`. Default value is `full`. See section
@@ -2945,6 +2953,12 @@ remote.<name>.prune:: remove any remote-tracking references that no longer exist on the remote (as if the `--prune` option was given on the command line). Overrides `fetch.prune` settings, if any. + +remote.<name>.pruneTags:: + When set to true, fetching from this remote by default will also + remove any local tags that no longer exist on the remote if pruning + is activated in general via `remote.<name>.prune`, `fetch.prune` or + `--prune`. Overrides `fetch.pruneTags` settings, if any. + See also `remote.<name>.prune` and the PRUNING section of linkgit:git-fetch[1].
Documentation/fetch-options.txt +13 -1
@@ -73,7 +73,19 @@ ifndef::git-pull[] are fetched due to an explicit refspec (either on the command line or in the remote configuration, for example if the remote was cloned with the --mirror option), then they are also - subject to pruning. + subject to pruning. Supplying `--prune-tags` is a shorthand for + providing the tag refspec. ++ +See the PRUNING section below for more details. + +-P:: +--prune-tags:: + Before fetching, remove any local tags that no longer exist on + the remote if `--prune` is enabled. This option should be used + more carefully, unlike `--prune` it will remove any local + references (local tags) that have been created. This option is + a shorthand for providing the explicit tag refspec along with + `--prune`, see the discussion about that in its documentation. + See the PRUNING section below for more details.
Documentation/git-fetch.txt +47 -0
53 lines not on this page · raw message
builtin/fetch.c +29 -3
@@ -38,6 +38,10 @@ static int fetch_prune_config = -1; /* unspecified */static int prune =-1 ;/* unspecified */ #define PRUNE_BY_DEFAULT 0/* do we prune by default? */ +static int fetch_prune_tags_config =-1 ;/* unspecified */ +static int prune_tags =-1 ;/* unspecified */ +#define PRUNE_TAGS_BY_DEFAULT 0/* do we prune tags by default? */ +static int all, append, dry_run, force, keep, multiple, update_head_ok, verbosity, deepen_relative;static int progress =-1 ;static int tags =TAGS_DEFAULT , unshallow, update_shallow, deepen;
@@ -64,6 +68,11 @@ static int git_fetch_config(const char *k, const char *v, void *cb)return 0 ; } +if (!strcmp (k,"fetch.prunetags" )) { + fetch_prune_tags_config =git_config_bool (k, v); +return 0 ; + } +if (!strcmp (k,"submodule.recurse" )) {int r =git_config_bool (k, v) ?RECURSE_SUBMODULES_ON :RECURSE_SUBMODULES_OFF ;
@@ -126,6 +135,8 @@ static struct option builtin_fetch_options[] = { N_("number of submodules fetched in parallel")),OPT_BOOL ('p' ,"prune" , &prune,N_ ("prune remote-tracking branches no longer on remote" )), +OPT_BOOL ('P' ,"prune-tags" , &prune_tags, +N_ ("prune local tags no longer on remote and clobber changed tags" )), { OPTION_CALLBACK, 0, "recurse-submodules", &recurse_submodules, N_("on-demand"),N_ ("control recursive fetching of submodules" ), PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules },
@@ -1212,6 +1223,8 @@ static void add_options_to_argv(struct argv_array *argv)argv_array_push (argv,"--dry-run" );if (prune !=-1 )argv_array_push (argv, prune ?"--prune" :"--no-prune" ); +if (prune_tags !=-1 ) +argv_array_push (argv, prune_tags ?"--prune-tags" :"--no-prune-tags" );if (update_head_ok)argv_array_push (argv,"--update-head-ok" );if (force)
@@ -1265,7 +1278,7 @@ static int fetch_multiple(struct string_list *list)return result; } -static int fetch_one (struct remote *remote,int argc,const char **argv) +static int fetch_one (struct remote *remote,int argc,const char **argv,int prune_tags_ok) {static const char **refs =NULL ;struct refspec *refspec;
@@ -1288,6 +1301,19 @@ static int fetch_one(struct remote *remote, int argc, const char **argv) prune =PRUNE_BY_DEFAULT ; } +if (prune_tags <0 ) { +/* no command line request */ +if (0 <= remote->prune_tags) + prune_tags = remote->prune_tags; +else if (0 <= fetch_prune_tags_config) + prune_tags = fetch_prune_tags_config; +else + prune_tags =PRUNE_TAGS_BY_DEFAULT ; + } + +if (prune_tags_ok && prune_tags &&remote_is_configured (remote,0 )) +add_prune_tags_to_fetch_refspec (remote); +if (argc >0 ) {int j =0 ;int i;
@@ -1368,7 +1394,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) }else if (argc ==0 ) {/* No arguments -- use default remote */ remote =remote_get (NULL ); - result =fetch_one (remote, argc, argv); + result =fetch_one (remote, argc, argv,1 ); }else if (multiple) {/* All arguments are assumed to be remotes or groups */ for (i =0 ; i < argc; i++)
@@ -1386,7 +1412,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) }else {/* Zero or one remotes */ remote =remote_get (argv[0 ]); - result =fetch_one (remote, argc-1 , argv+1 ); + result =fetch_one (remote, argc-1 , argv+1 , argc ==1 ); } }
contrib/completion/git-completion.bash +1 -1
@@ -1468,7 +1468,7 @@ __git_fetch_recurse_submodules="yes on-demand no" __git_fetch_options=" --quiet --verbose --append --upload-pack --force --keep --depth= --tags --no-tags --all --prune --dry-run --recurse-submodules= - --unshallow --update-shallow + --unshallow --update-shallow --prune-tags " _git_fetch ()
remote.c +14 -0
@@ -104,6 +104,17 @@ static void add_fetch_refspec(struct remote *remote, const char *ref) remote->fetch_refspec[remote->fetch_refspec_nr++] = ref; } +void add_prune_tags_to_fetch_refspec (struct remote *remote) +{ +int nr = remote->fetch_refspec_nr; +int bufsize = nr +1 ; +int size =sizeof (struct refspec ); + + remote->fetch =xrealloc (remote->fetch, size * bufsize); +memcpy (&remote->fetch[nr], tag_refspec, size); +add_fetch_refspec (remote,xstrdup (TAG_REFSPEC )); +} +static void add_url (struct remote *remote,const char *url) {ALLOC_GROW (remote->url, remote->url_nr +1 , remote->url_alloc);
@@ -174,6 +185,7 @@ static struct remote *make_remote(const char *name, int len) ret =xcalloc (1 ,sizeof (struct remote )); ret->prune =-1 ;/* unspecified */ + ret->prune_tags =-1 ;/* unspecified */ ALLOC_GROW (remotes, remotes_nr +1 , remotes_alloc); remotes[remotes_nr++] = ret; ret->name =xstrndup (name, len);
@@ -392,6 +404,8 @@ static int handle_config(const char *key, const char *value, void *cb) remote->skip_default_update =git_config_bool (key, value); else if (!strcmp(subkey, "prune")) remote->prune =git_config_bool (key, value); + else if (!strcmp(subkey, "prunetags")) + remote->prune_tags =git_config_bool (key, value); else if (!strcmp(subkey, "url")) {const char *v;if (git_config_string (&v, key, value))
remote.h +3 -0
@@ -47,6 +47,7 @@ struct remote {int skip_default_update;int mirror;int prune; +int prune_tags;const char *receivepack;const char *uploadpack;
@@ -299,4 +300,6 @@ void apply_push_cas(struct push_cas_option *, struct remote *, struct ref *); #define TAG_REFSPEC "refs/tags/*:refs/tags/*" +void add_prune_tags_to_fetch_refspec(struct remote *remote); +#endif
t/t5510-fetch.sh +69 -0
81 lines not on this page · raw message
Signature · 1 line
2.15.1.424.g9478a66081
Re: [PATCH v2 16/17] fetch: add a --fetch-prune option and fetch.pruneTags config
t/t5510-fetch.sh test_configured_prune_type () { · line 595
Quoted 11 lines · Ævar Arnfjörð Bjarmason
Add a --fetch-prune option to git-fetch, along with fetch.pruneTags config option. [...]
Signed-off-by Ævar Arnfjörð Bjarmason <[email protected]>---
t/t5510-fetch.sh +4
-0@@ -592,6 +592,15 @@ test_configured_prune_type () { + if test "$fetch_prune_tags" = 'true' || + test "$remote_origin_prune_tags" = 'true' + then + if ! printf '%s' "$cmdline" | grep -q refs/remotes/origin/
Is $cmdline guaranteed to end with a newline? Historically, not all 'grep's would be able to match the last line if it was not properly terminated. Perhaps you want '%s\n' instead?
t/t5510-fetch.sh test_configured_prune true true unset unset kept pruned \ · line 714
Quoted 7 lines
t/t5510-fetch.sh +6
-0+ then + new_cmdline="$new_cmdline refs/tags/*:refs/tags/*" + fi + fi +@@ -705,6 +714,66 @@ test_configured_prune true true unset unset kept pruned \ +# When --prune-tags is supplied it's ignored if an explict refspec is
s/explict/explicit/
t/t5510-fetch.sh test_configured_prune true true unset unset kept pruned \ · line 746
Quoted 6 lines
t/t5510-fetch.sh +6
-0+# given, same for the configuration options. + +# Pruning that also takes place if s!origin!<file:// url of remote>!, +# or otherwise uses the file://-specific codepath. However, because +# there's no implicit +refs/heads/*:refs/remotes/origin/* refspec and +# supplying it on the command-line negate --prune-tags the branches
s/negate/&s/
s/--prune-tags/&,/
Quoted 1 line
t/t5510-fetch.sh +1
-0+# will not be pruned.
Re: [PATCH v2 16/17] fetch: add a --fetch-prune option and fetch.pruneTags config
On Fri, Feb 09 2018, Eric Sunshine jotted:
Quoted 5 lines
Quoted 3 lines · Ævar Arnfjörð Bjarmason
Add a --fetch-prune option to git-fetch, along with fetch.pruneTags config option. [...]
Quoted 12 lines
t/t5510-fetch.sh test_configured_prune_type () { · line 595
Quoted 8 lines
Signed-off-by Ævar Arnfjörð Bjarmason <[email protected]>---
t/t5510-fetch.sh +4
-0@@ -592,6 +592,15 @@ test_configured_prune_type () { + if test "$fetch_prune_tags" = 'true' || + test "$remote_origin_prune_tags" = 'true' + then + if ! printf '%s' "$cmdline" | grep -q refs/remotes/origin/Is $cmdline guaranteed to end with a newline? Historically, not all 'grep's would be able to match the last line if it was not properly terminated. Perhaps you want '%s\n' instead?
Good point. No it's not, will fix that.
Quoted 10 lines
t/t5510-fetch.sh test_configured_prune true true unset unset kept pruned \ · line 714
Quoted 7 lines
t/t5510-fetch.sh +6
-0+ then + new_cmdline="$new_cmdline refs/tags/*:refs/tags/*" + fi + fi +@@ -705,6 +714,66 @@ test_configured_prune true true unset unset kept pruned \ +# When --prune-tags is supplied it's ignored if an explict refspec iss/explict/explicit/
Quoted 11 lines
t/t5510-fetch.sh test_configured_prune true true unset unset kept pruned \ · line 746
Quoted 6 lines
t/t5510-fetch.sh +6
-0+# given, same for the configuration options. + +# Pruning that also takes place if s!origin!<file:// url of remote>!, +# or otherwise uses the file://-specific codepath. However, because +# there's no implicit +refs/heads/*:refs/remotes/origin/* refspec and +# supplying it on the command-line negate --prune-tags the branchess/negate/&s/
s/--prune-tags/&,/Quoted 1 line
t/t5510-fetch.sh +1
-0+# will not be pruned.
*nod*
PATCH v2 17/17 fetch: make the --fetch-prune work with <url>
In reply to Ævar Arnfjörð BjarmasonMake the new --prune-tags option work properly when git-fetch is invoked with a <url> parameter instead of a <remote name> parameter.
This change is split off from the introduction of --prune-tags due to the relative complexity of munging the incoming argv, which is easier to review as a separate change.
--- Documentation/git-fetch.txt | 21 ++++++--------------- builtin/fetch.c | 17 ++++++++++++++--- t/t5510-fetch.sh | 6 +++--- 3 files changed, 23 insertions(+), 21 deletions(-)
Documentation/git-fetch.txt +6 -15
@@ -175,24 +175,15 @@ It's reasonable to e.g. configure `fetch.pruneTags=true` in run, without making every invocation of `git fetch` without `--prune` an error. -Another special case of `--prune-tags` is that -`refs/tags/*:refs/tags/*` will not be implicitly provided if an URL is -being fetched. I.e.: - ------------------------------------------------- -$ git fetch <url> --prune --prune-tags ------------------------------------------------- - -Will prune no tags, as opposed to: +Pruning tags with `--prune-tags` also works when fetching a URL +instead of a named remote. These will all prune tags not found on +origin: ------------------------------------------------ $ git fetch origin --prune --prune-tags ------------------------------------------------- - -To prune tags given a URL supply the refspec explicitly: - ------------------------------------------------- -$ git fetch <url> --prune 'refs/tags/*:refs/tags/*' +$ git fetch origin --prune 'refs/tags/*:refs/tags/*' +$ git fetch <url of origin> --prune --prune-tags +$ git fetch <url of origin> --prune 'refs/tags/*:refs/tags/*' ------------------------------------------------ OUTPUT
builtin/fetch.c +14 -3
@@ -1283,7 +1283,10 @@ static int fetch_one(struct remote *remote, int argc, const char **argv, int prustatic const char **refs =NULL ;struct refspec *refspec;int ref_nr =0 ; +int j =0 ;int exit_code; +int maybe_prune_tags; +int remote_via_config =remote_is_configured (remote,0 );if (!remote) die(_("No remote repository specified. Please, specify either a URL or a\n"
@@ -1311,13 +1314,21 @@ static int fetch_one(struct remote *remote, int argc, const char **argv, int pru prune_tags =PRUNE_TAGS_BY_DEFAULT ; } -if (prune_tags_ok && prune_tags &&remote_is_configured (remote,0 )) + maybe_prune_tags = prune_tags_ok && prune_tags; +if (maybe_prune_tags && remote_via_config)add_prune_tags_to_fetch_refspec (remote); +if (argc >0 || (maybe_prune_tags && !remote_via_config)) { +size_t nr_alloc =st_add3 (argc, maybe_prune_tags,1 ); + refs =xcalloc (nr_alloc,sizeof (const char *)); +if (maybe_prune_tags) { + refs[j++] =xstrdup ("refs/tags/*:refs/tags/*" ); + ref_nr++; + } + } +if (argc >0 ) { -int j =0 ;int i; - refs =xcalloc (st_add (argc,1 ),sizeof (const char *));for (i =0 ; i < argc; i++) {if (!strcmp (argv[i],"tag" )) { i++;
t/t5510-fetch.sh +3 -3
@@ -748,9 +748,9 @@ test_configured_prune unset unset unset true pruned kept \ test_configured_prune_type unset unset unset unset kept kept "origin --prune-tags" "name" test_configured_prune_type unset unset unset unset kept kept "origin --prune-tags" "link" test_configured_prune_type unset unset unset unset pruned pruned "origin --prune --prune-tags" "name" -test_configured_prune_type unset unset unset unset kept kept "origin --prune --prune-tags" "link" +test_configured_prune_type unset unset unset unset kept pruned "origin --prune --prune-tags" "link" test_configured_prune_type unset unset unset unset pruned pruned "--prune --prune-tags origin" "name" -test_configured_prune_type unset unset unset unset kept kept "--prune --prune-tags origin" "link" +test_configured_prune_type unset unset unset unset kept pruned "--prune --prune-tags origin" "link" test_configured_prune_type unset unset true unset pruned pruned "--prune origin" "name" test_configured_prune_type unset unset true unset kept pruned "--prune origin" "link" test_configured_prune_type unset unset unset true pruned pruned "--prune origin" "name"
@@ -772,7 +772,7 @@ test_expect_success 'remove remote.origin.fetch "one"' ' ) ' test_configured_prune_type unset unset unset unset kept pruned "origin --prune --prune-tags" "name" -test_configured_prune_type unset unset unset unset kept kept "origin --prune --prune-tags" "link" +test_configured_prune_type unset unset unset unset kept pruned "origin --prune --prune-tags" "link" test_expect_success 'all boundary commits are excluded' ' test_commit base &&
Signature · 1 line
2.15.1.424.g9478a66081
Re: [PATCH v2 17/17] fetch: make the --fetch-prune work with <url>
Quoted 1 line · Ævar Arnfjörð Bjarmason
fetch: make the --fetch-prune work with <url>
Do you mean s/--fetch-prune/--prune-tags/ ?
Quoted 9 lines
Make the new --prune-tags option work properly when git-fetch is invoked with a <url> parameter instead of a <remote name> parameter.
This change is split off from the introduction of --prune-tags due to the relative complexity of munging the incoming argv, which is easier to review as a separate change.
Signed-off-by Ævar Arnfjörð Bjarmason <[email protected]>
Re: [PATCH v2 17/17] fetch: make the --fetch-prune work with <url>
On Fri, Feb 09 2018, Eric Sunshine jotted:
Quoted 5 lines
Quoted 1 line · Ævar Arnfjörð Bjarmason
fetch: make the --fetch-prune work with <url>
Do you mean s/--fetch-prune/--prune-tags/ ?
Yes, sorry. Will fix.
Quoted 9 lines
Quoted 9 lines
Make the new --prune-tags option work properly when git-fetch is invoked with a <url> parameter instead of a <remote name> parameter.
This change is split off from the introduction of --prune-tags due to the relative complexity of munging the incoming argv, which is easier to review as a separate change.
Signed-off-by Ævar Arnfjörð Bjarmason <[email protected]>
Re: [PATCH v2 00/17] document & test fetch pruning & add fetch.pruneTags
In reply to Ævar Arnfjörð BjarmasonOn Thu, Feb 08 2018, Ævar Arnfjörð Bjarmason jotted:
Quoted 3 lines
As noted in my [email protected] there was a bug I noticed in v3 where it would segfault on some git-fetch invocations, but there were not tests anywhere that caught that.
...and of course this whole submission this time around should be v4, not v2, but I didn't notice that I didn't adjust the subject prefix before sending. Junio: Hopefully you can pick it up anyway without too much trouble, sorry.
FWIW I've deployed this to production @ work to some tens of k of machines (low "k" of which have users using git) without any issues.
Re: [PATCH v2 00/17] document & test fetch pruning & add fetch.pruneTags
Quoted 6 lines · Ævar Arnfjörð Bjarmason
On Thu, Feb 08 2018, Ævar Arnfjörð Bjarmason jotted:
Quoted 3 lines
As noted in my [email protected] there was a bug I noticed in v3 where it would segfault on some git-fetch invocations, but there were not tests anywhere that caught that.
Quoted 7 lines
...and of course this whole submission this time around should be v4, not v2, but I didn't notice that I didn't adjust the subject prefix before sending. Junio: Hopefully you can pick it up anyway without too much trouble, sorry.
FWIW I've deployed this to production @ work to some tens of k of machines (low "k" of which have users using git) without any issues.
Will replace. I found that the resolution of conflicts necessary with the jh/partial-clone topic is a bit different from the previous version due to addition of an extra parameter to fetch_one(), and I think I didn't botch it, but please double check when I push the results out in a few hours.
Thanks.
Re: [PATCH v2 00/17] document & test fetch pruning & add fetch.pruneTags
On Thu, Feb 08 2018, Junio C. Hamano jotted:
Quoted 13 lines
Quoted 11 lines · Ævar Arnfjörð Bjarmason
On Thu, Feb 08 2018, Ævar Arnfjörð Bjarmason jotted:
Quoted 3 lines
As noted in my [email protected] there was a bug I noticed in v3 where it would segfault on some git-fetch invocations, but there were not tests anywhere that caught that.
...and of course this whole submission this time around should be v4, not v2, but I didn't notice that I didn't adjust the subject prefix before sending. Junio: Hopefully you can pick it up anyway without too much trouble, sorry.
Quoted 8 lines
Quoted 2 lines
FWIW I've deployed this to production @ work to some tens of k of machines (low "k" of which have users using git) without any issues.
Will replace. I found that the resolution of conflicts necessary with the jh/partial-clone topic is a bit different from the previous version due to addition of an extra parameter to
fetch_one(), and I think I didn't botch it, but please double check when I push the results out in a few hours.
Thanks. FWIW I've looked over that resolution in pu and it LGTM.
v5 cover document & test fetch pruning & add fetch.pruneTags
In reply to Ævar Arnfjörð BjarmasonHere's a v5 (correct subject line this time!). Many thanks to Eric for a thorough review.
I'll spare you the per-patch changelog. These are all minor commit message / doc / comment wording changes, with the exception of making a bit of the test code better, and adding a \n for grep portability. tbdiff at the end.
Ævar Arnfjörð Bjarmason (17):
fetch: don't redundantly NULL something calloc() gave us
fetch: trivially refactor assignment to ref_nr
fetch: stop accessing "remote" variable indirectly
remote: add a macro for "refs/tags/*:refs/tags/*"
fetch tests: refactor in preparation for testing tag pruning fetch tests: re-arrange arguments for future readability fetch tests: add a tag to be deleted to the pruning tests fetch tests: test --prune and refspec interaction
fetch tests: double quote a variable for interpolation fetch tests: expand case/esac for later change
fetch tests: fetch <url> <spec> as well as fetch [<remote>] git fetch doc: add a new section to explain the ins & outs of pruning git remote doc: correct dangerous lies about what prune does git-fetch & config doc: link to the new PRUNING section fetch tests: add scaffolding for the new fetch.pruneTags
fetch: add a --prune-tags option and fetch.pruneTags config
fetch: make the --prune-tags work with <url>
Documentation/config.txt | 20 ++- Documentation/fetch-options.txt | 17 ++- Documentation/git-fetch.txt | 87 ++++++++++++ Documentation/git-remote.txt | 14 +- builtin/fetch.c | 54 ++++++-- contrib/completion/git-completion.bash | 2 +- remote.c | 15 +++ remote.h | 5 + t/t5510-fetch.sh | 238 +++++++++++++++++++++++++++------ 9 files changed, 391 insertions(+), 61 deletions(-)
tbdiff:
1: da0992d97c = 1: da0992d97c fetch: don't redundantly NULL something calloc() gave us 2: c781d18a29 ! 2: 899430a3b2 fetch: trivially refactor assignment to ref_nr
@@ -7,8 +7,8 @@
"j" is, and "j" is only incremented in that loop, so this change isn't a logic error.
- This change makes a subsequent change which splits the incrementing of
- "ref_nr" into two blocks.
+ This change simplifies a subsequent change, which will split the
+ incrementing of "ref_nr" into two blocks.
3: 1203ef6e35 = 3: 98e3a28bdf fetch: stop accessing "remote" variable indirectly
4: 1d7956c444 = 4: 384c1fc318 remote: add a macro for "refs/tags/*:refs/tags/*"
5: a0dc3eb024 = 5: ec92777861 fetch tests: refactor in preparation for testing tag pruning
6: 7ed1561e3d = 6: 1c23526223 fetch tests: re-arrange arguments for future readability
7: 6dbf0e688d = 7: 59abe07b71 fetch tests: add a tag to be deleted to the pruning tests
8: 9fc3589793 = 8: af7acef671 fetch tests: test --prune and refspec interaction
9: a4487f9389 = 9: cb72187362 fetch tests: double quote a variable for interpolation
10: b8c07e2d42 = 10: 300c1c0985 fetch tests: expand case/esac for later change
11: 6341131ee0 ! 11: 550e7df594 fetch tests: fetch <url> <spec> as well as fetch [<remote>]
@@ -60,15 +60,12 @@
- test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" '
+ mode=$6
+
-+ if ! test -e prune-type-setup-done
++ if test -z "$cmdline_setup"
+ then
-+ test_expect_success 'prune_type setup' '
-+ git -C one config remote.origin.url >one.remote-url &&
-+ git -C one config remote.origin.fetch >one.remote-fetch &&
-+ remote_url="file://$(cat one.remote-url)" &&
-+ remote_fetch="$(cat one.remote-fetch)" &&
-+ cmdline_setup="\"$remote_url\" \"$remote_fetch\"" &&
-+ touch prune-type-setup-done
++ test_expect_success 'setup cmdline_setup variable for subsequent test' '
++ remote_url="file://$(git -C one config remote.origin.url)" &&
++ remote_fetch="$(git -C one config remote.origin.fetch)" &&
++ cmdline_setup="\"$remote_url\" \"$remote_fetch\""
+ '
+ fi
+12: 824c3ed4c1 ! 12: 273f4c603f git fetch doc: add a new section to explain the ins & outs of pruning
@@ -32,7 +32,7 @@
+<commit>` needlessly verbose, as well as impacting anything else
+that'll work with the complete set of known references.
+
-+These remote tracking references can be deleted as a one-off with
++These remote-tracking references can be deleted as a one-off with
+either of:
+
+------------------------------------------------
@@ -44,13 +44,13 @@
+------------------------------------------------
+
+To prune references as part of your normal workflow without needing to
-+remember to run that set fetch.prune globally, or
++remember to run that, set fetch.prune globally, or
+remote.<name>.prune per-remote in the config. See
+linkgit:git-config[1].
+
+Here's where things get tricky and more specific. The pruning feature
+doesn't actually care about branches, instead it'll prune local <->
-+remote references as a function of the refspec of the remote (see ++remote-references as a function of the refspec of the remote (see
+<refspec> and <<CRTB,CONFIGURED REMOTE-TRACKING BRANCHES>> above).
+
+Therefore if the refspec for the remote includes
13: 59de997172 ! 13: ada313b523 git remote doc: correct dangerous lies about what prune does
@@ -11,7 +11,7 @@
oldest version I could be bothered to test).
E.g. if "refs/tags/*:refs/tags/*" is explicitly set in the refspec of
- the remote it'll delete all local tags <name> doesn't know about.
+ the remote, it'll delete all local tags <name> doesn't know about.
Instead, briefly give the reader just enough of a hint that this option might constitute a shotgun aimed at their foot, and point them
@@ -36,11 +36,11 @@
-These stale branches have already been removed from the remote repository -referenced by <name>, but are still locally available in -"remotes/<name>".
-+Deletes stale references associated with <name>. By default stale -+remote-tracking branches under <name>, but depending on global -+configuration and the configuration of the remote we might even prune -+local tags that haven't been pushed there. Equivalent to git fetch -+--prune <name>, except that no new references will be fetched. ++Deletes stale references associated with <name>. By default, stale ++remote-tracking branches under <name> are deleted, but depending on ++global configuration and the configuration of the remote we might even ++prune local tags that haven't been pushed there. Equivalent to git ++fetch --prune <name>, except that no new references will be fetched.
++
+See the PRUNING section of linkgit:git-fetch[1] for what it'll prune +depending on various configuration.
14: 5dfc9177bb = 14: eeb9102d9f git-fetch & config doc: link to the new PRUNING section 15: fc0890eb9c ! 15: 6c1facccda fetch tests: add scaffolding for the new fetch.pruneTags
@@ -28,7 +28,7 @@
+ cmdline=$7
+ mode=$8
- if ! test -e prune-type-setup-done
+ if test -z "$cmdline_setup"
then
@@
cmdline="$new_cmdline"16: 03b1393ea2 ! 16: 1d8d6e29fc fetch: add a --prune-tags option and fetch.pruneTags config
@@ -1,9 +1,10 @@
Author: Ævar Arnfjörð Bjarmason <[email protected]>
- fetch: add a --fetch-prune option and fetch.pruneTags config
-
- Add a --fetch-prune option to git-fetch, along with fetch.pruneTags
- config option. This allows for doing any of:
+ fetch: add a --prune-tags option and fetch.pruneTags config
+
+ Add a --prune-tags option to git-fetch, along with fetch.pruneTags
+ config option and a -P shorthand (-p is --prune). This allows for
+ doing any of:
git fetch -p -P
git fetch --prune --prune-tags
@@ -366,7 +367,7 @@
+ if test "$fetch_prune_tags" = 'true' ||
+ test "$remote_origin_prune_tags" = 'true'
+ then
-+ if ! printf '%s' "$cmdline" | grep -q refs/remotes/origin/
++ if ! printf '%s' "$cmdline\n" | grep -q refs/remotes/origin/
+ then
+ new_cmdline="$new_cmdline refs/tags/*:refs/tags/*"
+ fi
@@ -396,7 +397,7 @@
+test_configured_prune false true false true pruned pruned ""
+test_configured_prune true false true false kept kept ""
+
-+# When --prune-tags is supplied it's ignored if an explict refspec is
++# When --prune-tags is supplied it's ignored if an explicit refspec is
+# given, same for the configuration options.
+test_configured_prune unset unset unset unset pruned kept \
+ "--prune --prune-tags origin +refs/heads/*:refs/remotes/origin/*"
@@ -405,13 +406,15 @@
+test_configured_prune unset unset unset true pruned kept \
+ "--prune origin +refs/heads/*:refs/remotes/origin/*"
+
-+# Pruning that also takes place if s!origin!<file:// url of remote>!,
-+# or otherwise uses the file://-specific codepath. However, because
-+# there's no implicit +refs/heads/*:refs/remotes/origin/* refspec and
-+# supplying it on the command-line negate --prune-tags the branches
-+# will not be pruned.
-+test_configured_prune_type unset unset unset unset kept kept "origin --prune-tags" "name"
-+test_configured_prune_type unset unset unset unset kept kept "origin --prune-tags" "link"
++# Pruning that also takes place if a file:// url replaces a named
++# remote, with the exception of --prune-tags on the command-line
++# (arbitrary limitation).
++#
++# However, because there's no implicit
++# +refs/heads/*:refs/remotes/origin/* refspec and supplying it on the
++# command-line negates --prune-tags, the branches will not be pruned.
++test_configured_prune_type unset unset unset unset kept kept "origin --prune-tags" "name"
++test_configured_prune_type unset unset unset unset kept kept "origin --prune-tags" "link"
+test_configured_prune_type unset unset unset unset pruned pruned "origin --prune --prune-tags" "name"
+test_configured_prune_type unset unset unset unset kept kept "origin --prune --prune-tags" "link"
+test_configured_prune_type unset unset unset unset pruned pruned "--prune --prune-tags origin" "name"
@@ -427,9 +430,8 @@
+test_configured_prune_type unset true unset true pruned pruned "origin" "name"
+test_configured_prune_type unset true unset true kept pruned "origin" "link"
+
-+# When all remote.origin.fetch settings are deleted a --prune
-+# --prune-tags still implicitly supplies refs/tags/*:refs/tags/* so
-+# tags, but not tracking branches, will be deleted.
++# Interaction between --prune-tags and no "fetch" config in the remote
++# at all.
+test_expect_success 'remove remote.origin.fetch "one"' '
+ (
+ cd one &&
17: f0dca21626 ! 17: ddcd576c5c fetch: make the --prune-tags work with <url>
@@ -1,6 +1,6 @@
Author: Ævar Arnfjörð Bjarmason <[email protected]>
- fetch: make the --fetch-prune work with <url>
+ fetch: make the --prune-tags work with <url>
Make the new --prune-tags option work properly when git-fetch is
invoked with a <url> parameter instead of a <remote name>
@@ -91,8 +91,18 @@
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@
- test_configured_prune_type unset unset unset unset kept kept "origin --prune-tags" "name"
- test_configured_prune_type unset unset unset unset kept kept "origin --prune-tags" "link"
+ "--prune origin +refs/heads/*:refs/remotes/origin/*"
+
+ # Pruning that also takes place if a file:// url replaces a named
+-# remote, with the exception of --prune-tags on the command-line
+-# (arbitrary limitation).
+-#
+-# However, because there's no implicit
++# remote. However, because there's no implicit
+ # +refs/heads/*:refs/remotes/origin/* refspec and supplying it on the
+ # command-line negates --prune-tags, the branches will not be pruned.
+ test_configured_prune_type unset unset unset unset kept kept "origin --prune-tags" "name"
+ test_configured_prune_type unset unset unset unset kept kept "origin --prune-tags" "link"
test_configured_prune_type unset unset unset unset pruned pruned "origin --prune --prune-tags" "name"
-test_configured_prune_type unset unset unset unset kept kept "origin --prune --prune-tags" "link"
+test_configured_prune_type unset unset unset unset kept pruned "origin --prune --prune-tags" "link"
@@ -103,6 +113,18 @@
test_configured_prune_type unset unset true unset kept pruned "--prune origin" "link"
test_configured_prune_type unset unset unset true pruned pruned "--prune origin" "name"
@@
+ test_configured_prune_type unset true unset true pruned pruned "origin" "name"
+ test_configured_prune_type unset true unset true kept pruned "origin" "link"
+
+-# Interaction between --prune-tags and no "fetch" config in the remote
+-# at all.
++# When all remote.origin.fetch settings are deleted a --prune
++# --prune-tags still implicitly supplies refs/tags/*:refs/tags/* so
++# tags, but not tracking branches, will be deleted.
+ test_expect_success 'remove remote.origin.fetch "one"' '
+ (
+ cd one &&
+@@
)
'
test_configured_prune_type unset unset unset unset kept pruned "origin --prune --prune-tags" "name"
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v5 1/17 fetch: don't redundantly NULL something calloc() gave us
Stop redundantly NULL-ing the last element of the refs structure, which was retrieved via calloc(), and is thus guaranteed to be pre-NULL'd.
This code dates back to b888d61c83 ("Make fetch a builtin", 2007-09-10), where wasn't any reason to do this back then either, it's just boilerplate left over from when git-fetch was initially introduced.
The motivation for this change was to make a subsequent change which would also modify the refs variable smaller, since it won't have to copy this redundant "NULL the last + 1 item" pattern.
We may not end up keeping that change, but as this pattern is still pointless, so let's fix it.
--- builtin/fetch.c | 1 - 1 file changed, 1 deletion(-)
builtin/fetch.c +0 -1
@@ -1302,7 +1302,6 @@ static int fetch_one(struct remote *remote, int argc, const char **argv) }else refs[j++] = argv[i]; } - refs[j] =NULL ; ref_nr = j; }
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v5 2/17 fetch: trivially refactor assignment to ref_nr
In reply to Ævar Arnfjörð BjarmasonTrivially refactor an assignment to make a subsequent patch smaller. The "ref_nr" variable is initialized to 0 earlier, just as "j" is, and "j" is only incremented in that loop, so this change isn't a logic error.
This change simplifies a subsequent change, which will split the incrementing of "ref_nr" into two blocks.
--- builtin/fetch.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
builtin/fetch.c +1 -1
@@ -1301,8 +1301,8 @@ static int fetch_one(struct remote *remote, int argc, const char **argv) argv[i], argv[i]); }else refs[j++] = argv[i]; + ref_nr++; } - ref_nr = j; }sigchain_push_common (unlock_pack_on_signal);
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v5 3/17 fetch: stop accessing "remote" variable indirectly
In reply to Ævar Arnfjörð BjarmasonAccess the "remote" variable passed to the fetch_one() directly rather than through the gtransport wrapper struct constructed in this function for other purposes.
This makes the code more readable, as it's now obvious that the remote struct doesn't somehow get munged by the prepare_transport() function above, which takes the "remote" struct as an argument and constructs the "gtransport" struct, containing among other things the "remote" struct.
A subsequent change will copy this pattern to access a new remote->prune_tags field, but without the use of the gtransport variable. It's useful once that change lands to see that the two pieces of code behave exactly the same.
This pattern of accessing the container struct was added in 737c5a9cde ("fetch: make --prune configurable", 2013-07-13) when this code was initially introduced.
--- builtin/fetch.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
builtin/fetch.c +2 -2
@@ -1280,8 +1280,8 @@ static int fetch_one(struct remote *remote, int argc, const char **argv)if (prune <0 ) {/* no command line request */ -if (0 <= gtransport->remote->prune) - prune = gtransport->remote->prune; +if (0 <= remote->prune) + prune = remote->prune;else if (0 <= fetch_prune_config) prune = fetch_prune_config;else
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v5 4/17 remote: add a macro for "refs/tags/*:refs/tags/*"
In reply to Ævar Arnfjörð BjarmasonAdd a macro with the refspec string "refs/tags/*:refs/tags/*". There's been a pre-defined struct version of this since e0aaa29ff3 ("Have a constant extern refspec for "--tags"", 2008-04-17), but nothing that could be passed to e.g. add_fetch_refspec().
This will be used in subsequent commits to avoid hardcoding this string in multiple places.
--- remote.c | 1 + remote.h | 2 ++ 2 files changed, 3 insertions(+)
remote.c +1 -0
@@ -22,6 +22,7 @@ static struct refspec s_tag_refspec = {"refs/tags/*" }; +/* See TAG_REFSPEC for the string version */ const struct refspec *tag_refspec = &s_tag_refspec;struct counted_string {
remote.h +2 -0
@@ -297,4 +297,6 @@ extern int parseopt_push_cas_option(const struct option *, const char *arg, intextern int is_empty_cas (const struct push_cas_option *);void apply_push_cas (struct push_cas_option *,struct remote *,struct ref *); +#define TAG_REFSPEC "refs/tags/*:refs/tags/*" +#endif
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v5 5/17 fetch tests: refactor in preparation for testing tag pruning
In reply to Ævar Arnfjörð BjarmasonIn a subsequent commit this function will learn to test for tag pruning, prepare for that by making space for more variables, and making it clear that "expected" here refers to branches.
--- t/t5510-fetch.sh | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
t/t5510-fetch.sh +6 -3
@@ -549,9 +549,12 @@ set_config_tristate () { } test_configured_prune () { - fetch_prune=$1 remote_origin_prune=$2 cmdline=$3 expected=$4 + fetch_prune=$1 + remote_origin_prune=$2 + cmdline=$3 + expected_branch=$4 - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; $4" ' + test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && (
@@ -572,7 +575,7 @@ test_configured_prune () { set_config_tristate remote.origin.prune $remote_origin_prune && git fetch $cmdline && - case "$expected" in + case "$expected_branch" in pruned) test_must_fail git rev-parse --verify refs/remotes/origin/newbranch ;;
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v5 6/17 fetch tests: re-arrange arguments for future readability
In reply to Ævar Arnfjörð BjarmasonRe-arrange the arguments to the test_configured_prune() function used in this test to pass the arguments to --fetch last. A subsequent change will test for more elaborate fetch arguments, including long refspecs. It'll be more readable to be able to wrap those on a new line of their own.
--- t/t5510-fetch.sh | 82 ++++++++++++++++++++++++++++++-------------------------- 1 file changed, 44 insertions(+), 38 deletions(-)
t/t5510-fetch.sh +44 -38
@@ -551,10 +551,10 @@ set_config_tristate () { test_configured_prune () { fetch_prune=$1 remote_origin_prune=$2 - cmdline=$3 - expected_branch=$4 + expected_branch=$3 + cmdline=$4 - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${3:+ $3}; branch:$4" ' + test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && (
@@ -587,41 +587,47 @@ test_configured_prune () { ' } -test_configured_prune unset unset "" kept -test_configured_prune unset unset "--no-prune" kept -test_configured_prune unset unset "--prune" pruned - -test_configured_prune false unset "" kept -test_configured_prune false unset "--no-prune" kept -test_configured_prune false unset "--prune" pruned - -test_configured_prune true unset "" pruned -test_configured_prune true unset "--prune" pruned -test_configured_prune true unset "--no-prune" kept - -test_configured_prune unset false "" kept -test_configured_prune unset false "--no-prune" kept -test_configured_prune unset false "--prune" pruned - -test_configured_prune false false "" kept -test_configured_prune false false "--no-prune" kept -test_configured_prune false false "--prune" pruned - -test_configured_prune true false "" kept -test_configured_prune true false "--prune" pruned -test_configured_prune true false "--no-prune" kept - -test_configured_prune unset true "" pruned -test_configured_prune unset true "--no-prune" kept -test_configured_prune unset true "--prune" pruned - -test_configured_prune false true "" pruned -test_configured_prune false true "--no-prune" kept -test_configured_prune false true "--prune" pruned - -test_configured_prune true true "" pruned -test_configured_prune true true "--prune" pruned -test_configured_prune true true "--no-prune" kept +# $1 config: fetch.prune +# $2 config: remote.<name>.prune +# $3 expect: branch to be pruned? +# $4 git-fetch $cmdline: +# +# $1 $2 $3 $4 +test_configured_prune unset unset kept "" +test_configured_prune unset unset kept "--no-prune" +test_configured_prune unset unset pruned "--prune" + +test_configured_prune false unset kept "" +test_configured_prune false unset kept "--no-prune" +test_configured_prune false unset pruned "--prune" + +test_configured_prune true unset pruned "" +test_configured_prune true unset pruned "--prune" +test_configured_prune true unset kept "--no-prune" + +test_configured_prune unset false kept "" +test_configured_prune unset false kept "--no-prune" +test_configured_prune unset false pruned "--prune" + +test_configured_prune false false kept "" +test_configured_prune false false kept "--no-prune" +test_configured_prune false false pruned "--prune" + +test_configured_prune true false kept "" +test_configured_prune true false pruned "--prune" +test_configured_prune true false kept "--no-prune" + +test_configured_prune unset true pruned "" +test_configured_prune unset true kept "--no-prune" +test_configured_prune unset true pruned "--prune" + +test_configured_prune false true pruned "" +test_configured_prune false true kept "--no-prune" +test_configured_prune false true pruned "--prune" + +test_configured_prune true true pruned "" +test_configured_prune true true pruned "--prune" +test_configured_prune true true kept "--no-prune" test_expect_success 'all boundary commits are excluded' ' test_commit base &&
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v5 7/17 fetch tests: add a tag to be deleted to the pruning tests
In reply to Ævar Arnfjörð BjarmasonAdd a tag to be deleted to the fetch --prune tests. The tag is always kept for now, which is the expected behavior, but now I can add a test for tag pruning in a later commit.
--- t/t5510-fetch.sh | 93 ++++++++++++++++++++++++++++++++------------------------ 1 file changed, 53 insertions(+), 40 deletions(-)
t/t5510-fetch.sh +53 -40
@@ -552,21 +552,25 @@ test_configured_prune () { fetch_prune=$1 remote_origin_prune=$2 expected_branch=$3 - cmdline=$4 + expected_tag=$4 + cmdline=$5 - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${4:+ $4}; branch:$3" ' + test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && + git tag -f newtag && ( cd one && test_unconfig fetch.prune && test_unconfig remote.origin.prune && git fetch && - git rev-parse --verify refs/remotes/origin/newbranch + git rev-parse --verify refs/remotes/origin/newbranch && + git rev-parse --verify refs/tags/newtag ) && # now remove it git branch -d newbranch && + git tag -d newtag && # then test (
@@ -582,6 +586,14 @@ test_configured_prune () { kept) git rev-parse --verify refs/remotes/origin/newbranch ;; + esac && + case "$expected_tag" in + pruned) + test_must_fail git rev-parse --verify refs/tags/newtag + ;; + kept) + git rev-parse --verify refs/tags/newtag + ;; esac ) '
@@ -590,44 +602,45 @@ test_configured_prune () { # $1 config: fetch.prune # $2 config: remote.<name>.prune # $3 expect: branch to be pruned? -# $4 git-fetch $cmdline: +# $4 expect: tag to be pruned? +# $5 git-fetch $cmdline: # -# $1 $2 $3 $4 -test_configured_prune unset unset kept "" -test_configured_prune unset unset kept "--no-prune" -test_configured_prune unset unset pruned "--prune" - -test_configured_prune false unset kept "" -test_configured_prune false unset kept "--no-prune" -test_configured_prune false unset pruned "--prune" - -test_configured_prune true unset pruned "" -test_configured_prune true unset pruned "--prune" -test_configured_prune true unset kept "--no-prune" - -test_configured_prune unset false kept "" -test_configured_prune unset false kept "--no-prune" -test_configured_prune unset false pruned "--prune" - -test_configured_prune false false kept "" -test_configured_prune false false kept "--no-prune" -test_configured_prune false false pruned "--prune" - -test_configured_prune true false kept "" -test_configured_prune true false pruned "--prune" -test_configured_prune true false kept "--no-prune" - -test_configured_prune unset true pruned "" -test_configured_prune unset true kept "--no-prune" -test_configured_prune unset true pruned "--prune" - -test_configured_prune false true pruned "" -test_configured_prune false true kept "--no-prune" -test_configured_prune false true pruned "--prune" - -test_configured_prune true true pruned "" -test_configured_prune true true pruned "--prune" -test_configured_prune true true kept "--no-prune" +# $1 $2 $3 $4 $5 +test_configured_prune unset unset kept kept "" +test_configured_prune unset unset kept kept "--no-prune" +test_configured_prune unset unset pruned kept "--prune" + +test_configured_prune false unset kept kept "" +test_configured_prune false unset kept kept "--no-prune" +test_configured_prune false unset pruned kept "--prune" + +test_configured_prune true unset pruned kept "" +test_configured_prune true unset pruned kept "--prune" +test_configured_prune true unset kept kept "--no-prune" + +test_configured_prune unset false kept kept "" +test_configured_prune unset false kept kept "--no-prune" +test_configured_prune unset false pruned kept "--prune" + +test_configured_prune false false kept kept "" +test_configured_prune false false kept kept "--no-prune" +test_configured_prune false false pruned kept "--prune" + +test_configured_prune true false kept kept "" +test_configured_prune true false pruned kept "--prune" +test_configured_prune true false kept kept "--no-prune" + +test_configured_prune unset true pruned kept "" +test_configured_prune unset true kept kept "--no-prune" +test_configured_prune unset true pruned kept "--prune" + +test_configured_prune false true pruned kept "" +test_configured_prune false true kept kept "--no-prune" +test_configured_prune false true pruned kept "--prune" + +test_configured_prune true true pruned kept "" +test_configured_prune true true pruned kept "--prune" +test_configured_prune true true kept kept "--no-prune" test_expect_success 'all boundary commits are excluded' ' test_commit base &&
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v5 8/17 fetch tests: test --prune and refspec interaction
In reply to Ævar Arnfjörð BjarmasonAdd a test for the interaction between explicitly provided refspecs and fetch.prune.
There's no point in adding this boilerplate to every combination of unset/false/true, it's instructive and sufficient to show that no matter if the variable is unset, false or true the refspec on the command-line overrides any configuration variable.
--- t/t5510-fetch.sh | 12 ++++++++++++ 1 file changed, 12 insertions(+)
t/t5510-fetch.sh +12 -0
@@ -609,6 +609,10 @@ test_configured_prune () { test_configured_prune unset unset kept kept "" test_configured_prune unset unset kept kept "--no-prune" test_configured_prune unset unset pruned kept "--prune" +test_configured_prune unset unset kept pruned \ + "--prune origin refs/tags/*:refs/tags/*" +test_configured_prune unset unset pruned pruned \ + "--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" test_configured_prune false unset kept kept "" test_configured_prune false unset kept kept "--no-prune"
@@ -625,6 +629,10 @@ test_configured_prune unset false pruned kept "--prune" test_configured_prune false false kept kept "" test_configured_prune false false kept kept "--no-prune" test_configured_prune false false pruned kept "--prune" +test_configured_prune false false kept pruned \ + "--prune origin refs/tags/*:refs/tags/*" +test_configured_prune false false pruned pruned \ + "--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" test_configured_prune true false kept kept "" test_configured_prune true false pruned kept "--prune"
@@ -641,6 +649,10 @@ test_configured_prune false true pruned kept "--prune" test_configured_prune true true pruned kept "" test_configured_prune true true pruned kept "--prune" test_configured_prune true true kept kept "--no-prune" +test_configured_prune true true kept pruned \ + "--prune origin refs/tags/*:refs/tags/*" +test_configured_prune true true pruned pruned \ + "--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" test_expect_success 'all boundary commits are excluded' ' test_commit base &&
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v5 9/17 fetch tests: double quote a variable for interpolation
In reply to Ævar Arnfjörð BjarmasonIf the $cmdline variable contains arguments with spaces they won't be interpolated correctly, since the body of the test is single quoted, and because test-lib.sh does its own eval().
This will be used in a subsequent commit to pass arguments that need to be quoted to git-fetch, i.e. a file:// path to fetch, which will have a space in it.
--- t/t5510-fetch.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
t/t5510-fetch.sh +1 -1
@@ -578,7 +578,7 @@ test_configured_prune () { set_config_tristate fetch.prune $fetch_prune && set_config_tristate remote.origin.prune $remote_origin_prune && - git fetch $cmdline && + git fetch '"$cmdline"' && case "$expected_branch" in pruned) test_must_fail git rev-parse --verify refs/remotes/origin/newbranch
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v5 10/17 fetch tests: expand case/esac for later change
In reply to Ævar Arnfjörð BjarmasonExpand a compact case/esac statement for a later change that'll add more logic to the body of the "*" case. This is a whitespace-only change.
--- t/t5510-fetch.sh | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
t/t5510-fetch.sh +6 -2
@@ -543,8 +543,12 @@ test_expect_success "should be able to fetch with duplicate refspecs" ' set_config_tristate () { # var=$1 val=$2 case "$2" in - unset) test_unconfig "$1" ;; - *) git config "$1" "$2" ;; + unset) + test_unconfig "$1" + ;; + *) + git config "$1" "$2" + ;; esac }
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v5 11/17 fetch tests: fetch <url> <spec> as well as fetch [<remote>]
In reply to Ævar Arnfjörð BjarmasonWhen a remote URL is supplied on the command-line the internals of the fetch are different, in particular the code in get_ref_map(). An earlier version of the subsequent fetch.pruneTags patch hid a segfault because the difference wasn't tested for.
Now all the tests are run as both of the variants of:
git fetch
git -c [...] fetch $(git config remote.origin.url) $(git config remote.origin.fetch)I'm using -c because while the [fetch] config just set by set_config_tristate will be picked up, the remote.origin.* config won't override it as intended.
Work around that and turn this into a purely command-line test by always setting the variables on the command-line, and translate any setting of remote.origin.X into fetch.X.
The reason for choosing the names "name" and "link" as opposed to e.g. "named" and "url" is because they're the same length, which makes the test output easier to read as it will be aligned.
Due to shellscript quoting madness it's not worthwhile to do all of this within a test_expect_success, but do the parts that can easily be done there, including the one-time setting of variables that don't change between runs to be used by subsequent runs in the 'prune_type setup' test.
--- t/t5510-fetch.sh | 44 ++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 40 insertions(+), 4 deletions(-)
t/t5510-fetch.sh +40 -4
@@ -548,18 +548,49 @@ set_config_tristate () { ;; *) git config "$1" "$2" + key=$(echo $1 | sed -e 's/^remote\.origin/fetch/') + git_fetch_c="$git_fetch_c -c $key=$2" ;; esac } test_configured_prune () { + test_configured_prune_type "$@" "name" + test_configured_prune_type "$@" "link" +} + +test_configured_prune_type () { fetch_prune=$1 remote_origin_prune=$2 expected_branch=$3 expected_tag=$4 cmdline=$5 - - test_expect_success "prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" ' + mode=$6 + + if test -z "$cmdline_setup" + then + test_expect_success 'setup cmdline_setup variable for subsequent test' ' + remote_url="file://$(git -C one config remote.origin.url)" && + remote_fetch="$(git -C one config remote.origin.fetch)" && + cmdline_setup="\"$remote_url\" \"$remote_fetch\"" + ' + fi + + if test "$mode" = 'link' + then + new_cmdline="" + + if test "$cmdline" = "" + then + new_cmdline=$cmdline_setup + else + new_cmdline=$(printf "%s" "$cmdline" | perl -pe 's[origin(?!/)]["'"$remote_url"'"]g') + fi + + cmdline="$new_cmdline" + fi + + test_expect_success "$mode prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && git tag -f newtag &&
@@ -567,7 +598,7 @@ test_configured_prune () { cd one && test_unconfig fetch.prune && test_unconfig remote.origin.prune && - git fetch && + git fetch '"$cmdline_setup"' && git rev-parse --verify refs/remotes/origin/newbranch && git rev-parse --verify refs/tags/newtag ) &&
@@ -579,10 +610,15 @@ test_configured_prune () { # then test ( cd one && + git_fetch_c="" && set_config_tristate fetch.prune $fetch_prune && set_config_tristate remote.origin.prune $remote_origin_prune && - git fetch '"$cmdline"' && + if test "$mode" != "link" + then + git_fetch_c="" + fi && + git$git_fetch_c fetch '"$cmdline"' && case "$expected_branch" in pruned) test_must_fail git rev-parse --verify refs/remotes/origin/newbranch
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v5 12/17 git fetch doc: add a new section to explain the ins & outs of pruning
In reply to Ævar Arnfjörð BjarmasonAdd a new section to canonically explain how remote reference pruning works, and how users should be careful about using it in conjunction with tag refspecs in particular.
A subsequent commit will update the git-remote documentation to refer to this section, and details the motivation for writing this in the first place.
--- Documentation/git-fetch.txt | 49 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+)
Documentation/git-fetch.txt +49 -0
@@ -99,6 +99,55 @@ The latter use of the `remote.<repository>.fetch` values can be overridden by giving the `--refmap=<refspec>` parameter(s) on the command line. +PRUNING +------- + +Git has a default disposition of keeping data unless it's explicitly +thrown away; this extends to holding onto local references to branches +on remotes that have themselves deleted those branches. + +If left to accumulate, these stale references might make performance +worse on big and busy repos that have a lot of branch churn, and +e.g. make the output of commands like `git branch -a --contains +<commit>` needlessly verbose, as well as impacting anything else +that'll work with the complete set of known references. + +These remote-tracking references can be deleted as a one-off with +either of: + +------------------------------------------------ +# While fetching +$ git fetch --prune <name> + +# Only prune, don't fetch +$ git remote prune <name> +------------------------------------------------ + +To prune references as part of your normal workflow without needing to +remember to run that, set `fetch.prune` globally, or +`remote.<name>.prune` per-remote in the config. See +linkgit:git-config[1]. + +Here's where things get tricky and more specific. The pruning feature +doesn't actually care about branches, instead it'll prune local <-> +remote-references as a function of the refspec of the remote (see +`<refspec>` and <<CRTB,CONFIGURED REMOTE-TRACKING BRANCHES>> above). + +Therefore if the refspec for the remote includes +e.g. `refs/tags/*:refs/tags/*`, or you manually run e.g. `git fetch +--prune <name> "refs/tags/*:refs/tags/*"` it won't be stale remote +tracking branches that are deleted, but any local tag that doesn't +exist on the remote. + +This might not be what you expect, i.e. you want to prune remote +`<name>`, but also explicitly fetch tags from it, so when you fetch +from it you delete all your local tags, most of which may not have +come from the `<name>` remote in the first place. + +So be careful when using this with a refspec like +`refs/tags/*:refs/tags/*`, or any other refspec which might map +references from multiple remotes to the same local namespace. + OUTPUT ------
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v5 13/17 git remote doc: correct dangerous lies about what prune does
In reply to Ævar Arnfjörð BjarmasonThe "git remote prune <name>" command uses the same machinery as "git fetch <name> --prune", and shares all the same caveats, but its documentation has suggested that it'll just "delete stale remote-tracking branches under <name>".
This isn't true, and hasn't been true since at least v1.8.5.6 (the oldest version I could be bothered to test).
E.g. if "refs/tags/*:refs/tags/*" is explicitly set in the refspec of the remote, it'll delete all local tags <name> doesn't know about.
Instead, briefly give the reader just enough of a hint that this option might constitute a shotgun aimed at their foot, and point them to the new PRUNING section in the git-fetch documentation which explains all the nuances of what this facility does.
See "[BUG] git remote prune removes local tags, depending on fetch config" (CACi5S_39wNrbfjLfn0xhCY+uewtFN2YmnAcRc86z6pjUTjWPHQ@mail.gmail.com) by Michael Giuffrida for the initial report.
--- Documentation/git-remote.txt | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
Documentation/git-remote.txt +9 -5
@@ -172,10 +172,14 @@ With `-n` option, the remote heads are not queried first with 'prune':: -Deletes all stale remote-tracking branches under <name>. -These stale branches have already been removed from the remote repository -referenced by <name>, but are still locally available in -"remotes/<name>". +Deletes stale references associated with <name>. By default, stale +remote-tracking branches under <name> are deleted, but depending on +global configuration and the configuration of the remote we might even +prune local tags that haven't been pushed there. Equivalent to `git +fetch --prune <name>`, except that no new references will be fetched. ++ +See the PRUNING section of linkgit:git-fetch[1] for what it'll prune +depending on various configuration. + With `--dry-run` option, report what branches will be pruned, but do not actually prune them.
@@ -189,7 +193,7 @@ remotes.default is not defined, all remotes which do not have the configuration parameter remote.<name>.skipDefaultUpdate set to true will be updated. (See linkgit:git-config[1]). + -With `--prune` option, prune all the remotes that are updated. +With `--prune` option, run pruning against all the remotes that are updated. DISCUSSION
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v5 14/17 git-fetch & config doc: link to the new PRUNING section
In reply to Ævar Arnfjörð BjarmasonAmend the documentation for fetch.prune, fetch.<name>.prune and --prune to link to the recently added PRUNING section.
I'd have liked to link directly to it with "<<PRUNING>>" from fetch-options.txt, since it's included in git-fetch.txt (git-pull.txt also includes it, but doesn't include that option). However making a reference across files yields this error:
[...]/Documentation/git-fetch.xml:226: element xref: validity error : IDREF attribute linkend references an unknown ID "PRUNING"
--- Documentation/config.txt | 6 +++++- Documentation/fetch-options.txt | 3 +++ 2 files changed, 8 insertions(+), 1 deletion(-)
Documentation/config.txt +5 -1
@@ -1398,7 +1398,8 @@ fetch.unpackLimit:: fetch.prune:: If true, fetch will automatically behave as if the `--prune` - option was given on the command line. See also `remote.<name>.prune`. + option was given on the command line. See also `remote.<name>.prune` + and the PRUNING section of linkgit:git-fetch[1]. fetch.output:: Control how ref update status is printed. Valid values are
@@ -2944,6 +2945,9 @@ remote.<name>.prune:: remove any remote-tracking references that no longer exist on the remote (as if the `--prune` option was given on the command line). Overrides `fetch.prune` settings, if any. ++ +See also `remote.<name>.prune` and the PRUNING section of +linkgit:git-fetch[1]. remotes.<group>:: The list of remotes which are fetched by "git remote update
Documentation/fetch-options.txt +3 -0
@@ -74,6 +74,9 @@ ifndef::git-pull[] line or in the remote configuration, for example if the remote was cloned with the --mirror option), then they are also subject to pruning. ++ +See the PRUNING section below for more details. + endif::git-pull[] ifndef::git-pull[]
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v5 15/17 fetch tests: add scaffolding for the new fetch.pruneTags
In reply to Ævar Arnfjörð BjarmasonThe fetch.pruneTags configuration doesn't exist yet, but will be added in a subsequent commit. Since testing for it requires adding new parameters to the test_configured_prune function it's easier to review this patch first to assert that no functional changes are introduced yet.
--- t/t5510-fetch.sh | 92 ++++++++++++++++++++++++++++++-------------------------- 1 file changed, 50 insertions(+), 42 deletions(-)
t/t5510-fetch.sh +50 -42
@@ -562,10 +562,12 @@ test_configured_prune () { test_configured_prune_type () { fetch_prune=$1 remote_origin_prune=$2 - expected_branch=$3 - expected_tag=$4 - cmdline=$5 - mode=$6 + fetch_prune_tags=$3 + remote_origin_prune_tags=$4 + expected_branch=$5 + expected_tag=$6 + cmdline=$7 + mode=$8 if test -z "$cmdline_setup" then
@@ -590,14 +592,16 @@ test_configured_prune_type () { cmdline="$new_cmdline" fi - test_expect_success "$mode prune fetch.prune=$1 remote.origin.prune=$2${5:+ $5}; branch:$3 tag:$4" ' + test_expect_success "$mode prune fetch.prune=$1 remote.origin.prune=$2 fetch.pruneTags=$3 remote.origin.pruneTags=$4${7:+ $7}; branch:$5 tag:$6" ' # make sure a newbranch is there in . and also in one git branch -f newbranch && git tag -f newtag && ( cd one && test_unconfig fetch.prune && + test_unconfig fetch.pruneTags && test_unconfig remote.origin.prune && + test_unconfig remote.origin.pruneTags && git fetch '"$cmdline_setup"' && git rev-parse --verify refs/remotes/origin/newbranch && git rev-parse --verify refs/tags/newtag
@@ -612,7 +616,9 @@ test_configured_prune_type () { cd one && git_fetch_c="" && set_config_tristate fetch.prune $fetch_prune && + set_config_tristate fetch.pruneTags $fetch_prune_tags && set_config_tristate remote.origin.prune $remote_origin_prune && + set_config_tristate remote.origin.pruneTags $remote_origin_prune_tags && if test "$mode" != "link" then
@@ -641,57 +647,59 @@ test_configured_prune_type () { # $1 config: fetch.prune # $2 config: remote.<name>.prune -# $3 expect: branch to be pruned? -# $4 expect: tag to be pruned? -# $5 git-fetch $cmdline: +# $3 config: fetch.pruneTags +# $4 config: remote.<name>.pruneTags +# $5 expect: branch to be pruned? +# $6 expect: tag to be pruned? +# $7 git-fetch $cmdline: # -# $1 $2 $3 $4 $5 -test_configured_prune unset unset kept kept "" -test_configured_prune unset unset kept kept "--no-prune" -test_configured_prune unset unset pruned kept "--prune" -test_configured_prune unset unset kept pruned \ +# $1 $2 $3 $4 $5 $6 $7 +test_configured_prune unset unset unset unset kept kept "" +test_configured_prune unset unset unset unset kept kept "--no-prune" +test_configured_prune unset unset unset unset pruned kept "--prune" +test_configured_prune unset unset unset unset kept pruned \ "--prune origin refs/tags/*:refs/tags/*" -test_configured_prune unset unset pruned pruned \ +test_configured_prune unset unset unset unset pruned pruned \ "--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" -test_configured_prune false unset kept kept "" -test_configured_prune false unset kept kept "--no-prune" -test_configured_prune false unset pruned kept "--prune" +test_configured_prune false unset unset unset kept kept "" +test_configured_prune false unset unset unset kept kept "--no-prune" +test_configured_prune false unset unset unset pruned kept "--prune" -test_configured_prune true unset pruned kept "" -test_configured_prune true unset pruned kept "--prune" -test_configured_prune true unset kept kept "--no-prune" +test_configured_prune true unset unset unset pruned kept "" +test_configured_prune true unset unset unset pruned kept "--prune" +test_configured_prune true unset unset unset kept kept "--no-prune" -test_configured_prune unset false kept kept "" -test_configured_prune unset false kept kept "--no-prune" -test_configured_prune unset false pruned kept "--prune" +test_configured_prune unset false unset unset kept kept "" +test_configured_prune unset false unset unset kept kept "--no-prune" +test_configured_prune unset false unset unset pruned kept "--prune" -test_configured_prune false false kept kept "" -test_configured_prune false false kept kept "--no-prune" -test_configured_prune false false pruned kept "--prune" -test_configured_prune false false kept pruned \ +test_configured_prune false false unset unset kept kept "" +test_configured_prune false false unset unset kept kept "--no-prune" +test_configured_prune false false unset unset pruned kept "--prune" +test_configured_prune false false unset unset kept pruned \ "--prune origin refs/tags/*:refs/tags/*" -test_configured_prune false false pruned pruned \ +test_configured_prune false false unset unset pruned pruned \ "--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" -test_configured_prune true false kept kept "" -test_configured_prune true false pruned kept "--prune" -test_configured_prune true false kept kept "--no-prune" +test_configured_prune true false unset unset kept kept "" +test_configured_prune true false unset unset pruned kept "--prune" +test_configured_prune true false unset unset kept kept "--no-prune" -test_configured_prune unset true pruned kept "" -test_configured_prune unset true kept kept "--no-prune" -test_configured_prune unset true pruned kept "--prune" +test_configured_prune unset true unset unset pruned kept "" +test_configured_prune unset true unset unset kept kept "--no-prune" +test_configured_prune unset true unset unset pruned kept "--prune" -test_configured_prune false true pruned kept "" -test_configured_prune false true kept kept "--no-prune" -test_configured_prune false true pruned kept "--prune" +test_configured_prune false true unset unset pruned kept "" +test_configured_prune false true unset unset kept kept "--no-prune" +test_configured_prune false true unset unset pruned kept "--prune" -test_configured_prune true true pruned kept "" -test_configured_prune true true pruned kept "--prune" -test_configured_prune true true kept kept "--no-prune" -test_configured_prune true true kept pruned \ +test_configured_prune true true unset unset pruned kept "" +test_configured_prune true true unset unset pruned kept "--prune" +test_configured_prune true true unset unset kept kept "--no-prune" +test_configured_prune true true unset unset kept pruned \ "--prune origin refs/tags/*:refs/tags/*" -test_configured_prune true true pruned pruned \ +test_configured_prune true true unset unset pruned pruned \ "--prune origin refs/tags/*:refs/tags/* +refs/heads/*:refs/remotes/origin/*" test_expect_success 'all boundary commits are excluded' '
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v5 16/17 fetch: add a --prune-tags option and fetch.pruneTags config
In reply to Ævar Arnfjörð BjarmasonAdd a --prune-tags option to git-fetch, along with fetch.pruneTags config option and a -P shorthand (-p is --prune). This allows for doing any of:
git fetch -p -P
git fetch --prune --prune-tags
git fetch -p -P origin
git fetch --prune --prune-tags origin
Or simply:
git config fetch.prune true &&
git config fetch.pruneTags true &&
git fetchInstead of the much more verbose:
git fetch --prune origin 'refs/tags/*:refs/tags/*' '+refs/heads/*:refs/remotes/origin/*'
Before this feature it was painful to support the use-case of pulling from a repo which is having both its branches *and* tags deleted regularly, and have our local references to reflect upstream.
At work we create deployment tags in the repo for each rollout, and there's *lots* of those, so they're archived within weeks for performance reasons.
Without this change it's hard to centrally configure such repos in /etc/gitconfig (on servers that are only used for working with them). You need to set fetch.prune=true globally, and then for each repo:
git -C {} config --replace-all remote.origin.fetch "refs/tags/*:refs/tags/*" "^\+*refs/tags/\*:refs/tags/\*$"Now I can simply set fetch.pruneTags=true in /etc/gitconfig as well, and users running "git pull" will automatically get the pruning semantics I want.
Even though "git remote" has corresponding "prune" and "update --prune" subcommands I'm intentionally not adding a corresponding prune-tags or "update --prune --prune-tags" mode to that command.
It's advertised (as noted in my recent "git remote doc: correct dangerous lies about what prune does") as only modifying remote tracking references, whereas any --prune-tags option is always going to modify what from the user's perspective is a local copy of the tag, since there's no such thing as a remote tracking tag.
Ideally add_prune_tags_to_fetch_refspec() would be something that would use ALLOC_GROW() to grow the 'fetch` member of the 'remote' struct. Instead I'm realloc-ing remote->fetch and adding the tag_refspec to the end.
The reason is that parse_{fetch,push}_refspec which allocate the refspec (ultimately remote->fetch) struct are called many places that don't have access to a 'remote' struct. It would be hard to change all their callsites to be amenable to carry around the bookkeeping variables required for dynamic allocation.
All the other callers of the API first incrementally construct the string version of the refspec in remote->fetch_refspec via add_fetch_refspec(), before finally calling parse_fetch_refspec() via some variation of remote_get().
It's less of a pain to deal with the one special case that needs to modify already constructed refspecs than to chase down and change all the other callsites. The API I'm adding is intentionally not generalized because if we add more of these we'd probably want to re-visit how this is done.
See my "Re: [BUG] git remote prune removes local tags, depending on fetch config" ([email protected];
https://public-inbox.org/git/[email protected]/) for more background info.
--- Documentation/config.txt | 14 +++++++ Documentation/fetch-options.txt | 14 ++++++- Documentation/git-fetch.txt | 47 +++++++++++++++++++++++ builtin/fetch.c | 32 ++++++++++++++-- contrib/completion/git-completion.bash | 2 +- remote.c | 14 +++++++ remote.h | 3 ++ t/t5510-fetch.sh | 70 ++++++++++++++++++++++++++++++++++ 8 files changed, 191 insertions(+), 5 deletions(-)
Documentation/config.txt +14 -0
@@ -1401,6 +1401,14 @@ fetch.prune:: option was given on the command line. See also `remote.<name>.prune` and the PRUNING section of linkgit:git-fetch[1]. +fetch.pruneTags:: + If true, fetch will automatically behave as if the + `refs/tags/*:refs/tags/*` refspec was provided when pruning, + if not set already. This allows for setting both this option + and `fetch.prune` to maintain a 1=1 mapping to upstream + refs. See also `remote.<name>.pruneTags` and the PRUNING + section of linkgit:git-fetch[1]. + fetch.output:: Control how ref update status is printed. Valid values are `full` and `compact`. Default value is `full`. See section
@@ -2945,6 +2953,12 @@ remote.<name>.prune:: remove any remote-tracking references that no longer exist on the remote (as if the `--prune` option was given on the command line). Overrides `fetch.prune` settings, if any. + +remote.<name>.pruneTags:: + When set to true, fetching from this remote by default will also + remove any local tags that no longer exist on the remote if pruning + is activated in general via `remote.<name>.prune`, `fetch.prune` or + `--prune`. Overrides `fetch.pruneTags` settings, if any. + See also `remote.<name>.prune` and the PRUNING section of linkgit:git-fetch[1].
Documentation/fetch-options.txt +13 -1
@@ -73,7 +73,19 @@ ifndef::git-pull[] are fetched due to an explicit refspec (either on the command line or in the remote configuration, for example if the remote was cloned with the --mirror option), then they are also - subject to pruning. + subject to pruning. Supplying `--prune-tags` is a shorthand for + providing the tag refspec. ++ +See the PRUNING section below for more details. + +-P:: +--prune-tags:: + Before fetching, remove any local tags that no longer exist on + the remote if `--prune` is enabled. This option should be used + more carefully, unlike `--prune` it will remove any local + references (local tags) that have been created. This option is + a shorthand for providing the explicit tag refspec along with + `--prune`, see the discussion about that in its documentation. + See the PRUNING section below for more details.
Documentation/git-fetch.txt +47 -0
53 lines not on this page · raw message
builtin/fetch.c +29 -3
@@ -38,6 +38,10 @@ static int fetch_prune_config = -1; /* unspecified */static int prune =-1 ;/* unspecified */ #define PRUNE_BY_DEFAULT 0/* do we prune by default? */ +static int fetch_prune_tags_config =-1 ;/* unspecified */ +static int prune_tags =-1 ;/* unspecified */ +#define PRUNE_TAGS_BY_DEFAULT 0/* do we prune tags by default? */ +static int all, append, dry_run, force, keep, multiple, update_head_ok, verbosity, deepen_relative;static int progress =-1 ;static int tags =TAGS_DEFAULT , unshallow, update_shallow, deepen;
@@ -64,6 +68,11 @@ static int git_fetch_config(const char *k, const char *v, void *cb)return 0 ; } +if (!strcmp (k,"fetch.prunetags" )) { + fetch_prune_tags_config =git_config_bool (k, v); +return 0 ; + } +if (!strcmp (k,"submodule.recurse" )) {int r =git_config_bool (k, v) ?RECURSE_SUBMODULES_ON :RECURSE_SUBMODULES_OFF ;
@@ -126,6 +135,8 @@ static struct option builtin_fetch_options[] = { N_("number of submodules fetched in parallel")),OPT_BOOL ('p' ,"prune" , &prune,N_ ("prune remote-tracking branches no longer on remote" )), +OPT_BOOL ('P' ,"prune-tags" , &prune_tags, +N_ ("prune local tags no longer on remote and clobber changed tags" )), { OPTION_CALLBACK, 0, "recurse-submodules", &recurse_submodules, N_("on-demand"),N_ ("control recursive fetching of submodules" ), PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules },
@@ -1212,6 +1223,8 @@ static void add_options_to_argv(struct argv_array *argv)argv_array_push (argv,"--dry-run" );if (prune !=-1 )argv_array_push (argv, prune ?"--prune" :"--no-prune" ); +if (prune_tags !=-1 ) +argv_array_push (argv, prune_tags ?"--prune-tags" :"--no-prune-tags" );if (update_head_ok)argv_array_push (argv,"--update-head-ok" );if (force)
@@ -1265,7 +1278,7 @@ static int fetch_multiple(struct string_list *list)return result; } -static int fetch_one (struct remote *remote,int argc,const char **argv) +static int fetch_one (struct remote *remote,int argc,const char **argv,int prune_tags_ok) {static const char **refs =NULL ;struct refspec *refspec;
@@ -1288,6 +1301,19 @@ static int fetch_one(struct remote *remote, int argc, const char **argv) prune =PRUNE_BY_DEFAULT ; } +if (prune_tags <0 ) { +/* no command line request */ +if (0 <= remote->prune_tags) + prune_tags = remote->prune_tags; +else if (0 <= fetch_prune_tags_config) + prune_tags = fetch_prune_tags_config; +else + prune_tags =PRUNE_TAGS_BY_DEFAULT ; + } + +if (prune_tags_ok && prune_tags &&remote_is_configured (remote,0 )) +add_prune_tags_to_fetch_refspec (remote); +if (argc >0 ) {int j =0 ;int i;
@@ -1368,7 +1394,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) }else if (argc ==0 ) {/* No arguments -- use default remote */ remote =remote_get (NULL ); - result =fetch_one (remote, argc, argv); + result =fetch_one (remote, argc, argv,1 ); }else if (multiple) {/* All arguments are assumed to be remotes or groups */ for (i =0 ; i < argc; i++)
@@ -1386,7 +1412,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) }else {/* Zero or one remotes */ remote =remote_get (argv[0 ]); - result =fetch_one (remote, argc-1 , argv+1 ); + result =fetch_one (remote, argc-1 , argv+1 , argc ==1 ); } }
contrib/completion/git-completion.bash +1 -1
@@ -1468,7 +1468,7 @@ __git_fetch_recurse_submodules="yes on-demand no" __git_fetch_options=" --quiet --verbose --append --upload-pack --force --keep --depth= --tags --no-tags --all --prune --dry-run --recurse-submodules= - --unshallow --update-shallow + --unshallow --update-shallow --prune-tags " _git_fetch ()
remote.c +14 -0
@@ -104,6 +104,17 @@ static void add_fetch_refspec(struct remote *remote, const char *ref) remote->fetch_refspec[remote->fetch_refspec_nr++] = ref; } +void add_prune_tags_to_fetch_refspec (struct remote *remote) +{ +int nr = remote->fetch_refspec_nr; +int bufsize = nr +1 ; +int size =sizeof (struct refspec ); + + remote->fetch =xrealloc (remote->fetch, size * bufsize); +memcpy (&remote->fetch[nr], tag_refspec, size); +add_fetch_refspec (remote,xstrdup (TAG_REFSPEC )); +} +static void add_url (struct remote *remote,const char *url) {ALLOC_GROW (remote->url, remote->url_nr +1 , remote->url_alloc);
@@ -174,6 +185,7 @@ static struct remote *make_remote(const char *name, int len) ret =xcalloc (1 ,sizeof (struct remote )); ret->prune =-1 ;/* unspecified */ + ret->prune_tags =-1 ;/* unspecified */ ALLOC_GROW (remotes, remotes_nr +1 , remotes_alloc); remotes[remotes_nr++] = ret; ret->name =xstrndup (name, len);
@@ -392,6 +404,8 @@ static int handle_config(const char *key, const char *value, void *cb) remote->skip_default_update =git_config_bool (key, value); else if (!strcmp(subkey, "prune")) remote->prune =git_config_bool (key, value); + else if (!strcmp(subkey, "prunetags")) + remote->prune_tags =git_config_bool (key, value); else if (!strcmp(subkey, "url")) {const char *v;if (git_config_string (&v, key, value))
remote.h +3 -0
@@ -47,6 +47,7 @@ struct remote {int skip_default_update;int mirror;int prune; +int prune_tags;const char *receivepack;const char *uploadpack;
@@ -299,4 +300,6 @@ void apply_push_cas(struct push_cas_option *, struct remote *, struct ref *); #define TAG_REFSPEC "refs/tags/*:refs/tags/*" +void add_prune_tags_to_fetch_refspec(struct remote *remote); +#endif
t/t5510-fetch.sh +70 -0
82 lines not on this page · raw message
Signature · 1 line
2.15.1.424.g9478a66081
PATCH v5 17/17 fetch: make the --prune-tags work with <url>
In reply to Ævar Arnfjörð BjarmasonMake the new --prune-tags option work properly when git-fetch is invoked with a <url> parameter instead of a <remote name> parameter.
This change is split off from the introduction of --prune-tags due to the relative complexity of munging the incoming argv, which is easier to review as a separate change.
--- Documentation/git-fetch.txt | 21 ++++++--------------- builtin/fetch.c | 17 ++++++++++++++--- t/t5510-fetch.sh | 16 +++++++--------- 3 files changed, 27 insertions(+), 27 deletions(-)
Documentation/git-fetch.txt +6 -15
@@ -175,24 +175,15 @@ It's reasonable to e.g. configure `fetch.pruneTags=true` in run, without making every invocation of `git fetch` without `--prune` an error. -Another special case of `--prune-tags` is that -`refs/tags/*:refs/tags/*` will not be implicitly provided if an URL is -being fetched. I.e.: - ------------------------------------------------- -$ git fetch <url> --prune --prune-tags ------------------------------------------------- - -Will prune no tags, as opposed to: +Pruning tags with `--prune-tags` also works when fetching a URL +instead of a named remote. These will all prune tags not found on +origin: ------------------------------------------------ $ git fetch origin --prune --prune-tags ------------------------------------------------- - -To prune tags given a URL supply the refspec explicitly: - ------------------------------------------------- -$ git fetch <url> --prune 'refs/tags/*:refs/tags/*' +$ git fetch origin --prune 'refs/tags/*:refs/tags/*' +$ git fetch <url of origin> --prune --prune-tags +$ git fetch <url of origin> --prune 'refs/tags/*:refs/tags/*' ------------------------------------------------ OUTPUT
builtin/fetch.c +14 -3
@@ -1283,7 +1283,10 @@ static int fetch_one(struct remote *remote, int argc, const char **argv, int prustatic const char **refs =NULL ;struct refspec *refspec;int ref_nr =0 ; +int j =0 ;int exit_code; +int maybe_prune_tags; +int remote_via_config =remote_is_configured (remote,0 );if (!remote) die(_("No remote repository specified. Please, specify either a URL or a\n"
@@ -1311,13 +1314,21 @@ static int fetch_one(struct remote *remote, int argc, const char **argv, int pru prune_tags =PRUNE_TAGS_BY_DEFAULT ; } -if (prune_tags_ok && prune_tags &&remote_is_configured (remote,0 )) + maybe_prune_tags = prune_tags_ok && prune_tags; +if (maybe_prune_tags && remote_via_config)add_prune_tags_to_fetch_refspec (remote); +if (argc >0 || (maybe_prune_tags && !remote_via_config)) { +size_t nr_alloc =st_add3 (argc, maybe_prune_tags,1 ); + refs =xcalloc (nr_alloc,sizeof (const char *)); +if (maybe_prune_tags) { + refs[j++] =xstrdup ("refs/tags/*:refs/tags/*" ); + ref_nr++; + } + } +if (argc >0 ) { -int j =0 ;int i; - refs =xcalloc (st_add (argc,1 ),sizeof (const char *));for (i =0 ; i < argc; i++) {if (!strcmp (argv[i],"tag" )) { i++;
t/t5510-fetch.sh +7 -9
@@ -738,18 +738,15 @@ test_configured_prune unset unset unset true pruned kept \ "--prune origin +refs/heads/*:refs/remotes/origin/*" # Pruning that also takes place if a file:// url replaces a named -# remote, with the exception of --prune-tags on the command-line -# (arbitrary limitation). -# -# However, because there's no implicit +# remote. However, because there's no implicit # +refs/heads/*:refs/remotes/origin/* refspec and supplying it on the # command-line negates --prune-tags, the branches will not be pruned. test_configured_prune_type unset unset unset unset kept kept "origin --prune-tags" "name" test_configured_prune_type unset unset unset unset kept kept "origin --prune-tags" "link" test_configured_prune_type unset unset unset unset pruned pruned "origin --prune --prune-tags" "name" -test_configured_prune_type unset unset unset unset kept kept "origin --prune --prune-tags" "link" +test_configured_prune_type unset unset unset unset kept pruned "origin --prune --prune-tags" "link" test_configured_prune_type unset unset unset unset pruned pruned "--prune --prune-tags origin" "name" -test_configured_prune_type unset unset unset unset kept kept "--prune --prune-tags origin" "link" +test_configured_prune_type unset unset unset unset kept pruned "--prune --prune-tags origin" "link" test_configured_prune_type unset unset true unset pruned pruned "--prune origin" "name" test_configured_prune_type unset unset true unset kept pruned "--prune origin" "link" test_configured_prune_type unset unset unset true pruned pruned "--prune origin" "name"
@@ -761,8 +758,9 @@ test_configured_prune_type unset true true unset kept pruned "origin" "link" test_configured_prune_type unset true unset true pruned pruned "origin" "name" test_configured_prune_type unset true unset true kept pruned "origin" "link" -# Interaction between --prune-tags and no "fetch" config in the remote -# at all. +# When all remote.origin.fetch settings are deleted a --prune +# --prune-tags still implicitly supplies refs/tags/*:refs/tags/* so +# tags, but not tracking branches, will be deleted. test_expect_success 'remove remote.origin.fetch "one"' ' ( cd one &&
@@ -770,7 +768,7 @@ test_expect_success 'remove remote.origin.fetch "one"' ' ) ' test_configured_prune_type unset unset unset unset kept pruned "origin --prune --prune-tags" "name" -test_configured_prune_type unset unset unset unset kept kept "origin --prune --prune-tags" "link" +test_configured_prune_type unset unset unset unset kept pruned "origin --prune --prune-tags" "link" test_expect_success 'all boundary commits are excluded' ' test_commit base &&
Signature · 1 line
2.15.1.424.g9478a66081
Re: [PATCH v5 00/17] document & test fetch pruning & add fetch.pruneTags
In reply to Ævar Arnfjörð BjarmasonQuoted 2 lines · Ævar Arnfjörð Bjarmason
Here's a v5 (correct subject line this time!). Many thanks to Eric for a thorough review.
We haven't seen any comments on this round. Is everybody happy?
I do not have a strong opinion on the new feature, either for or against. I didn't find anything majorly questionable in the execution, though, so...
Re: [PATCH v5 00/17] document & test fetch pruning & add fetch.pruneTags
On Thu, Feb 22 2018, Junio C. Hamano jotted:
Quoted 10 lines
Quoted 2 lines · Ævar Arnfjörð Bjarmason
Here's a v5 (correct subject line this time!). Many thanks to Eric for a thorough review.
We haven't seen any comments on this round. Is everybody happy?
I do not have a strong opinion on the new feature, either for or against. I didn't find anything majorly questionable in the execution, though, so...
I've been running that here on thousands of boxes (that are actively using it) for 2 weeks now without issue. Would be great to have it merged down & in 2.17.
Re: [PATCH v5 00/17] document & test fetch pruning & add fetch.pruneTags
Quoted 9 lines · Ævar Arnfjörð Bjarmason
On Thu, Feb 22 2018, Junio C. Hamano jotted:
Quoted 7 lines
Quoted 2 lines · Ævar Arnfjörð Bjarmason
Here's a v5 (correct subject line this time!). Many thanks to Eric for a thorough review.
We haven't seen any comments on this round. Is everybody happy?
Quoted 7 lines
Quoted 3 lines
I do not have a strong opinion on the new feature, either for or against. I didn't find anything majorly questionable in the execution, though, so...
I've been running that here on thousands of boxes (that are actively using it) for 2 weeks now without issue. Would be great to have it merged down & in 2.17.
If those thousands of boxes are all employing one specific workflow that is helped by these changes, and the workflow is that other people do not care about (or even worse, actively do not want to let their junior project members to use without thinking), then a data-point from the original author does not amount to much ;-)
Let's see how others find it useful and/or if the changed code gets in the way of others (I am not absolutely sure if the changes are free of regression to existing users who do not use the new feature).
Thanks.
Re: [PATCH v5 00/17] document & test fetch pruning & add fetch.pruneTags
On Thu, Feb 22 2018, Junio C. Hamano jotted:
Quoted 19 lines
Quoted 9 lines · Ævar Arnfjörð Bjarmason
On Thu, Feb 22 2018, Junio C. Hamano jotted:
Quoted 7 lines
Quoted 2 lines · Ævar Arnfjörð Bjarmason
Here's a v5 (correct subject line this time!). Many thanks to Eric for a thorough review.
We haven't seen any comments on this round. Is everybody happy?
Quoted 7 lines
Quoted 3 lines
I do not have a strong opinion on the new feature, either for or against. I didn't find anything majorly questionable in the execution, though, so...
I've been running that here on thousands of boxes (that are actively using it) for 2 weeks now without issue. Would be great to have it merged down & in 2.17.
Quoted 5 lines
If those thousands of boxes are all employing one specific workflow that is helped by these changes, and the workflow is that other people do not care about (or even worse, actively do not want to let their junior project members to use without thinking), then a data-point from the original author does not amount to much ;-)
Of course, I should have been clearer. I just meant to chime in with the datapoint that I'm fairly sure this doesn't have any serious bugs given the wide internal testing it's gotten.
Quoted 4 lines
Let's see how others find it useful and/or if the changed code gets in the way of others (I am not absolutely sure if the changes are free of regression to existing users who do not use the new feature).
I think if you're on the fence about merging it down (and others don't chime in saying the want it / like it) it makes sense to merge down 1-14/17 and we could discard 15-17/17 for now for a later re-submission and discussion once the earlier part of the series lands in master.
The earlier part of the series is just trivial code changes that don't change any functionality, more documentation for how existing functionality works, and more thorough testing of existing functionality.
Re: [PATCH v5 00/17] document & test fetch pruning & add fetch.pruneTags
Quoted 9 lines · Ævar Arnfjörð Bjarmason
Quoted 4 lines
Let's see how others find it useful and/or if the changed code gets in the way of others (I am not absolutely sure if the changes are free of regression to existing users who do not use the new feature).
I think if you're on the fence about merging it down (and others don't chime in saying the want it / like it) it makes sense to merge down 1-14/17 and we could discard 15-17/17 for now for a later re-submission and discussion once the earlier part of the series lands in master.
Or we merge everything and let people discover glitches and complain. Reverting the last pieces can wait until then ;-).
Re: [PATCH v5 00/17] document & test fetch pruning & add fetch.pruneTags
On Fri, Feb 23 2018, Junio C. Hamano jotted:
Quoted 7 lines
Quoted 5 lines · Ævar Arnfjörð Bjarmason
Quoted 4 lines
Let's see how others find it useful and/or if the changed code gets in the way of others (I am not absolutely sure if the changes are free of regression to existing users who do not use the new feature).
Quoted 7 lines
Quoted 4 lines
I think if you're on the fence about merging it down (and others don't chime in saying the want it / like it) it makes sense to merge down 1-14/17 and we could discard 15-17/17 for now for a later re-submission and discussion once the earlier part of the series lands in master.
Or we merge everything and let people discover glitches and complain. Reverting the last pieces can wait until then ;-).
Thanks, that obviously works for me.
Quoted 3 lines · Michael Giuffrida
It doesn't seem like a useful feature -- you wouldn't expect
git fetch --pruneto remove your local branches that you were developing on, right?
Don't mix local and remote refs. There is a reason why remote tracking branches are put in a separate name space. If you fetch the remote tags into a separate name space (eg. refs/remote/tags/*:refs/tags/*) then there is no conflict.
Andreas.
Signature · 3 lines
Andreas Schwab, [email protected] GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different."
Quoted 6 lines · Andreas Schwab
Quoted 3 lines · Michael Giuffrida
It doesn't seem like a useful feature -- you wouldn't expect
git fetch --pruneto remove your local branches that you were developing on, right?
Quoted 6 lines
Don't mix local and remote refs. There is a reason why remote tracking branches are put in a separate name space. If you fetch the remote tags into a separate name space (eg. refs/remote/tags/*:refs/tags/*) then there is no conflict.
Andreas.
But then they are no longer considered tags, but remote tracking branches. Tools like git tag and git describe won't consider them, and git branch -r would show them as remote tracking branches.
Quoted 5 lines
Signature · 4 lines
-- Andreas Schwab, [email protected] GPG Key fingerprint = 58CA 54C7 6D53 942B 1756 01D3 44D5 214B 8276 4ED5 "And now for something completely different."