Skip to contentSkip to conversation

git remote update: New option --prune (-p)

PATCH In review Finn Arne Gangstad 4 participants 15 messages Jun 15, 2016 Documentation

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 git remote update: New option --prune (-p)

With the --prune (or -p) option, git remote update will also prune all the remotes that it fetches. Previously, you had to do a manual git remote prune <remote> for each of the remotes you wanted to prune, and this could be tedious with many remotes.

A single command will now update all remotes, and remove all stale
branches: git remote update -p

---
 Documentation/git-remote.txt |    4 ++-
 builtin-remote.c             |   73 ++++++++++++++++++++++++++----------------
 2 files changed, 48 insertions(+), 29 deletions(-)
Documentation/git-remote.txt +3 -1
index c9c0e6f..0b6e67d 100644
@@ -16,7 +16,7 @@ SYNOPSIS
 'git remote set-head' <name> [-a | -d | <branch>]
 'git remote show' [-n] <name>
 'git remote prune' [-n | --dry-run] <name>
-'git remote update' [group]
+'git remote update' [-p | --prune] [group]
 
 DESCRIPTION
 -----------
@@ -125,6 +125,8 @@ the configuration parameter remotes.default will get used; if
 remotes.default is not defined, all remotes which do not have the
 configuration parameter remote.<name>.skipDefaultUpdate set to true will
 be updated.  (See linkgit:git-config[1]).
++
+With `--prune` option, prune all the remotes that are updated.
 
 
 DISCUSSION
builtin-remote.c +45 -28

122 lines not on this page · raw message

Signature · 1 line
1.6.2.1.470.gd21ca.dirty
#1 lore raw

2009/4/2 Finn Arne Gangstad <[email protected]>:

Quoted 4 lines

With the --prune (or -p) option, git remote update will also prune all the remotes that it fetches.  Previously, you had to do a manual git remote prune <remote> for each of the remotes you wanted to prune, and this could be tedious with many remotes.

Yay!

But one question. It seem to me odd to put this as an option to git remote update, and not git remote prune.

I mean, it seems weird that one must say:

   git remote update --prune

and one cannot say:

   git remote prune --all

especially when there is a git remote prune already. It seems a bit counterintuitive to find pruning actions under "update", but not all that strange to find an all "--all" option for the "prune" action.

Although to me having both be allowed and mean the same thing also makes sense.

Anyway, thanks for this regardless, I am looking forward to this functionality. :-)

Cheers,
yves

Signature · 1 line
perl -Mre=debug -e "/just|another|perl|hacker/"
#2 lore raw
Quoted 10 lines · demerphq

But one question. It seem to me odd to put this as an option to git remote update, and not git remote prune.

I mean, it seems weird that one must say:

   git remote update --prune

and one cannot say:

   git remote prune --all

But "git remote update" actually respects "remote groups", so it is not just "--all". I think what you want is "git remote prune <group>".

Quoted 3 lines

especially when there is a git remote prune already. It seems a bit counterintuitive to find pruning actions under "update", but not all that strange to find an all "--all" option for the "prune" action.

I think it makes sense under update as pruning is really just a different (and perhaps slightly more dangerous) form of update. Generally I would only want to run prune after having run update, so combining them makes sense from a workflow perspective.

Quoted 2 lines

Although to me having both be allowed and mean the same thing also makes sense.

I think that would make sense, too.

-Peff

#3 lore raw
Quoted 4 lines · Jeff King

I think it makes sense under update as pruning is really just a different (and perhaps slightly more dangerous) form of update. Generally I would only want to run prune after having run update, so combining them makes sense from a workflow perspective.

I agree with you that "oh by the way please prune as well" makes perfect sense, but I actually would even go stronger than that---if we _were_ adding this command today, I would probably make "update" prune by default, perhaps with an option to skip the pruning step.

I gave the patch an only cursory look, so I wouldn't comment on the implementation; two things I would look at in the code would be if it makes two connections to the remote to learn the same information (which would be bad) and if it skips the pruning stage if the update stage failed (which would probably be a sane precaution).

#4 lore raw
Quoted 6 lines · Junio C Hamano

[...]

I gave the patch an only cursory look, so I wouldn't comment on the implementation; two things I would look at in the code would be if it makes two connections to the remote to learn the same information (which would be bad)

How bad? git remote update execs "git fetch <remote>" to do the fetching part, and after that the information is lost of course. It might be possible to do a --prune option to fetch instead, and just use that directly.

Quoted 2 lines

and if it skips the pruning stage if the update stage failed
(which would probably be a sane precaution).

Yes, this should be fixed.

- Finn Arne

#5 lore raw
Quoted 9 lines · Finn Arne Gangstad
Quoted 6 lines · Junio C Hamano

[...]

I gave the patch an only cursory look, so I wouldn't comment on the implementation; two things I would look at in the code would be if it makes two connections to the remote to learn the same information (which would be bad)

How bad?

I'd say only "could be improved later" bad.

#6 lore raw

v2 cover git remote update: New option --prune (-p)

Quoted 5 lines · Junio C Hamano
Quoted 3 lines · Finn Arne Gangstad
Quoted 2 lines · Junio C Hamano

[...]

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

I gave the patch an only cursory look, so I wouldn't comment on the implementation; two things I would look at in the code would be if it makes two connections to the remote to learn the same information (which would be bad)

How bad?

I'd say only "could be improved later" bad.

Ok. I split the patch into two to make it easier to review. 1/2 just splits out prune_remote() as a separate function, and does not change any behavior.
2/2 adds the new option to remote update.

Finn Arne Gangstad (2):
builtin-remote.c: Split out prune_remote as a separate function. git remote update: New option --prune

 Documentation/git-remote.txt |    4 ++-
 builtin-remote.c             |   76 ++++++++++++++++++++++++++----------------
 2 files changed, 50 insertions(+), 30 deletions(-)

- Finn Arne

#7 lore raw

PATCH v2 1/2 builtin-remote.c: Split out prune_remote as a separate function.

prune_remote will be used in update(), so this function was split out to avoid code duplication.

---
 builtin-remote.c |   56 +++++++++++++++++++++++++++++------------------------
 1 files changed, 31 insertions(+), 25 deletions(-)
builtin-remote.c +31 -25
index 9ef846f..9804d6c 100644
@@ -26,6 +26,7 @@ static const char * const builtin_remote_usage[] = {
 static int verbose;
 
 static int show_all(void);
+static int prune_remote(const char *remote, int dry_run);
 
 static inline int postfixcmp(const char *string, const char *postfix)
 {
@@ -1128,46 +1129,51 @@ static int prune(int argc, const char **argv)
 		OPT__DRY_RUN(&dry_run),
 		OPT_END()
 	};
-	struct ref_states states;
-	const char *dangling_msg;
 
 	argc = parse_options(argc, argv, options, builtin_remote_usage, 0);
 
 	if (argc < 1)
 		usage_with_options(builtin_remote_usage, options);
 
-	dangling_msg = (dry_run
-			? " %s will become dangling!\n"
-			: " %s has become dangling!\n");
+	for (; argc; argc--, argv++)
+		result |= prune_remote(*argv, dry_run);
 
-	memset(&states, 0, sizeof(states));
-	for (; argc; argc--, argv++) {
-		int i;
+	return result;
+}
 
-		get_remote_ref_states(*argv, &states, GET_REF_STATES);
+static int prune_remote(const char *remote, int dry_run)
+{
+	int result = 0;
+	struct ref_states states;
+	const char *dangling_msg = dry_run
+		? " %s will become dangling!\n"
+		: " %s has become dangling!\n";
 
-		if (states.stale.nr) {
-			printf("Pruning %s\n", *argv);
-			printf("URL: %s\n",
-			       states.remote->url_nr
-			       ? states.remote->url[0]
-			       : "(no URL)");
-		}
+	memset(&states, 0, sizeof(states));
+	int i;
 
-		for (i = 0; i < states.stale.nr; i++) {
-			const char *refname = states.stale.items[i].util;
+	get_remote_ref_states(remote, &states, GET_REF_STATES);
 
-			if (!dry_run)
-				result |= delete_ref(refname, NULL, 0);
+	if (states.stale.nr) {
+		printf("Pruning %s\n", remote);
+		printf("URL: %s\n",
+		       states.remote->url_nr
+		       ? states.remote->url[0]
+		       : "(no URL)");
+	}
 
-			printf(" * [%s] %s\n", dry_run ? "would prune" : "pruned",
-			       abbrev_ref(refname, "refs/remotes/"));
-			warn_dangling_symref(dangling_msg, refname);
-		}
+	for (i = 0; i < states.stale.nr; i++) {
+		const char *refname = states.stale.items[i].util;
 
-		free_remote_ref_states(&states);
+		if (!dry_run)
+			result |= delete_ref(refname, NULL, 0);
+
+		printf(" * [%s] %s\n", dry_run ? "would prune" : "pruned",
+		       abbrev_ref(refname, "refs/remotes/"));
+		warn_dangling_symref(dangling_msg, refname);
 	}
 
+	free_remote_ref_states(&states);
 	return result;
 }
 
Signature · 1 line
1.6.2.1.471.gdeb91.dirty
#8 lore raw

PATCH v2 2/2 git remote update: New option --prune

In reply to Finn Arne Gangstad

With the --prune (or -p) option, git remote update will also prune all the remotes that it fetches. Previously, you had to do a manual git remote prune <remote> for each of the remotes you wanted to prune, and this could be tedious with many remotes.

A single command will now update a set of remotes, and remove all stale branches: git remote update -p [group]

---
 Documentation/git-remote.txt |    4 +++-
 builtin-remote.c             |   20 ++++++++++++++++----
 2 files changed, 19 insertions(+), 5 deletions(-)
Documentation/git-remote.txt +3 -1
index c9c0e6f..0b6e67d 100644
@@ -16,7 +16,7 @@ SYNOPSIS
 'git remote set-head' <name> [-a | -d | <branch>]
 'git remote show' [-n] <name>
 'git remote prune' [-n | --dry-run] <name>
-'git remote update' [group]
+'git remote update' [-p | --prune] [group]
 
 DESCRIPTION
 -----------
@@ -125,6 +125,8 @@ the configuration parameter remotes.default will get used; if
 remotes.default is not defined, all remotes which do not have the
 configuration parameter remote.<name>.skipDefaultUpdate set to true will
 be updated.  (See linkgit:git-config[1]).
++
+With `--prune` option, prune all the remotes that are updated.
 
 
 DISCUSSION
builtin-remote.c +16 -4
index 9804d6c..c8e5b17 100644
@@ -15,7 +15,7 @@ static const char * const builtin_remote_usage[] = {
 	"git remote set-head <name> [-a | -d | <branch>]",
 	"git remote show [-n] <name>",
 	"git remote prune [-n | --dry-run] <name>",
-	"git remote [-v | --verbose] update [group]",
+	"git remote [-v | --verbose] update [-p | --prune] [group]",
 	NULL
 };
 
@@ -1210,10 +1210,18 @@ static int get_remote_group(const char *key, const char *value, void *cb)
 
 static int update(int argc, const char **argv)
 {
-	int i, result = 0;
+	int i, result = 0, prune = 0;
 	struct string_list list = { NULL, 0, 0, 0 };
 	static const char *default_argv[] = { NULL, "default", NULL };
+	struct option options[] = {
+		OPT_GROUP("update specific options"),
+		OPT_BOOLEAN('p', "prune", &prune,
+			    "prune remotes after fecthing"),
+		OPT_END()
+	};
 
+	argc = parse_options(argc, argv, options, builtin_remote_usage,
+			     PARSE_OPT_KEEP_ARGV0);
 	if (argc < 2) {
 		argc = 2;
 		argv = default_argv;
@@ -1228,8 +1236,12 @@ static int update(int argc, const char **argv)
 	if (!result && !list.nr  && argc == 2 && !strcmp(argv[1], "default"))
 		result = for_each_remote(get_one_remote_for_update, &list);
 
-	for (i = 0; i < list.nr; i++)
-		result |= fetch_remote(list.items[i].string);
+	for (i = 0; i < list.nr; i++) {
+		int err = fetch_remote(list.items[i].string);
+		result |= err;
+		if (!err && prune)
+			result |= prune_remote(list.items[i].string, 0);
+	}
 
 	/* all names were strdup()ed or strndup()ed */
 	list.strdup_strings = 1;
Signature · 1 line
1.6.2.1.471.gdeb91.dirty
#10 lore raw
In reply to Jeff King

2009/4/2 Jeff King <[email protected]>:

Quoted 7 lines · demerphq
Quoted 6 lines

But one question. It seem to me odd to put this as an option to git remote update, and not git remote prune.

I mean, it seems weird that one must say:

Quoted 8 lines
Quoted 5 lines

   git remote update --prune

and one cannot say:

   git remote prune --all

But "git remote update" actually respects "remote groups", so it is not just "--all". I think what you want is "git remote prune <group>".

Are there any implicit groups defined, like "all-remotes" or something? It seems less than desirable to have to define such a group for an operation that IMO is pretty reasonable to expect to happen regularly.

I personally haven't found any use for defining remote groups yet to be honest. Its a granularity of operation that hasnt served much purpose for me yet. Although i could see it being useful in the future.

Generally tho I either want to update and prune one remote only, with

git fetch $remote; git prune $remote,

or i want to update and prune all with something like:

  git remote update; for r in $(git remote); do git remote prune $r; done;

This patch makes the latter better huffman encoded, but I'd kind of expect both to be doable as single commands in terms of how often I want to do them.

Maybe git fetch --prune would be a nice complement to this patch.

Quoted 8 lines
Quoted 3 lines

especially when there is a git remote prune already. It seems a bit counterintuitive to find pruning actions under "update", but not all that strange to find an all "--all" option for the "prune" action.

I think it makes sense under update as pruning is really just a different (and perhaps slightly more dangerous) form of update. Generally I would only want to run prune after having run update, so combining them makes sense from a workflow perspective.

Yeah, conceptually they approach the same point from different angles.

Quoted 5 lines
Quoted 3 lines

Although to me having both be allowed and mean the same thing also makes sense.

I think that would make sense, too.

And the solution that presents the least surprise to the most users.

Yves

Signature · 1 line
perl -Mre=debug -e "/just|another|perl|hacker/"
#11 lore raw
Quoted 7 lines · demerphq
Quoted 2 lines

But "git remote update" actually respects "remote groups", so it is not just "--all". I think what you want is "git remote prune <group>".

Are there any implicit groups defined, like "all-remotes" or something? It seems less than desirable to have to define such a group for an operation that IMO is pretty reasonable to expect to happen regularly.

Yes. From "git help remote":

update
Fetch updates for a named set of remotes in the repository as defined by remotes.<group>. If a named group is not specified on the command line, the configuration parameter remotes.default will get used; if remotes.default is not defined, all remotes which do not have the configuration parameter remote.<name>.skipDefaultUpdate set to true will be updated. (See git-config(1)).

So without defining any other config, "git remote update" will by default update everything

Quoted 4 lines

I personally haven't found any use for defining remote groups yet to be honest. Its a granularity of operation that hasnt served much purpose for me yet. Although i could see it being useful in the future.

I haven't either. I suspect it would be useful if you had a complex set of repo relationships, like an integration manager pulling from an upstream but also from other developers.

Quoted 3 lines

Generally tho I either want to update and prune one remote only, with

git fetch $remote; git prune $remote,

It might be useful if "remote update" treated an unconfigured group as a simple remote. So that "git remote update --prune $remote" would do what you wanted here.

I could even see "remote.*.autoprune" config being useful so you could avoid --prune. It is living dangerously, I suppose, for some workflows; but I generally consider whatever is in my remote tracking branches to be throwaway, and automatically pruning is not really dangerous.

Quoted 9 lines

or i want to update and prune all with something like:

  git remote update; for r in $(git remote); do git remote prune $r; done;

This patch makes the latter better huffman encoded, but I'd kind of expect both to be doable as single commands in terms of how often I want to do them.

Maybe git fetch --prune would be a nice complement to this patch.

I think we have tried to keep pruning out of fetch, as fetch does not necessarily use or know about tracking branches. But the "git remote update $remote" proposal I gave above would do basically the same thing (except you would call it "remote update" instead of "fetch").

-Peff

#12 lore raw

2009/4/2 Jeff King <[email protected]>:

Quoted 21 lines · demerphq
Quoted 8 lines
Quoted 3 lines

But "git remote update" actually respects "remote groups", so it is not just "--all". I think what you want is "git remote prune <group>".

Are there any implicit groups defined, like "all-remotes" or something? It seems less than desirable to have to define such a group for an operation that IMO is pretty reasonable to expect to happen regularly.

Yes. From "git help remote":

      update
          Fetch updates for a named set of remotes in the repository as defined by remotes.<group>. If a named group is not specified on the command line, the configuration parameter remotes.default will get used; if remotes.default is not defined, all remotes which do not have the configuration parameter remote.<name>.skipDefaultUpdate set to true will be updated. (See git-config(1)).

Quoted 2 lines

So without defining any other config, "git remote update" will by default update everything

Er, personally i find that documentation pretty cryptic. And when i check git config for group, i see this:

remotes.<group>
The list of remotes which are fetched by "git remote update <group>". See git-remote(1).

and

remote.<name>.skipDefaultUpdate
If true, this remote will be skipped by default when updating using the update subcommand of git-remote(1).

Neither of which really explain groups, how to define them properly, (the list is separated by what? and includes the remote name?) or whether there are implicit groups. I mean, it seems logical that if you can have user defined groups that there are some built in ones too, like "all" and "none" or perhaps groups defined by transport "http" or "git" for instance.

Quoted 8 lines
Quoted 4 lines

I personally haven't found any use for defining  remote groups yet to be honest. Its a granularity of operation that hasnt served much purpose for me yet. Although i could see it being useful in the future.

I haven't either. I suspect it would be useful if you had a complex set of repo relationships, like an integration manager pulling from an upstream but also from other developers.

Now that you have called my attention to them in more detail i suspect ill end up using them for a few things. Maybe ill try to write up a doc patch once i have.

Quoted 7 lines
Quoted 3 lines

Generally tho I either want to update and prune one remote only, with

   git fetch $remote; git prune $remote,

It might be useful if "remote update" treated an unconfigured group as a simple remote. So that "git remote update --prune $remote" would do what you wanted here.

It seems reasonable to me that names for groups and remotes should stay distinct and that remotes are treated as being groups which contain only the remote of the same name. These would be yet more implicit groups.

Quoted 5 lines

I could even see "remote.*.autoprune" config being useful so you could avoid --prune. It is living dangerously, I suppose, for some workflows; but I generally consider whatever is in my remote tracking branches to be throwaway, and automatically pruning is not really dangerous.

Me too.

Quoted 8 lines
Quoted 8 lines

or i want to update and prune all with something like:

  git remote update; for r in $(git remote); do git remote prune $r; done;

This patch makes the latter better huffman encoded, but I'd kind of expect both to be doable as single commands in terms of how often I want to do them.

Quoted 6 lines
Quoted 1 line

Maybe git fetch --prune would be a nice complement to this patch.

I think we have tried to keep pruning out of fetch, as fetch does not necessarily use or know about tracking branches. But the "git remote update $remote" proposal I gave above would do basically the same thing (except you would call it "remote update" instead of "fetch").

Ok, that makes sense. I see why fetch would be left out. Thanks for explaining.

cheers,
Yves

Signature · 1 line
perl -Mre=debug -e "/just|another|perl|hacker/"
#13 lore raw
Quoted 6 lines · demerphq

Er, personally i find that documentation pretty cryptic. And when i check git config for group, i see this:

remotes.<group>
The list of remotes which are fetched by "git remote update <group>". See git-remote(1).

Yeah, this should probably say "separated by space" or whatever (I actually don't even know). I would assume it contains the remote name; I can't imagine what other thing it would include. But it wouldn't hurt to make that more explicit.

I'm sure a documentation patch would be welcome.

Quoted 10 lines

remote.<name>.skipDefaultUpdate
If true, this remote will be skipped by default when updating using the update subcommand of git-remote(1).

Neither of which really explain groups, how to define them properly, (the list is separated by what? and includes the remote name?) or whether there are implicit groups. I mean, it seems logical that if you can have user defined groups that there are some built in ones too, like "all" and "none" or perhaps groups defined by transport "http" or "git" for instance.

I think what is confusing is that there is exactly one implicit group, and it is "the default group", which contains every remote that doesn't have skipDefaultUpdate set. You refer to the "default group" by not mentioning any group.

So no, there aren't other implicit groups (AFAIK).

You can propose implicit groups, but I think they would have to have a compelling use case over simply creating them manually. To avoid conflict with groups people have already defined, they would only be used if no remotes.$whatever config existed.

I think having "git remote update foo" fall back to a group containing only the remote "foo" when "remotes.foo" does not exist makes sense. I'm not sure that "none", "http", or "git" is all that useful in practice (the only thing I can think of for the latter two is that you might use "git" versus "http" depending on restrictive firewall settings).

You could give the unnamed "default group" a name (like "all"), but then you risk conflict with existing "remotes.all". And in this case, it is hard to remain backwards compatible: "git remote update" will do something different now in the case that the user has configured remotes.all.

-Peff

#14 lore raw

2009/4/2 Jeff King <[email protected]>:

Quoted 9 lines · demerphq
Quoted 7 lines

Er, personally i find that documentation pretty cryptic. And when i check git config for group, i see this:

       remotes.<group>
           The list of remotes which are fetched by "git remote update <group>". See git-remote(1).

Quoted 6 lines

Yeah, this should probably say "separated by space" or whatever (I actually don't even know). I would assume it contains the remote name; I can't imagine what other thing it would include. But it wouldn't hurt to make that more explicit.

I'm sure a documentation patch would be welcome.

If/when i feel that i understand the subject sufficiently Ill make an attempt. :-)

Quoted 11 lines
Quoted 10 lines

       remote.<name>.skipDefaultUpdate
           If true, this remote will be skipped by default when updating using the update subcommand of git-remote(1).

Neither of which really explain groups, how to define them properly, (the list is separated by what? and includes the remote name?) or whether there are implicit groups. I mean, it seems logical that if you can have user defined groups that there are some built in ones too, like "all" and "none" or perhaps groups defined by transport "http" or "git" for instance.

Quoted 4 lines

I think what is confusing is that there is exactly one implicit group, and it is "the default group", which contains every remote that doesn't have skipDefaultUpdate set. You refer to the "default group" by not mentioning any group.

Yes well an implicit group "default" would be nice.

Quoted 7 lines

So no, there aren't other implicit groups (AFAIK).

You can propose implicit groups, but I think they would have to have a compelling use case over simply creating them manually. To avoid conflict with groups people have already defined, they would only be used if no remotes.$whatever config existed.

Or perhpas simply use syntax not likely or expected to be used in the name of a group. Like colon. Or something like that.

Quoted 7 lines

I think having "git remote update foo" fall back to a group containing only the remote "foo" when "remotes.foo" does not exist makes sense. I'm not sure that "none", "http", or "git" is all that useful in practice (the only thing I can think of for the latter two is that you might use "git" versus "http" depending on restrictive firewall settings).

Well i was think of situations where somebody has coded something that just must have a group. Thus 'none' would be essentially a no-op in this case. I know you can argue "well dont do that", but people tend to do silly things whatever they are told, and explicit arguments make for less special cases in wrapper scripts and the like...

Quoted 3 lines

You could give the unnamed "default group" a name (like "all"), but then you risk conflict with existing "remotes.all".

Id much prefer the default group to be called default. Not "all". Ideally "all" would really be "all". :-)

Quoted 4 lines

And in this case, it is
hard to remain backwards compatible: "git remote update" will do something different now in the case that the user has configured remotes.all.

Well, id have a thought a better approach is to use a character that cannot or is extremely unlikely to be used in an existing group definition and cannot be used in a remote definition. Like maybe ":all" or something.

Or maybe: git config remote.implicitgroups true could be used to enable it?

cheers,
Yves

Signature · 1 line
perl -Mre=debug -e "/just|another|perl|hacker/"