git-remote exclude

"git-remote exclude" can be used to prevent one or more unwanted remote branches from being tracked. After, for example,

    $ git-remote origin exclude man html
"git-fetch origin" will no longer fetch origin/man and origin/html.

"git-remote exclude" does not check whether the "unwanted" branch really exists on the remote repo.

---
 git-remote.perl |   51 +++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 51 insertions(+), 0 deletions(-)

index fc055b6..8ee1bea 100755
@@ -243,6 +243,47 @@ sub add_remote {
 		      "+refs/heads/*:refs/remotes/$name/*");
 }
 
+sub exclude_remote_branch {
+	my ($name, @branches) = @_;
+	if (!exists $remote->{$name}) {
+		print STDERR "remote $name does not exists.\n";
+		exit(1);
+	}
+
+	my $wildcard = "";
+	my %fetch_entries = ();
+	my @fetches = $git->command(qw(repo-config --get-all), "remote.$name.fetch");
+	foreach $fetch (@fetches) {
+		if ($fetch =~ m%^\+?refs/heads/\*:refs/remotes/$name/\*$%) {
+			$wildcard = $fetch;
+		}
+		else {
+			foreach $br (@branches) {
+				if ($fetch =~ m#^refs/heads/$br:.*$#) {
+					$fetch_entries{$br} = $fetch;
+				}
+			}
+		}
+	}
+
+	if (!$wildcard) {
+		print STDERR "No wildcard entry for $name. Config not changed\n";
+		exit(1);
+	}
+	my $clean;
+       	($clean = $wildcard) =~ s/([+*])/\\$1/g;
+	$git->command(qw(repo-config --unset), "remote.$name.fetch", $clean);
+	foreach $key (keys %fetch_entries) {
+		$git->command(qw(repo-config --unset), "remote.$name.fetch",
+				$fetch_entries{$key});
+	}
+	foreach $br (@branches) {
+		$git->command(qw(repo-config --add), "remote.$name.fetch",
+			"refs/heads/$br:");
+	}
+	$git->command(qw(repo-config --add), "remote.$name.fetch", $wildcard);
+}
+
 if (!@ARGV) {
 	for (sort keys %$remote) {
 		print "$_\n";

@@ -274,9 +315,19 @@ elsif ($ARGV[0] eq 'add') {
 	}
 	add_remote($ARGV[1], $ARGV[2]);
 }
+elsif ($ARGV[0] eq 'exclude') {
+	if (@ARGV < 3) {
+		print STDERR "Usage: git remote exclude <name> <branch>...\n";
+		exit(1);
+	}
+	my @branches = @ARGV;
+	shift(@branches); shift(@branches);
+	exclude_remote_branch($ARGV[1], @branches);
+}
 else {
 	print STDERR "Usage: git remote\n";
 	print STDERR "       git remote add <name> <url>\n";
+	print STDERR "       git remote exclude <name> <branch>...\n";
 	print STDERR "       git remote show <name>\n";
 	exit(1);
 }

Hi,

That is not what your patch does.

It rewrites the "remote.$name.fetch" entries so that those branches are not _updated_, but they are _fetched_ nevertheless.

But then, I don't really see _why_ you would want such a solution. After all, you are more likely to be interested in _specific_ branches, rather than all branches _except_ a few.

IMHO "git remote expand <name>" to expand the wildcards and "git remote copy <dest> <src>" would be more useful.

Ciao,
Dscho

That's not true. I have a patch (that doesn't work, so it's not submitted), that would allow me to have:

[remote "origin"]
url = git://git2.kernel.org/pub/scm/git/git.git
fetch = refs/heads/*:refs/remotes/up/* fetch = !refs/heads/html
fetch = !refs/heads/todo

i.e. I don't want those two branches, but I do want everything else. I'd also like to be able to do "!/refs/heads/temp/*" so I could block a whole subdirectory of branches. I have a feeling that this would come in handy for people like the person who was recently talking about having 880 branches in his repository, with only a few active.

Andy

Dr Andy Parkins, M Eng (hons), MIEE
[email protected]

I do not think you are interested in "not these two but everything else". You are interested in maint, master, next, pu and man. You can get away by saying "not these two but everything else" only because you are implicitly trusting me not to publish insane number of random throw-away branches left and right.

In an ideal world, I think you would want to be able to do something like this:

- Let's look at what they have right now.
- I want, among the above, this, that and that.
- I would never want any of the others I just checked.

- If there are new ones found in the future, please let me know and I'll decide then. or...
- If there are new ones found in the future, I would not want to be bothered with them. or...
- If there are new ones found in the future, I want them too.

The choice between the last three will heavily depend on the nature of the project and also your work habit to a certain degree.

A remote could frequently throw in random small-topic branches that are short lived, and are totally uninteresting for somebody who wants to track the big picture branches. You would say the first if you primarily work in a particular area of the project, because you will always be interested in the primary integration and you may or may not be interested in a new topic. You would say the second if you are only interested in the big picture branches after marking the primary integration branches as interesting. You would say the third only if you are a packrat, or have unlimited brain bandwidth to keep track of what's happening in every little corner of the project.

If a single repository has 1000 branches, that is a sign that there is something wrong in the project's organization. No single person should need to look at 1000 branches -- that goes directly against the distributed nature of git. Humans do not scale well.

Now, 'git-fetch' currently does not scale either, and I think it is a problem, but I think that is fixable by redoing the part below the while loop that calls append_fetch_head to make the processing more efficient.

But you cannot optimize humans the same way.

The move to BK formalized a change to the development process of the kernel project --- a central single integrator did not scale well, and there was a need for a group of people around the center to help the peer review and the integration process. The patch-flow in the development process places more integration work and responsibility on the lieutenants, and the tool helps people who are more central than the lieutenants trust, verify and integrate their lieutenants' work.

Helping this kind of project organization is what git inherited from BK as well. The key to scaling is to limit the product of the level of details times the size of the area one person needs to look at. A project may have 1000 simultaneous development going at any one time, but a lieutenant should not need to look at all 1000 branches but only a much smaller portion of them in his own area of expertise. A more central person should not need to look at all 1000 branches either, but what he will look at would be branches that the lieutenants would publish in their repositories, as the result of integration of these 1000 branches at the lowest level.

Nope. I think I'm right. You're right that I'm trusting you not to publish an insane number of branches, but that holds for the glob-mode only, regardless of whether the NOT fetches are in place. The sort of thing I imagine happening is that as maintainer you suddenly say - I think 1.5.0 is so good that we'll maintain it for longer than usual. In which case we would suddenly get a branch for maint-1.5.0. I would want that branch, but I still don't want man, html and todo.

- If there is one that was there but now isn't, ask me if I want to remove it

True; so all those should be possible to specify in the config.

That's true. However, if there were discipline in the naming, this wouldn't cause a major problem:

fetch = refs/heads/export/*:refs/remotes/origin/*

Now, only the branches under export/ will be sent across automatically. You can make whatever topic branches you want without polluting the automatic namespace.

Again, in a project of that size, the branches could be conventionally stored in particular namespaces.

fetch = refs/heads/topicA/*:refs/remotes/origin/topicA/* fetch = refs/heads/topicB/*:refs/remotes/origin/topicB/*

Without saying how those 1000 branches are organised, I think that it is an incorrect generalisation to say that something is wrong. I have tens of thousands of files in my home directory. I cannot possibly look at all those files; and yet it's not disorganised.

To me, one of git's strongest features is the ability to have hierarchically organised branch names. It solves the scalability problem in one fell swoop.

No; but you can optimise them in a different way. Trees are the way, as long as the number of choices at any level is within reasonable limits, then you can sort through thousands and thousands of items easily.

I actually think the way to go with this globbing stuff is to make it possible to specify more accurately what you want. The weak area at the moment is the fact that it's not possible to do a non-recursive glob. That is to say that

fetch = refs/heads/*:refs/remotes/*

would grab x/foo, y/bar, and z/baz. I know you keep your topic branches in places like jc/some-topic and js/some-other-topic. If you ever pushed them, then the default glob would pull them all. If we could have a non-recursive glob, then you could safely push them (making things all the more distributed) without impacting the master, maint, next, etc branch pulls.

All agreed. However, limiting the detail is not necessarily the same as hiding the detail. If those 1000 branches can be arranged in such a manner as to make 990 of them easily ignored, then that's just as good as hiding those 990.

Andy

Dr Andy Parkins, M Eng (hons), MIEE
[email protected]

Hi,

Fine. Submit a patch. And if that patch makes git-fetch not as fragile as I expect it to do, I am not opposed to inclusion. I'll just not use that feature. And I'll still find your usage confused.

Ciao,
Dscho

I have no doubt your opposition to inclusion is important here. But some people would (out-of-fashion'ly/arrogantly/dont-care'ly) submit their patches "just for fun", as they did in the olden days. (Not that old to me though).

Quy

Hi,

Hey, if others need that feature, I'm fine! And maybe I find myself in the need of exactly the same feature that I was opposed to, later in my life. Happened before, will happen again.

Ciao,
Dscho

Very good idea. I even thought that this feature is present in git already...

Jakub Narebski
Warsaw, Poland
ShadeHawk on #git

I meant to say "track", but somehow "fetch" got to my fingers. Old ages, perhaps. You are right, "update" is _the_ word. Thanks.

For different situations, there are different paths to choose to reach that ultimate "After All" (Zen? NO!). I offered one (little) path and expect to see more of bigger ones. That's the _why_ (and/or, the _why_ not).

Quy