Skip to contentSkip to conversation

git gc expanding packed data?

PATCH In review Andreas Schwab 7 participants 26 messages Jun 15, 2016

lore
Share & tools
lore
#0 lore raw
Quoted 7 lines · Nicolas Pitre

It appears that the git installation serving clone requests for
git://gcc.gnu.org/git/gcc.git generates lots of unreferenced objects. I just cloned it and the pack I was sent contains 1383356 objects (can be determined with 'git show-index < .git/objects/pack/*.idx | wc -l'). However, there are only 978501 actually referenced objects in that cloned repository ( 'git rev-list --all --objects | wc -l'). That makes for 404855 useless objects in the cloned repository.

Those objects are not useless. They are referenced by the remote refs on the remote side, which are not fetched by default. If you clone a mirror of the repository you'll see no unreferenced objects.

Andreas.

Signature · 3 lines
Andreas Schwab, [email protected]
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."
#1 lore raw
Quoted 14 lines · Andreas Schwab
Quoted 7 lines · Nicolas Pitre

It appears that the git installation serving clone requests for
git://gcc.gnu.org/git/gcc.git generates lots of unreferenced objects. I just cloned it and the pack I was sent contains 1383356 objects (can be determined with 'git show-index < .git/objects/pack/*.idx | wc -l'). However, there are only 978501 actually referenced objects in that cloned repository ( 'git rev-list --all --objects | wc -l').  That makes for 404855 useless objects in the cloned repository.

Those objects are not useless.  They are referenced by the remote refs on the remote side, which are not fetched by default.  If you clone a mirror of the repository you'll see no unreferenced objects.

Quoted 7 lines

Andreas.

--
Andreas Schwab, [email protected]
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5 "And now for something completely different."

Thanks... It is a difference between svn and git mentality probably - one only pushes reasonably reliable code to a public git repository, whereas anything transient is recorded in svn - I think many of the unreferenced objects are svn user-branches (which are probably of use to people who intend to work on gcc for fairly extended periods, rather than casual users like me).
The case with gcc is probably quite extreme - many user branches, and very large code base - but is there anything on the git side with git gc which can lessen this kind of pathological behavior (expanding packs)?

Thanks a lot for the explanation and the discussion.

Hin-Tak

#2 lore raw
Quoted 1 line · Hin-Tak Leung

Thanks... It is a difference between svn and git mentality probably -

It is just that the remote is a git-svn tree, with only a few branches created as local branches.

Quoted 4 lines

The case with gcc is probably quite extreme - many user branches, and very large code base - but is there anything on the git side with git gc which can lessen this kind of pathological behavior (expanding packs)?

If you fetch all refs, not only refs/heads/*, all objects will be referenced.

Andreas.

Signature · 3 lines
Andreas Schwab, [email protected]
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."
#3 lore raw
In reply to Andreas Schwab
Quoted 10 lines · Andreas Schwab
Quoted 7 lines · Nicolas Pitre

It appears that the git installation serving clone requests for
git://gcc.gnu.org/git/gcc.git generates lots of unreferenced objects. I just cloned it and the pack I was sent contains 1383356 objects (can be determined with 'git show-index < .git/objects/pack/*.idx | wc -l'). However, there are only 978501 actually referenced objects in that cloned repository ( 'git rev-list --all --objects | wc -l'). That makes for 404855 useless objects in the cloned repository.

Quoted 3 lines

Those objects are not useless. They are referenced by the remote refs on the remote side, which are not fetched by default. If you clone a mirror of the repository you'll see no unreferenced objects.

If you do a clone using the git:// protocol and the server sends you only the ref for the trunk branch, then it should send you only objects reachable from that branch. Any extra objects sent by the server are useless to me and wastes my and everyone else's bandwidth, and on my next repack those objects are pruned anyway. The point of the git protocol is _not_ necessarily to send a copy of the remote pack file over, even during a clone.

Nicolas

#4 lore raw
Quoted 2 lines · Nicolas Pitre

If you do a clone using the git:// protocol and the server sends you only the ref for the trunk branch,

A clone will fetch all branches from refs/heads/*.

Quoted 1 line

then it should send you only objects reachable from that branch.

Apparantly this does not work. I'd guess the extra objects are needed due to the delta compression.

Andreas.

Signature · 3 lines
Andreas Schwab, [email protected]
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."
#5 lore raw

Re: git clone sending unneeded objects (was : git gc expanding packed data?)

Quoted 11 lines · Andreas Schwab
Quoted 2 lines · Nicolas Pitre

If you do a clone using the git:// protocol and the server sends you only the ref for the trunk branch,

A clone will fetch all branches from refs/heads/*.

Quoted 1 line

then it should send you only objects reachable from that branch.

Apparantly this does not work. I'd guess the extra objects are needed due to the delta compression.

I just tried doing a clone of the GCC repository, then git gc --prune=now, and another clone specifying --reference to the first, and it wanted to download all the unreachable objects again. So it doesn't seem to be a compression issue.

This is with git 1.6.4 on both ends.

Jason

#6 lore raw

Re: git clone sending unneeded objects

Quoted 13 lines · Jason Merrill
Quoted 11 lines · Andreas Schwab
Quoted 2 lines · Nicolas Pitre

If you do a clone using the git:// protocol and the server sends you only the ref for the trunk branch,

A clone will fetch all branches from refs/heads/*.

Quoted 1 line

then it should send you only objects reachable from that branch.

Apparantly this does not work. I'd guess the extra objects are needed due to the delta compression.

Quoted 6 lines

I just tried doing a clone of the GCC repository, then git gc --prune=now, and another clone specifying --reference to the first, and it wanted to download all the unreachable objects again. So it doesn't seem to be a compression issue.

This is with git 1.6.4 on both ends.

Which protocol did you use?

If you use git:// or ssh://, it's normally a security feature that Git sends you only reachable objects. If it doesn't, it's a serious bug.

Signature · 2 lines
Matthieu Moy
http://www-verimag.imag.fr/~moy/
#7 lore raw

Re: git clone sending unneeded objects

Quoted 1 line · Matthieu Moy

Which protocol did you use?

git://

Jason

#8 lore raw

Re: git clone sending unneeded objects

In reply to Matthieu Moy
Quoted 20 lines · Matthieu Moy
Quoted 18 lines · Jason Merrill
Quoted 11 lines · Andreas Schwab
Quoted 2 lines · Nicolas Pitre

If you do a clone using the git:// protocol and the server sends you only the ref for the trunk branch,

A clone will fetch all branches from refs/heads/*.

Quoted 1 line

then it should send you only objects reachable from that branch.

Apparantly this does not work. I'd guess the extra objects are needed due to the delta compression.

I just tried doing a clone of the GCC repository, then git gc --prune=now, and another clone specifying --reference to the first, and it wanted to download all the unreachable objects again. So it doesn't seem to be a compression issue.

Quoted 6 lines
Quoted 1 line

This is with git 1.6.4 on both ends.

Which protocol did you use?

If you use git:// or ssh://, it's normally a security feature that Git sends you only reachable objects. If it doesn't, it's a serious bug.

I did reproduce the issue with git:// back when this discussion started. I also asked for more information about the remote which didn't come forth.

Nicolas

#9 lore raw

Re: git clone sending unneeded objects

Quoted 3 lines · Nicolas Pitre

I did reproduce the issue with git:// back when this discussion started. I also asked for more information about the remote which didn't come forth.

Looking back, I only see you asking about the git version on the server, which is 1.6.4.

So again:

git clone git://gcc.gnu.org/git/gcc.git
(1399509 objects, ~600MB .git dir)
git gc --prune=now (988906 objects, ~450MB .git dir)

...then

git clone git://gcc.gnu.org/git/gcc.git --reference $firstclone
(573401 objects, ~550MB .git dir)
git fsck (clean)
git gc --prune=now (5 objects, ~7MB .git dir)

What's going on here?

Jason

#10 lore raw

Re: git clone sending unneeded objects

Quoted 8 lines · Jason Merrill
Quoted 4 lines · Nicolas Pitre

I did reproduce the issue with git:// back when this discussion started. I also asked for more information about the remote which didn't come forth.

Looking back, I only see you asking about the git version on the server, which is 1.6.4.

Hmm, I was under the impression from the previous thread that the server is a bit older and/or have more backward compatible settings to cater for older git clients?

Quoted 7 lines

So again:

git clone git://gcc.gnu.org/git/gcc.git
(1399509 objects, ~600MB .git dir)
git gc --prune=now (988906 objects, ~450MB .git dir)

Quoted 8 lines

...then

git clone git://gcc.gnu.org/git/gcc.git --reference $firstclone
(573401 objects, ~550MB .git dir)
git fsck (clean)
git gc --prune=now (5 objects, ~7MB .git dir)

What's going on here?

FWIW, I still have my clone (git://) and do my periodic 'git fetch' and 'git gc prune=now' (learned my lessons!) and it is currently .git dir is about 350MB. (from previous discussion the optimal at the time was about 300MB, so it has grown a bit in the last couple of months).

And thanks everybody for all the discussion and advice. git is a great tool. (and I have essentially stopped using svn, prefering git-svn!).

#11 lore raw

Re: git clone sending unneeded objects

In reply to Jason Merrill
Quoted 14 lines · Jason Merrill
Quoted 3 lines · Nicolas Pitre

I did reproduce the issue with git:// back when this discussion started. I also asked for more information about the remote which didn't come forth.

Looking back, I only see you asking about the git version on the server, which is 1.6.4.

So again:

git clone git://gcc.gnu.org/git/gcc.git
(1399509 objects, ~600MB .git dir)
git gc --prune=now (988906 objects, ~450MB .git dir)

Quoted 8 lines

...then

git clone git://gcc.gnu.org/git/gcc.git --reference $firstclone
(573401 objects, ~550MB .git dir)
git fsck (clean)
git gc --prune=now (5 objects, ~7MB .git dir)

What's going on here?

Some screw up.

Do you have access to the remote machine? Is it possible to have a tarball of the gcc.git directory from there?

Nicolas

#12 lore raw

Re: git clone sending unneeded objects

Quoted 2 lines · Nicolas Pitre

Do you have access to the remote machine? Is it possible to have a tarball of the gcc.git directory from there?

http://gcc.gnu.org/gcc-git.tar.gz

I'll leave it there for a few days.

Jason

#13 lore raw

Re: git clone sending unneeded objects

Incidentally, somewhat related to this issue, I've noticed that if I fetch a branch which I don't currently have in my repository, and I have most of the commits on that branch in my object store (or in an alternate repository) but not the most recent commit, git fetch isn't smart enough to only grab the commits I'm actually missing, it wants to fetch much more.

I would expect that since the clone pulled down everything in the gcc.git repository, I could then do

git config remote.origin.fetch 'refs/remotes/*:refs/remotes/origin/*' git fetch

and have all the branches, not just the ones in refs/heads. But when I do this git fetch wants to fetch some 500k redundant objects.

Jason

#14 lore raw

Re: git clone sending unneeded objects

Quoted 2 lines · Jason Merrill

git config remote.origin.fetch 'refs/remotes/*:refs/remotes/origin/*' git fetch

git count-objects -v before:

count: 44
size: 1768
in-pack: 1399509
packs: 1
size-pack: 600456
prune-packable: 0
garbage: 0

and after (transferred 278MB):

count: 44
size: 1768
in-pack: 1947339
packs: 2
size-pack: 1178408
prune-packable: 8
garbage: 0

and then after git gc --prune=now:

count: 0
size: 0
in-pack: 1399613
packs: 1
size-pack: 839900
prune-packable: 0
garbage: 0

So I only actually needed 104 more objects, but fetch wasn't clever enough to see that, and my new pack is much less efficient.

I've run into the same issue using alternates to set up multiple working directories for different branches; if the alternate directory isn't completely up-to-date, fetch wants to pull down lots of data again rather than use what I have and only fetch the last one or two commits.

Jason

#15 lore raw

Re: git clone sending unneeded objects

Quoted 6 lines · Jason Merrill
Quoted 2 lines · Jason Merrill

git config remote.origin.fetch 'refs/remotes/*:refs/remotes/origin/*' git fetch

git count-objects -v before:

Quoted 7 lines

count: 44
size: 1768
in-pack: 1399509
packs: 1
size-pack: 600456
prune-packable: 0
garbage: 0

I'm sure if you had done 'git rev-list --all --objects | wc -l' at that point, the result would have been something around 900000. That's the actual number of objects git had a reference to, compared to the total objects contained in the object store.

Quoted 9 lines

and after (transferred 278MB):

count: 44
size: 1768
in-pack: 1947339
packs: 2
size-pack: 1178408
prune-packable: 8
garbage: 0

And those 500000 extra objects or so (minus a couple dozens which were probably used to "complete" the fetched thin pack and are duplicates of local objects -- the fetch progress message gave the exact number) were obtained from the remote repository because git has no way to tell the remote it already had them. That's what I was explaining in my previous email.

Quoted 10 lines

and then after git gc --prune=now:

count: 0
size: 0
in-pack: 1399613
packs: 1
size-pack: 839900
prune-packable: 0
garbage: 0

Quoted 2 lines

So I only actually needed 104 more objects, but fetch wasn't clever enough to see that, and my new pack is much less efficient.

Like I said, it's not that the fetch wasn't clever enough. Rather that your initial clone asked for way too many objects in the first place. That's what my patch fixed.

Now the pack efficiency can be explained as well. A single pack is always going to be more efficient than 2 packs. Problem is when you do a gc, by default git does the least costly operation which consists of copying as much data from existing packs without extra processing. That means that many objects were copied from the second (newly received) pack although a better delta representation was most probably available in the other larger pack (remember that most objects from that second pack already existed in the first pack). Git do select the second pack in preference to the other pack because it is more recent, and normally more recent packs contains more recent objects which is a good heuristic to optimizes the object enumeration. In this case this didn't produce a good result, but again we're talking about a scenario which is bogus from the start and shouldn't be.

So if you do a 'git gc --aggressive' and let it run for a while, you should get back a smaller pack, possibly even much smaller than the original
one.

Nicolas

#16 lore raw

Re: git clone sending unneeded objects

In reply to Jason Merrill
Quoted 6 lines · Jason Merrill

Incidentally, somewhat related to this issue, I've noticed that if I fetch a branch which I don't currently have in my repository, and I have most of the commits on that branch in my object store (or in an alternate repository) but not the most recent commit, git fetch isn't smart enough to only grab the commits I'm actually missing, it wants to fetch much more.

Quoted 8 lines

I would expect that since the clone pulled down everything in the gcc.git repository, I could then do

git config remote.origin.fetch 'refs/remotes/*:refs/remotes/origin/*' git fetch

and have all the branches, not just the ones in refs/heads. But when I do this git fetch wants to fetch some 500k redundant objects.

Well... Assuming a fixed git using the patch I posted yesterday, my clone of gcc.git has 988941 objects. The source repository used for the clone has 1399551 objects. Of course the source repo has more objects because it has extra branches in the refs/remotes/ namespace that the clone didn't fetch. If you wish to also fetch those branches as you illustrated above then you'll get the difference i.e. 410610 additional objects.

And even if the broken clone (before my patch) did pull everything from gcc.git, in the cloned repository those 410610 extra objects are considered as garbage because nothing actually reference them. So even if you decide to fetch the extra branches that the initial clone didn't pick up, or if you do reference that repository with "garbage" objects for another clone to which you want to add those extra branches, git has no way to know that it already had access to those objects locally and "ungarbage" them as they aren't referenced. Result is a useless fetch of 410610 objects that you already have, but that you weren't supposed to have in the first place.

Nicolas

#17 lore raw

Re: git clone sending unneeded objects

Quoted 10 lines · Nicolas Pitre

And even if the broken clone (before my patch) did pull everything from gcc.git, in the cloned repository those 410610 extra objects are considered as garbage because nothing actually reference them. So even if you decide to fetch the extra branches that the initial clone didn't pick up, or if you do reference that repository with "garbage" objects for another clone to which you want to add those extra branches, git has no way to know that it already had access to those objects locally and "ungarbage" them as they aren't referenced. Result is a useless fetch of 410610 objects that you already have, but that you weren't supposed to have in the first place.

Just to clarify a minor nit:

Actually, if those refs have not changed, quickfetch should kick in and realize that all 410610 objects are reachable locally without errors, permitting the client to avoid the object transfer.

However, if *ANY* of those refs were to change to something you don't actually have, quickfetch would fail, and we would need to fetch all 410610 objects.

Signature · 1 line
Shawn.
#18 lore raw

Re: git clone sending unneeded objects

Quoted 7 lines · Shawn O. Pearce

Actually, if those refs have not changed, quickfetch should kick in and realize that all 410610 objects are reachable locally without errors, permitting the client to avoid the object transfer.

However, if *ANY* of those refs were to change to something you don't actually have, quickfetch would fail, and we would need to fetch all 410610 objects.

Right. That seems unfortunate to me; couldn't fetch do a bit more checking before it decides to download the whole world again?

Jason

#19 lore raw

Re: git clone sending unneeded objects

Quoted 11 lines · Jason Merrill
Quoted 7 lines · Shawn O. Pearce

Actually, if those refs have not changed, quickfetch should kick in and realize that all 410610 objects are reachable locally without errors, permitting the client to avoid the object transfer.

However, if *ANY* of those refs were to change to something you don't actually have, quickfetch would fail, and we would need to fetch all 410610 objects.

Right. That seems unfortunate to me; couldn't fetch do a bit more checking before it decides to download the whole world again?

The quickfetch test could be turned into a filter so refs that are already available locally could simply not be fetched on a per ref basis. But that would be a rather expensive test which couldn't keep its "quick" qualifier anymore, and so for a case that shouldn't have happened normally anyway if git didn't have a bug with its clone operation as I've explained already.

Nicolas

#20 lore raw

Re: git clone sending unneeded objects

In reply to Shawn O. Pearce
Quoted 14 lines · Shawn O. Pearce
Quoted 10 lines · Nicolas Pitre

And even if the broken clone (before my patch) did pull everything from gcc.git, in the cloned repository those 410610 extra objects are considered as garbage because nothing actually reference them. So even if you decide to fetch the extra branches that the initial clone didn't pick up, or if you do reference that repository with "garbage" objects for another clone to which you want to add those extra branches, git has no way to know that it already had access to those objects locally and "ungarbage" them as they aren't referenced. Result is a useless fetch of 410610 objects that you already have, but that you weren't supposed to have in the first place.

Just to clarify a minor nit:

Quoted 7 lines

Actually, if those refs have not changed, quickfetch should kick in and realize that all 410610 objects are reachable locally without errors, permitting the client to avoid the object transfer.

However, if *ANY* of those refs were to change to something you don't actually have, quickfetch would fail, and we would need to fetch all 410610 objects.

Right. But since we're talking about a git mirror for the gcc svn repo and gcc is a rather active project, the likelyhood of any ref to change at any time is rather high.

Nicolas

#21 lore raw

Re: git clone sending unneeded objects

In reply to Jason Merrill
Quoted 7 lines · Jason Merrill
Quoted 2 lines · Nicolas Pitre

Do you have access to the remote machine? Is it possible to have a tarball of the gcc.git directory from there?

http://gcc.gnu.org/gcc-git.tar.gz

I'll leave it there for a few days.

Thanks, I got it now. And I was able to reproduce the issue locally.

Cloning the original repository does transfer objects which become unreferenced in the clone. But cloning that cloned repository (before pruning the unreferenced objects) does not transfer those objects again.

Just need to find out why.

Nicolas

#22 lore raw

PATCH 1/1 make 'git clone' ask the remote only for objects it cares about

Current behavior of 'git clone' when not using --mirror is to fetch everything from the peer, and then filter out unwanted refs just before writing them out to the cloned repository. This may become highly inefficient if the peer has an unusual ref namespace, or if it simply has "remotes" refs of its own, and those locally unwanted refs are connecting to a large set of objects which becomes unreferenced as soon as they are fetched.

Let's filter out those unwanted refs from the peer _before_ asking it what refs we want to fetch instead, which is the most logical thing to do anyway.

---

Quoted 10 lines · Nicolas Pitre
Quoted 7 lines · Jason Merrill
Quoted 2 lines · Nicolas Pitre

Do you have access to the remote machine? Is it possible to have a tarball of the gcc.git directory from there?

http://gcc.gnu.org/gcc-git.tar.gz

I'll leave it there for a few days.

Quoted 7 lines

Thanks, I got it now. And I was able to reproduce the issue locally.

Cloning the original repository does transfer objects which become unreferenced in the clone. But cloning that cloned repository (before pruning the unreferenced objects) does not transfer those objects again.

Just need to find out why.

And the "why" is described above. The problem was actually on the client side and was affecting clones of any repository containing anything outside refs/heads and refs/tags.

The fact that the git repository on gcc.gnu.org has lots of stuff in "remote" branches that don't get cloned by default is a separate configuration/policy issue on that server which might need (or not) to be looked into. For instance at least, as a bare repository, it should have all the git files in gcc.git/ directly instead of gcc.git/.git/.

builtin-clone.c +17 -10
index bab2d84..edf7c7f 100644
@@ -329,24 +329,28 @@ static void remove_junk_on_signal(int signo)
 	raise(signo);
 }
 
-static struct ref *write_remote_refs(const struct ref *refs,
-		struct refspec *refspec, const char *reflog)
+static struct ref *wanted_peer_refs(const struct ref *refs,
+		struct refspec *refspec)
 {
 	struct ref *local_refs = NULL;
 	struct ref **tail = &local_refs;
-	struct ref *r;
 
 	get_fetch_map(refs, refspec, &tail, 0);
 	if (!option_mirror)
 		get_fetch_map(refs, tag_refspec, &tail, 0);
 
+	return local_refs;
+}
+
+static void write_remote_refs(const struct ref *local_refs, const char *reflog)
+{
+	const struct ref *r;
+
 	for (r = local_refs; r; r = r->next)
 		add_extra_ref(r->peer_ref->name, r->old_sha1, 0);
 
 	pack_refs(PACK_REFS_ALL);
 	clear_extra_refs();
-
-	return local_refs;
 }
 
 int cmd_clone(int argc, const char **argv, const char *prefix)
@@ -495,9 +499,10 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 
 	strbuf_reset(&value);
 
-	if (path && !is_bundle)
+	if (path && !is_bundle) {
 		refs = clone_local(path, git_dir);
-	else {
+		mapped_refs = wanted_peer_refs(refs, refspec);
+	} else {
 		struct remote *remote = remote_get(argv[0]);
 		transport = transport_get(remote, remote->url[0]);
 
@@ -520,14 +525,16 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 					     option_upload_pack);
 
 		refs = transport_get_remote_refs(transport);
-		if (refs)
-			transport_fetch_refs(transport, refs);
+		if (refs) {
+			mapped_refs = wanted_peer_refs(refs, refspec);
+			transport_fetch_refs(transport, mapped_refs);
+		}
 	}
 
 	if (refs) {
 		clear_extra_refs();
 
-		mapped_refs = write_remote_refs(refs, refspec, reflog_msg.buf);
+		write_remote_refs(mapped_refs, reflog_msg.buf);
 
 		remote_head = find_ref_by_name(refs, "HEAD");
 		remote_head_points_at =
#23 lore raw

Re: [PATCH] make 'git clone' ask the remote only for objects it cares about

Quoted 7 lines · Nicolas Pitre

Current behavior of 'git clone' when not using --mirror is to fetch everything from the peer, and then filter out unwanted refs just before writing them out to the cloned repository. This may become highly inefficient if the peer has an unusual ref namespace, or if it simply has "remotes" refs of its own, and those locally unwanted refs are connecting to a large set of objects which becomes unreferenced as soon as they are fetched.

...

Quoted 1 line

+static void write_remote_refs(const struct ref *local_refs, const char *reflog)

Here reflog is now unused. I'm going to squash this in.

builtin-clone.c +2 -2
index edf7c7f..4992c25 100644
@@ -342,7 +342,7 @@ static struct ref *wanted_peer_refs(const struct ref *refs,
 	return local_refs;
 }
 
-static void write_remote_refs(const struct ref *local_refs, const char *reflog)
+static void write_remote_refs(const struct ref *local_refs)
 {
 	const struct ref *r;
 
@@ -534,7 +534,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	if (refs) {
 		clear_extra_refs();
 
-		write_remote_refs(mapped_refs, reflog_msg.buf);
+		write_remote_refs(mapped_refs);
 
 		remote_head = find_ref_by_name(refs, "HEAD");
 		remote_head_points_at =
Signature · 1 line
Shawn.
#24 lore raw

Re: [PATCH] make 'git clone' ask the remote only for objects it cares about

Quoted 8 lines · Shawn O. Pearce
Quoted 7 lines · Nicolas Pitre

Current behavior of 'git clone' when not using --mirror is to fetch everything from the peer, and then filter out unwanted refs just before writing them out to the cloned repository. This may become highly inefficient if the peer has an unusual ref namespace, or if it simply has "remotes" refs of its own, and those locally unwanted refs are connecting to a large set of objects which becomes unreferenced as soon as they are fetched.

Quoted 4 lines

...

Quoted 1 line

+static void write_remote_refs(const struct ref *local_refs, const char *reflog)

Here reflog is now unused. I'm going to squash this in.

Yeah, I noticed. Since I didn't know what was the original intent for it, I just left it there.

Nicolas

#25 lore raw

Re: [PATCH] make 'git clone' ask the remote only for objects it cares about

In reply to Nicolas Pitre
Quoted 5 lines · Nicolas Pitre

The fact that the git repository on gcc.gnu.org has lots of stuff in "remote" branches that don't get cloned by default is a separate configuration/policy issue on that server which might need (or not) to be looked into. For instance at least, as a bare repository, it should have all the git files in gcc.git/ directly instead of gcc.git/.git/.

The remote is just a git-svn tree.

Andreas.

Signature · 3 lines
Andreas Schwab, [email protected]
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."