HTTP repo referencing stale heads (can't clone)

Hi,

I maintain a small git repo. I upload it over ssh (with git-push) to a machine where it is distributed over http:

http://dsd.object4.net/git/zd1211.git/

For some reason it is no longer possible to clone this repo over http:

walk 35afe6b3a859242a18812e7485ea8b211e24abaf
walk 93d9a9f469282e1e392c16ce571da4c08805e8bb
error: Couldn't get http://dsd.object4.net/git/zd1211.git/refs/heads/softmac-old for heads/softmac-old The requested URL returned error: 404
error: Could not interpret heads/softmac-old as something to pull

"softmac-old" is an old branch, which I have recently deleted. I deleted it by removing the .git/refs/heads/softmac-old file, and relying on git-prune to clear out old objects.

Even on the server-side, there is no obvious reference to this old head:

$ find -name '*softmac*'
$ grep -R softmac *
(no results for either)

"git-fsck-objects" reports nothing, "git-fsck-objects --full" reports:
dangling commit 7cc423c942975005f96f308186537ad6e7808c2e dangling commit b36378de6231f1b5100b1517b9c8c243a21090fd

I have tried running git-prune and git-update-server-info, but that doesn't help.

Any ideas? I'm still new to git.
I am running git-1.2.4

Thanks,
Daniel

My standard answer would be

http-server$ cd /var/www/git/zd1211.git/ ;# or whereever http-server$ GIT_DIR=. git-update-server-info

but you said you have run it already... Can you try to see if there is some caching proxy involved that still serves stale info/refs file?

client$ wget http://dsd.object4.net/git/zd1211.git/info/refs

Is there any interest in making the HTTP transport slighly less dumb by using DAV?

I have a working patch to http-fetch that tries to use PROPFIND to get a remote pack list and falls back to using objects/info/packs. It's feasible to do something similar to get a remote ref list when cloning, although that's a bit more work as all refs would have to be fetched into a local repo and parsed to determine the object type.

Long term, this could give a repo admin the choice of either making sure git-update-server-info has been run after every ref/pack change or enabling DAV once. Assuming they need to use HTTP.

For a successful technology, reality must take precedence over public
relations, for nature cannot be fooled.

Dear diary, on Mon, Apr 03, 2006 at 08:09:30PM CEST, I got a letter where Nick Hengeveld <[email protected]> said that...

Well, what is the actual advantage of DAV compared to git-update-server-info? Why would I prefer enabling DAV?

				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Right now I am having amnesia and deja-vu at the same time.  I think
I have forgotten this before.

In theory, things should work the same either way. It seems that in practice though, the server info files continue to surface as a source of fetch problems.

For a successful technology, reality must take precedence over public
relations, for nature cannot be fooled.

Dear diary, on Tue, Apr 04, 2006 at 02:10:57PM CEST, I got a letter where Nick Hengeveld <[email protected]> said that...

Because people do not know they have to set up their post-update hook. When they are already going at lengths to find out how to set up DAV for git fetch, they would discover the post-update hook way as well.

So, it really seems rather redundant to me.

				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
Right now I am having amnesia and deja-vu at the same time.  I think
I have forgotten this before.

There are other reasons that a client can end up with a stale copy of the server info files - in this case the problem was an intermediate proxy out of the control of the repo admin.

While we should be able to fix that particular problem, it seems safer to go straight to the source if possible.

If I've already enabled DAV for pushing to a repo, I'd find it nice to be able to use it for fetches as well.

For a successful technology, reality must take precedence over public
relations, for nature cannot be fooled.

I personally feel PROPFIND is the right way to do "wget -r", and very much welcome a patch to replace objects/info/packs with it when able.

Faking info/refs with PROPFIND, if we do not have to peel the onion ^{}, should be relatively cheap operation, and could be done as an enhancement to git-ls-remote.sh. If your faked info/refs file lacks ^{} entries, git-fetch cannot auto-follow tags, but git-clone should work as before.

A clever sysadmins could mod_rewrite requests to info/refs and objects/info/packs with a custom CGI, but then probably they would be running git-daemon ;-).

Ah, should have known. I am behind a (lame) transparent proxy on port 80.

I opened that file in my web browser and it showed the old heads. After a force-refresh (ctrl+F5, which sends some additionally http headers to refresh the page from the real server), the old heads disappeared, and git now clones successfully.

git-http-fetch should probably send those extra headers too. I'll try to find some time to look at this next week.

Thanks!
Daniel

git-http-fetch uses the "Pragma: no-cache" header when requesting objects that shouldn't be cached. Is this the additional header you're referring to?

This patch adds the header to git-ls-remote for the info/refs request.

git-ls-remote: send no-cache header when fetching info/refs

Proxies should not cache this file as it can cause a client to end up with a stale version, as reported here:

http://marc.theaimsgroup.com/?l=git&m=114407944125389

---

 git-ls-remote.sh |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

da9b6fa01f1a8bd6ab5f6d4346584f3f032584aa

index 2c9a588..b6882a9 100755
@@ -53,7 +53,7 @@ http://* | https://* )
         if [ -n "$GIT_SSL_NO_VERIFY" ]; then
             curl_extra_args="-k"
         fi
-	curl -nsf $curl_extra_args "$peek_repo/info/refs" ||
+	curl -nsf $curl_extra_args --header "Pragma: no-cache" "$peek_repo/info/refs" ||
 		echo "failed	slurping"
 	;;
 

1.3.0.rc1.g9aef-dirty

As per HTTP 1.1, it should also send:
Cache-Control: no-cache

Pragma: no-cache is the deprecated extension.
It's safe to send both.

GPG Key id:  0xD1F10BA2
Fingerprint: 96E2 304A B9C4 949A 10A0  9105 9543 0453 D1F1 0BA2

AstralStorm