Skip to contentSkip to conversation

git-prune-script eats data

DISCUSSION Jeff Garzik 2 participants 3 messages Jun 15, 2016

lore
Share & tools
lore
#0 lore raw

Ok, I have a reproducible case (at least for me), of git-prune-script munching data. I'll just give reproduction instructions (everyone reading this can do what I did), and some output at the end.

$ cd /repos
$ mkdir libata-dev-test/.git
$ cd libata-dev-test
$ cp -al ../linux-2.6/.git/objects .git/
$ rsync -az --verbose --delete \
rsync://rsync.kernel.org/pub/scm/linux/kernel/git/jgarzik/libata-dev.git/ 
 .git/		# word-wrapped from previous line
$ git-checkout-script -f
$ git-fsck-cache
dangling commit 1b142a71f3b131317489edf806abfab4c347476c
dangling commit 51a7f407d9b600e3278449a12135a21ffb0791a2
dangling commit eb93f3e7284204379444137a660b64f9dbd2ec04
dangling commit fcf604172829176bc618663e8387c8943ff88b66

NOTE: These dangling commits are NORMAL -- stuff that really does need pruning.

$ git-prune-script
$ git-fsck-cache
error: cannot map sha1 file c39ae07f393806ccf406ef966e9a15afc43cc36a
bad object in tag 5dc01c595e6c6ec9ccda4f6f69c131c0dd945f8c
bad sha1 entry '5dc01c595e6c6ec9ccda4f6f69c131c0dd945f8c'
$ rsync -az --verbose --delete \
rsync://rsync.kernel.org/pub/scm/linux/kernel/git/jgarzik/libata-dev.git/ 
 .git/		# word-wrapped from previous line
		# this second invocation downloads a TON of objects,
		# most/all of which are in the vanilla linux-2.6 tree
		# and should not have been pruned
#1 lore raw
Quoted 7 lines · Jeff Garzik
$ git-prune-script
$ git-fsck-cache
error: cannot map sha1 file c39ae07f393806ccf406ef966e9a15afc43cc36a
bad object in tag 5dc01c595e6c6ec9ccda4f6f69c131c0dd945f8c
bad sha1 entry '5dc01c595e6c6ec9ccda4f6f69c131c0dd945f8c'

It's exactly the same thing that Jens had. You have a tag object for the v2.6.11-tree thing, but you don't have the reference to the tag.

And now I realize what the problem is. It's _not_ that "git prune" has removed too much, like the obvious implication would be: it's that "git prune" has not removed _enough_.

"git prune" normally never removes tag-objects, whether reachable or not. That's because git-fsck-cache was explicitly ignoring them, which was because early on, the way you found tags was you did

	git-fsck-cache --tags

and then you created refs to them manually.

But now that special case causes problems (and is no longer needed anyway), because it means that "git prune" will not remove unreachable tags, but it _will_ remove everything that an unreachable tag points to (because that is also unreachable).

So the trivial fix is to just remove the lines from fsck-cache.c that say

	/* Don't bother with tag reachability. */
	if (obj->type == tag_type)
		continue;

and that will fix it for you.

Will do.

Linus

#2 lore raw
Quoted 1 line · Linus Torvalds

And now I realize what the problem is. It's _not_ that "git prune" has removed too much, like the obvious implication would be: it's that "git prune" has not removed _enough_.

Makes a lot of sense. Looking at the repo, I did indeed forget to copy the tags.

Quoted 7 lines

So the trivial fix is to just remove the lines from fsck-cache.c that say

	/* Don't bother with tag reachability. */
	if (obj->type == tag_type)
		continue;

and that will fix it for you.

Sounds good, thanks.

Quoted 1 line

It's exactly the same thing that Jens had. You have a tag object for the v2.6.11-tree thing, but you don't have the reference to the tag.

Ref email just sent: Kernel hackers expect tags to come with the pull.
Four kernel hackers, and counting.

Just accept that our brains are wired that way ;-) We like having Linus-blessed-and-pushed-to-kernel.org tags from linux-2.6.git public tree follow us around.

	git-pull-script --tags $url

should accomplish that.

Jeff