Skip to contentSkip to conversation

cogito: Don't ever assume object type in cg-object-id

PATCH In review Pavel Roskin 2 participants 8 messages Jun 15, 2016

lore
Share & tools
lore
#0 lore raw

PATCH 6/8 cogito: Don't ever assume object type in cg-object-id

Don't ever assume object type in cg-object-id.

If type is unknown in normalize_id(), make it empty so it's checked later. Check parent ID that it's a commit.

cg-object-id +11 -9
@@ -24,22 +24,22 @@
 USAGE="cg-object-id [-p] [-t] [OBJECT_ID]"
 
 . ${COGITO_LIB}cg-Xlib
 
 
-# Normalize argument.  The normalized SHA1 ID is put to $normid,
-# type is put to $type.
+# Normalize argument.  The normalized SHA1 ID is put to $normid.
+# If type is known, it's put to $type.
 normalize_id()
 {
 	local id="$1"
+	local type=
 
 	if [ "${id:(-1):1}" = "^" ]; then
 		# find first parent
 		normalize_id "${id%^}"
 		normid=$(git-cat-file commit "$normid" | \
 			 awk '/^parent/{print $2; exit};/^$/{exit}') || exit 1
-		type="commit"
 		return
 	fi
 
 	if [ ! "$id" ] || [ "$id" = "this" ] || [ "$id" = "HEAD" ]; then
 		read id < "$_git/HEAD"
@@ -118,31 +118,33 @@ if [ "$show_parent" -a "$show_tree" ]; t
 	exit 1
 fi
 
 id="${ARGS[0]}"
 normalize_id "$id"
+[ "$type" ] || type=$(git-cat-file -t "$normid")
+
+if [ -z "$show_tree" ]; then
+	if [ "$type" != "commit" ]; then
+		echo "Invalid commit id: $normid" >&2
+		exit 1
+	fi
+fi
 
 if [ "$show_parent" ]; then
 	git-cat-file commit "$normid" | awk '/^parent/{print $2};/^$/{exit}'
 	exit 0
 fi
 
-[ "$type" ] || type=$(git-cat-file -t "$normid")
 if [ "$show_tree" ]; then
 	if [ "$type" = "commit" ]; then
 		normid=$(git-cat-file commit "$normid" | sed -e 's/tree //;q')
 		type=$(git-cat-file -t "$normid")
 	fi
 
 	if [ "$type" != "tree" ]; then
 		echo "Invalid tree id: $normid" >&2
 		exit 1
 	fi
-else
-	if [ "$type" != "commit" ]; then
-		echo "Invalid commit id: $normid" >&2
-		exit 1
-	fi
 fi
 
 echo $normid
 
Signature · 2 lines
Regards,
Pavel Roskin
#1 lore raw

Dear diary, on Tue, Sep 20, 2005 at 04:25:20AM CEST, I got a letter where Pavel Roskin <[email protected]> told me that...

Quoted 4 lines

Don't ever assume object type in cg-object-id.

If type is unknown in normalize_id(), make it empty so it's checked later. Check parent ID that it's a commit.

Why? In most of the cases, you have one pointless fork() more, and we "assume" only if it was in a parent line, and if that's not a commit, you've got a *serious* problem with your database. fsck should yell you to death, and I don't think you could even normally commit such a commit, so this should never happen. There's no point in wasting time trying to handle seriously messed up database.

Signature · 4 lines
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
#2 lore raw

Hello, Petr!

Quoted 7 lines · Petr Baudis

Dear diary, on Tue, Sep 20, 2005 at 04:25:20AM CEST, I got a letter where Pavel Roskin <[email protected]> told me that...

Quoted 4 lines

Don't ever assume object type in cg-object-id.

If type is unknown in normalize_id(), make it empty so it's checked later. Check parent ID that it's a commit.

Quoted 6 lines

Why? In most of the cases, you have one pointless fork() more, and we "assume" only if it was in a parent line, and if that's not a commit, you've got a *serious* problem with your database. fsck should yell you to death, and I don't think you could even normally commit such a commit, so this should never happen. There's no point in wasting time trying to handle seriously messed up database.

I agree in this case. I actually was unsure, but I decided to play safe after reading this comment in cg-tag:

# This is most usually the ID of the commit to tag. Tagging
# other objects than commits is possible, but rather "unusual".

As for fsck "yelling to death", when was the last time you tried that on the cogito repository fetched by cogito? That's what I get:

[proski@dv cogito]$ git-fsck-objects
missing tree 0842d4cb3cd36675c518c241b16cf25fad0c5384 broken link from commit 2c70421be7d88fbee49986d7a5584d1f010a25de

              to    tree 8c365c79293c9a8a86dc6802f6230e389e876acb
broken link from  commit 2c70421be7d88fbee49986d7a5584d1f010a25de
              to  commit 659bd6b6489ae254a07b1bc578ff44778ed7b8b4
broken link from  commit 463d05c7c4fe7f24da29749f4c7f25893fc20b8c
              to    tree 0842d4cb3cd36675c518c241b16cf25fad0c5384
broken link from  commit 463d05c7c4fe7f24da29749f4c7f25893fc20b8c
              to  commit ed34298a39b05e05bd333d9dac658df19e5b2dab

missing commit 659bd6b6489ae254a07b1bc578ff44778ed7b8b4 missing tree 8c365c79293c9a8a86dc6802f6230e389e876acb missing commit c4cd9bc72a6e0ed355041d331bb4034d99738f82 broken link from commit d14925c87cdb6ca6345bcb3c8e34a2d659c79451

              to    tree e5ea614d1884a02314a3be8e09cb3f8c786fc0e9
broken link from  commit d14925c87cdb6ca6345bcb3c8e34a2d659c79451
              to  commit c4cd9bc72a6e0ed355041d331bb4034d99738f82

missing tree e5ea614d1884a02314a3be8e09cb3f8c786fc0e9 missing commit ed34298a39b05e05bd333d9dac658df19e5b2dab

It's not like broken trees are exceedingly rare. Sure, this must be some "better" kind of breakage, but anyway.

Signature · 2 lines
Regards,
Pavel Roskin
#3 lore raw

Dear diary, on Thu, Sep 22, 2005 at 05:27:05AM CEST, I got a letter where Pavel Roskin <[email protected]> told me that...

Quoted 17 lines

As for fsck "yelling to death", when was the last time you tried that on the cogito repository fetched by cogito? That's what I get:

[proski@dv cogito]$ git-fsck-objects
missing tree 0842d4cb3cd36675c518c241b16cf25fad0c5384 broken link from commit 2c70421be7d88fbee49986d7a5584d1f010a25de

              to    tree 8c365c79293c9a8a86dc6802f6230e389e876acb
broken link from  commit 2c70421be7d88fbee49986d7a5584d1f010a25de
              to  commit 659bd6b6489ae254a07b1bc578ff44778ed7b8b4
broken link from  commit 463d05c7c4fe7f24da29749f4c7f25893fc20b8c
              to    tree 0842d4cb3cd36675c518c241b16cf25fad0c5384
broken link from  commit 463d05c7c4fe7f24da29749f4c7f25893fc20b8c
              to  commit ed34298a39b05e05bd333d9dac658df19e5b2dab

missing commit 659bd6b6489ae254a07b1bc578ff44778ed7b8b4 missing tree 8c365c79293c9a8a86dc6802f6230e389e876acb missing commit c4cd9bc72a6e0ed355041d331bb4034d99738f82 broken link from commit d14925c87cdb6ca6345bcb3c8e34a2d659c79451

Quoted 5 lines
              to    tree e5ea614d1884a02314a3be8e09cb3f8c786fc0e9
broken link from  commit d14925c87cdb6ca6345bcb3c8e34a2d659c79451
              to  commit c4cd9bc72a6e0ed355041d331bb4034d99738f82

missing tree e5ea614d1884a02314a3be8e09cb3f8c786fc0e9 missing commit ed34298a39b05e05bd333d9dac658df19e5b2dab

Heh, your database seems fairly broken.

Quoted 2 lines

It's not like broken trees are exceedingly rare. Sure, this must be some "better" kind of breakage, but anyway.

$ git-fsck-objects
error: b3e9704ecdf48869f635f0aa99ddfb513f885aff: object not found

and some dangling stuff, that's all I get. This seems to be caused by the mere existence of that (empty) file, so I just removed it from the objects database and all is fine now.

Signature · 4 lines
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
#4 lore raw

Hi, Petr!

Quoted 6 lines · Petr Baudis
$ git-fsck-objects
error: b3e9704ecdf48869f635f0aa99ddfb513f885aff: object not found

and some dangling stuff, that's all I get. This seems to be caused by the mere existence of that (empty) file, so I just removed it from the objects database and all is fine now.

Maybe kernel.org site is not in sync yet, but anyway, here's what I found.

I ran this script:

[proski@dv cogito]$ for i in .git/refs/tags/*; do git-tar-tree cat $i >/dev/null || echo $i; done
fatal: not a reference to a tag, commit or tree object: 463d05c7c4fe7f24da29749f4c7f25893fc20b8c .git/refs/tags/git-pasky-0.1
fatal: not a reference to a tag, commit or tree object: 2c70421be7d88fbee49986d7a5584d1f010a25de .git/refs/tags/git-pasky-0.2
fatal: not a reference to a tag, commit or tree object: d14925c87cdb6ca6345bcb3c8e34a2d659c79451 .git/refs/tags/git-pasky-0.3

If I delete the bad tags and prune the tree, git-fsck-objects has nothing to complain about:

[proski@dv cogito]$ rm .git/refs/tags/git-pasky-0.1 [proski@dv cogito]$ rm .git/refs/tags/git-pasky-0.2 [proski@dv cogito]$ rm .git/refs/tags/git-pasky-0.3 [proski@dv cogito]$ git-prune origin cat .git/refs/tags/* [proski@dv cogito]$ git-fsck-objects
[proski@dv cogito]$

That's probably what you should do on the master tree if you haven't done it.

I can run "git-prune origin", but then cg-fetch would refetch objects for more tags:

Missing object of tag cogito-0.12... retrieved
Missing object of tag cogito-0.12.1... retrieved
Missing object of tag cogito-0.13... retrieved
Missing object of tag cogito-0.14... retrieved
Missing object of tag cogito-0.14.1... retrieved
Missing object of tag cogito-0.14.2... retrieved
Missing object of tag cogito-0.15... retrieved
Missing object of tag git-pasky-0.1... retrieved
Missing object of tag git-pasky-0.2... retrieved
Missing object of tag git-pasky-0.3... retrieved

How come that cogito-0.12 - cogito-0.15 are not on the origin tree, but older cogito tags are?

Signature · 2 lines
Regards,
Pavel Roskin
#5 lore raw

Dear diary, on Thu, Sep 22, 2005 at 05:37:59PM CEST, I got a letter where Pavel Roskin <[email protected]> told me that...

Quoted 1 line

Hi, Petr!

Hi,

Quoted 10 lines · Petr Baudis
Quoted 6 lines
$ git-fsck-objects
error: b3e9704ecdf48869f635f0aa99ddfb513f885aff: object not found

and some dangling stuff, that's all I get. This seems to be caused by the mere existence of that (empty) file, so I just removed it from the objects database and all is fine now.

Maybe kernel.org site is not in sync yet, but anyway, here's what I found.

root 4036 2.7 2.8 106756 104080 ? SN 00:04 14:32 rsync

:/

Quoted 9 lines

I ran this script:

[proski@dv cogito]$ for i in .git/refs/tags/*; do git-tar-tree cat $i >/dev/null || echo $i; done
fatal: not a reference to a tag, commit or tree object: 463d05c7c4fe7f24da29749f4c7f25893fc20b8c .git/refs/tags/git-pasky-0.1
fatal: not a reference to a tag, commit or tree object: 2c70421be7d88fbee49986d7a5584d1f010a25de .git/refs/tags/git-pasky-0.2
fatal: not a reference to a tag, commit or tree object: d14925c87cdb6ca6345bcb3c8e34a2d659c79451 .git/refs/tags/git-pasky-0.3

Huh. Huh?

xpasky@machine[1:0]~/git-devel$ git-cat-file -t $(cg-object-id git-pasky-0.1)
commit
xpasky@machine[1:0]~/git-devel$ git-cat-file -t $(cg-object-id git-pasky-0.2)
commit
xpasky@machine[1:0]~/git-devel$ git-cat-file -t $(cg-object-id git-pasky-0.3)
commit

Same on master.kernel.org.

Quoted 14 lines

I can run "git-prune origin", but then cg-fetch would refetch objects for more tags:

Missing object of tag cogito-0.12... retrieved
Missing object of tag cogito-0.12.1... retrieved
Missing object of tag cogito-0.13... retrieved
Missing object of tag cogito-0.14... retrieved
Missing object of tag cogito-0.14.1... retrieved
Missing object of tag cogito-0.14.2... retrieved
Missing object of tag cogito-0.15... retrieved
Missing object of tag git-pasky-0.1... retrieved
Missing object of tag git-pasky-0.2... retrieved
Missing object of tag git-pasky-0.3... retrieved

Quoted 2 lines

How come that cogito-0.12 - cogito-0.15 are not on the origin tree, but older cogito tags are?

That's because the older tags tag commits directly, not tag objects. git-prune removed the tag objects because they were unreferenced by its perspective. Anyway, git-prune without any arguments should get it right.

Signature · 4 lines
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.
#6 lore raw
Quoted 10 lines · Petr Baudis

Dear diary, on Thu, Sep 22, 2005 at 05:37:59PM CEST, I got a letter where Pavel Roskin <[email protected]> told me that...

Quoted 7 lines

[proski@dv cogito]$ for i in .git/refs/tags/*; do git-tar-tree cat $i >/dev/null || echo $i; done
fatal: not a reference to a tag, commit or tree object: 463d05c7c4fe7f24da29749f4c7f25893fc20b8c .git/refs/tags/git-pasky-0.1
fatal: not a reference to a tag, commit or tree object: 2c70421be7d88fbee49986d7a5584d1f010a25de .git/refs/tags/git-pasky-0.2
fatal: not a reference to a tag, commit or tree object: d14925c87cdb6ca6345bcb3c8e34a2d659c79451 .git/refs/tags/git-pasky-0.3

Quoted 4 lines

Huh. Huh?

xpasky@machine[1:0]~/git-devel$ git-cat-file -t $(cg-object-id git-pasky-0.1)
commit

The error message is misleading. In fact, it's the tree referred to by the commit that was missing. Anyway, that object exists on the server, so it was my local problem.

It would be nice to have a script fetch an object by SHA1, possibly with the dependencies. git-fetch doesn't do it yet.

I copied the whole objects tree for cogito using rsync, and I see lots or error messages from git-fsck-objects. I'll wait for your changes to propagate. As it stands now, b3e9704ecdf48869f635f0aa99ddfb513f885aff (a file you said you have removed) and 11ed64c1b141c9ba397a1ca76aef2cd250976007 are still present and empty.

Are you sure rsync that copies to kernel.org uses the "delete" option? If not, we should treat kernel.org as "dumb transport" where directory listings should never be used.

Quoted 7 lines
Quoted 2 lines

How come that cogito-0.12 - cogito-0.15 are not on the origin tree, but older cogito tags are?

That's because the older tags tag commits directly, not tag objects. git-prune removed the tag objects because they were unreferenced by its perspective. Anyway, git-prune without any arguments should get it right.

Thanks for the explanation. I'll be more careful with git-prune.

Signature · 2 lines
Regards,
Pavel Roskin
#7 lore raw
In reply to Petr Baudis

Dear diary, on Thu, Sep 22, 2005 at 11:46:43AM CEST, I got a letter where Petr Baudis <[email protected]> told me that...

Quoted 26 lines

Dear diary, on Thu, Sep 22, 2005 at 05:27:05AM CEST, I got a letter where Pavel Roskin <[email protected]> told me that...

Quoted 17 lines

As for fsck "yelling to death", when was the last time you tried that on the cogito repository fetched by cogito? That's what I get:

[proski@dv cogito]$ git-fsck-objects
missing tree 0842d4cb3cd36675c518c241b16cf25fad0c5384 broken link from commit 2c70421be7d88fbee49986d7a5584d1f010a25de

              to    tree 8c365c79293c9a8a86dc6802f6230e389e876acb
broken link from  commit 2c70421be7d88fbee49986d7a5584d1f010a25de
              to  commit 659bd6b6489ae254a07b1bc578ff44778ed7b8b4
broken link from  commit 463d05c7c4fe7f24da29749f4c7f25893fc20b8c
              to    tree 0842d4cb3cd36675c518c241b16cf25fad0c5384
broken link from  commit 463d05c7c4fe7f24da29749f4c7f25893fc20b8c
              to  commit ed34298a39b05e05bd333d9dac658df19e5b2dab

missing commit 659bd6b6489ae254a07b1bc578ff44778ed7b8b4 missing tree 8c365c79293c9a8a86dc6802f6230e389e876acb missing commit c4cd9bc72a6e0ed355041d331bb4034d99738f82 broken link from commit d14925c87cdb6ca6345bcb3c8e34a2d659c79451

Quoted 5 lines
              to    tree e5ea614d1884a02314a3be8e09cb3f8c786fc0e9
broken link from  commit d14925c87cdb6ca6345bcb3c8e34a2d659c79451
              to  commit c4cd9bc72a6e0ed355041d331bb4034d99738f82

missing tree e5ea614d1884a02314a3be8e09cb3f8c786fc0e9 missing commit ed34298a39b05e05bd333d9dac658df19e5b2dab

Quoted 1 line

Heh, your database seems fairly broken.

Ok, this seems like broken database on kernel.org, sorry about that. It actually just contained some superfluous objects. I pruned it and it seems that fsck is happy now.

Signature · 4 lines
				Petr "Pasky" Baudis
Stuff: http://pasky.or.cz/
VI has two modes: the one in which it beeps and the one in which
it doesn't.