cat-file: add %(objectmode) avoid verifying submodules' OIDs
cover cat-file: add %(objectmode) avoid verifying submodules' OIDs
The cat-file --batch command is very valuable in server settings, but so far it is missing a bit of functionality that would come in handy there.
For example, it is sometimes necessary to determine the object mode of a batch of tree objects' children.
This came up in $dayjob recently, and applies cleanly to v2.44.0.
Johannes Schindelin (1):
cat-file: avoid verifying submodules' OIDs
Victoria Dye (2):
t1006: update 'run_tests' to test generic object specifiers
cat-file: add %(objectmode) atom
Documentation/git-cat-file.txt | 10 +++++ builtin/cat-file.c | 41 ++++++++++++++---- t/t1006-cat-file.sh | 79 +++++++++++++++++++++------------- 3 files changed, 92 insertions(+), 38 deletions(-)
base-commit: 3c2a3fdc388747b9eaf4a4a4f2035c1c9ddb26d0
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1689%2Fdscho%2Fcat-file-vs-submodules-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1689/dscho/cat-file-vs-submodules-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1689
Signature · 1 line
gitgitgadget
PATCH 1/3 t1006: update 'run_tests' to test generic object specifiers
From: Victoria Dye <[email protected]>
Update the 'run_tests' test wrapper so that the first argument may refer to any specifier that uniquely identifies an object (e.g. a ref name, '<OID>:<path>', '<OID>^{<type>}', etc.), rather than only a full object ID. Also, add a test that uses a non-OID identifier, ensuring appropriate parsing in 'cat-file'.
--- t/t1006-cat-file.sh | 46 +++++++++++++++++++++++---------------------- 1 file changed, 24 insertions(+), 22 deletions(-)
t/t1006-cat-file.sh +24 -22
@@ -112,65 +112,66 @@ strlen () { run_tests () { type=$1 - sha1=$2 + object_name=$2 + oid=$(git rev-parse --verify $object_name) size=$3 content=$4 pretty_content=$5 - batch_output="$sha1 $type $size + batch_output="$oid $type $size $content" test_expect_success "$type exists" ' - git cat-file -e $sha1 + git cat-file -e $object_name ' test_expect_success "Type of $type is correct" ' echo $type >expect && - git cat-file -t $sha1 >actual && + git cat-file -t $object_name >actual && test_cmp expect actual ' test_expect_success "Size of $type is correct" ' echo $size >expect && - git cat-file -s $sha1 >actual && + git cat-file -s $object_name >actual && test_cmp expect actual ' test_expect_success "Type of $type is correct using --allow-unknown-type" ' echo $type >expect && - git cat-file -t --allow-unknown-type $sha1 >actual && + git cat-file -t --allow-unknown-type $object_name >actual && test_cmp expect actual ' test_expect_success "Size of $type is correct using --allow-unknown-type" ' echo $size >expect && - git cat-file -s --allow-unknown-type $sha1 >actual && + git cat-file -s --allow-unknown-type $object_name >actual && test_cmp expect actual ' test -z "$content" || test_expect_success "Content of $type is correct" ' echo_without_newline "$content" >expect && - git cat-file $type $sha1 >actual && + git cat-file $type $object_name >actual && test_cmp expect actual ' test_expect_success "Pretty content of $type is correct" ' echo_without_newline "$pretty_content" >expect && - git cat-file -p $sha1 >actual && + git cat-file -p $object_name >actual && test_cmp expect actual ' test -z "$content" || test_expect_success "--batch output of $type is correct" ' echo "$batch_output" >expect && - echo $sha1 | git cat-file --batch >actual && + echo $object_name | git cat-file --batch >actual && test_cmp expect actual ' test_expect_success "--batch-check output of $type is correct" ' - echo "$sha1 $type $size" >expect && - echo_without_newline $sha1 | git cat-file --batch-check >actual && + echo "$oid $type $size" >expect && + echo_without_newline $object_name | git cat-file --batch-check >actual && test_cmp expect actual '
@@ -179,33 +180,33 @@ $content" test -z "$content" || test_expect_success "--batch-command $opt output of $type content is correct" ' echo "$batch_output" >expect && - test_write_lines "contents $sha1" | git cat-file --batch-command $opt >actual && + test_write_lines "contents $object_name" | git cat-file --batch-command $opt >actual && test_cmp expect actual ' test_expect_success "--batch-command $opt output of $type info is correct" ' - echo "$sha1 $type $size" >expect && - test_write_lines "info $sha1" | + echo "$oid $type $size" >expect && + test_write_lines "info $object_name" | git cat-file --batch-command $opt >actual && test_cmp expect actual ' done test_expect_success "custom --batch-check format" ' - echo "$type $sha1" >expect && - echo $sha1 | git cat-file --batch-check="%(objecttype) %(objectname)" >actual && + echo "$type $oid" >expect && + echo $object_name | git cat-file --batch-check="%(objecttype) %(objectname)" >actual && test_cmp expect actual ' test_expect_success "custom --batch-command format" ' - echo "$type $sha1" >expect && - echo "info $sha1" | git cat-file --batch-command="%(objecttype) %(objectname)" >actual && + echo "$type $oid" >expect && + echo "info $object_name" | git cat-file --batch-command="%(objecttype) %(objectname)" >actual && test_cmp expect actual ' test_expect_success '--batch-check with %(rest)' ' echo "$type this is some extra content" >expect && - echo "$sha1 this is some extra content" | + echo "$object_name this is some extra content" | git cat-file --batch-check="%(objecttype) %(rest)" >actual && test_cmp expect actual '
@@ -216,7 +217,7 @@ $content" echo "$size" && echo "$content" } >expect && - echo $sha1 | git cat-file --batch="%(objectsize)" >actual && + echo $object_name | git cat-file --batch="%(objectsize)" >actual && test_cmp expect actual '
@@ -226,7 +227,7 @@ $content" echo "$type" && echo "$content" } >expect && - echo $sha1 | git cat-file --batch="%(objecttype)" >actual && + echo $object_name | git cat-file --batch="%(objecttype)" >actual && test_cmp expect actual ' }
@@ -271,6 +272,7 @@ tree_size=$(($(test_oid rawsz) + 13)) tree_pretty_content="100644 blob $hello_sha1 hello${LF}" run_tests 'tree' $tree_sha1 $tree_size "" "$tree_pretty_content" +run_tests 'blob' "$tree_sha1:hello" $hello_size "" "$hello_content" commit_message="Initial commit" commit_sha1=$(echo_without_newline "$commit_message" | git commit-tree $tree_sha1)
Signature · 1 line
gitgitgadget
Re: [PATCH 1/3] t1006: update 'run_tests' to test generic object specifiers
Quoted 22 lines · Victoria Dye via GitGitGadget
From: Victoria Dye <[email protected]>
Update the '
run_tests' test wrapper so that the first argument may refer to any specifier that uniquely identifies an object (e.g. a ref name, '<OID>:<path>', '<OID>^{<type>}', etc.), rather than only a full object ID. Also, add a test that uses a non-OID identifier, ensuring appropriate parsing in 'cat-file'.Signed-off-by Victoria Dye <[email protected]>Signed-off-by Johannes Schindelin <[email protected]>--- t/t1006-cat-file.sh | 46 +++++++++++++++++++++++---------------------- 1 file changed, 24 insertions(+), 22 deletions(-)t/t1006-cat-file.sh +0
-0@@ -112,65 +112,66 @@ strlen () { run_tests () { type=$1
t/t1006-cat-file.sh strlen () { · line 122
Quoted 10 lines
t/t1006-cat-file.sh +3
-2- sha1=$2 + object_name=$2 + oid=$(git rev-parse --verify $object_name) size=$3 content=$4 pretty_content=$5 - batch_output="$sha1 $type $size + batch_output="$oid $type $size $content"
As "object_name" is now allowed to be any name in the 'extended SHA-1' syntax (cf. Documentation/revisions.txt), you should be a bit more careful in quoting.
oid=$(git rev-parse --verify "$object_name")
t/t1006-cat-file.sh strlen () { · line 126
Quoted 4 lines
t/t1006-cat-file.sh +1
-1test_expect_success "$type exists" ' - git cat-file -e $sha1 + git cat-file -e $object_name '
Likewise. You may not currently use a path with SP in it to name a tree object, e.g., "HEAD:Read Me.txt", but protecting against such a pathname is a cheap investment for futureproofing.
Looking good otherwise. Thanks.
PATCH 2/3 cat-file: add %(objectmode) atom
In reply to Johannes Schindelin via GitGitGadgetFrom: Victoria Dye <[email protected]>
Add a formatting atom, used with the --batch-check/--batch-command options, that prints the octal representation of the object mode if a given revision includes that information, e.g. one that follows the format <tree-ish>:<path>. If the mode information does not exist, an empty string is printed instead.
--- Documentation/git-cat-file.txt | 5 +++++ builtin/cat-file.c | 9 +++++++-- t/t1006-cat-file.sh | 23 +++++++++++++++-------- 3 files changed, 27 insertions(+), 10 deletions(-)
Documentation/git-cat-file.txt +5 -0
@@ -292,6 +292,11 @@ newline. The available atoms are: `objecttype`:: The type of the object (the same as `cat-file -t` reports). +`objectmode`:: + If the specified object has mode information (such as a tree or + index entry), the mode expressed as an octal integer. Otherwise, + empty string. + `objectsize`:: The size, in bytes, of the object (the same as `cat-file -s` reports).
builtin/cat-file.c +7 -2
@@ -272,6 +272,7 @@ struct expand_data {struct object_id oid;enum object_type type;unsigned long size; +unsigned short mode;off_t disk_size;const char *rest;struct object_id delta_base_oid;
@@ -303,6 +304,7 @@ struct expand_data {*/ unsigned skip_object_info : 1; }; +#define EXPAND_DATA_INIT { .mode = S_IFINVALID }static int is_atom (const char *atom,const char *s,int slen) {
@@ -342,6 +344,9 @@ static void expand_atom(struct strbuf *sb, const char *atom, int len,else strbuf_addstr (sb,oid_to_hex (&data->delta_base_oid)); + }else if (is_atom ("objectmode" , atom, len)) { +if (!data->mark_query && !(S_IFINVALID == data->mode)) +strbuf_addf (sb,"%06o" , data->mode); }else die ("unknown format element: %.*s" , len, atom); }
@@ -562,6 +567,7 @@ static void batch_one_object(const char *obj_name,return ; } + data->mode = ctx.mode;batch_object_write (obj_name, scratch, opt, data,NULL ,0 ); }
@@ -766,7 +772,7 @@ static int batch_objects(struct batch_options *opt) {struct strbuf input =STRBUF_INIT ;struct strbuf output =STRBUF_INIT ; -struct expand_data data; +struct expand_data data =EXPAND_DATA_INIT ;int save_warning;int retval =0 ;
@@ -775,7 +781,6 @@ static int batch_objects(struct batch_options *opt)* object_info to be handed to oid_object_info_extended for each * object. */ -memset (&data,0 ,sizeof (data)); data.mark_query =1 ; expand_format(&output, opt->format ? opt->format : DEFAULT_FORMAT,
t/t1006-cat-file.sh +15 -8
@@ -114,9 +114,10 @@ run_tests () { type=$1 object_name=$2 oid=$(git rev-parse --verify $object_name) - size=$3 - content=$4 - pretty_content=$5 + mode=$3 + size=$4 + content=$5 + pretty_content=$6 batch_output="$oid $type $size $content"
@@ -211,6 +212,12 @@ $content" test_cmp expect actual ' + test_expect_success '--batch-check with %(objectmode)' ' + echo "$mode $oid" >expect && + echo $object_name | git cat-file --batch-check="%(objectmode) %(objectname)" >actual && + test_cmp expect actual + ' + test -z "$content" || test_expect_success "--batch without type ($type)" ' {
@@ -241,7 +248,7 @@ test_expect_success "setup" ' git update-index --add hello ' -run_tests 'blob' $hello_sha1 $hello_size "$hello_content" "$hello_content" +run_tests 'blob' $hello_sha1 "" $hello_size "$hello_content" "$hello_content" test_expect_success '--batch-command --buffer with flush for blob info' ' echo "$hello_sha1 blob $hello_size" >expect &&
@@ -271,8 +278,8 @@ tree_sha1=$(git write-tree) tree_size=$(($(test_oid rawsz) + 13)) tree_pretty_content="100644 blob $hello_sha1 hello${LF}" -run_tests 'tree' $tree_sha1 $tree_size "" "$tree_pretty_content" -run_tests 'blob' "$tree_sha1:hello" $hello_size "" "$hello_content" +run_tests 'tree' $tree_sha1 "" $tree_size "" "$tree_pretty_content" +run_tests 'blob' "$tree_sha1:hello" "100644" $hello_size "" "$hello_content" commit_message="Initial commit" commit_sha1=$(echo_without_newline "$commit_message" | git commit-tree $tree_sha1)
@@ -283,7 +290,7 @@ committer $GIT_COMMITTER_NAME <$GIT_COMMITTER_EMAIL> $GIT_COMMITTER_DATE $commit_message" -run_tests 'commit' $commit_sha1 $commit_size "$commit_content" "$commit_content" +run_tests 'commit' $commit_sha1 "" $commit_size "$commit_content" "$commit_content" tag_header_without_timestamp="object $hello_sha1 type blob
@@ -297,7 +304,7 @@ $tag_description" tag_sha1=$(echo_without_newline "$tag_content" | git hash-object -t tag --stdin -w) tag_size=$(strlen "$tag_content") -run_tests 'tag' $tag_sha1 $tag_size "$tag_content" "$tag_content" +run_tests 'tag' $tag_sha1 "" $tag_size "$tag_content" "$tag_content" test_expect_success "Reach a blob from a tag pointing to it" ' echo_without_newline "$hello_content" >expect &&
Signature · 1 line
gitgitgadget
Re: [PATCH 2/3] cat-file: add %(objectmode) atom
builtin/cat-file.c struct expand_data { · line 276
Quoted 10 lines · Victoria Dye via GitGitGadget
builtin/cat-file.c +1
-0@@ -272,6 +272,7 @@ struct expand_data {struct object_id oid;enum object_type type;unsigned long size; +unsigned short mode;off_t disk_size;
We are not saving the storage used in this structure by using "unsigned short" due to alignment, so I got curious where the choice came from, but I do not think of any sensible explanation.
Let's to be consistent with the remainder of the system, like how the mode is stored in the in-core index (ce_mode) and in the in-core tree entry (name_entry.mode) and use "unsigned int" instead here.
builtin/cat-file.c struct expand_data { · line 307
Quoted 1 line
builtin/cat-file.c +1
-0+#define EXPAND_DATA_INIT { .mode = S_IFINVALID }
Thanks for knowing about and choosing to use the INVALID thing (I would have naively chosen 0 without looking around enough and made things inconsistent).
builtin/cat-file.c static void expand_atom(struct strbuf *sb, const char *atom, int len, · line 349
Quoted 3 lines
builtin/cat-file.c +3
-0+ }else if (is_atom ("objectmode" , atom, len)) { +if (!data->mark_query && !(S_IFINVALID == data->mode)) +strbuf_addf (sb,"%06o" , data->mode);
Nit. I think
if (!data->mark_query && data->mode != S_IFINVALID)
would be a more common way to write the same thing.
Quoted 10 lines
builtin/cat-file.c +1
-1@@ -766,7 +772,7 @@ static int batch_objects(struct batch_options *opt) {struct strbuf input =STRBUF_INIT ;struct strbuf output =STRBUF_INIT ; -struct expand_data data; +struct expand_data data =EXPAND_DATA_INIT ;int save_warning;int retval =0 ;
builtin/cat-file.c static int batch_objects(struct batch_options *opt) · line 778
Quoted 4 lines
builtin/cat-file.c +0
-1@@ -775,7 +781,6 @@ static int batch_objects(struct batch_options *opt)* object_info to be handed to oid_object_info_extended for each * object. */ -memset (&data,0 ,sizeof (data));
Nice to see this go with the _INIT thing.
Quoted 8 lines
t/t1006-cat-file.sh +0
-0@@ -114,9 +114,10 @@ run_tests () { type=$1 object_name=$2 oid=$(git rev-parse --verify $object_name)
t/t1006-cat-file.sh run_tests () { · line 123
Quoted 10 lines
t/t1006-cat-file.sh +4
-3- size=$3 - content=$4 - pretty_content=$5 + mode=$3 + size=$4 + content=$5 + pretty_content=$6 batch_output="$oid $type $size $content"
I wonder if appending $mode as an optional thing at the end would have made the patch less noisy? After all, the expectation above that does not have $mode, and the tests that are expected to produce output that match the expectation, do not have to change. And the existing invocation of run_tests that do not care about $mode do not have to change.
But I guess if the damage is only with the above 7-lines (which would become just 1 if we made mode the $6 last tthing), it is not a huge deal either way?
Re: [PATCH 2/3] cat-file: add %(objectmode) atom
Quoted 26 lines · Junio C Hamano
Quoted 8 lines
t/t1006-cat-file.sh +0
-0@@ -114,9 +114,10 @@ run_tests () { type=$1 object_name=$2 oid=$(git rev-parse --verify $object_name)t/t1006-cat-file.sh run_tests () { · line 123
Quoted 10 lines
t/t1006-cat-file.sh +4
-3- size=$3 - content=$4 - pretty_content=$5 + mode=$3 + size=$4 + content=$5 + pretty_content=$6 batch_output="$oid $type $size $content"I wonder if appending $mode as an optional thing at the end would have made the patch less noisy? After all, the expectation above that does not have $mode, and the tests that are expected to produce output that match the expectation, do not have to change. And the existing invocation of
run_teststhat do not care about $mode do not have to change.
Quoted 3 lines
But I guess if the damage is only with the above 7-lines (which would become just 1 if we made mode the $6 last tthing), it is not a huge deal either way?
Unfortunately, not really.
If we made the optional mode as the last thing, and allow run_tests() to be called without an explicit "", it may have avoided unnecessary conflicts with eb/hash-transition topic. Interested folks can see how well these three patches plays with the other topic by trying to merge it to 'seen'.
PATCH 3/3 cat-file: avoid verifying submodules' OIDs
In reply to Johannes Schindelin via GitGitGadgetFrom: Johannes Schindelin <[email protected]>
Submodules are strange creatures. They have OIDs, but the corresponding objects are not expected to be present in the current directory.
Let's teach cat-file about this: This command should not even attempt to look up those objects, let alone declare them "missing".
--- Documentation/git-cat-file.txt | 5 +++++ builtin/cat-file.c | 32 ++++++++++++++++++++++++++------ t/t1006-cat-file.sh | 10 ++++++++++ 3 files changed, 41 insertions(+), 6 deletions(-)
Documentation/git-cat-file.txt +5 -0
@@ -412,6 +412,11 @@ Note also that multiple copies of an object may be present in the object database; in this case, it is undefined which copy's size or delta base will be reported. +Submodules are handled specially in `git cat-file`, as the objects +corresponding to the recorded OIDs are not expected to be present in the +current repository. For that reason, submodules are reported as having +type `submodule` and mode 1600000 and all other fields are zeroed out. + GIT --- Part of the linkgit:git[1] suite
builtin/cat-file.c +26 -6
@@ -128,7 +128,9 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name,switch (opt) {case 't' : oi.type_name = &sb; -if (oid_object_info_extended (the_repository, &oid, &oi, flags) <0 ) +if (obj_context.mode ==S_IFGITLINK ) +strbuf_addstr (&sb,"submodule" ); +else if (oid_object_info_extended (the_repository, &oid, &oi, flags) <0 )die ("git cat-file: could not get object info" );if (sb.len) {printf ("%s\n" , sb.buf);
@@ -319,17 +321,26 @@ static void expand_atom(struct strbuf *sb, const char *atom, int len,if (!data->mark_query)strbuf_addstr (sb,oid_to_hex (&data->oid)); }else if (is_atom ("objecttype" , atom, len)) { -if (data->mark_query) +if (data->mode ==S_IFGITLINK ) { +if (!data->mark_query) +strbuf_addstr (sb,"submodule" ); + }else if (data->mark_query) data->info.typep = &data->type;else strbuf_addstr (sb,type_name (data->type)); }else if (is_atom ("objectsize" , atom, len)) { -if (data->mark_query) +if (data->mode ==S_IFGITLINK ) { +if (!data->mark_query) +strbuf_addstr (sb,"0" ); + }else if (data->mark_query) data->info.sizep = &data->size;else strbuf_addf (sb,"%" PRIuMAX , (uintmax_t )data->size); }else if (is_atom ("objectsize:disk" , atom, len)) { -if (data->mark_query) +if (data->mode ==S_IFGITLINK ) { +if (!data->mark_query) +strbuf_addstr (sb,"0" ); + }else if (data->mark_query) data->info.disk_sizep = &data->disk_size;else strbuf_addf (sb,"%" PRIuMAX, (uintmax_t )data->disk_size);
@@ -448,7 +459,8 @@ static void print_default_format(struct strbuf *scratch, struct expand_data *dat struct batch_options *opt) {strbuf_addf (scratch,"%s %s %" PRIuMAX"%c" ,oid_to_hex (&data->oid), -type_name (data->type), + data->mode ==S_IFGITLINK ? +"submodule" :type_name (data->type), (uintmax_t )data->size, opt->output_delim); }
@@ -470,7 +482,15 @@ static void batch_object_write(const char *obj_name,if (use_mailmap) data->info.typep = &data->type; -if (pack) +if (data->mode ==S_IFGITLINK ) { + data->type =OBJ_BAD ;/* `type_name()` does not know submodules */ + data->size =0 ; + data->disk_size =0 ; + data->rest =NULL ; +oidcpy (&data->delta_base_oid,null_oid ()); +memset (&data->info,0 ,sizeof (data->info)); + ret =0 ;/* no info to look up */ + }else if (pack) ret =packed_object_info (the_repository, pack, offset, &data->info);else
t/t1006-cat-file.sh +10 -0
@@ -1178,6 +1178,16 @@ test_expect_success 'cat-file --batch-check respects replace objects' ' test_cmp expect actual ' +test_expect_success 'batch-command with a submodule' ' + printf "160000 commit %0.*d\tsub\n" $(test_oid hexsz) 17 >tree-with-sub && + tree=$(git mktree <tree-with-sub) && + git cat-file --batch-check >actual <<-EOF && + $tree:sub + EOF + printf "%0.*d submodule 0\n" $(test_oid hexsz) 17 >expect && + test_cmp expect actual +' + # Pull the entry for object with oid "$1" out of the output of # "cat-file --batch", including its object content (which requires # parsing and reading a set amount of bytes, hence perl).
Signature · 1 line
gitgitgadget
Re: [PATCH 3/3] cat-file: avoid verifying submodules' OIDs
Documentation/git-cat-file.txt Note also that multiple copies of an object may be present in the object · line 418
Quoted 12 lines · Johannes Schindelin via GitGitGadget
Documentation/git-cat-file.txt +4
-0@@ -412,6 +412,11 @@ Note also that multiple copies of an object may be present in the object database; in this case, it is undefined which copy's size or delta base will be reported. +Submodules are handled specially in `git cat-file`, as the objects +corresponding to the recorded OIDs are not expected to be present in the +current repository. For that reason, submodules are reported as having +type `submodule` and mode 1600000 and all other fields are zeroed out.
I think there's an extra 0 in the mode here?
It may also be worth being more explicit about when Git knows something is a submodule. Naively, reading the above I might think that:
git ls-tree --format='%(objectname)' HEAD | git cat-file --batch-check
would do something special with submodules. But it can't, as there's no context carried in just the objectname. This is obvious if you are familiar with how Git works, but I'm not sure it would be for all end users. So we could say something along the lines of:
When cat-file is given a name within a tree that points to a submodule (e.g., HEAD:my-submodule), ...
-Peff
Re: [PATCH 3/3] cat-file: avoid verifying submodules' OIDs
In reply to Johannes Schindelin via GitGitGadget"Johannes Schindelin via GitGitGadget" <[email protected]>
Documentation/git-cat-file.txt Note also that multiple copies of an object may be present in the object · line 418
Quoted 4 lines · writes:
Documentation/git-cat-file.txt +4
-0+Submodules are handled specially in `git cat-file`, as the objects +corresponding to the recorded OIDs are not expected to be present in the +current repository. For that reason, submodules are reported as having +type `submodule` and mode 1600000 and all other fields are zeroed out.
While the above may not be technically wrong per-se, I am not sure if that is the more important part of what we want to tell our users. For example, "git ls-tree HEAD -- sha1collisiondetection" reports "160000 commit ...object.name.... sha1collisiondetection". Is it correct to say ...
Submodules are handled specially in git ls-tree, as the objects corresponding to the recorded OIDs are not expected to be present in the current repository.
...? I do not think so.
For the same reason, as an explanation for the reason why "git cat-file -t :sha1collisiondetection" just reports "submodule", the new text does not sit well.
I actually have to wonder if the new behaviour proposed by this patch is a solution that is in search of a problem, or trying to solve an unstated problem in a wrong way.
O=$(git rev-parse --verify :sha1collisiondetection) git cat-file -t "$O"
should fail because the object whose name is $O is not available. Why should then this succeed and give a different result?
git cat-file -t :sha1collisiondetection
The "cat-file" command is about objects. While object's type may sometimes be inferrable (by being contained in a tree), if the user asks us to determine the type of the object, we should actually hit the object store, whether the commit object in question happens to be on our history or somebody else's history that our gitlink points at.
So, I am not yet convinced that I should take this patch. Previous two steps looked good, though.
Thanks.
Quoted 12 lines
index
73bd78c0b63..c59ad682d1f100644builtin/cat-file.c +3
-1@@ -128,7 +128,9 @@ static int cat_one_file(int opt, const char *exp_type, const char *obj_name,switch (opt) {case 't' : oi.type_name = &sb; -if (oid_object_info_extended (the_repository, &oid, &oi, flags) <0 ) +if (obj_context.mode ==S_IFGITLINK ) +strbuf_addstr (&sb,"submodule" ); +else if (oid_object_info_extended (the_repository, &oid, &oi, flags) <0 )die ("git cat-file: could not get object info" );
Re: [PATCH 3/3] cat-file: avoid verifying submodules' OIDs
Quoted 19 lines · Junio C Hamano
I actually have to wonder if the new behaviour proposed by this patch is a solution that is in search of a problem, or trying to solve an unstated problem in a wrong way.
O=$(git rev-parse --verify :sha1collisiondetection) git cat-file -t "$O"
should fail because the object whose name is $O is not available. Why should then this succeed and give a different result?
git cat-file -t :sha1collisiondetection
The "cat-file" command is about objects. While object's type may sometimes be inferrable (by being contained in a tree), if the user asks us to determine the type of the object, we should actually hit the object store, whether the commit object in question happens to be on our history or somebody else's history that our gitlink points at.
Quoted 2 lines
So, I am not yet convinced that I should take this patch. Previous two steps looked good, though.
I'm not sure about "-t" in particular, but for batch output, I think if we stop at patch 2 it would be impossible to tell the difference between a submodule entry and a corrupt repo (or bad request). E.g., if I do this:
(echo HEAD:Makefile; echo HEAD:sha1collisiondetection) | git cat-file --batch-check='%(objectname) %(objectmode)'
after only patch 2, I'd get:
4e255c81f22386389c7460d8f5e59426673b5a5a 100644
HEAD:sha1collisiondetection missing
We can't tell if HEAD didn't resolve, or it doesn't have that path, or if it's a regular blob entry and the repository is corrupt. Whereas after patch 3, we get:
4e255c81f22386389c7460d8f5e59426673b5a5a 100644
855827c583bc30645ba427885caa40c5b81764d2 160000
and the mode tells us that we resolved it to a submodule.
The current behavior is not too surprising for cat-file, since it's whole purpose is to give you information about the objects themselves, and we don't have one here. But with this %(objectmode) format, we're really moving into a realm of "resolve this name for me and show me the context". We don't care about the details of the object at all!
I think you could make an argument that the problem is shoe-horning new, slightly-mismatched functionality into cat-file. But there are lots of practical reasons to want to do so, as we discussed elsewhere. Since gitlinks are the only place where we'd expect an object to be missing, "simulating" them here isn't too bad. But I suspect there's a more general solution where cat-file learns to print dummy values for any missing object, letting the caller see what we _could_ find out. And then the submodule case just falls out naturally. I doubt we could make it the default for historical compatibility; we'd need a new option.
This is all speculative on my part, of course. Probably Dscho or Victoria can explain their use case better. :)
-Peff
Re: [PATCH 3/3] cat-file: avoid verifying submodules' OIDs
Quoted 6 lines · Jeff King
I think you could make an argument that the problem is shoe-horning new, slightly-mismatched functionality into cat-file. But there are lots of practical reasons to want to do so, as we discussed elsewhere. Since gitlinks are the only place where we'd expect an object to be missing, "simulating" them here isn't too bad.
100% agreed. This is something we should be asking about "HEAD:" tree object, not about "HEAD:sha1collisiondetection" object, if we are to ask cat-file. After all "cat-file p HEAD:" tells us that the thing is a submodule already. But unfortunately the "--batch" thing is limited to "give me an object and what you want to know about the object, and I'll tell you what I know about it" exchange, so it is a very bad match when you cannot really give it an object (which you do not have, like the target of the gitlink). So...
Quoted 5 lines
But I suspect there's a more
general solution where cat-file learns to print dummy values for any missing object, letting the caller see what we _could_ find out. And then the submodule case just falls out naturally. I doubt we could make it the default for historical compatibility; we'd need a new option.
... "--batch" obviously needs to be extended, and %(objectmode) may be one direction to do so, but it would also work to allow us to ask about "HEAD:" and what it has at paths, which match a pathspec "sha1collisiondetection", an equivalent to give "cat-file --batch" a command to drive "ls-tree".
Quoted 2 lines
This is all speculative on my part, of course. Probably Dscho or Victoria can explain their use case better. :)
Likewise.
"Johannes Schindelin via GitGitGadget" <[email protected]>
Quoted 5 lines · writes:
The cat-file --batch command is very valuable in server settings, but so far it is missing a bit of functionality that would come in handy there.
For example, it is sometimes necessary to determine the object mode of a batch of tree objects' children.
OK.
It is somewhat unsatisfying that --batch/--batch-check lacks so much. Even with %(objectmode) its nature of one-object-at-a-time makes querying children of a large tree a chore, when you compare it with something like "cat-file -p HEAD:" that allows you to grab the needed information for all children with a single invocation.
This is orthogonal to what the patch wants to do, which is to enrich the output side with more formatting, bit I wonder if we want to consider enriching the input side? e.g. instead of feeding just a single object name from the standard input of "cat-file --batch/--batch-check", perhaps a syntax can say "Here I have the object name for a tree-ish object, but please pretend that I gave you all the objects contained within it", or something?
Thanks, will queue.
Quoted 6 lines · Junio C Hamano
It is somewhat unsatisfying that --batch/--batch-check lacks so much. Even with %(objectmode) its nature of one-object-at-a-time makes querying children of a large tree a chore, when you compare it with something like "cat-file -p HEAD:" that allows you to grab the needed information for all children with a single invocation.
Quoted 7 lines
This is orthogonal to what the patch wants to do, which is to enrich the output side with more formatting, bit I wonder if we want to consider enriching the input side? e.g. instead of feeding just a single object name from the standard input of "cat-file --batch/--batch-check", perhaps a syntax can say "Here I have the object name for a tree-ish object, but please pretend that I gave you all the objects contained within it", or something?
That is an interesting direction. In practice I guess you might want to expand trees (to show their contents) or perhaps commits (to traverse history and/or look at their trees). And we already have tools to do that.
So for example you can already do:
git ls-tree --format='%(objectname) %(objectmode)' HEAD
Or if you wanted to mix-and-match with other cat-file placeholders, you can do:
git ls-tree --format='%(objectname) %(objectmode)' HEAD | git cat-file --batch-check='%(objectname) %(deltabase) %(rest)'
That is a little less efficient (we look up the object twice), but once you are working with hex object ids it is not too bad (cat-file is heavily optimized here). Of course in the long run I think we should move to a future where the formatting code is shared, and you can just ask ls-tree for deltabase if you want to.
I think leaving this to specialized tools like ls-tree gives them a lot of flexibility that a special input mode to cat-file might find awkward. For example, recurse vs non-recursive tree listing. Or filtering with pathspecs. And of course when you get into commits and traversal, there are many rev-list options. :)
The strategy so far has been making sure cat-file can efficiently take in the output of these other tools to further describe objects. But moving towards a unified output formatting model would be even better, I think. In the meantime, I think cat-file learning %(objectmode) makes sense for single names (rather than listing trees), and fortunately it uses the same (obvious) name that ls-tree does, so we won't have a problem unifying them later.
The patch itself looked reasonable to me, modulo the comments you already made.
-Peff
Quoted 12 lines · Jeff King
That is an interesting direction. In practice I guess you might want to expand trees (to show their contents) or perhaps commits (to traverse history and/or look at their trees). And we already have tools to do that.
So for example you can already do:
git ls-tree --format='%(objectname) %(objectmode)' HEADOr if you wanted to mix-and-match with other cat-file placeholders, you can do:
Quoted 8 lines
git ls-tree --format='%(objectname) %(objectmode)' HEAD | git cat-file --batch-check='%(objectname) %(deltabase) %(rest)'That is a little less efficient (we look up the object twice), but once you are working with hex object ids it is not too bad (cat-file is heavily optimized here). Of course in the long run I think we should move to a future where the formatting code is shared, and you can just ask ls-tree for deltabase if you want to.
I was imagining more about a use case "cat-file --batch" was originally designed for---having a long-running single process and ask any and all questions you have about various objects in the object database by interacting with it. So "yes, ls-tree can already give us that information", while it is true, shoots at a different direction from what I had in mind.
Quoted 7 lines
The strategy so far has been making sure cat-file can efficiently take in the output of these other tools to further describe objects. But moving towards a unified output formatting model would be even better, I think. In the meantime, I think cat-file learning %(objectmode) makes sense for single names (rather than listing trees), and fortunately it uses the same (obvious) name that ls-tree does, so we won't have a problem unifying them later.
Yes, enriching the output format side is an orthogonal issue from the input side, and the %(objectmode) thing that gives a piece of information that is additionally available on top of the various pieces of information about the object itself does make sense.
Quoted 4 lines
The patch itself looked reasonable to me, modulo the comments you already made.
-Peff
Quoted 12 lines · Junio C Hamano
Quoted 11 lines
Or if you wanted to mix-and-match with other cat-file placeholders, you can do:
git ls-tree --format='%(objectname) %(objectmode)' HEAD | git cat-file --batch-check='%(objectname) %(deltabase) %(rest)'That is a little less efficient (we look up the object twice), but once you are working with hex object ids it is not too bad (cat-file is heavily optimized here). Of course in the long run I think we should move to a future where the formatting code is shared, and you can just ask ls-tree for deltabase if you want to.
Quoted 6 lines
I was imagining more about a use case "cat-file --batch" was originally designed for---having a long-running single process and ask any and all questions you have about various objects in the object database by interacting with it. So "yes, ls-tree can already give us that information", while it is true, shoots at a different direction from what I had in mind.
Ah, yeah, that is one thing that cat-file does that no other part of the system will. I do wonder in the long term if it is easier to teach cat-file everything that all of the other commands can do, or to teach all of the other commands some way of handling multiple requests in a single process. ;)
(All obviously orthogonal to this patch series).
-Peff