fetch: introduce machine-parseable output
cover fetch: introduce machine-parseable output
Hi,
this is the second part of my quest to introduce a machine-parseable output for git-fetch(1) after the initial refactorings that have been merged via e9dffbc7f1 (Merge branch 'ps/fetch-ref-update-reporting', 2023-04-06).
Parsing the output of fetches is mostly impossible. It prettifies reference names that are about to be updated, doesn't print the old and new object IDs the refs are being updated from and to, and prints all of that information in nice columns. In short, it is designed to be read by humans rather than machines.
This makes it hard to use in a script way though, e.g. to learn about which references actually have been updated or which have not been updated. This patch series intends to fix that by introducing a new machine-parseable interface:
``` $ git fetch --output-format=porcelain --no-progress fff5a5e7f528b2ed2c335991399a766c2cf01103 af67688dca57999fd848f051eeea1d375ba546b2 refs/remotes/origin/master * 0000000000000000000000000000000000000000 e046fe5a36a970bc14fbfbcb2074a48776f6b671 refs/remotes/origin/x86-rep-insns * 0000000000000000000000000000000000000000 bb81ed6862b864c9eb99447f04d49a84ecb647e5 refs/tags/v6.3-rc4 * 0000000000000000000000000000000000000000 83af7b1468c0dca86b4dc9e43e73bfa4f38d9637 refs/tags/v6.3-rc5 * 0000000000000000000000000000000000000000 ab3affb8ed84f68638162fe7e6fd4055e15bff5b refs/tags/v6.3-rc6 * 0000000000000000000000000000000000000000 1c8c28415e8743368a2b800520a6dd0b22ee6ec2 refs/tags/v6.3-rc7 ```
The series is structured as following:
- Patches 1 and 2 improve test coverage for output formats.
- Patch 3 fixes a bug with the current output format.
- Patch 4 to 6 perform some preliminary refactorings.
- Patch 7 introduces a new --output-format= option for git-fetch(1) that allows the user to configure the output more directly.
- Patch 8 introduces the new "porcelain" output format.
Patrick
Patrick Steinhardt (8):
fetch: split out tests for output format
fetch: add a test to exercise invalid output formats
fetch: fix missing from-reference when fetching HEAD:foo
fetch: introduce display_format enum
fetch: move display format parsing into main function
fetch: move option related variables into main function
fetch: introduce new --output-format option
fetch: introduce machine-parseable "porcelain" output format
Documentation/config/fetch.txt | 4 +- Documentation/fetch-options.txt | 5 + Documentation/git-fetch.txt | 17 +- builtin/fetch.c | 406 +++++++++++++++++++------------- t/t5510-fetch.sh | 53 ----- t/t5574-fetch-output.sh | 209 ++++++++++++++++ 6 files changed, 475 insertions(+), 219 deletions(-) create mode 100755 t/t5574-fetch-output.sh
Signature · 1 line
2.40.0
PATCH 1/8 fetch: split out tests for output format
We're about to introduce a new porcelain mode for the output of git-fetch(1). As part of that we'll be introducing a set of new tests that only relate to the output of this command.
Split out tests that exercise the output format of git-fetch(1) so that it becomes easier to verify this functionality as a standalone unit. As the tests assume that the default branch is called "main" we set up the corresponding GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME environment variable accordingly.
--- t/t5510-fetch.sh | 53 ---------------------------------- t/t5574-fetch-output.sh | 63 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+), 53 deletions(-) create mode 100755 t/t5574-fetch-output.sh
t/t5510-fetch.sh +0 -53
@@ -1118,59 +1118,6 @@ test_expect_success 'fetching with auto-gc does not lock up' ' ) ' -test_expect_success 'fetch aligned output' ' - git clone . full-output && - test_commit looooooooooooong-tag && - ( - cd full-output && - git -c fetch.output=full fetch origin >actual 2>&1 && - grep -e "->" actual | cut -c 22- >../actual - ) && - cat >expect <<-\EOF && - main -> origin/main - looooooooooooong-tag -> looooooooooooong-tag - EOF - test_cmp expect actual -' - -test_expect_success 'fetch compact output' ' - git clone . compact && - test_commit extraaa && - ( - cd compact && - git -c fetch.output=compact fetch origin >actual 2>&1 && - grep -e "->" actual | cut -c 22- >../actual - ) && - cat >expect <<-\EOF && - main -> origin/* - extraaa -> * - EOF - test_cmp expect actual -' - -test_expect_success '--no-show-forced-updates' ' - mkdir forced-updates && - ( - cd forced-updates && - git init && - test_commit 1 && - test_commit 2 - ) && - git clone forced-updates forced-update-clone && - git clone forced-updates no-forced-update-clone && - git -C forced-updates reset --hard HEAD~1 && - ( - cd forced-update-clone && - git fetch --show-forced-updates origin 2>output && - test_i18ngrep "(forced update)" output - ) && - ( - cd no-forced-update-clone && - git fetch --no-show-forced-updates origin 2>output && - test_i18ngrep ! "(forced update)" output - ) -' - for section in fetch transfer do test_expect_success "$section.hideRefs affects connectivity check" '
t/t5574-fetch-output.sh new file +63 -0
63 lines not on this page · raw message
Signature · 1 line
2.40.0
PATCH 2/8 fetch: add a test to exercise invalid output formats
In reply to Patrick SteinhardtAdd a testcase that exercises the logic when an invalid output format is passed via the fetch.output configuration.
--- t/t5574-fetch-output.sh | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
t/t5574-fetch-output.sh +17 -0
@@ -7,6 +7,23 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME . ./test-lib.sh +test_expect_success 'fetch with invalid output format configuration' ' + test_when_finished "rm -rf clone" && + git clone . clone && + + test_must_fail git -C clone -c fetch.output= fetch origin >actual 2>&1 && + cat >expect <<-EOF && + fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}${SQ} + EOF + test_cmp expect actual && + + test_must_fail git -C clone -c fetch.output=garbage fetch origin >actual 2>&1 && + cat >expect <<-EOF && + fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}garbage${SQ} + EOF + test_cmp expect actual +' + test_expect_success 'fetch aligned output' ' git clone . full-output && test_commit looooooooooooong-tag &&
Signature · 1 line
2.40.0
PATCH 3/8 fetch: fix missing from-reference when fetching HEAD:foo
In reply to Patrick SteinhardtWhen displaying reference updates, we print a line that looks similar to the following:
``` * branch master -> master ```
The "branch" bit changes depending on what kind of reference we're updating, while both of the right-hand references are computed by stripping well-known prefixes like "refs/heads/" or "refs/tags".
The logic is kind of intertwined though and not easy to follow: we precompute both the kind (e.g. "branch") and the what, which is the abbreviated remote reference name, in store_updated_refs() and then pass it down the call chain to display_ref_update().
There is a set of different cases here:
- When the remote reference name is "HEAD" we assume no kind and will thus instead print "[new ref]". We keep what at the empty string.
- When the remote reference name has a well-known prefix then the kind would be "branch", "tag" or "remote-tracking branch". The what is the reference with the well-known prefix stripped and in fact matches the output that prettify_refname() would return.
- Otherwise, we'll again assume no kind and keep the what set to the fully qualified reference name.
Now there is a bug with the first case here, where the remote reference name is "HEAD". As noted, "what" will be set to the empty string. And that seems to be intentional because we also use this information to update the FETCH_HEAD, and in case we're updating HEAD we seemingly don't want to append that to our FETCH_HEAD value.
But as mentioned, we also use this value to display reference updates. And while the call to display_ref_update() correctly figures out that we meant "HEAD" when what is empty, the call to update_local_ref() doesn't. update_local_ref() will then call display_ref_update() with the empty string and cause the following broken output:
``` $ git fetch --dry-run origin HEAD:foo From https://github.com/git/git * [new ref] -> foo ```
The HEAD string is clearly missing from the left-hand side of the arrow, which is further stressed by the point that the following commands work as expected:
``` $ git fetch --dry-run origin HEAD From https://github.com/git/git * branch HEAD -> FETCH_HEAD $ git fetch --dry-run origin master From https://github.com/git/git * branch master -> FETCH_HEAD * branch master -> origin/master ```
Fix this bug by instead unconditionally passing the full reference name to display_ref_update() which learns to call prettify_refname() on it. This does fix the above bug and is otherwise functionally the same as prettify_refname() would only ever strip the well-known prefixes just as intended. So at the same time, this also simplifies the code a bit.
Note that this patch also changes formatting of the block that computes the "kind" and "what" variables. This is done on purpose so that it is part of the diff, hopefully making the change easier to comprehend.
--- builtin/fetch.c | 37 +++++++++++++++++++------------------ t/t5574-fetch-output.sh | 19 +++++++++++++++++++ 2 files changed, 38 insertions(+), 18 deletions(-)
builtin/fetch.c +19 -18
@@ -918,12 +918,14 @@ static void display_ref_update(struct display_state *display_state, char code, } width = (summary_width +strlen (summary) -gettext_width (summary)); + remote =prettify_refname (remote); + local =prettify_refname (local);strbuf_addf (&display_state->buf," %c %-*s " , code, width, summary);if (!display_state->compact_format) -print_remote_to_local (display_state, remote,prettify_refname (local)); +print_remote_to_local (display_state, remote, local);else -print_compact (display_state, remote,prettify_refname (local)); +print_compact (display_state, remote, local);if (error)strbuf_addf (&display_state->buf," (%s)" , error);strbuf_addch (&display_state->buf,'\n' );
@@ -934,7 +936,7 @@ static void display_ref_update(struct display_state *display_state, char code,static int update_local_ref (struct ref *ref,struct ref_transaction *transaction,struct display_state *display_state, -const char *remote,const struct ref *remote_ref, +const struct ref *remote_ref,int summary_width) {struct commit *current =NULL , *updated;
@@ -946,7 +948,7 @@ static int update_local_ref(struct ref *ref,if (oideq (&ref->old_oid, &ref->new_oid)) {if (verbosity >0 )display_ref_update (display_state,'=' ,_ ("[up to date]" ),NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 0 ; }
@@ -959,7 +961,7 @@ static int update_local_ref(struct ref *ref,*/ display_ref_update (display_state,'!' ,_ ("[rejected]" ),_ ("can't fetch into checked-out branch" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 1 ; }
@@ -970,12 +972,12 @@ static int update_local_ref(struct ref *ref, r =s_update_ref ("updating tag" , ref, transaction,0 );display_ref_update (display_state, r ?'!' :'t' ,_ ("[tag update]" ), r ?_ ("unable to update local ref" ) :NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return r; }else {display_ref_update (display_state,'!' ,_ ("[rejected]" ),_ ("would clobber existing tag" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 1 ; } }
@@ -1008,7 +1010,7 @@ static int update_local_ref(struct ref *ref, r =s_update_ref (msg, ref, transaction,0 );display_ref_update (display_state, r ?'!' :'*' , what, r ?_ ("unable to update local ref" ) :NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return r; }
@@ -1030,7 +1032,7 @@ static int update_local_ref(struct ref *ref, r =s_update_ref ("fast-forward" , ref, transaction,1 );display_ref_update (display_state, r ?'!' :' ' , quickref.buf, r ?_ ("unable to update local ref" ) :NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);strbuf_release (&quickref);return r; }else if (force || ref->force) {
@@ -1042,12 +1044,12 @@ static int update_local_ref(struct ref *ref, r =s_update_ref ("forced-update" , ref, transaction,1 );display_ref_update (display_state, r ?'!' :'+' , quickref.buf, r ?_ ("unable to update local ref" ) :_ ("forced update" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);strbuf_release (&quickref);return r; }else {display_ref_update (display_state,'!' ,_ ("[rejected]" ),_ ("non-fast-forward" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 1 ; } }
@@ -1252,14 +1254,13 @@ static int store_updated_refs(struct display_state *display_state,if (!strcmp (rm->name,"HEAD" )) { kind ="" ; what ="" ; - } -else if (skip_prefix (rm->name,"refs/heads/" , &what)) + }else if (skip_prefix (rm->name,"refs/heads/" , &what)) { kind ="branch" ; -else if (skip_prefix (rm->name,"refs/tags/" , &what)) + }else if (skip_prefix (rm->name,"refs/tags/" , &what)) { kind ="tag" ; -else if (skip_prefix (rm->name,"refs/remotes/" , &what)) + }else if (skip_prefix (rm->name,"refs/remotes/" , &what)) { kind ="remote-tracking branch" ; -else { + }else { kind ="" ; what = rm->name; }
@@ -1277,7 +1278,7 @@ static int store_updated_refs(struct display_state *display_state, display_state->url_len);if (ref) { - rc |=update_local_ref (ref, transaction, display_state, what, + rc |=update_local_ref (ref, transaction, display_state, rm, summary_width);free (ref); }else if (write_fetch_head || dry_run) {
@@ -1288,7 +1289,7 @@ static int store_updated_refs(struct display_state *display_state,*/ display_ref_update (display_state,'*' , *kind ? kind :"branch" ,NULL , - *what ? what :"HEAD" , + rm->name,"FETCH_HEAD" , summary_width); } }
t/t5574-fetch-output.sh +19 -0
@@ -54,6 +54,25 @@ test_expect_success 'fetch compact output' ' test_cmp expect actual ' +test_expect_success 'fetch output with HEAD and --dry-run' ' + test_when_finished "rm -rf head" && + git clone . head && + + git -C head fetch --dry-run origin HEAD >actual 2>&1 && + cat >expect <<-EOF && + From $(test-tool path-utils real_path .)/. + * branch HEAD -> FETCH_HEAD + EOF + test_cmp expect actual && + + git -C head fetch --dry-run origin HEAD:foo >actual 2>&1 && + cat >expect <<-EOF && + From $(test-tool path-utils real_path .)/. + * [new ref] HEAD -> foo + EOF + test_cmp expect actual +' + test_expect_success '--no-show-forced-updates' ' mkdir forced-updates && (
Signature · 1 line
2.40.0
Re: [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo
Quoted 70 lines · Patrick Steinhardt
When displaying reference updates, we print a line that looks similar to the following:
``` * branch master -> master ```The "branch" bit changes depending on what kind of reference we're updating, while both of the right-hand references are computed by stripping well-known prefixes like "refs/heads/" or "refs/tags".
The logic is kind of intertwined though and not easy to follow: we precompute both the kind (e.g. "branch") and the what, which is the abbreviated remote reference name, in
store_updated_refs()and then pass it down the call chain todisplay_ref_update().There is a set of different cases here:
- When the remote reference name is "HEAD" we assume no kind and will thus instead print "[new ref]". We keep what at the empty string.
- When the remote reference name has a well-known prefix then the kind would be "branch", "tag" or "remote-tracking branch". The what is the reference with the well-known prefix stripped and in fact matches the output that
prettify_refname()would return.- Otherwise, we'll again assume no kind and keep the what set to the fully qualified reference name.
Now there is a bug with the first case here, where the remote reference name is "HEAD". As noted, "what" will be set to the empty string. And that seems to be intentional because we also use this information to update the
FETCH_HEAD, and in case we're updating HEAD we seemingly don't want to append that to ourFETCH_HEADvalue.But as mentioned, we also use this value to display reference updates. And while the call to
display_ref_update()correctly figures out that we meant "HEAD" whenwhatis empty, the call toupdate_local_ref()doesn't.update_local_ref()will then calldisplay_ref_update()with the empty string and cause the following broken output:``` $ git fetch --dry-run origin HEAD:foo From https://github.com/git/git * [new ref] -> foo ```The HEAD string is clearly missing from the left-hand side of the arrow, which is further stressed by the point that the following commands work as expected:
``` $ git fetch --dry-run origin HEAD From https://github.com/git/git * branch HEAD -> FETCH_HEAD $ git fetch --dry-run origin master From https://github.com/git/git * branch master -> FETCH_HEAD * branch master -> origin/master ```Fix this bug by instead unconditionally passing the full reference name to
display_ref_update()which learns to callprettify_refname()on it. This does fix the above bug and is otherwise functionally the same asprettify_refname()would only ever strip the well-known prefixes just as intended. So at the same time, this also simplifies the code a bit.
Quoted 5 lines
Note that this patch also changes formatting of the block that computes the "kind" and "what" variables. This is done on purpose so that it is part of the diff, hopefully making the change easier to comprehend.
Signed-off-by Patrick Steinhardt <[email protected]>
The commit message here has a lot of context, but I found it a bit hard to parse through, especially relative to the actual fix in code.
One suggestion was to load the paragraphs a bit more with the actual problem being solved first, before beginning a lot of the context.
We also discussed the block of format changes and felt a bit mixed on whether to include it or not. It does match the coding style guidelines, but there is no actual functional change made to those lines in this series.
I think its a good improvement, and it does force some extra context into the diff which makes reading the resulting change easier.
Re: [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo
Quoted 1 line · Jacob Keller
On 4/19/2023 5:31 AM, Patrick Steinhardt wrote:
[snip]
Quoted 6 lines
The commit message here has a lot of context, but I found it a bit hard to parse through, especially relative to the actual fix in code.
One suggestion was to load the paragraphs a bit more with the actual problem being solved first, before beginning a lot of the context.
Quoted 6 lines
We also discussed the block of format changes and felt a bit mixed on whether to include it or not. It does match the coding style guidelines, but there is no actual functional change made to those lines in this series.
I think its a good improvement, and it does force some extra context into the diff which makes reading the resulting change easier.
I initially really struggled to explain to myself why my change actually works, and that shows in the commit message. I've reworded it a bit and took some of the suggested changes that Glen posted to hopefully make this clearer.
Patrick
Re: [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo
In reply to Patrick SteinhardtQuoted 10 lines · Patrick Steinhardt
t/t5574-fetch-output.sh +9
-0+test_expect_success 'fetch output with HEAD and --dry-run' ' + test_when_finished "rm -rf head" && + git clone . head && + + git -C head fetch --dry-run origin HEAD >actual 2>&1 && + cat >expect <<-EOF && + From $(test-tool path-utils real_path .)/. + * branch HEAD -> FETCH_HEAD + EOF
t/t5574-fetch-output.sh test_expect_success 'fetch compact output' ' · line 75
Quoted 10 lines
t/t5574-fetch-output.sh +10
-0+ test_cmp expect actual && + + git -C head fetch --dry-run origin HEAD:foo >actual 2>&1 && + cat >expect <<-EOF && + From $(test-tool path-utils real_path .)/. + * [new ref] HEAD -> foo + EOF + test_cmp expect actual +' +
The test mentions HEAD and --dry-run, but the bug seems to exist regardless of whether --dry-run is used. I understand the use of --dry-run for testing fetch output so that you can repeatably run git fetch and get the same results.
The tests here should probably also have a test that covers fetch without --dry-run though.
Thanks,
Jake
Re: [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo
Quoted 29 lines · Jacob Keller
Quoted 10 lines · Patrick Steinhardt
t/t5574-fetch-output.sh +9
-0+test_expect_success 'fetch output with HEAD and --dry-run' ' + test_when_finished "rm -rf head" && + git clone . head && + + git -C head fetch --dry-run origin HEAD >actual 2>&1 && + cat >expect <<-EOF && + From $(test-tool path-utils real_path .)/. + * branch HEAD -> FETCH_HEAD + EOFt/t5574-fetch-output.sh test_expect_success 'fetch compact output' ' · line 75
Quoted 10 lines
t/t5574-fetch-output.sh +10
-0+ test_cmp expect actual && + + git -C head fetch --dry-run origin HEAD:foo >actual 2>&1 && + cat >expect <<-EOF && + From $(test-tool path-utils real_path .)/. + * [new ref] HEAD -> foo + EOF + test_cmp expect actual +' +The test mentions HEAD and --dry-run, but the bug seems to exist regardless of whether --dry-run is used. I understand the use of --dry-run for testing fetch output so that you can repeatably run git fetch and get the same results.
Quoted 2 lines
The tests here should probably also have a test that covers fetch without --dry-run though.
Makes sense, will amend!
Patrick
Re: [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo
In reply to Patrick SteinhardtRearranging the lines slightly,
Quoted 24 lines · Patrick Steinhardt
When displaying reference updates, we print a line that looks similar to the following:
``` * branch master -> master ```The "branch" bit changes depending on what kind of reference we're updating, while both of the right-hand references are computed by stripping well-known prefixes like "refs/heads/" or "refs/tags".
[...]
we also use this value to display reference updates.
And while the call todisplay_ref_update()correctly figures out that we meant "HEAD" whenwhatis empty, the call toupdate_local_ref()doesn't.update_local_ref()will then calldisplay_ref_update()with the empty string and cause the following broken output:``` $ git fetch --dry-run origin HEAD:foo From https://github.com/git/git * [new ref] -> foo ```
Quoted 8 lines
[...]
Fix this bug by instead unconditionally passing the full reference name to
display_ref_update()which learns to callprettify_refname()on it. This does fix the above bug and is otherwise functionally the same asprettify_refname()would only ever strip the well-known prefixes just as intended. So at the same time, this also simplifies the code a bit.
The bug fix is obviously good. I'm surprised we hadn't caught this sooner.
As a nitpicky comment, the commit message goes into a lot of detail, which makes it tricky to read on its own (though the level of detail makes it easy to match to the diff, making the diff quite easy to follow). I would have found this easier to read by summarizing the high-level mental model before diving into the background, e.g.
store_updated_refs() parses the remote ref name to create a 'note' to write to FETCH_HEAD. This note is usually the prettified ref name, so it is used to diplay ref updates (display_ref_update()). But if the remote ref is HEAD, the note is the empty string [insert bug description]. Instead, use the note only as a note and have display_ref_update() prettify the ref name itself...
Quoted 11 lines
builtin/fetch.c +2
-0@@ -918,12 +918,14 @@ static void display_ref_update(struct display_state *display_state, char code, } width = (summary_width +strlen (summary) -gettext_width (summary)); + remote =prettify_refname (remote); + local =prettify_refname (local);
builtin/fetch.c static void display_ref_update(struct display_state *display_state, char code, · line 931
Quoted 10 lines
builtin/fetch.c +2
-2strbuf_addf (&display_state->buf," %c %-*s " , code, width, summary);if (!display_state->compact_format) -print_remote_to_local (display_state, remote,prettify_refname (local)); +print_remote_to_local (display_state, remote, local);else -print_compact (display_state, remote,prettify_refname (local)); +print_compact (display_state, remote, local);if (error)strbuf_addf (&display_state->buf," (%s)" , error);strbuf_addch (&display_state->buf,'\n' );
As expected, we now prettify the name isntead of trusting the 'note' that came in the parameter...
Quoted 75 lines
builtin/fetch.c +9
-9@@ -934,7 +936,7 @@ static void display_ref_update(struct display_state *display_state, char code,static int update_local_ref (struct ref *ref,struct ref_transaction *transaction,struct display_state *display_state, -const char *remote,const struct ref *remote_ref, +const struct ref *remote_ref,int summary_width) {struct commit *current =NULL , *updated;@@ -946,7 +948,7 @@ static int update_local_ref(struct ref *ref,if (oideq (&ref->old_oid, &ref->new_oid)) {if (verbosity >0 )display_ref_update (display_state,'=' ,_ ("[up to date]" ),NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 0 ; }@@ -959,7 +961,7 @@ static int update_local_ref(struct ref *ref,*/ display_ref_update (display_state,'!' ,_ ("[rejected]" ),_ ("can't fetch into checked-out branch" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 1 ; }@@ -970,12 +972,12 @@ static int update_local_ref(struct ref *ref, r =s_update_ref ("updating tag" , ref, transaction,0 );display_ref_update (display_state, r ?'!' :'t' ,_ ("[tag update]" ), r ?_ ("unable to update local ref" ) :NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return r; }else {display_ref_update (display_state,'!' ,_ ("[rejected]" ),_ ("would clobber existing tag" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 1 ; } }@@ -1008,7 +1010,7 @@ static int update_local_ref(struct ref *ref, r =s_update_ref (msg, ref, transaction,0 );display_ref_update (display_state, r ?'!' :'*' , what, r ?_ ("unable to update local ref" ) :NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return r; }@@ -1030,7 +1032,7 @@ static int update_local_ref(struct ref *ref, r =s_update_ref ("fast-forward" , ref, transaction,1 );display_ref_update (display_state, r ?'!' :' ' , quickref.buf, r ?_ ("unable to update local ref" ) :NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);strbuf_release (&quickref);return r; }else if (force || ref->force) {@@ -1042,12 +1044,12 @@ static int update_local_ref(struct ref *ref, r =s_update_ref ("forced-update" , ref, transaction,1 );display_ref_update (display_state, r ?'!' :'+' , quickref.buf, r ?_ ("unable to update local ref" ) :_ ("forced update" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);strbuf_release (&quickref);return r; }else {display_ref_update (display_state,'!' ,_ ("[rejected]" ),_ ("non-fast-forward" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 1 ; } }
...
Quoted 10 lines
builtin/fetch.c +1
-1@@ -1277,7 +1278,7 @@ static int store_updated_refs(struct display_state *display_state, display_state->url_len);if (ref) { - rc |=update_local_ref (ref, transaction, display_state, what, + rc |=update_local_ref (ref, transaction, display_state, rm, summary_width);free (ref); }else if (write_fetch_head || dry_run) {
builtin/fetch.c static int store_updated_refs(struct display_state *display_state, · line 1295
Quoted 8 lines
builtin/fetch.c +1
-1@@ -1288,7 +1289,7 @@ static int store_updated_refs(struct display_state *display_state,*/ display_ref_update (display_state,'*' , *kind ? kind :"branch" ,NULL , - *what ? what :"HEAD" , + rm->name,"FETCH_HEAD" , summary_width); } }
and we stop passing the 'note' as a parameter. Looks good.
Quoted 9 lines
builtin/fetch.c +1
-3@@ -1252,14 +1254,13 @@ static int store_updated_refs(struct display_state *display_state,if (!strcmp (rm->name,"HEAD" )) { kind ="" ; what ="" ; - } -else if (skip_prefix (rm->name,"refs/heads/" , &what)) + }else if (skip_prefix (rm->name,"refs/heads/" , &what)) { kind ="branch" ; -else if (skip_prefix (rm->name,"refs/tags/" , &what))
builtin/fetch.c static int store_updated_refs(struct display_state *display_state, · line 1266
Quoted 10 lines
builtin/fetch.c +3
-2+ }else if (skip_prefix (rm->name,"refs/tags/" , &what)) { kind ="tag" ; - else if (skip_prefix(rm->name, "refs/remotes/", &what)) + }else if (skip_prefix (rm->name,"refs/remotes/" , &what)) { kind ="remote-tracking branch" ; - else { + }else { kind ="" ; what = rm->name; }
I really appreciate that this makes the patch easier to read. I don't really appreciate this sort of churn, but it _is_ following CodingGuidelines:
- When there are multiple arms to a conditional and some of them
require braces, enclose even a single line block in braces for
consistency. E.g.:
if (foo) {
doit();
} else {
one();
two();
three();
}(I initialy thought it wasn't. Thanks to other Review Club participants for pointing this out).
t/t5574-fetch-output.sh test_expect_success 'fetch compact output' ' · line 57
Quoted 9 lines
t/t5574-fetch-output.sh +1
-0@@ -54,6 +54,25 @@ test_expect_success 'fetch compact output' ' test_cmp expect actual ' +test_expect_success 'fetch output with HEAD and --dry-run' '
The commit message and diff didn't imply that this is a --dry-run only bug. I tested locally, and it seems to reproduce without --dry-run too, so I think we should drop "--dry-run" from this test name. In a later patch, you also add a test for porcelain output with --dry-run, but since this test seems designed for just this bug, I think we can drop the later test.
Re: [PATCH 3/8] fetch: fix missing from-reference when fetching HEAD:foo
Quoted 48 lines · Glen Choo
Rearranging the lines slightly,
Quoted 24 lines · Patrick Steinhardt
When displaying reference updates, we print a line that looks similar to the following:
``` * branch master -> master ```The "branch" bit changes depending on what kind of reference we're updating, while both of the right-hand references are computed by stripping well-known prefixes like "refs/heads/" or "refs/tags".
[...]
we also use this value to display reference updates.
And while the call todisplay_ref_update()correctly figures out that we meant "HEAD" whenwhatis empty, the call toupdate_local_ref()doesn't.update_local_ref()will then calldisplay_ref_update()with the empty string and cause the following broken output:``` $ git fetch --dry-run origin HEAD:foo From https://github.com/git/git * [new ref] -> foo ```Quoted 8 lines
[...]
Fix this bug by instead unconditionally passing the full reference name to
display_ref_update()which learns to callprettify_refname()on it. This does fix the above bug and is otherwise functionally the same asprettify_refname()would only ever strip the well-known prefixes just as intended. So at the same time, this also simplifies the code a bit.The bug fix is obviously good. I'm surprised we hadn't caught this sooner.
As a nitpicky comment, the commit message goes into a lot of detail, which makes it tricky to read on its own (though the level of detail makes it easy to match to the diff, making the diff quite easy to follow). I would have found this easier to read by summarizing the high-level mental model before diving into the background, e.g.
Quoted 6 lines
store_updated_refs()parses the remote ref name to create a 'note' to write toFETCH_HEAD. This note is usually the prettified ref name, so it is used to diplay ref updates (display_ref_update()). But if the remote ref is HEAD, the note is the empty string [insert bug description]. Instead, use the note only as a note and havedisplay_ref_update()prettify the ref name itself...
I like that and will use a variant of this.
[snip]
Quoted 8 lines
Quoted 8 lines
t/t5574-fetch-output.sh +0
-0@@ -54,6 +54,25 @@ test_expect_success 'fetch compact output' ' test_cmp expect actual '
Quoted 8 lines
t/t5574-fetch-output.sh test_expect_success 'fetch compact output' ' · line 57
Quoted 1 line
t/t5574-fetch-output.sh +1
-0+test_expect_success 'fetch output with HEAD and --dry-run' 'The commit message and diff didn't imply that this is a --dry-run only bug. I tested locally, and it seems to reproduce without --dry-run too, so I think we should drop "--dry-run" from this test name. In a later patch, you also add a test for porcelain output with --dry-run, but since this test seems designed for just this bug, I think we can drop the later test.
True, I'll amend the test.
Patrick
PATCH 4/8 fetch: introduce `display_format` enum
In reply to Patrick SteinhardtWe currently have two different display formats in git-fetch(1) with the "full" and "compact" formats. This is tracked with a boolean value that simply denotes whether the display format is supposed to be compacted or not. This works reasonably well while there are only two formats, but we're about to introduce another format that will make this a bit more awkward to use.
Introduce a enum display_format that is more readily extensible.
--- builtin/fetch.c | 101 ++++++++++++++++++++++++++++++------------------ 1 file changed, 64 insertions(+), 37 deletions(-)
builtin/fetch.c +64 -37
@@ -48,11 +48,17 @@ enum {TAGS_SET =2 }; +enum display_format { +DISPLAY_FORMAT_UNKNOWN =0 , +DISPLAY_FORMAT_FULL , +DISPLAY_FORMAT_COMPACT , +}; +struct display_state {struct strbuf buf;int refcol_width; -int compact_format; +enum display_format format;char *url;int url_len, shown_url;
@@ -784,7 +790,6 @@ static int refcol_width(const struct ref *ref, int compact_format)static void display_state_init (struct display_state *display_state,struct ref *ref_map,const char *raw_url) { -struct ref *rm;const char *format ="full" ;int i;
@@ -809,31 +814,42 @@ static void display_state_init(struct display_state *display_state, struct ref *git_config_get_string_tmp ("fetch.output" , &format);if (!strcasecmp (format,"full" )) - display_state->compact_format =0 ; + display_state->format =DISPLAY_FORMAT_FULL ;else if (!strcasecmp (format,"compact" )) - display_state->compact_format =1 ; + display_state->format =DISPLAY_FORMAT_COMPACT ;else die (_ ("invalid value for '%s': '%s'" ),"fetch.output" , format); - display_state->refcol_width =10 ; -for (rm = ref_map; rm; rm = rm->next) { -int width; +switch (display_state->format) { +case DISPLAY_FORMAT_FULL : +case DISPLAY_FORMAT_COMPACT : { +struct ref *rm; -if (rm->status ==REF_STATUS_REJECT_SHALLOW || - !rm->peer_ref || - !strcmp (rm->name,"HEAD" )) -continue ; + display_state->refcol_width =10 ; +for (rm = ref_map; rm; rm = rm->next) { +int width; - width =refcol_width (rm, display_state->compact_format); +if (rm->status ==REF_STATUS_REJECT_SHALLOW || + !rm->peer_ref || + !strcmp (rm->name,"HEAD" )) +continue ; -/* -* Not precise calculation for compact mode because '*' can -* appear on the left hand side of '->' and shrink the column -* back. -*/ -if (display_state->refcol_width < width) - display_state->refcol_width = width; + width =refcol_width (rm, display_state->format ==DISPLAY_FORMAT_COMPACT ); + +/* +* Not precise calculation for compact mode because '*' can +* appear on the left hand side of '->' and shrink the column +* back. +*/ +if (display_state->refcol_width < width) + display_state->refcol_width = width; + } + +break ; + } +default : +BUG ("unexpected display foramt %d" , display_state->format); } }
@@ -904,30 +920,41 @@ static void display_ref_update(struct display_state *display_state, char code, const char *remote, const char *local, int summary_width) { - int width; - if (verbosity < 0) return;strbuf_reset (&display_state->buf); -if (!display_state->shown_url) { -strbuf_addf (&display_state->buf,_ ("From %.*s\n" ), - display_state->url_len, display_state->url); - display_state->shown_url =1 ; +switch (display_state->format) { +case DISPLAY_FORMAT_FULL : +case DISPLAY_FORMAT_COMPACT : { +int width; + +if (!display_state->shown_url) { +strbuf_addf (&display_state->buf,_ ("From %.*s\n" ), + display_state->url_len, display_state->url); + display_state->shown_url =1 ; + } + + width = (summary_width +strlen (summary) -gettext_width (summary)); + remote =prettify_refname (remote); + local =prettify_refname (local); + +strbuf_addf (&display_state->buf," %c %-*s " , code, width, summary); + +if (display_state->format !=DISPLAY_FORMAT_COMPACT ) +print_remote_to_local (display_state, remote, local); +else +print_compact (display_state, remote, local); + +if (error) +strbuf_addf (&display_state->buf," (%s)" , error); + +break ; } - - width = (summary_width +strlen (summary) -gettext_width (summary)); - remote =prettify_refname (remote); - local =prettify_refname (local); - -strbuf_addf (&display_state->buf," %c %-*s " , code, width, summary); -if (!display_state->compact_format) -print_remote_to_local (display_state, remote, local); -else -print_compact (display_state, remote, local); -if (error) -strbuf_addf (&display_state->buf," (%s)" , error); +default : +BUG ("unexpected display format %d" , display_state->format); + };strbuf_addch (&display_state->buf,'\n' );fputs (display_state->buf.buf, stderr);
Signature · 1 line
2.40.0
PATCH 5/8 fetch: move display format parsing into main function
In reply to Patrick SteinhardtParsing the display format happens inside of display_state_init(). As we only need to check for a simple config entry, this is a natural location to put this code as it means that display-state logic is neatly contained in a single location.
We're about to introduce a output format though that is intended to be parseable by machines, for example inside of a script. In that case it becomes a bit awkward of an interface if you have to call git-fetch(1) with the fetch.output config key set. We're thus going to introduce a new --output-format switch for git-fetch(1) so that the output format can be configured more directly.
This means we'll have to hook parsing of the display format into the command line options parser. Having the code to determine the actual output format scattered across two different sites is hard to reason about though.
Refactor the code such that callers are expected to pass the display format that is to be used into display_state_init(). This allows us to lift up the code into the main function, where we can then hook it into command line options parser in a follow-up commit.
--- builtin/fetch.c | 41 ++++++++++++++++++++++++----------------- 1 file changed, 24 insertions(+), 17 deletions(-)
builtin/fetch.c +24 -17
@@ -788,14 +788,13 @@ static int refcol_width(const struct ref *ref, int compact_format) }static void display_state_init (struct display_state *display_state,struct ref *ref_map, -const char *raw_url) +const char *raw_url,enum display_format format) { -const char *format ="full" ;int i;memset (display_state,0 ,sizeof (*display_state)); -strbuf_init (&display_state->buf,0 ); + display_state->format = format;if (raw_url) display_state->url =transport_anonymize_url (raw_url);
@@ -812,15 +811,6 @@ static void display_state_init(struct display_state *display_state, struct ref *if (verbosity <0 )return ; -git_config_get_string_tmp ("fetch.output" , &format); -if (!strcasecmp (format,"full" )) - display_state->format =DISPLAY_FORMAT_FULL ; -else if (!strcasecmp (format,"compact" )) - display_state->format =DISPLAY_FORMAT_COMPACT ; -else -die (_ ("invalid value for '%s': '%s'" ), -"fetch.output" , format); -switch (display_state->format) {case DISPLAY_FORMAT_FULL :case DISPLAY_FORMAT_COMPACT : {
@@ -1614,7 +1604,8 @@ static int backfill_tags(struct display_state *display_state, }static int do_fetch (struct transport *transport, -struct refspec *rs) +struct refspec *rs, +enum display_format display_format) {struct ref_transaction *transaction =NULL ;struct ref *ref_map =NULL ;
@@ -1700,7 +1691,7 @@ static int do_fetch(struct transport *transport,if (retcode) goto cleanup; -display_state_init (&display_state, ref_map, transport->url); +display_state_init (&display_state, ref_map, transport->url, display_format);if (atomic_fetch) { transaction =ref_transaction_begin (&err);
@@ -2076,7 +2067,8 @@ static inline void fetch_one_setup_partial(struct remote *remote) }static int fetch_one (struct remote *remote,int argc,const char **argv, -int prune_tags_ok,int use_stdin_refspecs) +int prune_tags_ok,int use_stdin_refspecs, +enum display_format display_format) {struct refspec rs =REFSPEC_INIT_FETCH ;int i;
@@ -2143,7 +2135,7 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,sigchain_push_common (unlock_pack_on_signal);atexit (unlock_pack_atexit);sigchain_push (SIGPIPE ,SIG_IGN ); - exit_code =do_fetch (gtransport, &rs); + exit_code =do_fetch (gtransport, &rs, display_format);sigchain_pop (SIGPIPE );refspec_clear (&rs);transport_disconnect (gtransport);
@@ -2155,6 +2147,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) {int i;const char *bundle_uri; +enum display_format display_format =DISPLAY_FORMAT_UNKNOWN ;struct string_list list =STRING_LIST_INIT_DUP ;struct remote *remote =NULL ;int result =0 ;
@@ -2181,6 +2174,19 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) argc =parse_options (argc, argv, prefix, builtin_fetch_options, builtin_fetch_usage,0 ); +if (display_format ==DISPLAY_FORMAT_UNKNOWN ) { +const char *format ="full" ; + +git_config_get_string_tmp ("fetch.output" , &format); +if (!strcasecmp (format,"full" )) + display_format =DISPLAY_FORMAT_FULL ; +else if (!strcasecmp (format,"compact" )) + display_format =DISPLAY_FORMAT_COMPACT ; +else +die (_ ("invalid value for '%s': '%s'" ), +"fetch.output" , format); + } +if (recurse_submodules_cli !=RECURSE_SUBMODULES_DEFAULT ) recurse_submodules = recurse_submodules_cli;
@@ -2309,7 +2315,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) }else if (remote) {if (filter_options.choice ||has_promisor_remote ())fetch_one_setup_partial (remote); - result =fetch_one (remote, argc, argv, prune_tags_ok, stdin_refspecs); + result =fetch_one (remote, argc, argv, prune_tags_ok, stdin_refspecs, + display_format); }else {int max_children = max_jobs;
Signature · 1 line
2.40.0
PATCH 6/8 fetch: move option related variables into main function
In reply to Patrick SteinhardtThe options of git-fetch(1) which we pass to parse_options() are declared globally in builtin/fetch.c. This means we're forced to use global variables for all the options, which is more likely to cause confusion than explicitly passing state around.
Refactor the code to move the options into cmd_fetch(). Move variables that were previously forced to be declared globally and which are only used by cmd_fetch() into function-local scope.
--- builtin/fetch.c | 194 ++++++++++++++++++++++++------------------------ 1 file changed, 98 insertions(+), 96 deletions(-)
builtin/fetch.c +98 -96
@@ -79,9 +79,8 @@ static int all, append, dry_run, force, keep, multiple, update_head_ok;static int write_fetch_head =1 ;static int verbosity, deepen_relative, set_upstream, refetch;static int progress =-1 ; -static int enable_auto_gc =1 ; -static int tags =TAGS_DEFAULT , unshallow, update_shallow, deepen; -static int max_jobs =-1 , submodule_fetch_jobs_config =-1 ; +static int tags =TAGS_DEFAULT , update_shallow, deepen; +static int submodule_fetch_jobs_config =-1 ;static int fetch_parallel_config =1 ;static int atomic_fetch;static enum transport_family family;
@@ -92,17 +91,11 @@ static struct string_list deepen_not = STRING_LIST_INIT_NODUP;static struct strbuf default_rla =STRBUF_INIT ;static struct transport *gtransport;static struct transport *gsecondary; -static const char *submodule_prefix ="" ;static int recurse_submodules =RECURSE_SUBMODULES_DEFAULT ; -static int recurse_submodules_cli =RECURSE_SUBMODULES_DEFAULT ; -static int recurse_submodules_default =RECURSE_SUBMODULES_ON_DEMAND ;static struct refspec refmap =REFSPEC_INIT_FETCH ;static struct list_objects_filter_options filter_options =LIST_OBJECTS_FILTER_INIT ;static struct string_list server_options =STRING_LIST_INIT_DUP ;static struct string_list negotiation_tip =STRING_LIST_INIT_NODUP ; -static int fetch_write_commit_graph =-1 ; -static int stdin_refspecs =0 ; -static int negotiate_only;static int git_fetch_config (const char *k,const char *v,void *cb) {
@@ -160,92 +153,6 @@ static int parse_refmap_arg(const struct option *opt, const char *arg, int unsetreturn 0 ; } -static struct option builtin_fetch_options[] = { -OPT__VERBOSITY (&verbosity), -OPT_BOOL (0 ,"all" , &all, -N_ ("fetch from all remotes" )), -OPT_BOOL (0 ,"set-upstream" , &set_upstream, -N_ ("set upstream for git pull/fetch" )), -OPT_BOOL ('a' ,"append" , &append, -N_ ("append to .git/FETCH_HEAD instead of overwriting" )), -OPT_BOOL (0 ,"atomic" , &atomic_fetch, -N_ ("use atomic transaction to update references" )), -OPT_STRING (0 ,"upload-pack" , &upload_pack,N_ ("path" ), -N_ ("path to upload pack on remote end" )), -OPT__FORCE (&force,N_ ("force overwrite of local reference" ),0 ), -OPT_BOOL ('m' ,"multiple" , &multiple, -N_ ("fetch from multiple remotes" )), -OPT_SET_INT ('t' ,"tags" , &tags, -N_ ("fetch all tags and associated objects" ),TAGS_SET ), -OPT_SET_INT ('n' ,NULL , &tags, -N_ ("do not fetch all tags (--no-tags)" ),TAGS_UNSET ), -OPT_INTEGER ('j' ,"jobs" , &max_jobs, -N_ ("number of submodules fetched in parallel" )), -OPT_BOOL (0 ,"prefetch" , &prefetch, -N_ ("modify the refspec to place all refs within refs/prefetch/" )), -OPT_BOOL ('p' ,"prune" , &prune, -N_ ("prune remote-tracking branches no longer on remote" )), -OPT_BOOL ('P' ,"prune-tags" , &prune_tags, -N_ ("prune local tags no longer on remote and clobber changed tags" )), -OPT_CALLBACK_F (0 ,"recurse-submodules" , &recurse_submodules_cli,N_ ("on-demand" ), -N_ ("control recursive fetching of submodules" ), -PARSE_OPT_OPTARG , option_fetch_parse_recurse_submodules), -OPT_BOOL (0 ,"dry-run" , &dry_run, -N_ ("dry run" )), -OPT_BOOL (0 ,"write-fetch-head" , &write_fetch_head, -N_ ("write fetched references to the FETCH_HEAD file" )), -OPT_BOOL ('k' ,"keep" , &keep,N_ ("keep downloaded pack" )), -OPT_BOOL ('u' ,"update-head-ok" , &update_head_ok, -N_ ("allow updating of HEAD ref" )), -OPT_BOOL (0 ,"progress" , &progress,N_ ("force progress reporting" )), -OPT_STRING (0 ,"depth" , &depth,N_ ("depth" ), -N_ ("deepen history of shallow clone" )), -OPT_STRING (0 ,"shallow-since" , &deepen_since,N_ ("time" ), -N_ ("deepen history of shallow repository based on time" )), -OPT_STRING_LIST (0 ,"shallow-exclude" , &deepen_not,N_ ("revision" ), -N_ ("deepen history of shallow clone, excluding rev" )), -OPT_INTEGER (0 ,"deepen" , &deepen_relative, -N_ ("deepen history of shallow clone" )), -OPT_SET_INT_F (0 ,"unshallow" , &unshallow, -N_ ("convert to a complete repository" ), -1 ,PARSE_OPT_NONEG ), -OPT_SET_INT_F (0 ,"refetch" , &refetch, -N_ ("re-fetch without negotiating common commits" ), -1 ,PARSE_OPT_NONEG ), - {OPTION_STRING ,0 ,"submodule-prefix" , &submodule_prefix,N_ ("dir" ), -N_ ("prepend this to submodule path output" ),PARSE_OPT_HIDDEN }, -OPT_CALLBACK_F (0 ,"recurse-submodules-default" , - &recurse_submodules_default,N_ ("on-demand" ), -N_ ("default for recursive fetching of submodules " -"(lower priority than config files)" ), -PARSE_OPT_HIDDEN , option_fetch_parse_recurse_submodules), -OPT_BOOL (0 ,"update-shallow" , &update_shallow, -N_ ("accept refs that update .git/shallow" )), -OPT_CALLBACK_F (0 ,"refmap" ,NULL ,N_ ("refmap" ), -N_ ("specify fetch refmap" ),PARSE_OPT_NONEG , parse_refmap_arg), -OPT_STRING_LIST ('o' ,"server-option" , &server_options,N_ ("server-specific" ),N_ ("option to transmit" )), -OPT_SET_INT ('4' ,"ipv4" , &family,N_ ("use IPv4 addresses only" ), -TRANSPORT_FAMILY_IPV4 ), -OPT_SET_INT ('6' ,"ipv6" , &family,N_ ("use IPv6 addresses only" ), -TRANSPORT_FAMILY_IPV6 ), -OPT_STRING_LIST (0 ,"negotiation-tip" , &negotiation_tip,N_ ("revision" ), -N_ ("report that we have only objects reachable from this object" )), -OPT_BOOL (0 ,"negotiate-only" , &negotiate_only, -N_ ("do not fetch a packfile; instead, print ancestors of negotiation tips" )), -OPT_PARSE_LIST_OBJECTS_FILTER (&filter_options), -OPT_BOOL (0 ,"auto-maintenance" , &enable_auto_gc, -N_ ("run 'maintenance --auto' after fetching" )), -OPT_BOOL (0 ,"auto-gc" , &enable_auto_gc, -N_ ("run 'maintenance --auto' after fetching" )), -OPT_BOOL (0 ,"show-forced-updates" , &fetch_show_forced_updates, -N_ ("check for forced-updates on all updated branches" )), -OPT_BOOL (0 ,"write-commit-graph" , &fetch_write_commit_graph, -N_ ("write the commit-graph after fetching" )), -OPT_BOOL (0 ,"stdin" , &stdin_refspecs, -N_ ("accept refspecs from stdin" )), -OPT_END () -}; -static void unlock_pack (unsigned int flags) {if (gtransport)
@@ -2145,13 +2052,108 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,int cmd_fetch (int argc,const char **argv,const char *prefix) { -int i;const char *bundle_uri; +const char *submodule_prefix ="" ;enum display_format display_format =DISPLAY_FORMAT_UNKNOWN ;struct string_list list =STRING_LIST_INIT_DUP ;struct remote *remote =NULL ;int result =0 ;int prune_tags_ok =1 ; +int enable_auto_gc =1 ; +int unshallow =0 ; +int max_jobs =-1 ; +int recurse_submodules_cli =RECURSE_SUBMODULES_DEFAULT ; +int recurse_submodules_default =RECURSE_SUBMODULES_ON_DEMAND ; +int fetch_write_commit_graph =-1 ; +int stdin_refspecs =0 ; +int negotiate_only =0 ; +int i; + +struct option builtin_fetch_options[] = { +OPT__VERBOSITY (&verbosity), +OPT_BOOL (0 ,"all" , &all, +N_ ("fetch from all remotes" )), +OPT_BOOL (0 ,"set-upstream" , &set_upstream, +N_ ("set upstream for git pull/fetch" )), +OPT_BOOL ('a' ,"append" , &append, +N_ ("append to .git/FETCH_HEAD instead of overwriting" )), +OPT_BOOL (0 ,"atomic" , &atomic_fetch, +N_ ("use atomic transaction to update references" )), +OPT_STRING (0 ,"upload-pack" , &upload_pack,N_ ("path" ), +N_ ("path to upload pack on remote end" )), +OPT__FORCE (&force,N_ ("force overwrite of local reference" ),0 ), +OPT_BOOL ('m' ,"multiple" , &multiple, +N_ ("fetch from multiple remotes" )), +OPT_SET_INT ('t' ,"tags" , &tags, +N_ ("fetch all tags and associated objects" ),TAGS_SET ), +OPT_SET_INT ('n' ,NULL , &tags, +N_ ("do not fetch all tags (--no-tags)" ),TAGS_UNSET ), +OPT_INTEGER ('j' ,"jobs" , &max_jobs, +N_ ("number of submodules fetched in parallel" )), +OPT_BOOL (0 ,"prefetch" , &prefetch, +N_ ("modify the refspec to place all refs within refs/prefetch/" )), +OPT_BOOL ('p' ,"prune" , &prune, +N_ ("prune remote-tracking branches no longer on remote" )), +OPT_BOOL ('P' ,"prune-tags" , &prune_tags, +N_ ("prune local tags no longer on remote and clobber changed tags" )), +OPT_CALLBACK_F (0 ,"recurse-submodules" , &recurse_submodules_cli,N_ ("on-demand" ), +N_ ("control recursive fetching of submodules" ), +PARSE_OPT_OPTARG , option_fetch_parse_recurse_submodules), +OPT_BOOL (0 ,"dry-run" , &dry_run, +N_ ("dry run" )), +OPT_BOOL (0 ,"write-fetch-head" , &write_fetch_head, +N_ ("write fetched references to the FETCH_HEAD file" )), +OPT_BOOL ('k' ,"keep" , &keep,N_ ("keep downloaded pack" )), +OPT_BOOL ('u' ,"update-head-ok" , &update_head_ok, +N_ ("allow updating of HEAD ref" )), +OPT_BOOL (0 ,"progress" , &progress,N_ ("force progress reporting" )), +OPT_STRING (0 ,"depth" , &depth,N_ ("depth" ), +N_ ("deepen history of shallow clone" )), +OPT_STRING (0 ,"shallow-since" , &deepen_since,N_ ("time" ), +N_ ("deepen history of shallow repository based on time" )), +OPT_STRING_LIST (0 ,"shallow-exclude" , &deepen_not,N_ ("revision" ), +N_ ("deepen history of shallow clone, excluding rev" )), +OPT_INTEGER (0 ,"deepen" , &deepen_relative, +N_ ("deepen history of shallow clone" )), +OPT_SET_INT_F (0 ,"unshallow" , &unshallow, +N_ ("convert to a complete repository" ), +1 ,PARSE_OPT_NONEG ), +OPT_SET_INT_F (0 ,"refetch" , &refetch, +N_ ("re-fetch without negotiating common commits" ), +1 ,PARSE_OPT_NONEG ), + {OPTION_STRING ,0 ,"submodule-prefix" , &submodule_prefix,N_ ("dir" ), +N_ ("prepend this to submodule path output" ),PARSE_OPT_HIDDEN }, +OPT_CALLBACK_F (0 ,"recurse-submodules-default" , + &recurse_submodules_default,N_ ("on-demand" ), +N_ ("default for recursive fetching of submodules " +"(lower priority than config files)" ), +PARSE_OPT_HIDDEN , option_fetch_parse_recurse_submodules), +OPT_BOOL (0 ,"update-shallow" , &update_shallow, +N_ ("accept refs that update .git/shallow" )), +OPT_CALLBACK_F (0 ,"refmap" ,NULL ,N_ ("refmap" ), +N_ ("specify fetch refmap" ),PARSE_OPT_NONEG , parse_refmap_arg), +OPT_STRING_LIST ('o' ,"server-option" , &server_options,N_ ("server-specific" ),N_ ("option to transmit" )), +OPT_SET_INT ('4' ,"ipv4" , &family,N_ ("use IPv4 addresses only" ), +TRANSPORT_FAMILY_IPV4 ), +OPT_SET_INT ('6' ,"ipv6" , &family,N_ ("use IPv6 addresses only" ), +TRANSPORT_FAMILY_IPV6 ), +OPT_STRING_LIST (0 ,"negotiation-tip" , &negotiation_tip,N_ ("revision" ), +N_ ("report that we have only objects reachable from this object" )), +OPT_BOOL (0 ,"negotiate-only" , &negotiate_only, +N_ ("do not fetch a packfile; instead, print ancestors of negotiation tips" )), +OPT_PARSE_LIST_OBJECTS_FILTER (&filter_options), +OPT_BOOL (0 ,"auto-maintenance" , &enable_auto_gc, +N_ ("run 'maintenance --auto' after fetching" )), +OPT_BOOL (0 ,"auto-gc" , &enable_auto_gc, +N_ ("run 'maintenance --auto' after fetching" )), +OPT_BOOL (0 ,"show-forced-updates" , &fetch_show_forced_updates, +N_ ("check for forced-updates on all updated branches" )), +OPT_BOOL (0 ,"write-commit-graph" , &fetch_write_commit_graph, +N_ ("write the commit-graph after fetching" )), +OPT_BOOL (0 ,"stdin" , &stdin_refspecs, +N_ ("accept refspecs from stdin" )), +OPT_END () + };packet_trace_identity ("fetch" );
Signature · 1 line
2.40.0
PATCH 7/8 fetch: introduce new `--output-format` option
In reply to Patrick SteinhardtIt is only possible to configure the output format that git-fetch(1) uses by setting it via a config key. While this interface may be fine as long as we only have the current "full" and "compact" output formats, where it is unlikely that the user will have to change them regularly. But we're about to introduce a new machine-parseable interface where the current mechanism feels a little bit indirect and rigid.
Introduce a new --output-format option that allows the user to change the desired output format more directly.
--- Documentation/fetch-options.txt | 5 ++++ builtin/fetch.c | 17 +++++++++++ t/t5574-fetch-output.sh | 50 +++++++++++++++++++++++++-------- 3 files changed, 60 insertions(+), 12 deletions(-)
Documentation/fetch-options.txt +5 -0
@@ -78,6 +78,11 @@ linkgit:git-config[1]. --dry-run:: Show what would be done, without making any changes. +--output-format:: + Control how ref update status is printed. Valid values are + `full` and `compact`. Default value is `full`. See section + OUTPUT in linkgit:git-fetch[1] for detail. + ifndef::git-pull[] --[no-]write-fetch-head:: Write the list of remote refs fetched in the `FETCH_HEAD`
builtin/fetch.c +17 -0
@@ -2050,6 +2050,21 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,return exit_code; } +static int opt_parse_output_format (const struct option *opt,const char *arg,int unset) +{ +enum display_format *format = opt->value; +if (unset || !arg) +return 1 ; +else if (!strcmp (arg,"full" )) + *format =DISPLAY_FORMAT_FULL ; +else if (!strcmp (arg,"compact" )) + *format =DISPLAY_FORMAT_COMPACT ; +else +return error (_ ("unsupported output format '%s'" ), arg); + +return 0 ; +} +int cmd_fetch (int argc,const char **argv,const char *prefix) {const char *bundle_uri;
@@ -2101,6 +2116,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),OPT_BOOL (0 ,"dry-run" , &dry_run,N_ ("dry run" )), +OPT_CALLBACK (0 ,"output-format" , &display_format,N_ ("format" ),N_ ("output format" ), + opt_parse_output_format),OPT_BOOL (0 ,"write-fetch-head" , &write_fetch_head,N_ ("write fetched references to the FETCH_HEAD file" )), OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
t/t5574-fetch-output.sh +38 -12
65 lines not on this page · raw message
Signature · 1 line
2.40.0
Re: [PATCH 7/8] fetch: introduce new `--output-format` option
builtin/fetch.c int cmd_fetch(int argc, const char **argv, const char *prefix) · line 2123
Quoted 9 lines · Patrick Steinhardt
builtin/fetch.c +2
-0@@ -2101,6 +2116,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),OPT_BOOL (0 ,"dry-run" , &dry_run,N_ ("dry run" )), +OPT_CALLBACK (0 ,"output-format" , &display_format,N_ ("format" ),N_ ("output format" ), + opt_parse_output_format),OPT_BOOL (0 ,"write-fetch-head" , &write_fetch_head,N_ ("write fetched references to the FETCH_HEAD file" )), OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
This change is good enough for fetching from a single remote, but if we want to support "--all", we'd also need to propagate the CLI flag to the child "fetch" processes. (The config option wouldn't have this bug because the child processes would parse config and get the correct value.)
Re: [PATCH 7/8] fetch: introduce new `--output-format` option
Quoted 17 lines · Glen Choo
builtin/fetch.c int cmd_fetch(int argc, const char **argv, const char *prefix) · line 2123
Quoted 9 lines · Patrick Steinhardt
builtin/fetch.c +2
-0@@ -2101,6 +2116,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),OPT_BOOL (0 ,"dry-run" , &dry_run,N_ ("dry run" )), +OPT_CALLBACK (0 ,"output-format" , &display_format,N_ ("format" ),N_ ("output format" ), + opt_parse_output_format),OPT_BOOL (0 ,"write-fetch-head" , &write_fetch_head,N_ ("write fetched references to the FETCH_HEAD file" )), OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),This change is good enough for fetching from a single remote, but if we want to support "--all", we'd also need to propagate the CLI flag to the child "fetch" processes. (The config option wouldn't have this bug because the child processes would parse config and get the correct value.)
Oh, right, good catch. Will fix, thanks!
Patrick
PATCH 8/8 fetch: introduce machine-parseable "porcelain" output format
In reply to Patrick SteinhardtThe output of git-fetch(1) is obviously designed for consumption by users, only: we neatly columnize data, we abbreviate reference names, we print neat arrows and we don't provide information about actual object IDs that have changed. This makes the output format basically unusable in the context of scripted invocations of git-fetch(1) that want to learn about the exact changes that the command performs.
Introduce a new machine-parseable "porcelain" output format that is supposed to fix this shortcoming. This output format is intended to provide information about every reference that is about to be updated, the old object ID that the reference has been pointing to and the new object ID it will be updated to. Furthermore, the output format provides the same flags as the human-readable format to indicate basic conditions for each reference update like whether it was a fast-forward update, a branch deletion, a rejected update or others.
The output format is quite simple:
``` <flag> <old-object-id> <new-object-id> <local-reference>\n ```
We assume two conditions which are generally true:
- The old and new object IDs have fixed known widths and cannot contain spaces.
- References cannot contain newlines.
With these assumptions, the output format becomes unambiguously parseable. Furthermore, given that this output is designed to be consumed by scripts, the machine-readable data is printed to stdout instead of stderr like the human-readable output is. This is mostly done so that other data printed to stderr, like error messages or progress meters, don't interfere with the parseable data.
--- Documentation/config/fetch.txt | 4 +- Documentation/fetch-options.txt | 4 +- Documentation/git-fetch.txt | 17 ++++++- builtin/fetch.c | 50 +++++++++++++++----- t/t5574-fetch-output.sh | 84 +++++++++++++++++++++++++++++++++ 5 files changed, 142 insertions(+), 17 deletions(-)
Documentation/config/fetch.txt +2 -2
@@ -52,8 +52,8 @@ fetch.pruneTags:: fetch.output:: Control how ref update status is printed. Valid values are - `full` and `compact`. Default value is `full`. See section - OUTPUT in linkgit:git-fetch[1] for detail. + `full`, `compact` and `porcelain`. Default value is `full`. + See section OUTPUT in linkgit:git-fetch[1] for detail. fetch.negotiationAlgorithm:: Control how information about the commits in the local repository
Documentation/fetch-options.txt +2 -2
@@ -80,8 +80,8 @@ linkgit:git-config[1]. --output-format:: Control how ref update status is printed. Valid values are - `full` and `compact`. Default value is `full`. See section - OUTPUT in linkgit:git-fetch[1] for detail. + `full`, `compact` and `porcelain`. Default value is `full`. + See section OUTPUT in linkgit:git-fetch[1] for detail. ifndef::git-pull[] --[no-]write-fetch-head::
Documentation/git-fetch.txt +15 -2
@@ -197,13 +197,26 @@ The output of "git fetch" depends on the transport method used; this section describes the output when fetching over the Git protocol (either locally or via ssh) and Smart HTTP protocol. -The status of the fetch is output in tabular form, with each line -representing the status of a single ref. Each line is of the form: +The output format can be chosen either via the `fetch.output` config +(see linkgit:git-config[1]), or via the `--output-format` switch. +Supported values include: + +For the `full` and `compact` output formats, the status of the fetch is +output in tabular, with each line representing the status of a single +ref. Each line is of the form: ------------------------------- <flag> <summary> <from> -> <to> [<reason>] ------------------------------- +The `porcelain` output format is intended to be machine-parseable. In +contrast to the human-readable output formats it thus prints to standard +output instead of standard error. Each line is of the form: + +------------------------------- +<flag> <old-object-id> <new-object-id> <local-reference> +------------------------------- + The status of up-to-date refs is shown only if the --verbose option is used.
builtin/fetch.c +39 -11
150 lines not on this page · raw message
t/t5574-fetch-output.sh +84 -0
96 lines not on this page · raw message
Signature · 1 line
2.40.0
Re: [PATCH 8/8] fetch: introduce machine-parseable "porcelain" output format
Quoted 5 lines · Patrick Steinhardt
The output format is quite simple:
``` <flag> <old-object-id> <new-object-id> <local-reference>\n ```
This format doesn't show the remote name or url that was fetched. That seems okay when fetching with a single remote, but it seems necessary with "--all". Perhaps you were planning to add that in a later series? If so, I think it's okay to call the "porcelain" format experimental, and forbid porcelain + --all until then.
Quoted 7 lines
We assume two conditions which are generally true:
- The old and new object IDs have fixed known widths and cannot contain spaces.
- References cannot contain newlines.
This seems like a non-issue if we add a -z CLI option to indicate that entries should be NUL terminated instead of newline terminated, but that can be done as a followup.
Quoted 6 lines
With these assumptions, the output format becomes unambiguously parseable. Furthermore, given that this output is designed to be consumed by scripts, the machine-readable data is printed to stdout instead of stderr like the human-readable output is. This is mostly done so that other data printed to stderr, like error messages or progress meters, don't interfere with the parseable data.
Sending the 'main output' to stdout makes sense to me, but this (and possibly respecting -z) sounds like a different mode of operation, not just a matter of formats. It seems different enough that I'd prefer not to piggyback on "fetch.output" for this (even though this adds more surface to the interface...).
We could add --porcelain and say that "fetch.output" is ignored if --porcelain is also given. That also eliminates the need for --output-format, I think.
The .c changes look good to me.
Quoted 7 lines
t/t5574-fetch-output.sh +7
-0+test_expect_success 'fetch porcelain output with HEAD and --dry-run' ' + test_when_finished "rm -rf head" && + git clone . head && + COMMIT_ID=$(git rev-parse HEAD) && + + git -C head fetch --output-format=porcelain --dry-run origin HEAD >actual && + cat >expect <<-EOF &&
t/t5574-fetch-output.sh test_expect_success 'fetch output with HEAD and --dry-run' ' · line 184
Quoted 10 lines
t/t5574-fetch-output.sh +10
-0+ * $ZERO_OID $COMMIT_ID FETCH_HEAD + EOF + test_cmp expect actual && + + git -C head fetch --output-format=porcelain --dry-run origin HEAD:foo >actual && + cat >expect <<-EOF && + * $ZERO_OID $COMMIT_ID refs/heads/foo + EOF + test_cmp expect actual +'
As mentioned upthread, I think this test isn't needed because "porcelain" wouldn't run into the bug we are checking for anyway.
Re: [PATCH 8/8] fetch: introduce machine-parseable "porcelain" output format
Quoted 8 lines · Glen Choo
Quoted 5 lines · Patrick Steinhardt
The output format is quite simple:
``` <flag> <old-object-id> <new-object-id> <local-reference>\n ```
Quoted 5 lines
This format doesn't show the remote name or url that was fetched. That seems okay when fetching with a single remote, but it seems necessary with "--all". Perhaps you were planning to add that in a later series? If so, I think it's okay to call the "porcelain" format experimental, and forbid porcelain + --all until then.
The reason is mostly that I didn't find an output format that I really liked here. We'd basically have to repeat the remote URL for every single reference: just repeating it once per remote doesn't fly because with --parallel the output could be intermingled. But doing that feels wasteful to me, so I bailed. I guess I'm also biased here because it just wouldn't be useful to myself.
So with that in mind, I'd like to continue ignoring this issue for now and just not report the remote that the ref came from. But I'd also argue that we don't have to restrict porcelain mode to single-remote
fetches: it can still be useful to do multi-remote fetches even without the information where a certain reference update comes from. So any kind of restriction would feel artificial to me here.
Furthermore, I'd argue that it is not necessary to label the format as experimental only because of this limitation. With the refactorings done in this and the preceding patch series it is easy to add a new format in case there indeed is somebody that would have a usecase for this. The "porcelain" format should stay stable, and if we decide that we want to also report the remote for each reference in a follow-up we can easily add a "porcelain-v2" or "porcelain-with-remote" format.
As I said though: I'm clearly biased, so if you feel like my train of though is simply me being lazy then I'd carve in and adapt.
Quoted 10 lines
Quoted 6 lines
We assume two conditions which are generally true:
- The old and new object IDs have fixed known widths and cannot contain spaces.
- References cannot contain newlines.
This seems like a non-issue if we add a -z CLI option to indicate that entries should be NUL terminated instead of newline terminated, but that can be done as a followup.
Yeah, either via -z or a new porcelain output format. But both of these conditions should generally be true anyway, so I don't see that those should become a problem.
Quoted 13 lines
Quoted 6 lines
With these assumptions, the output format becomes unambiguously parseable. Furthermore, given that this output is designed to be consumed by scripts, the machine-readable data is printed to stdout instead of stderr like the human-readable output is. This is mostly done so that other data printed to stderr, like error messages or progress meters, don't interfere with the parseable data.
Sending the 'main output' to stdout makes sense to me, but this (and possibly respecting -z) sounds like a different mode of operation, not just a matter of formats. It seems different enough that I'd prefer not to piggyback on "fetch.output" for this (even though this adds more surface to the interface...).
Quoted 3 lines
We could add --porcelain and say that "fetch.output" is ignored if --porcelain is also given. That also eliminates the need for --output-format, I think.
I was thinking about this initially, as well. But ultimately I decided against this especially because of your second paragraph: we'd now need to think about precedence of options and mutual exclusion, and that to me feels like an interface that is less obvious than a single knob that works as you'd expect.
Quoted 14 lines
The .c changes look good to me.
Quoted 12 lines
t/t5574-fetch-output.sh +12
-0+test_expect_success 'fetch porcelain output with HEAD and --dry-run' ' + test_when_finished "rm -rf head" && + git clone . head && + COMMIT_ID=$(git rev-parse HEAD) && + + git -C head fetch --output-format=porcelain --dry-run origin HEAD >actual && + cat >expect <<-EOF && + * $ZERO_OID $COMMIT_ID FETCH_HEAD + EOF + test_cmp expect actual && + + git -C head fetch --output-format=porcelain --dry-run origin HEAD:foo >actual &&
Quoted 8 lines
t/t5574-fetch-output.sh test_expect_success 'fetch output with HEAD and --dry-run' ' · line 184
Quoted 5 lines
t/t5574-fetch-output.sh +5
-0+ cat >expect <<-EOF && + * $ZERO_OID $COMMIT_ID refs/heads/foo + EOF + test_cmp expect actual +'As mentioned upthread, I think this test isn't needed because "porcelain" wouldn't run into the bug we are checking for anyway.
The only reason that the other bug was able to survive for so long was that we didn't have test coverage there. So I think it makes sense to explicitly test this, too, also because it causes us to walk a different code path.
Last but not least: this test uncovered a segfault I had in a previous version. So I'd rather keep it :)
Patrick
Re: [PATCH 8/8] fetch: introduce machine-parseable "porcelain" output format
Quoted 15 lines · Patrick Steinhardt
Quoted 8 lines · Glen Choo
Quoted 5 lines · Patrick Steinhardt
The output format is quite simple:
``` <flag> <old-object-id> <new-object-id> <local-reference>\n ```Quoted 5 lines
This format doesn't show the remote name or url that was fetched. That seems okay when fetching with a single remote, but it seems necessary with "--all". Perhaps you were planning to add that in a later series? If so, I think it's okay to call the "porcelain" format experimental, and forbid porcelain + --all until then.
Quoted 6 lines
The reason is mostly that I didn't find an output format that I really liked here. We'd basically have to repeat the remote URL for every single reference: just repeating it once per remote doesn't fly because with
--parallelthe output could be intermingled. But doing that feels wasteful to me, so I bailed. I guess I'm also biased here because it just wouldn't be useful to myself.
Yeah, I couldn't think of a good way to output the remote URL either.
Quoted 7 lines
So with that in mind, I'd like to continue ignoring this issue for now and just not report the remote that the ref came from. But I'd also argue that we don't have to restrict porcelain mode to single-remote
fetches: it can still be useful to do multi-remote fetches even without the information where a certain reference update comes from. So any kind of restriction would feel artificial to me here.
Quoted 7 lines
Furthermore, I'd argue that it is not necessary to label the format as experimental only because of this limitation. With the refactorings done in this and the preceding patch series it is easy to add a new format in case there indeed is somebody that would have a usecase for this. The "porcelain" format should stay stable, and if we decide that we want to also report the remote for each reference in a follow-up we can easily add a "porcelain-v2" or "porcelain-with-remote" format.
I agree that the current form is useful, but it seems quite likely that users will want the remote name/url at some point. In the majority of cases, the user can parse the remote name from the updated local ref and if they want, they look up the url by asking "git config", but that breaks down quite quickly if they have a custom refspec.
My suggestion to label it experimental was meant to give us the freedom to change the "first version" in backwards-incompatible ways that we otherwise couldn't. We could say "here's porcelain-v2, use this and ignore porcelain-v1", but it would be even nicer if the user didn't need to think about porcelain-v1. The idea is that we could call it experimental, see what users need in the wild, then make changes without having to create a new "porcelain-v2" mode.
(I realize in hindsight that calling it experimental sounds a bit insulting. I thought your proposal was quite sensible, actually. i apologize if it came across that way)
Perhaps part of the reason why you'd prefer not to make it experimental is that users who might want this feature are too afraid of backwards-incompatible changes. I think that's fair. A different way of achieving the same flexibility would be to make the "porcelain" format extensible (like git status --porcelain=v2). A future, multivalued "--porcelain-options=show-remote-url,show-remote-ref" would cover our bases.
I'm happy with either approach as long as we don't have to bikeshed about the "perfect" porcelain output :)
Quoted 17 lines
Quoted 13 lines
Quoted 6 lines
With these assumptions, the output format becomes unambiguously parseable. Furthermore, given that this output is designed to be consumed by scripts, the machine-readable data is printed to stdout instead of stderr like the human-readable output is. This is mostly done so that other data printed to stderr, like error messages or progress meters, don't interfere with the parseable data.
Sending the 'main output' to stdout makes sense to me, but this (and possibly respecting -z) sounds like a different mode of operation, not just a matter of formats. It seems different enough that I'd prefer not to piggyback on "fetch.output" for this (even though this adds more surface to the interface...).
Quoted 3 lines
We could add --porcelain and say that "fetch.output" is ignored if --porcelain is also given. That also eliminates the need for --output-format, I think.
Quoted 5 lines
I was thinking about this initially, as well. But ultimately I decided against this especially because of your second paragraph: we'd now need to think about precedence of options and mutual exclusion, and that to me feels like an interface that is less obvious than a single knob that works as you'd expect.
My full thoughts on this are in
https://lore.kernel.org/git/[email protected]
but the short version is that I'm not sure if I expect something as innocuous-looking as --output-format would imply other, machine-friendly things (like stdout instead of stderr), and using --porcelain might make option precedence clearer in some situtations (like if -z is given).
Quoted 21 lines
Quoted 12 lines
Quoted 12 lines
t/t5574-fetch-output.sh +12
-0+test_expect_success 'fetch porcelain output with HEAD and --dry-run' ' + test_when_finished "rm -rf head" && + git clone . head && + COMMIT_ID=$(git rev-parse HEAD) && + + git -C head fetch --output-format=porcelain --dry-run origin HEAD >actual && + cat >expect <<-EOF && + * $ZERO_OID $COMMIT_ID FETCH_HEAD + EOF + test_cmp expect actual && + + git -C head fetch --output-format=porcelain --dry-run origin HEAD:foo >actual &&Quoted 8 lines
t/t5574-fetch-output.sh test_expect_success 'fetch output with HEAD and --dry-run' ' · line 184
Quoted 5 lines
t/t5574-fetch-output.sh +5
-0+ cat >expect <<-EOF && + * $ZERO_OID $COMMIT_ID refs/heads/foo + EOF + test_cmp expect actual +'As mentioned upthread, I think this test isn't needed because "porcelain" wouldn't run into the bug we are checking for anyway.
Quoted 7 lines
The only reason that the other bug was able to survive for so long was that we didn't have test coverage there. So I think it makes sense to explicitly test this, too, also because it causes us to walk a different code path.
Last but not least: this test uncovered a segfault I had in a previous version. So I'd rather keep it :)
Ah, okay, both are fair. In that case, it probably makes sense to drop the "--dry-run".
Re: [PATCH 8/8] fetch: introduce machine-parseable "porcelain" output format
Quoted 3 lines · Glen Choo
Quoted 2 lines · Patrick Steinhardt
Quoted 1 line · Glen Choo
Patrick Steinhardt <[email protected]> writes:
[snip]
Quoted 29 lines
Quoted 7 lines
So with that in mind, I'd like to continue ignoring this issue for now and just not report the remote that the ref came from. But I'd also argue that we don't have to restrict porcelain mode to single-remote
fetches: it can still be useful to do multi-remote fetches even without the information where a certain reference update comes from. So any kind of restriction would feel artificial to me here.Quoted 7 lines
Furthermore, I'd argue that it is not necessary to label the format as experimental only because of this limitation. With the refactorings done in this and the preceding patch series it is easy to add a new format in case there indeed is somebody that would have a usecase for this. The "porcelain" format should stay stable, and if we decide that we want to also report the remote for each reference in a follow-up we can easily add a "porcelain-v2" or "porcelain-with-remote" format.
I agree that the current form is useful, but it seems quite likely that users will want the remote name/url at some point. In the majority of cases, the user can parse the remote name from the updated local ref and if they want, they look up the url by asking "git config", but that breaks down quite quickly if they have a custom refspec.
My suggestion to label it experimental was meant to give us the freedom to change the "first version" in backwards-incompatible ways that we otherwise couldn't. We could say "here's porcelain-v2, use this and ignore porcelain-v1", but it would be even nicer if the user didn't need to think about porcelain-v1. The idea is that we could call it experimental, see what users need in the wild, then make changes without having to create a new "porcelain-v2" mode.
Quoted 3 lines
(I realize in hindsight that calling it experimental sounds a bit insulting. I thought your proposal was quite sensible, actually. i apologize if it came across that way)
No offense taken, so no need to apologize.
Quoted 3 lines
Perhaps part of the reason why you'd prefer not to make it experimental is that users who might want this feature are too afraid of backwards-incompatible changes. I think that's fair.
Exactly. I'm implementing this whole feature with a concrete usecase at GitLab in mind, and if we declare this as experimental then it will become a whole lot more risky for us to actually adopt it. Because if the output format may change at any point in time, then the code that builds on top of it is likely to be broken.
Quoted 5 lines
A different way of
achieving the same flexibility would be to make the "porcelain" format extensible (like git status --porcelain=v2). A future, multivalued "--porcelain-options=show-remote-url,show-remote-ref" would cover our bases.
Yeah, that would indeed be quite flexible. Theoretically, there is no reason why we couldn't have --output-format=porcelain,show-remote-url though.
Quoted 2 lines
I'm happy with either approach as long as we don't have to bikeshed about the "perfect" porcelain output :)
Agreed, and that's why I'm currently defending the "good enough" format. It should likely work for most usecases that exist out there. The target audience is going to be quite small here as this is not a user-directed feature. Furthermore, I assume that the overlap of machines that want to parse all reference updates done in a multi-remote fetch while also being able to exactly tell which remote brought in what updates is going to be tiny.
Quoted 23 lines
Quoted 17 lines
Quoted 13 lines
Quoted 6 lines
With these assumptions, the output format becomes unambiguously parseable. Furthermore, given that this output is designed to be consumed by scripts, the machine-readable data is printed to stdout instead of stderr like the human-readable output is. This is mostly done so that other data printed to stderr, like error messages or progress meters, don't interfere with the parseable data.
Sending the 'main output' to stdout makes sense to me, but this (and possibly respecting -z) sounds like a different mode of operation, not just a matter of formats. It seems different enough that I'd prefer not to piggyback on "fetch.output" for this (even though this adds more surface to the interface...).
Quoted 3 lines
We could add --porcelain and say that "fetch.output" is ignored if --porcelain is also given. That also eliminates the need for --output-format, I think.
Quoted 5 lines
I was thinking about this initially, as well. But ultimately I decided against this especially because of your second paragraph: we'd now need to think about precedence of options and mutual exclusion, and that to me feels like an interface that is less obvious than a single knob that works as you'd expect.
Quoted 8 lines
My full thoughts on this are in
https://lore.kernel.org/git/[email protected]
but the short version is that I'm not sure if I expect something as innocuous-looking as --output-format would imply other, machine-friendly things (like stdout instead of stderr), and using --porcelain might make option precedence clearer in some situtations (like if -z is given).
I'm not even sure that -z makes sense in this context. If we see cases where not using -z can cause the machine-readable interface to become unparseable then this is a bug in the output format, if you ask me. Mostly because the whole intent of it is to be machine-parseable. So if we output data that can e.g. contain newlines, then we must not use newlines as part of the output format or alternatively escape them. Why let the author of the script shoot themselves into the foot?
Anyway, I'm digressing. It's hard for me to decide what to do right now. The thread with Junio and Jacob points into the direction of keeping the --output-format= interface, while this thread points into the other direction. I'm naturally more inclined to keep --output-format=, mostly because I personally feel like it's the more obvious interface. But I also see your point, so it's not really a choice of right-or-wrong here, but rather of style.
Patrick
Re: [PATCH 8/8] fetch: introduce machine-parseable "porcelain" output format
Quoted 7 lines · Patrick Steinhardt
Quoted 2 lines
I'm happy with either approach as long as we don't have to bikeshed about the "perfect" porcelain output :)
Agreed, and that's why I'm currently defending the "good enough" format. It should likely work for most usecases that exist out there. The target audience is going to be quite small here as this is not a user-directed feature.
Okay, I agree that this is "good enough" for most cases and we can extend it if needed.
Quoted 9 lines
Quoted 8 lines
My full thoughts on this are in
https://lore.kernel.org/git/[email protected]
but the short version is that I'm not sure if I expect something as innocuous-looking as --output-format would imply other, machine-friendly things (like stdout instead of stderr), and using --porcelain might make option precedence clearer in some situtations (like if -z is given).
Quoted 7 lines
I'm not even sure that
-zmakes sense in this context. If we see cases where not using-zcan cause the machine-readable interface to become unparseable then this is a bug in the output format, if you ask me. Mostly because the whole intent of it is to be machine-parseable. So if we output data that can e.g. contain newlines, then we must not use newlines as part of the output format or alternatively escape them. Why let the author of the script shoot themselves into the foot?
Agreed. I think the current output format is resilient enough, just that "-z" is something we choose to support for plumbing anyway.
Quoted 7 lines
Anyway, I'm digressing. It's hard for me to decide what to do right now. The thread with Junio and Jacob points into the direction of keeping the
--output-format=interface, while this thread points into the other direction. I'm naturally more inclined to keep--output-format=, mostly because I personally feel like it's the more obvious interface. But I also see your point, so it's not really a choice of right-or-wrong here, but rather of style.
Yes. All sides seem to understand the tradeoffs, but we value them differently.
I'd personally prefer to err on the side of consistency, because even if the existing behavior is less-than-ideal since:
1) This makes it possible for us to fix it in a consistent, well-thought out way, so we don't have to decide the future for the whole project in a git-fetch series.
2) At least it still behaves how _some_ users have come to expect it.
As for which is more consistent, a grep for --porcelain over Documentation/* shows:
- git-push
- git-status
- git-worktree
- git-blame
- git-commit
Most of them already output to stdout. The lone exception is git-push, which does exactly this "use stdout for the main output, but keep using stderr for debugging output" behavior you added, so using --porcelain seems somewhat consistent.
On the other hand, I couldn't find any other option that switches from stderr to stdout (we sometimes say --stdout to say "use stdout instead of a file", but that's different), so if we added this behavior to "git fetch --[output-]format", this might be the first.
(Sidenote: most instances of --format are for templated-style format, but "git-replace" does accept an enum for it. I don't think this should weigh heavily in our decision though.)
For the other, non git-push commands, --porcelain is treated like an output format and respects "last one wins" regular option precedence, e.g. in the case of "git status --porcelain --short". I find this behavior somewhat confusing (I'd expect --porcelain to win, or at least be incompatible), but at least we can consistently change this in the future [*].
That said, I'm willing to accept that I'm biased toward my own ideas. If others don't find "--format=porcelain implies stdout" confusing, I can accept that decision.
[*] In all likelihood, we probably can't make the change to --porcelain for backwards-compatibility reasons, but we could introduce a synonym with the behavior we want ("--machine"?) and have OPT_PARSE_MACHINE handle both --porcelain and --machine.
Re: [PATCH 8/8] fetch: introduce machine-parseable "porcelain" output format
In reply to Patrick SteinhardtQuoted 15 lines · Patrick Steinhardt
Quoted 8 lines · Glen Choo
Quoted 5 lines · Patrick Steinhardt
The output format is quite simple:
``` <flag> <old-object-id> <new-object-id> <local-reference>\n ```Quoted 5 lines
This format doesn't show the remote name or url that was fetched. That seems okay when fetching with a single remote, but it seems necessary with "--all". Perhaps you were planning to add that in a later series? If so, I think it's okay to call the "porcelain" format experimental, and forbid porcelain + --all until then.
Quoted 6 lines
The reason is mostly that I didn't find an output format that I really liked here. We'd basically have to repeat the remote URL for every single reference: just repeating it once per remote doesn't fly because with
--parallelthe output could be intermingled. But doing that feels wasteful to me, so I bailed. I guess I'm also biased here because it just wouldn't be useful to myself.
Couldn't each URL be assigned an index and then just show 0 in the output? I think it should be straightforward for a parser to figure out what 0 means but if not, that's something that could be figured out once the use case arrives (which very well could be never).
Signature · 1 line
Felipe Contreras
Quoted 11 lines · Patrick Steinhardt
Parsing the output of fetches is mostly impossible. It prettifies reference names that are about to be updated, doesn't print the old and new object IDs the refs are being updated from and to, and prints all of that information in nice columns. In short, it is designed to be read by humans rather than machines.
This makes it hard to use in a script way though, e.g. to learn about which references actually have been updated or which have not been updated. This patch series intends to fix that by introducing a new machine-parseable interface:
Quoted 9 lines
``` $ git fetch --output-format=porcelain --no-progress fff5a5e7f528b2ed2c335991399a766c2cf01103 af67688dca57999fd848f051eeea1d375ba546b2 refs/remotes/origin/master * 0000000000000000000000000000000000000000 e046fe5a36a970bc14fbfbcb2074a48776f6b671 refs/remotes/origin/x86-rep-insns * 0000000000000000000000000000000000000000 bb81ed6862b864c9eb99447f04d49a84ecb647e5 refs/tags/v6.3-rc4 * 0000000000000000000000000000000000000000 83af7b1468c0dca86b4dc9e43e73bfa4f38d9637 refs/tags/v6.3-rc5 * 0000000000000000000000000000000000000000 ab3affb8ed84f68638162fe7e6fd4055e15bff5b refs/tags/v6.3-rc6 * 0000000000000000000000000000000000000000 1c8c28415e8743368a2b800520a6dd0b22ee6ec2 refs/tags/v6.3-rc7 ```
Makes sense, my only question is what other format could git fetch have? I think --format=porcelain is clear enough.
Signature · 1 line
Felipe Contreras
Quoted 22 lines · Felipe Contreras
Quoted 11 lines · Patrick Steinhardt
Parsing the output of fetches is mostly impossible. It prettifies reference names that are about to be updated, doesn't print the old and new object IDs the refs are being updated from and to, and prints all of that information in nice columns. In short, it is designed to be read by humans rather than machines.
This makes it hard to use in a script way though, e.g. to learn about which references actually have been updated or which have not been updated. This patch series intends to fix that by introducing a new machine-parseable interface:
Quoted 9 lines
``` $ git fetch --output-format=porcelain --no-progress fff5a5e7f528b2ed2c335991399a766c2cf01103 af67688dca57999fd848f051eeea1d375ba546b2 refs/remotes/origin/master * 0000000000000000000000000000000000000000 e046fe5a36a970bc14fbfbcb2074a48776f6b671 refs/remotes/origin/x86-rep-insns * 0000000000000000000000000000000000000000 bb81ed6862b864c9eb99447f04d49a84ecb647e5 refs/tags/v6.3-rc4 * 0000000000000000000000000000000000000000 83af7b1468c0dca86b4dc9e43e73bfa4f38d9637 refs/tags/v6.3-rc5 * 0000000000000000000000000000000000000000 ab3affb8ed84f68638162fe7e6fd4055e15bff5b refs/tags/v6.3-rc6 * 0000000000000000000000000000000000000000 1c8c28415e8743368a2b800520a6dd0b22ee6ec2 refs/tags/v6.3-rc7 ```
Quoted 2 lines
Makes sense, my only question is what other format could
git fetchhave? I think--format=porcelainis clear enough.
Yeah, I'd be perfectly happy to rename this to --format=porcelain. I'll wait for the Review Club that discusses this patch set tomorrow and will send a new version with that change afterwards if nobody disagrees.
Patrick
Quoted 23 lines · Patrick Steinhardt
Quoted 22 lines · Felipe Contreras
Quoted 20 lines · Patrick Steinhardt
Parsing the output of fetches is mostly impossible. It prettifies reference names that are about to be updated, doesn't print the old and new object IDs the refs are being updated from and to, and prints all of that information in nice columns. In short, it is designed to be read by humans rather than machines.
This makes it hard to use in a script way though, e.g. to learn about which references actually have been updated or which have not been updated. This patch series intends to fix that by introducing a new machine-parseable interface:
``` $ git fetch --output-format=porcelain --no-progress fff5a5e7f528b2ed2c335991399a766c2cf01103 af67688dca57999fd848f051eeea1d375ba546b2 refs/remotes/origin/master * 0000000000000000000000000000000000000000 e046fe5a36a970bc14fbfbcb2074a48776f6b671 refs/remotes/origin/x86-rep-insns * 0000000000000000000000000000000000000000 bb81ed6862b864c9eb99447f04d49a84ecb647e5 refs/tags/v6.3-rc4 * 0000000000000000000000000000000000000000 83af7b1468c0dca86b4dc9e43e73bfa4f38d9637 refs/tags/v6.3-rc5 * 0000000000000000000000000000000000000000 ab3affb8ed84f68638162fe7e6fd4055e15bff5b refs/tags/v6.3-rc6 * 0000000000000000000000000000000000000000 1c8c28415e8743368a2b800520a6dd0b22ee6ec2 refs/tags/v6.3-rc7 ```
Quoted 8 lines
Quoted 2 lines
Makes sense, my only question is what other format could
git fetchhave? I think--format=porcelainis clear enough.Yeah, I'd be perfectly happy to rename this to
--format=porcelain. I'll wait for the Review Club that discusses this patch set tomorrow and will send a new version with that change afterwards if nobody disagrees.Patrick
We had some discussion during review club about this, where the idea of using "--porcelain" came up because many commands use that when switching into a machine readable format.
In addition, this format not only changes the output but also moves it from being on stderr to stdout, which is a hint that the intended usage of the command is now a little different.
I don't have strong opinion here but want to note that "--format" is often used by commands like log which changes how we structure the output of git objects.
Obviously using "--porcelain" is a bit weird when dealing with the pre-existing compact and full outputs, and perhaps --format wouldn't be confusing.
I didn't find any command which used --output-format today, and all the uses of --format I saw were for object formatting like git log.
I'm ok with --format, but wanted to note the potential confusion.
git status also uses --porcelain and has a -z option for NUL terminating instead of newline terminating. We thought -z might be useful to handle the potential of weird refnames that include characters. No one on the review could remember what rules were enforced on refnames to confirm if it was legal to have '\n' in a refname or not.
Quoted 6 lines · Jacob Keller
Quoted 5 lines
Yeah, I'd be perfectly happy to rename this to
--format=porcelain. I'll wait for the Review Club that discusses this patch set tomorrow and will send a new version with that change afterwards if nobody disagrees.Patrick
Quoted 7 lines
We had some discussion during review club about this, where the idea of using "--porcelain" came up because many commands use that when switching into a machine readable format.
In addition, this format not only changes the output but also moves it from being on stderr to stdout, which is a hint that the intended usage of the command is now a little different.
A little different from what? I do not think the answer would be "other program's --porcelain mode", as sending them to stdout would be one of the things that make the output easier for programs to parse, so it does sound like very much in the same spirit as "git status --porcelain" where its output format gets tweaked to be more machine friendly.
The output with "--porcelain" option enabled tend to be less human friendly and the distinction between Porcelain (for humans) and plumbing (for scripts) is reversed in the use of the word there---it started as "this is the option for those who write Porcelain commands to use", but still it is not a very good name for the option.
I am perfectly OK if the plan is to uniformly use --output-format (or something equally more descriptive) and migrate and deprecate the "--porcelain" option away from existing commands.
Thanks.
Quoted 16 lines · Junio C Hamano
Quoted 6 lines · Jacob Keller
Quoted 5 lines
Yeah, I'd be perfectly happy to rename this to
--format=porcelain. I'll wait for the Review Club that discusses this patch set tomorrow and will send a new version with that change afterwards if nobody disagrees.Patrick
Quoted 7 lines
We had some discussion during review club about this, where the idea of using "--porcelain" came up because many commands use that when switching into a machine readable format.
In addition, this format not only changes the output but also moves it from being on stderr to stdout, which is a hint that the intended usage of the command is now a little different.
Quoted 6 lines
A little different from what? I do not think the answer would be "other program's --porcelain mode", as sending them to stdout would be one of the things that make the output easier for programs to parse, so it does sound like very much in the same spirit as "git status --porcelain" where its output format gets tweaked to be more machine friendly.
A little different from using git fetch normally where all output is stderr and is generally "this is what I did" but which was obviously not intended to be parsed by scripts but instead by the human who ran the command.
Quoted 7 lines
The output with "--porcelain" option enabled tend to be less human friendly and the distinction between Porcelain (for humans) and plumbing (for scripts) is reversed in the use of the word there---it started as "this is the option for those who write Porcelain commands to use", but still it is not a very good name for the option.
Yea the option sort of means "to be used by those implementing porcelain" and its definitely a bit confusing.
Quoted 6 lines
I am perfectly OK if the plan is to uniformly use --output-format (or something equally more descriptive) and migrate and deprecate the "--porcelain" option away from existing commands.
Thanks.
That makes sense to me as well.
Quoted 23 lines · Junio C Hamano
Quoted 6 lines · Jacob Keller
Quoted 5 lines
Yeah, I'd be perfectly happy to rename this to
--format=porcelain. I'll wait for the Review Club that discusses this patch set tomorrow and will send a new version with that change afterwards if nobody disagrees.Patrick
Quoted 7 lines
We had some discussion during review club about this, where the idea of using "--porcelain" came up because many commands use that when switching into a machine readable format.
In addition, this format not only changes the output but also moves it from being on stderr to stdout, which is a hint that the intended usage of the command is now a little different.
A little different from what? I do not think the answer would be "other program's --porcelain mode", as sending them to stdout would be one of the things that make the output easier for programs to parse, so it does sound like very much in the same spirit as "git status --porcelain" where its output format gets tweaked to be more machine friendly.
Quoted 6 lines
The output with "--porcelain" option enabled tend to be less human friendly and the distinction between Porcelain (for humans) and plumbing (for scripts) is reversed in the use of the word there---it started as "this is the option for those who write Porcelain commands to use", but still it is not a very good name for the option.
Ah, thanks for explaining where this reversed meaning actually comes from. I really wondered why we use "porcelain" in preexisting code to reflect machine-parseable interface, but that explanation does make sense.
Quoted 3 lines
I am perfectly OK if the plan is to uniformly use --output-format (or something equally more descriptive) and migrate and deprecate the "--porcelain" option away from existing commands.
I'm still not quite clear on where the consensus lies now. Personally, I think that both --format and --output-format work well and are a bit more descriptive than simply --porcelain, and wouldn't mind also migrating other binaries to use either of them.
Furthermore, I think that --[output-]format has the advantage that you don't need to handle priorities or mutual exclusion of different options that all apply to the reference format. To a user, it is not immediately obvious what git fetch --format=compact --porcelain would do, and which of both options ultimately get respected. But that's likely only true for future commands, because any migration would create the same kind of ambiguity for preexisting commands.
If we were to also migrate preexisting code to use --[output-]format then I'd argue that --output-format is likely the better name, mostly because it is less likely to be ambiguous compared to --format. The latter could e.g. easily confused with --object-format.
So I think I'll stick with --output-format for the time being.
Patrick
Quoted 9 lines · Patrick Steinhardt
On Wed, Apr 26, 2023 at 01:23:12PM -0700, Junio C Hamano wrote:
Furthermore, I think that--[output-]formathas the advantage that you don't need to handle priorities or mutual exclusion of different options that all apply to the reference format. To a user, it is not immediately obvious whatgit fetch --format=compact --porcelainwould do, and which of both options ultimately get respected. But that's likely only true for future commands, because any migration would create the same kind of ambiguity for preexisting commands.
Quoted 7 lines
If we were to also migrate preexisting code to use
--[output-]formatthen I'd argue that--output-formatis likely the better name, mostly because it is less likely to be ambiguous compared to--format. The latter could e.g. easily confused with--object-format.So I think I'll stick with
--output-formatfor the time being.
I agree. I think using --output-format and migrating existing commands that have --porcelain to use --output-format is good. I'm not sure whether to keep using --output-format=porcelain or whether to use a different term there.
I definitely think avoiding confusion with --format is good, and I don't think its too much of a burden to have the longer --output-format as the option name.
Thanks,
Jake
Quoted 1 line
Patrick
Quoted 22 lines · Junio C Hamano
Quoted 7 lines
We had some discussion during review club about this, where the idea of using "--porcelain" came up because many commands use that when switching into a machine readable format.
In addition, this format not only changes the output but also moves it from being on stderr to stdout, which is a hint that the intended usage of the command is now a little different.
A little different from what? I do not think the answer would be "other program's --porcelain mode", as sending them to stdout would be one of the things that make the output easier for programs to parse, so it does sound like very much in the same spirit as "git status --porcelain" where its output format gets tweaked to be more machine friendly.
The output with "--porcelain" option enabled tend to be less human friendly and the distinction between Porcelain (for humans) and plumbing (for scripts) is reversed in the use of the word there---it started as "this is the option for those who write Porcelain commands to use", but still it is not a very good name for the option.
Quoted 3 lines
I am perfectly OK if the plan is to uniformly use --output-format (or something equally more descriptive) and migrate and deprecate the "--porcelain" option away from existing commands.
I agree that --porcelain is a confusing name that would be nice to deprecate, but I don't think --output-format captures all of the intent of "operate in a machine-friendly mode instead of a human-friendly one". Unfortunately, if we had picked --plumbing from the beginning, I doubt we would be having this discussion today :/
E.g. machines (Unix ones at least?) like to have output on stdout and to be able to request NUL-terminated output. It's unfortunate that if we don't piggyback onto --output-format, we run into option precedence problems (like Patrick mentioned), but I'd find it more confusing that --output-format=[porcelain|full|compact] don't behave the same way.
I don't think this puts us in a better spot with regards to option precedence either. Consider:
git fetch --output-format=full -z <...>
The only way to respect both options is to have -z affect the human-readable output, which isn't the end of the world, but it seems unnecessary.
Perhaps something like --machine instead?
Quoted 6 lines · Jacob Keller
Quoted 5 lines
Yeah, I'd be perfectly happy to rename this to
--format=porcelain. I'll wait for the Review Club that discusses this patch set tomorrow and will send a new version with that change afterwards if nobody disagrees.Patrick
Quoted 7 lines
We had some discussion during review club about this, where the idea of using "--porcelain" came up because many commands use that when switching into a machine readable format.
In addition, this format not only changes the output but also moves it from being on stderr to stdout, which is a hint that the intended usage of the command is now a little different.
A little different from what? I do not think the answer would be "other program's --porcelain mode", as sending them to stdout would be one of the things that make the output easier for programs to parse, so it does sound like very much in the same spirit as "git status --porcelain" where its output format gets tweaked to be more machine friendly.
The output with "--porcelain" option enabled tend to be less human friendly and the distinction between Porcelain (for humans) and plumbing (for scripts) is reversed in the use of the word there---it started as "this is the option for those who write Porcelain commands to use", but still it is not a very good name for the option.
I am perfectly OK if the plan is to uniformly use --output-format (or something equally more descriptive) and migrate and deprecate the "--porcelain" option away from existing commands.
Thanks.
Hi Patrick!
Thanks for the pleasant read! I thought this was a great topic for Review Club. It's too bad that we missed you, but we post all relevant feedback here anyway.
Nevertheless, if you'd like to see the meeting notes, you can find them at:
https://docs.google.com/document/d/14L8BAumGTpsXpjDY8VzZ4rRtpAjuGrFSRqn3stCuS_w/edit
Quoted 11 lines · Patrick Steinhardt
Parsing the output of fetches is mostly impossible. It prettifies reference names that are about to be updated, doesn't print the old and new object IDs the refs are being updated from and to, and prints all of that information in nice columns. In short, it is designed to be read by humans rather than machines.
This makes it hard to use in a script way though, e.g. to learn about which references actually have been updated or which have not been updated. This patch series intends to fix that by introducing a new machine-parseable interface:
Quoted 9 lines
``` $ git fetch --output-format=porcelain --no-progress fff5a5e7f528b2ed2c335991399a766c2cf01103 af67688dca57999fd848f051eeea1d375ba546b2 refs/remotes/origin/master * 0000000000000000000000000000000000000000 e046fe5a36a970bc14fbfbcb2074a48776f6b671 refs/remotes/origin/x86-rep-insns * 0000000000000000000000000000000000000000 bb81ed6862b864c9eb99447f04d49a84ecb647e5 refs/tags/v6.3-rc4 * 0000000000000000000000000000000000000000 83af7b1468c0dca86b4dc9e43e73bfa4f38d9637 refs/tags/v6.3-rc5 * 0000000000000000000000000000000000000000 ab3affb8ed84f68638162fe7e6fd4055e15bff5b refs/tags/v6.3-rc6 * 0000000000000000000000000000000000000000 1c8c28415e8743368a2b800520a6dd0b22ee6ec2 refs/tags/v6.3-rc7 ```
Having machine-parseable output seems like an obviously good goal to me. The finer points of the interface and output format are worth discussing. I'll do so in the patches themselves.
Overall, I found the series very well-structured and easy to follow along. Thanks.
Quoted 10 lines · Glen Choo
Hi Patrick!
Thanks for the pleasant read! I thought this was a great topic for Review Club. It's too bad that we missed you, but we post all relevant feedback here anyway.
Nevertheless, if you'd like to see the meeting notes, you can find them at:
https://docs.google.com/document/d/14L8BAumGTpsXpjDY8VzZ4rRtpAjuGrFSRqn3stCuS_w/edit
It seems like Jacob Keller and I have mostly the same comments (since we did both go to the same Review Club, after all :)), so feel free to respond to only Jacob's messages and ignore mine (where the messages overlap) - I'll catch up on those conversations.
Quoted 11 lines · Patrick Steinhardt
``` $ git fetch --output-format=porcelain --no-progress fff5a5e7f528b2ed2c335991399a766c2cf01103 af67688dca57999fd848f051eeea1d375ba546b2 refs/remotes/origin/master * 0000000000000000000000000000000000000000 e046fe5a36a970bc14fbfbcb2074a48776f6b671 refs/remotes/origin/x86-rep-insns * 0000000000000000000000000000000000000000 bb81ed6862b864c9eb99447f04d49a84ecb647e5 refs/tags/v6.3-rc4 * 0000000000000000000000000000000000000000 83af7b1468c0dca86b4dc9e43e73bfa4f38d9637 refs/tags/v6.3-rc5 * 0000000000000000000000000000000000000000 ab3affb8ed84f68638162fe7e6fd4055e15bff5b refs/tags/v6.3-rc6 * 0000000000000000000000000000000000000000 1c8c28415e8743368a2b800520a6dd0b22ee6ec2 refs/tags/v6.3-rc7 ```
One thing that the standard output (maybe with --progress?) shows to stderr is which remote is being fetched and what URL was fetched.
This seems like useful data to include in a machine readable format. It wasn't clear if such output would still be made and whether it would go to stderr or stdout, nor whether that existing output was machine readable.
Obviously you can somewhat infer remotes based on the refs/remotes, but that doesn't include other refs like tags.
When fetching from multiple reports (--all or explicitly asking), I wonder if it makes sense to include some lines to differentiate where each block of updates for a given remote starts and ends?
Thanks,
Jake
v2 cover fetch: introduce machine-parseable output
In reply to Patrick SteinhardtHi,
this is the second version of my patch series that introduces a machine-parseable output.
Changes compared to v1:
- Patch 3/8: I've reworded the commit message to hopefully be more straight-forward. I've also amended tests so that we don't only test with --dry-run, but also without.
- Patch 6/8: I've also moved the all and multiple variables from their global scope into cmd__fetch.
- Patch 7/8: Fixed a bug where --output-format= wasn't honored for child processes when doing multi-remote fetches. Furthermore, I've unified parsing of the actual format so that we don't have to repeat it thrice.
- Patch 8/8: Added a note to the commit message that tries to argue why I didn't add remote information to the interface. I'm still open to change this if you disagree with my reasoning here.
Thanks a bunch for all the feedback received so far, really appreciate it!
Patrick
Patrick Steinhardt (8):
fetch: split out tests for output format
fetch: add a test to exercise invalid output formats
fetch: fix missing from-reference when fetching HEAD:foo
fetch: introduce display_format enum
fetch: move display format parsing into main function
fetch: move option related variables into main function
fetch: introduce new --output-format option
fetch: introduce machine-parseable "porcelain" output format
Documentation/config/fetch.txt | 4 +- Documentation/fetch-options.txt | 5 + Documentation/git-fetch.txt | 17 +- builtin/fetch.c | 427 ++++++++++++++++++++------------ t/t5510-fetch.sh | 53 ---- t/t5574-fetch-output.sh | 237 ++++++++++++++++++ 6 files changed, 521 insertions(+), 222 deletions(-) create mode 100755 t/t5574-fetch-output.sh
Range-diff against v1:
1: 0d0d50d14c = 1: 0d0d50d14c fetch: split out tests for output format 2: 29d2c58914 = 2: 29d2c58914 fetch: add a test to exercise invalid output formats 3: 596e12f03a ! 3: d1fb6eeae7 fetch: fix missing from-reference when fetching HEAD:foo
@@ Metadata
## Commit message ##
fetch: fix missing from-reference when fetching HEAD:foo
- When displaying reference updates, we print a line that looks similar to
- the following:
+ store_updated_refs() parses the remote reference for two purposes:
+
+ - It gets used as a note when writing FETCH_HEAD.
+
+ - It is passed through to display_ref_update() to display
+ updated references in the following format:
+
+ ```
+ * branch master -> master
+ ```
+
+ In most cases, the parsed remote reference is the prettified reference
+ name and can thus be used for both cases. But if the remote reference is
+ HEAD, the parsed remote reference becomes empty. This is intended when
+ we write the FETCH_HEAD, where we skip writing the note in that case.
+ But it is not intended when displaying the updated references and would
+ cause us to miss the left-hand side of the displayed reference update:
```
- * branch master -> master
- ```
-
- The "branch" bit changes depending on what kind of reference we're
- updating, while both of the right-hand references are computed by
- stripping well-known prefixes like "refs/heads/" or "refs/tags".
-
- The logic is kind of intertwined though and not easy to follow: we
- precompute both the kind (e.g. "branch") and the what, which is the
- abbreviated remote reference name, in store_updated_refs() and then
- pass it down the call chain to display_ref_update().
-
- There is a set of different cases here:
-
- - When the remote reference name is "HEAD" we assume no kind and
- will thus instead print "[new ref]". We keep what at the empty
- string.
-
- - When the remote reference name has a well-known prefix then the
- kind would be "branch", "tag" or "remote-tracking branch". The
- what is the reference with the well-known prefix stripped and in
- fact matches the output that prettify_refname() would return.
-
- - Otherwise, we'll again assume no kind and keep the what set to the
- fully qualified reference name.
-
- Now there is a bug with the first case here, where the remote reference
- name is "HEAD". As noted, "what" will be set to the empty string. And
- that seems to be intentional because we also use this information to
- update the FETCH_HEAD, and in case we're updating HEAD we seemingly
- don't want to append that to our FETCH_HEAD value.
-
- But as mentioned, we also use this value to display reference updates.
- And while the call to display_ref_update() correctly figures out that
- we meant "HEAD" when what is empty, the call to update_local_ref()
- doesn't. update_local_ref() will then call display_ref_update() with
- the empty string and cause the following broken output:
-
- ```
- $ git fetch --dry-run origin HEAD:foo
+ $ git fetch origin HEAD:foo
From https://github.com/git/git
* [new ref] -> foo
```
The HEAD string is clearly missing from the left-hand side of the arrow,
- which is further stressed by the point that the following commands work
- as expected:
+ which is further stressed by the point that the following commands show
+ the left-hand side as expected:
```
- $ git fetch --dry-run origin HEAD
+ $ git fetch origin HEAD
From https://github.com/git/git
* branch HEAD -> FETCH_HEAD
- $ git fetch --dry-run origin master
+ $ git fetch origin master
From https://github.com/git/git
* branch master -> FETCH_HEAD
* branch master -> origin/master
```
- Fix this bug by instead unconditionally passing the full reference name
- to display_ref_update() which learns to call prettify_refname() on
- it. This does fix the above bug and is otherwise functionally the same
- as prettify_refname() would only ever strip the well-known prefixes
- just as intended. So at the same time, this also simplifies the code a
- bit.
+ The logic of how we compute the remote reference name that we ultimately
+ pass to display_ref_update() is not easy to follow. There are three
+ different cases here:
+
+ - When the remote reference name is "HEAD" we set the remote
+ reference name to the empty string. This is the case that causes
+ the bug to occur, where we would indeed want to print "HEAD"
+ instead of the empty string. This is what prettify_refname()
+ would return.
+
+ - When the remote reference name has a well-known prefix then we
+ strip this prefix. This matches what prettify_refname() does.
+
+ - Otherwise, we keep the fully qualified reference name. This also
+ matches what prettify_refname() does.
+
+ As the return value of prettify_refname() would do the correct thing
+ for us in all three cases, we can fix the bug by passing through the
+ full remote reference name to display_ref_update(), which learns to
+ call prettify_refname(). At the same time, this also simplifies the
+ code a bit.
Note that this patch also changes formatting of the block that computes the "kind" and "what" variables. This is done on purpose so that it is
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
+ EOF
+ test_cmp expect actual &&
+
++ git -C head fetch origin HEAD >actual 2>&1 &&
++ test_cmp expect actual &&
++
+ git -C head fetch --dry-run origin HEAD:foo >actual 2>&1 &&
+ cat >expect <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ * [new ref] HEAD -> foo
+ EOF
++ test_cmp expect actual &&
++
++ git -C head fetch origin HEAD:foo >actual 2>&1 &&
+ test_cmp expect actual
+'
+
4: 8571363be1 = 4: b545bf8bb9 fetch: introduce `display_format` enum
5: d98c3ee0ce = 5: 4990d35998 fetch: move display format parsing into main function
6: 640a8840e1 ! 6: cfe84129ab fetch: move option related variables into main function
@@ Commit message
Signed-off-by: Patrick Steinhardt <[email protected]>
## builtin/fetch.c ##
-@@ builtin/fetch.c: static int all, append, dry_run, force, keep, multiple, update_head_ok;
+@@ builtin/fetch.c: static int fetch_prune_tags_config = -1; /* unspecified */
+ static int prune_tags = -1; /* unspecified */
+ #define PRUNE_TAGS_BY_DEFAULT 0 /* do we prune tags by default? */
+
+-static int all, append, dry_run, force, keep, multiple, update_head_ok;
++static int append, dry_run, force, keep, update_head_ok;
static int write_fetch_head = 1;
static int verbosity, deepen_relative, set_upstream, refetch;
static int progress = -1;
@@ builtin/fetch.c: static int fetch_one(struct remote *remote, int argc, const cha
enum display_format display_format = DISPLAY_FORMAT_UNKNOWN;
struct string_list list = STRING_LIST_INIT_DUP;
struct remote *remote = NULL;
++ int all = 0, multiple = 0;
int result = 0;
int prune_tags_ok = 1;
+ int enable_auto_gc = 1;
7: 3b2cad066a ! 7: 0335e5eeb4 fetch: introduce new `--output-format` option
@@ Commit message
current mechanism feels a little bit indirect and rigid.
Introduce a new `--output-format` option that allows the user to change
- the desired output format more directly.
+ the desired format more directly.
Signed-off-by: Patrick Steinhardt <[email protected]>
@@ Documentation/fetch-options.txt: linkgit:git-config[1].
Write the list of remote refs fetched in the `FETCH_HEAD`
## builtin/fetch.c ##
+@@ builtin/fetch.c: enum display_format {
+ DISPLAY_FORMAT_UNKNOWN = 0,
+ DISPLAY_FORMAT_FULL,
+ DISPLAY_FORMAT_COMPACT,
++ DISPLAY_FORMAT_MAX,
++};
++
++static const char * const display_formats[DISPLAY_FORMAT_MAX] = {
++ NULL,
++ "full",
++ "compact",
+ };
+
+ struct display_state {
+@@ builtin/fetch.c: static int fetch_finished(int result, struct strbuf *out,
+ return 0;
+ }
+
+-static int fetch_multiple(struct string_list *list, int max_children)
++static int fetch_multiple(struct string_list *list, int max_children,
++ enum display_format format)
+ {
+ int i, result = 0;
+ struct strvec argv = STRVEC_INIT;
+@@ builtin/fetch.c: static int fetch_multiple(struct string_list *list, int max_children)
+ "--no-write-commit-graph", NULL);
+ add_options_to_argv(&argv);
+
++ if (format != DISPLAY_FORMAT_UNKNOWN)
++ strvec_pushf(&argv, "--output-format=%s", display_formats[format]);
++
+ if (max_children != 1 && list->nr != 1) {
+ struct parallel_fetch_state state = { argv.v, list, 0, 0 };
+ const struct run_process_parallel_opts opts = {
@@ builtin/fetch.c: static int fetch_one(struct remote *remote, int argc, const char **argv,
return exit_code;
}
++static enum display_format parse_display_format(const char *format)
++{
++ for (int i = 0; i < ARRAY_SIZE(display_formats); i++)
++ if (display_formats[i] && !strcmp(display_formats[i], format))
++ return i;
++ return DISPLAY_FORMAT_UNKNOWN;
++}
++
+static int opt_parse_output_format(const struct option *opt, const char *arg, int unset)
+{
-+ enum display_format *format = opt->value;
++ enum display_format *format = opt->value, parsed;
++
+ if (unset || !arg)
+ return 1;
-+ else if (!strcmp(arg, "full"))
-+ *format = DISPLAY_FORMAT_FULL;
-+ else if (!strcmp(arg, "compact"))
-+ *format = DISPLAY_FORMAT_COMPACT;
-+ else
++
++ parsed = parse_display_format(arg);
++ if (parsed == DISPLAY_FORMAT_UNKNOWN)
+ return error(_("unsupported output format '%s'"), arg);
++ *format = parsed;
+
+ return 0;
+}
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
N_("write fetched references to the FETCH_HEAD file")),
OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
+@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ const char *format = "full";
+
+ git_config_get_string_tmp("fetch.output", &format);
+- if (!strcasecmp(format, "full"))
+- display_format = DISPLAY_FORMAT_FULL;
+- else if (!strcasecmp(format, "compact"))
+- display_format = DISPLAY_FORMAT_COMPACT;
+- else
++
++ display_format = parse_display_format(format);
++ if (display_format == DISPLAY_FORMAT_UNKNOWN)
+ die(_("invalid value for '%s': '%s'"),
+ "fetch.output", format);
+ }
+@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ max_children = fetch_parallel_config;
+
+ /* TODO should this also die if we have a previous partial-clone? */
+- result = fetch_multiple(&list, max_children);
++ result = fetch_multiple(&list, max_children, display_format);
+ }
+
+
## t/t5574-fetch-output.sh ##
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch with invalid output format configuration' '
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch aligned output' '
cat >expect <<-\EOF &&
main -> origin/*
extraaa -> *
+@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
+ test_cmp expect actual
+ '
+
++test_expect_success 'fetch compact output with multiple remotes' '
++ test_when_finished "rm -rf compact-cfg compact-cli" &&
++
++ git clone . compact-cli &&
++ git -C compact-cli remote add second-remote "$PWD" &&
++ git clone . compact-cfg &&
++ git -C compact-cfg remote add second-remote "$PWD" &&
++ test_commit multi-commit &&
++
++ git -C compact-cfg -c fetch.output=compact fetch --all >actual-cfg 2>&1 &&
++ git -C compact-cli fetch --output-format=compact --all >actual-cli 2>&1 &&
++ test_cmp actual-cfg actual-cli &&
++
++ grep -e "->" actual-cfg | cut -c 22- >actual &&
++ cat >expect <<-\EOF &&
++ main -> origin/*
++ multi-commit -> *
++ main -> second-remote/*
++ EOF
++ test_cmp expect actual
++'
++
+ test_expect_success 'fetch output with HEAD and --dry-run' '
+ test_when_finished "rm -rf head" &&
+ git clone . head &&8: 301138da03 ! 8: d7c1bc1a80 fetch: introduce machine-parseable "porcelain" output format
@@ Commit message
so that other data printed to stderr, like error messages or progress meters, don't interfere with the parseable data.
+ A notable ommission here is that the output format does not include the
+ remote from which a reference was fetched, which might be important
+ information especially in the context of multi-remote fetches. But as
+ such a format would require us to print the remote for every single
+ reference update due to parallelizable fetches it feels wasteful for the
+ most likely usecase, which is when fetching from a single remote. If
+ usecases come up for this in the future though it is easy enough to add
+ a new "porcelain-v2" format that adds this information.
+
## Documentation/config/fetch.txt ##
@@ builtin/fetch.c: enum display_format {
DISPLAY_FORMAT_FULL,
DISPLAY_FORMAT_COMPACT,
+ DISPLAY_FORMAT_PORCELAIN,
+ DISPLAY_FORMAT_MAX,
+ };
+
+@@ builtin/fetch.c: static const char * const display_formats[DISPLAY_FORMAT_MAX] = {
+ NULL,
+ "full",
+ "compact",
++ "porcelain",
};
struct display_state {
@@ builtin/fetch.c: static int prune_refs(struct display_state *display_state,
summary_width);
warn_dangling_symref(stderr, dangling_msg, ref->name);
}
-@@ builtin/fetch.c: static int opt_parse_output_format(const struct option *opt, const char *arg, in
- *format = DISPLAY_FORMAT_FULL;
- else if (!strcmp(arg, "compact"))
- *format = DISPLAY_FORMAT_COMPACT;
-+ else if (!strcmp(arg, "porcelain"))
-+ *format = DISPLAY_FORMAT_PORCELAIN;
- else
- return error(_("unsupported output format '%s'"), arg);
-
-@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
- display_format = DISPLAY_FORMAT_FULL;
- else if (!strcasecmp(format, "compact"))
- display_format = DISPLAY_FORMAT_COMPACT;
-+ else if (!strcasecmp(format, "porcelain"))
-+ display_format = DISPLAY_FORMAT_PORCELAIN;
- else
- die(_("invalid value for '%s': '%s'"),
- "fetch.output", format);
## t/t5574-fetch-output.sh ##
-@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
+@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output with multiple remotes' '
test_cmp expect actual
'
Signature · 1 line
2.40.1
PATCH v2 1/8 fetch: split out tests for output format
We're about to introduce a new porcelain mode for the output of git-fetch(1). As part of that we'll be introducing a set of new tests that only relate to the output of this command.
Split out tests that exercise the output format of git-fetch(1) so that it becomes easier to verify this functionality as a standalone unit. As the tests assume that the default branch is called "main" we set up the corresponding GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME environment variable accordingly.
--- t/t5510-fetch.sh | 53 ---------------------------------- t/t5574-fetch-output.sh | 63 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+), 53 deletions(-) create mode 100755 t/t5574-fetch-output.sh
t/t5510-fetch.sh +0 -53
@@ -1118,59 +1118,6 @@ test_expect_success 'fetching with auto-gc does not lock up' ' ) ' -test_expect_success 'fetch aligned output' ' - git clone . full-output && - test_commit looooooooooooong-tag && - ( - cd full-output && - git -c fetch.output=full fetch origin >actual 2>&1 && - grep -e "->" actual | cut -c 22- >../actual - ) && - cat >expect <<-\EOF && - main -> origin/main - looooooooooooong-tag -> looooooooooooong-tag - EOF - test_cmp expect actual -' - -test_expect_success 'fetch compact output' ' - git clone . compact && - test_commit extraaa && - ( - cd compact && - git -c fetch.output=compact fetch origin >actual 2>&1 && - grep -e "->" actual | cut -c 22- >../actual - ) && - cat >expect <<-\EOF && - main -> origin/* - extraaa -> * - EOF - test_cmp expect actual -' - -test_expect_success '--no-show-forced-updates' ' - mkdir forced-updates && - ( - cd forced-updates && - git init && - test_commit 1 && - test_commit 2 - ) && - git clone forced-updates forced-update-clone && - git clone forced-updates no-forced-update-clone && - git -C forced-updates reset --hard HEAD~1 && - ( - cd forced-update-clone && - git fetch --show-forced-updates origin 2>output && - test_i18ngrep "(forced update)" output - ) && - ( - cd no-forced-update-clone && - git fetch --no-show-forced-updates origin 2>output && - test_i18ngrep ! "(forced update)" output - ) -' - for section in fetch transfer do test_expect_success "$section.hideRefs affects connectivity check" '
t/t5574-fetch-output.sh new file +63 -0
63 lines not on this page · raw message
Signature · 1 line
2.40.1
Re: [PATCH v2 1/8] fetch: split out tests for output format
Quoted 29 lines · Patrick Steinhardt
We're about to introduce a new porcelain mode for the output of git-fetch(1). As part of that we'll be introducing a set of new tests that only relate to the output of this command.
Split out tests that exercise the output format of git-fetch(1) so that it becomes easier to verify this functionality as a standalone unit. As the tests assume that the default branch is called "main" we set up the corresponding
GIT_TEST_DEFAULT_INITIAL_BRANCH_NAMEenvironment variable accordingly.Signed-off-by Patrick Steinhardt <[email protected]>--- t/t5510-fetch.sh | 53 ---------------------------------- t/t5574-fetch-output.sh | 63 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+), 53 deletions(-) create mode 100755 t/t5574-fetch-output.sht/t5574-fetch-output.sh new file +5
-0@@ -0,0 +1,63 @@ +#!/bin/sh + +test_description='git fetch output format' + +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
t/t5574-fetch-output.sh line 15
Quoted 10 lines
t/t5574-fetch-output.sh new file +10
-0+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME + +. ./test-lib.sh + +test_expect_success 'fetch aligned output' ' + git clone . full-output && + test_commit looooooooooooong-tag && + ( + cd full-output && + git -c fetch.output=full fetch origin >actual 2>&1 &&
Why is that 2>&1 redirection used here?
If the output the test case is interested in goes to the command's standard output, then it's unnecessary. However, if it goes to standard error, then why is standard output redirected as well?
I understand that this patch just moves existing test cases around as-is, so this is not something you introduced, but I point it out here, because later patches of this series add several new test cases following this anti-pattern.
Since this series creates a new test script, perhaps this might be the right time to clean this up.
Quoted 28 lines
t/t5574-fetch-output.sh new file +28
-0+ grep -e "->" actual | cut -c 22- >../actual + ) && + cat >expect <<-\EOF && + main -> origin/main + looooooooooooong-tag -> looooooooooooong-tag + EOF + test_cmp expect actual +' + +test_expect_success 'fetch compact output' ' + git clone . compact && + test_commit extraaa && + ( + cd compact && + git -c fetch.output=compact fetch origin >actual 2>&1 && + grep -e "->" actual | cut -c 22- >../actual + ) && + cat >expect <<-\EOF && + main -> origin/* + extraaa -> * + EOF + test_cmp expect actual +' + +test_expect_success '--no-show-forced-updates' ' + mkdir forced-updates && + ( + cd forced-updates &&
t/t5574-fetch-output.sh line 53
Quoted 10 lines
t/t5574-fetch-output.sh new file +10
-0+ git init && + test_commit 1 && + test_commit 2 + ) && + git clone forced-updates forced-update-clone && + git clone forced-updates no-forced-update-clone && + git -C forced-updates reset --hard HEAD~1 && + ( + cd forced-update-clone && + git fetch --show-forced-updates origin 2>output &&
Oh, look, there are some good examples to follow here as well :)
Quoted 13 lines
t/t5574-fetch-output.sh new file +10
-0+ test_i18ngrep "(forced update)" output + ) && + ( + cd no-forced-update-clone && + git fetch --no-show-forced-updates origin 2>output && + test_i18ngrep ! "(forced update)" output + ) +' + +test_doneSignature · 1 line
2.40.1
Re: [PATCH v2 1/8] fetch: split out tests for output format
Quoted 46 lines · SZEDER Gábor
Quoted 29 lines · Patrick Steinhardt
We're about to introduce a new porcelain mode for the output of git-fetch(1). As part of that we'll be introducing a set of new tests that only relate to the output of this command.
Split out tests that exercise the output format of git-fetch(1) so that it becomes easier to verify this functionality as a standalone unit. As the tests assume that the default branch is called "main" we set up the corresponding
GIT_TEST_DEFAULT_INITIAL_BRANCH_NAMEenvironment variable accordingly.Signed-off-by Patrick Steinhardt <[email protected]>--- t/t5510-fetch.sh | 53 ---------------------------------- t/t5574-fetch-output.sh | 63 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+), 53 deletions(-) create mode 100755 t/t5574-fetch-output.sht/t5574-fetch-output.sh new file +5
-0@@ -0,0 +1,63 @@ +#!/bin/sh + +test_description='git fetch output format' + +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=maint/t5574-fetch-output.sh line 15
Quoted 10 lines
t/t5574-fetch-output.sh new file +10
-0+export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME + +. ./test-lib.sh + +test_expect_success 'fetch aligned output' ' + git clone . full-output && + test_commit looooooooooooong-tag && + ( + cd full-output && + git -c fetch.output=full fetch origin >actual 2>&1 &&Why is that 2>&1 redirection used here?
If the output the test case is interested in goes to the command's standard output, then it's unnecessary. However, if it goes to standard error, then why is standard output redirected as well?
Quoted 7 lines
I understand that this patch just moves existing test cases around as-is, so this is not something you introduced, but I point it out here, because later patches of this series add several new test cases following this anti-pattern.
Since this series creates a new test script, perhaps this might be the right time to clean this up.
I feel like this patch series is big enough on its own already, so for now I'd like to refrain from touching up the old tests. But you've got a good point here, and I'll make sure that any new tests don't followw the same pattern.
That being said, I think we should keep on testing both stdout and stderr. The tests here are explicitly about the output format, so it does make sense to verify that both of them match what we expect. We should treat them as separate things though.
Patrick
PATCH v2 2/8 fetch: add a test to exercise invalid output formats
In reply to Patrick SteinhardtAdd a testcase that exercises the logic when an invalid output format is passed via the fetch.output configuration.
--- t/t5574-fetch-output.sh | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
t/t5574-fetch-output.sh +17 -0
@@ -7,6 +7,23 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME . ./test-lib.sh +test_expect_success 'fetch with invalid output format configuration' ' + test_when_finished "rm -rf clone" && + git clone . clone && + + test_must_fail git -C clone -c fetch.output= fetch origin >actual 2>&1 && + cat >expect <<-EOF && + fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}${SQ} + EOF + test_cmp expect actual && + + test_must_fail git -C clone -c fetch.output=garbage fetch origin >actual 2>&1 && + cat >expect <<-EOF && + fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}garbage${SQ} + EOF + test_cmp expect actual +' + test_expect_success 'fetch aligned output' ' git clone . full-output && test_commit looooooooooooong-tag &&
Signature · 1 line
2.40.1
PATCH v2 3/8 fetch: fix missing from-reference when fetching HEAD:foo
In reply to Patrick Steinhardtstore_updated_refs() parses the remote reference for two purposes:
- It gets used as a note when writing FETCH_HEAD.
- It is passed through to display_ref_update() to display updated references in the following format:
```
* branch master -> master
```
In most cases, the parsed remote reference is the prettified reference name and can thus be used for both cases. But if the remote reference is HEAD, the parsed remote reference becomes empty. This is intended when we write the FETCH_HEAD, where we skip writing the note in that case. But it is not intended when displaying the updated references and would cause us to miss the left-hand side of the displayed reference update:
``` $ git fetch origin HEAD:foo From https://github.com/git/git * [new ref] -> foo ```
The HEAD string is clearly missing from the left-hand side of the arrow, which is further stressed by the point that the following commands show the left-hand side as expected:
``` $ git fetch origin HEAD From https://github.com/git/git * branch HEAD -> FETCH_HEAD $ git fetch origin master From https://github.com/git/git * branch master -> FETCH_HEAD * branch master -> origin/master ```
The logic of how we compute the remote reference name that we ultimately pass to display_ref_update() is not easy to follow. There are three different cases here:
- When the remote reference name is "HEAD" we set the remote reference name to the empty string. This is the case that causes the bug to occur, where we would indeed want to print "HEAD" instead of the empty string. This is what prettify_refname() would return.
- When the remote reference name has a well-known prefix then we strip this prefix. This matches what prettify_refname() does.
- Otherwise, we keep the fully qualified reference name. This also matches what prettify_refname() does.
As the return value of prettify_refname() would do the correct thing for us in all three cases, we can fix the bug by passing through the full remote reference name to display_ref_update(), which learns to call prettify_refname(). At the same time, this also simplifies the code a bit.
Note that this patch also changes formatting of the block that computes the "kind" and "what" variables. This is done on purpose so that it is part of the diff, hopefully making the change easier to comprehend.
--- builtin/fetch.c | 37 +++++++++++++++++++------------------ t/t5574-fetch-output.sh | 25 +++++++++++++++++++++++++ 2 files changed, 44 insertions(+), 18 deletions(-)
builtin/fetch.c +19 -18
@@ -918,12 +918,14 @@ static void display_ref_update(struct display_state *display_state, char code, } width = (summary_width +strlen (summary) -gettext_width (summary)); + remote =prettify_refname (remote); + local =prettify_refname (local);strbuf_addf (&display_state->buf," %c %-*s " , code, width, summary);if (!display_state->compact_format) -print_remote_to_local (display_state, remote,prettify_refname (local)); +print_remote_to_local (display_state, remote, local);else -print_compact (display_state, remote,prettify_refname (local)); +print_compact (display_state, remote, local);if (error)strbuf_addf (&display_state->buf," (%s)" , error);strbuf_addch (&display_state->buf,'\n' );
@@ -934,7 +936,7 @@ static void display_ref_update(struct display_state *display_state, char code,static int update_local_ref (struct ref *ref,struct ref_transaction *transaction,struct display_state *display_state, -const char *remote,const struct ref *remote_ref, +const struct ref *remote_ref,int summary_width) {struct commit *current =NULL , *updated;
@@ -946,7 +948,7 @@ static int update_local_ref(struct ref *ref,if (oideq (&ref->old_oid, &ref->new_oid)) {if (verbosity >0 )display_ref_update (display_state,'=' ,_ ("[up to date]" ),NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 0 ; }
@@ -959,7 +961,7 @@ static int update_local_ref(struct ref *ref,*/ display_ref_update (display_state,'!' ,_ ("[rejected]" ),_ ("can't fetch into checked-out branch" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 1 ; }
@@ -970,12 +972,12 @@ static int update_local_ref(struct ref *ref, r =s_update_ref ("updating tag" , ref, transaction,0 );display_ref_update (display_state, r ?'!' :'t' ,_ ("[tag update]" ), r ?_ ("unable to update local ref" ) :NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return r; }else {display_ref_update (display_state,'!' ,_ ("[rejected]" ),_ ("would clobber existing tag" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 1 ; } }
@@ -1008,7 +1010,7 @@ static int update_local_ref(struct ref *ref, r =s_update_ref (msg, ref, transaction,0 );display_ref_update (display_state, r ?'!' :'*' , what, r ?_ ("unable to update local ref" ) :NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return r; }
@@ -1030,7 +1032,7 @@ static int update_local_ref(struct ref *ref, r =s_update_ref ("fast-forward" , ref, transaction,1 );display_ref_update (display_state, r ?'!' :' ' , quickref.buf, r ?_ ("unable to update local ref" ) :NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);strbuf_release (&quickref);return r; }else if (force || ref->force) {
@@ -1042,12 +1044,12 @@ static int update_local_ref(struct ref *ref, r =s_update_ref ("forced-update" , ref, transaction,1 );display_ref_update (display_state, r ?'!' :'+' , quickref.buf, r ?_ ("unable to update local ref" ) :_ ("forced update" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);strbuf_release (&quickref);return r; }else {display_ref_update (display_state,'!' ,_ ("[rejected]" ),_ ("non-fast-forward" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 1 ; } }
@@ -1252,14 +1254,13 @@ static int store_updated_refs(struct display_state *display_state,if (!strcmp (rm->name,"HEAD" )) { kind ="" ; what ="" ; - } -else if (skip_prefix (rm->name,"refs/heads/" , &what)) + }else if (skip_prefix (rm->name,"refs/heads/" , &what)) { kind ="branch" ; -else if (skip_prefix (rm->name,"refs/tags/" , &what)) + }else if (skip_prefix (rm->name,"refs/tags/" , &what)) { kind ="tag" ; -else if (skip_prefix (rm->name,"refs/remotes/" , &what)) + }else if (skip_prefix (rm->name,"refs/remotes/" , &what)) { kind ="remote-tracking branch" ; -else { + }else { kind ="" ; what = rm->name; }
@@ -1277,7 +1278,7 @@ static int store_updated_refs(struct display_state *display_state, display_state->url_len);if (ref) { - rc |=update_local_ref (ref, transaction, display_state, what, + rc |=update_local_ref (ref, transaction, display_state, rm, summary_width);free (ref); }else if (write_fetch_head || dry_run) {
@@ -1288,7 +1289,7 @@ static int store_updated_refs(struct display_state *display_state,*/ display_ref_update (display_state,'*' , *kind ? kind :"branch" ,NULL , - *what ? what :"HEAD" , + rm->name,"FETCH_HEAD" , summary_width); } }
t/t5574-fetch-output.sh +25 -0
@@ -54,6 +54,31 @@ test_expect_success 'fetch compact output' ' test_cmp expect actual ' +test_expect_success 'fetch output with HEAD and --dry-run' ' + test_when_finished "rm -rf head" && + git clone . head && + + git -C head fetch --dry-run origin HEAD >actual 2>&1 && + cat >expect <<-EOF && + From $(test-tool path-utils real_path .)/. + * branch HEAD -> FETCH_HEAD + EOF + test_cmp expect actual && + + git -C head fetch origin HEAD >actual 2>&1 && + test_cmp expect actual && + + git -C head fetch --dry-run origin HEAD:foo >actual 2>&1 && + cat >expect <<-EOF && + From $(test-tool path-utils real_path .)/. + * [new ref] HEAD -> foo + EOF + test_cmp expect actual && + + git -C head fetch origin HEAD:foo >actual 2>&1 && + test_cmp expect actual +' + test_expect_success '--no-show-forced-updates' ' mkdir forced-updates && (
Signature · 1 line
2.40.1
Re: [PATCH v2 3/8] fetch: fix missing from-reference when fetching HEAD:foo
For v2, I've only read through this patch for now. I'm still formulating my thoughts on the output format and the CLI option name.
Quoted 18 lines · Patrick Steinhardt
store_updated_refs()parses the remote reference for two purposes:- It gets used as a note when writing
FETCH_HEAD.- It is passed through to
display_ref_update()to display updated references in the following format:```
* branch master -> master
```In most cases, the parsed remote reference is the prettified reference name and can thus be used for both cases. But if the remote reference is HEAD, the parsed remote reference becomes empty. This is intended when we write the
FETCH_HEAD, where we skip writing the note in that case. But it is not intended when displaying the updated references and would cause us to miss the left-hand side of the displayed reference update:
Quoted 7 lines
[...]
As the return value of
prettify_refname()would do the correct thing for us in all three cases, we can fix the bug by passing through the full remote reference name todisplay_ref_update(), which learns to callprettify_refname(). At the same time, this also simplifies the code a bit.
This rewritten section explains the bug and the fix very nicely :) Thanks!
Quoted 11 lines
t/t5574-fetch-output.sh +3
-0@@ -54,6 +54,31 @@ test_expect_success 'fetch compact output' ' test_cmp expect actual ' +test_expect_success 'fetch output with HEAD and --dry-run' ' + test_when_finished "rm -rf head" && + git clone . head &&
t/t5574-fetch-output.sh test_expect_success 'fetch compact output' ' · line 69
Quoted 10 lines
t/t5574-fetch-output.sh +10
-0+ + git -C head fetch --dry-run origin HEAD >actual 2>&1 && + cat >expect <<-EOF && + From $(test-tool path-utils real_path .)/. + * branch HEAD -> FETCH_HEAD + EOF + test_cmp expect actual && + + git -C head fetch origin HEAD >actual 2>&1 && + test_cmp expect actual &&
Maybe rename the test to just 'fetch output with HEAD'? The 'and --dry-run' part seems completely obsolete.
Re: [PATCH v2 3/8] fetch: fix missing from-reference when fetching HEAD:foo
In reply to Patrick Steinhardtt/t5574-fetch-output.sh test_expect_success 'fetch compact output' ' · line 58
Quoted 9 lines · Patrick Steinhardt
index
0e45c27007..b9dcdade63100755t/t5574-fetch-output.sh +2
-0@@ -54,6 +54,31 @@ test_expect_success 'fetch compact output' ' test_cmp expect actual ' +test_expect_success 'fetch output with HEAD and --dry-run' ' + test_when_finished "rm -rf head" &&
The test name could drop the --dry-run mention since this now tests both variants.
Everything else looks good on this patch. The new commit message is much better to me now!
PATCH v2 4/8 fetch: introduce `display_format` enum
In reply to Patrick SteinhardtWe currently have two different display formats in git-fetch(1) with the "full" and "compact" formats. This is tracked with a boolean value that simply denotes whether the display format is supposed to be compacted or not. This works reasonably well while there are only two formats, but we're about to introduce another format that will make this a bit more awkward to use.
Introduce a enum display_format that is more readily extensible.
--- builtin/fetch.c | 101 ++++++++++++++++++++++++++++++------------------ 1 file changed, 64 insertions(+), 37 deletions(-)
builtin/fetch.c +64 -37
@@ -48,11 +48,17 @@ enum {TAGS_SET =2 }; +enum display_format { +DISPLAY_FORMAT_UNKNOWN =0 , +DISPLAY_FORMAT_FULL , +DISPLAY_FORMAT_COMPACT , +}; +struct display_state {struct strbuf buf;int refcol_width; -int compact_format; +enum display_format format;char *url;int url_len, shown_url;
@@ -784,7 +790,6 @@ static int refcol_width(const struct ref *ref, int compact_format)static void display_state_init (struct display_state *display_state,struct ref *ref_map,const char *raw_url) { -struct ref *rm;const char *format ="full" ;int i;
@@ -809,31 +814,42 @@ static void display_state_init(struct display_state *display_state, struct ref *git_config_get_string_tmp ("fetch.output" , &format);if (!strcasecmp (format,"full" )) - display_state->compact_format =0 ; + display_state->format =DISPLAY_FORMAT_FULL ;else if (!strcasecmp (format,"compact" )) - display_state->compact_format =1 ; + display_state->format =DISPLAY_FORMAT_COMPACT ;else die (_ ("invalid value for '%s': '%s'" ),"fetch.output" , format); - display_state->refcol_width =10 ; -for (rm = ref_map; rm; rm = rm->next) { -int width; +switch (display_state->format) { +case DISPLAY_FORMAT_FULL : +case DISPLAY_FORMAT_COMPACT : { +struct ref *rm; -if (rm->status ==REF_STATUS_REJECT_SHALLOW || - !rm->peer_ref || - !strcmp (rm->name,"HEAD" )) -continue ; + display_state->refcol_width =10 ; +for (rm = ref_map; rm; rm = rm->next) { +int width; - width =refcol_width (rm, display_state->compact_format); +if (rm->status ==REF_STATUS_REJECT_SHALLOW || + !rm->peer_ref || + !strcmp (rm->name,"HEAD" )) +continue ; -/* -* Not precise calculation for compact mode because '*' can -* appear on the left hand side of '->' and shrink the column -* back. -*/ -if (display_state->refcol_width < width) - display_state->refcol_width = width; + width =refcol_width (rm, display_state->format ==DISPLAY_FORMAT_COMPACT ); + +/* +* Not precise calculation for compact mode because '*' can +* appear on the left hand side of '->' and shrink the column +* back. +*/ +if (display_state->refcol_width < width) + display_state->refcol_width = width; + } + +break ; + } +default : +BUG ("unexpected display foramt %d" , display_state->format); } }
@@ -904,30 +920,41 @@ static void display_ref_update(struct display_state *display_state, char code, const char *remote, const char *local, int summary_width) { - int width; - if (verbosity < 0) return;strbuf_reset (&display_state->buf); -if (!display_state->shown_url) { -strbuf_addf (&display_state->buf,_ ("From %.*s\n" ), - display_state->url_len, display_state->url); - display_state->shown_url =1 ; +switch (display_state->format) { +case DISPLAY_FORMAT_FULL : +case DISPLAY_FORMAT_COMPACT : { +int width; + +if (!display_state->shown_url) { +strbuf_addf (&display_state->buf,_ ("From %.*s\n" ), + display_state->url_len, display_state->url); + display_state->shown_url =1 ; + } + + width = (summary_width +strlen (summary) -gettext_width (summary)); + remote =prettify_refname (remote); + local =prettify_refname (local); + +strbuf_addf (&display_state->buf," %c %-*s " , code, width, summary); + +if (display_state->format !=DISPLAY_FORMAT_COMPACT ) +print_remote_to_local (display_state, remote, local); +else +print_compact (display_state, remote, local); + +if (error) +strbuf_addf (&display_state->buf," (%s)" , error); + +break ; } - - width = (summary_width +strlen (summary) -gettext_width (summary)); - remote =prettify_refname (remote); - local =prettify_refname (local); - -strbuf_addf (&display_state->buf," %c %-*s " , code, width, summary); -if (!display_state->compact_format) -print_remote_to_local (display_state, remote, local); -else -print_compact (display_state, remote, local); -if (error) -strbuf_addf (&display_state->buf," (%s)" , error); +default : +BUG ("unexpected display format %d" , display_state->format); + };strbuf_addch (&display_state->buf,'\n' );fputs (display_state->buf.buf, stderr);
Signature · 1 line
2.40.1
PATCH v2 5/8 fetch: move display format parsing into main function
In reply to Patrick SteinhardtParsing the display format happens inside of display_state_init(). As we only need to check for a simple config entry, this is a natural location to put this code as it means that display-state logic is neatly contained in a single location.
We're about to introduce a output format though that is intended to be parseable by machines, for example inside of a script. In that case it becomes a bit awkward of an interface if you have to call git-fetch(1) with the fetch.output config key set. We're thus going to introduce a new --output-format switch for git-fetch(1) so that the output format can be configured more directly.
This means we'll have to hook parsing of the display format into the command line options parser. Having the code to determine the actual output format scattered across two different sites is hard to reason about though.
Refactor the code such that callers are expected to pass the display format that is to be used into display_state_init(). This allows us to lift up the code into the main function, where we can then hook it into command line options parser in a follow-up commit.
--- builtin/fetch.c | 41 ++++++++++++++++++++++++----------------- 1 file changed, 24 insertions(+), 17 deletions(-)
builtin/fetch.c +24 -17
@@ -788,14 +788,13 @@ static int refcol_width(const struct ref *ref, int compact_format) }static void display_state_init (struct display_state *display_state,struct ref *ref_map, -const char *raw_url) +const char *raw_url,enum display_format format) { -const char *format ="full" ;int i;memset (display_state,0 ,sizeof (*display_state)); -strbuf_init (&display_state->buf,0 ); + display_state->format = format;if (raw_url) display_state->url =transport_anonymize_url (raw_url);
@@ -812,15 +811,6 @@ static void display_state_init(struct display_state *display_state, struct ref *if (verbosity <0 )return ; -git_config_get_string_tmp ("fetch.output" , &format); -if (!strcasecmp (format,"full" )) - display_state->format =DISPLAY_FORMAT_FULL ; -else if (!strcasecmp (format,"compact" )) - display_state->format =DISPLAY_FORMAT_COMPACT ; -else -die (_ ("invalid value for '%s': '%s'" ), -"fetch.output" , format); -switch (display_state->format) {case DISPLAY_FORMAT_FULL :case DISPLAY_FORMAT_COMPACT : {
@@ -1614,7 +1604,8 @@ static int backfill_tags(struct display_state *display_state, }static int do_fetch (struct transport *transport, -struct refspec *rs) +struct refspec *rs, +enum display_format display_format) {struct ref_transaction *transaction =NULL ;struct ref *ref_map =NULL ;
@@ -1700,7 +1691,7 @@ static int do_fetch(struct transport *transport,if (retcode) goto cleanup; -display_state_init (&display_state, ref_map, transport->url); +display_state_init (&display_state, ref_map, transport->url, display_format);if (atomic_fetch) { transaction =ref_transaction_begin (&err);
@@ -2076,7 +2067,8 @@ static inline void fetch_one_setup_partial(struct remote *remote) }static int fetch_one (struct remote *remote,int argc,const char **argv, -int prune_tags_ok,int use_stdin_refspecs) +int prune_tags_ok,int use_stdin_refspecs, +enum display_format display_format) {struct refspec rs =REFSPEC_INIT_FETCH ;int i;
@@ -2143,7 +2135,7 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,sigchain_push_common (unlock_pack_on_signal);atexit (unlock_pack_atexit);sigchain_push (SIGPIPE ,SIG_IGN ); - exit_code =do_fetch (gtransport, &rs); + exit_code =do_fetch (gtransport, &rs, display_format);sigchain_pop (SIGPIPE );refspec_clear (&rs);transport_disconnect (gtransport);
@@ -2155,6 +2147,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) {int i;const char *bundle_uri; +enum display_format display_format =DISPLAY_FORMAT_UNKNOWN ;struct string_list list =STRING_LIST_INIT_DUP ;struct remote *remote =NULL ;int result =0 ;
@@ -2181,6 +2174,19 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) argc =parse_options (argc, argv, prefix, builtin_fetch_options, builtin_fetch_usage,0 ); +if (display_format ==DISPLAY_FORMAT_UNKNOWN ) { +const char *format ="full" ; + +git_config_get_string_tmp ("fetch.output" , &format); +if (!strcasecmp (format,"full" )) + display_format =DISPLAY_FORMAT_FULL ; +else if (!strcasecmp (format,"compact" )) + display_format =DISPLAY_FORMAT_COMPACT ; +else +die (_ ("invalid value for '%s': '%s'" ), +"fetch.output" , format); + } +if (recurse_submodules_cli !=RECURSE_SUBMODULES_DEFAULT ) recurse_submodules = recurse_submodules_cli;
@@ -2309,7 +2315,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) }else if (remote) {if (filter_options.choice ||has_promisor_remote ())fetch_one_setup_partial (remote); - result =fetch_one (remote, argc, argv, prune_tags_ok, stdin_refspecs); + result =fetch_one (remote, argc, argv, prune_tags_ok, stdin_refspecs, + display_format); }else {int max_children = max_jobs;
Signature · 1 line
2.40.1
PATCH v2 6/8 fetch: move option related variables into main function
In reply to Patrick SteinhardtThe options of git-fetch(1) which we pass to parse_options() are declared globally in builtin/fetch.c. This means we're forced to use global variables for all the options, which is more likely to cause confusion than explicitly passing state around.
Refactor the code to move the options into cmd_fetch(). Move variables that were previously forced to be declared globally and which are only used by cmd_fetch() into function-local scope.
--- builtin/fetch.c | 197 ++++++++++++++++++++++++------------------------ 1 file changed, 100 insertions(+), 97 deletions(-)
builtin/fetch.c +100 -97
@@ -75,13 +75,12 @@ static int fetch_prune_tags_config = -1; /* unspecified */static int prune_tags =-1 ;/* unspecified */ #define PRUNE_TAGS_BY_DEFAULT 0/* do we prune tags by default? */ -static int all, append, dry_run, force, keep, multiple, update_head_ok; +static int append, dry_run, force, keep, update_head_ok;static int write_fetch_head =1 ;static int verbosity, deepen_relative, set_upstream, refetch;static int progress =-1 ; -static int enable_auto_gc =1 ; -static int tags =TAGS_DEFAULT , unshallow, update_shallow, deepen; -static int max_jobs =-1 , submodule_fetch_jobs_config =-1 ; +static int tags =TAGS_DEFAULT , update_shallow, deepen; +static int submodule_fetch_jobs_config =-1 ;static int fetch_parallel_config =1 ;static int atomic_fetch;static enum transport_family family;
@@ -92,17 +91,11 @@ static struct string_list deepen_not = STRING_LIST_INIT_NODUP;static struct strbuf default_rla =STRBUF_INIT ;static struct transport *gtransport;static struct transport *gsecondary; -static const char *submodule_prefix ="" ;static int recurse_submodules =RECURSE_SUBMODULES_DEFAULT ; -static int recurse_submodules_cli =RECURSE_SUBMODULES_DEFAULT ; -static int recurse_submodules_default =RECURSE_SUBMODULES_ON_DEMAND ;static struct refspec refmap =REFSPEC_INIT_FETCH ;static struct list_objects_filter_options filter_options =LIST_OBJECTS_FILTER_INIT ;static struct string_list server_options =STRING_LIST_INIT_DUP ;static struct string_list negotiation_tip =STRING_LIST_INIT_NODUP ; -static int fetch_write_commit_graph =-1 ; -static int stdin_refspecs =0 ; -static int negotiate_only;static int git_fetch_config (const char *k,const char *v,void *cb) {
@@ -160,92 +153,6 @@ static int parse_refmap_arg(const struct option *opt, const char *arg, int unsetreturn 0 ; } -static struct option builtin_fetch_options[] = { -OPT__VERBOSITY (&verbosity), -OPT_BOOL (0 ,"all" , &all, -N_ ("fetch from all remotes" )), -OPT_BOOL (0 ,"set-upstream" , &set_upstream, -N_ ("set upstream for git pull/fetch" )), -OPT_BOOL ('a' ,"append" , &append, -N_ ("append to .git/FETCH_HEAD instead of overwriting" )), -OPT_BOOL (0 ,"atomic" , &atomic_fetch, -N_ ("use atomic transaction to update references" )), -OPT_STRING (0 ,"upload-pack" , &upload_pack,N_ ("path" ), -N_ ("path to upload pack on remote end" )), -OPT__FORCE (&force,N_ ("force overwrite of local reference" ),0 ), -OPT_BOOL ('m' ,"multiple" , &multiple, -N_ ("fetch from multiple remotes" )), -OPT_SET_INT ('t' ,"tags" , &tags, -N_ ("fetch all tags and associated objects" ),TAGS_SET ), -OPT_SET_INT ('n' ,NULL , &tags, -N_ ("do not fetch all tags (--no-tags)" ),TAGS_UNSET ), -OPT_INTEGER ('j' ,"jobs" , &max_jobs, -N_ ("number of submodules fetched in parallel" )), -OPT_BOOL (0 ,"prefetch" , &prefetch, -N_ ("modify the refspec to place all refs within refs/prefetch/" )), -OPT_BOOL ('p' ,"prune" , &prune, -N_ ("prune remote-tracking branches no longer on remote" )), -OPT_BOOL ('P' ,"prune-tags" , &prune_tags, -N_ ("prune local tags no longer on remote and clobber changed tags" )), -OPT_CALLBACK_F (0 ,"recurse-submodules" , &recurse_submodules_cli,N_ ("on-demand" ), -N_ ("control recursive fetching of submodules" ), -PARSE_OPT_OPTARG , option_fetch_parse_recurse_submodules), -OPT_BOOL (0 ,"dry-run" , &dry_run, -N_ ("dry run" )), -OPT_BOOL (0 ,"write-fetch-head" , &write_fetch_head, -N_ ("write fetched references to the FETCH_HEAD file" )), -OPT_BOOL ('k' ,"keep" , &keep,N_ ("keep downloaded pack" )), -OPT_BOOL ('u' ,"update-head-ok" , &update_head_ok, -N_ ("allow updating of HEAD ref" )), -OPT_BOOL (0 ,"progress" , &progress,N_ ("force progress reporting" )), -OPT_STRING (0 ,"depth" , &depth,N_ ("depth" ), -N_ ("deepen history of shallow clone" )), -OPT_STRING (0 ,"shallow-since" , &deepen_since,N_ ("time" ), -N_ ("deepen history of shallow repository based on time" )), -OPT_STRING_LIST (0 ,"shallow-exclude" , &deepen_not,N_ ("revision" ), -N_ ("deepen history of shallow clone, excluding rev" )), -OPT_INTEGER (0 ,"deepen" , &deepen_relative, -N_ ("deepen history of shallow clone" )), -OPT_SET_INT_F (0 ,"unshallow" , &unshallow, -N_ ("convert to a complete repository" ), -1 ,PARSE_OPT_NONEG ), -OPT_SET_INT_F (0 ,"refetch" , &refetch, -N_ ("re-fetch without negotiating common commits" ), -1 ,PARSE_OPT_NONEG ), - {OPTION_STRING ,0 ,"submodule-prefix" , &submodule_prefix,N_ ("dir" ), -N_ ("prepend this to submodule path output" ),PARSE_OPT_HIDDEN }, -OPT_CALLBACK_F (0 ,"recurse-submodules-default" , - &recurse_submodules_default,N_ ("on-demand" ), -N_ ("default for recursive fetching of submodules " -"(lower priority than config files)" ), -PARSE_OPT_HIDDEN , option_fetch_parse_recurse_submodules), -OPT_BOOL (0 ,"update-shallow" , &update_shallow, -N_ ("accept refs that update .git/shallow" )), -OPT_CALLBACK_F (0 ,"refmap" ,NULL ,N_ ("refmap" ), -N_ ("specify fetch refmap" ),PARSE_OPT_NONEG , parse_refmap_arg), -OPT_STRING_LIST ('o' ,"server-option" , &server_options,N_ ("server-specific" ),N_ ("option to transmit" )), -OPT_SET_INT ('4' ,"ipv4" , &family,N_ ("use IPv4 addresses only" ), -TRANSPORT_FAMILY_IPV4 ), -OPT_SET_INT ('6' ,"ipv6" , &family,N_ ("use IPv6 addresses only" ), -TRANSPORT_FAMILY_IPV6 ), -OPT_STRING_LIST (0 ,"negotiation-tip" , &negotiation_tip,N_ ("revision" ), -N_ ("report that we have only objects reachable from this object" )), -OPT_BOOL (0 ,"negotiate-only" , &negotiate_only, -N_ ("do not fetch a packfile; instead, print ancestors of negotiation tips" )), -OPT_PARSE_LIST_OBJECTS_FILTER (&filter_options), -OPT_BOOL (0 ,"auto-maintenance" , &enable_auto_gc, -N_ ("run 'maintenance --auto' after fetching" )), -OPT_BOOL (0 ,"auto-gc" , &enable_auto_gc, -N_ ("run 'maintenance --auto' after fetching" )), -OPT_BOOL (0 ,"show-forced-updates" , &fetch_show_forced_updates, -N_ ("check for forced-updates on all updated branches" )), -OPT_BOOL (0 ,"write-commit-graph" , &fetch_write_commit_graph, -N_ ("write the commit-graph after fetching" )), -OPT_BOOL (0 ,"stdin" , &stdin_refspecs, -N_ ("accept refspecs from stdin" )), -OPT_END () -}; -static void unlock_pack (unsigned int flags) {if (gtransport)
@@ -2145,13 +2052,109 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,int cmd_fetch (int argc,const char **argv,const char *prefix) { -int i;const char *bundle_uri; +const char *submodule_prefix ="" ;enum display_format display_format =DISPLAY_FORMAT_UNKNOWN ;struct string_list list =STRING_LIST_INIT_DUP ;struct remote *remote =NULL ; +int all =0 , multiple =0 ;int result =0 ;int prune_tags_ok =1 ; +int enable_auto_gc =1 ; +int unshallow =0 ; +int max_jobs =-1 ; +int recurse_submodules_cli =RECURSE_SUBMODULES_DEFAULT ; +int recurse_submodules_default =RECURSE_SUBMODULES_ON_DEMAND ; +int fetch_write_commit_graph =-1 ; +int stdin_refspecs =0 ; +int negotiate_only =0 ; +int i; + +struct option builtin_fetch_options[] = { +OPT__VERBOSITY (&verbosity), +OPT_BOOL (0 ,"all" , &all, +N_ ("fetch from all remotes" )), +OPT_BOOL (0 ,"set-upstream" , &set_upstream, +N_ ("set upstream for git pull/fetch" )), +OPT_BOOL ('a' ,"append" , &append, +N_ ("append to .git/FETCH_HEAD instead of overwriting" )), +OPT_BOOL (0 ,"atomic" , &atomic_fetch, +N_ ("use atomic transaction to update references" )), +OPT_STRING (0 ,"upload-pack" , &upload_pack,N_ ("path" ), +N_ ("path to upload pack on remote end" )), +OPT__FORCE (&force,N_ ("force overwrite of local reference" ),0 ), +OPT_BOOL ('m' ,"multiple" , &multiple, +N_ ("fetch from multiple remotes" )), +OPT_SET_INT ('t' ,"tags" , &tags, +N_ ("fetch all tags and associated objects" ),TAGS_SET ), +OPT_SET_INT ('n' ,NULL , &tags, +N_ ("do not fetch all tags (--no-tags)" ),TAGS_UNSET ), +OPT_INTEGER ('j' ,"jobs" , &max_jobs, +N_ ("number of submodules fetched in parallel" )), +OPT_BOOL (0 ,"prefetch" , &prefetch, +N_ ("modify the refspec to place all refs within refs/prefetch/" )), +OPT_BOOL ('p' ,"prune" , &prune, +N_ ("prune remote-tracking branches no longer on remote" )), +OPT_BOOL ('P' ,"prune-tags" , &prune_tags, +N_ ("prune local tags no longer on remote and clobber changed tags" )), +OPT_CALLBACK_F (0 ,"recurse-submodules" , &recurse_submodules_cli,N_ ("on-demand" ), +N_ ("control recursive fetching of submodules" ), +PARSE_OPT_OPTARG , option_fetch_parse_recurse_submodules), +OPT_BOOL (0 ,"dry-run" , &dry_run, +N_ ("dry run" )), +OPT_BOOL (0 ,"write-fetch-head" , &write_fetch_head, +N_ ("write fetched references to the FETCH_HEAD file" )), +OPT_BOOL ('k' ,"keep" , &keep,N_ ("keep downloaded pack" )), +OPT_BOOL ('u' ,"update-head-ok" , &update_head_ok, +N_ ("allow updating of HEAD ref" )), +OPT_BOOL (0 ,"progress" , &progress,N_ ("force progress reporting" )), +OPT_STRING (0 ,"depth" , &depth,N_ ("depth" ), +N_ ("deepen history of shallow clone" )), +OPT_STRING (0 ,"shallow-since" , &deepen_since,N_ ("time" ), +N_ ("deepen history of shallow repository based on time" )), +OPT_STRING_LIST (0 ,"shallow-exclude" , &deepen_not,N_ ("revision" ), +N_ ("deepen history of shallow clone, excluding rev" )), +OPT_INTEGER (0 ,"deepen" , &deepen_relative, +N_ ("deepen history of shallow clone" )), +OPT_SET_INT_F (0 ,"unshallow" , &unshallow, +N_ ("convert to a complete repository" ), +1 ,PARSE_OPT_NONEG ), +OPT_SET_INT_F (0 ,"refetch" , &refetch, +N_ ("re-fetch without negotiating common commits" ), +1 ,PARSE_OPT_NONEG ), + {OPTION_STRING ,0 ,"submodule-prefix" , &submodule_prefix,N_ ("dir" ), +N_ ("prepend this to submodule path output" ),PARSE_OPT_HIDDEN }, +OPT_CALLBACK_F (0 ,"recurse-submodules-default" , + &recurse_submodules_default,N_ ("on-demand" ), +N_ ("default for recursive fetching of submodules " +"(lower priority than config files)" ), +PARSE_OPT_HIDDEN , option_fetch_parse_recurse_submodules), +OPT_BOOL (0 ,"update-shallow" , &update_shallow, +N_ ("accept refs that update .git/shallow" )), +OPT_CALLBACK_F (0 ,"refmap" ,NULL ,N_ ("refmap" ), +N_ ("specify fetch refmap" ),PARSE_OPT_NONEG , parse_refmap_arg), +OPT_STRING_LIST ('o' ,"server-option" , &server_options,N_ ("server-specific" ),N_ ("option to transmit" )), +OPT_SET_INT ('4' ,"ipv4" , &family,N_ ("use IPv4 addresses only" ), +TRANSPORT_FAMILY_IPV4 ), +OPT_SET_INT ('6' ,"ipv6" , &family,N_ ("use IPv6 addresses only" ), +TRANSPORT_FAMILY_IPV6 ), +OPT_STRING_LIST (0 ,"negotiation-tip" , &negotiation_tip,N_ ("revision" ), +N_ ("report that we have only objects reachable from this object" )), +OPT_BOOL (0 ,"negotiate-only" , &negotiate_only, +N_ ("do not fetch a packfile; instead, print ancestors of negotiation tips" )), +OPT_PARSE_LIST_OBJECTS_FILTER (&filter_options), +OPT_BOOL (0 ,"auto-maintenance" , &enable_auto_gc, +N_ ("run 'maintenance --auto' after fetching" )), +OPT_BOOL (0 ,"auto-gc" , &enable_auto_gc, +N_ ("run 'maintenance --auto' after fetching" )), +OPT_BOOL (0 ,"show-forced-updates" , &fetch_show_forced_updates, +N_ ("check for forced-updates on all updated branches" )), +OPT_BOOL (0 ,"write-commit-graph" , &fetch_write_commit_graph, +N_ ("write the commit-graph after fetching" )), +OPT_BOOL (0 ,"stdin" , &stdin_refspecs, +N_ ("accept refspecs from stdin" )), +OPT_END () + };packet_trace_identity ("fetch" );
Signature · 1 line
2.40.1
Re: [PATCH v2 6/8] fetch: move option related variables into main function
Quoted 8 lines · Patrick Steinhardt
The options of git-fetch(1) which we pass to
parse_options()are declared globally inbuiltin/fetch.c. This means we're forced to use global variables for all the options, which is more likely to cause confusion than explicitly passing state around.Refactor the code to move the options into
cmd_fetch(). Move variables that were previously forced to be declared globally and which are only used bycmd_fetch()into function-local scope.
Very well done.
PATCH v2 7/8 fetch: introduce new `--output-format` option
In reply to Patrick SteinhardtIt is only possible to configure the output format that git-fetch(1) uses by setting it via a config key. While this interface may be fine as long as we only have the current "full" and "compact" output formats, where it is unlikely that the user will have to change them regularly. But we're about to introduce a new machine-parseable interface where the current mechanism feels a little bit indirect and rigid.
Introduce a new --output-format option that allows the user to change the desired format more directly.
--- Documentation/fetch-options.txt | 5 +++ builtin/fetch.c | 48 ++++++++++++++++++---- t/t5574-fetch-output.sh | 72 +++++++++++++++++++++++++++------ 3 files changed, 106 insertions(+), 19 deletions(-)
Documentation/fetch-options.txt +5 -0
@@ -78,6 +78,11 @@ linkgit:git-config[1]. --dry-run:: Show what would be done, without making any changes. +--output-format:: + Control how ref update status is printed. Valid values are + `full` and `compact`. Default value is `full`. See section + OUTPUT in linkgit:git-fetch[1] for detail. + ifndef::git-pull[] --[no-]write-fetch-head:: Write the list of remote refs fetched in the `FETCH_HEAD`
builtin/fetch.c +41 -7
90 lines not on this page · raw message
t/t5574-fetch-output.sh +60 -12
93 lines not on this page · raw message
Signature · 1 line
2.40.1
Re: [PATCH v2 7/8] fetch: introduce new `--output-format` option
Quoted 6 lines · Patrick Steinhardt
builtin/fetch.c +0
-0@@ -52,6 +52,13 @@ enum display_format { DISPLAY_FORMAT_UNKNOWN = 0,
builtin/fetch.c enum display_format { · line 62
Quoted 10 lines
builtin/fetch.c +7
-0DISPLAY_FORMAT_FULL ,DISPLAY_FORMAT_COMPACT , + DISPLAY_FORMAT_MAX, +}; + +static const char *const display_formats[DISPLAY_FORMAT_MAX ] = { +NULL , +"full" , +"compact" , };
Hmph, the _MAX thing that is only needed to size the array and never used elsewhere (i.e. parse_display_format() uses ARRAY_SIZE() of the thing, instead of the constant, and that is just fine) is an eyesore.
I wonder if
static const char *const display_format[] = {
[DISPLAY_FORMAT_UNKNOWN] = NULL,
[DISPLAY_FORMAT_FULL] = "full",
[DISPLAY_FORMAT_COMPACT] = "compact",
};would be easier to maintain?
I'll omit my usual "name your array in singular" lecture, as I think you've heard it already.
Thanks.
Re: [PATCH v2 7/8] fetch: introduce new `--output-format` option
Quoted 9 lines · Junio C Hamano
I wonder if
static const char *const display_format[] = { [DISPLAY_FORMAT_UNKNOWN] = NULL, [DISPLAY_FORMAT_FULL] = "full", [DISPLAY_FORMAT_COMPACT] = "compact", };would be easier to maintain?
It's easier to read, so I'd think so.
Re: [PATCH v2 7/8] fetch: introduce new `--output-format` option
Quoted 10 lines · Glen Choo
Quoted 8 lines · Junio C Hamano
I wonder if
static const char *const display_format[] = { [DISPLAY_FORMAT_UNKNOWN] = NULL, [DISPLAY_FORMAT_FULL] = "full", [DISPLAY_FORMAT_COMPACT] = "compact", };
Quoted 3 lines
Quoted 1 line
would be easier to maintain?
It's easier to read, so I'd think so.
Yeah, I'll adopt this approach in v3.
Patrick
Re: [PATCH v2 7/8] fetch: introduce new `--output-format` option
In reply to Patrick Steinhardtbuiltin/fetch.c static int fetch_multiple(struct string_list *list, int max_children) · line 1907
Quoted 7 lines · Patrick Steinhardt
builtin/fetch.c +3
-0@@ -1894,6 +1902,9 @@ static int fetch_multiple(struct string_list *list, int max_children) "--no-write-commit-graph", NULL);add_options_to_argv (&argv); +if (format !=DISPLAY_FORMAT_UNKNOWN ) +strvec_pushf (&argv,"--output-format=%s" , display_formats[format]); +
I think these lines belong inside add_options_to_argv(), since that's also used to prepare argv for fetch_submodules(), so we'd also get support for --recurse-submodules. (I wish I had spotted that in v1, sorry. Thankfully they use the same helper function, so we only have to do this once.)
----- >8 --------- >8 --------- >8 --------- >8 --------- >8 ----
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 422e29a914..7aa385aed5 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1796,8 +1796,11 @@ static int add_remote_or_group(const char *name, struct string_list *list)
return 1;
}
-static void add_options_to_argv(struct strvec *argv)
+static void add_options_to_argv(struct strvec *argv,
+ enum display_format format)
{
/* Maybe this shouldn't be first, idk */
+ if (format != DISPLAY_FORMAT_UNKNOWN)
+ strvec_pushf(argv, "--output-format=%s", display_formats[format]);
if (dry_run)
strvec_push(argv, "--dry-run");
if (prune != -1)
@@ -1908,10 +1911,7 @@ static int fetch_multiple(struct string_list *list, int max_children,
strvec_pushl(&argv, "-c", "fetch.bundleURI=",
"fetch", "--append", "--no-auto-gc",
"--no-write-commit-graph", NULL);
- add_options_to_argv(&argv);
-
- if (format != DISPLAY_FORMAT_UNKNOWN)
- strvec_pushf(&argv, "--output-format=%s", display_formats[format]);
+ add_options_to_argv(&argv, format); if (max_children != 1 && list->nr != 1) {
struct parallel_fetch_state state = { argv.v, list, 0, 0 }; @@ -2403,7 +2403,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)
if (max_children < 0)
max_children = fetch_parallel_config;
- add_options_to_argv(&options);
+ add_options_to_argv(&options, display_format);
result = fetch_submodules(the_repository,
&options,
submodule_prefix,----- >8 --------- >8 --------- >8 --------- >8 --------- >8 ----
I tested the result of that locally with --recurse-submodules, and it works.
Re: [PATCH v2 7/8] fetch: introduce new `--output-format` option
Quoted 57 lines · Glen Choo
builtin/fetch.c static int fetch_multiple(struct string_list *list, int max_children) · line 1907
Quoted 7 lines · Patrick Steinhardt
builtin/fetch.c +3
-0@@ -1894,6 +1902,9 @@ static int fetch_multiple(struct string_list *list, int max_children) "--no-write-commit-graph", NULL);add_options_to_argv (&argv); +if (format !=DISPLAY_FORMAT_UNKNOWN ) +strvec_pushf (&argv,"--output-format=%s" , display_formats[format]); +I think these lines belong inside
add_options_to_argv(), since that's also used to prepare argv forfetch_submodules(), so we'd also get support for --recurse-submodules. (I wish I had spotted that in v1, sorry. Thankfully they use the same helper function, so we only have to do this once.)----- >8 --------- >8 --------- >8 --------- >8 --------- >8 ----
diff --gita/builtin/fetch.cb/builtin/fetch.c
index422e29a914..7aa385aed5100644
---a/builtin/fetch.c
+++b/builtin/fetch.c
@@ -1796,8 +1796,11 @@ static intadd_remote_or_group(const char *name, struct string_list *list)return 1;}
-static void add_options_to_argv(struct strvec *argv) +static void add_options_to_argv(struct strvec *argv, + enum display_format format) { /* Maybe this shouldn't be first, idk */ + if (format != DISPLAY_FORMAT_UNKNOWN) + strvec_pushf(argv, "--output-format=%s", display_formats[format]); if (dry_run) strvec_push(argv, "--dry-run"); if (prune != -1) @@ -1908,10 +1911,7 @@ static int fetch_multiple(struct string_list *list, int max_children, strvec_pushl(&argv, "-c", "fetch.bundleURI=", "fetch", "--append", "--no-auto-gc", "--no-write-commit-graph", NULL); - add_options_to_argv(&argv); - - if (format != DISPLAY_FORMAT_UNKNOWN) - strvec_pushf(&argv, "--output-format=%s", display_formats[format]); + add_options_to_argv(&argv, format);if (max_children != 1 && list->nr != 1) { struct parallel_fetch_state state = { argv.v, list, 0, 0 };@@ -2403,7 +2403,7 @@ int
cmd_fetch(int argc, const char **argv, const char *prefix)
if (max_children< 0)
max_children=fetch_parallel_config;- add_options_to_argv(&options); + add_options_to_argv(&options, display_format); result = fetch_submodules(the_repository, &options, submodule_prefix,
Quoted 4 lines
----- >8 --------- >8 --------- >8 --------- >8 --------- >8 ----
I tested the result of that locally with --recurse-submodules, and it works.
Unfortunately it doesn't quite work alright: while the porcelain format does indeed get inherited to the child process correctly, the parallel process API will cause us to group output per submodule-fetch. This has the consequence that stdout will be redirected into stderr, and that then breaks the assumption that all machine-parseable output goes to stdout.
My initial reflex is to just outright reject porcelain mode when submodule fetches are enabled. But that would require the caller to always explicitly pass --recurse-submodules=off, which isn't exactly great usability-wise.
The alternative would be to ungroup the output so that we can continue to print to the correct output streams. That works alright, and I've got a working version that does exactly that. But now we have the issue that the porcelain output is misleading: you cannot tell whether a specific reference update happens in the parent repository or in the submodule as that information is not part of the output.
I consider the second option to be much worse than the first option because it can cause scripts do to the wrong thing. So I'll send v3 with the first option, even though it's kind of an awful workaround. I'd be happy to hear any alternative proposals though.
Patrick
Re: [PATCH v2 7/8] fetch: introduce new `--output-format` option
Quoted 82 lines · Patrick Steinhardt
Quoted 57 lines · Glen Choo
builtin/fetch.c static int fetch_multiple(struct string_list *list, int max_children) · line 1907
Quoted 7 lines · Patrick Steinhardt
builtin/fetch.c +3
-0@@ -1894,6 +1902,9 @@ static int fetch_multiple(struct string_list *list, int max_children) "--no-write-commit-graph", NULL);add_options_to_argv (&argv); +if (format !=DISPLAY_FORMAT_UNKNOWN ) +strvec_pushf (&argv,"--output-format=%s" , display_formats[format]); +I think these lines belong inside
add_options_to_argv(), since that's also used to prepare argv forfetch_submodules(), so we'd also get support for --recurse-submodules. (I wish I had spotted that in v1, sorry. Thankfully they use the same helper function, so we only have to do this once.)----- >8 --------- >8 --------- >8 --------- >8 --------- >8 ----
diff --gita/builtin/fetch.cb/builtin/fetch.c
index422e29a914..7aa385aed5100644
---a/builtin/fetch.c
+++b/builtin/fetch.c
@@ -1796,8 +1796,11 @@ static intadd_remote_or_group(const char *name, struct string_list *list)return 1;}
-static void add_options_to_argv(struct strvec *argv) +static void add_options_to_argv(struct strvec *argv, + enum display_format format) { /* Maybe this shouldn't be first, idk */ + if (format != DISPLAY_FORMAT_UNKNOWN) + strvec_pushf(argv, "--output-format=%s", display_formats[format]); if (dry_run) strvec_push(argv, "--dry-run"); if (prune != -1) @@ -1908,10 +1911,7 @@ static int fetch_multiple(struct string_list *list, int max_children, strvec_pushl(&argv, "-c", "fetch.bundleURI=", "fetch", "--append", "--no-auto-gc", "--no-write-commit-graph", NULL); - add_options_to_argv(&argv); - - if (format != DISPLAY_FORMAT_UNKNOWN) - strvec_pushf(&argv, "--output-format=%s", display_formats[format]); + add_options_to_argv(&argv, format);if (max_children != 1 && list->nr != 1) { struct parallel_fetch_state state = { argv.v, list, 0, 0 };@@ -2403,7 +2403,7 @@ int
cmd_fetch(int argc, const char **argv, const char *prefix)
if (max_children< 0)
max_children=fetch_parallel_config;- add_options_to_argv(&options); + add_options_to_argv(&options, display_format); result = fetch_submodules(the_repository, &options, submodule_prefix,Quoted 4 lines
----- >8 --------- >8 --------- >8 --------- >8 --------- >8 ----
I tested the result of that locally with --recurse-submodules, and it works.
Unfortunately it doesn't quite work alright: while the porcelain format does indeed get inherited to the child process correctly, the parallel process API will cause us to group output per submodule-fetch. This has the consequence that stdout will be redirected into stderr, and that then breaks the assumption that all machine-parseable output goes to stdout.
My initial reflex is to just outright reject porcelain mode when submodule fetches are enabled. But that would require the caller to always explicitly pass
--recurse-submodules=off, which isn't exactly great usability-wise.The alternative would be to ungroup the output so that we can continue to print to the correct output streams. That works alright, and I've got a working version that does exactly that. But now we have the issue that the porcelain output is misleading: you cannot tell whether a specific reference update happens in the parent repository or in the submodule as that information is not part of the output.
Quoted 6 lines
I consider the second option to be much worse than the first option because it can cause scripts do to the wrong thing. So I'll send v3 with the first option, even though it's kind of an awful workaround. I'd be happy to hear any alternative proposals though.
Patrick
I've gone with a slightly different variant of the first option that is inspired by --negotiate-only: instead of refusing to run, we disable submodule-fetches unless explicitly specified on the command line. In that case we return an error.
Patrick
PATCH v2 8/8 fetch: introduce machine-parseable "porcelain" output format
In reply to Patrick SteinhardtThe output of git-fetch(1) is obviously designed for consumption by users, only: we neatly columnize data, we abbreviate reference names, we print neat arrows and we don't provide information about actual object IDs that have changed. This makes the output format basically unusable in the context of scripted invocations of git-fetch(1) that want to learn about the exact changes that the command performs.
Introduce a new machine-parseable "porcelain" output format that is supposed to fix this shortcoming. This output format is intended to provide information about every reference that is about to be updated, the old object ID that the reference has been pointing to and the new object ID it will be updated to. Furthermore, the output format provides the same flags as the human-readable format to indicate basic conditions for each reference update like whether it was a fast-forward update, a branch deletion, a rejected update or others.
The output format is quite simple:
``` <flag> <old-object-id> <new-object-id> <local-reference>\n ```
We assume two conditions which are generally true:
- The old and new object IDs have fixed known widths and cannot contain spaces.
- References cannot contain newlines.
With these assumptions, the output format becomes unambiguously parseable. Furthermore, given that this output is designed to be consumed by scripts, the machine-readable data is printed to stdout instead of stderr like the human-readable output is. This is mostly done so that other data printed to stderr, like error messages or progress meters, don't interfere with the parseable data.
A notable ommission here is that the output format does not include the remote from which a reference was fetched, which might be important information especially in the context of multi-remote fetches. But as such a format would require us to print the remote for every single reference update due to parallelizable fetches it feels wasteful for the most likely usecase, which is when fetching from a single remote. If usecases come up for this in the future though it is easy enough to add a new "porcelain-v2" format that adds this information.
--- Documentation/config/fetch.txt | 4 +- Documentation/fetch-options.txt | 4 +- Documentation/git-fetch.txt | 17 ++++++- builtin/fetch.c | 47 +++++++++++++----- t/t5574-fetch-output.sh | 84 +++++++++++++++++++++++++++++++++ 5 files changed, 139 insertions(+), 17 deletions(-)
Documentation/config/fetch.txt +2 -2
@@ -52,8 +52,8 @@ fetch.pruneTags:: fetch.output:: Control how ref update status is printed. Valid values are - `full` and `compact`. Default value is `full`. See section - OUTPUT in linkgit:git-fetch[1] for detail. + `full`, `compact` and `porcelain`. Default value is `full`. + See section OUTPUT in linkgit:git-fetch[1] for detail. fetch.negotiationAlgorithm:: Control how information about the commits in the local repository
Documentation/fetch-options.txt +2 -2
@@ -80,8 +80,8 @@ linkgit:git-config[1]. --output-format:: Control how ref update status is printed. Valid values are - `full` and `compact`. Default value is `full`. See section - OUTPUT in linkgit:git-fetch[1] for detail. + `full`, `compact` and `porcelain`. Default value is `full`. + See section OUTPUT in linkgit:git-fetch[1] for detail. ifndef::git-pull[] --[no-]write-fetch-head::
Documentation/git-fetch.txt +15 -2
@@ -197,13 +197,26 @@ The output of "git fetch" depends on the transport method used; this section describes the output when fetching over the Git protocol (either locally or via ssh) and Smart HTTP protocol. -The status of the fetch is output in tabular form, with each line -representing the status of a single ref. Each line is of the form: +The output format can be chosen either via the `fetch.output` config +(see linkgit:git-config[1]), or via the `--output-format` switch. +Supported values include: + +For the `full` and `compact` output formats, the status of the fetch is +output in tabular, with each line representing the status of a single +ref. Each line is of the form: ------------------------------- <flag> <summary> <from> -> <to> [<reason>] ------------------------------- +The `porcelain` output format is intended to be machine-parseable. In +contrast to the human-readable output formats it thus prints to standard +output instead of standard error. Each line is of the form: + +------------------------------- +<flag> <old-object-id> <new-object-id> <local-reference> +------------------------------- + The status of up-to-date refs is shown only if the --verbose option is used.
builtin/fetch.c +36 -11
141 lines not on this page · raw message
t/t5574-fetch-output.sh +84 -0
96 lines not on this page · raw message
Signature · 1 line
2.40.1
Re: [PATCH v2 8/8] fetch: introduce machine-parseable "porcelain" output format
Quoted 8 lines · Patrick Steinhardt
A notable ommission here is that the output format does not include the remote from which a reference was fetched, which might be important information especially in the context of multi-remote fetches. But as such a format would require us to print the remote for every single reference update due to parallelizable fetches it feels wasteful for the most likely usecase, which is when fetching from a single remote. If usecases come up for this in the future though it is easy enough to add a new "porcelain-v2" format that adds this information.
Thanks for clarifying this. I think generally its ok to leave out since you can often infer what remote a ref came from (either because of refs/remotes, or because of how you configured your refspecs for fetching).
We do have the option to extend this in the future if needed, so I think that's ok. In that case if we wanted an even more verbose output I'd probably argue for something more structured like adding it as part of a JSON output.
Thanks,
Jake
Re: [PATCH v2 8/8] fetch: introduce machine-parseable "porcelain" output format
In reply to Patrick SteinhardtQuoted 8 lines · Patrick Steinhardt
A notable ommission here is that the output format does not include the remote from which a reference was fetched, which might be important information especially in the context of multi-remote fetches. But as such a format would require us to print the remote for every single reference update due to parallelizable fetches it feels wasteful for the most likely usecase, which is when fetching from a single remote. If usecases come up for this in the future though it is easy enough to add a new "porcelain-v2" format that adds this information.
We discussed this elsewhere in the thread, but if we are just adding information (and not omitting existing information or shuffling it around), I would prefer for us to make the format extensible using flags than to add a whole new format enum. We can't imagine what other information users might want (maybe the remote ref name?), and it would be nice to avoid bumping the 'porcelain version' unnecessarily.
I agree that this format is good enough as a starting point, though.
It's not new to v2, but I've mentioned my reservations on --output-format on:
https://lore.kernel.org/git/[email protected]/
I will be out of office all of next week. If, in the meantime, everyone else decides that --output-format=porcelain is good enough, I'm happy to accept the result.
Quoted 12 lines
t/t5574-fetch-output.sh +8
-0@@ -127,6 +193,24 @@ test_expect_success 'fetch output with HEAD and --dry-run' ' test_cmp expect actual ' +test_expect_success 'fetch porcelain output with HEAD and --dry-run' ' + test_when_finished "rm -rf head" && + git clone . head && + COMMIT_ID=$(git rev-parse HEAD) && + + git -C head fetch --output-format=porcelain --dry-run origin HEAD >actual && + cat >expect <<-EOF && + * $ZERO_OID $COMMIT_ID FETCH_HEAD
t/t5574-fetch-output.sh test_expect_success 'fetch output with HEAD and --dry-run' ' · line 213
Quoted 10 lines
t/t5574-fetch-output.sh +10
-0+ EOF + test_cmp expect actual && + + git -C head fetch --output-format=porcelain --dry-run origin HEAD:foo >actual && + cat >expect <<-EOF && + * $ZERO_OID $COMMIT_ID refs/heads/foo + EOF + test_cmp expect actual +' +
Now that the earlier test also tests without --dry-run, shouldn't this one too?
v3 cover fetch: introduce machine-parseable output
In reply to Patrick SteinhardtHi,
this is the third version of my patch series to introduce a machine-parseable output for git-fetch(1).
Changes compared to v2:
- A new commit that fixes --no-recurse-submodules not being honored for multi-remote fetches was added. This fixes issues that would otherwise be hit later in the patch series.
- Tests were adjusted to explicitly verify stdout and stderr as standalone units instead of redirecting 2>&1.
- Instead of introducing --output-format and a new "porcelain" format I've changed this to instead introduce --porcelain. What finally convinced me to go this way is Glen's list of preexisting commands that have --porcelain. So I decided to not be the only outlier here.
- The DISPLAY_FORMAT_MAX macro was dropped in favor of Junio's proposal that uses designated initializers for the array.
- Similar to --negotiate-only, the porcelain output is now incompatible with --recurse-submodules=[yes|on-demand]. This is done to avoid ambiguity when references in both the superproject and any of its submodules is updated.
Thanks again for all the feedback!
Patrick
Patrick Steinhardt (8):
fetch: fix --no-recurse-submodules with multi-remote fetches
fetch: split out tests for output format
fetch: add a test to exercise invalid output formats
fetch: fix missing from-reference when fetching HEAD:foo
fetch: introduce display_format enum
fetch: move display format parsing into main function
fetch: move option related variables into main function
fetch: introduce machine-parseable "porcelain" output format
Documentation/fetch-options.txt | 6 + Documentation/git-fetch.txt | 9 + builtin/fetch.c | 437 +++++++++++++++++++------------- t/t5510-fetch.sh | 53 ---- t/t5526-fetch-submodules.sh | 31 +++ t/t5574-fetch-output.sh | 231 +++++++++++++++++ 6 files changed, 543 insertions(+), 224 deletions(-) create mode 100755 t/t5574-fetch-output.sh
Range-diff against v2:
-: ---------- > 1: 4b2b0cfe15 fetch: fix `--no-recurse-submodules` with multi-remote fetches
1: 0d0d50d14c = 2: 6ebc7450ba fetch: split out tests for output format
2: 29d2c58914 ! 3: 78479922ac fetch: add a test to exercise invalid output formats
@@ t/t5574-fetch-output.sh: export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
+ test_when_finished "rm -rf clone" &&
+ git clone . clone &&
+
-+ test_must_fail git -C clone -c fetch.output= fetch origin >actual 2>&1 &&
++ test_must_fail git -C clone -c fetch.output= fetch origin >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}${SQ}
+ EOF
-+ test_cmp expect actual &&
++ test_must_be_empty actual.out &&
++ test_cmp expect actual.err &&
+
-+ test_must_fail git -C clone -c fetch.output=garbage fetch origin >actual 2>&1 &&
++ test_must_fail git -C clone -c fetch.output=garbage fetch origin >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}garbage${SQ}
+ EOF
-+ test_cmp expect actual
++ test_must_be_empty actual.out &&
++ test_cmp expect actual.err
+'
+
test_expect_success 'fetch aligned output' '
3: d1fb6eeae7 ! 4: 46e1266ab0 fetch: fix missing from-reference when fetching HEAD:foo
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
test_cmp expect actual
'
-+test_expect_success 'fetch output with HEAD and --dry-run' '
++test_expect_success 'fetch output with HEAD' '
+ test_when_finished "rm -rf head" &&
+ git clone . head &&
+
-+ git -C head fetch --dry-run origin HEAD >actual 2>&1 &&
++ git -C head fetch --dry-run origin HEAD >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ * branch HEAD -> FETCH_HEAD
+ EOF
-+ test_cmp expect actual &&
++ test_must_be_empty actual.out &&
++ test_cmp expect actual.err &&
+
-+ git -C head fetch origin HEAD >actual 2>&1 &&
-+ test_cmp expect actual &&
++ git -C head fetch origin HEAD >actual.out 2>actual.err &&
++ test_must_be_empty actual.out &&
++ test_cmp expect actual.err &&
+
-+ git -C head fetch --dry-run origin HEAD:foo >actual 2>&1 &&
++ git -C head fetch --dry-run origin HEAD:foo >actual.out 2>actual.err &&
+ cat >expect <<-EOF &&
+ From $(test-tool path-utils real_path .)/.
+ * [new ref] HEAD -> foo
+ EOF
-+ test_cmp expect actual &&
++ test_must_be_empty actual.out &&
++ test_cmp expect actual.err &&
+
-+ git -C head fetch origin HEAD:foo >actual 2>&1 &&
-+ test_cmp expect actual
++ git -C head fetch origin HEAD:foo >actual.out 2>actual.err &&
++ test_must_be_empty actual.out &&
++ test_cmp expect actual.err
+'
+
test_expect_success '--no-show-forced-updates' '
4: b545bf8bb9 ! 5: acc0f7f520 fetch: introduce `display_format` enum
@@ builtin/fetch.c: static void display_state_init(struct display_state *display_st
+ break;
+ }
+ default:
-+ BUG("unexpected display foramt %d", display_state->format);
++ BUG("unexpected display format %d", display_state->format);
}
}5: 4990d35998 = 6: cd23440128 fetch: move display format parsing into main function 6: cfe84129ab = 7: edbc31013f fetch: move option related variables into main function 7: 0335e5eeb4 < -: ---------- fetch: introduce new `--output-format` option 8: d7c1bc1a80 ! 8: e132d9494e fetch: introduce machine-parseable "porcelain" output format
@@ Commit message
information especially in the context of multi-remote fetches. But as such a format would require us to print the remote for every single reference update due to parallelizable fetches it feels wasteful for the
- most likely usecase, which is when fetching from a single remote. If
- usecases come up for this in the future though it is easy enough to add
- a new "porcelain-v2" format that adds this information.
+ most likely usecase, which is when fetching from a single remote.
+
+ In a similar spirit, a second restriction is that this cannot be used
+ with --recurse-submodules. This is because any reference updates would
+ be ambiguous without also printing the repository in which the update
+ happens.
+
+ Considering that both multi-remote and submodule fetches are rather
+ niche and likely not going to be useful for the majority of usecases
+ these omissions feel acceptable. If usecases for either of these come up
+ in the future though it is easy enough to add a new "porcelain-v2"
+ format that adds this information.
- ## Documentation/config/fetch.txt ##
-@@ Documentation/config/fetch.txt: fetch.pruneTags::
-
- fetch.output::
- Control how ref update status is printed. Valid values are
-- full and compact. Default value is full. See section
-- OUTPUT in linkgit:git-fetch[1] for detail.
-+ full, compact and porcelain. Default value is full.
-+ See section OUTPUT in linkgit:git-fetch[1] for detail.
-
- fetch.negotiationAlgorithm::
- Control how information about the commits in the local repository
-
## Documentation/fetch-options.txt ##
@@ Documentation/fetch-options.txt: linkgit:git-config[1].
+ --dry-run::
+ Show what would be done, without making any changes.
- --output-format::
- Control how ref update status is printed. Valid values are
-- full and compact. Default value is full. See section
-- OUTPUT in linkgit:git-fetch[1] for detail.
-+ full, compact and porcelain. Default value is full.
-+ See section OUTPUT in linkgit:git-fetch[1] for detail.
-
++--porcelain::
++ Print the output to standard output in an easy-to-parse format for
++ scripts. See section OUTPUT in linkgit:git-fetch[1] for details.
+++
++This is incompatible with --recurse-submodules=[yes|on-demand]. ++
ifndef::git-pull[]
--[no-]write-fetch-head::
+ Write the list of remote refs fetched in the FETCH_HEAD
## Documentation/git-fetch.txt ##
-@@ Documentation/git-fetch.txt: The output of "git fetch" depends on the transport method used; this
- section describes the output when fetching over the Git protocol
- (either locally or via ssh) and Smart HTTP protocol.
-
--The status of the fetch is output in tabular form, with each line
--representing the status of a single ref. Each line is of the form:
-+The output format can be chosen either via the fetch.output config
-+(see linkgit:git-config[1]), or via the --output-format switch.
-+Supported values include:
-+
-+For the full and compact output formats, the status of the fetch is
-+output in tabular, with each line representing the status of a single
-+ref. Each line is of the form:
-
- -------------------------------
+@@ Documentation/git-fetch.txt: representing the status of a single ref. Each line is of the form:
<flag> <summary> <from> -> <to> [<reason>]
-------------------------------
-+The porcelain output format is intended to be machine-parseable. In -+contrast to the human-readable output formats it thus prints to standard -+output instead of standard error. Each line is of the form:
++When using --porcelain, the output format is intended to be ++machine-parseable. In contrast to the human-readable output formats it ++thus prints to standard output instead of standard error. Each line is ++of the form:
+
+-------------------------------
+<flag> <old-object-id> <new-object-id> <local-reference>
@@ builtin/fetch.c: enum display_format {
DISPLAY_FORMAT_FULL,
DISPLAY_FORMAT_COMPACT,
+ DISPLAY_FORMAT_PORCELAIN,
- DISPLAY_FORMAT_MAX,
- };
-
-@@ builtin/fetch.c: static const char * const display_formats[DISPLAY_FORMAT_MAX] = {
- NULL,
- "full",
- "compact",
-+ "porcelain",
};
struct display_state {
@@ builtin/fetch.c: static void display_state_init(struct display_state *display_st
+ /* We don't need to precompute anything here. */
+ break;
default:
-- BUG("unexpected display foramt %d", display_state->format);
-+ BUG("unexpected display format %d", display_state->format);
+ BUG("unexpected display format %d", display_state->format);
}
- }
-
@@ builtin/fetch.c: static void print_compact(struct display_state *display_state,
static void display_ref_update(struct display_state *display_state, char code,
const char *summary, const char *error,
@@ builtin/fetch.c: static int prune_refs(struct display_state *display_state,
summary_width);
warn_dangling_symref(stderr, dangling_msg, ref->name);
}
+@@ builtin/fetch.c: static int add_remote_or_group(const char *name, struct string_list *list)
+ return 1;
+ }
+
+-static void add_options_to_argv(struct strvec *argv)
++static void add_options_to_argv(struct strvec *argv,
++ enum display_format format)
+ {
+ if (dry_run)
+ strvec_push(argv, "--dry-run");
+@@ builtin/fetch.c: static void add_options_to_argv(struct strvec *argv)
+ strvec_push(argv, "--ipv6");
+ if (!write_fetch_head)
+ strvec_push(argv, "--no-write-fetch-head");
++ if (format == DISPLAY_FORMAT_PORCELAIN)
++ strvec_pushf(argv, "--porcelain");
+ }
+
+ /* Fetch multiple remotes in parallel */
+@@ builtin/fetch.c: struct parallel_fetch_state {
+ const char **argv;
+ struct string_list *remotes;
+ int next, result;
++ enum display_format format;
+ };
+
+ static int fetch_next_remote(struct child_process *cp,
+@@ builtin/fetch.c: static int fetch_next_remote(struct child_process *cp,
+ strvec_push(&cp->args, remote);
+ cp->git_cmd = 1;
+
+- if (verbosity >= 0)
++ if (verbosity >= 0 && state->format != DISPLAY_FORMAT_PORCELAIN)
+ printf(_("Fetching %s\n"), remote);
+
+ return 1;
+@@ builtin/fetch.c: static int fetch_finished(int result, struct strbuf *out,
+ return 0;
+ }
+
+-static int fetch_multiple(struct string_list *list, int max_children) ++static int fetch_multiple(struct string_list *list, int max_children,
++ enum display_format format)
+ {
+ int i, result = 0;
+ struct strvec argv = STRVEC_INIT;
+@@ builtin/fetch.c: static int fetch_multiple(struct string_list *list, int max_children) +
+ strvec_pushl(&argv, "fetch", "--append", "--no-auto-gc",
+ "--no-write-commit-graph", NULL);
+- add_options_to_argv(&argv);
++ add_options_to_argv(&argv, format);
+
+ if (max_children != 1 && list->nr != 1) {
+- struct parallel_fetch_state state = { argv.v, list, 0, 0 };
++ struct parallel_fetch_state state = { argv.v, list, 0, 0, format };
+ const struct run_process_parallel_opts opts = {
+ .tr2_category = "fetch",
+ .tr2_label = "parallel/fetch",
+@@ builtin/fetch.c: static int fetch_multiple(struct string_list *list, int max_children) +
+ strvec_pushv(&cmd.args, argv.v);
+ strvec_push(&cmd.args, name);
+- if (verbosity >= 0)
++ if (verbosity >= 0 && format != DISPLAY_FORMAT_PORCELAIN)
+ printf(_("Fetching %s\n"), name);
+ cmd.git_cmd = 1;
+ if (run_command(&cmd)) {
+@@ builtin/fetch.c: static int fetch_one(struct remote *remote, int argc, const char **argv,
+ return exit_code;
+ }
+
++static int opt_parse_porcelain(const struct option *opt, const char *arg, int unset) ++{
++ enum display_format *format = opt->value;
++ *format = DISPLAY_FORMAT_PORCELAIN;
++ return 0;
++}
++
+ int cmd_fetch(int argc, const char **argv, const char *prefix)
+ {
+ const char *bundle_uri;
+@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
+ OPT_BOOL(0, "dry-run", &dry_run,
+ N_("dry run")),
++ OPT_CALLBACK_F(0, "porcelain", &display_format, NULL, N_("machine-readable output"),
++ PARSE_OPT_NOARG|PARSE_OPT_NONEG, opt_parse_porcelain),
+ OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
+ N_("write fetched references to the FETCH_HEAD file")),
+ OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
+@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ fetch_config_from_gitmodules(sfjc, rs);
+ }
+
++ if (display_format == DISPLAY_FORMAT_PORCELAIN) {
++ switch (recurse_submodules_cli) {
++ case RECURSE_SUBMODULES_OFF:
++ case RECURSE_SUBMODULES_DEFAULT:
++ /*
++ * Reference updates in submodules would be ambiguous
++ * in porcelain mode, so we reject this combination.
++ */
++ recurse_submodules = RECURSE_SUBMODULES_OFF;
++ break;
++
++ default:
++ die(_("options '%s' and '%s' cannot be used together"),
++ "--porcelain", "--recurse-submodules");
++ }
++ }
++
+ if (negotiate_only && !negotiation_tip.nr)
+ die(_("--negotiate-only needs one or more --negotiation-tip=*"));
+
+@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ max_children = fetch_parallel_config;
+
+ /* TODO should this also die if we have a previous partial-clone? */
+- result = fetch_multiple(&list, max_children);
++ result = fetch_multiple(&list, max_children, display_format);
+ }
+
+
+@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ if (max_children < 0)
+ max_children = fetch_parallel_config;
+
+- add_options_to_argv(&options);
++ add_options_to_argv(&options, display_format);
+ result = fetch_submodules(the_repository,
+ &options,
+ submodule_prefix,
## t/t5574-fetch-output.sh ##
-@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output with multiple remotes' ' +@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
test_cmp expect actual
'
+test_expect_success 'fetch porcelain output' '
-+ test_when_finished "rm -rf porcelain-cfg porcelain-cli" &&
++ test_when_finished "rm -rf porcelain" &&
+
+ # Set up a bunch of references that we can use to demonstrate different
+ # kinds of flag symbols in the output format.
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output with multiple
+ # namespaces so that we can test that rejected and force-updated
+ # references are reported properly.
+ refspecs="refs/heads/*:refs/unforced/* +refs/heads/*:refs/forced/*" &&
-+ git clone . porcelain-cli &&
-+ git clone . porcelain-cfg &&
-+ git -C porcelain-cfg fetch origin $refspecs &&
-+ git -C porcelain-cli fetch origin $refspecs &&
++ git clone . porcelain &&
++ git -C porcelain fetch origin $refspecs &&
+
+ # Now that we have set up the client repositories we can change our
+ # local references.
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output with multiple
+ # Execute a dry-run fetch first. We do this to assert that the dry-run
+ # and non-dry-run fetches produces the same output. Execution of the
+ # fetch is expected to fail as we have a rejected reference update.
-+ test_must_fail git -C porcelain-cfg -c fetch.output=porcelain fetch --dry-run --prune origin $refspecs >actual-dry-run-cfg &&
-+ test_must_fail git -C porcelain-cli fetch --output-format=porcelain --dry-run --prune origin $refspecs >actual-dry-run-cli &&
-+ test_cmp actual-dry-run-cfg actual-dry-run-cli &&
-+ test_cmp expect actual-dry-run-cfg &&
++ test_must_fail git -C porcelain fetch \
++ --porcelain --dry-run --prune origin $refspecs >actual &&
++ test_cmp expect actual &&
+
+ # And now we perform a non-dry-run fetch.
-+ test_must_fail git -C porcelain-cfg -c fetch.output=porcelain fetch --prune origin $refspecs >actual-cfg &&
-+ test_must_fail git -C porcelain-cli fetch --output-format=porcelain --prune origin $refspecs >actual-cli &&
-+ test_cmp actual-cfg actual-cli &&
-+ test_cmp expect actual-cfg &&
-+
-+ # Ensure that the dry-run and non-dry-run output matches.
-+ test_cmp actual-dry-run-cfg actual-cfg
++ test_must_fail git -C porcelain fetch \
++ --porcelain --prune origin $refspecs >actual 2>stderr &&
++ test_cmp expect actual &&
++ test_must_be_empty stderr
+'
+
- test_expect_success 'fetch output with HEAD and --dry-run' ' ++test_expect_success 'fetch porcelain with multiple remotes' '
++ test_when_finished "rm -rf porcelain" &&
++
++ git clone . porcelain &&
++ git -C porcelain remote add second-remote "$PWD" &&
++ git -C porcelain fetch second-remote &&
++
++ test_commit --no-tag multi-commit &&
++ old_commit=$(git rev-parse HEAD~) &&
++ new_commit=$(git rev-parse HEAD) &&
++
++ cat >expect <<-EOF &&
++ $old_commit $new_commit refs/remotes/origin/force-updated
++ $old_commit $new_commit refs/remotes/second-remote/force-updated
++ EOF
++
++ git -C porcelain fetch --porcelain --all >actual 2>stderr &&
++ test_cmp expect actual &&
++ test_must_be_empty stderr
++'
++
++test_expect_success 'fetch porcelain refuses to work with submodules' '
++ test_when_finished "rm -rf porcelain" &&
++
++ cat >expect <<-EOF &&
++ fatal: options ${SQ}--porcelain${SQ} and ${SQ}--recurse-submodules${SQ} cannot be used together
++ EOF
++
++ git init porcelain &&
++ test_must_fail git -C porcelain fetch --porcelain --recurse-submodules=yes 2>stderr &&
++ test_cmp expect stderr &&
++
++ test_must_fail git -C porcelain fetch --porcelain --recurse-submodules=on-demand 2>stderr &&
++ test_cmp expect stderr
++'
++
+ test_expect_success 'fetch output with HEAD' '
test_when_finished "rm -rf head" &&
git clone . head &&
-@@ t/t5574-fetch-output.sh: test_expect_success 'fetch output with HEAD and --dry-run' '
- test_cmp expect actual
+@@ t/t5574-fetch-output.sh: test_expect_success 'fetch output with HEAD' '
+ test_cmp expect actual.err
'
-+test_expect_success 'fetch porcelain output with HEAD and --dry-run' ' ++test_expect_success 'fetch porcelain output with HEAD' '
+ test_when_finished "rm -rf head" &&
+ git clone . head &&
+ COMMIT_ID=$(git rev-parse HEAD) &&
+
-+ git -C head fetch --output-format=porcelain --dry-run origin HEAD >actual &&
++ git -C head fetch --porcelain --dry-run origin HEAD >actual &&
+ cat >expect <<-EOF &&
+ * $ZERO_OID $COMMIT_ID FETCH_HEAD
+ EOF
+ test_cmp expect actual &&
+
-+ git -C head fetch --output-format=porcelain --dry-run origin HEAD:foo >actual &&
++ git -C head fetch --porcelain origin HEAD >actual &&
++ test_cmp expect actual &&
++
++ git -C head fetch --porcelain --dry-run origin HEAD:foo >actual &&
+ cat >expect <<-EOF &&
+ * $ZERO_OID $COMMIT_ID refs/heads/foo
+ EOF
++ test_cmp expect actual &&
++
++ git -C head fetch --porcelain origin HEAD:foo >actual &&
+ test_cmp expect actual
+'
+
Signature · 1 line
2.40.1
PATCH v3 1/8 fetch: fix `--no-recurse-submodules` with multi-remote fetches
When running git fetch --no-recurse-submodules, the exectation is that we don't fetch any submodules. And while this works for fetches of a single remote, it doesn't when fetching multiple remotes at once. The result is that we do recurse into submodules even though the user has explicitly asked us not to.
This is because while we pass on --recurse-submodules={yes,on-demand} if specified by the user, we don't pass on --no-recurse-submodules to the subprocess spawned to perform the submodule fetch.
Fix this by also forwarding this flag as expected.
--- builtin/fetch.c | 2 ++ t/t5526-fetch-submodules.sh | 31 +++++++++++++++++++++++++++++++ 2 files changed, 33 insertions(+)
builtin/fetch.c +2 -0
@@ -1876,6 +1876,8 @@ static void add_options_to_argv(struct strvec *argv)strvec_push (argv,"--keep" );if (recurse_submodules ==RECURSE_SUBMODULES_ON )strvec_push (argv,"--recurse-submodules" ); +else if (recurse_submodules ==RECURSE_SUBMODULES_OFF ) +strvec_push (argv,"--no-recurse-submodules" );else if (recurse_submodules ==RECURSE_SUBMODULES_ON_DEMAND )strvec_push (argv,"--recurse-submodules=on-demand" );if (tags ==TAGS_SET )
t/t5526-fetch-submodules.sh +31 -0
@@ -1180,4 +1180,35 @@ test_expect_success 'fetch --all with --recurse-submodules with multiple' ' test_line_count = 2 fetch-subs ' +test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" ' + test_when_finished "git -C downstream remote remove second" && + + # We need to add a second remote, otherwise --all falls back to the + # normal fetch-one case. + git -C downstream remote add second .. && + git -C downstream fetch --all && + + add_submodule_commits && + add_superproject_commits && + old_commit=$(git rev-parse --short HEAD~) && + new_commit=$(git rev-parse --short HEAD) && + + git -C downstream fetch --all --no-recurse-submodules >actual.out 2>actual.err && + + cat >expect.out <<-EOF && + Fetching origin + Fetching second + EOF + + cat >expect.err <<-EOF && + From $(test-tool path-utils real_path .)/. + $old_commit..$new_commit super -> origin/super + From .. + $old_commit..$new_commit super -> second/super + EOF + + test_cmp expect.out actual.out && + test_cmp expect.err actual.err +' + test_done
Signature · 1 line
2.40.1
Re: [PATCH v3 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches
Quoted 23 lines · Patrick Steinhardt
When running
git fetch --no-recurse-submodules, the exectation is that we don't fetch any submodules. And while this works for fetches of a single remote, it doesn't when fetching multiple remotes at once. The result is that we do recurse into submodules even though the user has explicitly asked us not to.This is because while we pass on
--recurse-submodules={yes,on-demand}if specified by the user, we don't pass on--no-recurse-submodulesto the subprocess spawned to perform the submodule fetch.Fix this by also forwarding this flag as expected.
Signed-off-by Patrick Steinhardt <[email protected]>--- builtin/fetch.c | 2 ++ t/t5526-fetch-submodules.sh | 31 +++++++++++++++++++++++++++++++ 2 files changed, 33 insertions(+)builtin/fetch.c +0
-0
builtin/fetch.c static void add_options_to_argv(struct strvec *argv) · line 1883
Quoted 8 lines
builtin/fetch.c +2
-0@@ -1876,6 +1876,8 @@ static void add_options_to_argv(struct strvec *argv)strvec_push (argv,"--keep" );if (recurse_submodules ==RECURSE_SUBMODULES_ON )strvec_push (argv,"--recurse-submodules" ); +else if (recurse_submodules ==RECURSE_SUBMODULES_OFF ) +strvec_push (argv,"--no-recurse-submodules" );else if (recurse_submodules ==RECURSE_SUBMODULES_ON_DEMAND )strvec_push (argv,"--recurse-submodules=on-demand" );if (tags ==TAGS_SET )
Makes sense.
I wondered for a bit whether this should have been checking recurse_submodules_cli (the actual CLI flag) back in 386c076a86 (fetch --negotiate-only: do not update submodules, 2022-01-18). I think this current state is correct, though. After we have told the superproject what submodule recursion mode to use, we want to continue using that mode when recursing through submodules regardless of whether that mode originally came from the CLI or config.
Quoted 20 lines
builtin/fetch.c +20
-0+test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" ' + test_when_finished "git -C downstream remote remove second" && + +# We need to add a second remote, otherwise --all falls back to the +# normal fetch-one case. + git -C downstream remote add second .. && + git -C downstream fetch --all && + + add_submodule_commits && + add_superproject_commits && + old_commit=$(git rev-parse --short HEAD~) && + new_commit=$(git rev-parse --short HEAD) && + + git -C downstream fetch --all --no-recurse-submodules >actual.out 2>actual.err && + + cat >expect.out <<-EOF && + Fetching origin + Fetching second + EOF +
builtin/fetch.c
Quoted 10 lines
builtin/fetch.c +10
-0+ cat >expect.err <<-EOF && + From $(test-tool path-utils real_path .)/. + $old_commit..$new_commit super -> origin/super + From .. + $old_commit..$new_commit super -> second/super +EOF + + test_cmp expect.out actual.out && + test_cmp expect.err actual.err +'
The test looks okay, though is there a reason you didn't copy the style of the previous test? It is nearly exactly what you want, I think, like (untested)
test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" '
test_when_finished "rm -fr src_clone" &&
git clone --recurse-submodules src src_clone &&
(
cd src_clone &&
git remote add secondary ../src &&
git config submodule.recurse true &&
git config fetch.parallel 0 &&
git fetch --all 2>../fetch-log
) &&
grep "Fetching submodule" fetch-log >fetch-subs &&
test_must_be_empty fetch-subs
'which has the handy benefit of not needing the test-tools invocation.
Re: [PATCH v3 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches
Quoted 1 line · Glen Choo
Patrick Steinhardt <[email protected]> writes:
[snip]
Quoted 49 lines
Quoted 20 lines
t/t5526-fetch-submodules.sh +20
-0+test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" ' + test_when_finished "git -C downstream remote remove second" && + + # We need to add a second remote, otherwise --all falls back to the + # normal fetch-one case. + git -C downstream remote add second .. && + git -C downstream fetch --all && + + add_submodule_commits && + add_superproject_commits && + old_commit=$(git rev-parse --short HEAD~) && + new_commit=$(git rev-parse --short HEAD) && + + git -C downstream fetch --all --no-recurse-submodules >actual.out 2>actual.err && + + cat >expect.out <<-EOF && + Fetching origin + Fetching second + EOF +t/t5526-fetch-submodules.sh test_expect_success 'fetch --all with --recurse-submodules with multiple' ' · line 1212
Quoted 10 lines
t/t5526-fetch-submodules.sh +10
-0+ cat >expect.err <<-EOF && + From $(test-tool path-utils real_path .)/. + $old_commit..$new_commit super -> origin/super + From .. + $old_commit..$new_commit super -> second/super + EOF + + test_cmp expect.out actual.out && + test_cmp expect.err actual.err +'The test looks okay, though is there a reason you didn't copy the style of the previous test? It is nearly exactly what you want, I think, like (untested)
test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" ' test_when_finished "rm -fr src_clone" && git clone --recurse-submodules src src_clone && ( cd src_clone && git remote add secondary ../src && git config submodule.recurse true && git config fetch.parallel 0 && git fetch --all 2>../fetch-log ) && grep "Fetching submodule" fetch-log >fetch-subs && test_must_be_empty fetch-subs '
Quoted 1 line
which has the handy benefit of not needing the test-tools invocation.
That is indeed much simpler, thanks.
Patrick
PATCH v3 2/8 fetch: split out tests for output format
In reply to Patrick SteinhardtWe're about to introduce a new porcelain mode for the output of git-fetch(1). As part of that we'll be introducing a set of new tests that only relate to the output of this command.
Split out tests that exercise the output format of git-fetch(1) so that it becomes easier to verify this functionality as a standalone unit. As the tests assume that the default branch is called "main" we set up the corresponding GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME environment variable accordingly.
--- t/t5510-fetch.sh | 53 ---------------------------------- t/t5574-fetch-output.sh | 63 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+), 53 deletions(-) create mode 100755 t/t5574-fetch-output.sh
t/t5510-fetch.sh +0 -53
@@ -1118,59 +1118,6 @@ test_expect_success 'fetching with auto-gc does not lock up' ' ) ' -test_expect_success 'fetch aligned output' ' - git clone . full-output && - test_commit looooooooooooong-tag && - ( - cd full-output && - git -c fetch.output=full fetch origin >actual 2>&1 && - grep -e "->" actual | cut -c 22- >../actual - ) && - cat >expect <<-\EOF && - main -> origin/main - looooooooooooong-tag -> looooooooooooong-tag - EOF - test_cmp expect actual -' - -test_expect_success 'fetch compact output' ' - git clone . compact && - test_commit extraaa && - ( - cd compact && - git -c fetch.output=compact fetch origin >actual 2>&1 && - grep -e "->" actual | cut -c 22- >../actual - ) && - cat >expect <<-\EOF && - main -> origin/* - extraaa -> * - EOF - test_cmp expect actual -' - -test_expect_success '--no-show-forced-updates' ' - mkdir forced-updates && - ( - cd forced-updates && - git init && - test_commit 1 && - test_commit 2 - ) && - git clone forced-updates forced-update-clone && - git clone forced-updates no-forced-update-clone && - git -C forced-updates reset --hard HEAD~1 && - ( - cd forced-update-clone && - git fetch --show-forced-updates origin 2>output && - test_i18ngrep "(forced update)" output - ) && - ( - cd no-forced-update-clone && - git fetch --no-show-forced-updates origin 2>output && - test_i18ngrep ! "(forced update)" output - ) -' - for section in fetch transfer do test_expect_success "$section.hideRefs affects connectivity check" '
t/t5574-fetch-output.sh new file +63 -0
63 lines not on this page · raw message
Signature · 1 line
2.40.1
PATCH v3 3/8 fetch: add a test to exercise invalid output formats
In reply to Patrick SteinhardtAdd a testcase that exercises the logic when an invalid output format is passed via the fetch.output configuration.
--- t/t5574-fetch-output.sh | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
t/t5574-fetch-output.sh +19 -0
@@ -7,6 +7,25 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME . ./test-lib.sh +test_expect_success 'fetch with invalid output format configuration' ' + test_when_finished "rm -rf clone" && + git clone . clone && + + test_must_fail git -C clone -c fetch.output= fetch origin >actual.out 2>actual.err && + cat >expect <<-EOF && + fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}${SQ} + EOF + test_must_be_empty actual.out && + test_cmp expect actual.err && + + test_must_fail git -C clone -c fetch.output=garbage fetch origin >actual.out 2>actual.err && + cat >expect <<-EOF && + fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}garbage${SQ} + EOF + test_must_be_empty actual.out && + test_cmp expect actual.err +' + test_expect_success 'fetch aligned output' ' git clone . full-output && test_commit looooooooooooong-tag &&
Signature · 1 line
2.40.1
PATCH v3 4/8 fetch: fix missing from-reference when fetching HEAD:foo
In reply to Patrick Steinhardtstore_updated_refs() parses the remote reference for two purposes:
- It gets used as a note when writing FETCH_HEAD.
- It is passed through to display_ref_update() to display updated references in the following format:
```
* branch master -> master
```
In most cases, the parsed remote reference is the prettified reference name and can thus be used for both cases. But if the remote reference is HEAD, the parsed remote reference becomes empty. This is intended when we write the FETCH_HEAD, where we skip writing the note in that case. But it is not intended when displaying the updated references and would cause us to miss the left-hand side of the displayed reference update:
``` $ git fetch origin HEAD:foo From https://github.com/git/git * [new ref] -> foo ```
The HEAD string is clearly missing from the left-hand side of the arrow, which is further stressed by the point that the following commands show the left-hand side as expected:
``` $ git fetch origin HEAD From https://github.com/git/git * branch HEAD -> FETCH_HEAD $ git fetch origin master From https://github.com/git/git * branch master -> FETCH_HEAD * branch master -> origin/master ```
The logic of how we compute the remote reference name that we ultimately pass to display_ref_update() is not easy to follow. There are three different cases here:
- When the remote reference name is "HEAD" we set the remote reference name to the empty string. This is the case that causes the bug to occur, where we would indeed want to print "HEAD" instead of the empty string. This is what prettify_refname() would return.
- When the remote reference name has a well-known prefix then we strip this prefix. This matches what prettify_refname() does.
- Otherwise, we keep the fully qualified reference name. This also matches what prettify_refname() does.
As the return value of prettify_refname() would do the correct thing for us in all three cases, we can fix the bug by passing through the full remote reference name to display_ref_update(), which learns to call prettify_refname(). At the same time, this also simplifies the code a bit.
Note that this patch also changes formatting of the block that computes the "kind" and "what" variables. This is done on purpose so that it is part of the diff, hopefully making the change easier to comprehend.
--- builtin/fetch.c | 37 +++++++++++++++++++------------------ t/t5574-fetch-output.sh | 29 +++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+), 18 deletions(-)
builtin/fetch.c +19 -18
@@ -918,12 +918,14 @@ static void display_ref_update(struct display_state *display_state, char code, } width = (summary_width +strlen (summary) -gettext_width (summary)); + remote =prettify_refname (remote); + local =prettify_refname (local);strbuf_addf (&display_state->buf," %c %-*s " , code, width, summary);if (!display_state->compact_format) -print_remote_to_local (display_state, remote,prettify_refname (local)); +print_remote_to_local (display_state, remote, local);else -print_compact (display_state, remote,prettify_refname (local)); +print_compact (display_state, remote, local);if (error)strbuf_addf (&display_state->buf," (%s)" , error);strbuf_addch (&display_state->buf,'\n' );
@@ -934,7 +936,7 @@ static void display_ref_update(struct display_state *display_state, char code,static int update_local_ref (struct ref *ref,struct ref_transaction *transaction,struct display_state *display_state, -const char *remote,const struct ref *remote_ref, +const struct ref *remote_ref,int summary_width) {struct commit *current =NULL , *updated;
@@ -946,7 +948,7 @@ static int update_local_ref(struct ref *ref,if (oideq (&ref->old_oid, &ref->new_oid)) {if (verbosity >0 )display_ref_update (display_state,'=' ,_ ("[up to date]" ),NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 0 ; }
@@ -959,7 +961,7 @@ static int update_local_ref(struct ref *ref,*/ display_ref_update (display_state,'!' ,_ ("[rejected]" ),_ ("can't fetch into checked-out branch" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 1 ; }
@@ -970,12 +972,12 @@ static int update_local_ref(struct ref *ref, r =s_update_ref ("updating tag" , ref, transaction,0 );display_ref_update (display_state, r ?'!' :'t' ,_ ("[tag update]" ), r ?_ ("unable to update local ref" ) :NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return r; }else {display_ref_update (display_state,'!' ,_ ("[rejected]" ),_ ("would clobber existing tag" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 1 ; } }
@@ -1008,7 +1010,7 @@ static int update_local_ref(struct ref *ref, r =s_update_ref (msg, ref, transaction,0 );display_ref_update (display_state, r ?'!' :'*' , what, r ?_ ("unable to update local ref" ) :NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return r; }
@@ -1030,7 +1032,7 @@ static int update_local_ref(struct ref *ref, r =s_update_ref ("fast-forward" , ref, transaction,1 );display_ref_update (display_state, r ?'!' :' ' , quickref.buf, r ?_ ("unable to update local ref" ) :NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);strbuf_release (&quickref);return r; }else if (force || ref->force) {
@@ -1042,12 +1044,12 @@ static int update_local_ref(struct ref *ref, r =s_update_ref ("forced-update" , ref, transaction,1 );display_ref_update (display_state, r ?'!' :'+' , quickref.buf, r ?_ ("unable to update local ref" ) :_ ("forced update" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);strbuf_release (&quickref);return r; }else {display_ref_update (display_state,'!' ,_ ("[rejected]" ),_ ("non-fast-forward" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 1 ; } }
@@ -1252,14 +1254,13 @@ static int store_updated_refs(struct display_state *display_state,if (!strcmp (rm->name,"HEAD" )) { kind ="" ; what ="" ; - } -else if (skip_prefix (rm->name,"refs/heads/" , &what)) + }else if (skip_prefix (rm->name,"refs/heads/" , &what)) { kind ="branch" ; -else if (skip_prefix (rm->name,"refs/tags/" , &what)) + }else if (skip_prefix (rm->name,"refs/tags/" , &what)) { kind ="tag" ; -else if (skip_prefix (rm->name,"refs/remotes/" , &what)) + }else if (skip_prefix (rm->name,"refs/remotes/" , &what)) { kind ="remote-tracking branch" ; -else { + }else { kind ="" ; what = rm->name; }
@@ -1277,7 +1278,7 @@ static int store_updated_refs(struct display_state *display_state, display_state->url_len);if (ref) { - rc |=update_local_ref (ref, transaction, display_state, what, + rc |=update_local_ref (ref, transaction, display_state, rm, summary_width);free (ref); }else if (write_fetch_head || dry_run) {
@@ -1288,7 +1289,7 @@ static int store_updated_refs(struct display_state *display_state,*/ display_ref_update (display_state,'*' , *kind ? kind :"branch" ,NULL , - *what ? what :"HEAD" , + rm->name,"FETCH_HEAD" , summary_width); } }
t/t5574-fetch-output.sh +29 -0
@@ -56,6 +56,35 @@ test_expect_success 'fetch compact output' ' test_cmp expect actual ' +test_expect_success 'fetch output with HEAD' ' + test_when_finished "rm -rf head" && + git clone . head && + + git -C head fetch --dry-run origin HEAD >actual.out 2>actual.err && + cat >expect <<-EOF && + From $(test-tool path-utils real_path .)/. + * branch HEAD -> FETCH_HEAD + EOF + test_must_be_empty actual.out && + test_cmp expect actual.err && + + git -C head fetch origin HEAD >actual.out 2>actual.err && + test_must_be_empty actual.out && + test_cmp expect actual.err && + + git -C head fetch --dry-run origin HEAD:foo >actual.out 2>actual.err && + cat >expect <<-EOF && + From $(test-tool path-utils real_path .)/. + * [new ref] HEAD -> foo + EOF + test_must_be_empty actual.out && + test_cmp expect actual.err && + + git -C head fetch origin HEAD:foo >actual.out 2>actual.err && + test_must_be_empty actual.out && + test_cmp expect actual.err +' + test_expect_success '--no-show-forced-updates' ' mkdir forced-updates && (
Signature · 1 line
2.40.1
PATCH v3 5/8 fetch: introduce `display_format` enum
In reply to Patrick SteinhardtWe currently have two different display formats in git-fetch(1) with the "full" and "compact" formats. This is tracked with a boolean value that simply denotes whether the display format is supposed to be compacted or not. This works reasonably well while there are only two formats, but we're about to introduce another format that will make this a bit more awkward to use.
Introduce a enum display_format that is more readily extensible.
--- builtin/fetch.c | 101 ++++++++++++++++++++++++++++++------------------ 1 file changed, 64 insertions(+), 37 deletions(-)
builtin/fetch.c +64 -37
@@ -48,11 +48,17 @@ enum {TAGS_SET =2 }; +enum display_format { +DISPLAY_FORMAT_UNKNOWN =0 , +DISPLAY_FORMAT_FULL , +DISPLAY_FORMAT_COMPACT , +}; +struct display_state {struct strbuf buf;int refcol_width; -int compact_format; +enum display_format format;char *url;int url_len, shown_url;
@@ -784,7 +790,6 @@ static int refcol_width(const struct ref *ref, int compact_format)static void display_state_init (struct display_state *display_state,struct ref *ref_map,const char *raw_url) { -struct ref *rm;const char *format ="full" ;int i;
@@ -809,31 +814,42 @@ static void display_state_init(struct display_state *display_state, struct ref *git_config_get_string_tmp ("fetch.output" , &format);if (!strcasecmp (format,"full" )) - display_state->compact_format =0 ; + display_state->format =DISPLAY_FORMAT_FULL ;else if (!strcasecmp (format,"compact" )) - display_state->compact_format =1 ; + display_state->format =DISPLAY_FORMAT_COMPACT ;else die (_ ("invalid value for '%s': '%s'" ),"fetch.output" , format); - display_state->refcol_width =10 ; -for (rm = ref_map; rm; rm = rm->next) { -int width; +switch (display_state->format) { +case DISPLAY_FORMAT_FULL : +case DISPLAY_FORMAT_COMPACT : { +struct ref *rm; -if (rm->status ==REF_STATUS_REJECT_SHALLOW || - !rm->peer_ref || - !strcmp (rm->name,"HEAD" )) -continue ; + display_state->refcol_width =10 ; +for (rm = ref_map; rm; rm = rm->next) { +int width; - width =refcol_width (rm, display_state->compact_format); +if (rm->status ==REF_STATUS_REJECT_SHALLOW || + !rm->peer_ref || + !strcmp (rm->name,"HEAD" )) +continue ; -/* -* Not precise calculation for compact mode because '*' can -* appear on the left hand side of '->' and shrink the column -* back. -*/ -if (display_state->refcol_width < width) - display_state->refcol_width = width; + width =refcol_width (rm, display_state->format ==DISPLAY_FORMAT_COMPACT ); + +/* +* Not precise calculation for compact mode because '*' can +* appear on the left hand side of '->' and shrink the column +* back. +*/ +if (display_state->refcol_width < width) + display_state->refcol_width = width; + } + +break ; + } +default : +BUG ("unexpected display format %d" , display_state->format); } }
@@ -904,30 +920,41 @@ static void display_ref_update(struct display_state *display_state, char code, const char *remote, const char *local, int summary_width) { - int width; - if (verbosity < 0) return;strbuf_reset (&display_state->buf); -if (!display_state->shown_url) { -strbuf_addf (&display_state->buf,_ ("From %.*s\n" ), - display_state->url_len, display_state->url); - display_state->shown_url =1 ; +switch (display_state->format) { +case DISPLAY_FORMAT_FULL : +case DISPLAY_FORMAT_COMPACT : { +int width; + +if (!display_state->shown_url) { +strbuf_addf (&display_state->buf,_ ("From %.*s\n" ), + display_state->url_len, display_state->url); + display_state->shown_url =1 ; + } + + width = (summary_width +strlen (summary) -gettext_width (summary)); + remote =prettify_refname (remote); + local =prettify_refname (local); + +strbuf_addf (&display_state->buf," %c %-*s " , code, width, summary); + +if (display_state->format !=DISPLAY_FORMAT_COMPACT ) +print_remote_to_local (display_state, remote, local); +else +print_compact (display_state, remote, local); + +if (error) +strbuf_addf (&display_state->buf," (%s)" , error); + +break ; } - - width = (summary_width +strlen (summary) -gettext_width (summary)); - remote =prettify_refname (remote); - local =prettify_refname (local); - -strbuf_addf (&display_state->buf," %c %-*s " , code, width, summary); -if (!display_state->compact_format) -print_remote_to_local (display_state, remote, local); -else -print_compact (display_state, remote, local); -if (error) -strbuf_addf (&display_state->buf," (%s)" , error); +default : +BUG ("unexpected display format %d" , display_state->format); + };strbuf_addch (&display_state->buf,'\n' );fputs (display_state->buf.buf, stderr);
Signature · 1 line
2.40.1
PATCH v3 6/8 fetch: move display format parsing into main function
In reply to Patrick SteinhardtParsing the display format happens inside of display_state_init(). As we only need to check for a simple config entry, this is a natural location to put this code as it means that display-state logic is neatly contained in a single location.
We're about to introduce a output format though that is intended to be parseable by machines, for example inside of a script. In that case it becomes a bit awkward of an interface if you have to call git-fetch(1) with the fetch.output config key set. We're thus going to introduce a new --output-format switch for git-fetch(1) so that the output format can be configured more directly.
This means we'll have to hook parsing of the display format into the command line options parser. Having the code to determine the actual output format scattered across two different sites is hard to reason about though.
Refactor the code such that callers are expected to pass the display format that is to be used into display_state_init(). This allows us to lift up the code into the main function, where we can then hook it into command line options parser in a follow-up commit.
--- builtin/fetch.c | 41 ++++++++++++++++++++++++----------------- 1 file changed, 24 insertions(+), 17 deletions(-)
builtin/fetch.c +24 -17
@@ -788,14 +788,13 @@ static int refcol_width(const struct ref *ref, int compact_format) }static void display_state_init (struct display_state *display_state,struct ref *ref_map, -const char *raw_url) +const char *raw_url,enum display_format format) { -const char *format ="full" ;int i;memset (display_state,0 ,sizeof (*display_state)); -strbuf_init (&display_state->buf,0 ); + display_state->format = format;if (raw_url) display_state->url =transport_anonymize_url (raw_url);
@@ -812,15 +811,6 @@ static void display_state_init(struct display_state *display_state, struct ref *if (verbosity <0 )return ; -git_config_get_string_tmp ("fetch.output" , &format); -if (!strcasecmp (format,"full" )) - display_state->format =DISPLAY_FORMAT_FULL ; -else if (!strcasecmp (format,"compact" )) - display_state->format =DISPLAY_FORMAT_COMPACT ; -else -die (_ ("invalid value for '%s': '%s'" ), -"fetch.output" , format); -switch (display_state->format) {case DISPLAY_FORMAT_FULL :case DISPLAY_FORMAT_COMPACT : {
@@ -1614,7 +1604,8 @@ static int backfill_tags(struct display_state *display_state, }static int do_fetch (struct transport *transport, -struct refspec *rs) +struct refspec *rs, +enum display_format display_format) {struct ref_transaction *transaction =NULL ;struct ref *ref_map =NULL ;
@@ -1700,7 +1691,7 @@ static int do_fetch(struct transport *transport,if (retcode) goto cleanup; -display_state_init (&display_state, ref_map, transport->url); +display_state_init (&display_state, ref_map, transport->url, display_format);if (atomic_fetch) { transaction =ref_transaction_begin (&err);
@@ -2078,7 +2069,8 @@ static inline void fetch_one_setup_partial(struct remote *remote) }static int fetch_one (struct remote *remote,int argc,const char **argv, -int prune_tags_ok,int use_stdin_refspecs) +int prune_tags_ok,int use_stdin_refspecs, +enum display_format display_format) {struct refspec rs =REFSPEC_INIT_FETCH ;int i;
@@ -2145,7 +2137,7 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,sigchain_push_common (unlock_pack_on_signal);atexit (unlock_pack_atexit);sigchain_push (SIGPIPE ,SIG_IGN ); - exit_code =do_fetch (gtransport, &rs); + exit_code =do_fetch (gtransport, &rs, display_format);sigchain_pop (SIGPIPE );refspec_clear (&rs);transport_disconnect (gtransport);
@@ -2157,6 +2149,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) {int i;const char *bundle_uri; +enum display_format display_format =DISPLAY_FORMAT_UNKNOWN ;struct string_list list =STRING_LIST_INIT_DUP ;struct remote *remote =NULL ;int result =0 ;
@@ -2183,6 +2176,19 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) argc =parse_options (argc, argv, prefix, builtin_fetch_options, builtin_fetch_usage,0 ); +if (display_format ==DISPLAY_FORMAT_UNKNOWN ) { +const char *format ="full" ; + +git_config_get_string_tmp ("fetch.output" , &format); +if (!strcasecmp (format,"full" )) + display_format =DISPLAY_FORMAT_FULL ; +else if (!strcasecmp (format,"compact" )) + display_format =DISPLAY_FORMAT_COMPACT ; +else +die (_ ("invalid value for '%s': '%s'" ), +"fetch.output" , format); + } +if (recurse_submodules_cli !=RECURSE_SUBMODULES_DEFAULT ) recurse_submodules = recurse_submodules_cli;
@@ -2311,7 +2317,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) }else if (remote) {if (filter_options.choice ||has_promisor_remote ())fetch_one_setup_partial (remote); - result =fetch_one (remote, argc, argv, prune_tags_ok, stdin_refspecs); + result =fetch_one (remote, argc, argv, prune_tags_ok, stdin_refspecs, + display_format); }else {int max_children = max_jobs;
Signature · 1 line
2.40.1
PATCH v3 7/8 fetch: move option related variables into main function
In reply to Patrick SteinhardtThe options of git-fetch(1) which we pass to parse_options() are declared globally in builtin/fetch.c. This means we're forced to use global variables for all the options, which is more likely to cause confusion than explicitly passing state around.
Refactor the code to move the options into cmd_fetch(). Move variables that were previously forced to be declared globally and which are only used by cmd_fetch() into function-local scope.
--- builtin/fetch.c | 197 ++++++++++++++++++++++++------------------------ 1 file changed, 100 insertions(+), 97 deletions(-)
builtin/fetch.c +100 -97
@@ -75,13 +75,12 @@ static int fetch_prune_tags_config = -1; /* unspecified */static int prune_tags =-1 ;/* unspecified */ #define PRUNE_TAGS_BY_DEFAULT 0/* do we prune tags by default? */ -static int all, append, dry_run, force, keep, multiple, update_head_ok; +static int append, dry_run, force, keep, update_head_ok;static int write_fetch_head =1 ;static int verbosity, deepen_relative, set_upstream, refetch;static int progress =-1 ; -static int enable_auto_gc =1 ; -static int tags =TAGS_DEFAULT , unshallow, update_shallow, deepen; -static int max_jobs =-1 , submodule_fetch_jobs_config =-1 ; +static int tags =TAGS_DEFAULT , update_shallow, deepen; +static int submodule_fetch_jobs_config =-1 ;static int fetch_parallel_config =1 ;static int atomic_fetch;static enum transport_family family;
@@ -92,17 +91,11 @@ static struct string_list deepen_not = STRING_LIST_INIT_NODUP;static struct strbuf default_rla =STRBUF_INIT ;static struct transport *gtransport;static struct transport *gsecondary; -static const char *submodule_prefix ="" ;static int recurse_submodules =RECURSE_SUBMODULES_DEFAULT ; -static int recurse_submodules_cli =RECURSE_SUBMODULES_DEFAULT ; -static int recurse_submodules_default =RECURSE_SUBMODULES_ON_DEMAND ;static struct refspec refmap =REFSPEC_INIT_FETCH ;static struct list_objects_filter_options filter_options =LIST_OBJECTS_FILTER_INIT ;static struct string_list server_options =STRING_LIST_INIT_DUP ;static struct string_list negotiation_tip =STRING_LIST_INIT_NODUP ; -static int fetch_write_commit_graph =-1 ; -static int stdin_refspecs =0 ; -static int negotiate_only;static int git_fetch_config (const char *k,const char *v,void *cb) {
@@ -160,92 +153,6 @@ static int parse_refmap_arg(const struct option *opt, const char *arg, int unsetreturn 0 ; } -static struct option builtin_fetch_options[] = { -OPT__VERBOSITY (&verbosity), -OPT_BOOL (0 ,"all" , &all, -N_ ("fetch from all remotes" )), -OPT_BOOL (0 ,"set-upstream" , &set_upstream, -N_ ("set upstream for git pull/fetch" )), -OPT_BOOL ('a' ,"append" , &append, -N_ ("append to .git/FETCH_HEAD instead of overwriting" )), -OPT_BOOL (0 ,"atomic" , &atomic_fetch, -N_ ("use atomic transaction to update references" )), -OPT_STRING (0 ,"upload-pack" , &upload_pack,N_ ("path" ), -N_ ("path to upload pack on remote end" )), -OPT__FORCE (&force,N_ ("force overwrite of local reference" ),0 ), -OPT_BOOL ('m' ,"multiple" , &multiple, -N_ ("fetch from multiple remotes" )), -OPT_SET_INT ('t' ,"tags" , &tags, -N_ ("fetch all tags and associated objects" ),TAGS_SET ), -OPT_SET_INT ('n' ,NULL , &tags, -N_ ("do not fetch all tags (--no-tags)" ),TAGS_UNSET ), -OPT_INTEGER ('j' ,"jobs" , &max_jobs, -N_ ("number of submodules fetched in parallel" )), -OPT_BOOL (0 ,"prefetch" , &prefetch, -N_ ("modify the refspec to place all refs within refs/prefetch/" )), -OPT_BOOL ('p' ,"prune" , &prune, -N_ ("prune remote-tracking branches no longer on remote" )), -OPT_BOOL ('P' ,"prune-tags" , &prune_tags, -N_ ("prune local tags no longer on remote and clobber changed tags" )), -OPT_CALLBACK_F (0 ,"recurse-submodules" , &recurse_submodules_cli,N_ ("on-demand" ), -N_ ("control recursive fetching of submodules" ), -PARSE_OPT_OPTARG , option_fetch_parse_recurse_submodules), -OPT_BOOL (0 ,"dry-run" , &dry_run, -N_ ("dry run" )), -OPT_BOOL (0 ,"write-fetch-head" , &write_fetch_head, -N_ ("write fetched references to the FETCH_HEAD file" )), -OPT_BOOL ('k' ,"keep" , &keep,N_ ("keep downloaded pack" )), -OPT_BOOL ('u' ,"update-head-ok" , &update_head_ok, -N_ ("allow updating of HEAD ref" )), -OPT_BOOL (0 ,"progress" , &progress,N_ ("force progress reporting" )), -OPT_STRING (0 ,"depth" , &depth,N_ ("depth" ), -N_ ("deepen history of shallow clone" )), -OPT_STRING (0 ,"shallow-since" , &deepen_since,N_ ("time" ), -N_ ("deepen history of shallow repository based on time" )), -OPT_STRING_LIST (0 ,"shallow-exclude" , &deepen_not,N_ ("revision" ), -N_ ("deepen history of shallow clone, excluding rev" )), -OPT_INTEGER (0 ,"deepen" , &deepen_relative, -N_ ("deepen history of shallow clone" )), -OPT_SET_INT_F (0 ,"unshallow" , &unshallow, -N_ ("convert to a complete repository" ), -1 ,PARSE_OPT_NONEG ), -OPT_SET_INT_F (0 ,"refetch" , &refetch, -N_ ("re-fetch without negotiating common commits" ), -1 ,PARSE_OPT_NONEG ), - {OPTION_STRING ,0 ,"submodule-prefix" , &submodule_prefix,N_ ("dir" ), -N_ ("prepend this to submodule path output" ),PARSE_OPT_HIDDEN }, -OPT_CALLBACK_F (0 ,"recurse-submodules-default" , - &recurse_submodules_default,N_ ("on-demand" ), -N_ ("default for recursive fetching of submodules " -"(lower priority than config files)" ), -PARSE_OPT_HIDDEN , option_fetch_parse_recurse_submodules), -OPT_BOOL (0 ,"update-shallow" , &update_shallow, -N_ ("accept refs that update .git/shallow" )), -OPT_CALLBACK_F (0 ,"refmap" ,NULL ,N_ ("refmap" ), -N_ ("specify fetch refmap" ),PARSE_OPT_NONEG , parse_refmap_arg), -OPT_STRING_LIST ('o' ,"server-option" , &server_options,N_ ("server-specific" ),N_ ("option to transmit" )), -OPT_SET_INT ('4' ,"ipv4" , &family,N_ ("use IPv4 addresses only" ), -TRANSPORT_FAMILY_IPV4 ), -OPT_SET_INT ('6' ,"ipv6" , &family,N_ ("use IPv6 addresses only" ), -TRANSPORT_FAMILY_IPV6 ), -OPT_STRING_LIST (0 ,"negotiation-tip" , &negotiation_tip,N_ ("revision" ), -N_ ("report that we have only objects reachable from this object" )), -OPT_BOOL (0 ,"negotiate-only" , &negotiate_only, -N_ ("do not fetch a packfile; instead, print ancestors of negotiation tips" )), -OPT_PARSE_LIST_OBJECTS_FILTER (&filter_options), -OPT_BOOL (0 ,"auto-maintenance" , &enable_auto_gc, -N_ ("run 'maintenance --auto' after fetching" )), -OPT_BOOL (0 ,"auto-gc" , &enable_auto_gc, -N_ ("run 'maintenance --auto' after fetching" )), -OPT_BOOL (0 ,"show-forced-updates" , &fetch_show_forced_updates, -N_ ("check for forced-updates on all updated branches" )), -OPT_BOOL (0 ,"write-commit-graph" , &fetch_write_commit_graph, -N_ ("write the commit-graph after fetching" )), -OPT_BOOL (0 ,"stdin" , &stdin_refspecs, -N_ ("accept refspecs from stdin" )), -OPT_END () -}; -static void unlock_pack (unsigned int flags) {if (gtransport)
@@ -2147,13 +2054,109 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,int cmd_fetch (int argc,const char **argv,const char *prefix) { -int i;const char *bundle_uri; +const char *submodule_prefix ="" ;enum display_format display_format =DISPLAY_FORMAT_UNKNOWN ;struct string_list list =STRING_LIST_INIT_DUP ;struct remote *remote =NULL ; +int all =0 , multiple =0 ;int result =0 ;int prune_tags_ok =1 ; +int enable_auto_gc =1 ; +int unshallow =0 ; +int max_jobs =-1 ; +int recurse_submodules_cli =RECURSE_SUBMODULES_DEFAULT ; +int recurse_submodules_default =RECURSE_SUBMODULES_ON_DEMAND ; +int fetch_write_commit_graph =-1 ; +int stdin_refspecs =0 ; +int negotiate_only =0 ; +int i; + +struct option builtin_fetch_options[] = { +OPT__VERBOSITY (&verbosity), +OPT_BOOL (0 ,"all" , &all, +N_ ("fetch from all remotes" )), +OPT_BOOL (0 ,"set-upstream" , &set_upstream, +N_ ("set upstream for git pull/fetch" )), +OPT_BOOL ('a' ,"append" , &append, +N_ ("append to .git/FETCH_HEAD instead of overwriting" )), +OPT_BOOL (0 ,"atomic" , &atomic_fetch, +N_ ("use atomic transaction to update references" )), +OPT_STRING (0 ,"upload-pack" , &upload_pack,N_ ("path" ), +N_ ("path to upload pack on remote end" )), +OPT__FORCE (&force,N_ ("force overwrite of local reference" ),0 ), +OPT_BOOL ('m' ,"multiple" , &multiple, +N_ ("fetch from multiple remotes" )), +OPT_SET_INT ('t' ,"tags" , &tags, +N_ ("fetch all tags and associated objects" ),TAGS_SET ), +OPT_SET_INT ('n' ,NULL , &tags, +N_ ("do not fetch all tags (--no-tags)" ),TAGS_UNSET ), +OPT_INTEGER ('j' ,"jobs" , &max_jobs, +N_ ("number of submodules fetched in parallel" )), +OPT_BOOL (0 ,"prefetch" , &prefetch, +N_ ("modify the refspec to place all refs within refs/prefetch/" )), +OPT_BOOL ('p' ,"prune" , &prune, +N_ ("prune remote-tracking branches no longer on remote" )), +OPT_BOOL ('P' ,"prune-tags" , &prune_tags, +N_ ("prune local tags no longer on remote and clobber changed tags" )), +OPT_CALLBACK_F (0 ,"recurse-submodules" , &recurse_submodules_cli,N_ ("on-demand" ), +N_ ("control recursive fetching of submodules" ), +PARSE_OPT_OPTARG , option_fetch_parse_recurse_submodules), +OPT_BOOL (0 ,"dry-run" , &dry_run, +N_ ("dry run" )), +OPT_BOOL (0 ,"write-fetch-head" , &write_fetch_head, +N_ ("write fetched references to the FETCH_HEAD file" )), +OPT_BOOL ('k' ,"keep" , &keep,N_ ("keep downloaded pack" )), +OPT_BOOL ('u' ,"update-head-ok" , &update_head_ok, +N_ ("allow updating of HEAD ref" )), +OPT_BOOL (0 ,"progress" , &progress,N_ ("force progress reporting" )), +OPT_STRING (0 ,"depth" , &depth,N_ ("depth" ), +N_ ("deepen history of shallow clone" )), +OPT_STRING (0 ,"shallow-since" , &deepen_since,N_ ("time" ), +N_ ("deepen history of shallow repository based on time" )), +OPT_STRING_LIST (0 ,"shallow-exclude" , &deepen_not,N_ ("revision" ), +N_ ("deepen history of shallow clone, excluding rev" )), +OPT_INTEGER (0 ,"deepen" , &deepen_relative, +N_ ("deepen history of shallow clone" )), +OPT_SET_INT_F (0 ,"unshallow" , &unshallow, +N_ ("convert to a complete repository" ), +1 ,PARSE_OPT_NONEG ), +OPT_SET_INT_F (0 ,"refetch" , &refetch, +N_ ("re-fetch without negotiating common commits" ), +1 ,PARSE_OPT_NONEG ), + {OPTION_STRING ,0 ,"submodule-prefix" , &submodule_prefix,N_ ("dir" ), +N_ ("prepend this to submodule path output" ),PARSE_OPT_HIDDEN }, +OPT_CALLBACK_F (0 ,"recurse-submodules-default" , + &recurse_submodules_default,N_ ("on-demand" ), +N_ ("default for recursive fetching of submodules " +"(lower priority than config files)" ), +PARSE_OPT_HIDDEN , option_fetch_parse_recurse_submodules), +OPT_BOOL (0 ,"update-shallow" , &update_shallow, +N_ ("accept refs that update .git/shallow" )), +OPT_CALLBACK_F (0 ,"refmap" ,NULL ,N_ ("refmap" ), +N_ ("specify fetch refmap" ),PARSE_OPT_NONEG , parse_refmap_arg), +OPT_STRING_LIST ('o' ,"server-option" , &server_options,N_ ("server-specific" ),N_ ("option to transmit" )), +OPT_SET_INT ('4' ,"ipv4" , &family,N_ ("use IPv4 addresses only" ), +TRANSPORT_FAMILY_IPV4 ), +OPT_SET_INT ('6' ,"ipv6" , &family,N_ ("use IPv6 addresses only" ), +TRANSPORT_FAMILY_IPV6 ), +OPT_STRING_LIST (0 ,"negotiation-tip" , &negotiation_tip,N_ ("revision" ), +N_ ("report that we have only objects reachable from this object" )), +OPT_BOOL (0 ,"negotiate-only" , &negotiate_only, +N_ ("do not fetch a packfile; instead, print ancestors of negotiation tips" )), +OPT_PARSE_LIST_OBJECTS_FILTER (&filter_options), +OPT_BOOL (0 ,"auto-maintenance" , &enable_auto_gc, +N_ ("run 'maintenance --auto' after fetching" )), +OPT_BOOL (0 ,"auto-gc" , &enable_auto_gc, +N_ ("run 'maintenance --auto' after fetching" )), +OPT_BOOL (0 ,"show-forced-updates" , &fetch_show_forced_updates, +N_ ("check for forced-updates on all updated branches" )), +OPT_BOOL (0 ,"write-commit-graph" , &fetch_write_commit_graph, +N_ ("write the commit-graph after fetching" )), +OPT_BOOL (0 ,"stdin" , &stdin_refspecs, +N_ ("accept refspecs from stdin" )), +OPT_END () + };packet_trace_identity ("fetch" );
Signature · 1 line
2.40.1
PATCH v3 8/8 fetch: introduce machine-parseable "porcelain" output format
In reply to Patrick SteinhardtThe output of git-fetch(1) is obviously designed for consumption by users, only: we neatly columnize data, we abbreviate reference names, we print neat arrows and we don't provide information about actual object IDs that have changed. This makes the output format basically unusable in the context of scripted invocations of git-fetch(1) that want to learn about the exact changes that the command performs.
Introduce a new machine-parseable "porcelain" output format that is supposed to fix this shortcoming. This output format is intended to provide information about every reference that is about to be updated, the old object ID that the reference has been pointing to and the new object ID it will be updated to. Furthermore, the output format provides the same flags as the human-readable format to indicate basic conditions for each reference update like whether it was a fast-forward update, a branch deletion, a rejected update or others.
The output format is quite simple:
``` <flag> <old-object-id> <new-object-id> <local-reference>\n ```
We assume two conditions which are generally true:
- The old and new object IDs have fixed known widths and cannot contain spaces.
- References cannot contain newlines.
With these assumptions, the output format becomes unambiguously parseable. Furthermore, given that this output is designed to be consumed by scripts, the machine-readable data is printed to stdout instead of stderr like the human-readable output is. This is mostly done so that other data printed to stderr, like error messages or progress meters, don't interfere with the parseable data.
A notable ommission here is that the output format does not include the remote from which a reference was fetched, which might be important information especially in the context of multi-remote fetches. But as such a format would require us to print the remote for every single reference update due to parallelizable fetches it feels wasteful for the most likely usecase, which is when fetching from a single remote.
In a similar spirit, a second restriction is that this cannot be used with --recurse-submodules. This is because any reference updates would be ambiguous without also printing the repository in which the update happens.
Considering that both multi-remote and submodule fetches are rather niche and likely not going to be useful for the majority of usecases these omissions feel acceptable. If usecases for either of these come up in the future though it is easy enough to add a new "porcelain-v2" format that adds this information.
--- Documentation/fetch-options.txt | 6 ++ Documentation/git-fetch.txt | 9 +++ builtin/fetch.c | 91 +++++++++++++++++++----- t/t5574-fetch-output.sh | 120 ++++++++++++++++++++++++++++++++ 4 files changed, 208 insertions(+), 18 deletions(-)
Documentation/fetch-options.txt +6 -0
@@ -78,6 +78,12 @@ linkgit:git-config[1]. --dry-run:: Show what would be done, without making any changes. +--porcelain:: + Print the output to standard output in an easy-to-parse format for + scripts. See section OUTPUT in linkgit:git-fetch[1] for details. ++ +This is incompatible with `--recurse-submodules=[yes|on-demand]`. + ifndef::git-pull[] --[no-]write-fetch-head:: Write the list of remote refs fetched in the `FETCH_HEAD`
Documentation/git-fetch.txt +9 -0
@@ -204,6 +204,15 @@ representing the status of a single ref. Each line is of the form: <flag> <summary> <from> -> <to> [<reason>] ------------------------------- +When using `--porcelain`, the output format is intended to be +machine-parseable. In contrast to the human-readable output formats it +thus prints to standard output instead of standard error. Each line is +of the form: + +------------------------------- +<flag> <old-object-id> <new-object-id> <local-reference> +------------------------------- + The status of up-to-date refs is shown only if the --verbose option is used.
builtin/fetch.c +73 -18
252 lines not on this page · raw message
t/t5574-fetch-output.sh +120 -0
132 lines not on this page · raw message
Signature · 1 line
2.40.1
Re: [PATCH v3 8/8] fetch: introduce machine-parseable "porcelain" output format
This version looks great! I only have minor comments.
Quoted 12 lines · Patrick Steinhardt
A notable ommission here is that the output format does not include the remote from which a reference was fetched, which might be important information especially in the context of multi-remote fetches. But as such a format would require us to print the remote for every single reference update due to parallelizable fetches it feels wasteful for the most likely usecase, which is when fetching from a single remote.
In a similar spirit, a second restriction is that this cannot be used with
--recurse-submodules. This is because any reference updates would be ambiguous without also printing the repository in which the update happens.
Quoted 5 lines
Considering that both multi-remote and submodule fetches are rather niche and likely not going to be useful for the majority of usecases these omissions feel acceptable. If usecases for either of these come up in the future though it is easy enough to add a new "porcelain-v2" format that adds this information.
As a point of clarification, I think these options aren't niche in themselves, but they are more user-facing, so using them _in conjunction with_ --porcelain is probably pretty niche, so I think this is okay for now.
Quoted 24 lines
Documentation/fetch-options.txt +6
-0@@ -78,6 +78,12 @@ linkgit:git-config[1]. --dry-run:: Show what would be done, without making any changes. +--porcelain:: + Print the output to standard output in an easy-to-parse format for + scripts. See section OUTPUT in linkgit:git-fetch[1] for details. ++ +This is incompatible with `--recurse-submodules=[yes|on-demand]`. + ifndef::git-pull[] --[no-]write-fetch-head:: Write the list of remote refs fetched in the `FETCH_HEAD`Documentation/git-fetch.txt +1
-0@@ -204,6 +204,15 @@ representing the status of a single ref. Each line is of the form: <flag> <summary> <from> -> <to> [<reason>] ------------------------------- +When using `--porcelain`, the output format is intended to be
Documentation/git-fetch.txt representing the status of a single ref. Each line is of the form: · line 217
Quoted 10 lines
Documentation/git-fetch.txt +8
-0+machine-parseable. In contrast to the human-readable output formats it +thus prints to standard output instead of standard error. Each line is +of the form: + +------------------------------- +<flag> <old-object-id> <new-object-id> <local-reference> +------------------------------- + The status of up-to-date refs is shown only if the --verbose option is used.
I hadn't commented on the docs before, but they are very clear. Thanks!
Quoted 9 lines
Documentation/git-fetch.txt +1
-0@@ -1830,6 +1857,7 @@ struct parallel_fetch_state { const char **argv; struct string_list *remotes; int next, result; + enum display_format format; }; static int fetch_next_remote(struct child_process *cp,
Documentation/git-fetch.txt static int fetch_next_remote(struct child_process *cp, · line 1883
Quoted 8 lines
Documentation/git-fetch.txt +1
-1@@ -1849,7 +1877,7 @@ static int fetch_next_remote(struct child_process *cp, strvec_push(&cp->args, remote); cp->git_cmd = 1; - if (verbosity >= 0) + if (verbosity >= 0 && state->format != DISPLAY_FORMAT_PORCELAIN) printf(_("Fetching %s\n"), remote); return 1;
Here and elsewhere, I wonder if it's clearer to name the variable "porcelain" and separate it from "enum display_format". Then we can check "porcelain" directly instead of using "format == DISPLAY_FORMAT_PORCELAIN" as a proxy...
Quoted 11 lines
Documentation/git-fetch.txt +7
-0+static int opt_parse_porcelain(const struct option *opt, const char *arg, int unset) +{ + enum display_format *format = opt->value; + *format = DISPLAY_FORMAT_PORCELAIN; + return 0; +} + int cmd_fetch(int argc, const char **argv, const char *prefix) { const char *bundle_uri;
Documentation/git-fetch.txt int cmd_fetch(int argc, const char **argv, const char *prefix) · line 2147
Quoted 8 lines
Documentation/git-fetch.txt +2
-0@@ -2104,6 +2140,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules), OPT_BOOL(0, "dry-run", &dry_run, N_("dry run")), + OPT_CALLBACK_F(0, "porcelain", &display_format, NULL, N_("machine-readable output"), + PARSE_OPT_NOARG|PARSE_OPT_NONEG, opt_parse_porcelain), OPT_BOOL(0, "write-fetch-head", &write_fetch_head, N_("write fetched references to the FETCH_HEAD file")), OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
e.g. since we are reusing the "display_format" variable, we need to make sure we parse "--porcelain" after we read "fetch.output". I double-checked to make sure we were doing the right thing, though it would be nice to not have to worry about those sorts of things. This shouldn't hold up the series though.
Quoted 5 lines
Documentation/git-fetch.txt +5
-0+test_expect_success 'fetch porcelain with multiple remotes' ' + test_when_finished "rm -rf porcelain" && + + git clone . porcelain && + git -C porcelain remote add second-remote "$PWD" &&
Documentation/git-fetch.txt
Quoted 10 lines
Documentation/git-fetch.txt +10
-0+ git -C porcelain fetch second-remote && + + test_commit --no-tag multi-commit && + old_commit=$(git rev-parse HEAD~) && + new_commit=$(git rev-parse HEAD) && + + cat >expect <<-EOF && + $old_commit $new_commit refs/remotes/origin/force-updated + $old_commit $new_commit refs/remotes/second-remote/force-updated + EOF
The only thing in this test that relies on the previous test is that HEAD is pointing to "force-updated", and it's hard to tell where HEAD is since the previous test is so long. Could we create a new branch instead?
Re: [PATCH v3 8/8] fetch: introduce machine-parseable "porcelain" output format
Quoted 22 lines · Glen Choo
This version looks great! I only have minor comments.
Quoted 12 lines · Patrick Steinhardt
A notable ommission here is that the output format does not include the remote from which a reference was fetched, which might be important information especially in the context of multi-remote fetches. But as such a format would require us to print the remote for every single reference update due to parallelizable fetches it feels wasteful for the most likely usecase, which is when fetching from a single remote.
In a similar spirit, a second restriction is that this cannot be used with
--recurse-submodules. This is because any reference updates would be ambiguous without also printing the repository in which the update happens.Quoted 5 lines
Considering that both multi-remote and submodule fetches are rather niche and likely not going to be useful for the majority of usecases these omissions feel acceptable. If usecases for either of these come up in the future though it is easy enough to add a new "porcelain-v2" format that adds this information.
Quoted 4 lines
As a point of clarification, I think these options aren't niche in themselves, but they are more user-facing, so using them
_inconjunctionwith_--porcelain is probably pretty niche, so I think this is okay for now.
Yeah, that's what I indeed intended to say. Will clarify.
[snip]
Quoted 16 lines
Quoted 16 lines
builtin/fetch.c +2
-1@@ -1830,6 +1857,7 @@ struct parallel_fetch_state {const char **argv;struct string_list *remotes;int next, result; +enum display_format format; }; static int fetch_next_remote(struct child_process *cp,@@ -1849,7 +1877,7 @@ static int fetch_next_remote(struct child_process *cp,strvec_push (&cp->args, remote); cp->git_cmd =1 ; -if (verbosity >=0 ) +if (verbosity >=0 && state->format !=DISPLAY_FORMAT_PORCELAIN )printf (_ ("Fetching %s\n" ), remote);
Quoted 6 lines
Review static int fetch_next_remote(struct child_process *cp, · line 1883
Quoted 1 line
(quoted diff) +0
-0return 1;Here and elsewhere, I wonder if it's clearer to name the variable "porcelain" and separate it from "enum
display_format". Then we can check "porcelain" directly instead of using "format ==DISPLAY_FORMAT_PORCELAIN" as a proxy...
For now I'd like to keep this as-is: it's easier to keep track of this when there is only a single variable that keeps track of the output format. But if we were to add additional porcelain formats in the future I agree that it would be nice to refactor the code as you propose.
I'd rather keep the simpler version for now though where we only have a single state to worry about.
Quoted 17 lines
Quoted 17 lines
builtin/fetch.c +9
-0+static int opt_parse_porcelain (const struct option *opt,const char *arg,int unset) +{ +enum display_format *format = opt->value; + *format =DISPLAY_FORMAT_PORCELAIN ; +return 0 ; +} +int cmd_fetch (int argc,const char **argv,const char *prefix) {const char *bundle_uri;@@ -2104,6 +2140,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),OPT_BOOL (0 ,"dry-run" , &dry_run,N_ ("dry run" )), +OPT_CALLBACK_F (0 ,"porcelain" , &display_format,NULL ,N_ ("machine-readable output" ), +PARSE_OPT_NOARG |PARSE_OPT_NONEG , opt_parse_porcelain), OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
Quoted 8 lines
builtin/fetch.c int cmd_fetch(int argc, const char **argv, const char *prefix) · line 2147
Quoted 2 lines
builtin/fetch.c +0
-0N_("write fetched references to the FETCH_HEAD file")), OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),e.g. since we are reusing the "
display_format" variable, we need to make sure we parse "--porcelain" after we read "fetch.output". I double-checked to make sure we were doing the right thing, though it would be nice to not have to worry about those sorts of things. This shouldn't hold up the series though.
In fact it is the other way round: we parse --porcelain first and then only read "fetch.output" in the case where the display_format variable is still set to DISPLAY_FORMAT_UNKNOWN.
So in the end there is no fragile order dependence here -- it would work just the same regardless of whether we first parse command line options or the configuration.
Quoted 12 lines
Quoted 12 lines
t/t5574-fetch-output.sh +12
-0+test_expect_success 'fetch porcelain with multiple remotes' ' + test_when_finished "rm -rf porcelain" && + + git clone . porcelain && + git -C porcelain remote add second-remote "$PWD" && + git -C porcelain fetch second-remote && + + test_commit --no-tag multi-commit && + old_commit=$(git rev-parse HEAD~) && + new_commit=$(git rev-parse HEAD) && + + cat >expect <<-EOF &&
Quoted 8 lines
t/t5574-fetch-output.sh test_expect_success 'fetch compact output' ' · line 133
Quoted 3 lines
t/t5574-fetch-output.sh +3
-0+ $old_commit $new_commit refs/remotes/origin/force-updated + $old_commit $new_commit refs/remotes/second-remote/force-updated + EOFThe only thing in this test that relies on the previous test is that HEAD is pointing to "force-updated", and it's hard to tell where HEAD is since the previous test is so long. Could we create a new branch instead?
Makes sense, done.
Patrick
Re: [PATCH v3 8/8] fetch: introduce machine-parseable "porcelain" output format
In reply to Patrick SteinhardtDocumentation/fetch-options.txt linkgit:git-config[1]. · line 86
Quoted 6 lines · Patrick Steinhardt
+--porcelain::
Documentation/fetch-options.txt +5
-0+ Print the output to standard output in an easy-to-parse format for + scripts. See section OUTPUT in linkgit:git-fetch[1] for details. ++ +This is incompatible with `--recurse-submodules=[yes|on-demand]`. +
We should probably specify that this will override any setting of "fetch.output" (and perhaps add a test for it). Most readers will probably guess that this is the case, but the extra clarity wouldn't hurt.
Quoted 2 lines · Patrick Steinhardt
this is the third version of my patch series to introduce a machine-parseable output for git-fetch(1).
Which base did you pick to build these 8 patches? It seems that 6/8 is not happy when the series is attempted to be applied on top of many recent tips of 'master' that I tried (including 667fcf4e15 that the previous iteration was queued on top of in my tree).
Thanks.
Quoted 9 lines · Junio C Hamano
Quoted 2 lines · Patrick Steinhardt
this is the third version of my patch series to introduce a machine-parseable output for git-fetch(1).
Which base did you pick to build these 8 patches? It seems that 6/8 is not happy when the series is attempted to be applied on top of many recent tips of 'master' that I tried (including
667fcf4e15that the previous iteration was queued on top of in my tree).
OK, I found e9dffbc7 (Merge branch 'ps/fetch-ref-update-reporting', 2023-04-06). I can manage from here.
Thanks.
Quoted 12 lines · Junio C Hamano
Quoted 9 lines · Junio C Hamano
Quoted 2 lines · Patrick Steinhardt
this is the third version of my patch series to introduce a machine-parseable output for git-fetch(1).
Which base did you pick to build these 8 patches? It seems that 6/8 is not happy when the series is attempted to be applied on top of many recent tips of 'master' that I tried (including
667fcf4e15that the previous iteration was queued on top of in my tree).
Quoted 4 lines
OK, I found
e9dffbc7(Merge branch 'ps/fetch-ref-update-reporting', 2023-04-06). I can manage from here.Thanks.
Yeah, that's where I based it off from. Sorry for not explicitly mentioning this fact.
Patrick
Quoted 4 lines · Patrick Steinhardt
Hi,
this is the third version of my patch series to introduce a machine-parseable output for git-fetch(1).
This version looks great! I only had minor comments this time, mainly on docs and tests.
Thanks for your patience and receptiveness, and for the fruitful discussions :)
Quoted 4 lines · Glen Choo
Quoted 2 lines · Patrick Steinhardt
Hi,
Quoted 8 lines
Quoted 2 lines
this is the third version of my patch series to introduce a machine-parseable output for git-fetch(1).
This version looks great! I only had minor comments this time, mainly on docs and tests.
Thanks for your patience and receptiveness, and for the fruitful discussions :)
Thank _you_ for the extensive feedback and constructive criticism.
Patrick
v4 cover fetch: introduce machine-parseable output
In reply to Patrick SteinhardtHi,
this is the fourth version of my patch series to introduce a machine-parseable output format for git-fetch(1). It applies on top of e9dffbc7f1 (Merge branch 'ps/fetch-ref-update-reporting', 2023-04-06).
There's only a small set of change compared to v3:
- Patch 1/8: The test added by this change has been significantly simplified.
- Patch 8/8: Reworded the commit message to clarify that parallel fetches and --recurse-submodules aren't niche on their own, but are likely niche combined with --porcelain given that they are user-facing while --porcelain is script-facing.
- Patch 8/8: Clarified that --porcelain takes precedence over the fetch.output config option in the user-facing documentation. I've also added a test to verify that this is the case.
- Patch 8/8: Amended one test to not depend on the state that its preceding test cretaes.
Thanks to Glen for the feedback.
Patrick
Patrick Steinhardt (8):
fetch: fix --no-recurse-submodules with multi-remote fetches
fetch: split out tests for output format
fetch: add a test to exercise invalid output formats
fetch: fix missing from-reference when fetching HEAD:foo
fetch: introduce display_format enum
fetch: move display format parsing into main function
fetch: move option related variables into main function
fetch: introduce machine-parseable "porcelain" output format
Documentation/fetch-options.txt | 7 + Documentation/git-fetch.txt | 9 + builtin/fetch.c | 437 +++++++++++++++++++------------- t/t5510-fetch.sh | 53 ---- t/t5526-fetch-submodules.sh | 19 ++ t/t5574-fetch-output.sh | 251 ++++++++++++++++++ 6 files changed, 552 insertions(+), 224 deletions(-) create mode 100755 t/t5574-fetch-output.sh
Range-diff against v3:
1: 4b2b0cfe15 ! 1: d82b42ed34 fetch: fix --no-recurse-submodules with multi-remote fetches
@@ t/t5526-fetch-submodules.sh: test_expect_success 'fetch --all with --recurse-sub
'
+test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" '
-+ test_when_finished "git -C downstream remote remove second" &&
++ test_when_finished "rm -rf src_clone" &&
+
-+ # We need to add a second remote, otherwise --all falls back to the
-+ # normal fetch-one case.
-+ git -C downstream remote add second .. &&
-+ git -C downstream fetch --all &&
++ git clone --recurse-submodules src src_clone &&
++ (
++ cd src_clone &&
++ git remote add secondary ../src &&
++ git config submodule.recurse true &&
++ git config fetch.parallel 0 &&
++ git fetch --all --no-recurse-submodules 2>../actual
++ ) &&
+
-+ add_submodule_commits &&
-+ add_superproject_commits &&
-+ old_commit=$(git rev-parse --short HEAD~) &&
-+ new_commit=$(git rev-parse --short HEAD) &&
-+
-+ git -C downstream fetch --all --no-recurse-submodules >actual.out 2>actual.err &&
-+
-+ cat >expect.out <<-EOF &&
-+ Fetching origin
-+ Fetching second
++ cat >expect <<-EOF &&
++ From ../src
++ * [new branch] master -> secondary/master
+ EOF
-+
-+ cat >expect.err <<-EOF &&
-+ From $(test-tool path-utils real_path .)/.
-+ $old_commit..$new_commit super -> origin/super
-+ From ..
-+ $old_commit..$new_commit super -> second/super
-+ EOF
-+
-+ test_cmp expect.out actual.out &&
-+ test_cmp expect.err actual.err
++ test_cmp expect actual
+'
+
test_done
2: 6ebc7450ba = 2: 33112dc51a fetch: split out tests for output format
3: 78479922ac = 3: 006ea93afb fetch: add a test to exercise invalid output formats
4: 46e1266ab0 = 4: e599ea6d33 fetch: fix missing from-reference when fetching HEAD:foo
5: acc0f7f520 = 5: 80ac00b0c4 fetch: introduce `display_format` enum
6: cd23440128 = 6: 826b8b7bc0 fetch: move display format parsing into main function
7: edbc31013f = 7: 20f2e061d6 fetch: move option related variables into main function
8: e132d9494e ! 8: 24ae381950 fetch: introduce machine-parseable "porcelain" output format
@@ Commit message
be ambiguous without also printing the repository in which the update
happens.
- Considering that both multi-remote and submodule fetches are rather
- niche and likely not going to be useful for the majority of usecases
- these omissions feel acceptable. If usecases for either of these come up
- in the future though it is easy enough to add a new "porcelain-v2"
- format that adds this information.
+ Considering that both multi-remote and submodule fetches are user-facing
+ features, using them in conjunction with `--porcelain` that is intended
+ for scripting purposes is likely not going to be useful in the majority
+ of cases. With that in mind these restrictions feel acceptable. If
+ usecases for either of these come up in the future though it is easy
+ enough to add a new "porcelain-v2" format that adds this information.
Signed-off-by: Patrick Steinhardt <[email protected]>
@@ Documentation/fetch-options.txt: linkgit:git-config[1].
+ Print the output to standard output in an easy-to-parse format for
+ scripts. See section OUTPUT in linkgit:git-fetch[1] for details.
++
-+This is incompatible with `--recurse-submodules=[yes|on-demand]`.
++This is incompatible with `--recurse-submodules=[yes|on-demand]` and takes
++precedence over the `fetch.output` config option.
+
ifndef::git-pull[]
--[no-]write-fetch-head::
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
+test_expect_success 'fetch porcelain with multiple remotes' '
+ test_when_finished "rm -rf porcelain" &&
+
++ git switch --create multiple-remotes &&
+ git clone . porcelain &&
+ git -C porcelain remote add second-remote "$PWD" &&
+ git -C porcelain fetch second-remote &&
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
+ new_commit=$(git rev-parse HEAD) &&
+
+ cat >expect <<-EOF &&
-+ $old_commit $new_commit refs/remotes/origin/force-updated
-+ $old_commit $new_commit refs/remotes/second-remote/force-updated
++ $old_commit $new_commit refs/remotes/origin/multiple-remotes
++ $old_commit $new_commit refs/remotes/second-remote/multiple-remotes
+ EOF
+
+ git -C porcelain fetch --porcelain --all >actual 2>stderr &&
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
+ test_must_fail git -C porcelain fetch --porcelain --recurse-submodules=on-demand 2>stderr &&
+ test_cmp expect stderr
+'
++
++test_expect_success 'fetch porcelain overrides fetch.output config' '
++ test_when_finished "rm -rf porcelain" &&
++
++ git switch --create config-override &&
++ git clone . porcelain &&
++ test_commit new-commit &&
++ old_commit=$(git rev-parse HEAD~) &&
++ new_commit=$(git rev-parse HEAD) &&
++
++ cat >expect <<-EOF &&
++ $old_commit $new_commit refs/remotes/origin/config-override
++ * $ZERO_OID $new_commit refs/tags/new-commit
++ EOF
++
++ git -C porcelain -c fetch.output=compact fetch --porcelain >stdout 2>stderr &&
++ test_must_be_empty stderr &&
++ test_cmp expect stdout
++'
+
test_expect_success 'fetch output with HEAD' '
test_when_finished "rm -rf head" &&Signature · 1 line
2.40.1
PATCH v4 1/8 fetch: fix `--no-recurse-submodules` with multi-remote fetches
When running git fetch --no-recurse-submodules, the exectation is that we don't fetch any submodules. And while this works for fetches of a single remote, it doesn't when fetching multiple remotes at once. The result is that we do recurse into submodules even though the user has explicitly asked us not to.
This is because while we pass on --recurse-submodules={yes,on-demand} if specified by the user, we don't pass on --no-recurse-submodules to the subprocess spawned to perform the submodule fetch.
Fix this by also forwarding this flag as expected.
--- builtin/fetch.c | 2 ++ t/t5526-fetch-submodules.sh | 19 +++++++++++++++++++ 2 files changed, 21 insertions(+)
builtin/fetch.c +2 -0
@@ -1876,6 +1876,8 @@ static void add_options_to_argv(struct strvec *argv)strvec_push (argv,"--keep" );if (recurse_submodules ==RECURSE_SUBMODULES_ON )strvec_push (argv,"--recurse-submodules" ); +else if (recurse_submodules ==RECURSE_SUBMODULES_OFF ) +strvec_push (argv,"--no-recurse-submodules" );else if (recurse_submodules ==RECURSE_SUBMODULES_ON_DEMAND )strvec_push (argv,"--recurse-submodules=on-demand" );if (tags ==TAGS_SET )
t/t5526-fetch-submodules.sh +19 -0
@@ -1180,4 +1180,23 @@ test_expect_success 'fetch --all with --recurse-submodules with multiple' ' test_line_count = 2 fetch-subs ' +test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" ' + test_when_finished "rm -rf src_clone" && + + git clone --recurse-submodules src src_clone && + ( + cd src_clone && + git remote add secondary ../src && + git config submodule.recurse true && + git config fetch.parallel 0 && + git fetch --all --no-recurse-submodules 2>../actual + ) && + + cat >expect <<-EOF && + From ../src + * [new branch] master -> secondary/master + EOF + test_cmp expect actual +' + test_done
Signature · 1 line
2.40.1
Re: [PATCH v4 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches
Quoted 11 lines · Patrick Steinhardt
When running
git fetch --no-recurse-submodules, the exectation is that we don't fetch any submodules. And while this works for fetches of a single remote, it doesn't when fetching multiple remotes at once. The result is that we do recurse into submodules even though the user has explicitly asked us not to.This is because while we pass on
--recurse-submodules={yes,on-demand}if specified by the user, we don't pass on--no-recurse-submodulesto the subprocess spawned to perform the submodule fetch.Fix this by also forwarding this flag as expected.
Makes sense.
t/t5526-fetch-submodules.sh test_expect_success 'fetch --all with --recurse-submodules with multiple' ' · line 1190
Quoted 8 lines
t/t5526-fetch-submodules.sh +8
-0+test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" ' + test_when_finished "rm -rf src_clone" && + + git clone --recurse-submodules src src_clone && + ( + cd src_clone && + git remote add secondary ../src && + git config submodule.recurse true &&
The above two is essential to this test; we are interested in making sure that --no-recurse-submodules is propagated down even when the "--all" option is used, and we want another remote for that. We set the default to recurse, so that passing "--no-recurse-submodules" would defeat it, but just refraining to pass "--recurse-submodules" would cause us to recurse.
t/t5526-fetch-submodules.sh test_expect_success 'fetch --all with --recurse-submodules with multiple' ' · line 1191
Quoted 1 line
t/t5526-fetch-submodules.sh +1
-0+ git config fetch.parallel 0 &&
Is this necessary for the purpose of the test, though? It should not hurt, but we do not require the end-users to set it in real life for the parallel fetching to work, either, right?
Just being curious.
t/t5526-fetch-submodules.sh test_expect_success 'fetch --all with --recurse-submodules with multiple' ' · line 1200
Quoted 9 lines
t/t5526-fetch-submodules.sh +9
-0+ git fetch --all --no-recurse-submodules 2>../actual + ) && + + cat >expect <<-EOF && + From ../src + * [new branch] master -> secondary/master + EOF + test_cmp expect actual +'
In the context of a series that attempts to introduce a new stable output format for machine consumption, which implies the traditional output can change to match human users' preference, this test feels a bit brittle, but let's wait until the end of the series to judge that.
Looking good. Thanks.
Quoted 1 line
test_done
Re: [PATCH v4 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches
Quoted 5 lines · Junio C Hamano
t/t5526-fetch-submodules.sh test_expect_success 'fetch --all with --recurse-submodules with multiple' ' · line 1191
Quoted 1 line
t/t5526-fetch-submodules.sh +1
-0+ git config fetch.parallel 0 &&Is this necessary for the purpose of the test, though? It should not hurt, but we do not require the end-users to set it in real life for the parallel fetching to work, either, right?
IIUC it would make the test output deterministic if we fetched from both remotes. That doesn't happen here though, so I guess it's not doing anything right now.
Quoted 7 lines
Quoted 7 lines
t/t5526-fetch-submodules.sh +7
-0+ git fetch --all --no-recurse-submodules 2>../actual + ) && + + cat >expect <<-EOF && + From ../src + * [new branch] master -> secondary/master + EOF
Quoted 8 lines
t/t5526-fetch-submodules.sh test_expect_success 'fetch --all with --recurse-submodules with multiple' ' · line 1200
Quoted 2 lines
t/t5526-fetch-submodules.sh +2
-0+ test_cmp expect actual +'In the context of a series that attempts to introduce a new stable output format for machine consumption, which implies the traditional output can change to match human users' preference, this test feels a bit brittle, but let's wait until the end of the series to judge that.
I also find it a bit brittle to assert on the whole output when this test is about checking that we do not fetch the superproject.
Is there a reason you didn't go with the "grep for submodule lines" approach in the previous tests? If it's about catching regressions, IMO your PATCH 2/8 does a good enough job of doing that.
Wondering out loud, I wonder if it makes sense for us to make a bigger distinction between "tests whose purpose is to guard against unexpected changes in output" (i.e. snapshot tests) vs "tests that happen to use output as a way to assert behavior" (i.e. 'regular' behavioral tests). Many GUI app codebases have such a distinction and have different best practices around them.
Re: [PATCH v4 1/8] fetch: fix `--no-recurse-submodules` with multi-remote fetches
Quoted 11 lines · Glen Choo
Quoted 5 lines · Junio C Hamano
t/t5526-fetch-submodules.sh test_expect_success 'fetch --all with --recurse-submodules with multiple' ' · line 1191
Quoted 1 line
t/t5526-fetch-submodules.sh +1
-0+ git config fetch.parallel 0 &&Is this necessary for the purpose of the test, though? It should not hurt, but we do not require the end-users to set it in real life for the parallel fetching to work, either, right?
IIUC it would make the test output deterministic if we fetched from both remotes. That doesn't happen here though, so I guess it's not doing anything right now.
Right, will drop.
Quoted 23 lines
Quoted 7 lines
Quoted 7 lines
t/t5526-fetch-submodules.sh +7
-0+ git fetch --all --no-recurse-submodules 2>../actual + ) && + + cat >expect <<-EOF && + From ../src + * [new branch] master -> secondary/master + EOFQuoted 8 lines
t/t5526-fetch-submodules.sh test_expect_success 'fetch --all with --recurse-submodules with multiple' ' · line 1200
Quoted 2 lines
t/t5526-fetch-submodules.sh +2
-0+ test_cmp expect actual +'In the context of a series that attempts to introduce a new stable output format for machine consumption, which implies the traditional output can change to match human users' preference, this test feels a bit brittle, but let's wait until the end of the series to judge that.
I also find it a bit brittle to assert on the whole output when this test is about checking that we do not fetch the superproject.
Is there a reason you didn't go with the "grep for submodule lines" approach in the previous tests? If it's about catching regressions, IMO your PATCH 2/8 does a good enough job of doing that.
Quoted 6 lines
Wondering out loud, I wonder if it makes sense for us to make a bigger distinction between "tests whose purpose is to guard against unexpected changes in output" (i.e. snapshot tests) vs "tests that happen to use output as a way to assert behavior" (i.e. 'regular' behavioral tests). Many GUI app codebases have such a distinction and have different best practices around them.
Fair enough, will switch to ! grep "Fetching submodule".
Patrick
PATCH v4 2/8 fetch: split out tests for output format
In reply to Patrick SteinhardtWe're about to introduce a new porcelain mode for the output of git-fetch(1). As part of that we'll be introducing a set of new tests that only relate to the output of this command.
Split out tests that exercise the output format of git-fetch(1) so that it becomes easier to verify this functionality as a standalone unit. As the tests assume that the default branch is called "main" we set up the corresponding GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME environment variable accordingly.
--- t/t5510-fetch.sh | 53 ---------------------------------- t/t5574-fetch-output.sh | 63 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+), 53 deletions(-) create mode 100755 t/t5574-fetch-output.sh
t/t5510-fetch.sh +0 -53
@@ -1118,59 +1118,6 @@ test_expect_success 'fetching with auto-gc does not lock up' ' ) ' -test_expect_success 'fetch aligned output' ' - git clone . full-output && - test_commit looooooooooooong-tag && - ( - cd full-output && - git -c fetch.output=full fetch origin >actual 2>&1 && - grep -e "->" actual | cut -c 22- >../actual - ) && - cat >expect <<-\EOF && - main -> origin/main - looooooooooooong-tag -> looooooooooooong-tag - EOF - test_cmp expect actual -' - -test_expect_success 'fetch compact output' ' - git clone . compact && - test_commit extraaa && - ( - cd compact && - git -c fetch.output=compact fetch origin >actual 2>&1 && - grep -e "->" actual | cut -c 22- >../actual - ) && - cat >expect <<-\EOF && - main -> origin/* - extraaa -> * - EOF - test_cmp expect actual -' - -test_expect_success '--no-show-forced-updates' ' - mkdir forced-updates && - ( - cd forced-updates && - git init && - test_commit 1 && - test_commit 2 - ) && - git clone forced-updates forced-update-clone && - git clone forced-updates no-forced-update-clone && - git -C forced-updates reset --hard HEAD~1 && - ( - cd forced-update-clone && - git fetch --show-forced-updates origin 2>output && - test_i18ngrep "(forced update)" output - ) && - ( - cd no-forced-update-clone && - git fetch --no-show-forced-updates origin 2>output && - test_i18ngrep ! "(forced update)" output - ) -' - for section in fetch transfer do test_expect_success "$section.hideRefs affects connectivity check" '
t/t5574-fetch-output.sh new file +63 -0
63 lines not on this page · raw message
Signature · 1 line
2.40.1
PATCH v4 3/8 fetch: add a test to exercise invalid output formats
In reply to Patrick SteinhardtAdd a testcase that exercises the logic when an invalid output format is passed via the fetch.output configuration.
--- t/t5574-fetch-output.sh | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
t/t5574-fetch-output.sh +19 -0
@@ -7,6 +7,25 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME . ./test-lib.sh +test_expect_success 'fetch with invalid output format configuration' ' + test_when_finished "rm -rf clone" && + git clone . clone && + + test_must_fail git -C clone -c fetch.output= fetch origin >actual.out 2>actual.err && + cat >expect <<-EOF && + fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}${SQ} + EOF + test_must_be_empty actual.out && + test_cmp expect actual.err && + + test_must_fail git -C clone -c fetch.output=garbage fetch origin >actual.out 2>actual.err && + cat >expect <<-EOF && + fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}garbage${SQ} + EOF + test_must_be_empty actual.out && + test_cmp expect actual.err +' + test_expect_success 'fetch aligned output' ' git clone . full-output && test_commit looooooooooooong-tag &&
Signature · 1 line
2.40.1
Re: [PATCH v4 3/8] fetch: add a test to exercise invalid output formats
Quoted 7 lines · Patrick Steinhardt
Add a testcase that exercises the logic when an invalid output format is passed via the
fetch.outputconfiguration.Signed-off-by Patrick Steinhardt <[email protected]>--- t/t5574-fetch-output.sh | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
It makes perfect sense to make sure that invalid input gets rejected and the command exits with non-zero status, and it is probably a good thing that the end-user sees a message that explains why the particular input is rejected (even though it adds one more thing that needs to be updated when the message gets reworded).
But do we need to insist on no output to the standard output stream when the command errors out?
Other than that, looking good.
THanks.
Quoted 30 lines
t/t5574-fetch-output.sh +19
-0@@ -7,6 +7,25 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME . ./test-lib.sh +test_expect_success 'fetch with invalid output format configuration' ' + test_when_finished "rm -rf clone" && + git clone . clone && + + test_must_fail git -C clone -c fetch.output= fetch origin >actual.out 2>actual.err && + cat >expect <<-EOF && + fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}${SQ} + EOF + test_must_be_empty actual.out && + test_cmp expect actual.err && + + test_must_fail git -C clone -c fetch.output=garbage fetch origin >actual.out 2>actual.err && + cat >expect <<-EOF && + fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}garbage${SQ} + EOF + test_must_be_empty actual.out && + test_cmp expect actual.err +' + test_expect_success 'fetch aligned output' ' git clone . full-output && test_commit looooooooooooong-tag &&
Re: [PATCH v4 3/8] fetch: add a test to exercise invalid output formats
Quoted 16 lines · Junio C Hamano
Quoted 7 lines · Patrick Steinhardt
Add a testcase that exercises the logic when an invalid output format is passed via the
fetch.outputconfiguration.Signed-off-by Patrick Steinhardt <[email protected]>--- t/t5574-fetch-output.sh | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)It makes perfect sense to make sure that invalid input gets rejected and the command exits with non-zero status, and it is probably a good thing that the end-user sees a message that explains why the particular input is rejected (even though it adds one more thing that needs to be updated when the message gets reworded).
Quoted 4 lines
But do we need to insist on no output to the standard output stream when the command errors out?
Other than that, looking good.
Not really, no. Will change.
Patrick
PATCH v4 4/8 fetch: fix missing from-reference when fetching HEAD:foo
In reply to Patrick Steinhardtstore_updated_refs() parses the remote reference for two purposes:
- It gets used as a note when writing FETCH_HEAD.
- It is passed through to display_ref_update() to display updated references in the following format:
```
* branch master -> master
```
In most cases, the parsed remote reference is the prettified reference name and can thus be used for both cases. But if the remote reference is HEAD, the parsed remote reference becomes empty. This is intended when we write the FETCH_HEAD, where we skip writing the note in that case. But it is not intended when displaying the updated references and would cause us to miss the left-hand side of the displayed reference update:
``` $ git fetch origin HEAD:foo From https://github.com/git/git * [new ref] -> foo ```
The HEAD string is clearly missing from the left-hand side of the arrow, which is further stressed by the point that the following commands show the left-hand side as expected:
``` $ git fetch origin HEAD From https://github.com/git/git * branch HEAD -> FETCH_HEAD $ git fetch origin master From https://github.com/git/git * branch master -> FETCH_HEAD * branch master -> origin/master ```
The logic of how we compute the remote reference name that we ultimately pass to display_ref_update() is not easy to follow. There are three different cases here:
- When the remote reference name is "HEAD" we set the remote reference name to the empty string. This is the case that causes the bug to occur, where we would indeed want to print "HEAD" instead of the empty string. This is what prettify_refname() would return.
- When the remote reference name has a well-known prefix then we strip this prefix. This matches what prettify_refname() does.
- Otherwise, we keep the fully qualified reference name. This also matches what prettify_refname() does.
As the return value of prettify_refname() would do the correct thing for us in all three cases, we can fix the bug by passing through the full remote reference name to display_ref_update(), which learns to call prettify_refname(). At the same time, this also simplifies the code a bit.
Note that this patch also changes formatting of the block that computes the "kind" and "what" variables. This is done on purpose so that it is part of the diff, hopefully making the change easier to comprehend.
--- builtin/fetch.c | 37 +++++++++++++++++++------------------ t/t5574-fetch-output.sh | 29 +++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+), 18 deletions(-)
builtin/fetch.c +19 -18
@@ -918,12 +918,14 @@ static void display_ref_update(struct display_state *display_state, char code, } width = (summary_width +strlen (summary) -gettext_width (summary)); + remote =prettify_refname (remote); + local =prettify_refname (local);strbuf_addf (&display_state->buf," %c %-*s " , code, width, summary);if (!display_state->compact_format) -print_remote_to_local (display_state, remote,prettify_refname (local)); +print_remote_to_local (display_state, remote, local);else -print_compact (display_state, remote,prettify_refname (local)); +print_compact (display_state, remote, local);if (error)strbuf_addf (&display_state->buf," (%s)" , error);strbuf_addch (&display_state->buf,'\n' );
@@ -934,7 +936,7 @@ static void display_ref_update(struct display_state *display_state, char code,static int update_local_ref (struct ref *ref,struct ref_transaction *transaction,struct display_state *display_state, -const char *remote,const struct ref *remote_ref, +const struct ref *remote_ref,int summary_width) {struct commit *current =NULL , *updated;
@@ -946,7 +948,7 @@ static int update_local_ref(struct ref *ref,if (oideq (&ref->old_oid, &ref->new_oid)) {if (verbosity >0 )display_ref_update (display_state,'=' ,_ ("[up to date]" ),NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 0 ; }
@@ -959,7 +961,7 @@ static int update_local_ref(struct ref *ref,*/ display_ref_update (display_state,'!' ,_ ("[rejected]" ),_ ("can't fetch into checked-out branch" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 1 ; }
@@ -970,12 +972,12 @@ static int update_local_ref(struct ref *ref, r =s_update_ref ("updating tag" , ref, transaction,0 );display_ref_update (display_state, r ?'!' :'t' ,_ ("[tag update]" ), r ?_ ("unable to update local ref" ) :NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return r; }else {display_ref_update (display_state,'!' ,_ ("[rejected]" ),_ ("would clobber existing tag" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 1 ; } }
@@ -1008,7 +1010,7 @@ static int update_local_ref(struct ref *ref, r =s_update_ref (msg, ref, transaction,0 );display_ref_update (display_state, r ?'!' :'*' , what, r ?_ ("unable to update local ref" ) :NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return r; }
@@ -1030,7 +1032,7 @@ static int update_local_ref(struct ref *ref, r =s_update_ref ("fast-forward" , ref, transaction,1 );display_ref_update (display_state, r ?'!' :' ' , quickref.buf, r ?_ ("unable to update local ref" ) :NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);strbuf_release (&quickref);return r; }else if (force || ref->force) {
@@ -1042,12 +1044,12 @@ static int update_local_ref(struct ref *ref, r =s_update_ref ("forced-update" , ref, transaction,1 );display_ref_update (display_state, r ?'!' :'+' , quickref.buf, r ?_ ("unable to update local ref" ) :_ ("forced update" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);strbuf_release (&quickref);return r; }else {display_ref_update (display_state,'!' ,_ ("[rejected]" ),_ ("non-fast-forward" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 1 ; } }
@@ -1252,14 +1254,13 @@ static int store_updated_refs(struct display_state *display_state,if (!strcmp (rm->name,"HEAD" )) { kind ="" ; what ="" ; - } -else if (skip_prefix (rm->name,"refs/heads/" , &what)) + }else if (skip_prefix (rm->name,"refs/heads/" , &what)) { kind ="branch" ; -else if (skip_prefix (rm->name,"refs/tags/" , &what)) + }else if (skip_prefix (rm->name,"refs/tags/" , &what)) { kind ="tag" ; -else if (skip_prefix (rm->name,"refs/remotes/" , &what)) + }else if (skip_prefix (rm->name,"refs/remotes/" , &what)) { kind ="remote-tracking branch" ; -else { + }else { kind ="" ; what = rm->name; }
@@ -1277,7 +1278,7 @@ static int store_updated_refs(struct display_state *display_state, display_state->url_len);if (ref) { - rc |=update_local_ref (ref, transaction, display_state, what, + rc |=update_local_ref (ref, transaction, display_state, rm, summary_width);free (ref); }else if (write_fetch_head || dry_run) {
@@ -1288,7 +1289,7 @@ static int store_updated_refs(struct display_state *display_state,*/ display_ref_update (display_state,'*' , *kind ? kind :"branch" ,NULL , - *what ? what :"HEAD" , + rm->name,"FETCH_HEAD" , summary_width); } }
t/t5574-fetch-output.sh +29 -0
@@ -56,6 +56,35 @@ test_expect_success 'fetch compact output' ' test_cmp expect actual ' +test_expect_success 'fetch output with HEAD' ' + test_when_finished "rm -rf head" && + git clone . head && + + git -C head fetch --dry-run origin HEAD >actual.out 2>actual.err && + cat >expect <<-EOF && + From $(test-tool path-utils real_path .)/. + * branch HEAD -> FETCH_HEAD + EOF + test_must_be_empty actual.out && + test_cmp expect actual.err && + + git -C head fetch origin HEAD >actual.out 2>actual.err && + test_must_be_empty actual.out && + test_cmp expect actual.err && + + git -C head fetch --dry-run origin HEAD:foo >actual.out 2>actual.err && + cat >expect <<-EOF && + From $(test-tool path-utils real_path .)/. + * [new ref] HEAD -> foo + EOF + test_must_be_empty actual.out && + test_cmp expect actual.err && + + git -C head fetch origin HEAD:foo >actual.out 2>actual.err && + test_must_be_empty actual.out && + test_cmp expect actual.err +' + test_expect_success '--no-show-forced-updates' ' mkdir forced-updates && (
Signature · 1 line
2.40.1
Re: [PATCH v4 4/8] fetch: fix missing from-reference when fetching HEAD:foo
Quoted 8 lines · Patrick Steinhardt
But it is not intended when displaying the updated references and would cause us to miss the left-hand side of the displayed reference update:
``` $ git fetch origin HEAD:foo From https://github.com/git/git * [new ref] -> foo ```
Quoted 8 lines
The HEAD string is clearly missing from the left-hand side of the arrow, which is further stressed by the point that the following commands show the left-hand side as expected:
```
$ git fetch origin HEAD From https://github.com/git/git * branch HEAD -> FETCH_HEAD
I do not mind being explicit and showing HEAD in this case for the sake of consistency.
But speaking for the past developers, it was deliberate to omit what is common from the output to make it more terse, IIRC, and I think it is unfair to call it a "BUG".
Back when we wrote git-fetch-script, the output was a lot more verbose, and through efforts like 165f3902 (git-fetch: more terse fetch output, 2007-11-03) and numerous others over time, we got to the current output.
Quoted 3 lines
Note that this patch also changes formatting of the block that computes the "kind" and "what" variables. This is done on purpose so that it is part of the diff, hopefully making the change easier to comprehend.
Just to help readers, "kind" is the category like branch, tag, etc. and "what" is the concrete name like 'master' and 'foo'.
Quoted 31 lines
builtin/fetch.c +5
-3@@ -918,12 +918,14 @@ static void display_ref_update(struct display_state *display_state, char code, } width = (summary_width +strlen (summary) -gettext_width (summary)); + remote =prettify_refname (remote); + local =prettify_refname (local);strbuf_addf (&display_state->buf," %c %-*s " , code, width, summary);if (!display_state->compact_format) -print_remote_to_local (display_state, remote,prettify_refname (local)); +print_remote_to_local (display_state, remote, local);else -print_compact (display_state, remote,prettify_refname (local)); +print_compact (display_state, remote, local);if (error)strbuf_addf (&display_state->buf," (%s)" , error);strbuf_addch (&display_state->buf,'\n' );@@ -934,7 +936,7 @@ static void display_ref_update(struct display_state *display_state, char code,static int update_local_ref (struct ref *ref,struct ref_transaction *transaction,struct display_state *display_state, -const char *remote,const struct ref *remote_ref, +const struct ref *remote_ref,int summary_width) {struct commit *current =NULL , *updated;
builtin/fetch.c static int update_local_ref(struct ref *ref, · line 951
Quoted 5 lines
builtin/fetch.c +1
-1@@ -946,7 +948,7 @@ static int update_local_ref(struct ref *ref,if (oideq (&ref->old_oid, &ref->new_oid)) {if (verbosity >0 )display_ref_update (display_state,'=' ,_ ("[up to date]" ),NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);
Makes sense. The variable "remote" (now removed) holds what to write to FETCH_HEAD to be used to formulate a merge message by the caller, but this function is purely to report the ref updates and has no need to have access to that information.
Quoted 9 lines
builtin/fetch.c +1
-3@@ -1252,14 +1254,13 @@ static int store_updated_refs(struct display_state *display_state,if (!strcmp (rm->name,"HEAD" )) { kind ="" ; what ="" ; - } -else if (skip_prefix (rm->name,"refs/heads/" , &what)) + }else if (skip_prefix (rm->name,"refs/heads/" , &what)) { kind ="branch" ; -else if (skip_prefix (rm->name,"refs/tags/" , &what))
builtin/fetch.c static int store_updated_refs(struct display_state *display_state, · line 1266
Quoted 10 lines
builtin/fetch.c +3
-2+ }else if (skip_prefix (rm->name,"refs/tags/" , &what)) { kind ="tag" ; - else if (skip_prefix(rm->name, "refs/remotes/", &what)) + }else if (skip_prefix (rm->name,"refs/remotes/" , &what)) { kind ="remote-tracking branch" ; - else { + }else { kind ="" ; what = rm->name; }
This is a bit noisier than necessary. It took me a while until I realized that this hunk is a no-op.
builtin/fetch.c static int store_updated_refs(struct display_state *display_state, · line 1283
Quoted 8 lines
builtin/fetch.c +1
-1@@ -1277,7 +1278,7 @@ static int store_updated_refs(struct display_state *display_state, display_state->url_len);if (ref) { - rc |=update_local_ref (ref, transaction, display_state, what, + rc |=update_local_ref (ref, transaction, display_state, rm, summary_width);free (ref);
Good.
builtin/fetch.c static int store_updated_refs(struct display_state *display_state, · line 1293
Quoted 7 lines
builtin/fetch.c +1
-1@@ -1288,7 +1289,7 @@ static int store_updated_refs(struct display_state *display_state,*/ display_ref_update (display_state,'*' , *kind ? kind :"branch" ,NULL , - *what ? what :"HEAD" , + rm->name,"FETCH_HEAD" , summary_width);
Good, too. The original cleared "what" and then to compensate for that had a logic to turn it back to "HEAD", but that is all gone by passing rm->name down.
I think we could pass "rm" and leave it to display_ref_update() what string to use, if we wanted to further refine the output later.
Then somebody in the future may even want to see "HEAD" to be shown as an empty string and that can all be done in display_ref_update(). It would fix the inconsistency that "git fetch origin HEAD" reports "HEAD -> FETCH_HEAD" by hiding "HEAD" just like the case where fetching "HEAD:foo" does, going in the other direction, I would think.
Thanks.
Re: [PATCH v4 4/8] fetch: fix missing from-reference when fetching HEAD:foo
Quoted 22 lines · Junio C Hamano
Quoted 8 lines · Patrick Steinhardt
But it is not intended when displaying the updated references and would cause us to miss the left-hand side of the displayed reference update:
``` $ git fetch origin HEAD:foo From https://github.com/git/git * [new ref] -> foo ```Quoted 8 lines
The HEAD string is clearly missing from the left-hand side of the arrow, which is further stressed by the point that the following commands show the left-hand side as expected:
```
$ git fetch origin HEAD From https://github.com/git/git * branch HEAD -> FETCH_HEADI do not mind being explicit and showing HEAD in this case for the sake of consistency.
Quoted 8 lines
But speaking for the past developers, it was deliberate to omit what is common from the output to make it more terse, IIRC, and I think it is unfair to call it a "BUG".
Back when we wrote git-fetch-script, the output was a lot more verbose, and through efforts like
165f3902(git-fetch: more terse fetch output, 2007-11-03) and numerous others over time, we got to the current output.
That's fair. It's still not quite clear whether this behaviour is in fact intentional though. Quoting 165f3902 (git-fetch: more terse fetch output, 2007-11-03), the weird corner case is not documented:
This makes the fetch output much more terse and prettier on a 80 column
display, based on a consensus reached on the mailing list. Here's an
example output:
Receiving objects: 100% (5439/5439), 1.60 MiB | 636 KiB/s, done.
Resolving deltas: 100% (4604/4604), done.
From git://git.kernel.org/pub/scm/git/git
! [rejected] html -> origin/html (non fast forward)
136e631..f45e867 maint -> origin/maint (fast forward)
9850e2e..44dd7e0 man -> origin/man (fast forward)
3e4bb08..e3d6d56 master -> origin/master (fast forward)
fa3665c..536f64a next -> origin/next (fast forward)
+ 4f6d9d6...768326f pu -> origin/pu (forced update)
* [new branch] todo -> origin/todoI've reformulated the commit message to talk about an inconsistency instead of a bug.
Patrick
PATCH v4 5/8 fetch: introduce `display_format` enum
In reply to Patrick SteinhardtWe currently have two different display formats in git-fetch(1) with the "full" and "compact" formats. This is tracked with a boolean value that simply denotes whether the display format is supposed to be compacted or not. This works reasonably well while there are only two formats, but we're about to introduce another format that will make this a bit more awkward to use.
Introduce a enum display_format that is more readily extensible.
--- builtin/fetch.c | 101 ++++++++++++++++++++++++++++++------------------ 1 file changed, 64 insertions(+), 37 deletions(-)
builtin/fetch.c +64 -37
@@ -48,11 +48,17 @@ enum {TAGS_SET =2 }; +enum display_format { +DISPLAY_FORMAT_UNKNOWN =0 , +DISPLAY_FORMAT_FULL , +DISPLAY_FORMAT_COMPACT , +}; +struct display_state {struct strbuf buf;int refcol_width; -int compact_format; +enum display_format format;char *url;int url_len, shown_url;
@@ -784,7 +790,6 @@ static int refcol_width(const struct ref *ref, int compact_format)static void display_state_init (struct display_state *display_state,struct ref *ref_map,const char *raw_url) { -struct ref *rm;const char *format ="full" ;int i;
@@ -809,31 +814,42 @@ static void display_state_init(struct display_state *display_state, struct ref *git_config_get_string_tmp ("fetch.output" , &format);if (!strcasecmp (format,"full" )) - display_state->compact_format =0 ; + display_state->format =DISPLAY_FORMAT_FULL ;else if (!strcasecmp (format,"compact" )) - display_state->compact_format =1 ; + display_state->format =DISPLAY_FORMAT_COMPACT ;else die (_ ("invalid value for '%s': '%s'" ),"fetch.output" , format); - display_state->refcol_width =10 ; -for (rm = ref_map; rm; rm = rm->next) { -int width; +switch (display_state->format) { +case DISPLAY_FORMAT_FULL : +case DISPLAY_FORMAT_COMPACT : { +struct ref *rm; -if (rm->status ==REF_STATUS_REJECT_SHALLOW || - !rm->peer_ref || - !strcmp (rm->name,"HEAD" )) -continue ; + display_state->refcol_width =10 ; +for (rm = ref_map; rm; rm = rm->next) { +int width; - width =refcol_width (rm, display_state->compact_format); +if (rm->status ==REF_STATUS_REJECT_SHALLOW || + !rm->peer_ref || + !strcmp (rm->name,"HEAD" )) +continue ; -/* -* Not precise calculation for compact mode because '*' can -* appear on the left hand side of '->' and shrink the column -* back. -*/ -if (display_state->refcol_width < width) - display_state->refcol_width = width; + width =refcol_width (rm, display_state->format ==DISPLAY_FORMAT_COMPACT ); + +/* +* Not precise calculation for compact mode because '*' can +* appear on the left hand side of '->' and shrink the column +* back. +*/ +if (display_state->refcol_width < width) + display_state->refcol_width = width; + } + +break ; + } +default : +BUG ("unexpected display format %d" , display_state->format); } }
@@ -904,30 +920,41 @@ static void display_ref_update(struct display_state *display_state, char code, const char *remote, const char *local, int summary_width) { - int width; - if (verbosity < 0) return;strbuf_reset (&display_state->buf); -if (!display_state->shown_url) { -strbuf_addf (&display_state->buf,_ ("From %.*s\n" ), - display_state->url_len, display_state->url); - display_state->shown_url =1 ; +switch (display_state->format) { +case DISPLAY_FORMAT_FULL : +case DISPLAY_FORMAT_COMPACT : { +int width; + +if (!display_state->shown_url) { +strbuf_addf (&display_state->buf,_ ("From %.*s\n" ), + display_state->url_len, display_state->url); + display_state->shown_url =1 ; + } + + width = (summary_width +strlen (summary) -gettext_width (summary)); + remote =prettify_refname (remote); + local =prettify_refname (local); + +strbuf_addf (&display_state->buf," %c %-*s " , code, width, summary); + +if (display_state->format !=DISPLAY_FORMAT_COMPACT ) +print_remote_to_local (display_state, remote, local); +else +print_compact (display_state, remote, local); + +if (error) +strbuf_addf (&display_state->buf," (%s)" , error); + +break ; } - - width = (summary_width +strlen (summary) -gettext_width (summary)); - remote =prettify_refname (remote); - local =prettify_refname (local); - -strbuf_addf (&display_state->buf," %c %-*s " , code, width, summary); -if (!display_state->compact_format) -print_remote_to_local (display_state, remote, local); -else -print_compact (display_state, remote, local); -if (error) -strbuf_addf (&display_state->buf," (%s)" , error); +default : +BUG ("unexpected display format %d" , display_state->format); + };strbuf_addch (&display_state->buf,'\n' );fputs (display_state->buf.buf, stderr);
Signature · 1 line
2.40.1
Re: [PATCH v4 5/8] fetch: introduce `display_format` enum
Quoted 8 lines · Patrick Steinhardt
We currently have two different display formats in git-fetch(1) with the "full" and "compact" formats. This is tracked with a boolean value that simply denotes whether the display format is supposed to be compacted or not. This works reasonably well while there are only two formats, but we're about to introduce another format that will make this a bit more awkward to use.
Introduce a
enum display_formatthat is more readily extensible.
Makes sense.
builtin/fetch.c enum {
Quoted 12 lines
builtin/fetch.c +6
-1+enum display_format { +DISPLAY_FORMAT_UNKNOWN =0 , +DISPLAY_FORMAT_FULL , +DISPLAY_FORMAT_COMPACT , +};struct display_state {struct strbuf buf;int refcol_width; -int compact_format; +enum display_format format;
OK. Preparatory conversion without adding anything new.
builtin/fetch.c static void display_state_init(struct display_state *display_state, struct ref * · line 822
Quoted 12 lines
builtin/fetch.c +2
-2@@ -809,31 +814,42 @@ static void display_state_init(struct display_state *display_state, struct ref *git_config_get_string_tmp ("fetch.output" , &format);if (!strcasecmp (format,"full" )) - display_state->compact_format =0 ; + display_state->format =DISPLAY_FORMAT_FULL ;else if (!strcasecmp (format,"compact" )) - display_state->compact_format =1 ; + display_state->format =DISPLAY_FORMAT_COMPACT ;else die (_ ("invalid value for '%s': '%s'" ),"fetch.output" , format);
Naturally.
Quoted 35 lines
builtin/fetch.c +17
-15- display_state->refcol_width =10 ; -for (rm = ref_map; rm; rm = rm->next) { -int width; +switch (display_state->format) { +case DISPLAY_FORMAT_FULL : +case DISPLAY_FORMAT_COMPACT : { +struct ref *rm; -if (rm->status ==REF_STATUS_REJECT_SHALLOW || - !rm->peer_ref || - !strcmp (rm->name,"HEAD" )) -continue ; + display_state->refcol_width =10 ; +for (rm = ref_map; rm; rm = rm->next) { +int width; - width =refcol_width (rm, display_state->compact_format); +if (rm->status ==REF_STATUS_REJECT_SHALLOW || + !rm->peer_ref || + !strcmp (rm->name,"HEAD" )) +continue ; -/* -* Not precise calculation for compact mode because '*' can -* appear on the left hand side of '->' and shrink the column -* back. -*/ -if (display_state->refcol_width < width) - display_state->refcol_width = width; + width =refcol_width (rm, display_state->format ==DISPLAY_FORMAT_COMPACT ); + +/* +* Not precise calculation for compact mode because '*' can +* appear on the left hand side of '->' and shrink the column +* back.
builtin/fetch.c static void display_state_init(struct display_state *display_state, struct ref * · line 853
Quoted 10 lines
builtin/fetch.c +9
-0+*/ +if (display_state->refcol_width < width) + display_state->refcol_width = width; + } + +break ; + } +default : +BUG ("unexpected display format %d" , display_state->format); }
Due to reindentation, the patch is noisier than what it does (which should be "nothing, other than allowing another value in the .format member").
It makes me wonder if it would make it easier to read to move the bulk of this code to a helper function. If we are to give a name to what is being done in the above hunk, what would it be? It computes display->refcol_width in which all records would fit, but presumably if we are to add more things to be shown per ref and align them in a simlar way, we would compute widths for these other things there as well. Perhaps compute_display_alignment() or somesuch?
Re: [PATCH v4 5/8] fetch: introduce `display_format` enum
Quoted 1 line · Junio C Hamano
Patrick Steinhardt <[email protected]> writes:
[snip]
Quoted 50 lines
Quoted 35 lines
builtin/fetch.c +17
-15- display_state->refcol_width =10 ; -for (rm = ref_map; rm; rm = rm->next) { -int width; +switch (display_state->format) { +case DISPLAY_FORMAT_FULL : +case DISPLAY_FORMAT_COMPACT : { +struct ref *rm; -if (rm->status ==REF_STATUS_REJECT_SHALLOW || - !rm->peer_ref || - !strcmp (rm->name,"HEAD" )) -continue ; + display_state->refcol_width =10 ; +for (rm = ref_map; rm; rm = rm->next) { +int width; - width =refcol_width (rm, display_state->compact_format); +if (rm->status ==REF_STATUS_REJECT_SHALLOW || + !rm->peer_ref || + !strcmp (rm->name,"HEAD" )) +continue ; -/* -* Not precise calculation for compact mode because '*' can -* appear on the left hand side of '->' and shrink the column -* back. -*/ -if (display_state->refcol_width < width) - display_state->refcol_width = width; + width =refcol_width (rm, display_state->format ==DISPLAY_FORMAT_COMPACT ); + +/* +* Not precise calculation for compact mode because '*' can +* appear on the left hand side of '->' and shrink the column +* back. builtin/fetch.c static void display_state_init(struct display_state *display_state, struct ref * · line 853
Quoted 10 lines
builtin/fetch.c +9
-0+*/ +if (display_state->refcol_width < width) + display_state->refcol_width = width; + } + +break ; + } +default : +BUG ("unexpected display format %d" , display_state->format); }Due to reindentation, the patch is noisier than what it does (which should be "nothing, other than allowing another value in the .format member").
Quoted 7 lines
It makes me wonder if it would make it easier to read to move the bulk of this code to a helper function. If we are to give a name to what is being done in the above hunk, what would it be? It computes
display->refcol_widthin which all records would fit, but presumably if we are to add more things to be shown per ref and align them in a simlar way, we would compute widths for these other things there as well. Perhapscompute_display_alignment()or somesuch?
The code already has such a function and calls it refcol_width(), but it only computes the width for a single reference. The most natural thing to do here would thus be to merge the loop over the references into that function. This would also allow us to skip some weirdness, like the fact that we skip some references inside refcol_width() while we skip other references in the ref_map loop.
This refactoring is also quite noisy, but it makes the code simpler overall and will make the patch introducing the enum less so.
Patrick
PATCH v4 6/8 fetch: move display format parsing into main function
In reply to Patrick SteinhardtParsing the display format happens inside of display_state_init(). As we only need to check for a simple config entry, this is a natural location to put this code as it means that display-state logic is neatly contained in a single location.
We're about to introduce a output format though that is intended to be parseable by machines, for example inside of a script. In that case it becomes a bit awkward of an interface if you have to call git-fetch(1) with the fetch.output config key set. We're thus going to introduce a new --output-format switch for git-fetch(1) so that the output format can be configured more directly.
This means we'll have to hook parsing of the display format into the command line options parser. Having the code to determine the actual output format scattered across two different sites is hard to reason about though.
Refactor the code such that callers are expected to pass the display format that is to be used into display_state_init(). This allows us to lift up the code into the main function, where we can then hook it into command line options parser in a follow-up commit.
--- builtin/fetch.c | 41 ++++++++++++++++++++++++----------------- 1 file changed, 24 insertions(+), 17 deletions(-)
builtin/fetch.c +24 -17
@@ -788,14 +788,13 @@ static int refcol_width(const struct ref *ref, int compact_format) }static void display_state_init (struct display_state *display_state,struct ref *ref_map, -const char *raw_url) +const char *raw_url,enum display_format format) { -const char *format ="full" ;int i;memset (display_state,0 ,sizeof (*display_state)); -strbuf_init (&display_state->buf,0 ); + display_state->format = format;if (raw_url) display_state->url =transport_anonymize_url (raw_url);
@@ -812,15 +811,6 @@ static void display_state_init(struct display_state *display_state, struct ref *if (verbosity <0 )return ; -git_config_get_string_tmp ("fetch.output" , &format); -if (!strcasecmp (format,"full" )) - display_state->format =DISPLAY_FORMAT_FULL ; -else if (!strcasecmp (format,"compact" )) - display_state->format =DISPLAY_FORMAT_COMPACT ; -else -die (_ ("invalid value for '%s': '%s'" ), -"fetch.output" , format); -switch (display_state->format) {case DISPLAY_FORMAT_FULL :case DISPLAY_FORMAT_COMPACT : {
@@ -1614,7 +1604,8 @@ static int backfill_tags(struct display_state *display_state, }static int do_fetch (struct transport *transport, -struct refspec *rs) +struct refspec *rs, +enum display_format display_format) {struct ref_transaction *transaction =NULL ;struct ref *ref_map =NULL ;
@@ -1700,7 +1691,7 @@ static int do_fetch(struct transport *transport,if (retcode) goto cleanup; -display_state_init (&display_state, ref_map, transport->url); +display_state_init (&display_state, ref_map, transport->url, display_format);if (atomic_fetch) { transaction =ref_transaction_begin (&err);
@@ -2078,7 +2069,8 @@ static inline void fetch_one_setup_partial(struct remote *remote) }static int fetch_one (struct remote *remote,int argc,const char **argv, -int prune_tags_ok,int use_stdin_refspecs) +int prune_tags_ok,int use_stdin_refspecs, +enum display_format display_format) {struct refspec rs =REFSPEC_INIT_FETCH ;int i;
@@ -2145,7 +2137,7 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,sigchain_push_common (unlock_pack_on_signal);atexit (unlock_pack_atexit);sigchain_push (SIGPIPE ,SIG_IGN ); - exit_code =do_fetch (gtransport, &rs); + exit_code =do_fetch (gtransport, &rs, display_format);sigchain_pop (SIGPIPE );refspec_clear (&rs);transport_disconnect (gtransport);
@@ -2157,6 +2149,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) {int i;const char *bundle_uri; +enum display_format display_format =DISPLAY_FORMAT_UNKNOWN ;struct string_list list =STRING_LIST_INIT_DUP ;struct remote *remote =NULL ;int result =0 ;
@@ -2183,6 +2176,19 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) argc =parse_options (argc, argv, prefix, builtin_fetch_options, builtin_fetch_usage,0 ); +if (display_format ==DISPLAY_FORMAT_UNKNOWN ) { +const char *format ="full" ; + +git_config_get_string_tmp ("fetch.output" , &format); +if (!strcasecmp (format,"full" )) + display_format =DISPLAY_FORMAT_FULL ; +else if (!strcasecmp (format,"compact" )) + display_format =DISPLAY_FORMAT_COMPACT ; +else +die (_ ("invalid value for '%s': '%s'" ), +"fetch.output" , format); + } +if (recurse_submodules_cli !=RECURSE_SUBMODULES_DEFAULT ) recurse_submodules = recurse_submodules_cli;
@@ -2311,7 +2317,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) }else if (remote) {if (filter_options.choice ||has_promisor_remote ())fetch_one_setup_partial (remote); - result =fetch_one (remote, argc, argv, prune_tags_ok, stdin_refspecs); + result =fetch_one (remote, argc, argv, prune_tags_ok, stdin_refspecs, + display_format); }else {int max_children = max_jobs;
Signature · 1 line
2.40.1
Re: [PATCH v4 6/8] fetch: move display format parsing into main function
Quoted 6 lines · Patrick Steinhardt
We're about to introduce a output format though that is intended to be parseable by machines, for example inside of a script. In that case it becomes a bit awkward of an interface if you have to call git-fetch(1) with the
fetch.outputconfig key set. We're thus going to introduce a new--output-formatswitch for git-fetch(1) so that the output format can be configured more directly.
Good. I was wondering about that code in the context of the previous patch, especially the error message had a hard-coded assumption that it comes from a configuration variable.
And the changes to display_state_init() to lift the responsibility of finding and validating .format out of the function, and the changes to intermediate functions to pass the .format through the callchain, are all expected and there was nothing questionable.
Quoted 14 lines
builtin/fetch.c +3
-0@@ -2157,6 +2149,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) {int i;const char *bundle_uri; +enum display_format display_format =DISPLAY_FORMAT_UNKNOWN ;struct string_list list =STRING_LIST_INIT_DUP ;struct remote *remote =NULL ;int result =0 ;@@ -2183,6 +2176,19 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) argc =parse_options (argc, argv, prefix, builtin_fetch_options, builtin_fetch_usage,0 ); +if (display_format ==DISPLAY_FORMAT_UNKNOWN ) { +const char *format ="full" ;
builtin/fetch.c int cmd_fetch(int argc, const char **argv, const char *prefix) · line 2190
Quoted 10 lines
builtin/fetch.c +10
-0+ +git_config_get_string_tmp ("fetch.output" , &format); +if (!strcasecmp (format,"full" )) + display_format =DISPLAY_FORMAT_FULL ; +else if (!strcasecmp (format,"compact" )) + display_format =DISPLAY_FORMAT_COMPACT ; +else +die (_ ("invalid value for '%s': '%s'" ), +"fetch.output" , format); + }
OK, but isn't the usual way to do this to have configuration parser before parse_options() and then let parse_options() override whatever display_format set by it?
That way, we do not have to worry about DISPLAY_FORMAT_UNKNOWN at all. Just initialize the variable to whatever default format at the beginning of this function, read "fetch.output" to override it if the configuration exists, and then let parse_options() to handle "--output-format" or "--porcelain" or whatever to further override it.
Thanks.
Re: [PATCH v4 6/8] fetch: move display format parsing into main function
In reply to Patrick SteinhardtQuoted 3 lines · Patrick Steinhardt
with the
fetch.outputconfig key set. We're thus going to introduce a new--output-formatswitch for git-fetch(1) so that the output format can be configured more directly.
This is stale as of v3, since isn't named --output-format any more. Let me see if there are other instances of this.
(I should have caught this earlier; I only saw it because Junio quoted it in a reply of his.)
Re: [PATCH v4 6/8] fetch: move display format parsing into main function
Quoted 11 lines · Glen Choo
Quoted 3 lines · Patrick Steinhardt
with the
fetch.outputconfig key set. We're thus going to introduce a new--output-formatswitch for git-fetch(1) so that the output format can be configured more directly.This is stale as of v3, since isn't named --output-format any more. Let me see if there are other instances of this.
(I should have caught this earlier; I only saw it because Junio quoted it in a reply of his.)
Good catch, fixed now.
Patrick
PATCH v4 7/8 fetch: move option related variables into main function
In reply to Patrick SteinhardtThe options of git-fetch(1) which we pass to parse_options() are declared globally in builtin/fetch.c. This means we're forced to use global variables for all the options, which is more likely to cause confusion than explicitly passing state around.
Refactor the code to move the options into cmd_fetch(). Move variables that were previously forced to be declared globally and which are only used by cmd_fetch() into function-local scope.
--- builtin/fetch.c | 197 ++++++++++++++++++++++++------------------------ 1 file changed, 100 insertions(+), 97 deletions(-)
builtin/fetch.c +100 -97
@@ -75,13 +75,12 @@ static int fetch_prune_tags_config = -1; /* unspecified */static int prune_tags =-1 ;/* unspecified */ #define PRUNE_TAGS_BY_DEFAULT 0/* do we prune tags by default? */ -static int all, append, dry_run, force, keep, multiple, update_head_ok; +static int append, dry_run, force, keep, update_head_ok;static int write_fetch_head =1 ;static int verbosity, deepen_relative, set_upstream, refetch;static int progress =-1 ; -static int enable_auto_gc =1 ; -static int tags =TAGS_DEFAULT , unshallow, update_shallow, deepen; -static int max_jobs =-1 , submodule_fetch_jobs_config =-1 ; +static int tags =TAGS_DEFAULT , update_shallow, deepen; +static int submodule_fetch_jobs_config =-1 ;static int fetch_parallel_config =1 ;static int atomic_fetch;static enum transport_family family;
@@ -92,17 +91,11 @@ static struct string_list deepen_not = STRING_LIST_INIT_NODUP;static struct strbuf default_rla =STRBUF_INIT ;static struct transport *gtransport;static struct transport *gsecondary; -static const char *submodule_prefix ="" ;static int recurse_submodules =RECURSE_SUBMODULES_DEFAULT ; -static int recurse_submodules_cli =RECURSE_SUBMODULES_DEFAULT ; -static int recurse_submodules_default =RECURSE_SUBMODULES_ON_DEMAND ;static struct refspec refmap =REFSPEC_INIT_FETCH ;static struct list_objects_filter_options filter_options =LIST_OBJECTS_FILTER_INIT ;static struct string_list server_options =STRING_LIST_INIT_DUP ;static struct string_list negotiation_tip =STRING_LIST_INIT_NODUP ; -static int fetch_write_commit_graph =-1 ; -static int stdin_refspecs =0 ; -static int negotiate_only;static int git_fetch_config (const char *k,const char *v,void *cb) {
@@ -160,92 +153,6 @@ static int parse_refmap_arg(const struct option *opt, const char *arg, int unsetreturn 0 ; } -static struct option builtin_fetch_options[] = { -OPT__VERBOSITY (&verbosity), -OPT_BOOL (0 ,"all" , &all, -N_ ("fetch from all remotes" )), -OPT_BOOL (0 ,"set-upstream" , &set_upstream, -N_ ("set upstream for git pull/fetch" )), -OPT_BOOL ('a' ,"append" , &append, -N_ ("append to .git/FETCH_HEAD instead of overwriting" )), -OPT_BOOL (0 ,"atomic" , &atomic_fetch, -N_ ("use atomic transaction to update references" )), -OPT_STRING (0 ,"upload-pack" , &upload_pack,N_ ("path" ), -N_ ("path to upload pack on remote end" )), -OPT__FORCE (&force,N_ ("force overwrite of local reference" ),0 ), -OPT_BOOL ('m' ,"multiple" , &multiple, -N_ ("fetch from multiple remotes" )), -OPT_SET_INT ('t' ,"tags" , &tags, -N_ ("fetch all tags and associated objects" ),TAGS_SET ), -OPT_SET_INT ('n' ,NULL , &tags, -N_ ("do not fetch all tags (--no-tags)" ),TAGS_UNSET ), -OPT_INTEGER ('j' ,"jobs" , &max_jobs, -N_ ("number of submodules fetched in parallel" )), -OPT_BOOL (0 ,"prefetch" , &prefetch, -N_ ("modify the refspec to place all refs within refs/prefetch/" )), -OPT_BOOL ('p' ,"prune" , &prune, -N_ ("prune remote-tracking branches no longer on remote" )), -OPT_BOOL ('P' ,"prune-tags" , &prune_tags, -N_ ("prune local tags no longer on remote and clobber changed tags" )), -OPT_CALLBACK_F (0 ,"recurse-submodules" , &recurse_submodules_cli,N_ ("on-demand" ), -N_ ("control recursive fetching of submodules" ), -PARSE_OPT_OPTARG , option_fetch_parse_recurse_submodules), -OPT_BOOL (0 ,"dry-run" , &dry_run, -N_ ("dry run" )), -OPT_BOOL (0 ,"write-fetch-head" , &write_fetch_head, -N_ ("write fetched references to the FETCH_HEAD file" )), -OPT_BOOL ('k' ,"keep" , &keep,N_ ("keep downloaded pack" )), -OPT_BOOL ('u' ,"update-head-ok" , &update_head_ok, -N_ ("allow updating of HEAD ref" )), -OPT_BOOL (0 ,"progress" , &progress,N_ ("force progress reporting" )), -OPT_STRING (0 ,"depth" , &depth,N_ ("depth" ), -N_ ("deepen history of shallow clone" )), -OPT_STRING (0 ,"shallow-since" , &deepen_since,N_ ("time" ), -N_ ("deepen history of shallow repository based on time" )), -OPT_STRING_LIST (0 ,"shallow-exclude" , &deepen_not,N_ ("revision" ), -N_ ("deepen history of shallow clone, excluding rev" )), -OPT_INTEGER (0 ,"deepen" , &deepen_relative, -N_ ("deepen history of shallow clone" )), -OPT_SET_INT_F (0 ,"unshallow" , &unshallow, -N_ ("convert to a complete repository" ), -1 ,PARSE_OPT_NONEG ), -OPT_SET_INT_F (0 ,"refetch" , &refetch, -N_ ("re-fetch without negotiating common commits" ), -1 ,PARSE_OPT_NONEG ), - {OPTION_STRING ,0 ,"submodule-prefix" , &submodule_prefix,N_ ("dir" ), -N_ ("prepend this to submodule path output" ),PARSE_OPT_HIDDEN }, -OPT_CALLBACK_F (0 ,"recurse-submodules-default" , - &recurse_submodules_default,N_ ("on-demand" ), -N_ ("default for recursive fetching of submodules " -"(lower priority than config files)" ), -PARSE_OPT_HIDDEN , option_fetch_parse_recurse_submodules), -OPT_BOOL (0 ,"update-shallow" , &update_shallow, -N_ ("accept refs that update .git/shallow" )), -OPT_CALLBACK_F (0 ,"refmap" ,NULL ,N_ ("refmap" ), -N_ ("specify fetch refmap" ),PARSE_OPT_NONEG , parse_refmap_arg), -OPT_STRING_LIST ('o' ,"server-option" , &server_options,N_ ("server-specific" ),N_ ("option to transmit" )), -OPT_SET_INT ('4' ,"ipv4" , &family,N_ ("use IPv4 addresses only" ), -TRANSPORT_FAMILY_IPV4 ), -OPT_SET_INT ('6' ,"ipv6" , &family,N_ ("use IPv6 addresses only" ), -TRANSPORT_FAMILY_IPV6 ), -OPT_STRING_LIST (0 ,"negotiation-tip" , &negotiation_tip,N_ ("revision" ), -N_ ("report that we have only objects reachable from this object" )), -OPT_BOOL (0 ,"negotiate-only" , &negotiate_only, -N_ ("do not fetch a packfile; instead, print ancestors of negotiation tips" )), -OPT_PARSE_LIST_OBJECTS_FILTER (&filter_options), -OPT_BOOL (0 ,"auto-maintenance" , &enable_auto_gc, -N_ ("run 'maintenance --auto' after fetching" )), -OPT_BOOL (0 ,"auto-gc" , &enable_auto_gc, -N_ ("run 'maintenance --auto' after fetching" )), -OPT_BOOL (0 ,"show-forced-updates" , &fetch_show_forced_updates, -N_ ("check for forced-updates on all updated branches" )), -OPT_BOOL (0 ,"write-commit-graph" , &fetch_write_commit_graph, -N_ ("write the commit-graph after fetching" )), -OPT_BOOL (0 ,"stdin" , &stdin_refspecs, -N_ ("accept refspecs from stdin" )), -OPT_END () -}; -static void unlock_pack (unsigned int flags) {if (gtransport)
@@ -2147,13 +2054,109 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,int cmd_fetch (int argc,const char **argv,const char *prefix) { -int i;const char *bundle_uri; +const char *submodule_prefix ="" ;enum display_format display_format =DISPLAY_FORMAT_UNKNOWN ;struct string_list list =STRING_LIST_INIT_DUP ;struct remote *remote =NULL ; +int all =0 , multiple =0 ;int result =0 ;int prune_tags_ok =1 ; +int enable_auto_gc =1 ; +int unshallow =0 ; +int max_jobs =-1 ; +int recurse_submodules_cli =RECURSE_SUBMODULES_DEFAULT ; +int recurse_submodules_default =RECURSE_SUBMODULES_ON_DEMAND ; +int fetch_write_commit_graph =-1 ; +int stdin_refspecs =0 ; +int negotiate_only =0 ; +int i; + +struct option builtin_fetch_options[] = { +OPT__VERBOSITY (&verbosity), +OPT_BOOL (0 ,"all" , &all, +N_ ("fetch from all remotes" )), +OPT_BOOL (0 ,"set-upstream" , &set_upstream, +N_ ("set upstream for git pull/fetch" )), +OPT_BOOL ('a' ,"append" , &append, +N_ ("append to .git/FETCH_HEAD instead of overwriting" )), +OPT_BOOL (0 ,"atomic" , &atomic_fetch, +N_ ("use atomic transaction to update references" )), +OPT_STRING (0 ,"upload-pack" , &upload_pack,N_ ("path" ), +N_ ("path to upload pack on remote end" )), +OPT__FORCE (&force,N_ ("force overwrite of local reference" ),0 ), +OPT_BOOL ('m' ,"multiple" , &multiple, +N_ ("fetch from multiple remotes" )), +OPT_SET_INT ('t' ,"tags" , &tags, +N_ ("fetch all tags and associated objects" ),TAGS_SET ), +OPT_SET_INT ('n' ,NULL , &tags, +N_ ("do not fetch all tags (--no-tags)" ),TAGS_UNSET ), +OPT_INTEGER ('j' ,"jobs" , &max_jobs, +N_ ("number of submodules fetched in parallel" )), +OPT_BOOL (0 ,"prefetch" , &prefetch, +N_ ("modify the refspec to place all refs within refs/prefetch/" )), +OPT_BOOL ('p' ,"prune" , &prune, +N_ ("prune remote-tracking branches no longer on remote" )), +OPT_BOOL ('P' ,"prune-tags" , &prune_tags, +N_ ("prune local tags no longer on remote and clobber changed tags" )), +OPT_CALLBACK_F (0 ,"recurse-submodules" , &recurse_submodules_cli,N_ ("on-demand" ), +N_ ("control recursive fetching of submodules" ), +PARSE_OPT_OPTARG , option_fetch_parse_recurse_submodules), +OPT_BOOL (0 ,"dry-run" , &dry_run, +N_ ("dry run" )), +OPT_BOOL (0 ,"write-fetch-head" , &write_fetch_head, +N_ ("write fetched references to the FETCH_HEAD file" )), +OPT_BOOL ('k' ,"keep" , &keep,N_ ("keep downloaded pack" )), +OPT_BOOL ('u' ,"update-head-ok" , &update_head_ok, +N_ ("allow updating of HEAD ref" )), +OPT_BOOL (0 ,"progress" , &progress,N_ ("force progress reporting" )), +OPT_STRING (0 ,"depth" , &depth,N_ ("depth" ), +N_ ("deepen history of shallow clone" )), +OPT_STRING (0 ,"shallow-since" , &deepen_since,N_ ("time" ), +N_ ("deepen history of shallow repository based on time" )), +OPT_STRING_LIST (0 ,"shallow-exclude" , &deepen_not,N_ ("revision" ), +N_ ("deepen history of shallow clone, excluding rev" )), +OPT_INTEGER (0 ,"deepen" , &deepen_relative, +N_ ("deepen history of shallow clone" )), +OPT_SET_INT_F (0 ,"unshallow" , &unshallow, +N_ ("convert to a complete repository" ), +1 ,PARSE_OPT_NONEG ), +OPT_SET_INT_F (0 ,"refetch" , &refetch, +N_ ("re-fetch without negotiating common commits" ), +1 ,PARSE_OPT_NONEG ), + {OPTION_STRING ,0 ,"submodule-prefix" , &submodule_prefix,N_ ("dir" ), +N_ ("prepend this to submodule path output" ),PARSE_OPT_HIDDEN }, +OPT_CALLBACK_F (0 ,"recurse-submodules-default" , + &recurse_submodules_default,N_ ("on-demand" ), +N_ ("default for recursive fetching of submodules " +"(lower priority than config files)" ), +PARSE_OPT_HIDDEN , option_fetch_parse_recurse_submodules), +OPT_BOOL (0 ,"update-shallow" , &update_shallow, +N_ ("accept refs that update .git/shallow" )), +OPT_CALLBACK_F (0 ,"refmap" ,NULL ,N_ ("refmap" ), +N_ ("specify fetch refmap" ),PARSE_OPT_NONEG , parse_refmap_arg), +OPT_STRING_LIST ('o' ,"server-option" , &server_options,N_ ("server-specific" ),N_ ("option to transmit" )), +OPT_SET_INT ('4' ,"ipv4" , &family,N_ ("use IPv4 addresses only" ), +TRANSPORT_FAMILY_IPV4 ), +OPT_SET_INT ('6' ,"ipv6" , &family,N_ ("use IPv6 addresses only" ), +TRANSPORT_FAMILY_IPV6 ), +OPT_STRING_LIST (0 ,"negotiation-tip" , &negotiation_tip,N_ ("revision" ), +N_ ("report that we have only objects reachable from this object" )), +OPT_BOOL (0 ,"negotiate-only" , &negotiate_only, +N_ ("do not fetch a packfile; instead, print ancestors of negotiation tips" )), +OPT_PARSE_LIST_OBJECTS_FILTER (&filter_options), +OPT_BOOL (0 ,"auto-maintenance" , &enable_auto_gc, +N_ ("run 'maintenance --auto' after fetching" )), +OPT_BOOL (0 ,"auto-gc" , &enable_auto_gc, +N_ ("run 'maintenance --auto' after fetching" )), +OPT_BOOL (0 ,"show-forced-updates" , &fetch_show_forced_updates, +N_ ("check for forced-updates on all updated branches" )), +OPT_BOOL (0 ,"write-commit-graph" , &fetch_write_commit_graph, +N_ ("write the commit-graph after fetching" )), +OPT_BOOL (0 ,"stdin" , &stdin_refspecs, +N_ ("accept refspecs from stdin" )), +OPT_END () + };packet_trace_identity ("fetch" );
Signature · 1 line
2.40.1
PATCH v4 8/8 fetch: introduce machine-parseable "porcelain" output format
In reply to Patrick SteinhardtThe output of git-fetch(1) is obviously designed for consumption by users, only: we neatly columnize data, we abbreviate reference names, we print neat arrows and we don't provide information about actual object IDs that have changed. This makes the output format basically unusable in the context of scripted invocations of git-fetch(1) that want to learn about the exact changes that the command performs.
Introduce a new machine-parseable "porcelain" output format that is supposed to fix this shortcoming. This output format is intended to provide information about every reference that is about to be updated, the old object ID that the reference has been pointing to and the new object ID it will be updated to. Furthermore, the output format provides the same flags as the human-readable format to indicate basic conditions for each reference update like whether it was a fast-forward update, a branch deletion, a rejected update or others.
The output format is quite simple:
``` <flag> <old-object-id> <new-object-id> <local-reference>\n ```
We assume two conditions which are generally true:
- The old and new object IDs have fixed known widths and cannot contain spaces.
- References cannot contain newlines.
With these assumptions, the output format becomes unambiguously parseable. Furthermore, given that this output is designed to be consumed by scripts, the machine-readable data is printed to stdout instead of stderr like the human-readable output is. This is mostly done so that other data printed to stderr, like error messages or progress meters, don't interfere with the parseable data.
A notable ommission here is that the output format does not include the remote from which a reference was fetched, which might be important information especially in the context of multi-remote fetches. But as such a format would require us to print the remote for every single reference update due to parallelizable fetches it feels wasteful for the most likely usecase, which is when fetching from a single remote.
In a similar spirit, a second restriction is that this cannot be used with --recurse-submodules. This is because any reference updates would be ambiguous without also printing the repository in which the update happens.
Considering that both multi-remote and submodule fetches are user-facing features, using them in conjunction with --porcelain that is intended for scripting purposes is likely not going to be useful in the majority of cases. With that in mind these restrictions feel acceptable. If usecases for either of these come up in the future though it is easy enough to add a new "porcelain-v2" format that adds this information.
--- Documentation/fetch-options.txt | 7 ++ Documentation/git-fetch.txt | 9 ++ builtin/fetch.c | 91 +++++++++++++++++---- t/t5574-fetch-output.sh | 140 ++++++++++++++++++++++++++++++++ 4 files changed, 229 insertions(+), 18 deletions(-)
Documentation/fetch-options.txt +7 -0
@@ -78,6 +78,13 @@ linkgit:git-config[1]. --dry-run:: Show what would be done, without making any changes. +--porcelain:: + Print the output to standard output in an easy-to-parse format for + scripts. See section OUTPUT in linkgit:git-fetch[1] for details. ++ +This is incompatible with `--recurse-submodules=[yes|on-demand]` and takes +precedence over the `fetch.output` config option. + ifndef::git-pull[] --[no-]write-fetch-head:: Write the list of remote refs fetched in the `FETCH_HEAD`
Documentation/git-fetch.txt +9 -0
@@ -204,6 +204,15 @@ representing the status of a single ref. Each line is of the form: <flag> <summary> <from> -> <to> [<reason>] ------------------------------- +When using `--porcelain`, the output format is intended to be +machine-parseable. In contrast to the human-readable output formats it +thus prints to standard output instead of standard error. Each line is +of the form: + +------------------------------- +<flag> <old-object-id> <new-object-id> <local-reference> +------------------------------- + The status of up-to-date refs is shown only if the --verbose option is used.
builtin/fetch.c +73 -18
252 lines not on this page · raw message
t/t5574-fetch-output.sh +140 -0
152 lines not on this page · raw message
Signature · 1 line
2.40.1
Re: [PATCH v4 8/8] fetch: introduce machine-parseable "porcelain" output format
Quoted 6 lines · Patrick Steinhardt
Considering that both multi-remote and submodule fetches are user-facing features, using them in conjunction with
--porcelainthat is intended for scripting purposes is likely not going to be useful in the majority of cases. With that in mind these restrictions feel acceptable. If usecases for either of these come up in the future though it is easy enough to add a new "porcelain-v2" format that adds this information.
Two steps ago, the proposed log message still mentioned "--output-format", which may want to be proofread again and revised.
Quoted 11 lines
builtin/fetch.c +2
-1@@ -1786,7 +1810,8 @@ static int add_remote_or_group(const char *name, struct string_list *list)return 1 ; } -static void add_options_to_argv (struct strvec *argv) +static void add_options_to_argv (struct strvec *argv, +enum display_format format) {if (dry_run)strvec_push (argv,"--dry-run" );
builtin/fetch.c static void add_options_to_argv(struct strvec *argv) · line 1852
Quoted 6 lines
builtin/fetch.c +2
-0@@ -1822,6 +1847,8 @@ static void add_options_to_argv(struct strvec *argv)strvec_push (argv,"--ipv6" );if (!write_fetch_head)strvec_push (argv,"--no-write-fetch-head" ); +if (format ==DISPLAY_FORMAT_PORCELAIN ) +strvec_pushf (argv,"--porcelain" ); }
Hmph.
[PATCH 9/8] may want to also introduce and pass down the "--output-format=full/compact" option, but that is clearly outside of the scope of this step.
builtin/fetch.c static int fetch_one(struct remote *remote, int argc, const char **argv, · line 2089
Quoted 6 lines
builtin/fetch.c +6
-0+static int opt_parse_porcelain (const struct option *opt,const char *arg,int unset) +{ +enum display_format *format = opt->value; + *format =DISPLAY_FORMAT_PORCELAIN ; +return 0 ; +}
Lack of "if (unset) ..." worries me. Shouldn't the code allow
git fetch --porcelain --no-porcelain
to defeat an earlier one and revert back to the default?
Other than that the changes in this step were mostly expected and I didn't spot anything glaringly wrong, but I didn't spend as much time as I did for the other steps.
Thanks.
Re: [PATCH v4 8/8] fetch: introduce machine-parseable "porcelain" output format
Quoted 11 lines · Junio C Hamano
Quoted 6 lines · Patrick Steinhardt
Considering that both multi-remote and submodule fetches are user-facing features, using them in conjunction with
--porcelainthat is intended for scripting purposes is likely not going to be useful in the majority of cases. With that in mind these restrictions feel acceptable. If usecases for either of these come up in the future though it is easy enough to add a new "porcelain-v2" format that adds this information.Two steps ago, the proposed log message still mentioned "--output-format", which may want to be proofread again and revised.
It's a relict from previous iterations. I've fixed the preceding patch to talk about --porcelain instead.
Quoted 20 lines
Quoted 11 lines
builtin/fetch.c +2
-1@@ -1786,7 +1810,8 @@ static int add_remote_or_group(const char *name, struct string_list *list)return 1 ; } -static void add_options_to_argv (struct strvec *argv) +static void add_options_to_argv (struct strvec *argv, +enum display_format format) {if (dry_run)strvec_push (argv,"--dry-run" );builtin/fetch.c static void add_options_to_argv(struct strvec *argv) · line 1852
Quoted 6 lines
builtin/fetch.c +2
-0@@ -1822,6 +1847,8 @@ static void add_options_to_argv(struct strvec *argv)strvec_push (argv,"--ipv6" );if (!write_fetch_head)strvec_push (argv,"--no-write-fetch-head" ); +if (format ==DISPLAY_FORMAT_PORCELAIN ) +strvec_pushf (argv,"--porcelain" ); }Hmph.
Quoted 3 lines
[PATCH 9/8] may want to also introduce and pass down the "--output-format=full/compact" option, but that is clearly outside of the scope of this step.
We don't have --output-format now though, but only the --porcelain switch. The only way to configure "full"/"compact" output formats is via the "fetch.output" config variable right now.
Quoted 9 lines
builtin/fetch.c static int fetch_one(struct remote *remote, int argc, const char **argv, · line 2089
Quoted 6 lines
builtin/fetch.c +6
-0+static int opt_parse_porcelain (const struct option *opt,const char *arg,int unset) +{ +enum display_format *format = opt->value; + *format =DISPLAY_FORMAT_PORCELAIN ; +return 0 ; +}Lack of "if (unset) ..." worries me. Shouldn't the code allow
Quoted 3 lines
git fetch --porcelain --no-porcelainto defeat an earlier one and revert back to the default?
We pass PARSE_OPT_NOARG|PARSE_OPT_NONEG, so this isn't really much of a concern. But yeah, it wouldn't allow --no-porcelain, and supporting it is trivial enough to do. I'll change this to a OPT_BOOL().
Patrick
v5 cover fetch: introduce machine-parseable output
In reply to Patrick SteinhardtHi,
this is the fourth version of my patch series to introduce a machine-parseable output format for git-fetch(1). It applies on top of e9dffbc7f1 (Merge branch 'ps/fetch-ref-update-reporting', 2023-04-06).
Changes compared to v4:
- Patch 1/9: Simplified the test as proposed by Junio and Glen.
- Patch 3/9: Added a test to verify that git fetch -c fetch.output without a value set fails as expected. Also dropped the tests that checked whether stdout was empty.
- Patch 4/9: Reformulated the commit message to treat the missing left-hand side of displayed references as an inconsistency instead of a bug. I've also added a testcase to verify that direct OID fetches continue to work as expected.
- Patch 5/9: New patch that makes calculation of the table width for displayed reference updates self-contained in refcol_width(). This is a preparatory refactoring that should make patch 6/9 easier to review.
- Patch 7/9: Refactored the code to parse the "fetch.output" config variable inside of git_fetch_config() before we parse command line options. Also fixed that the commit message was still referring to --output-format=porcelain instead of the new --porcelain switch.
- Patch 9/9: The --porcelain option is now a simple OPT_BOOL() that can be negated. Added a test that --no-porcelain works as expected.
Thanks for your feedback, Junio and Glen!
Patrick
Patrick Steinhardt (9):
fetch: fix --no-recurse-submodules with multi-remote fetches
fetch: split out tests for output format
fetch: add a test to exercise invalid output formats
fetch: print left-hand side when fetching HEAD:foo
fetch: refactor calculation of the display table width
fetch: introduce display_format enum
fetch: lift up parsing of "fetch.output" config variable
fetch: move option related variables into main function
fetch: introduce machine-parseable "porcelain" output format
Documentation/fetch-options.txt | 7 + Documentation/git-fetch.txt | 9 + builtin/fetch.c | 490 +++++++++++++++++++------------- t/t5510-fetch.sh | 53 ---- t/t5526-fetch-submodules.sh | 13 + t/t5574-fetch-output.sh | 293 +++++++++++++++++++ 6 files changed, 611 insertions(+), 254 deletions(-) create mode 100755 t/t5574-fetch-output.sh
Range-diff against v4:
1: d82b42ed34 ! 1: 02ee4fab7e fetch: fix --no-recurse-submodules with multi-remote fetches
@@ t/t5526-fetch-submodules.sh: test_expect_success 'fetch --all with --recurse-sub
+ cd src_clone &&
+ git remote add secondary ../src &&
+ git config submodule.recurse true &&
-+ git config fetch.parallel 0 &&
-+ git fetch --all --no-recurse-submodules 2>../actual
++ git fetch --all --no-recurse-submodules 2>../fetch-log
+ ) &&
-+
-+ cat >expect <<-EOF &&
-+ From ../src
-+ * [new branch] master -> secondary/master
-+ EOF
-+ test_cmp expect actual
++ ! grep "Fetching submodule" fetch-log
+'
+
test_done
2: 33112dc51a = 2: e459d8a1ec fetch: split out tests for output format
3: 006ea93afb ! 3: d503c425fe fetch: add a test to exercise invalid output formats
@@ t/t5574-fetch-output.sh: export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
+ test_when_finished "rm -rf clone" &&
+ git clone . clone &&
+
-+ test_must_fail git -C clone -c fetch.output= fetch origin >actual.out 2>actual.err &&
++ test_must_fail git -C clone -c fetch.output fetch origin 2>actual.err &&
++ cat >expect <<-EOF &&
++ error: missing value for ${SQ}fetch.output${SQ}
++ fatal: unable to parse ${SQ}fetch.output${SQ} from command-line config
++ EOF
++ test_cmp expect actual.err &&
++
++ test_must_fail git -C clone -c fetch.output= fetch origin 2>actual.err &&
+ cat >expect <<-EOF &&
+ fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}${SQ}
+ EOF
-+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err &&
+
-+ test_must_fail git -C clone -c fetch.output=garbage fetch origin >actual.out 2>actual.err &&
++ test_must_fail git -C clone -c fetch.output=garbage fetch origin 2>actual.err &&
+ cat >expect <<-EOF &&
+ fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}garbage${SQ}
+ EOF
-+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err
+'
+ 4: e599ea6d33 ! 4: 2cc7318697 fetch: fix missing from-reference when fetching HEAD:foo
@@ Metadata
Author: Patrick Steinhardt <[email protected]>
## Commit message ##
- fetch: fix missing from-reference when fetching HEAD:foo
+ fetch: print left-hand side when fetching HEAD:foo
store_updated_refs() parses the remote reference for two purposes:
@@ Commit message
name and can thus be used for both cases. But if the remote reference is HEAD, the parsed remote reference becomes empty. This is intended when we write the FETCH_HEAD, where we skip writing the note in that case.
- But it is not intended when displaying the updated references and would
- cause us to miss the left-hand side of the displayed reference update:
+ But when displaying the updated references this leads to inconsistent
+ output where the left-hand side of reference updates is missing in some
+ cases:
```
- $ git fetch origin HEAD:foo
- From https://github.com/git/git
- * [new ref] -> foo
- ```
-
- The HEAD string is clearly missing from the left-hand side of the arrow,
- which is further stressed by the point that the following commands show
- the left-hand side as expected:
-
- ```
- $ git fetch origin HEAD
+ $ git fetch origin HEAD HEAD:explicit-head :implicit-head main
From https://github.com/git/git
* branch HEAD -> FETCH_HEAD
-
- $ git fetch origin master
- From https://github.com/git/git
- * branch master -> FETCH_HEAD
- * branch master -> origin/master
+ * [new ref] -> explicit-head
+ * [new ref] -> implicit-head
+ * branch main -> FETCH_HEAD
```
+ This behaviour has existed ever since the table-based output has been
+ introduced for git-fetch(1) via 165f390250 (git-fetch: more terse fetch
+ output, 2007-11-03) and was never explicitly documented either in the
+ commit message or in any of our tests. So while it may not be a bug per
+ se, it feels like a weird inconsistency and not like it was a concious
+ design decision.
+
The logic of how we compute the remote reference name that we ultimately pass to display_ref_update() is not easy to follow. There are three different cases here:
- When the remote reference name is "HEAD" we set the remote reference name to the empty string. This is the case that causes
- the bug to occur, where we would indeed want to print "HEAD"
- instead of the empty string. This is what prettify_refname()
- would return.
+ the left-hand side to go missing, where we would indeed want to
+ print "HEAD" instead of the empty string. This is what
+ prettify_refname() would return.
- When the remote reference name has a well-known prefix then we strip this prefix. This matches what prettify_refname() does.
@@ Commit message
matches what prettify_refname() does.
As the return value of prettify_refname() would do the correct thing
- for us in all three cases, we can fix the bug by passing through the
- full remote reference name to display_ref_update(), which learns to
- call prettify_refname(). At the same time, this also simplifies the
- code a bit.
+ for us in all three cases, we can thus fix the inconsistency by passing
+ through the full remote reference name to display_ref_update(), which
+ learns to call prettify_refname(). At the same time, this also
+ simplifies the code a bit.
Note that this patch also changes formatting of the block that computes
- the "kind" and "what" variables. This is done on purpose so that it is
- part of the diff, hopefully making the change easier to comprehend.
+ the "kind" (which is the category like "branch" or "tag") and "what"
+ (which is the prettified reference name like "master" or "v1.0")
+ variables. This is done on purpose so that it is part of the diff,
+ hopefully making the change easier to comprehend.
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
+ test_must_be_empty actual.out &&
+ test_cmp expect actual.err
+'
++
++test_expect_success 'fetch output with object ID' '
++ test_when_finished "rm -rf object-id" &&
++ git clone . object-id &&
++ commit=$(git rev-parse HEAD) &&
++
++ git -C object-id fetch --dry-run origin $commit:object-id >actual.out 2>actual.err &&
++ cat >expect <<-EOF &&
++ From $(test-tool path-utils real_path .)/.
++ * [new ref] $commit -> object-id
++ EOF
++ test_must_be_empty actual.out &&
++ test_cmp expect actual.err &&
++
++ git -C object-id fetch origin $commit:object-id >actual.out 2>actual.err &&
++ test_must_be_empty actual.out &&
++ test_cmp expect actual.err
++'
+
test_expect_success '--no-show-forced-updates' '
mkdir forced-updates &&
-: ---------- > 5: bb1a591c2f fetch: refactor calculation of the display table width
5: 80ac00b0c4 ! 6: 3cac552f5f fetch: introduce `display_format` enum
@@ builtin/fetch.c: enum {
char *url;
int url_len, shown_url;
-@@ builtin/fetch.c: static int refcol_width(const struct ref *ref, int compact_format)
- static void display_state_init(struct display_state *display_state, struct ref *ref_map,
- const char *raw_url)
- {
-- struct ref *rm;
- const char *format = "full";
- int i;
-
@@ builtin/fetch.c: static void display_state_init(struct display_state *display_state, struct ref *
git_config_get_string_tmp("fetch.output", &format);
@@ builtin/fetch.c: static void display_state_init(struct display_state *display_st
die(_("invalid value for '%s': '%s'"),
"fetch.output", format);
-- display_state->refcol_width = 10;
-- for (rm = ref_map; rm; rm = rm->next) {
-- int width;
+- display_state->refcol_width = refcol_width(ref_map, display_state->compact_format);
+ switch (display_state->format) {
+ case DISPLAY_FORMAT_FULL:
-+ case DISPLAY_FORMAT_COMPACT: {
-+ struct ref *rm;
-
-- if (rm->status == REF_STATUS_REJECT_SHALLOW ||
-- !rm->peer_ref ||
-- !strcmp(rm->name, "HEAD"))
-- continue;
-+ display_state->refcol_width = 10;
-+ for (rm = ref_map; rm; rm = rm->next) {
-+ int width;
-
-- width = refcol_width(rm, display_state->compact_format);
-+ if (rm->status == REF_STATUS_REJECT_SHALLOW ||
-+ !rm->peer_ref ||
-+ !strcmp(rm->name, "HEAD"))
-+ continue;
-
-- /*
-- * Not precise calculation for compact mode because '*' can
-- * appear on the left hand side of '->' and shrink the column
-- * back.
-- */
-- if (display_state->refcol_width < width)
-- display_state->refcol_width = width;
-+ width = refcol_width(rm, display_state->format == DISPLAY_FORMAT_COMPACT);
-+
-+ /*
-+ * Not precise calculation for compact mode because '*' can
-+ * appear on the left hand side of '->' and shrink the column
-+ * back.
-+ */
-+ if (display_state->refcol_width < width)
-+ display_state->refcol_width = width;
-+ }
-+
++ case DISPLAY_FORMAT_COMPACT:
++ display_state->refcol_width = refcol_width(ref_map,
++ display_state->format == DISPLAY_FORMAT_COMPACT);
+ break;
-+ }
+ default:
+ BUG("unexpected display format %d", display_state->format);
- }
++ }
}
+ static void display_state_release(struct display_state *display_state)
@@ builtin/fetch.c: static void display_ref_update(struct display_state *display_state, char code,
const char *remote, const char *local,
int summary_width) 6: 826b8b7bc0 ! 7: 0c3dbcd09f fetch: move display format parsing into main function
@@ Metadata
Author: Patrick Steinhardt <[email protected]>
## Commit message ##
- fetch: move display format parsing into main function
+ fetch: lift up parsing of "fetch.output" config variable
Parsing the display format happens inside of display_state_init(). As we only need to check for a simple config entry, this is a natural location to put this code as it means that display-state logic is neatly contained in a single location.
- We're about to introduce a output format though that is intended to be
- parseable by machines, for example inside of a script. In that case it
- becomes a bit awkward of an interface if you have to call git-fetch(1)
- with the fetch.output config key set. We're thus going to introduce a
- new --output-format switch for git-fetch(1) so that the output format
- can be configured more directly.
-
- This means we'll have to hook parsing of the display format into the
- command line options parser. Having the code to determine the actual
- output format scattered across two different sites is hard to reason
- about though.
+ We're about to introduce a new "porcelain" output format though that is
+ intended to be parseable by machines, for example inside of a script.
+ This format can be enabled by passing the --porcelain switch to
+ git-fetch(1). As a consequence, we'll have to add a second callsite that
+ influences the output format, which will become awkward to handle.
Refactor the code such that callers are expected to pass the display format that is to be used into display_state_init(). This allows us to
@@ Commit message
## builtin/fetch.c ##
-@@ builtin/fetch.c: static int refcol_width(const struct ref *ref, int compact_format) +@@ builtin/fetch.c: static int fetch_write_commit_graph = -1;
+ static int stdin_refspecs = 0;
+ static int negotiate_only;
+
++struct fetch_config {
++ enum display_format display_format;
++};
++
+ static int git_fetch_config(const char *k, const char *v, void *cb)
+ {
++ struct fetch_config *fetch_config = cb;
++
+ if (!strcmp(k, "fetch.prune")) {
+ fetch_prune_config = git_config_bool(k, v);
+ return 0;
+@@ builtin/fetch.c: static int git_fetch_config(const char *k, const char *v, void *cb)
+ return 0;
+ }
+
++ if (!strcmp(k, "fetch.output")) {
++ if (!v)
++ return config_error_nonbool(k);
++ else if (!strcasecmp(v, "full"))
++ fetch_config->display_format = DISPLAY_FORMAT_FULL;
++ else if (!strcasecmp(v, "compact"))
++ fetch_config->display_format = DISPLAY_FORMAT_COMPACT;
++ else
++ die(_("invalid value for '%s': '%s'"),
++ "fetch.output", v);
++ }
++
+ return git_default_config(k, v, cb);
+ }
+
+@@ builtin/fetch.c: static int refcol_width(const struct ref *ref_map, int compact_format)
}
static void display_state_init(struct display_state *display_state, struct ref *ref_map,
@@ builtin/fetch.c: static void display_state_init(struct display_state *display_st
-
switch (display_state->format) {
case DISPLAY_FORMAT_FULL:
- case DISPLAY_FORMAT_COMPACT: {
+ case DISPLAY_FORMAT_COMPACT:
@@ builtin/fetch.c: static int backfill_tags(struct display_state *display_state,
}
@@ builtin/fetch.c: static int fetch_one(struct remote *remote, int argc, const cha
sigchain_pop(SIGPIPE);
refspec_clear(&rs);
transport_disconnect(gtransport);
-@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix) +@@ builtin/fetch.c: static int fetch_one(struct remote *remote, int argc, const char **argv, +
+ int cmd_fetch(int argc, const char **argv, const char *prefix)
{
++ struct fetch_config config = {
++ .display_format = DISPLAY_FORMAT_FULL,
++ };
int i;
const char *bundle_uri;
-+ enum display_format display_format = DISPLAY_FORMAT_UNKNOWN;
struct string_list list = STRING_LIST_INIT_DUP;
- struct remote *remote = NULL;
- int result = 0;
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
- argc = parse_options(argc, argv, prefix,
- builtin_fetch_options, builtin_fetch_usage, 0);
-
-+ if (display_format == DISPLAY_FORMAT_UNKNOWN) {
-+ const char *format = "full";
-+
-+ git_config_get_string_tmp("fetch.output", &format);
-+ if (!strcasecmp(format, "full"))
-+ display_format = DISPLAY_FORMAT_FULL;
-+ else if (!strcasecmp(format, "compact"))
-+ display_format = DISPLAY_FORMAT_COMPACT;
-+ else
-+ die(_("invalid value for '%s': '%s'"),
-+ "fetch.output", format);
-+ }
-+
- if (recurse_submodules_cli != RECURSE_SUBMODULES_DEFAULT)
- recurse_submodules = recurse_submodules_cli;
+ free(anon);
+ }
+- git_config(git_fetch_config, NULL);
++ git_config(git_fetch_config, &config);
+ if (the_repository->gitdir) {
+ prepare_repo_settings(the_repository);
+ the_repository->settings.command_requires_full_index = 0;
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
} else if (remote) {
if (filter_options.choice || has_promisor_remote())
fetch_one_setup_partial(remote);
- result = fetch_one(remote, argc, argv, prune_tags_ok, stdin_refspecs);
+ result = fetch_one(remote, argc, argv, prune_tags_ok, stdin_refspecs,
-+ display_format);
++ config.display_format);
} else {
int max_children = max_jobs;
7: 20f2e061d6 ! 8: 8e33a08c35 fetch: move option related variables into main function
@@ builtin/fetch.c: static struct string_list deepen_not = STRING_LIST_INIT_NODUP;
-static int stdin_refspecs = 0;
-static int negotiate_only;
- static int git_fetch_config(const char *k, const char *v, void *cb)
- {
+ struct fetch_config {
+ enum display_format display_format;
@@ builtin/fetch.c: static int parse_refmap_arg(const struct option *opt, const char *arg, int unset
return 0;
}
@@ builtin/fetch.c: static int parse_refmap_arg(const struct option *opt, const cha
static void unlock_pack(unsigned int flags)
{
if (gtransport)
-@@ builtin/fetch.c: static int fetch_one(struct remote *remote, int argc, const char **argv, -
- int cmd_fetch(int argc, const char **argv, const char *prefix)
- {
+@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ struct fetch_config config = {
+ .display_format = DISPLAY_FORMAT_FULL,
+ };
- int i;
- const char *bundle_uri;
+ const char *submodule_prefix = "";
- enum display_format display_format = DISPLAY_FORMAT_UNKNOWN;
+ const char *bundle_uri;
struct string_list list = STRING_LIST_INIT_DUP;
struct remote *remote = NULL;
+ int all = 0, multiple = 0;
8: 24ae381950 ! 9: d49152c220 fetch: introduce machine-parseable "porcelain" output format
@@ builtin/fetch.c: enum display_format {
struct display_state {
@@ builtin/fetch.c: static void display_state_init(struct display_state *display_state, struct ref *
-
+ display_state->refcol_width = refcol_width(ref_map,
+ display_state->format == DISPLAY_FORMAT_COMPACT);
break;
- }
+ case DISPLAY_FORMAT_PORCELAIN:
+ /* We don't need to precompute anything here. */
+ break;
@@ builtin/fetch.c: static int fetch_multiple(struct string_list *list, int max_chi
printf(_("Fetching %s\n"), name);
cmd.git_cmd = 1;
if (run_command(&cmd)) {
-@@ builtin/fetch.c: static int fetch_one(struct remote *remote, int argc, const char **argv,
- return exit_code;
- }
+@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ int fetch_write_commit_graph = -1;
+ int stdin_refspecs = 0;
+ int negotiate_only = 0;
++ int porcelain = 0;
+ int i;
-+static int opt_parse_porcelain(const struct option *opt, const char *arg, int unset)
-+{
-+ enum display_format *format = opt->value;
-+ *format = DISPLAY_FORMAT_PORCELAIN;
-+ return 0;
-+}
-+
- int cmd_fetch(int argc, const char **argv, const char *prefix)
- {
- const char *bundle_uri;
+ struct option builtin_fetch_options[] = {
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
PARSE_OPT_OPTARG, option_fetch_parse_recurse_submodules),
OPT_BOOL(0, "dry-run", &dry_run,
N_("dry run")),
-+ OPT_CALLBACK_F(0, "porcelain", &display_format, NULL, N_("machine-readable output"),
-+ PARSE_OPT_NOARG|PARSE_OPT_NONEG, opt_parse_porcelain),
++ OPT_BOOL(0, "porcelain", &porcelain, N_("machine-readable output")),
OPT_BOOL(0, "write-fetch-head", &write_fetch_head,
N_("write fetched references to the FETCH_HEAD file")),
OPT_BOOL('k', "keep", &keep, N_("keep downloaded pack")),
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
fetch_config_from_gitmodules(sfjc, rs);
}
-+ if (display_format == DISPLAY_FORMAT_PORCELAIN) {
++
++ if (porcelain) {
+ switch (recurse_submodules_cli) {
+ case RECURSE_SUBMODULES_OFF:
+ case RECURSE_SUBMODULES_DEFAULT:
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
+ die(_("options '%s' and '%s' cannot be used together"),
+ "--porcelain", "--recurse-submodules");
+ }
++
++ config.display_format = DISPLAY_FORMAT_PORCELAIN;
+ }
+
if (negotiate_only && !negotiation_tip.nr)
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
/* TODO should this also die if we have a previous partial-clone? */
- result = fetch_multiple(&list, max_children);
-+ result = fetch_multiple(&list, max_children, display_format);
++ result = fetch_multiple(&list, max_children, config.display_format);
}
-
+-
+ /*
+ * This is only needed after fetch_one(), which does not fetch
+ * submodules by itself.
@@ builtin/fetch.c: int cmd_fetch(int argc, const char **argv, const char *prefix)
if (max_children < 0)
max_children = fetch_parallel_config;
- add_options_to_argv(&options);
-+ add_options_to_argv(&options, display_format);
++ add_options_to_argv(&options, config.display_format);
result = fetch_submodules(the_repository,
&options,
submodule_prefix,
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch compact output' '
+ test_must_be_empty stderr &&
+ test_cmp expect stdout
+'
++
++test_expect_success 'fetch --no-porcelain overrides previous --porcelain' '
++ test_when_finished "rm -rf no-porcelain" &&
++
++ git switch --create no-porcelain &&
++ git clone . no-porcelain &&
++ test_commit --no-tag no-porcelain &&
++ old_commit=$(git rev-parse --short HEAD~) &&
++ new_commit=$(git rev-parse --short HEAD) &&
++
++ cat >expect <<-EOF &&
++ From $(test-tool path-utils real_path .)/.
++ $old_commit..$new_commit no-porcelain -> origin/no-porcelain
++ EOF
++
++ git -C no-porcelain fetch --porcelain --no-porcelain >stdout 2>stderr &&
++ test_cmp expect stderr &&
++ test_must_be_empty stdout
++'
+
test_expect_success 'fetch output with HEAD' '
test_when_finished "rm -rf head" &&
@@ t/t5574-fetch-output.sh: test_expect_success 'fetch output with HEAD' '
+ test_cmp expect actual
+'
+
- test_expect_success '--no-show-forced-updates' '
- mkdir forced-updates &&
- (
+ test_expect_success 'fetch output with object ID' '
+ test_when_finished "rm -rf object-id" &&
+ git clone . object-id &&Signature · 1 line
2.40.1
PATCH v5 1/9 fetch: fix `--no-recurse-submodules` with multi-remote fetches
When running git fetch --no-recurse-submodules, the exectation is that we don't fetch any submodules. And while this works for fetches of a single remote, it doesn't when fetching multiple remotes at once. The result is that we do recurse into submodules even though the user has explicitly asked us not to.
This is because while we pass on --recurse-submodules={yes,on-demand} if specified by the user, we don't pass on --no-recurse-submodules to the subprocess spawned to perform the submodule fetch.
Fix this by also forwarding this flag as expected.
--- builtin/fetch.c | 2 ++ t/t5526-fetch-submodules.sh | 13 +++++++++++++ 2 files changed, 15 insertions(+)
builtin/fetch.c +2 -0
@@ -1876,6 +1876,8 @@ static void add_options_to_argv(struct strvec *argv)strvec_push (argv,"--keep" );if (recurse_submodules ==RECURSE_SUBMODULES_ON )strvec_push (argv,"--recurse-submodules" ); +else if (recurse_submodules ==RECURSE_SUBMODULES_OFF ) +strvec_push (argv,"--no-recurse-submodules" );else if (recurse_submodules ==RECURSE_SUBMODULES_ON_DEMAND )strvec_push (argv,"--recurse-submodules=on-demand" );if (tags ==TAGS_SET )
t/t5526-fetch-submodules.sh +13 -0
@@ -1180,4 +1180,17 @@ test_expect_success 'fetch --all with --recurse-submodules with multiple' ' test_line_count = 2 fetch-subs ' +test_expect_success "fetch --all with --no-recurse-submodules only fetches superproject" ' + test_when_finished "rm -rf src_clone" && + + git clone --recurse-submodules src src_clone && + ( + cd src_clone && + git remote add secondary ../src && + git config submodule.recurse true && + git fetch --all --no-recurse-submodules 2>../fetch-log + ) && + ! grep "Fetching submodule" fetch-log +' + test_done
Signature · 1 line
2.40.1
PATCH v5 2/9 fetch: split out tests for output format
In reply to Patrick SteinhardtWe're about to introduce a new porcelain mode for the output of git-fetch(1). As part of that we'll be introducing a set of new tests that only relate to the output of this command.
Split out tests that exercise the output format of git-fetch(1) so that it becomes easier to verify this functionality as a standalone unit. As the tests assume that the default branch is called "main" we set up the corresponding GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME environment variable accordingly.
--- t/t5510-fetch.sh | 53 ---------------------------------- t/t5574-fetch-output.sh | 63 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 63 insertions(+), 53 deletions(-) create mode 100755 t/t5574-fetch-output.sh
t/t5510-fetch.sh +0 -53
@@ -1118,59 +1118,6 @@ test_expect_success 'fetching with auto-gc does not lock up' ' ) ' -test_expect_success 'fetch aligned output' ' - git clone . full-output && - test_commit looooooooooooong-tag && - ( - cd full-output && - git -c fetch.output=full fetch origin >actual 2>&1 && - grep -e "->" actual | cut -c 22- >../actual - ) && - cat >expect <<-\EOF && - main -> origin/main - looooooooooooong-tag -> looooooooooooong-tag - EOF - test_cmp expect actual -' - -test_expect_success 'fetch compact output' ' - git clone . compact && - test_commit extraaa && - ( - cd compact && - git -c fetch.output=compact fetch origin >actual 2>&1 && - grep -e "->" actual | cut -c 22- >../actual - ) && - cat >expect <<-\EOF && - main -> origin/* - extraaa -> * - EOF - test_cmp expect actual -' - -test_expect_success '--no-show-forced-updates' ' - mkdir forced-updates && - ( - cd forced-updates && - git init && - test_commit 1 && - test_commit 2 - ) && - git clone forced-updates forced-update-clone && - git clone forced-updates no-forced-update-clone && - git -C forced-updates reset --hard HEAD~1 && - ( - cd forced-update-clone && - git fetch --show-forced-updates origin 2>output && - test_i18ngrep "(forced update)" output - ) && - ( - cd no-forced-update-clone && - git fetch --no-show-forced-updates origin 2>output && - test_i18ngrep ! "(forced update)" output - ) -' - for section in fetch transfer do test_expect_success "$section.hideRefs affects connectivity check" '
t/t5574-fetch-output.sh new file +63 -0
63 lines not on this page · raw message
Signature · 1 line
2.40.1
PATCH v5 3/9 fetch: add a test to exercise invalid output formats
In reply to Patrick SteinhardtAdd a testcase that exercises the logic when an invalid output format is passed via the fetch.output configuration.
--- t/t5574-fetch-output.sh | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)
t/t5574-fetch-output.sh +24 -0
@@ -7,6 +7,30 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME . ./test-lib.sh +test_expect_success 'fetch with invalid output format configuration' ' + test_when_finished "rm -rf clone" && + git clone . clone && + + test_must_fail git -C clone -c fetch.output fetch origin 2>actual.err && + cat >expect <<-EOF && + error: missing value for ${SQ}fetch.output${SQ} + fatal: unable to parse ${SQ}fetch.output${SQ} from command-line config + EOF + test_cmp expect actual.err && + + test_must_fail git -C clone -c fetch.output= fetch origin 2>actual.err && + cat >expect <<-EOF && + fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}${SQ} + EOF + test_cmp expect actual.err && + + test_must_fail git -C clone -c fetch.output=garbage fetch origin 2>actual.err && + cat >expect <<-EOF && + fatal: invalid value for ${SQ}fetch.output${SQ}: ${SQ}garbage${SQ} + EOF + test_cmp expect actual.err +' + test_expect_success 'fetch aligned output' ' git clone . full-output && test_commit looooooooooooong-tag &&
Signature · 1 line
2.40.1
PATCH v5 4/9 fetch: print left-hand side when fetching HEAD:foo
In reply to Patrick Steinhardtstore_updated_refs() parses the remote reference for two purposes:
- It gets used as a note when writing FETCH_HEAD.
- It is passed through to display_ref_update() to display updated references in the following format:
```
* branch master -> master
```
In most cases, the parsed remote reference is the prettified reference name and can thus be used for both cases. But if the remote reference is HEAD, the parsed remote reference becomes empty. This is intended when we write the FETCH_HEAD, where we skip writing the note in that case. But when displaying the updated references this leads to inconsistent output where the left-hand side of reference updates is missing in some cases:
``` $ git fetch origin HEAD HEAD:explicit-head :implicit-head main From https://github.com/git/git * branch HEAD -> FETCH_HEAD * [new ref] -> explicit-head * [new ref] -> implicit-head * branch main -> FETCH_HEAD ```
This behaviour has existed ever since the table-based output has been introduced for git-fetch(1) via 165f390250 (git-fetch: more terse fetch output, 2007-11-03) and was never explicitly documented either in the commit message or in any of our tests. So while it may not be a bug per se, it feels like a weird inconsistency and not like it was a concious design decision.
The logic of how we compute the remote reference name that we ultimately pass to display_ref_update() is not easy to follow. There are three different cases here:
- When the remote reference name is "HEAD" we set the remote reference name to the empty string. This is the case that causes the left-hand side to go missing, where we would indeed want to print "HEAD" instead of the empty string. This is what prettify_refname() would return.
- When the remote reference name has a well-known prefix then we strip this prefix. This matches what prettify_refname() does.
- Otherwise, we keep the fully qualified reference name. This also matches what prettify_refname() does.
As the return value of prettify_refname() would do the correct thing for us in all three cases, we can thus fix the inconsistency by passing through the full remote reference name to display_ref_update(), which learns to call prettify_refname(). At the same time, this also simplifies the code a bit.
Note that this patch also changes formatting of the block that computes the "kind" (which is the category like "branch" or "tag") and "what" (which is the prettified reference name like "master" or "v1.0") variables. This is done on purpose so that it is part of the diff, hopefully making the change easier to comprehend.
--- builtin/fetch.c | 37 ++++++++++++++++---------------- t/t5574-fetch-output.sh | 47 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 66 insertions(+), 18 deletions(-)
builtin/fetch.c +19 -18
@@ -918,12 +918,14 @@ static void display_ref_update(struct display_state *display_state, char code, } width = (summary_width +strlen (summary) -gettext_width (summary)); + remote =prettify_refname (remote); + local =prettify_refname (local);strbuf_addf (&display_state->buf," %c %-*s " , code, width, summary);if (!display_state->compact_format) -print_remote_to_local (display_state, remote,prettify_refname (local)); +print_remote_to_local (display_state, remote, local);else -print_compact (display_state, remote,prettify_refname (local)); +print_compact (display_state, remote, local);if (error)strbuf_addf (&display_state->buf," (%s)" , error);strbuf_addch (&display_state->buf,'\n' );
@@ -934,7 +936,7 @@ static void display_ref_update(struct display_state *display_state, char code,static int update_local_ref (struct ref *ref,struct ref_transaction *transaction,struct display_state *display_state, -const char *remote,const struct ref *remote_ref, +const struct ref *remote_ref,int summary_width) {struct commit *current =NULL , *updated;
@@ -946,7 +948,7 @@ static int update_local_ref(struct ref *ref,if (oideq (&ref->old_oid, &ref->new_oid)) {if (verbosity >0 )display_ref_update (display_state,'=' ,_ ("[up to date]" ),NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 0 ; }
@@ -959,7 +961,7 @@ static int update_local_ref(struct ref *ref,*/ display_ref_update (display_state,'!' ,_ ("[rejected]" ),_ ("can't fetch into checked-out branch" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 1 ; }
@@ -970,12 +972,12 @@ static int update_local_ref(struct ref *ref, r =s_update_ref ("updating tag" , ref, transaction,0 );display_ref_update (display_state, r ?'!' :'t' ,_ ("[tag update]" ), r ?_ ("unable to update local ref" ) :NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return r; }else {display_ref_update (display_state,'!' ,_ ("[rejected]" ),_ ("would clobber existing tag" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 1 ; } }
@@ -1008,7 +1010,7 @@ static int update_local_ref(struct ref *ref, r =s_update_ref (msg, ref, transaction,0 );display_ref_update (display_state, r ?'!' :'*' , what, r ?_ ("unable to update local ref" ) :NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return r; }
@@ -1030,7 +1032,7 @@ static int update_local_ref(struct ref *ref, r =s_update_ref ("fast-forward" , ref, transaction,1 );display_ref_update (display_state, r ?'!' :' ' , quickref.buf, r ?_ ("unable to update local ref" ) :NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);strbuf_release (&quickref);return r; }else if (force || ref->force) {
@@ -1042,12 +1044,12 @@ static int update_local_ref(struct ref *ref, r =s_update_ref ("forced-update" , ref, transaction,1 );display_ref_update (display_state, r ?'!' :'+' , quickref.buf, r ?_ ("unable to update local ref" ) :_ ("forced update" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);strbuf_release (&quickref);return r; }else {display_ref_update (display_state,'!' ,_ ("[rejected]" ),_ ("non-fast-forward" ), - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 1 ; } }
@@ -1252,14 +1254,13 @@ static int store_updated_refs(struct display_state *display_state,if (!strcmp (rm->name,"HEAD" )) { kind ="" ; what ="" ; - } -else if (skip_prefix (rm->name,"refs/heads/" , &what)) + }else if (skip_prefix (rm->name,"refs/heads/" , &what)) { kind ="branch" ; -else if (skip_prefix (rm->name,"refs/tags/" , &what)) + }else if (skip_prefix (rm->name,"refs/tags/" , &what)) { kind ="tag" ; -else if (skip_prefix (rm->name,"refs/remotes/" , &what)) + }else if (skip_prefix (rm->name,"refs/remotes/" , &what)) { kind ="remote-tracking branch" ; -else { + }else { kind ="" ; what = rm->name; }
@@ -1277,7 +1278,7 @@ static int store_updated_refs(struct display_state *display_state, display_state->url_len);if (ref) { - rc |=update_local_ref (ref, transaction, display_state, what, + rc |=update_local_ref (ref, transaction, display_state, rm, summary_width);free (ref); }else if (write_fetch_head || dry_run) {
@@ -1288,7 +1289,7 @@ static int store_updated_refs(struct display_state *display_state,*/ display_ref_update (display_state,'*' , *kind ? kind :"branch" ,NULL , - *what ? what :"HEAD" , + rm->name,"FETCH_HEAD" , summary_width); } }
t/t5574-fetch-output.sh +47 -0
53 lines not on this page · raw message
Signature · 1 line
2.40.1
Re: [PATCH v5 4/9] fetch: print left-hand side when fetching HEAD:foo
Quoted 6 lines · Patrick Steinhardt
This behaviour has existed ever since the table-based output has been introduced for git-fetch(1) via
165f390250(git-fetch: more terse fetch output, 2007-11-03) and was never explicitly documented either in the commit message or in any of our tests. So while it may not be a bug per se, it feels like a weird inconsistency and not like it was a concious design decision.
The change seems well-justified. I agree that it does feel strange and unintended. I was really surprised to learn that it was not a bug.
Re: [PATCH v5 4/9] fetch: print left-hand side when fetching HEAD:foo
In reply to Patrick Steinhardtbuiltin/fetch.c static int update_local_ref(struct ref *ref, · line 953
Quoted 8 lines · Patrick Steinhardt
builtin/fetch.c +1
-1@@ -946,7 +948,7 @@ static int update_local_ref(struct ref *ref,if (oideq (&ref->old_oid, &ref->new_oid)) {if (verbosity >0 )display_ref_update (display_state,'=' ,_ ("[up to date]" ),NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 0 ; }
Here (and in other hunks) we now dereference remote_ref unconditionally. But in existing parts of the code, we guard against remote_ref being NULL. E.g., later on:
if (!current || !updated) {
const char *msg;
const char *what;
int r;
/*
* Nicely describe the new ref we're fetching.
* Base this on the remote's ref name, as it's
* more likely to follow a standard layout.
*/
const char *name = remote_ref ? remote_ref->name : "";
[...]I'm not sure if the old code was being overly defensive, or if the new code is ripe for a segfault. But it's probably worth looking into (it was noticed by Coverity).
Looking at the caller, it is always store_update_refs() which passes its own "rm", a pointer iterating over ref_map. And it should always be non-NULL, since that's the loop condition.
So I think your code is fine, but you might want to double-check my logic. (And it may be worth cleaning up the existing redundant check to prevent confusion).
-Peff
Re: [PATCH v5 4/9] fetch: print left-hand side when fetching HEAD:foo
Quoted 31 lines · Jeff King
builtin/fetch.c static int update_local_ref(struct ref *ref, · line 953
Quoted 8 lines · Patrick Steinhardt
builtin/fetch.c +1
-1@@ -946,7 +948,7 @@ static int update_local_ref(struct ref *ref,if (oideq (&ref->old_oid, &ref->new_oid)) {if (verbosity >0 )display_ref_update (display_state,'=' ,_ ("[up to date]" ),NULL , - remote, ref->name, summary_width); + remote_ref->name, ref->name, summary_width);return 0 ; }Here (and in other hunks) we now dereference
remote_refunconditionally. But in existing parts of the code, we guard againstremote_refbeing NULL. E.g., later on:if (!current || !updated) { const char *msg; const char *what; int r; /* * Nicely describe the new ref we're fetching. * Base this on the remote's ref name, as it's * more likely to follow a standard layout. */ const char *name = remote_ref ? remote_ref->name : ""; [...]I'm not sure if the old code was being overly defensive, or if the new code is ripe for a segfault. But it's probably worth looking into (it was noticed by Coverity).
Quoted 7 lines
Looking at the caller, it is always
store_update_refs()which passes its own "rm", a pointer iterating overref_map. And it should always be non-NULL, since that's the loop condition.So I think your code is fine, but you might want to double-check my logic. (And it may be worth cleaning up the existing redundant check to prevent confusion).
I really think that the code is overly defensive. As you mention, there is a single caller of update_local_ref(), only, and that caller already dereferences the remote reference unconditionally anyway. So if there was any way for rm to become NULL we'd already have a segfault earlier than in update_local_ref().
I'll send a follow-up patch series after the dust has settled that removes the check.
Thanks!
Patrick
PATCH v5 5/9 fetch: refactor calculation of the display table width
In reply to Patrick SteinhardtWhen displaying reference updates, we try to print the references in a neat table. As the table's width is determined its contents we thus need to precalculate the overall width before we can start printing updated references.
The calculation is driven by display_state_init(), which invokes refcol_width() for every reference that is to be printed. This split is somewhat confusing. For one, we filter references that shall be attributed to the overall width in both places. And second, we needlessly recalculate the maximum line length based on the terminal columns and display format for every reference.
Refactor the code so that the complete width calculations are neatly contained in refcol_width().
--- builtin/fetch.c | 81 ++++++++++++++++++++++--------------------------- 1 file changed, 37 insertions(+), 44 deletions(-)
builtin/fetch.c +37 -44
@@ -751,40 +751,51 @@ static int s_update_ref(const char *action,return ret; } -static int refcol_width (const struct ref *ref,int compact_format) +static int refcol_width (const struct ref *ref_map,int compact_format) { -int max, rlen, llen, len; +const struct ref *ref; +int max, width =10 ; -/* uptodate lines are only shown on high verbosity level */ -if (verbosity <=0 &&oideq (&ref->peer_ref->old_oid, &ref->old_oid)) -return 0 ; - - max =term_columns (); - rlen =utf8_strwidth (prettify_refname (ref->name)); - - llen =utf8_strwidth (prettify_refname (ref->peer_ref->name)); - -/* -* rough estimation to see if the output line is too long and -* should not be counted (we can't do precise calculation -* anyway because we don't know if the error explanation part -* will be printed in update_local_ref) -*/ -if (compact_format) { - llen =0 ; + max =term_columns (); +if (compact_format) max = max *2 /3 ; - } - len =21 /* flag and summary */ + rlen +4 /* -> */ + llen; -if (len >= max) -return 0 ; -return rlen; +for (ref = ref_map; ref; ref = ref->next) { +int rlen, llen =0 , len; + +if (ref->status ==REF_STATUS_REJECT_SHALLOW || + !ref->peer_ref || + !strcmp (ref->name,"HEAD" )) +continue ; + +/* uptodate lines are only shown on high verbosity level */ +if (verbosity <=0 &&oideq (&ref->peer_ref->old_oid, &ref->old_oid)) +continue ; + + rlen =utf8_strwidth (prettify_refname (ref->name)); +if (!compact_format) + llen =utf8_strwidth (prettify_refname (ref->peer_ref->name)); + +/* +* rough estimation to see if the output line is too long and +* should not be counted (we can't do precise calculation +* anyway because we don't know if the error explanation part +* will be printed in update_local_ref) +*/ + len =21 /* flag and summary */ + rlen +4 /* -> */ + llen; +if (len >= max) +continue ; + +if (width < rlen) + width = rlen; + } + +return width; }static void display_state_init (struct display_state *display_state,struct ref *ref_map,const char *raw_url) { -struct ref *rm;const char *format ="full" ;int i;
@@ -816,25 +827,7 @@ static void display_state_init(struct display_state *display_state, struct ref *die (_ ("invalid value for '%s': '%s'" ),"fetch.output" , format); - display_state->refcol_width =10 ; -for (rm = ref_map; rm; rm = rm->next) { -int width; - -if (rm->status ==REF_STATUS_REJECT_SHALLOW || - !rm->peer_ref || - !strcmp (rm->name,"HEAD" )) -continue ; - - width =refcol_width (rm, display_state->compact_format); - -/* -* Not precise calculation for compact mode because '*' can -* appear on the left hand side of '->' and shrink the column -* back. -*/ -if (display_state->refcol_width < width) - display_state->refcol_width = width; - } + display_state->refcol_width =refcol_width (ref_map, display_state->compact_format); }static void display_state_release (struct display_state *display_state)
Signature · 1 line
2.40.1
Re: [PATCH v5 5/9] fetch: refactor calculation of the display table width
Quoted 12 lines · Patrick Steinhardt
When displaying reference updates, we try to print the references in a neat table. As the table's width is determined its contents we thus need to precalculate the overall width before we can start printing updated references.
The calculation is driven by
display_state_init(), which invokesrefcol_width()for every reference that is to be printed. This split is somewhat confusing. For one, we filter references that shall be attributed to the overall width in both places. And second, we needlessly recalculate the maximum line length based on the terminal columns and display format for every reference.
Quoted 2 lines
Refactor the code so that the complete width calculations are neatly contained in
refcol_width().
Through no fault of yours, I have to admit that I found this refactor quite hard to read. I ended up redoing the refactor and ended up with a result very similar to yours. That was probably overkill since we have pretty extensive tests in this area, but I'm quite happy with the change since it's far more readable.
Quoted 5 lines
builtin/fetch.c +0
-5-/* -* rough estimation to see if the output line is too long and -* should not be counted (we can't do precise calculation -* anyway because we don't know if the error explanation part -* will be printed in update_local_ref)
builtin/fetch.c static int s_update_ref(const char *action, · line 779
Quoted 10 lines
builtin/fetch.c +2
-7-*/ -if (compact_format) { - llen =0 ; + max =term_columns (); +if (compact_format) max = max *2 /3 ; - } - len =21 /* flag and summary */ + rlen +4 /* -> */ + llen; -if (len >= max) -return 0 ;
The one thing that changed for the better (vs keeping the lines in the same order as before) is that this comment that used to be anchored on "if (compact_format) {"...
builtin/fetch.c static int s_update_ref(const char *action, · line 787
Quoted 9 lines
builtin/fetch.c +9
-0+/* +* rough estimation to see if the output line is too long and +* should not be counted (we can't do precise calculation +* anyway because we don't know if the error explanation part +* will be printed in update_local_ref) +*/ + len =21 /* flag and summary */ + rlen +4 /* -> */ + llen; +if (len >= max) +continue ;
is now more accurately anchored to the check that throws away the line.
Looks good.
PATCH v5 6/9 fetch: introduce `display_format` enum
In reply to Patrick SteinhardtWe currently have two different display formats in git-fetch(1) with the "full" and "compact" formats. This is tracked with a boolean value that simply denotes whether the display format is supposed to be compacted or not. This works reasonably well while there are only two formats, but we're about to introduce another format that will make this a bit more awkward to use.
Introduce a enum display_format that is more readily extensible.
--- builtin/fetch.c | 69 +++++++++++++++++++++++++++++++++---------------- 1 file changed, 47 insertions(+), 22 deletions(-)
builtin/fetch.c +47 -22
@@ -48,11 +48,17 @@ enum {TAGS_SET =2 }; +enum display_format { +DISPLAY_FORMAT_UNKNOWN =0 , +DISPLAY_FORMAT_FULL , +DISPLAY_FORMAT_COMPACT , +}; +struct display_state {struct strbuf buf;int refcol_width; -int compact_format; +enum display_format format;char *url;int url_len, shown_url;
@@ -820,14 +826,22 @@ static void display_state_init(struct display_state *display_state, struct ref *git_config_get_string_tmp ("fetch.output" , &format);if (!strcasecmp (format,"full" )) - display_state->compact_format =0 ; + display_state->format =DISPLAY_FORMAT_FULL ;else if (!strcasecmp (format,"compact" )) - display_state->compact_format =1 ; + display_state->format =DISPLAY_FORMAT_COMPACT ;else die (_ ("invalid value for '%s': '%s'" ),"fetch.output" , format); - display_state->refcol_width =refcol_width (ref_map, display_state->compact_format); +switch (display_state->format) { +case DISPLAY_FORMAT_FULL : +case DISPLAY_FORMAT_COMPACT : + display_state->refcol_width =refcol_width (ref_map, + display_state->format ==DISPLAY_FORMAT_COMPACT ); +break ; +default : +BUG ("unexpected display format %d" , display_state->format); + } }static void display_state_release (struct display_state *display_state)
@@ -897,30 +911,41 @@ static void display_ref_update(struct display_state *display_state, char code, const char *remote, const char *local, int summary_width) { - int width; - if (verbosity < 0) return;strbuf_reset (&display_state->buf); -if (!display_state->shown_url) { -strbuf_addf (&display_state->buf,_ ("From %.*s\n" ), - display_state->url_len, display_state->url); - display_state->shown_url =1 ; +switch (display_state->format) { +case DISPLAY_FORMAT_FULL : +case DISPLAY_FORMAT_COMPACT : { +int width; + +if (!display_state->shown_url) { +strbuf_addf (&display_state->buf,_ ("From %.*s\n" ), + display_state->url_len, display_state->url); + display_state->shown_url =1 ; + } + + width = (summary_width +strlen (summary) -gettext_width (summary)); + remote =prettify_refname (remote); + local =prettify_refname (local); + +strbuf_addf (&display_state->buf," %c %-*s " , code, width, summary); + +if (display_state->format !=DISPLAY_FORMAT_COMPACT ) +print_remote_to_local (display_state, remote, local); +else +print_compact (display_state, remote, local); + +if (error) +strbuf_addf (&display_state->buf," (%s)" , error); + +break ; } - - width = (summary_width +strlen (summary) -gettext_width (summary)); - remote =prettify_refname (remote); - local =prettify_refname (local); - -strbuf_addf (&display_state->buf," %c %-*s " , code, width, summary); -if (!display_state->compact_format) -print_remote_to_local (display_state, remote, local); -else -print_compact (display_state, remote, local); -if (error) -strbuf_addf (&display_state->buf," (%s)" , error); +default : +BUG ("unexpected display format %d" , display_state->format); + };strbuf_addch (&display_state->buf,'\n' );fputs (display_state->buf.buf, stderr);
Signature · 1 line
2.40.1
PATCH v5 7/9 fetch: lift up parsing of "fetch.output" config variable
In reply to Patrick SteinhardtParsing the display format happens inside of display_state_init(). As we only need to check for a simple config entry, this is a natural location to put this code as it means that display-state logic is neatly contained in a single location.
We're about to introduce a new "porcelain" output format though that is intended to be parseable by machines, for example inside of a script. This format can be enabled by passing the --porcelain switch to git-fetch(1). As a consequence, we'll have to add a second callsite that influences the output format, which will become awkward to handle.
Refactor the code such that callers are expected to pass the display format that is to be used into display_state_init(). This allows us to lift up the code into the main function, where we can then hook it into command line options parser in a follow-up commit.
--- builtin/fetch.c | 50 +++++++++++++++++++++++++++++++------------------ 1 file changed, 32 insertions(+), 18 deletions(-)
builtin/fetch.c +32 -18
@@ -104,8 +104,14 @@ static int fetch_write_commit_graph = -1;static int stdin_refspecs =0 ;static int negotiate_only; +struct fetch_config { +enum display_format display_format; +}; +static int git_fetch_config (const char *k,const char *v,void *cb) { +struct fetch_config *fetch_config = cb; +if (!strcmp (k,"fetch.prune" )) { fetch_prune_config =git_config_bool (k, v);return 0 ;
@@ -144,6 +150,18 @@ static int git_fetch_config(const char *k, const char *v, void *cb)return 0 ; } +if (!strcmp (k,"fetch.output" )) { +if (!v) +return config_error_nonbool (k); +else if (!strcasecmp (v,"full" )) + fetch_config->display_format =DISPLAY_FORMAT_FULL ; +else if (!strcasecmp (v,"compact" )) + fetch_config->display_format =DISPLAY_FORMAT_COMPACT ; +else +die (_ ("invalid value for '%s': '%s'" ), +"fetch.output" , v); + } +return git_default_config (k, v, cb); }
@@ -800,14 +818,13 @@ static int refcol_width(const struct ref *ref_map, int compact_format) }static void display_state_init (struct display_state *display_state,struct ref *ref_map, -const char *raw_url) +const char *raw_url,enum display_format format) { -const char *format ="full" ;int i;memset (display_state,0 ,sizeof (*display_state)); -strbuf_init (&display_state->buf,0 ); + display_state->format = format;if (raw_url) display_state->url =transport_anonymize_url (raw_url);
@@ -824,15 +841,6 @@ static void display_state_init(struct display_state *display_state, struct ref *if (verbosity <0 )return ; -git_config_get_string_tmp ("fetch.output" , &format); -if (!strcasecmp (format,"full" )) - display_state->format =DISPLAY_FORMAT_FULL ; -else if (!strcasecmp (format,"compact" )) - display_state->format =DISPLAY_FORMAT_COMPACT ; -else -die (_ ("invalid value for '%s': '%s'" ), -"fetch.output" , format); -switch (display_state->format) {case DISPLAY_FORMAT_FULL :case DISPLAY_FORMAT_COMPACT :
@@ -1605,7 +1613,8 @@ static int backfill_tags(struct display_state *display_state, }static int do_fetch (struct transport *transport, -struct refspec *rs) +struct refspec *rs, +enum display_format display_format) {struct ref_transaction *transaction =NULL ;struct ref *ref_map =NULL ;
@@ -1691,7 +1700,7 @@ static int do_fetch(struct transport *transport,if (retcode) goto cleanup; -display_state_init (&display_state, ref_map, transport->url); +display_state_init (&display_state, ref_map, transport->url, display_format);if (atomic_fetch) { transaction =ref_transaction_begin (&err);
@@ -2069,7 +2078,8 @@ static inline void fetch_one_setup_partial(struct remote *remote) }static int fetch_one (struct remote *remote,int argc,const char **argv, -int prune_tags_ok,int use_stdin_refspecs) +int prune_tags_ok,int use_stdin_refspecs, +enum display_format display_format) {struct refspec rs =REFSPEC_INIT_FETCH ;int i;
@@ -2136,7 +2146,7 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,sigchain_push_common (unlock_pack_on_signal);atexit (unlock_pack_atexit);sigchain_push (SIGPIPE ,SIG_IGN ); - exit_code =do_fetch (gtransport, &rs); + exit_code =do_fetch (gtransport, &rs, display_format);sigchain_pop (SIGPIPE );refspec_clear (&rs);transport_disconnect (gtransport);
@@ -2146,6 +2156,9 @@ static int fetch_one(struct remote *remote, int argc, const char **argv,int cmd_fetch (int argc,const char **argv,const char *prefix) { +struct fetch_config config = { + .display_format =DISPLAY_FORMAT_FULL , + };int i;const char *bundle_uri;struct string_list list =STRING_LIST_INIT_DUP ;
@@ -2165,7 +2178,7 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)free (anon); } -git_config (git_fetch_config,NULL ); +git_config (git_fetch_config, &config);if (the_repository->gitdir) {prepare_repo_settings (the_repository); the_repository->settings.command_requires_full_index =0 ;
@@ -2302,7 +2315,8 @@ int cmd_fetch(int argc, const char **argv, const char *prefix) }else if (remote) {if (filter_options.choice ||has_promisor_remote ())fetch_one_setup_partial (remote); - result =fetch_one (remote, argc, argv, prune_tags_ok, stdin_refspecs); + result =fetch_one (remote, argc, argv, prune_tags_ok, stdin_refspecs, + config.display_format); }else {int max_children = max_jobs;
Signature · 1 line
2.40.1
PATCH v5 8/9 fetch: move option related variables into main function
In reply to Patrick SteinhardtThe options of git-fetch(1) which we pass to parse_options() are declared globally in builtin/fetch.c. This means we're forced to use global variables for all the options, which is more likely to cause confusion than explicitly passing state around.
Refactor the code to move the options into cmd_fetch(). Move variables that were previously forced to be declared globally and which are only used by cmd_fetch() into function-local scope.
--- builtin/fetch.c | 197 ++++++++++++++++++++++++------------------------ 1 file changed, 100 insertions(+), 97 deletions(-)
builtin/fetch.c +100 -97
@@ -75,13 +75,12 @@ static int fetch_prune_tags_config = -1; /* unspecified */static int prune_tags =-1 ;/* unspecified */ #define PRUNE_TAGS_BY_DEFAULT 0/* do we prune tags by default? */ -static int all, append, dry_run, force, keep, multiple, update_head_ok; +static int append, dry_run, force, keep, update_head_ok;static int write_fetch_head =1 ;static int verbosity, deepen_relative, set_upstream, refetch;static int progress =-1 ; -static int enable_auto_gc =1 ; -static int tags =TAGS_DEFAULT , unshallow, update_shallow, deepen; -static int max_jobs =-1 , submodule_fetch_jobs_config =-1 ; +static int tags =TAGS_DEFAULT , update_shallow, deepen; +static int submodule_fetch_jobs_config =-1 ;static int fetch_parallel_config =1 ;static int atomic_fetch;static enum transport_family family;
@@ -92,17 +91,11 @@ static struct string_list deepen_not = STRING_LIST_INIT_NODUP;static struct strbuf default_rla =STRBUF_INIT ;static struct transport *gtransport;static struct transport *gsecondary; -static const char *submodule_prefix ="" ;static int recurse_submodules =RECURSE_SUBMODULES_DEFAULT ; -static int recurse_submodules_cli =RECURSE_SUBMODULES_DEFAULT ; -static int recurse_submodules_default =RECURSE_SUBMODULES_ON_DEMAND ;static struct refspec refmap =REFSPEC_INIT_FETCH ;static struct list_objects_filter_options filter_options =LIST_OBJECTS_FILTER_INIT ;static struct string_list server_options =STRING_LIST_INIT_DUP ;static struct string_list negotiation_tip =STRING_LIST_INIT_NODUP ; -static int fetch_write_commit_graph =-1 ; -static int stdin_refspecs =0 ; -static int negotiate_only;struct fetch_config {enum display_format display_format;
@@ -178,92 +171,6 @@ static int parse_refmap_arg(const struct option *opt, const char *arg, int unsetreturn 0 ; } -static struct option builtin_fetch_options[] = { -OPT__VERBOSITY (&verbosity), -OPT_BOOL (0 ,"all" , &all, -N_ ("fetch from all remotes" )), -OPT_BOOL (0 ,"set-upstream" , &set_upstream, -N_ ("set upstream for git pull/fetch" )), -OPT_BOOL ('a' ,"append" , &append, -N_ ("append to .git/FETCH_HEAD instead of overwriting" )), -OPT_BOOL (0 ,"atomic" , &atomic_fetch, -N_ ("use atomic transaction to update references" )), -OPT_STRING (0 ,"upload-pack" , &upload_pack,N_ ("path" ), -N_ ("path to upload pack on remote end" )), -OPT__FORCE (&force,N_ ("force overwrite of local reference" ),0 ), -OPT_BOOL ('m' ,"multiple" , &multiple, -N_ ("fetch from multiple remotes" )), -OPT_SET_INT ('t' ,"tags" , &tags, -N_ ("fetch all tags and associated objects" ),TAGS_SET ), -OPT_SET_INT ('n' ,NULL , &tags, -N_ ("do not fetch all tags (--no-tags)" ),TAGS_UNSET ), -OPT_INTEGER ('j' ,"jobs" , &max_jobs, -N_ ("number of submodules fetched in parallel" )), -OPT_BOOL (0 ,"prefetch" , &prefetch, -N_ ("modify the refspec to place all refs within refs/prefetch/" )), -OPT_BOOL ('p' ,"prune" , &prune, -N_ ("prune remote-tracking branches no longer on remote" )), -OPT_BOOL ('P' ,"prune-tags" , &prune_tags, -N_ ("prune local tags no longer on remote and clobber changed tags" )), -OPT_CALLBACK_F (0 ,"recurse-submodules" , &recurse_submodules_cli,N_ ("on-demand" ), -N_ ("control recursive fetching of submodules" ), -PARSE_OPT_OPTARG , option_fetch_parse_recurse_submodules), -OPT_BOOL (0 ,"dry-run" , &dry_run, -N_ ("dry run" )), -OPT_BOOL (0 ,"write-fetch-head" , &write_fetch_head, -N_ ("write fetched references to the FETCH_HEAD file" )), -OPT_BOOL ('k' ,"keep" , &keep,N_ ("keep downloaded pack" )), -OPT_BOOL ('u' ,"update-head-ok" , &update_head_ok, -N_ ("allow updating of HEAD ref" )), -OPT_BOOL (0 ,"progress" , &progress,N_ ("force progress reporting" )), -OPT_STRING (0 ,"depth" , &depth,N_ ("depth" ), -N_ ("deepen history of shallow clone" )), -OPT_STRING (0 ,"shallow-since" , &deepen_since,N_ ("time" ), -N_ ("deepen history of shallow repository based on time" )), -OPT_STRING_LIST (0 ,"shallow-exclude" , &deepen_not,N_ ("revision" ), -N_ ("deepen history of shallow clone, excluding rev" )), -OPT_INTEGER (0 ,"deepen" , &deepen_relative, -N_ ("deepen history of shallow clone" )), -OPT_SET_INT_F (0 ,"unshallow" , &unshallow, -N_ ("convert to a complete repository" ), -1 ,PARSE_OPT_NONEG ), -OPT_SET_INT_F (0 ,"refetch" , &refetch, -N_ ("re-fetch without negotiating common commits" ), -1 ,PARSE_OPT_NONEG ), - {OPTION_STRING ,0 ,"submodule-prefix" , &submodule_prefix,N_ ("dir" ), -N_ ("prepend this to submodule path output" ),PARSE_OPT_HIDDEN }, -OPT_CALLBACK_F (0 ,"recurse-submodules-default" , - &recurse_submodules_default,N_ ("on-demand" ), -N_ ("default for recursive fetching of submodules " -"(lower priority than config files)" ), -PARSE_OPT_HIDDEN , option_fetch_parse_recurse_submodules), -OPT_BOOL (0 ,"update-shallow" , &update_shallow, -N_ ("accept refs that update .git/shallow" )), -OPT_CALLBACK_F (0 ,"refmap" ,NULL ,N_ ("refmap" ), -N_ ("specify fetch refmap" ),PARSE_OPT_NONEG , parse_refmap_arg), -OPT_STRING_LIST ('o' ,"server-option" , &server_options,N_ ("server-specific" ),N_ ("option to transmit" )), -OPT_SET_INT ('4' ,"ipv4" , &family,N_ ("use IPv4 addresses only" ), -TRANSPORT_FAMILY_IPV4 ), -OPT_SET_INT ('6' ,"ipv6" , &family,N_ ("use IPv6 addresses only" ), -TRANSPORT_FAMILY_IPV6 ), -OPT_STRING_LIST (0 ,"negotiation-tip" , &negotiation_tip,N_ ("revision" ), -N_ ("report that we have only objects reachable from this object" )), -OPT_BOOL (0 ,"negotiate-only" , &negotiate_only, -N_ ("do not fetch a packfile; instead, print ancestors of negotiation tips" )), -OPT_PARSE_LIST_OBJECTS_FILTER (&filter_options), -OPT_BOOL (0 ,"auto-maintenance" , &enable_auto_gc, -N_ ("run 'maintenance --auto' after fetching" )), -OPT_BOOL (0 ,"auto-gc" , &enable_auto_gc, -N_ ("run 'maintenance --auto' after fetching" )), -OPT_BOOL (0 ,"show-forced-updates" , &fetch_show_forced_updates, -N_ ("check for forced-updates on all updated branches" )), -OPT_BOOL (0 ,"write-commit-graph" , &fetch_write_commit_graph, -N_ ("write the commit-graph after fetching" )), -OPT_BOOL (0 ,"stdin" , &stdin_refspecs, -N_ ("accept refspecs from stdin" )), -OPT_END () -}; -static void unlock_pack (unsigned int flags) {if (gtransport)
@@ -2159,12 +2066,108 @@ int cmd_fetch(int argc, const char **argv, const char *prefix)struct fetch_config config = { .display_format =DISPLAY_FORMAT_FULL , }; -int i; +const char *submodule_prefix ="" ;const char *bundle_uri;struct string_list list =STRING_LIST_INIT_DUP ;struct remote *remote =NULL ; +int all =0 , multiple =0 ;int result =0 ;int prune_tags_ok =1 ; +int enable_auto_gc =1 ; +int unshallow =0 ; +int max_jobs =-1 ; +int recurse_submodules_cli =RECURSE_SUBMODULES_DEFAULT ; +int recurse_submodules_default =RECURSE_SUBMODULES_ON_DEMAND ; +int fetch_write_commit_graph =-1 ; +int stdin_refspecs =0 ; +int negotiate_only =0 ; +int i; + +struct option builtin_fetch_options[] = { +OPT__VERBOSITY (&verbosity), +OPT_BOOL (0 ,"all" , &all, +N_ ("fetch from all remotes" )), +OPT_BOOL (0 ,"set-upstream" , &set_upstream, +N_ ("set upstream for git pull/fetch" )), +OPT_BOOL ('a' ,"append" , &append, +N_ ("append to .git/FETCH_HEAD instead of overwriting" )), +OPT_BOOL (0 ,"atomic" , &atomic_fetch, +N_ ("use atomic transaction to update references" )), +OPT_STRING (0 ,"upload-pack" , &upload_pack,N_ ("path" ), +N_ ("path to upload pack on remote end" )), +OPT__FORCE (&force,N_ ("force overwrite of local reference" ),0 ), +OPT_BOOL ('m' ,"multiple" , &multiple, +N_ ("fetch from multiple remotes" )), +OPT_SET_INT ('t' ,"tags" , &tags, +N_ ("fetch all tags and associated objects" ),TAGS_SET ), +OPT_SET_INT ('n' ,NULL , &tags, +N_ ("do not fetch all tags (--no-tags)" ),TAGS_UNSET ), +OPT_INTEGER ('j' ,"jobs" , &max_jobs, +N_ ("number of submodules fetched in parallel" )), +OPT_BOOL (0 ,"prefetch" , &prefetch, +N_ ("modify the refspec to place all refs within refs/prefetch/" )), +OPT_BOOL ('p' ,"prune" , &prune, +N_ ("prune remote-tracking branches no longer on remote" )), +OPT_BOOL ('P' ,"prune-tags" , &prune_tags, +N_ ("prune local tags no longer on remote and clobber changed tags" )), +OPT_CALLBACK_F (0 ,"recurse-submodules" , &recurse_submodules_cli,N_ ("on-demand" ), +N_ ("control recursive fetching of submodules" ), +PARSE_OPT_OPTARG , option_fetch_parse_recurse_submodules), +OPT_BOOL (0 ,"dry-run" , &dry_run, +N_ ("dry run" )), +OPT_BOOL (0 ,"write-fetch-head" , &write_fetch_head, +N_ ("write fetched references to the FETCH_HEAD file" )), +OPT_BOOL ('k' ,"keep" , &keep,N_ ("keep downloaded pack" )), +OPT_BOOL ('u' ,"update-head-ok" , &update_head_ok, +N_ ("allow updating of HEAD ref" )), +OPT_BOOL (0 ,"progress" , &progress,N_ ("force progress reporting" )), +OPT_STRING (0 ,"depth" , &depth,N_ ("depth" ), +N_ ("deepen history of shallow clone" )), +OPT_STRING (0 ,"shallow-since" , &deepen_since,N_ ("time" ), +N_ ("deepen history of shallow repository based on time" )), +OPT_STRING_LIST (0 ,"shallow-exclude" , &deepen_not,N_ ("revision" ), +N_ ("deepen history of shallow clone, excluding rev" )), +OPT_INTEGER (0 ,"deepen" , &deepen_relative, +N_ ("deepen history of shallow clone" )), +OPT_SET_INT_F (0 ,"unshallow" , &unshallow, +N_ ("convert to a complete repository" ), +1 ,PARSE_OPT_NONEG ), +OPT_SET_INT_F (0 ,"refetch" , &refetch, +N_ ("re-fetch without negotiating common commits" ), +1 ,PARSE_OPT_NONEG ), + {OPTION_STRING ,0 ,"submodule-prefix" , &submodule_prefix,N_ ("dir" ), +N_ ("prepend this to submodule path output" ),PARSE_OPT_HIDDEN }, +OPT_CALLBACK_F (0 ,"recurse-submodules-default" , + &recurse_submodules_default,N_ ("on-demand" ), +N_ ("default for recursive fetching of submodules " +"(lower priority than config files)" ), +PARSE_OPT_HIDDEN , option_fetch_parse_recurse_submodules), +OPT_BOOL (0 ,"update-shallow" , &update_shallow, +N_ ("accept refs that update .git/shallow" )), +OPT_CALLBACK_F (0 ,"refmap" ,NULL ,N_ ("refmap" ), +N_ ("specify fetch refmap" ),PARSE_OPT_NONEG , parse_refmap_arg), +OPT_STRING_LIST ('o' ,"server-option" , &server_options,N_ ("server-specific" ),N_ ("option to transmit" )), +OPT_SET_INT ('4' ,"ipv4" , &family,N_ ("use IPv4 addresses only" ), +TRANSPORT_FAMILY_IPV4 ), +OPT_SET_INT ('6' ,"ipv6" , &family,N_ ("use IPv6 addresses only" ), +TRANSPORT_FAMILY_IPV6 ), +OPT_STRING_LIST (0 ,"negotiation-tip" , &negotiation_tip,N_ ("revision" ), +N_ ("report that we have only objects reachable from this object" )), +OPT_BOOL (0 ,"negotiate-only" , &negotiate_only, +N_ ("do not fetch a packfile; instead, print ancestors of negotiation tips" )), +OPT_PARSE_LIST_OBJECTS_FILTER (&filter_options), +OPT_BOOL (0 ,"auto-maintenance" , &enable_auto_gc, +N_ ("run 'maintenance --auto' after fetching" )), +OPT_BOOL (0 ,"auto-gc" , &enable_auto_gc, +N_ ("run 'maintenance --auto' after fetching" )), +OPT_BOOL (0 ,"show-forced-updates" , &fetch_show_forced_updates, +N_ ("check for forced-updates on all updated branches" )), +OPT_BOOL (0 ,"write-commit-graph" , &fetch_write_commit_graph, +N_ ("write the commit-graph after fetching" )), +OPT_BOOL (0 ,"stdin" , &stdin_refspecs, +N_ ("accept refspecs from stdin" )), +OPT_END () + };packet_trace_identity ("fetch" );
Signature · 1 line
2.40.1
PATCH v5 9/9 fetch: introduce machine-parseable "porcelain" output format
In reply to Patrick SteinhardtThe output of git-fetch(1) is obviously designed for consumption by users, only: we neatly columnize data, we abbreviate reference names, we print neat arrows and we don't provide information about actual object IDs that have changed. This makes the output format basically unusable in the context of scripted invocations of git-fetch(1) that want to learn about the exact changes that the command performs.
Introduce a new machine-parseable "porcelain" output format that is supposed to fix this shortcoming. This output format is intended to provide information about every reference that is about to be updated, the old object ID that the reference has been pointing to and the new object ID it will be updated to. Furthermore, the output format provides the same flags as the human-readable format to indicate basic conditions for each reference update like whether it was a fast-forward update, a branch deletion, a rejected update or others.
The output format is quite simple:
``` <flag> <old-object-id> <new-object-id> <local-reference>\n ```
We assume two conditions which are generally true:
- The old and new object IDs have fixed known widths and cannot contain spaces.
- References cannot contain newlines.
With these assumptions, the output format becomes unambiguously parseable. Furthermore, given that this output is designed to be consumed by scripts, the machine-readable data is printed to stdout instead of stderr like the human-readable output is. This is mostly done so that other data printed to stderr, like error messages or progress meters, don't interfere with the parseable data.
A notable ommission here is that the output format does not include the remote from which a reference was fetched, which might be important information especially in the context of multi-remote fetches. But as such a format would require us to print the remote for every single reference update due to parallelizable fetches it feels wasteful for the most likely usecase, which is when fetching from a single remote.
In a similar spirit, a second restriction is that this cannot be used with --recurse-submodules. This is because any reference updates would be ambiguous without also printing the repository in which the update happens.
Considering that both multi-remote and submodule fetches are user-facing features, using them in conjunction with --porcelain that is intended for scripting purposes is likely not going to be useful in the majority of cases. With that in mind these restrictions feel acceptable. If usecases for either of these come up in the future though it is easy enough to add a new "porcelain-v2" format that adds this information.
--- Documentation/fetch-options.txt | 7 ++ Documentation/git-fetch.txt | 9 ++ builtin/fetch.c | 88 ++++++++++++++---- t/t5574-fetch-output.sh | 159 ++++++++++++++++++++++++++++++++ 4 files changed, 244 insertions(+), 19 deletions(-)
Documentation/fetch-options.txt +7 -0
@@ -78,6 +78,13 @@ linkgit:git-config[1]. --dry-run:: Show what would be done, without making any changes. +--porcelain:: + Print the output to standard output in an easy-to-parse format for + scripts. See section OUTPUT in linkgit:git-fetch[1] for details. ++ +This is incompatible with `--recurse-submodules=[yes|on-demand]` and takes +precedence over the `fetch.output` config option. + ifndef::git-pull[] --[no-]write-fetch-head:: Write the list of remote refs fetched in the `FETCH_HEAD`
Documentation/git-fetch.txt +9 -0
@@ -204,6 +204,15 @@ representing the status of a single ref. Each line is of the form: <flag> <summary> <from> -> <to> [<reason>] ------------------------------- +When using `--porcelain`, the output format is intended to be +machine-parseable. In contrast to the human-readable output formats it +thus prints to standard output instead of standard error. Each line is +of the form: + +------------------------------- +<flag> <old-object-id> <new-object-id> <local-reference> +------------------------------- + The status of up-to-date refs is shown only if the --verbose option is used.
builtin/fetch.c +69 -19
251 lines not on this page · raw message
t/t5574-fetch-output.sh +159 -0
171 lines not on this page · raw message
Signature · 1 line
2.40.1
Re: [PATCH v5 9/9] fetch: introduce machine-parseable "porcelain" output format
Quoted 8 lines · Patrick Steinhardt
t/t5574-fetch-output.sh +8
-0+test_expect_success 'fetch --no-porcelain overrides previous --porcelain' ' + test_when_finished "rm -rf no-porcelain" && + + git switch --create no-porcelain && + git clone . no-porcelain && + test_commit --no-tag no-porcelain && + old_commit=$(git rev-parse --short HEAD~) && + new_commit=$(git rev-parse --short HEAD) &&
t/t5574-fetch-output.sh test_expect_success 'fetch compact output' ' · line 197
Quoted 10 lines
t/t5574-fetch-output.sh +10
-0+ + cat >expect <<-EOF && + From $(test-tool path-utils real_path .)/. + $old_commit..$new_commit no-porcelain -> origin/no-porcelain + EOF + + git -C no-porcelain fetch --porcelain --no-porcelain >stdout 2>stderr && + test_cmp expect stderr && + test_must_be_empty stdout +'
(This shouldn't block the series) As a matter of taste, it feels like overkill to construct the whole output to check if --no-porcelain wins. Maybe in a future cleanup, we would go through the tests in this file and see where it is really important that we have the exact output and where it is not.
Quoted 3 lines · Patrick Steinhardt
- Patch 3/9: Added a test to verify that
git fetch -c fetch.outputwithout a value set fails as expected. Also dropped the tests that checked whether stdout was empty.
Thank you for the attention to this small detail of valueless configuration variable definition.
Quoted 4 lines
- Patch 4/9: Reformulated the commit message to treat the missing left-hand side of displayed references as an inconsistency instead of a bug. I've also added a testcase to verify that direct OID fetches continue to work as expected.
Again, the direct OID fetch is a good thing to test here. I noticed that the test added here insists that the standard output stream is empty when the command errors out, which is not consistent with [3/9] above.
Quoted 4 lines
- Patch 5/9: New patch that makes calculation of the table width for displayed reference updates self-contained in
refcol_width(). This is a preparatory refactoring that should make patch 6/9 easier to review.
It is an excellent idea to avoid calls to refcol_width() for each ref that gets shown and make the helper responsible for computing the required maxwidth. The result indeed has become easier to follow as you mentioned in your response to my review on the previous round.
Quoted 5 lines
- Patch 7/9: Refactored the code to parse the "fetch.output" config variable inside of
git_fetch_config()before we parse command line options. Also fixed that the commit message was still referring to--output-format=porcelaininstead of the new--porcelainswitch.
As a standalone step, it leaves an impression that the step makes the way we handle the output-format configuration variable inconsistent with the way we handle the other configuration variables, but I think it is a good place to stop for the purpose of this topic. It lays a good foundation for future clean-up after the dust settles from this topic---we may want to move global variables assigned in the git_fetch_config() into the fetch_config structure.
Quoted 3 lines
- Patch 9/9: The
--porcelainoption is now a simpleOPT_BOOL()that can be negated. Added a test that--no-porcelainworks as expected.
OK, this time the familiar "prepare a variable to its default, let config callback to overwrite it by reading configuration variables, and then let the command line option override it" is used and the result easy to follow. I do not think .display_format is never assigned DISPLAY_FORMAT_UNKNOWN with this change, so [6/9] could lose the value from the enum, I think. The defensive switch statement that has BUG() to notice an erroneous caller that pass values other than DISPLAY_FORMAT_{FULL,COMPACT} is still a good idea.
Looking good.
Thanks.
Quoted 1 line · Junio C Hamano
Patrick Steinhardt <[email protected]> writes:
[snip]
Quoted 9 lines
Quoted 4 lines
- Patch 4/9: Reformulated the commit message to treat the missing left-hand side of displayed references as an inconsistency instead of a bug. I've also added a testcase to verify that direct OID fetches continue to work as expected.
Again, the direct OID fetch is a good thing to test here. I noticed that the test added here insists that the standard output stream is empty when the command errors out, which is not consistent with [3/9] above.
I think you misread the test: yes, we do test stdout and stderr separately and in many cases assert that stdout is in fact empty. But none of the added tests are about failing commands. So given that:
- The added tests explicitly are about verifying the output format.
- The distinction between stdout and stderr matters.
- The distinction matters even more with the addition of --porcelain.
I think that explicitly verifing both output streams is the correct thing to do.
Quoted 4 lines
Quoted 3 lines
- Patch 9/9: The
--porcelainoption is now a simpleOPT_BOOL()that can be negated. Added a test that--no-porcelainworks as expected.
Quoted 9 lines
OK, this time the familiar "prepare a variable to its default, let config callback to overwrite it by reading configuration variables, and then let the command line option override it" is used and the result easy to follow. I do not think .display_format is never assigned
DISPLAY_FORMAT_UNKNOWNwith this change, so [6/9] could lose the value from the enum, I think. The defensive switch statement that hasBUG()to notice an erroneous caller that pass values other than DISPLAY_FORMAT_{FULL,COMPACT} is still a good idea.
Ah, right, DISPLAY_FORMAT_UNKNOWN isn't really needed anymore. I think it's still good to have valid values of the enum start with 1 so that it becomes easier to detect cases where we accidentally use a default initialized variable. But that can be achieved without giving the default value an explicit name.
I'll refrain from sending a new version just to remove this constant as it doesn't really feel worth it, though. But please, let me know in case you disagree and I'll send an updated version.
Thanks
Patrick
Quoted 8 lines · Patrick Steinhardt
Quoted 1 line · Junio C Hamano
Patrick Steinhardt <[email protected]> writes:
[snip]
Quoted 5 lines
Quoted 4 lines
- Patch 4/9: Reformulated the commit message to treat the missing left-hand side of displayed references as an inconsistency instead of a bug. I've also added a testcase to verify that direct OID fetches continue to work as expected.
Quoted 8 lines
Quoted 4 lines
Again, the direct OID fetch is a good thing to test here. I noticed that the test added here insists that the standard output stream is empty when the command errors out, which is not consistent with [3/9] above.
I think you misread the test: yes, we do test stdout and stderr separately and in many cases assert that stdout is in fact empty. But none of the added tests are about failing commands.
Ah, you are absolutely right. Sorry for the noise---I do not have an objection to ensure that program output in the successful case is predictable. My main concern is to avoid giving unneeded assurance in the failing case.
Quoted 11 lines
Quoted 5 lines
assigned
DISPLAY_FORMAT_UNKNOWNwith this change, so [6/9] could lose the value from the enum, I think. The defensive switch statement that hasBUG()to notice an erroneous caller that pass values other than DISPLAY_FORMAT_{FULL,COMPACT} is still a good idea.Ah, right,
DISPLAY_FORMAT_UNKNOWNisn't really needed anymore. I think it's still good to have valid values of the enum start with1so that it becomes easier to detect cases where we accidentally use a default initialized variable. But that can be achieved without giving the default value an explicit name.
Unless we explicitly take advantage of UNKNOWN being 0 and perform clever defaulting, the result is far safer if you get rid of it, and instead give 0 to a real choice that is used as the default, for two
reasons: (1) naturally 0 initialization work fine, (2) we have one less enum constant people (or editor's auto-completer) assign to a variable by mistake instead of a real one they intended to. I agree with the last sentence of your paragraph above.
Quoted 12 lines · Patrick Steinhardt
Quoted 9 lines · Junio C Hamano
OK, this time the familiar "prepare a variable to its default, let config callback to overwrite it by reading configuration variables, and then let the command line option override it" is used and the result easy to follow. I do not think .display_format is never assigned
DISPLAY_FORMAT_UNKNOWNwith this change, so [6/9] could lose the value from the enum, I think. The defensive switch statement that hasBUG()to notice an erroneous caller that pass values other than DISPLAY_FORMAT_{FULL,COMPACT} is still a good idea.
Quoted 5 lines
Ah, right,
DISPLAY_FORMAT_UNKNOWNisn't really needed anymore. I think it's still good to have valid values of the enum start with1so that it becomes easier to detect cases where we accidentally use a default initialized variable. But that can be achieved without giving the default value an explicit name.
I think it's standard to have an element like that. It could be UNKNOWN, NULL, INVALID, or even DEFAULT.
For example, you could have code like this:
if (format == DISPLAY_FORMAT_DEFAULT)
format = DISPLAY_FORMAT_FULL;
This would distinguish cases in which the user did not specify a display format and we choose a default, from those where the user explicitely chose the format that happens to be the default.
Signature · 1 line
Felipe Contreras
Quoted 24 lines · Patrick Steinhardt
Changes compared to v4:
- Patch 1/9: Simplified the test as proposed by Junio and Glen.
- Patch 3/9: Added a test to verify that
git fetch -c fetch.outputwithout a value set fails as expected. Also dropped the tests that checked whether stdout was empty.- Patch 4/9: Reformulated the commit message to treat the missing left-hand side of displayed references as an inconsistency instead of a bug. I've also added a testcase to verify that direct OID fetches continue to work as expected.
- Patch 5/9: New patch that makes calculation of the table width for displayed reference updates self-contained in
refcol_width(). This is a preparatory refactoring that should make patch 6/9 easier to review.- Patch 7/9: Refactored the code to parse the "fetch.output" config variable inside of
git_fetch_config()before we parse command line options. Also fixed that the commit message was still referring to--output-format=porcelaininstead of the new--porcelainswitch.
Quoted 3 lines
- Patch 9/9: The
--porcelainoption is now a simpleOPT_BOOL()that can be negated. Added a test that--no-porcelainworks as expected.
I didn't spot any blocking issues in this version, and the various improvements (especially 6-7/9) are really welcome. I also read through Junio's comments, but I didn't spot anything that I thought should block the series, so I'm happy to leave
Quoted 30 lines · Glen Choo
Quoted 24 lines · Patrick Steinhardt
Changes compared to v4:
- Patch 1/9: Simplified the test as proposed by Junio and Glen.
- Patch 3/9: Added a test to verify that
git fetch -c fetch.outputwithout a value set fails as expected. Also dropped the tests that checked whether stdout was empty.- Patch 4/9: Reformulated the commit message to treat the missing left-hand side of displayed references as an inconsistency instead of a bug. I've also added a testcase to verify that direct OID fetches continue to work as expected.
- Patch 5/9: New patch that makes calculation of the table width for displayed reference updates self-contained in
refcol_width(). This is a preparatory refactoring that should make patch 6/9 easier to review.- Patch 7/9: Refactored the code to parse the "fetch.output" config variable inside of
git_fetch_config()before we parse command line options. Also fixed that the commit message was still referring to--output-format=porcelaininstead of the new--porcelainswitch.Quoted 3 lines
- Patch 9/9: The
--porcelainoption is now a simpleOPT_BOOL()that can be negated. Added a test that--no-porcelainworks as expected.
Quoted 6 lines
I didn't spot any blocking issues in this version, and the various improvements (especially 6-7/9) are really welcome. I also read through Junio's comments, but I didn't spot anything that I thought should block the series, so I'm happy to leave
Reviewed-by Glen Choo <[email protected]>
Thanks a lot for your reviews!
Patrick