set-head/fetch remote/HEAD updates
A message that is not in the archive (sent off-list or not yet received): [email protected]
v10 cover set-head/fetch remote/HEAD updates
Sorry for sending a v10 so soon after v9 without really waiting for feedback, but as I started digging into
https://lore.kernel.org/git/[email protected]/ I realized that the root cause is in fetch and not remote add, so the solution belongs to this series. I'll leave my overall comments for v9 in here for clarity.
v10: added a new patch (8/8) on top of v9 to handle HEAD in mirrored bare repositories, rest of the patches are unchanged
v9:
the two most notable changes are
- the new patch 1/7 which address
https://lore.kernel.org/git/Zw8IKyPkG0Hr6%[email protected]/, but see
https://lore.kernel.org/git/[email protected]/
- the refs_update_symref -> refs_update_symref_extended change in 2/7, reflecting on Phillip's comments (see
https://lore.kernel.org/git/[email protected]/)
Hopefully with 1/7 the series is ready to move back to seen :)
Best,
Bence
Bence Ferdinandy (8):
t/t5505-remote: set default branch to main
refs: atomically record overwritten ref in update_symref remote set-head: refactor for readability
remote set-head: better output for --auto
refs: add TRANSACTION_CREATE_EXISTS error
refs: add create_only option to refs_update_symref_extended
fetch: set remote/HEAD if it does not exist
fetch set_head: handle mirrored bare repositories
builtin/fetch.c | 90 ++++++++++++ builtin/remote.c | 52 +++++-- refs.c | 41 +++++- refs.h | 8 +- refs/files-backend.c | 24 ++-- refs/reftable-backend.c | 6 +- t/t4207-log-decoration-colors.sh | 3 +- t/t5505-remote.sh | 83 ++++++++++- t/t5510-fetch.sh | 229 ++++++++++++++++--------------- t/t5512-ls-remote.sh | 2 + t/t5514-fetch-multiple.sh | 17 ++- t/t5516-fetch-push.sh | 3 +- t/t5527-fetch-odd-refs.sh | 3 +- t/t7900-maintenance.sh | 3 +- t/t9210-scalar.sh | 5 +- t/t9211-scalar-clone.sh | 6 +- t/t9902-completion.sh | 65 +++++++++ 17 files changed, 486 insertions(+), 154 deletions(-)
Signature · 1 line
2.47.0.94.g8861098b6d
PATCH v10 1/8 t/t5505-remote: set default branch to main
Consider the bare repository called "mirror" in the test. Running git remote add --mirror -f origin ../one will not change HEAD, consequently if init.defaultBranch is not the same as what HEAD in the remote ("one"), HEAD in "mirror" will be pointing to a non-existent reference. Hence if "mirror" is used as a remote by yet another repository, ls-remote will not show HEAD. On the other hand, if init.defaultBranch happens to match HEAD in "one", then ls-remote will show HEAD.
Since the CI globally exports GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main, there's a drift between how the test repositories are set up in the CI and during local testing. This issue does not manifest currently, as the test does not do any remote HEAD manipulation where this would come up, but should such things be added, a locally passing test would break the CI vice-versa.
Set GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main in the test to be consistent with the CI.
---
Notes:
v9: - new patch
- a bandaid for the CI issue noticed by Taylor (cf:
https://lore.kernel.org/git/Zw8IKyPkG0Hr6%[email protected]/), but
see
https://lore.kernel.org/git/[email protected]/ for the root cause in detail
v10: no change
t/t5505-remote.sh | 3 +++ 1 file changed, 3 insertions(+)
t/t5505-remote.sh +3 -0
@@ -2,6 +2,9 @@ test_description='git remote porcelain-ish' +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME + TEST_PASSES_SANITIZE_LEAK=true . ./test-lib.sh
Signature · 1 line
2.47.0.94.g8861098b6d
PATCH v10 2/8 refs: atomically record overwritten ref in update_symref
In reply to Bence FerdinandyWhen updating a symref with update_symref it's currently not possible to know for sure what was the previous value that was overwritten. Extend refs_update_symref under a new function name, to record the value after the ref has been locked if the caller of refs_update_symref_extended requests it via a new variable in the function call. Keep the original refs_update_symref function with the same signature, but now as a wrapper around refs_update_symref_extended.
---
Notes:
v4: new patch
v5: - added before_target to reftables backend
- added an extra safety check for transaction's existence in refs.c
v6: - no change
v7: - remove the whole before_target concept from the backends and
handle checking it in refs.c instead (thanks Karthik)
- rename the before_target to referent which is how the same concept is called in the backends
- change commit prefix to be more in line with project standards
v8: no change
v9: - instead of adding parameters to refs_update_symref, rename what
was in v8 as refs_update_symref_extended and make refs_update_symref a wrapper for that. This significantly reduces the number of files that need to be touched, and avoids adding a lot of dummy NULL-s in unrelated places.
v10: no change
refs.c | 19 ++++++++++++++++--- refs.h | 4 ++++ 2 files changed, 20 insertions(+), 3 deletions(-)
refs.c +16 -3
@@ -2115,6 +2115,13 @@ int peel_iterated_oid(struct repository *r, const struct object_id *base, structint refs_update_symref (struct ref_store *refs,const char *ref,const char *target,const char *logmsg) +{ +return refs_update_symref_extended (refs, ref, target, logmsg,NULL ); +} + +int refs_update_symref_extended (struct ref_store *refs,const char *ref, +const char *target,const char *logmsg, +struct strbuf *referent) {struct ref_transaction *transaction;struct strbuf err =STRBUF_INIT ;
@@ -2122,13 +2129,20 @@ int refs_update_symref(struct ref_store *refs, const char *ref, transaction =ref_store_transaction_begin (refs, &err);if (!transaction || -ref_transaction_update (transaction, ref,NULL ,NULL , +ref_transaction_update (transaction, ref,NULL ,NULL , target,NULL ,REF_NO_DEREF , logmsg, &err) || -ref_transaction_commit (transaction, &err)) { +ref_transaction_prepare (transaction, &err)) { ret =error ("%s" , err.buf); + goto cleanup; } +if (referent) +refs_read_symbolic_ref (refs, ref, referent); +if (ref_transaction_commit (transaction, &err)) + ret =error ("%s" , err.buf); + +cleanup:strbuf_release (&err);if (transaction)ref_transaction_free (transaction);
@@ -2948,4 +2962,3 @@ int ref_update_expects_existing_old_ref(struct ref_update *update)return (update->flags &REF_HAVE_OLD ) && (!is_null_oid (&update->old_oid) || update->old_target); } -
refs.h +4 -0
@@ -573,6 +573,10 @@ int refs_copy_existing_ref(struct ref_store *refs, const char *oldref,int refs_update_symref (struct ref_store *refs,const char *refname,const char *target,const char *logmsg); +int refs_update_symref_extended (struct ref_store *refs,const char *refname, +const char *target,const char *logmsg, +struct strbuf *referent); +enum action_on_err {UPDATE_REFS_MSG_ON_ERR ,UPDATE_REFS_DIE_ON_ERR ,
Signature · 1 line
2.47.0.94.g8861098b6d
PATCH v10 3/8 remote set-head: refactor for readability
In reply to Bence FerdinandyRename buf and buf2 to something more explanatory.
Instead of calling get_main_ref_store(the_repository) multiple times, call it once and store in a new refs variable. Although this change probably offers some performance benefits, the main purpose is to shorten the line lengths of function calls using this variable.
---
Notes:
v5: new patch (split from the next patch as a preparatory step)
v6: no change
v7: - change commit prefix to be more in line with project standards
v8: no change
v9: - further improve readability by renaming buf, and buf2 consistently
with how patch 6 was already done
v10: no change
builtin/remote.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-)
builtin/remote.c +12 -11
@@ -1402,8 +1402,9 @@ static int show(int argc, const char **argv, const char *prefix) static int set_head(int argc, const char **argv, const char *prefix) { int i, opt_a = 0, opt_d = 0, result = 0; - struct strbuf buf = STRBUF_INIT, buf2 = STRBUF_INIT; + struct strbuf b_head = STRBUF_INIT, b_remote_head = STRBUF_INIT; char *head_name = NULL; + struct ref_store *refs = get_main_ref_store(the_repository); struct option options[] = { OPT_BOOL('a', "auto", &opt_a,
@@ -1415,7 +1416,7 @@ static int set_head(int argc, const char **argv, const char *prefix) argc =parse_options (argc, argv, prefix, options, builtin_remote_sethead_usage,0 );if (argc) -strbuf_addf (&buf,"refs/remotes/%s/HEAD" , argv[0 ]); +strbuf_addf (&b_head,"refs/remotes/%s/HEAD" , argv[0 ]);if (!opt_a && !opt_d && argc ==2 ) { head_name =xstrdup (argv[1 ]);
@@ -1434,25 +1435,25 @@ static int set_head(int argc, const char **argv, const char *prefix) head_name =xstrdup (states.heads.items[0 ].string);free_remote_ref_states (&states); }else if (opt_d && !opt_a && argc ==1 ) { -if (refs_delete_ref (get_main_ref_store (the_repository),NULL , buf.buf,NULL ,REF_NO_DEREF )) - result |=error (_ ("Could not delete %s" ), buf.buf); +if (refs_delete_ref (refs,NULL , b_head.buf,NULL ,REF_NO_DEREF )) + result |=error (_ ("Could not delete %s" ), b_head.buf); }else usage_with_options (builtin_remote_sethead_usage, options);if (head_name) { -strbuf_addf (&buf2,"refs/remotes/%s/%s" , argv[0 ], head_name); +strbuf_addf (&b_remote_head,"refs/remotes/%s/%s" , argv[0 ], head_name);/* make sure it's valid */ -if (!refs_ref_exists (get_main_ref_store (the_repository), buf2.buf)) - result |=error (_ ("Not a valid ref: %s" ), buf2.buf); -else if (refs_update_symref (get_main_ref_store (the_repository), buf.buf, buf2.buf,"remote set-head" )) - result |=error (_ ("Could not setup %s" ), buf.buf); +if (!refs_ref_exists (refs, b_remote_head.buf)) + result |=error (_ ("Not a valid ref: %s" ), b_remote_head.buf); +else if (refs_update_symref (refs, b_head.buf, b_remote_head.buf,"remote set-head" )) + result |=error (_ ("Could not setup %s" ), b_head.buf);else if (opt_a)printf ("%s/HEAD set to %s\n" , argv[0 ], head_name);free (head_name); } -strbuf_release (&buf); -strbuf_release (&buf2); +strbuf_release (&b_head); +strbuf_release (&b_remote_head);return result; }
Signature · 1 line
2.47.0.94.g8861098b6d
PATCH v10 4/8 remote set-head: better output for --auto
In reply to Bence FerdinandyCurrently, set-head --auto will print a message saying "remote/HEAD set to branch", which implies something was changed.
Change the output of --auto, so the output actually reflects what was
done: a) set a previously unset HEAD, b) change HEAD because remote changed or c) no updates. As a fourth output, if HEAD is changed from a previous value that was not a remote branch, explicitly call attention to this fact.
---
Notes:
v1-v2:
was RFC in https://lore.kernel.org/git/[email protected]/
v3:
This patch was originally sent along when I thought set-head was going to be invoked by fetch, but the discussion on the RFC concluded that it should be not. This opened the possibility to make it more explicit.
Note: although I feel both things the patch does are really just cosmetic, an argument could be made for breaking it into two, one for the no-op part and one for the --auto print update.
Was sent in:
https://lore.kernel.org/git/[email protected]/
v4:
- changes are now handled atomically via the ref update transaction
- outputs have changed along the lines of Junio's suggestion
- minor refactor to set_head for improved legibility
v5: - the minor refactor has been split out into its own patch
v6: - fixed uninitialized prev_head
- fixed case of unusual previous target
- fixed a test that would have been actually broken at this patch (the output was only correct with the later update to fetch)
- added 4 tests for the 4 output cases
v7: - change commit prefix to be more in line with project standards
- fixed tests to also work with the reftable backend
- renamed report function, fixed style issue with checking buf len
- fixed not releasing an strbuf
v8: no change
v9: - mark output strings in report_set_head_auto as translatable
- rename buf_prev to b_local_head for consistency
- use ${SQ} in tests instead of '\''
v10: no change
builtin/remote.c | 33 +++++++++++++++++++++++++++--- t/t5505-remote.sh | 51 ++++++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 80 insertions(+), 4 deletions(-)
builtin/remote.c +30 -3
@@ -1399,10 +1399,35 @@ static int show(int argc, const char **argv, const char *prefix)return result; } +static void report_set_head_auto (const char *remote,const char *head_name, +struct strbuf *b_local_head) { +struct strbuf buf_prefix =STRBUF_INIT ; +const char *prev_head =NULL ; + +strbuf_addf (&buf_prefix,"refs/remotes/%s/" , remote); +skip_prefix (b_local_head->buf, buf_prefix.buf, &prev_head); + +if (prev_head && !strcmp (prev_head, head_name)) +printf (_ ("'%s/HEAD' is unchanged and points to '%s'\n" ), + remote, head_name); +else if (prev_head) +printf (_ ("'%s/HEAD' has changed from '%s' and now points to '%s'\n" ), + remote, prev_head, head_name); +else if (!b_local_head->len) +printf (_ ("'%s/HEAD' is now created and points to '%s'\n" ), + remote, head_name); +else +printf (_ ("'%s/HEAD' used to point to '%s' " +"(which is not a remote branch), but now points to '%s'\n" ), + remote, b_local_head->buf, head_name); +strbuf_release (&buf_prefix); +} +static int set_head (int argc,const char **argv,const char *prefix) {int i, opt_a =0 , opt_d =0 , result =0 ; -struct strbuf b_head =STRBUF_INIT , b_remote_head =STRBUF_INIT ; +struct strbuf b_head =STRBUF_INIT , b_remote_head =STRBUF_INIT , + b_local_head =STRBUF_INIT ;char *head_name =NULL ;struct ref_store *refs =get_main_ref_store (the_repository);
@@ -1445,15 +1470,17 @@ static int set_head(int argc, const char **argv, const char *prefix)/* make sure it's valid */ if (!refs_ref_exists (refs, b_remote_head.buf)) result |=error (_ ("Not a valid ref: %s" ), b_remote_head.buf); -else if (refs_update_symref (refs, b_head.buf, b_remote_head.buf,"remote set-head" )) +else if (refs_update_symref_extended (refs, b_head.buf, b_remote_head.buf, +"remote set-head" , &b_local_head)) result |=error (_ ("Could not setup %s" ), b_head.buf);else if (opt_a) -printf ("%s/HEAD set to %s\n" , argv[0 ], head_name); +report_set_head_auto (argv[0 ], head_name, &b_local_head);free (head_name); }strbuf_release (&b_head);strbuf_release (&b_remote_head); +strbuf_release (&b_local_head);return result; }
t/t5505-remote.sh +50 -1
68 lines not on this page · raw message
Signature · 1 line
2.47.0.94.g8861098b6d
PATCH v10 5/8 refs: add TRANSACTION_CREATE_EXISTS error
In reply to Bence FerdinandyCurrently there is only one special error for transaction, for when there is a naming conflict, all other errors are dumped under a generic error. Add a new special error case for when the caller requests the reference to be updated only when it does not yet exist and the reference actually does exist.
---
Notes:
v4: new patch
v5: no change
v6: no change
v7: - change commit prefix to be more in line with project standards
- changed error checking to Karthik's suggestion
v8: no change
v9: - no change
v10: no change
refs.h | 4 +++- refs/files-backend.c | 24 ++++++++++++++++-------- refs/reftable-backend.c | 6 ++++-- 3 files changed, 23 insertions(+), 11 deletions(-)
refs.h +3 -1
@@ -762,8 +762,10 @@ int ref_transaction_verify(struct ref_transaction *transaction,/* Naming conflict (for example, the ref names A and A/B conflict). */ #define TRANSACTION_NAME_CONFLICT -1 +/* When only creation was requested, but the ref already exists. */ +#define TRANSACTION_CREATE_EXISTS -2/* All other errors. */ -#define TRANSACTION_GENERIC_ERROR -2 +#define TRANSACTION_GENERIC_ERROR -3/* * Perform the preparatory stages of committing `transaction`. Acquire
refs/files-backend.c +16 -8
@@ -2502,14 +2502,18 @@ static int split_symref_update(struct ref_update *update,static int check_old_oid (struct ref_update *update,struct object_id *oid,struct strbuf *err) { +int ret =TRANSACTION_GENERIC_ERROR ; +if (!(update->flags &REF_HAVE_OLD ) ||oideq (oid, &update->old_oid))return 0 ; -if (is_null_oid (&update->old_oid)) +if (is_null_oid (&update->old_oid)) {strbuf_addf (err,"cannot lock ref '%s': " "reference already exists" ,ref_update_original_update_refname (update)); + ret =TRANSACTION_CREATE_EXISTS ; + }else if (is_null_oid (oid)) strbuf_addf(err, "cannot lock ref '%s': " "reference is missing but expected %s",
@@ -2522,7 +2526,7 @@ static int check_old_oid(struct ref_update *update, struct object_id *oid,oid_to_hex (oid), oid_to_hex(&update->old_oid)); -return -1 ; +return ret; }/*
@@ -2602,9 +2606,11 @@ static int lock_ref_for_update(struct files_ref_store *refs, ret =TRANSACTION_GENERIC_ERROR ; goto out; } - }else if (check_old_oid (update, &lock->old_oid, err)) { - ret =TRANSACTION_GENERIC_ERROR ; - goto out; + }else { + ret =check_old_oid (update, &lock->old_oid, err); +if (ret) { + goto out; + } } }else {/*
@@ -2635,9 +2641,11 @@ static int lock_ref_for_update(struct files_ref_store *refs, update->old_target); ret =TRANSACTION_GENERIC_ERROR ; goto out; - }else if (check_old_oid (update, &lock->old_oid, err)) { - ret =TRANSACTION_GENERIC_ERROR ; - goto out; + }else { + ret =check_old_oid (update, &lock->old_oid, err); +if (ret) { + goto out; + } }/*
refs/reftable-backend.c +4 -2
@@ -1206,10 +1206,13 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store, goto done; } }else if ((u->flags &REF_HAVE_OLD ) && !oideq (¤t_oid, &u->old_oid)) { -if (is_null_oid (&u->old_oid)) + ret =TRANSACTION_NAME_CONFLICT ; +if (is_null_oid (&u->old_oid)) {strbuf_addf (err,_ ("cannot lock ref '%s': " "reference already exists" ),ref_update_original_update_refname (u)); + ret =TRANSACTION_CREATE_EXISTS ; + }else if (is_null_oid (¤t_oid)) strbuf_addf(err, _("cannot lock ref '%s': " "reference is missing but expected %s"),
@@ -1221,7 +1224,6 @@ static int reftable_be_transaction_prepare(struct ref_store *ref_store,ref_update_original_update_refname (u),oid_to_hex (¤t_oid), oid_to_hex(&u->old_oid)); - ret =-1 ; goto done; }
Signature · 1 line
2.47.0.94.g8861098b6d
PATCH v10 6/8 refs: add create_only option to refs_update_symref_extended
In reply to Bence FerdinandyAllow the caller to specify that it only wants to update the symref if it does not already exist. Silently ignore the error from the transaction API if the symref already exists.
---
Notes:
v4: new patch
v5: no change
v6: - switched from bool to int for create_only
- refactored logic in refs_update_symref with goto as layed out by Junio
v7: - change commit prefix to be more in line with project standards
- refactored code to accommodate changes in the first patch, but otherwise no change
v8: no change
v9: - no change (except for following through the
refs_update_symref_extended refactoring)
v10: no change
builtin/remote.c | 2 +- refs.c | 32 +++++++++++++++++++++++--------- refs.h | 2 +- 3 files changed, 25 insertions(+), 11 deletions(-)
builtin/remote.c +1 -1
@@ -1471,7 +1471,7 @@ static int set_head(int argc, const char **argv, const char *prefix)if (!refs_ref_exists (refs, b_remote_head.buf)) result |=error (_ ("Not a valid ref: %s" ), b_remote_head.buf);else if (refs_update_symref_extended (refs, b_head.buf, b_remote_head.buf, -"remote set-head" , &b_local_head)) +"remote set-head" , &b_local_head,0 )) result |=error (_ ("Could not setup %s" ), b_head.buf);else if (opt_a)report_set_head_auto (argv[0 ], head_name, &b_local_head);
refs.c +23 -9
@@ -2116,31 +2116,45 @@ int peel_iterated_oid(struct repository *r, const struct object_id *base, structint refs_update_symref (struct ref_store *refs,const char *ref,const char *target,const char *logmsg) { -return refs_update_symref_extended (refs, ref, target, logmsg,NULL ); +return refs_update_symref_extended (refs, ref, target, logmsg,NULL ,0 ); }int refs_update_symref_extended (struct ref_store *refs,const char *ref,const char *target,const char *logmsg, -struct strbuf *referent) +struct strbuf *referent,int create_only) {struct ref_transaction *transaction;struct strbuf err =STRBUF_INIT ; -int ret =0 ; +int ret =0 , prepret =0 ; transaction =ref_store_transaction_begin (refs, &err); -if (!transaction || -ref_transaction_update (transaction, ref,NULL ,NULL , - target,NULL ,REF_NO_DEREF , - logmsg, &err) || -ref_transaction_prepare (transaction, &err)) { +if (!transaction) { + error_return: ret =error ("%s" , err.buf); goto cleanup; } +if (create_only) { +if (ref_transaction_create (transaction, ref,NULL , target, +REF_NO_DEREF , logmsg, &err)) + goto error_return; + prepret =ref_transaction_prepare (transaction, &err); +if (prepret && prepret !=TRANSACTION_CREATE_EXISTS ) + goto error_return; + }else { +if (ref_transaction_update (transaction, ref,NULL ,NULL , + target,NULL ,REF_NO_DEREF , + logmsg, &err) || +ref_transaction_prepare (transaction, &err)) + goto error_return; + } +if (referent)refs_read_symbolic_ref (refs, ref, referent); +if (prepret ==TRANSACTION_CREATE_EXISTS ) + goto cleanup;if (ref_transaction_commit (transaction, &err)) - ret =error ("%s" , err.buf); + goto error_return; cleanup:strbuf_release (&err);
refs.h +1 -1
@@ -575,7 +575,7 @@ int refs_update_symref(struct ref_store *refs, const char *refname,int refs_update_symref_extended (struct ref_store *refs,const char *refname,const char *target,const char *logmsg, -struct strbuf *referent); +struct strbuf *referent,int create_only);enum action_on_err {UPDATE_REFS_MSG_ON_ERR ,
Signature · 1 line
2.47.0.94.g8861098b6d
PATCH v10 7/8 fetch: set remote/HEAD if it does not exist
In reply to Bence FerdinandyIf the user has remote/HEAD set already and it looks like it has changed on the server, then print a message, otherwise set it if we can. Silently pass if the user already has the same remote/HEAD set as reported by the server or if we encounter any errors along the way.
---
Notes:
v3: - does not rely on remote set-head anymore so it only authenticates
once
- uses the new REF_CREATE_ONLY to atomically check if the ref exists and only write it if it doesn't
- in all other cases the maximum it does is print a warning
v4: - instead of the discarded REF_CREATE_ONLY, it uses the existing,
but updated transaction api to request a silent create only
- it now uses the atomic before_target to determine reporting
- refactored for legibility
v5: - instead of printing a not too useful message, it now fails
silently, this in line with the objective to only set up remote/HEAD automatically if the right thing is trivial, for everything else there is remote set-head
- fixed all failing tests
- added two new tests, one for checking if remote/HEAD is set to the correct one, and one to test that we do not override remote/HEAD if it has changed on the server from what we have locally
v6: - fixed style issues and unintended extra empty line
- updated function call with bool to int from previous patch's change
- removed calls to error(...) inherited from builtin/remote.c so we actually fail silently
- set the test for remote set-head --auto to the correct value here, which was previously erronously set in the remote set-head patch
v7: - no change
v8: - changed logmsg in call to refs_update_symref from "remote
set-head" to "fetch"
v9: - follow through with refs_update_symref_extended
- fix test errors uncovered by the new patch
v10: no change
builtin/fetch.c | 82 +++++++++++ t/t4207-log-decoration-colors.sh | 3 +- t/t5505-remote.sh | 21 ++- t/t5510-fetch.sh | 229 ++++++++++++++++--------------- t/t5512-ls-remote.sh | 2 + t/t5514-fetch-multiple.sh | 17 ++- t/t5516-fetch-push.sh | 3 +- t/t5527-fetch-odd-refs.sh | 3 +- t/t7900-maintenance.sh | 3 +- t/t9210-scalar.sh | 5 +- t/t9211-scalar-clone.sh | 6 +- t/t9902-completion.sh | 65 +++++++++ 12 files changed, 315 insertions(+), 124 deletions(-)
builtin/fetch.c +82 -0
@@ -1578,6 +1578,80 @@ static int backfill_tags(struct display_state *display_state,return retcode; } +static void report_set_head (const char *remote,const char *head_name, +struct strbuf *buf_prev) { +struct strbuf buf_prefix =STRBUF_INIT ; +const char *prev_head =NULL ; + +strbuf_addf (&buf_prefix,"refs/remotes/%s/" , remote); +skip_prefix (buf_prev->buf, buf_prefix.buf, &prev_head); + +if (prev_head &&strcmp (prev_head, head_name)) { +printf ("'HEAD' at '%s' has changed from '%s' to '%s'\n" , + remote, prev_head, head_name); +printf ("Run 'git remote set-head %s %s' to follow the change.\n" , + remote, head_name); + } +} + +static const char *strip_refshead (const char *name){ +skip_prefix (name,"refs/heads/" , &name); +return name; +} + +static int set_head (const struct ref *remote_refs) +{ +int result =0 ; +struct strbuf b_head =STRBUF_INIT , b_remote_head =STRBUF_INIT , + b_local_head =STRBUF_INIT ; +const char *remote = gtransport->remote->name; +char *head_name =NULL ; +struct ref *ref, *matches; +struct ref *fetch_map =NULL , **fetch_map_tail = &fetch_map; +struct refspec_item refspec = { + .force =0 , + .pattern =1 , + .src = (char *)"refs/heads/*" , + .dst = (char *)"refs/heads/*" , + }; +struct string_list heads =STRING_LIST_INIT_DUP ; +struct ref_store *refs =get_main_ref_store (the_repository); + +get_fetch_map (remote_refs, &refspec, &fetch_map_tail,0 ); + matches =guess_remote_head (find_ref_by_name (remote_refs,"HEAD" ), + fetch_map,1 ); +for (ref = matches; ref; ref = ref->next) { +string_list_append (&heads,strip_refshead (ref->name)); + } + + +if (!heads.nr) + result =1 ; +else if (heads.nr >1 ) + result =1 ; +else + head_name =xstrdup (heads.items[0 ].string); +if (head_name) { +strbuf_addf (&b_head,"refs/remotes/%s/HEAD" , remote); +strbuf_addf (&b_remote_head,"refs/remotes/%s/%s" , remote, head_name); +/* make sure it's valid */ +if (!refs_ref_exists (refs, b_remote_head.buf)) + result =1 ; +else if (refs_update_symref_extended (refs, b_head.buf, b_remote_head.buf, +"fetch" , &b_local_head,1 )) + result =1 ; +else +report_set_head (remote, head_name, &b_local_head); + +free (head_name); + } + +strbuf_release (&b_head); +strbuf_release (&b_local_head); +strbuf_release (&b_remote_head); +return result; +} +static int do_fetch (struct transport *transport,struct refspec *rs,const struct fetch_config *config)
@@ -1647,6 +1721,8 @@ static int do_fetch(struct transport *transport, "refs/tags/"); } +strvec_push (&transport_ls_refs_options.ref_prefixes,"HEAD" ); +if (must_list_refs) {trace2_region_enter ("fetch" ,"remote_refs" , the_repository); remote_refs = transport_get_remote_refs(transport,
@@ -1791,6 +1867,12 @@ static int do_fetch(struct transport *transport, "you need to specify exactly one branch with the --set-upstream option")); } } +if (set_head (remote_refs)) + ; +/* +* Way too many cases where this can go wrong +* so let's just fail silently for now. +*/ cleanup:if (retcode) {
t/t4207-log-decoration-colors.sh +2 -1
@@ -59,7 +59,8 @@ ${c_reset}${c_tag}tag: ${c_reset}${c_tag}v1.0${c_reset}${c_commit}, \ ${c_reset}${c_tag}tag: ${c_reset}${c_tag}B${c_reset}${c_commit})${c_reset} B ${c_commit}COMMIT_ID${c_reset}${c_commit} (${c_reset}\ ${c_tag}tag: ${c_reset}${c_tag}A1${c_reset}${c_commit}, \ -${c_reset}${c_remoteBranch}other/main${c_reset}${c_commit})${c_reset} A1 +${c_reset}${c_remoteBranch}other/main${c_reset}${c_commit}, \ +${c_reset}${c_remoteBranch}other/HEAD${c_reset}${c_commit})${c_reset} A1 ${c_commit}COMMIT_ID${c_reset}${c_commit} (${c_reset}\ ${c_stash}refs/stash${c_reset}${c_commit})${c_reset} On main: Changes to A.t ${c_commit}COMMIT_ID${c_reset}${c_commit} (${c_reset}\
t/t5505-remote.sh +15 -6
@@ -74,7 +74,7 @@ test_expect_success 'add another remote' ' cd test && git remote add -f second ../two && tokens_match "origin second" "$(git remote)" && - check_tracking_branch second main side another && + check_tracking_branch second main side another HEAD && git for-each-ref "--format=%(refname)" refs/remotes | sed -e "/^refs\/remotes\/origin\//d" \ -e "/^refs\/remotes\/second\//d" >actual &&
@@ -476,7 +476,7 @@ test_expect_success 'set-head --auto has no problem w/multiple HEADs' ' cd test && git fetch two "refs/heads/*:refs/remotes/two/*" && git remote set-head --auto two >output 2>&1 && - echo "${SQ}two/HEAD${SQ} is now created and points to ${SQ}main${SQ}" >expect && + echo "${SQ}two/HEAD${SQ} is unchanged and points to ${SQ}main${SQ}" >expect && test_cmp expect output ) '
@@ -764,8 +764,10 @@ test_expect_success 'reject --no-no-tags' ' ' cat >one/expect <<\EOF + apis/HEAD -> apis/main apis/main apis/side + drosophila/HEAD -> drosophila/main drosophila/another drosophila/main drosophila/side
@@ -783,11 +785,14 @@ test_expect_success 'update' ' ' cat >one/expect <<\EOF + drosophila/HEAD -> drosophila/main drosophila/another drosophila/main drosophila/side + manduca/HEAD -> manduca/main manduca/main manduca/side + megaloprepus/HEAD -> megaloprepus/main megaloprepus/main megaloprepus/side EOF
@@ -795,7 +800,7 @@ EOF test_expect_success 'update with arguments' ' ( cd one && - for b in $(git branch -r) + for b in $(git branch -r | grep -v HEAD) do git branch -r -d $b || exit 1 done &&
@@ -827,10 +832,13 @@ test_expect_success 'update --prune' ' ' cat >one/expect <<-\EOF + apis/HEAD -> apis/main apis/main apis/side + manduca/HEAD -> manduca/main manduca/main manduca/side + megaloprepus/HEAD -> megaloprepus/main megaloprepus/main megaloprepus/side EOF
@@ -838,7 +846,7 @@ EOF test_expect_success 'update default' ' ( cd one && - for b in $(git branch -r) + for b in $(git branch -r | grep -v HEAD) do git branch -r -d $b || exit 1 done &&
@@ -850,6 +858,7 @@ test_expect_success 'update default' ' ' cat >one/expect <<\EOF + drosophila/HEAD -> drosophila/main drosophila/another drosophila/main drosophila/side
@@ -858,7 +867,7 @@ EOF test_expect_success 'update default (overridden, with funny whitespace)' ' ( cd one && - for b in $(git branch -r) + for b in $(git branch -r | grep -v HEAD) do git branch -r -d $b || exit 1 done &&
@@ -872,7 +881,7 @@ test_expect_success 'update default (overridden, with funny whitespace)' ' test_expect_success 'update (with remotes.default defined)' ' ( cd one && - for b in $(git branch -r) + for b in $(git branch -r | grep -v HEAD) do git branch -r -d $b || exit 1 done &&
t/t5510-fetch.sh +122 -107
356 lines not on this page · raw message
t/t5512-ls-remote.sh +2 -0
@@ -293,6 +293,8 @@ test_expect_success 'ls-remote with filtered symref (refname)' ' cat >expect <<-EOF && ref: refs/heads/main HEAD $rev HEAD + ref: refs/remotes/origin/main refs/remotes/origin/HEAD + $rev refs/remotes/origin/HEAD EOF git ls-remote --symref . HEAD >actual && test_cmp expect actual
t/t5514-fetch-multiple.sh +15 -2
@@ -45,14 +45,17 @@ test_expect_success setup ' ' cat > test/expect << EOF + one/HEAD -> one/main one/main one/side origin/HEAD -> origin/main origin/main origin/side + three/HEAD -> three/main three/another three/main three/side + two/HEAD -> two/main two/another two/main two/side
@@ -97,6 +100,7 @@ cat > expect << EOF origin/HEAD -> origin/main origin/main origin/side + three/HEAD -> three/main three/another three/main three/side
@@ -112,8 +116,10 @@ test_expect_success 'git fetch --multiple (but only one remote)' ' ' cat > expect << EOF + one/HEAD -> one/main one/main one/side + two/HEAD -> two/main two/another two/main two/side
@@ -141,7 +147,7 @@ test_expect_success 'git fetch --multiple (bad remote names)' ' test_expect_success 'git fetch --all (skipFetchAll)' ' (cd test4 && - for b in $(git branch -r) + for b in $(git branch -r | grep -v HEAD) do git branch -r -d $b || exit 1 done &&
@@ -153,11 +159,14 @@ test_expect_success 'git fetch --all (skipFetchAll)' ' ' cat > expect << EOF + one/HEAD -> one/main one/main one/side + three/HEAD -> three/main three/another three/main three/side + two/HEAD -> two/main two/another two/main two/side
@@ -165,7 +174,7 @@ EOF test_expect_success 'git fetch --multiple (ignoring skipFetchAll)' ' (cd test4 && - for b in $(git branch -r) + for b in $(git branch -r | grep -v HEAD) do git branch -r -d $b || exit 1 done &&
@@ -221,14 +230,17 @@ test_expect_success 'git fetch --multiple --jobs=0 picks a default' ' create_fetch_all_expect () { cat >expect <<-\EOF + one/HEAD -> one/main one/main one/side origin/HEAD -> origin/main origin/main origin/side + three/HEAD -> three/main three/another three/main three/side + two/HEAD -> two/main two/another two/main two/side
@@ -265,6 +277,7 @@ test_expect_success 'git fetch (fetch all remotes with fetch.all = true)' ' create_fetch_one_expect () { cat >expect <<-\EOF + one/HEAD -> one/main one/main one/side origin/HEAD -> origin/main
t/t5516-fetch-push.sh +2 -1
@@ -1395,7 +1395,8 @@ test_expect_success 'fetch follows tags by default' ' git tag -m "annotated" tag && git for-each-ref >tmp1 && sed -n "p; s|refs/heads/main$|refs/remotes/origin/main|p" tmp1 | - sort -k 3 >../expect + sed -n "p; s|refs/heads/main$|refs/remotes/origin/HEAD|p" | + sort -k 4 >../expect ) && test_when_finished "rm -rf dst" && git init dst &&
t/t5527-fetch-odd-refs.sh +2 -1
@@ -52,7 +52,8 @@ test_expect_success LONG_REF 'fetch handles extremely long refname' ' long main EOF - git for-each-ref --format="%(subject)" refs/remotes/long >actual && + git for-each-ref --format="%(subject)" refs/remotes/long \ + --exclude=refs/remotes/long/HEAD >actual && test_cmp expect actual '
t/t7900-maintenance.sh +2 -1
@@ -329,7 +329,8 @@ test_expect_success 'incremental-repack task' ' # Delete refs that have not been repacked in these packs. git for-each-ref --format="delete %(refname)" \ - refs/prefetch refs/tags refs/remotes >refs && + refs/prefetch refs/tags refs/remotes \ + --exclude=refs/remotes/*/HEAD >refs && git update-ref --stdin <refs && # Replace the object directory with this pack layout.
t/t9210-scalar.sh +3 -2
@@ -151,7 +151,8 @@ test_expect_success 'scalar clone' ' "$(pwd)" && git for-each-ref --format="%(refname)" refs/remotes/origin/ >actual && - echo "refs/remotes/origin/parallel" >expect && + echo "refs/remotes/origin/HEAD" >>expect && + echo "refs/remotes/origin/parallel" >>expect && test_cmp expect actual && test_path_is_missing 1/2 &&
@@ -220,7 +221,7 @@ test_expect_success 'scalar reconfigure --all with includeIf.onbranch' ' done ' - test_expect_success 'scalar reconfigure --all with detached HEADs' ' +test_expect_success 'scalar reconfigure --all with detached HEADs' ' repos="two three four" && for num in $repos do
t/t9211-scalar-clone.sh +3 -3
@@ -32,7 +32,7 @@ test_expect_success 'set up repository to clone' ' ) ' -cleanup_clone () { +cleanup_clone() { rm -rf "$1" }
@@ -128,7 +128,7 @@ test_expect_success '--single-branch clones HEAD only' ' ( cd $enlistment/src && git for-each-ref refs/remotes/origin >out && - test_line_count = 1 out && + test_line_count = 2 out && grep "refs/remotes/origin/base" out ) &&
@@ -142,7 +142,7 @@ test_expect_success '--no-single-branch clones all branches' ' ( cd $enlistment/src && git for-each-ref refs/remotes/origin >out && - test_line_count = 2 out && + test_line_count = 3 out && grep "refs/remotes/origin/base" out && grep "refs/remotes/origin/parallel" out ) &&
t/t9902-completion.sh +65 -0
443 lines not on this page · raw message
Signature · 1 line
2.47.0.94.g8861098b6d
PATCH v10 8/8 fetch set_head: handle mirrored bare repositories
In reply to Bence FerdinandyWhen adding a remote to bare repository with "git remote add --mirror", running fetch will fail to update HEAD to the remote's HEAD, since it does not know how to handle bare repositories. On the other hand HEAD already has content, since "git init --bare" has already set HEAD to whatever is the default branch set for the user. Unless this - by chance
- is the same as the remote's HEAD, HEAD will be pointing to a bad symref. Teach set_head to handle bare repositories, by overwriting HEAD so it mirrors the remote's HEAD.
Note, that in this case overriding the local HEAD reference is necessary, since HEAD will exist before fetch can be run, but this should not be an issue, since the whole purpose of --mirror is to be an exact mirror of the remote, so following any changes to HEAD makes sense.
Also note, that although "git remote set-head" also fails when trying to update the remote's locally tracked HEAD in a mirrored bare repository, the usage of the command does not make much sense after this patch:
fetch will update the remote HEAD correctly, and setting it manually to something else is antithetical to the concept of mirroring.
---
Notes:
v10: - new patch
- handles the issue discovered in
https://lore.kernel.org/git/[email protected]/T/
builtin/fetch.c | 16 ++++++++++++---- t/t5505-remote.sh | 10 ++++++++++ 2 files changed, 22 insertions(+), 4 deletions(-)
builtin/fetch.c +12 -4
@@ -1632,14 +1632,22 @@ static int set_head(const struct ref *remote_refs)else head_name =xstrdup (heads.items[0 ].string);if (head_name) { -strbuf_addf (&b_head,"refs/remotes/%s/HEAD" , remote); -strbuf_addf (&b_remote_head,"refs/remotes/%s/%s" , remote, head_name); +int is_bare =is_bare_repository (); +if (is_bare) { +strbuf_addstr (&b_head,"HEAD" ); +strbuf_addf (&b_remote_head,"refs/heads/%s" , head_name); + }else { +strbuf_addf (&b_head,"refs/remotes/%s/HEAD" , remote); +strbuf_addf (&b_remote_head,"refs/remotes/%s/%s" , remote, head_name); + }/* make sure it's valid */ -if (!refs_ref_exists (refs, b_remote_head.buf)) +if (!is_bare && !refs_ref_exists (refs, b_remote_head.buf)) { result =1 ; + }else if (refs_update_symref_extended (refs, b_head.buf, b_remote_head.buf, -"fetch" , &b_local_head,1 )) +"fetch" , &b_local_head, !is_bare)) { result =1 ; + }else report_set_head (remote, head_name, &b_local_head);
t/t5505-remote.sh +10 -0
@@ -545,6 +545,16 @@ test_expect_success 'add --mirror && prune' ' ) ' +test_expect_success 'add --mirror setting HEAD' ' + mkdir headmirror && + ( + cd headmirror && + git init --bare -b notmain && + git remote add --mirror -f origin ../one && + test "$(git symbolic-ref HEAD)" = "refs/heads/main" + ) +' + test_expect_success 'add --mirror=fetch' ' mkdir mirror-fetch && git init -b main mirror-fetch/parent &&
Signature · 1 line
2.47.0.94.g8861098b6d
Quoted 9 lines · Bence Ferdinandy
Sorry for sending a v10 so soon after v9 without really waiting for feedback, but as I started digging into
https://lore.kernel.org/git/[email protected]/ I realized that the root cause is in fetch and not remote add, so the solution belongs to this series. I'll leave my overall comments for v9 in here for clarity.v10: added a new patch (8/8) on top of v9 to handle HEAD in mirrored bare repositories, rest of the patches are unchanged
Thanks. For other readers, this is a continuation of the v9 from this thread:
https://lore.kernel.org/git/[email protected]/
Thanks,
Taylor
2024. okt. 21. 22:43:38 Taylor Blau <[email protected]>:
Quoted 8 lines · Bence Ferdinandy
Quoted 7 lines
Sorry for sending a v10 so soon after v9 without really waiting for feedback, but as I started digging into
https://lore.kernel.org/git/[email protected]/ I realized that the root cause is in fetch and not remote add, so the solution belongs to this series. I'll leave my overall comments for v9 in here for clarity.
Quoted 7 lines
Quoted 2 lines
v10: added a new patch (8/8) on top of v9 to handle HEAD in mirrored bare repositories, rest of the patches are unchanged
Thanks. For other readers, this is a continuation of the v9 from this thread:
And people may also want to drop the accidental noreply address in Cc when replying :) Sorry for the mixup!
Quoted 9 lines · Bence Ferdinandy
Bence Ferdinandy (8):
t/t5505-remote: set default branch to main
refs: atomically record overwritten ref inupdate_symrefremote set-head: refactor for readability
remote set-head: better output for --auto
refs: addTRANSACTION_CREATE_EXISTSerror
refs: addcreate_onlyoption torefs_update_symref_extended
fetch: set remote/HEAD if it does not exist
fetchset_head: handle mirrored bare repositories
After applying this topic to 'seen', I noticed some new CI breakage that appears to be caused by this topic:
https://github.com/git/git/actions/runs/11449483611/job/31855171514#step:4:2506
After dropping the topic locally and building with 'make SANITIZE=leak' and then running t0410-partial-clone.sh, I was able to resolve the failures.
Would you mind building with 'SANITIZE=leak' and running that script to see if you can address the issue? Thanks.
Thanks,
Taylor
Quoted 14 lines · Taylor Blau
Quoted 9 lines · Bence Ferdinandy
Bence Ferdinandy (8):
t/t5505-remote: set default branch to main
refs: atomically record overwritten ref inupdate_symrefremote set-head: refactor for readability
remote set-head: better output for --auto
refs: addTRANSACTION_CREATE_EXISTSerror
refs: addcreate_onlyoption torefs_update_symref_extended
fetch: set remote/HEAD if it does not exist
fetchset_head: handle mirrored bare repositoriesAfter applying this topic to 'seen', I noticed some new CI breakage that appears to be caused by this topic:
Quoted 8 lines
https://github.com/git/git/actions/runs/11449483611/job/31855171514#step:4:2506After dropping the topic locally and building with 'make SANITIZE=leak' and then running
t0410-partial-clone.sh, I was able to resolve the failures.Would you mind building with 'SANITIZE=leak' and running that script to see if you can address the issue? Thanks.
Thanks, I can reproduce the issue. I'm hoping there are no more tricks to testing things :D
Let's see how fast I can handle this ...
Best,
Bence
Signature · 1 line
bence.ferdinandy.com
Quoted 22 lines · Bence Ferdinandy
Quoted 20 lines · Taylor Blau
Quoted 9 lines · Bence Ferdinandy
Bence Ferdinandy (8):
t/t5505-remote: set default branch to main
refs: atomically record overwritten ref inupdate_symrefremote set-head: refactor for readability
remote set-head: better output for --auto
refs: addTRANSACTION_CREATE_EXISTSerror
refs: addcreate_onlyoption torefs_update_symref_extended
fetch: set remote/HEAD if it does not exist
fetchset_head: handle mirrored bare repositoriesAfter applying this topic to 'seen', I noticed some new CI breakage that appears to be caused by this topic:
https://github.com/git/git/actions/runs/11449483611/job/31855171514#step:4:2506After dropping the topic locally and building with 'make SANITIZE=leak' and then running
t0410-partial-clone.sh, I was able to resolve the failures.
Quoted 5 lines
Quoted 2 lines
Would you mind building with 'SANITIZE=leak' and running that script to see if you can address the issue? Thanks.
Thanks, I can reproduce the issue. I'm hoping there are no more tricks to testing things :D
Great, I'm glad that it was easily reproducible on your end. This one should have minimal differences from the setup in CI, as I think building with 'SANITIZE=leak' will suffice.
Quoted 1 line
Let's see how fast I can handle this ...
Thanks for looking.
Thanks,
Taylor
Quoted 29 lines · Taylor Blau
Quoted 22 lines · Bence Ferdinandy
Quoted 20 lines · Taylor Blau
Quoted 9 lines · Bence Ferdinandy
Bence Ferdinandy (8):
t/t5505-remote: set default branch to main
refs: atomically record overwritten ref inupdate_symrefremote set-head: refactor for readability
remote set-head: better output for --auto
refs: addTRANSACTION_CREATE_EXISTSerror
refs: addcreate_onlyoption torefs_update_symref_extended
fetch: set remote/HEAD if it does not exist
fetchset_head: handle mirrored bare repositoriesAfter applying this topic to 'seen', I noticed some new CI breakage that appears to be caused by this topic:
https://github.com/git/git/actions/runs/11449483611/job/31855171514#step:4:2506After dropping the topic locally and building with 'make SANITIZE=leak' and then running
t0410-partial-clone.sh, I was able to resolve the failures.Quoted 5 lines
Quoted 2 lines
Would you mind building with 'SANITIZE=leak' and running that script to see if you can address the issue? Thanks.
Thanks, I can reproduce the issue. I'm hoping there are no more tricks to testing things :D
Quoted 7 lines
Great, I'm glad that it was easily reproducible on your end. This one should have minimal differences from the setup in CI, as I think building with 'SANITIZE=leak' will suffice.
Quoted 1 line
Let's see how fast I can handle this ...
Thanks for looking.
I'll send v11 to the correct thread, hopefully with no random companies in Cc :)