Bugfix for partial clones and ref-in-want servers
cover Bugfix for partial clones and ref-in-want servers
A coworker discovered a bug when having a partial clone against a server that supports ref-in-want - the lazy fetches no longer work, because the client attempts to fetch with "want-ref <hash>", which is not permitted by the protocol.
The first patch deduplicates some code, so that the bugfix need only be applied once, and the second patch contains the actual bugfix.
Jonathan Tan (2):
fetch-object: provide only one fetching function
fetch-object: set exact_oid when fetching
fetch-object.c | 17 ++++++----------- fetch-object.h | 8 ++------ sha1-file.c | 2 +- t/t0410-partial-clone.sh | 12 ++++++++++++ unpack-trees.c | 2 +- 5 files changed, 22 insertions(+), 19 deletions(-)
Signature · 1 line
2.19.0.397.gdd90340f6a-goog
PATCH 1/2 fetch-object: provide only one fetching function
fetch-object.h currently provides two functions (fetch_object() and fetch_objects()) that could be replaced by a single, more flexible function. Replace those two functions with the more flexible function.
--- fetch-object.c | 16 +++++----------- fetch-object.h | 8 ++------ sha1-file.c | 2 +- unpack-trees.c | 2 +- 4 files changed, 9 insertions(+), 19 deletions(-)
fetch-object.c +5 -11
@@ -23,21 +23,15 @@ static void fetch_refs(const char *remote_name, struct ref *ref) fetch_if_missing = original_fetch_if_missing; } -void fetch_object (const char *remote_name,const unsigned char *sha1) -{ -struct ref *ref =alloc_ref (sha1_to_hex (sha1)); -hashcpy (ref->old_oid.hash, sha1); -fetch_refs (remote_name, ref); -} - -void fetch_objects (const char *remote_name,const struct oid_array *to_fetch) +void fetch_objects (const char *remote_name,const struct object_id *oids, +int oid_nr) {struct ref *ref =NULL ;int i; -for (i =0 ; i < to_fetch->nr; i++) { -struct ref *new_ref =alloc_ref (oid_to_hex (&to_fetch->oid[i])); -oidcpy (&new_ref->old_oid, &to_fetch->oid[i]); +for (i =0 ; i < oid_nr; i++) { +struct ref *new_ref =alloc_ref (oid_to_hex (&oids[i])); +oidcpy (&new_ref->old_oid, &oids[i]); new_ref->next = ref; ref = new_ref; }
fetch-object.h +2 -6
@@ -1,11 +1,7 @@#ifndef FETCH_OBJECT_H #define FETCH_OBJECT_H -#include "sha1-array.h" - -extern void fetch_object (const char *remote_name,const unsigned char *sha1); - -extern void fetch_objects (const char *remote_name, -const struct oid_array *to_fetch); +void fetch_objects (const char *remote_name,const struct object_id *oids, +int oid_nr);#endif
sha1-file.c +1 -1
@@ -1317,7 +1317,7 @@ int oid_object_info_extended(struct repository *r, const struct object_id *oid,* TODO Pass a repository struct through fetch_object, * such that arbitrary repositories work. */ -fetch_object (repository_format_partial_clone, real->hash); +fetch_objects (repository_format_partial_clone, real,1 ); already_retried =1 ;continue ; }
unpack-trees.c +1 -1
@@ -392,7 +392,7 @@ static int check_updates(struct unpack_trees_options *o) }if (to_fetch.nr)fetch_objects (repository_format_partial_clone, - &to_fetch); + to_fetch.oid, to_fetch.nr); fetch_if_missing = fetch_if_missing_store;oid_array_clear (&to_fetch); }
Signature · 1 line
2.19.0.397.gdd90340f6a-goog
Re: [PATCH 1/2] fetch-object: provide only one fetching function
Quoted 3 lines · Jonathan Tan
fetch-object.hcurrently provides two functions (fetch_object()andfetch_objects()) that could be replaced by a single, more flexible function. Replace those two functions with the more flexible function.
The latter half of the first sentence and the second sentence are pretty-much repeating the same thing.
I think you wanted to justify two changes:
(1) There are two helpers. fetch_object() fetches a single object from a given remote; fetch_objects() fetches a set of objects from a given remote. It is not like the former is implemented as a specialization of the latter (i.e. passing length=1 array), and it is not like the former is optimized specially for a single object fetch that cannot be made into such a thin wrapper. It is not like the latter is implemented as a repeated call to the former in a loop, either. There is no justification to keep two independent implementations, and using a single helper that can be used by all the callers of these two would make sense.
(2) The variant that fetches multiple objects takes oid_array. To be used by all the existing callers of these two helpers, it is better to use a different calling convention, namely, a array[] and its length passed as separate parameters.
Instead of explaining why the new convention is better to justify (2), the above three lines handwave by saying "more flexible" twice. We should do better.
fetch-object: unify fetch_object[s] functions
There are fetch_object() and fetch_objects() helpers in fetch-object.h; as the latter takes "struct oid_array", the former cannot be made into a thin wrapper around the latter without an extra allocation and set-up cost.
Update fetch_objects() to take an array of "struct object_id" and number of elements in it as separate parameters, remove fetch_object(), and adjust all existing callers of these functions to use the new fetch_objects().
perhaps?
Quoted 18 lines
sha1-file.c +1
-1@@ -1317,7 +1317,7 @@ int oid_object_info_extended(struct repository *r, const struct object_id *oid,* TODO Pass a repository struct through fetch_object, * such that arbitrary repositories work. */ -fetch_object (repository_format_partial_clone, real->hash); +fetch_objects (repository_format_partial_clone, real,1 ); already_retried =1 ;continue ; }unpack-trees.c +0
-0
unpack-trees.c static int check_updates(struct unpack_trees_options *o) · line 398
Quoted 8 lines
unpack-trees.c +1
-1@@ -392,7 +392,7 @@ static int check_updates(struct unpack_trees_options *o) }if (to_fetch.nr)fetch_objects (repository_format_partial_clone, - &to_fetch); + to_fetch.oid, to_fetch.nr); fetch_if_missing = fetch_if_missing_store;oid_array_clear (&to_fetch); }
Changes to these two callers do explain why <ptr, nr> is an interface that is easier to use. Those who already have an oid_array can pass its fields as separate parameters without too much trouble, and those who only have an oid (or an array of oid and knows how many are in the array) do not have to wrap them into an oid_array.
Thanks.
Re: [PATCH 1/2] fetch-object: provide only one fetching function
Quoted 11 lines
Instead of explaining why the new convention is better to justify (2), the above three lines handwave by saying "more flexible" twice. We should do better.
fetch-object: unify fetch_object[s] functions
There are
fetch_object()andfetch_objects()helpers infetch-object.h; as the latter takes "structoid_array", the former cannot be made into a thin wrapper around the latter without an extra allocation and set-up cost.
Quoted 6 lines
Update
fetch_objects()to take an array of "structobject_id" and number of elements in it as separate parameters, removefetch_object(), and adjust all existing callers of these functions to use the newfetch_objects().perhaps?
Thanks - your explanation is much clearer than mine. Let me know if you want a reroll (or if you can update the commit message yourself, that's fine too).
Re: [PATCH 1/2] fetch-object: provide only one fetching function
Quoted 16 lines · Jonathan Tan
Quoted 16 lines
Instead of explaining why the new convention is better to justify (2), the above three lines handwave by saying "more flexible" twice. We should do better.
fetch-object: unify fetch_object[s] functions
There are
fetch_object()andfetch_objects()helpers infetch-object.h; as the latter takes "structoid_array", the former cannot be made into a thin wrapper around the latter without an extra allocation and set-up cost.Update
fetch_objects()to take an array of "structobject_id" and number of elements in it as separate parameters, removefetch_object(), and adjust all existing callers of these functions to use the newfetch_objects().
Quoted 5 lines
Quoted 1 line
perhaps?
Thanks - your explanation is much clearer than mine. Let me know if you want a reroll (or if you can update the commit message yourself, that's fine too).
If there is no other change needed for either of the patches, I do not mind rewording the 1/2 myself to save a round-trip.
Thanks.
PATCH 2/2 fetch-object: set exact_oid when fetching
In reply to Jonathan Tanfetch_objects() currently does not set exact_oid in struct ref when invoking transport_fetch_refs(). If the server supports ref-in-want, fetch_pack() uses this field to determine whether a wanted ref should be requested as a "want-ref" line or a "want" line; without the setting of exact_oid, the wrong line will be sent.
Set exact_oid, so that the correct line is sent.
--- fetch-object.c | 1 + t/t0410-partial-clone.sh | 12 ++++++++++++ 2 files changed, 13 insertions(+)
fetch-object.c +1 -0
@@ -32,6 +32,7 @@ void fetch_objects(const char *remote_name, const struct object_id *oids,for (i =0 ; i < oid_nr; i++) {struct ref *new_ref =alloc_ref (oid_to_hex (&oids[i]));oidcpy (&new_ref->old_oid, &oids[i]); + new_ref->exact_oid =1 ; new_ref->next = ref; ref = new_ref; }
t/t0410-partial-clone.sh +12 -0
@@ -170,6 +170,18 @@ test_expect_success 'fetching of missing objects' ' git verify-pack --verbose "$IDX" | grep "$HASH" ' +test_expect_success 'fetching of missing objects works with ref-in-want enabled' ' + # ref-in-want requires protocol version 2 + git -C server config protocol.version 2 && + git -C server config uploadpack.allowrefinwant 1 && + git -C repo config protocol.version 2 && + + rm -rf repo/.git/objects/* && + rm -f trace && + GIT_TRACE_PACKET="$(pwd)/trace" git -C repo cat-file -p "$HASH" && + grep "git< fetch=.*ref-in-want" trace +' + test_expect_success 'rev-list stops traversal at missing and promised commit' ' rm -rf repo && test_create_repo repo &&
Signature · 1 line
2.19.0.397.gdd90340f6a-goog