More add_submodule_odb() cleanup in merge code
cover More add_submodule_odb() cleanup in merge code
(CC-ing Elijah in case he has insight into the merge-ort part.)
While continuing the effort into removing add_submodule_odb() (as part of the submodule partial clone effort) I came across this part of the merge code that specifies the repository being operated on in two ways:
one as a struct repository pointer and the other as a path. This patch set unifies the two.
I normally would not send refactoring patches unless I also have a feature patch that uses the results of said refactoring, but in this case, I think that these patches are worth having since they clarify a potentially unclear part of the API.
Note that these patches mean that the merging code no longer supports submodules that have their .git dirs in the worktree, but from what I can tell, this seems to be the direction we're going in (repo_submodule_init() does not support such submodules).
Patch 3 is included to show how I'm verifying some things. Including something like that in the master branch would probably require conditional compilation (to exclude the additional field in struct object used for checking, among other things), so I'm just including it here for informational purposes.
All these patches work under GIT_TEST_MERGE_ALGORITHM=recursive and GIT_TEST_MERGE_ALGORITHM=ort (and when that envvar is unset, for good measure).
Jonathan Tan (3):
t6437: run absorbgitdirs on repos
revision: remove "submodule" from opt struct DO NOT SUBMIT commit-reach,revision: verify non-mixing
alloc.c | 2 + commit-reach.c | 60 +++++++++++++++++------- merge-ort.c | 55 +++++++++++++++------- merge-recursive.c | 51 +++++++++++++------- object.h | 1 + revision.c | 96 ++++++++++++++++++++++---------------- revision.h | 1 - t/t6437-submodule-merge.sh | 9 ++-- 8 files changed, 179 insertions(+), 96 deletions(-)
Signature · 1 line
2.33.0.309.g3052b89438-goog
PATCH 1/3 t6437: run absorbgitdirs on repos
The submodule merge code is being transitioned away from add_submodule_odb() to repo_submodule_init(), and the latter does not support submodules that have their .git directories in the worktree (instead of in .git/modules). Migrate the test code by calling absorbgitdirs wherever necessary to place the .git directories of submodules in .git/modules of the superproject.
--- t/t6437-submodule-merge.sh | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-)
t/t6437-submodule-merge.sh +6 -3
@@ -27,7 +27,8 @@ test_expect_success setup ' git add file && test_tick && git commit -m sub-root) && - git add sub && + git submodule add ./sub sub && + git submodule absorbgitdirs && test_tick && git commit -m root &&
@@ -82,7 +83,8 @@ test_expect_success 'setup for merge search' ' git branch sub-a) && git commit --allow-empty -m init && git branch init && - git add sub && + git submodule add ./sub sub && + git submodule absorbgitdirs && git commit -m "a" && git branch a &&
@@ -112,7 +114,8 @@ test_expect_success 'setup for merge search' ' git checkout -b g init && (cd sub && git checkout -b sub-g sub-c) && - git add sub && + git submodule add ./sub sub && + git submodule absorbgitdirs && git commit -a -m "g") '
Signature · 1 line
2.33.0.309.g3052b89438-goog
PATCH 2/3 revision: remove "submodule" from opt struct
In reply to Jonathan TanClean up a TODO in revision.h by removing the "submodule" field from struct setup_revision_opt. This field is only used to specify the ref store to use, so use rev_info->repo to determine the ref store instead.
The only users of this field are merge-ort.c and merge-recursive.c. However, both these files specify the superproject as rev_info->repo and the submodule as setup_revision_opt->submodule. In order to be able to pass the submodule as rev_info->repo, all commits must be parsed with the submodule explicitly specified; this patch does that as well. (An incremental solution in which only some commits are parsed with explicit submodule will not work, because if the same commit is parsed twice in different repositories, there will be 2 heap-allocated object structs corresponding to that commit, and any flag set by the revision walking mechanism on one of them will not be reflected onto the other.)
--- merge-ort.c | 55 +++++++++++++++++++++++++++++++---------------- merge-recursive.c | 51 ++++++++++++++++++++++++++++--------------- revision.c | 16 +++++--------- revision.h | 1 - 4 files changed, 76 insertions(+), 47 deletions(-)
merge-ort.c +37 -18
@@ -32,6 +32,7 @@#include "promisor-remote.h" #include "revision.h" #include "strmap.h" +#include "submodule-config.h" #include "submodule.h" #include "tree.h" #include "unpack-trees.h"
@@ -1511,7 +1512,6 @@ static int find_first_merges(struct repository *repo,xsnprintf (merged_revision,sizeof (merged_revision),"^%s" ,oid_to_hex (&a->object.oid));repo_init_revisions (repo, &revs,NULL ); - rev_opts.submodule = path;/* FIXME: can't handle linked worktrees in submodules yet */ revs.single_worktree = path !=NULL ;setup_revisions (ARRAY_SIZE (rev_args)-1 , rev_args, &revs, &rev_opts);
@@ -1521,7 +1521,7 @@ static int find_first_merges(struct repository *repo,die ("revision walk setup failed" );while ((commit =get_revision (&revs)) !=NULL ) {struct object *o = &(commit->object); -if (in_merge_bases (b, commit)) +if (repo_in_merge_bases (repo, b, commit))add_object_array (o,NULL , &merges); }reset_revision_walk ();
@@ -1536,7 +1536,7 @@ static int find_first_merges(struct repository *repo, contains_another =0 ;for (j =0 ; j < merges.nr; j++) {struct commit *m2 = (struct commit *) merges.objects[j].item; -if (i != j &&in_merge_bases (m2, m1)) { +if (i != j &&repo_in_merge_bases (repo, m2, m1)) { contains_another =1 ;break ; }
@@ -1557,10 +1557,13 @@ static int merge_submodule(struct merge_options *opt,const struct object_id *b, struct object_id *result) { + const struct submodule *submodule; +struct repository subrepo; +struct strbuf sb =STRBUF_INIT ; +int ret =0 ;struct commit *commit_o, *commit_a, *commit_b;int parent_count;struct object_array merges; -struct strbuf sb =STRBUF_INIT ;int i;int search = !opt->priv->call_depth;
@@ -1576,6 +1579,10 @@ static int merge_submodule(struct merge_options *opt,if (is_null_oid (b))return 0 ; +/* +* NEEDSWORK: Remove this when all submodule object accesses are +* through explicitly specified repositores. +*/ if (add_submodule_odb (path)) { path_msg(opt, path, 0, _("Failed to merge submodule %s (not checked out)"),
@@ -1583,39 +1590,49 @@ static int merge_submodule(struct merge_options *opt,return 0 ; } -if (!(commit_o =lookup_commit_reference (opt->repo, o)) || - !(commit_a =lookup_commit_reference (opt->repo, a)) || - !(commit_b =lookup_commit_reference (opt->repo, b))) { + submodule =submodule_from_path (opt->repo,null_oid (), path); +if (repo_submodule_init (&subrepo, opt->repo, submodule)) { +path_msg (opt, path,0 , +_ ("Failed to merge submodule %s (not checked out)" ), + path); +return 0 ; + } + +if (!(commit_o =lookup_commit_reference (&subrepo, o)) || + !(commit_a =lookup_commit_reference (&subrepo, a)) || + !(commit_b =lookup_commit_reference (&subrepo, b))) {path_msg (opt, path,0 ,_ ("Failed to merge submodule %s (commits not present)" ), path); -return 0 ; + goto cleanup; }/* check whether both changes are forward */ -if (!in_merge_bases (commit_o, commit_a) || - !in_merge_bases (commit_o, commit_b)) { +if (!repo_in_merge_bases (&subrepo, commit_o, commit_a) || + !repo_in_merge_bases (&subrepo, commit_o, commit_b)) {path_msg (opt, path,0 ,_ ("Failed to merge submodule %s " "(commits don't follow merge-base)" ), path); -return 0 ; + goto cleanup; }/* Case #1: a is contained in b or vice versa */ -if (in_merge_bases (commit_a, commit_b)) { +if (repo_in_merge_bases (&subrepo, commit_a, commit_b)) {oidcpy (result, b);path_msg (opt, path,1 ,_ ("Note: Fast-forwarding submodule %s to %s" ), path,oid_to_hex (b)); -return 1 ; + ret =1 ; + goto cleanup; } -if (in_merge_bases (commit_b, commit_a)) { +if (repo_in_merge_bases (&subrepo, commit_b, commit_a)) {oidcpy (result, a);path_msg (opt, path,1 ,_ ("Note: Fast-forwarding submodule %s to %s" ), path,oid_to_hex (a)); -return 1 ; + ret =1 ; + goto cleanup; }/*
@@ -1627,10 +1644,10 @@ static int merge_submodule(struct merge_options *opt,/* Skip the search if makes no sense to the calling context. */ if (!search) -return 0 ; + goto cleanup;/* find commit which merges them */ - parent_count =find_first_merges (opt->repo, path, commit_a, commit_b, + parent_count =find_first_merges (&subrepo, path, commit_a, commit_b, &merges);switch (parent_count) {case 0 :
@@ -1664,7 +1681,9 @@ static int merge_submodule(struct merge_options *opt, }object_array_clear (&merges); -return 0 ; +cleanup: +repo_clear (&subrepo); +return ret; }static void initialize_attr_index (struct merge_options *opt)
merge-recursive.c +34 -17
127 lines not on this page · raw message
revision.c +5 -11
@@ -2563,8 +2563,7 @@ static int for_each_good_bisect_ref(struct ref_store *refs, each_ref_fn fn, voidreturn for_each_bisect_ref (refs, fn, cb_data, term_good); } -static int handle_revision_pseudo_opt (const char *submodule, -struct rev_info *revs, +static int handle_revision_pseudo_opt (struct rev_info *revs,const char **argv,int *flags) {const char *arg = argv[0 ];
@@ -2572,7 +2571,7 @@ static int handle_revision_pseudo_opt(const char *submodule,struct ref_store *refs;int argcount; -if (submodule) { +if (revs->repo != the_repository) {/* * We need some something like get_submodule_worktrees() * before we can go through all worktrees of a submodule,
@@ -2581,9 +2580,8 @@ static int handle_revision_pseudo_opt(const char *submodule,*/ if (!revs->single_worktree)BUG ("--single-worktree cannot be used together with submodule" ); - refs =get_submodule_ref_store (submodule); - }else - refs =get_main_ref_store (revs->repo); + } + refs =get_main_ref_store (revs->repo);/* * NOTE!
@@ -2707,12 +2705,8 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s {int i, flags, left, seen_dashdash, revarg_opt;struct strvec prune_data =STRVEC_INIT ; -const char *submodule =NULL ;int seen_end_of_options =0 ; -if (opt) - submodule = opt->submodule; -/* First, search for "--" */ if (opt && opt->assume_dashdash) { seen_dashdash =1 ;
@@ -2741,7 +2735,7 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct sif (!seen_end_of_options && *arg =='-' ) {int opts; - opts =handle_revision_pseudo_opt (submodule, + opts =handle_revision_pseudo_opt ( revs, argv + i, &flags);if (opts >0 ) {
revision.h +0 -1
@@ -336,7 +336,6 @@ extern volatile show_early_output_fn_t show_early_output;struct setup_revision_opt {const char *def;void (*tweak)(struct rev_info *,struct setup_revision_opt *); -const char *submodule;/* TODO: drop this and use rev_info->repo */ unsigned int assume_dashdash:1 , allow_exclude_promisor_objects:1 ;unsigned revarg_opt;
Signature · 1 line
2.33.0.309.g3052b89438-goog
PATCH 3/3 DO NOT SUBMIT commit-reach,revision: verify non-mixing
In reply to Jonathan TanThis patch is provided to show reviewers how I'm verifying that there is no mixing of object structs in the revision walking mechanisms. I've intercepted any creation of commit nodes in alloc.c (whether through alloc_commit_node() directly or through alloc_object_node()) to add the repository in which that node is allocated, and checked all additions of commits to data structures (as far as I can tell) in commit-reach.c and revision.c to make sure that the repository in the added objects is what we expect. This ensures that any flags set are set on the correct nodes (for example, it won't be set on an object allocated in the_repository and then checked from an object allocated in a submodule).
I'll leave out this patch in subsequent revisions of this patch set.
--- alloc.c | 2 ++ commit-reach.c | 60 +++++++++++++++++++++++++------------ object.h | 1 + revision.c | 80 ++++++++++++++++++++++++++++++++------------------ 4 files changed, 97 insertions(+), 46 deletions(-)
alloc.c +2 -0
@@ -96,6 +96,7 @@ void *alloc_object_node(struct repository *r) {struct object *obj =alloc_node (r->parsed_objects->object_state,sizeof (union any_object )); obj->type =OBJ_NONE ; + obj->repo = r;return obj; }
@@ -120,6 +121,7 @@ void *alloc_commit_node(struct repository *r) {struct commit *c =alloc_node (r->parsed_objects->commit_state,sizeof (struct commit ));init_commit_node (c); + c->object.repo = r;return c; }
commit-reach.c +42 -18
171 lines not on this page · raw message
object.h +1 -0
@@ -90,6 +90,7 @@ struct object { unsigned type : TYPE_BITS; unsigned flags : FLAG_BITS;struct object_id oid; +struct repository *repo; };const char *type_name (unsigned int type);
revision.c +52 -28
235 lines not on this page · raw message
Signature · 1 line
2.33.0.309.g3052b89438-goog
Re: [PATCH 1/3] t6437: run absorbgitdirs on repos
In reply to Jonathan TanQuoted 6 lines
The submodule merge code is being transitioned away from
add_submodule_odb()torepo_submodule_init(), and the latter does not support submodules that have their .git directories in the worktree (instead of in .git/modules). Migrate the test code by calling absorbgitdirs wherever necessary to place the .git directories of submodules in .git/modules of the superproject.
Upon further work, there seems to be quite a lot of test code to migrate, and it might be easier instead to add support for unabsorbed submodules to repo_submodule_init() (say, following what open_submodule() in submodule.c does). Feel free to comment on this or other aspects of the patch set, and in the meantime, I'll investigate that.
Re: [PATCH 1/3] t6437: run absorbgitdirs on repos
In reply to Jonathan TanQuoted 6 lines · Jonathan Tan
The submodule merge code is being transitioned away from
add_submodule_odb()torepo_submodule_init(), and the latter does not support submodules that have their .git directories in the worktree (instead of in .git/modules). Migrate the test code by calling absorbgitdirs wherever necessary to place the .git directories of submodules in .git/modules of the superproject.
It is wonderful that we can work with the "absorbed" form of the submodule repositories, and having all the submodules in "absorbed" form may be ideal in the longer term, but I fail to convince myself that this particular change to test script is a good idea.
After all, if you are not cloning from somebody else but creating a new submodule in a superproject you have (whether that was locally created or cloned from elsewhere), wouldn't it more natural to have a submodule that is not "absorbed" there? Is it easy to create an "absorbed" submodule locally?
The change to tests in this patch may make things work with the repo_submodule_init() that is incapable of work with submodules with in-worktree .git repository. In theory, the users can do the same, i.e. have an extra "git submodule absorbgitdirs" call, when adding a new submodule, to make things work with repo_submodule_init(), too.
But that sounds like a workaround, not a feature.
Quite honestly, if repo_submodule_init() can only work with the "absorbed" form, isn't it simply a bug?
Two independent (i.e. we can do either or both) things may improve the situation:
- allow repo_submodule_init() to work on both forms (in place---no cheating by calling "absorb" behind user's back).
- teach "git submodule add" to transition an old-style submodule into the "absorbed" form (either with an option, or do so by default with an escape hatch to disable).
The latter indirectly attacks the "repo_submodule_init() can only work with absorbed form" issue by making it harder to create a non-absorbed submodule to begin with.
Hmm?
Quoted 2 lines · Jonathan Tan
(CC-ing Elijah in case he has insight into the merge-ort part.)
All the submodule merging related functions were lifted from merge-recursive.c and minimally modified to fit the new structure. The only substantive change I made was to fix the merge result for virtual merge bases, but that's like one or two lines of code. In particular, everything relating to submodule objects was totally untouched...and I think that's reflected in the fact that your PATCH 2 basically is the same patch twice, once for merge-recursive and once for merge-ort.
I read over PATCH 2 and I didn't find anything that looked problematic, but I'm not up-to-speed on the add_submodule_odb and repo handling bits of the codebase so I'm not sure I would catch anything. But I am encouraged by the fact that it looks like you did the same stuff to merge-recursive and merge-ort; I'd be worried you missed something if that weren't the case.
As a sidenote, though...
This does remind me that I noticed that the following functions from object-store.h do not take an explicit repository:
write_object_file()hash_object_file()hash_object_file_literally()force_object_loose()
I have a patch sitting around somewhere (possibly only still referenced in my 'temp' branch) to make repo_*() variants of the above functions, and turn the above into simple wrappers of the repo_*() variants which just pass the_repository (much like someone else did with read_object_file() and repo_read_object_file()). It also updates merge-ort to use the new repo_*() functions. However, I ended up excluding it from my merge-ort submissions since it wasn't necessary. Would this be of interest in your submodule work, though? I guess it'd only matter if we started doing real merges of submodules as part of a parent repo merge. (As opposed to the fast-forward-only merges that merge-recursive/merge-ort do right now for submodules.)
Quoted 44 lines
While continuing the effort into removing
add_submodule_odb()(as part of the submodule partial clone effort) I came across this part of the merge code that specifies the repository being operated on in two ways:
one as a struct repository pointer and the other as a path. This patch set unifies the two.I normally would not send refactoring patches unless I also have a feature patch that uses the results of said refactoring, but in this case, I think that these patches are worth having since they clarify a potentially unclear part of the API.
Note that these patches mean that the merging code no longer supports submodules that have their .git dirs in the worktree, but from what I can tell, this seems to be the direction we're going in (
repo_submodule_init()does not support such submodules).Patch 3 is included to show how I'm verifying some things. Including something like that in the master branch would probably require conditional compilation (to exclude the additional field in struct object used for checking, among other things), so I'm just including it here for informational purposes.
All these patches work under GIT_TEST_MERGE_ALGORITHM=recursive and GIT_TEST_MERGE_ALGORITHM=ort (and when that envvar is unset, for good measure).
Jonathan Tan (3):
t6437: run absorbgitdirs on repos
revision: remove "submodule" from opt struct DO NOT SUBMIT commit-reach,revision: verify non-mixingalloc.c | 2 + commit-reach.c | 60 +++++++++++++++++------- merge-ort.c | 55 +++++++++++++++------- merge-recursive.c | 51 +++++++++++++------- object.h | 1 + revision.c | 96 ++++++++++++++++++++++---------------- revision.h | 1 - t/t6437-submodule-merge.sh | 9 ++-- 8 files changed, 179 insertions(+), 96 deletions(-)--
2.33.0.309.g3052b89438-goog
Re: [PATCH 1/3] t6437: run absorbgitdirs on repos
In reply to Junio C HamanoQuoted 9 lines
Quite honestly, if
repo_submodule_init()can only work with the "absorbed" form, isn't it simply a bug?Two independent (i.e. we can do either or both) things may improve the situation:
- allow
repo_submodule_init()to work on both forms (in place---no cheating by calling "absorb" behind user's back).
Quoted 7 lines
- teach "git submodule add" to transition an old-style submodule into the "absorbed" form (either with an option, or do so by default with an escape hatch to disable).
The latter indirectly attacks the "
repo_submodule_init()can only work with absorbed form" issue by making it harder to create a non-absorbed submodule to begin with.
Thanks for taking a look.
I took a further look and it doesn't look too difficult to teach repo_submodule_init() to work on both forms (without calling "absorbgitdirs") - I'll send an updated patch set soon. As for "git submodule add" transitioning it into the absorbed form, I'll leave that for someone else to do - although if repo_submodule_init() supports the unabsorbed form now, that is not so urgent to do, I think.
Quoted 13 lines · Jonathan Tan
Quoted 2 lines
(CC-ing Elijah in case he has insight into the merge-ort part.)
All the submodule merging related functions were lifted from
merge-recursive.cand minimally modified to fit the new structure. The only substantive change I made was to fix the merge result for virtual merge bases, but that's like one or two lines of code. In particular, everything relating to submodule objects was totally untouched...and I think that's reflected in the fact that your PATCH 2 basically is the same patch twice, once for merge-recursive and once for merge-ort.
Quoted 6 lines
I read over PATCH 2 and I didn't find anything that looked problematic, but I'm not up-to-speed on the
add_submodule_odband repo handling bits of the codebase so I'm not sure I would catch anything. But I am encouraged by the fact that it looks like you did the same stuff to merge-recursive and merge-ort; I'd be worried you missed something if that weren't the case.
Thanks for taking a look.
Quoted 10 lines
As a sidenote, though...
This does remind me that I noticed that the following functions from
object-store.hdo not take an explicit repository:
write_object_file()hash_object_file()hash_object_file_literally()force_object_loose()
Quoted 11 lines
I have a patch sitting around somewhere (possibly only still referenced in my 'temp' branch) to make repo_*() variants of the above functions, and turn the above into simple wrappers of the repo_*() variants which just pass
the_repository(much like someone else did withread_object_file()andrepo_read_object_file()). It also updates merge-ort to use the new repo_*() functions. However, I ended up excluding it from my merge-ort submissions since it wasn't necessary. Would this be of interest in your submodule work, though? I guess it'd only matter if we started doing real merges of submodules as part of a parent repo merge. (As opposed to the fast-forward-only merges that merge-recursive/merge-ort do right now for submodules.)
I think that these functions would be useful if we were to write into submodules (for example, in a real merge of submodules, as you said). Right now my submodule work is just making it support partial clones, so I'm not planning to introduce any new submodule writes. (Well, besides the fact that a lazy-fetch in a submodule would end up with new objects, but that's already handled because we do the fetch in a separate process in the submodule's gitdir.)
v2 cover More add_submodule_odb() cleanup in merge code
In reply to Jonathan TanThis is on jt/grep-wo-submodule-odb-as-alternate (unlike version 1) because that branch introduces another usage of repo_submodule_init(), and I need to update it.
The main change from version 1 is that I have introduced a patch (patch
2) that teaches repo_submodule_init() to support submodules with unabsorbed gitdirs, which means that I no longer need the original patch 1 that updated t6437 to absorb all git dirs.
Once again, all these patches work under GIT_TEST_MERGE_ALGORITHM=recursive and GIT_TEST_MERGE_ALGORITHM=ort (and when that envvar is unset, for good measure).
I have also tested it with the "DO NOT SUBMIT" patch 3 from version 1.
Jonathan Tan (3):
submodule: remove unnecessary unabsorbed fallback
repository: support unabsorbed in repo_submodule_init
revision: remove "submodule" from opt struct
builtin/grep.c | 5 +- builtin/ls-files.c | 4 +- builtin/submodule--helper.c | 7 +-- merge-ort.c | 53 +++++++++++++------- merge-recursive.c | 49 +++++++++++------- repository.c | 21 ++++---- repository.h | 15 +++--- revision.c | 16 ++---- revision.h | 1 - submodule.c | 24 ++------- t/helper/test-submodule-nested-repo-config.c | 4 +- 11 files changed, 102 insertions(+), 97 deletions(-)
Signature · 1 line
2.33.0.309.g3052b89438-goog
PATCH v2 1/3 submodule: remove unnecessary unabsorbed fallback
In get_submodule_repo_for(), there is a fallback code path for the case in which a submodule has an unabsorbed gitdir. (See the documentation for "git submodule absorbgitdirs" for more information about absorbed and unabsorbed gitdirs.) However, this code path is unnecessary, because such submodules are already handled: when the fetch_task is created in fetch_task_create(), it will create its own struct submodule with a path and name, and repo_submodule_init() can handle such a struct.
This fallback was introduced in 26f80ccfc1 ("submodule: migrate get_next_submodule to use repository structs", 2018-12-05). It was unnecessary even then, but perhaps it escaped notice because its parent commit d5498e0871 ("repository: repo_submodule_init to take a submodule struct", 2018-12-05) was the one that taught repo_submodule_init() to handle such created structs. Before, it took a path and always checked .gitmodules, so it truly would have failed if there were no entry in .gitmodules.
(Note to reviewers: in 26f80ccfc1, the "own struct submodule" I mentioned is in get_next_submodule(), not fetch_task_create().)
--- submodule.c | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-)
submodule.c +2 -13
@@ -1409,19 +1409,8 @@ static struct repository *get_submodule_repo_for(struct repository *r,struct repository *ret =xmalloc (sizeof (*ret));if (repo_submodule_init (ret, r, sub)) { -/* -* No entry in .gitmodules? Technically not a submodule, -* but historically we supported repositories that happen to be -* in-place where a gitlink is. Keep supporting them. -*/ -struct strbuf gitdir =STRBUF_INIT ; -strbuf_repo_worktree_path (&gitdir, r,"%s/.git" , sub->path); -if (repo_init (ret, gitdir.buf,NULL )) { -strbuf_release (&gitdir); -free (ret); -return NULL ; - } -strbuf_release (&gitdir); +free (ret); +return NULL ; }return ret;
Signature · 1 line
2.33.0.309.g3052b89438-goog
PATCH v2 2/3 repository: support unabsorbed in repo_submodule_init
In reply to Jonathan TanIn preparation for a subsequent commit that migrates code using add_submodule_odb() to repo_submodule_init(), teach repo_submodule_init() to support submodules with unabsorbed gitdirs. (See the documentation for "git submodule absorbgitdirs" for more information about absorbed and unabsorbed gitdirs.)
--- builtin/grep.c | 5 +---- builtin/ls-files.c | 4 +--- builtin/submodule--helper.c | 7 +------ repository.c | 21 +++++++++++--------- repository.h | 15 ++++++++------ submodule.c | 9 +++------ t/helper/test-submodule-nested-repo-config.c | 4 +--- 7 files changed, 28 insertions(+), 37 deletions(-)
builtin/grep.c +1 -4
@@ -433,17 +433,14 @@ static int grep_submodule(struct grep_opt *opt, {struct repository *subrepo;struct repository *superproject = opt->repo; -const struct submodule *sub;struct grep_opt subopt;int hit =0 ; - sub =submodule_from_path (superproject,null_oid (), path); -if (!is_submodule_active (superproject, path))return 0 ; subrepo =xmalloc (sizeof (*subrepo)); -if (repo_submodule_init (subrepo, superproject, sub)) { +if (repo_submodule_init (subrepo, superproject, path,null_oid ())) {free (subrepo);return 0 ; }
builtin/ls-files.c +1 -3
@@ -209,10 +209,8 @@ static void show_submodule(struct repository *superproject, struct dir_struct *dir, const char *path) { struct repository subrepo; -const struct submodule *sub =submodule_from_path (superproject, -null_oid (), path); -if (repo_submodule_init (&subrepo, superproject, sub)) +if (repo_submodule_init (&subrepo, superproject, path,null_oid ()))return ;if (repo_read_index (&subrepo) <0 )
builtin/submodule--helper.c +1 -6
@@ -2540,7 +2540,6 @@ static int push_check(int argc, const char **argv, const char *prefix)static int ensure_core_worktree (int argc,const char **argv,const char *prefix) { -const struct submodule *sub;const char *path;const char *cw;struct repository subrepo;
@@ -2550,11 +2549,7 @@ static int ensure_core_worktree(int argc, const char **argv, const char *prefix) path = argv[1 ]; - sub =submodule_from_path (the_repository,null_oid (), path); -if (!sub) -BUG ("We could get the submodule handle before?" ); - -if (repo_submodule_init (&subrepo, the_repository, sub)) +if (repo_submodule_init (&subrepo, the_repository, path,null_oid ()))die (_ ("could not get a repository handle for submodule '%s'" ), path);if (!repo_config_get_string_tmp (&subrepo,"core.worktree" , &cw)) {
repository.c +12 -9
@@ -190,19 +190,15 @@ int repo_init(struct repository *repo,int repo_submodule_init (struct repository *subrepo,struct repository *superproject, -const struct submodule *sub) +const char *path, +const struct object_id *treeish_name) {struct strbuf gitdir =STRBUF_INIT ;struct strbuf worktree =STRBUF_INIT ;int ret =0 ; -if (!sub) { - ret =-1 ; - goto out; - } - -strbuf_repo_worktree_path (&gitdir, superproject,"%s/.git" , sub->path); -strbuf_repo_worktree_path (&worktree, superproject,"%s" , sub->path); +strbuf_repo_worktree_path (&gitdir, superproject,"%s/.git" , path); +strbuf_repo_worktree_path (&worktree, superproject,"%s" , path);if (repo_init (subrepo, gitdir.buf, worktree.buf)) {/*
@@ -212,6 +208,13 @@ int repo_submodule_init(struct repository *subrepo,* in the superproject's 'modules' directory. In this case the * submodule would not have a worktree. */ +const struct submodule *sub = +submodule_from_path (superproject, treeish_name, path); +if (!sub) { + ret =-1 ; + goto out; + } +strbuf_reset (&gitdir);strbuf_repo_git_path (&gitdir, superproject,"modules/%s" , sub->name);
@@ -225,7 +228,7 @@ int repo_submodule_init(struct repository *subrepo, subrepo->submodule_prefix =xstrfmt ("%s%s/" , superproject->submodule_prefix ? superproject->submodule_prefix : -"" , sub->path); +"" , path); out:strbuf_release (&gitdir);
repository.h +9 -6
@@ -172,15 +172,18 @@ void initialize_the_repository(void);int repo_init (struct repository *r,const char *gitdir,const char *worktree);/* -* Initialize the repository 'subrepo' as the submodule given by the -* struct submodule 'sub' in parent repository 'superproject'. -* Return 0 upon success and a non-zero value upon failure, which may happen -* if the submodule is not found, or 'sub' is NULL. +* Initialize the repository 'subrepo' as the submodule at the given path. If +* the submodule's gitdir cannot be found at <path>/.git, this function calls +* submodule_from_path() to try to find it. treeish_name is only used if +* submodule_from_path() needs to be called; see its documentation for more +* information. +* Return 0 upon success and a non-zero value upon failure. */ -struct submodule ; +struct object_id ;int repo_submodule_init (struct repository *subrepo,struct repository *superproject, -const struct submodule *sub); +const char *path, +const struct object_id *treeish_name);void repo_clear (struct repository *repo);/*
submodule.c +3 -6
@@ -520,9 +520,6 @@ static void prepare_submodule_repo_env_in_gitdir(struct strvec *out)/* * Initialize a repository struct for a submodule based on the provided 'path'. * -* Unlike repo_submodule_init, this tolerates submodules not present -* in .gitmodules. This function exists only to preserve historical behavior, -* * Returns the repository struct on success, * NULL when the submodule is not present. */
@@ -1404,11 +1401,11 @@ static void fetch_task_release(struct fetch_task *p) }static struct repository *get_submodule_repo_for (struct repository *r, -const struct submodule *sub) +const char *path) {struct repository *ret =xmalloc (sizeof (*ret)); -if (repo_submodule_init (ret, r, sub)) { +if (repo_submodule_init (ret, r, path,null_oid ())) {free (ret);return NULL ; }
@@ -1452,7 +1449,7 @@ static int get_next_submodule(struct child_process *cp,continue ; } - task->repo =get_submodule_repo_for (spf->r, task->sub); + task->repo =get_submodule_repo_for (spf->r, task->sub->path);if (task->repo) {struct strbuf submodule_prefix =STRBUF_INIT ;child_process_init (cp);
t/helper/test-submodule-nested-repo-config.c +1 -3
@@ -11,15 +11,13 @@ static void die_usage(const char **argv, const char *msg)int cmd__submodule_nested_repo_config (int argc,const char **argv) {struct repository subrepo; -const struct submodule *sub;if (argc <3 )die_usage (argv,"Wrong number of arguments." );setup_git_directory (); - sub =submodule_from_path (the_repository,null_oid (), argv[1 ]); -if (repo_submodule_init (&subrepo, the_repository, sub)) { +if (repo_submodule_init (&subrepo, the_repository, argv[1 ],null_oid ())) {die_usage (argv,"Submodule not found." ); }
Signature · 1 line
2.33.0.309.g3052b89438-goog
PATCH v2 3/3 revision: remove "submodule" from opt struct
In reply to Jonathan TanClean up a TODO in revision.h by removing the "submodule" field from struct setup_revision_opt. This field is only used to specify the ref store to use, so use rev_info->repo to determine the ref store instead.
The only users of this field are merge-ort.c and merge-recursive.c. However, both these files specify the superproject as rev_info->repo and the submodule as setup_revision_opt->submodule. In order to be able to pass the submodule as rev_info->repo, all commits must be parsed with the submodule explicitly specified; this patch does that as well. (An incremental solution in which only some commits are parsed with explicit submodule will not work, because if the same commit is parsed twice in different repositories, there will be 2 heap-allocated object structs corresponding to that commit, and any flag set by the revision walking mechanism on one of them will not be reflected onto the other.)
--- merge-ort.c | 53 +++++++++++++++++++++++++++++++---------------- merge-recursive.c | 49 ++++++++++++++++++++++++++++--------------- revision.c | 16 +++++--------- revision.h | 1 - 4 files changed, 72 insertions(+), 47 deletions(-)
merge-ort.c +35 -18
@@ -32,6 +32,7 @@#include "promisor-remote.h" #include "revision.h" #include "strmap.h" +#include "submodule-config.h" #include "submodule.h" #include "tree.h" #include "unpack-trees.h"
@@ -1499,7 +1500,6 @@ static int find_first_merges(struct repository *repo,xsnprintf (merged_revision,sizeof (merged_revision),"^%s" ,oid_to_hex (&a->object.oid));repo_init_revisions (repo, &revs,NULL ); - rev_opts.submodule = path;/* FIXME: can't handle linked worktrees in submodules yet */ revs.single_worktree = path !=NULL ;setup_revisions (ARRAY_SIZE (rev_args)-1 , rev_args, &revs, &rev_opts);
@@ -1509,7 +1509,7 @@ static int find_first_merges(struct repository *repo,die ("revision walk setup failed" );while ((commit =get_revision (&revs)) !=NULL ) {struct object *o = &(commit->object); -if (in_merge_bases (b, commit)) +if (repo_in_merge_bases (repo, b, commit))add_object_array (o,NULL , &merges); }reset_revision_walk ();
@@ -1524,7 +1524,7 @@ static int find_first_merges(struct repository *repo, contains_another =0 ;for (j =0 ; j < merges.nr; j++) {struct commit *m2 = (struct commit *) merges.objects[j].item; -if (i != j &&in_merge_bases (m2, m1)) { +if (i != j &&repo_in_merge_bases (repo, m2, m1)) { contains_another =1 ;break ; }
@@ -1545,10 +1545,12 @@ static int merge_submodule(struct merge_options *opt,const struct object_id *b, struct object_id *result) { + struct repository subrepo; +struct strbuf sb =STRBUF_INIT ; +int ret =0 ;struct commit *commit_o, *commit_a, *commit_b;int parent_count;struct object_array merges; -struct strbuf sb =STRBUF_INIT ;int i;int search = !opt->priv->call_depth;
@@ -1564,6 +1566,10 @@ static int merge_submodule(struct merge_options *opt,if (is_null_oid (b))return 0 ; +/* +* NEEDSWORK: Remove this when all submodule object accesses are +* through explicitly specified repositores. +*/ if (add_submodule_odb (path)) { path_msg(opt, path, 0, _("Failed to merge submodule %s (not checked out)"),
@@ -1571,39 +1577,48 @@ static int merge_submodule(struct merge_options *opt,return 0 ; } -if (!(commit_o =lookup_commit_reference (opt->repo, o)) || - !(commit_a =lookup_commit_reference (opt->repo, a)) || - !(commit_b =lookup_commit_reference (opt->repo, b))) { +if (repo_submodule_init (&subrepo, opt->repo, path,null_oid ())) { +path_msg (opt, path,0 , +_ ("Failed to merge submodule %s (not checked out)" ), + path); +return 0 ; + } + +if (!(commit_o =lookup_commit_reference (&subrepo, o)) || + !(commit_a =lookup_commit_reference (&subrepo, a)) || + !(commit_b =lookup_commit_reference (&subrepo, b))) {path_msg (opt, path,0 ,_ ("Failed to merge submodule %s (commits not present)" ), path); -return 0 ; + goto cleanup; }/* check whether both changes are forward */ -if (!in_merge_bases (commit_o, commit_a) || - !in_merge_bases (commit_o, commit_b)) { +if (!repo_in_merge_bases (&subrepo, commit_o, commit_a) || + !repo_in_merge_bases (&subrepo, commit_o, commit_b)) {path_msg (opt, path,0 ,_ ("Failed to merge submodule %s " "(commits don't follow merge-base)" ), path); -return 0 ; + goto cleanup; }/* Case #1: a is contained in b or vice versa */ -if (in_merge_bases (commit_a, commit_b)) { +if (repo_in_merge_bases (&subrepo, commit_a, commit_b)) {oidcpy (result, b);path_msg (opt, path,1 ,_ ("Note: Fast-forwarding submodule %s to %s" ), path,oid_to_hex (b)); -return 1 ; + ret =1 ; + goto cleanup; } -if (in_merge_bases (commit_b, commit_a)) { +if (repo_in_merge_bases (&subrepo, commit_b, commit_a)) {oidcpy (result, a);path_msg (opt, path,1 ,_ ("Note: Fast-forwarding submodule %s to %s" ), path,oid_to_hex (a)); -return 1 ; + ret =1 ; + goto cleanup; }/*
@@ -1615,10 +1630,10 @@ static int merge_submodule(struct merge_options *opt,/* Skip the search if makes no sense to the calling context. */ if (!search) -return 0 ; + goto cleanup;/* find commit which merges them */ - parent_count =find_first_merges (opt->repo, path, commit_a, commit_b, + parent_count =find_first_merges (&subrepo, path, commit_a, commit_b, &merges);switch (parent_count) {case 0 :
@@ -1652,7 +1667,9 @@ static int merge_submodule(struct merge_options *opt, }object_array_clear (&merges); -return 0 ; +cleanup: +repo_clear (&subrepo); +return ret; }static void initialize_attr_index (struct merge_options *opt)
merge-recursive.c +32 -17
125 lines not on this page · raw message
revision.c +5 -11
@@ -2561,8 +2561,7 @@ static int for_each_good_bisect_ref(struct ref_store *refs, each_ref_fn fn, voidreturn for_each_bisect_ref (refs, fn, cb_data, term_good); } -static int handle_revision_pseudo_opt (const char *submodule, -struct rev_info *revs, +static int handle_revision_pseudo_opt (struct rev_info *revs,const char **argv,int *flags) {const char *arg = argv[0 ];
@@ -2570,7 +2569,7 @@ static int handle_revision_pseudo_opt(const char *submodule,struct ref_store *refs;int argcount; -if (submodule) { +if (revs->repo != the_repository) {/* * We need some something like get_submodule_worktrees() * before we can go through all worktrees of a submodule,
@@ -2579,9 +2578,8 @@ static int handle_revision_pseudo_opt(const char *submodule,*/ if (!revs->single_worktree)BUG ("--single-worktree cannot be used together with submodule" ); - refs =get_submodule_ref_store (submodule); - }else - refs =get_main_ref_store (revs->repo); + } + refs =get_main_ref_store (revs->repo);/* * NOTE!
@@ -2699,12 +2697,8 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct s {int i, flags, left, seen_dashdash, revarg_opt;struct strvec prune_data =STRVEC_INIT ; -const char *submodule =NULL ;int seen_end_of_options =0 ; -if (opt) - submodule = opt->submodule; -/* First, search for "--" */ if (opt && opt->assume_dashdash) { seen_dashdash =1 ;
@@ -2733,7 +2727,7 @@ int setup_revisions(int argc, const char **argv, struct rev_info *revs, struct sif (!seen_end_of_options && *arg =='-' ) {int opts; - opts =handle_revision_pseudo_opt (submodule, + opts =handle_revision_pseudo_opt ( revs, argv + i, &flags);if (opts >0 ) {
revision.h +0 -1
@@ -339,7 +339,6 @@ extern volatile show_early_output_fn_t show_early_output;struct setup_revision_opt {const char *def;void (*tweak)(struct rev_info *,struct setup_revision_opt *); -const char *submodule;/* TODO: drop this and use rev_info->repo */ unsigned int assume_dashdash:1 , allow_exclude_promisor_objects:1 ;unsigned revarg_opt;
Signature · 1 line
2.33.0.309.g3052b89438-goog
Quoted 10 lines · Jonathan Tan
This is on jt/grep-wo-submodule-odb-as-alternate (unlike version 1) because that branch introduces another usage of
repo_submodule_init(), and I need to update it.The main change from version 1 is that I have introduced a patch (patch
2) that teachesrepo_submodule_init()to support submodules with unabsorbed gitdirs, which means that I no longer need the original patch 1 that updated t6437 to absorb all git dirs.
Quoted 5 lines
Once again, all these patches work under GIT_TEST_MERGE_ALGORITHM=recursive and GIT_TEST_MERGE_ALGORITHM=ort (and when that envvar is unset, for good measure).
I have also tested it with the "DO NOT SUBMIT" patch 3 from version 1.
While I don't yet know the details of repositories and submodules, I know the merge-recursive and merge-ort side of things and those changes all look reasonable to me. So here's an Ack for that side of patch 3.
Despite not being familiar with the repository and submodule storage parts of the code, I took a look over the remainder of the 3 patches to look for anything that might stand out and look amiss. I didn't spot anything; it all looks reasonable to me.