Skip to contentSkip to conversation

submodule: convert parts of 'update' to C

PATCH Superseded Glen Choo 3 participants 75 messages Mar 1, 2022 – Mar 5, 2022

lore
Share & tools
lore

Glen Choo

author
#0 lore raw

cover submodule: convert parts of 'update' to C

Original series: https://lore.kernel.org/git/[email protected]
(I've trimmed the cc list down to the 'most interested' parties)

= Overview

This is part 1 of 2 series that will supersede ar/submodule-update (as laid out in [1]). This series prepares for the eventual conversion of "git submodule update" to C by doing 'obvious' conversions first, and leaving more involved conversions for later.

Part 1 is a lot simpler than the original series in its entirety, and should play better with topics that Junio identified:

- This series is based off a later version of 'master' that already has 'js/apply-partial-clone-filters-recursively' merged in [2].
- There is only one, trivial, conflict with 'es/superproject-aware-submodules' (both add tests to the end of t7406) [3].

Most of these patches were originally from ar/submodule-update, but because of the new organization, some commit messages have been amended to make more sense in context. However, patches 12 and 13 are new - they were added to handle the "--filter" option introduced by 'js/apply-partial-clone-filters-recursively'.

Cc-ed Josh, who might be interested in "--filter" changes e.g. the new tests.

[1] https://lore.kernel.org/git/[email protected]
[2] This also fixes some trivial merge conflicts with 'master'.
[3] Part 2 has nontrival conflicts though. Offline, Emily mentioned that conflicts might go away in the next iteration of 'es/superproject-aware-submodules', but if not, the next round of patches will probably be based on a merge of this series + 'es/superproject-aware-submodules'.

= Patch summary

I'm not certain whether to keep patch 13, see the extra discussion in the --- description for details.

- Patch 1 adds extra tests to "git submodule update" to make sure we don't break anything
- Patch 2 removes dead code that used to be part of "git submodule update"
- Patch 3 prepares for later changes by introducing the C function that will hold most of the newly converted code
- Patch 4 moves run-update-procedure's --suboid option into C
- Patch 5 moves ensure-core-worktree into C
- Patches 6-8 move run-update-procedure's --remote option into C
- Patches 9-11 move "git submodule update"'s --init into C
- Patches 12-13 move "git submodule update"'s --filter option into C

Atharva Raykar (3):
submodule--helper: get remote names from any repository
submodule--helper: refactor get_submodule_displaypath()
submodule--helper: allow setting superprefix for init_submodule()

Glen Choo (8):
submodule--helper: remove update-module-mode
submodule--helper: reorganize code for sh to C conversion submodule--helper run-update-procedure: remove --suboid
submodule--helper: remove ensure-core-worktree submodule--helper run-update-procedure: learn --remote submodule--helper update-clone: learn --init submodule update: add tests for --filter submodule--helper update-clone: check for --filter and --init

Ævar Arnfjörð Bjarmason (2):
submodule tests: test for init and update failure output
submodule--helper: don't use bitfield indirection for parse_options()

 builtin/submodule--helper.c    | 230 ++++++++++++++++++++-------------
 git-submodule.sh               |  54 +-------
 t/t7406-submodule-update.sh    |  54 +++++++-
 t/t7408-submodule-reference.sh |  14 +-
 4 files changed, 211 insertions(+), 141 deletions(-)

base-commit: 715d08a9e51251ad8290b181b6ac3b9e1f9719d7

Signature · 1 line
2.33.GIT
author
#1 lore raw

PATCH 1/13 submodule tests: test for init and update failure output

From: Ævar Arnfjörð Bjarmason <[email protected]>

Amend some submodule tests to test for the failure output of "git submodule [update|init]". The lack of such tests hid a regression in an earlier version of a subsequent commit.

---
 t/t7406-submodule-update.sh    | 14 ++++++++++++--
 t/t7408-submodule-reference.sh | 14 +++++++++++++-
 2 files changed, 25 insertions(+), 3 deletions(-)
t/t7406-submodule-update.sh +12 -2
index 11cccbb333..7764c1c3cb 100755
@@ -205,8 +205,18 @@ test_expect_success 'submodule update should fail due to local changes' '
 	 (cd submodule &&
 	  compare_head
 	 ) &&
-	 test_must_fail git submodule update submodule
-	)
+	 test_must_fail git submodule update submodule 2>../actual.raw
+	) &&
+	sed "s/^> //" >expect <<-\EOF &&
+	> error: Your local changes to the following files would be overwritten by checkout:
+	> 	file
+	> Please commit your changes or stash them before you switch branches.
+	> Aborting
+	> fatal: Unable to checkout OID in submodule path '\''submodule'\''
+	EOF
+	sed -e "s/checkout $SQ[^$SQ]*$SQ/checkout OID/" <actual.raw >actual &&
+	test_cmp expect actual
+
 '
 test_expect_success 'submodule update should throw away changes with --force ' '
 	(cd super &&
t/t7408-submodule-reference.sh +13 -1
index a3892f494b..c3a4545510 100755
@@ -193,7 +193,19 @@ test_expect_success 'missing nested submodule alternate fails clone and submodul
 		cd supersuper-clone &&
 		check_that_two_of_three_alternates_are_used &&
 		# update of the submodule fails
-		test_must_fail git submodule update --init --recursive
+		cat >expect <<-\EOF &&
+		fatal: submodule '\''sub'\'' cannot add alternate: path ... does not exist
+		Failed to clone '\''sub'\''. Retry scheduled
+		fatal: submodule '\''sub-dissociate'\'' cannot add alternate: path ... does not exist
+		Failed to clone '\''sub-dissociate'\''. Retry scheduled
+		fatal: submodule '\''sub'\'' cannot add alternate: path ... does not exist
+		Failed to clone '\''sub'\'' a second time, aborting
+		fatal: Failed to recurse into submodule path ...
+		EOF
+		test_must_fail git submodule update --init --recursive 2>err &&
+		grep -e fatal: -e ^Failed err >actual.raw &&
+		sed -e "s/path $SQ[^$SQ]*$SQ/path .../" <actual.raw >actual &&
+		test_cmp expect actual
 	)
 '
 
Signature · 1 line
2.33.GIT
author
#2 lore raw

PATCH 2/13 submodule--helper: remove update-module-mode

In reply to Glen Choo

This is dead code - it has not been used since c51f8f94e5 (submodule--helper: run update procedures from C, 2021-08-24).

---
 builtin/submodule--helper.c | 24 ------------------------
 1 file changed, 24 deletions(-)
builtin/submodule--helper.c +0 -24
index eeacefcc38..c11ee1ea2b 100644
@@ -1957,29 +1957,6 @@ static void determine_submodule_update_strategy(struct repository *r,
 	free(key);
 }
 
-static int module_update_module_mode(int argc, const char **argv, const char *prefix)
-{
-	const char *path, *update = NULL;
-	int just_cloned;
-	struct submodule_update_strategy update_strategy = { .type = SM_UPDATE_CHECKOUT };
-
-	if (argc < 3 || argc > 4)
-		die("submodule--helper update-module-clone expects <just-cloned> <path> [<update>]");
-
-	just_cloned = git_config_int("just_cloned", argv[1]);
-	path = argv[2];
-
-	if (argc == 4)
-		update = argv[3];
-
-	determine_submodule_update_strategy(the_repository,
-					    just_cloned, path, update,
-					    &update_strategy);
-	fputs(submodule_strategy_to_string(&update_strategy), stdout);
-
-	return 0;
-}
-
 struct update_clone_data {
 	const struct submodule *sub;
 	struct object_id oid;
@@ -3430,7 +3407,6 @@ static struct cmd_struct commands[] = {
 	{"name", module_name, 0},
 	{"clone", module_clone, 0},
 	{"add", module_add, SUPPORT_SUPER_PREFIX},
-	{"update-module-mode", module_update_module_mode, 0},
 	{"update-clone", update_clone, 0},
 	{"run-update-procedure", run_update_procedure, 0},
 	{"ensure-core-worktree", ensure_core_worktree, 0},
Signature · 1 line
2.33.GIT
author
#3 lore raw

PATCH 3/13 submodule--helper: reorganize code for sh to C conversion

In reply to Glen Choo

Introduce a function, update_submodule2(), that will implement the functionality of run-update-procedure and its surrounding shell code in submodule.sh. This name is temporary; it will replace update_submodule() when the sh to C conversion is complete.

---
 builtin/submodule--helper.c | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)
builtin/submodule--helper.c +21 -5
index c11ee1ea2b..1b67a3887c 100644
@@ -2452,6 +2452,16 @@ static int do_run_update_procedure(struct update_data *ud)
 	return run_update_command(ud, subforce);
 }
 
+/*
+ * NEEDSWORK: As we convert "git submodule update" to C,
+ * update_submodule2() will invoke more and more functions, making it
+ * difficult to preserve the function ordering without forward
+ * declarations.
+ *
+ * When the conversion is complete, this forward declaration will be
+ * unnecessary and should be removed.
+ */
+static int update_submodule2(struct update_data *update_data);
 static void update_submodule(struct update_clone_data *ucd)
 {
 	fprintf(stdout, "dummy %s %d\t%s\n",
@@ -2618,11 +2628,7 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)
 					    &update_data.update_strategy);
 
 	free(prefixed_path);
-
-	if (!oideq(&update_data.oid, &update_data.suboid) || update_data.force)
-		return do_run_update_procedure(&update_data);
-
-	return 3;
+	return update_submodule2(&update_data);
 }
 
 static int resolve_relative_path(int argc, const char **argv, const char *prefix)
@@ -3022,6 +3028,16 @@ static int module_create_branch(int argc, const char **argv, const char *prefix)
 				    force, reflog, quiet, track, dry_run);
 	return 0;
 }
+
+/* NEEDSWORK: this is a temporary name until we delete update_submodule() */
+static int update_submodule2(struct update_data *update_data)
+{
+	if (!oideq(&update_data->oid, &update_data->suboid) || update_data->force)
+		return do_run_update_procedure(update_data);
+
+	return 3;
+}
+
 struct add_data {
 	const char *prefix;
 	const char *branch;
Signature · 1 line
2.33.GIT
author
#4 lore raw

PATCH 4/13 submodule--helper run-update-procedure: remove --suboid

In reply to Glen Choo

Teach run-update-procedure to determine the oid of the submodule's HEAD instead of doing it in git-subomdule.sh.

---
 builtin/submodule--helper.c | 9 ++++++---
 git-submodule.sh            | 8 +-------
 2 files changed, 7 insertions(+), 10 deletions(-)
builtin/submodule--helper.c +6 -3
index 1b67a3887c..77ca4270f4 100644
@@ -2594,9 +2594,6 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)
 		OPT_CALLBACK_F(0, "oid", &update_data.oid, N_("sha1"),
 			       N_("SHA1 expected by superproject"), PARSE_OPT_NONEG,
 			       parse_opt_object_id),
-		OPT_CALLBACK_F(0, "suboid", &update_data.suboid, N_("subsha1"),
-			       N_("SHA1 of submodule's HEAD"), PARSE_OPT_NONEG,
-			       parse_opt_object_id),
 		OPT_END()
 	};
 
@@ -3032,6 +3029,12 @@ static int module_create_branch(int argc, const char **argv, const char *prefix)
 /* NEEDSWORK: this is a temporary name until we delete update_submodule() */
 static int update_submodule2(struct update_data *update_data)
 {
+	if (update_data->just_cloned)
+		oidcpy(&update_data->suboid, null_oid());
+	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD", &update_data->suboid))
+		die(_("Unable to find current revision in submodule path '%s'"),
+			update_data->displaypath);
+
 	if (!oideq(&update_data->oid, &update_data->suboid) || update_data->force)
 		return do_run_update_procedure(update_data);
 
git-submodule.sh +1 -7
index 87772ac891..32a09302ab 100755
@@ -406,14 +406,9 @@ cmd_update()
 
 		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
 
-		if test $just_cloned -eq 1
+		if test $just_cloned -eq 0
 		then
-			subsha1=
-		else
 			just_cloned=
-			subsha1=$(sanitize_submodule_env; cd "$sm_path" &&
-				git rev-parse --verify HEAD) ||
-			die "fatal: $(eval_gettext "Unable to find current revision in submodule path '\$displaypath'")"
 		fi
 
 		if test -n "$remote"
@@ -441,7 +436,6 @@ cmd_update()
 			  ${update:+--update "$update"} \
 			  ${prefix:+--recursive-prefix "$prefix"} \
 			  ${sha1:+--oid "$sha1"} \
-			  ${subsha1:+--suboid "$subsha1"} \
 			  "--" \
 			  "$sm_path")
 
Signature · 1 line
2.33.GIT
author
#5 lore raw

PATCH 5/13 submodule--helper: remove ensure-core-worktree

In reply to Glen Choo

Move the logic of "git submodule--helper ensure-core-worktree" into run-update-procedure. Since the ensure-core-worktree command is obsolete, remove it.

---
 builtin/submodule--helper.c | 12 ++----------
 git-submodule.sh            |  2 --
 2 files changed, 2 insertions(+), 12 deletions(-)
builtin/submodule--helper.c +2 -10
index 77ca4270f4..6b473fc0d2 100644
@@ -2771,17 +2771,11 @@ static int push_check(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
-static int ensure_core_worktree(int argc, const char **argv, const char *prefix)
+static void ensure_core_worktree(const char *path)
 {
-	const char *path;
 	const char *cw;
 	struct repository subrepo;
 
-	if (argc != 2)
-		BUG("submodule--helper ensure-core-worktree <path>");
-
-	path = argv[1];
-
 	if (repo_submodule_init(&subrepo, the_repository, path, null_oid()))
 		die(_("could not get a repository handle for submodule '%s'"), path);
 
@@ -2801,8 +2795,6 @@ static int ensure_core_worktree(int argc, const char **argv, const char *prefix)
 		free(abs_path);
 		strbuf_release(&sb);
 	}
-
-	return 0;
 }
 
 static int absorb_git_dirs(int argc, const char **argv, const char *prefix)
@@ -3029,6 +3021,7 @@ static int module_create_branch(int argc, const char **argv, const char *prefix)
 /* NEEDSWORK: this is a temporary name until we delete update_submodule() */
 static int update_submodule2(struct update_data *update_data)
 {
+	ensure_core_worktree(update_data->sm_path);
 	if (update_data->just_cloned)
 		oidcpy(&update_data->suboid, null_oid());
 	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD", &update_data->suboid))
@@ -3428,7 +3421,6 @@ static struct cmd_struct commands[] = {
 	{"add", module_add, SUPPORT_SUPER_PREFIX},
 	{"update-clone", update_clone, 0},
 	{"run-update-procedure", run_update_procedure, 0},
-	{"ensure-core-worktree", ensure_core_worktree, 0},
 	{"relative-path", resolve_relative_path, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
 	{"foreach", module_foreach, SUPPORT_SUPER_PREFIX},
git-submodule.sh +0 -2
index 32a09302ab..458ce73ac6 100755
@@ -402,8 +402,6 @@ cmd_update()
 	do
 		die_if_unmatched "$quickabort" "$sha1"
 
-		git submodule--helper ensure-core-worktree "$sm_path" || exit 1
-
 		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
 
 		if test $just_cloned -eq 0
Signature · 1 line
2.33.GIT
author
#6 lore raw

PATCH 6/13 submodule--helper: get remote names from any repository

In reply to Glen Choo

From: Atharva Raykar <[email protected]>

get_default_remote() retrieves the name of a remote by resolving the refs from of the current repository's ref store.

Thus in order to use it for retrieving the remote name of a submodule, we have to start a new subprocess which runs from the submodule directory.

Let's instead introduce a function called repo_get_default_remote() which takes any repository object and retrieves the remote accordingly.

get_default_remote() is then defined as a call to repo_get_default_remote() with 'the_repository' passed to it.

Now that we have repo_get_default_remote(), we no longer have to start a subprocess that called submodule--helper get-default-remote from within the submodule directory.

So let's make a function called get_default_remote_submodule() which takes a submodule path, and returns the default remote for that submodule, all within the same process.

We can now use this function to save an unnecessary subprocess spawn in sync_submodule(), and also in the next patch, which will require this functionality.

---
 builtin/submodule--helper.c | 39 +++++++++++++++++++++++--------------
 1 file changed, 24 insertions(+), 15 deletions(-)
builtin/submodule--helper.c +24 -15
index 6b473fc0d2..f934e33c7e 100644
@@ -31,11 +31,14 @@
 typedef void (*each_submodule_fn)(const struct cache_entry *list_item,
 				  void *cb_data);
 
-static char *get_default_remote(void)
+static char *repo_get_default_remote(struct repository *repo)
 {
 	char *dest = NULL, *ret;
 	struct strbuf sb = STRBUF_INIT;
-	const char *refname = resolve_ref_unsafe("HEAD", 0, NULL, NULL);
+	struct ref_store *store = get_main_ref_store(repo);
+	int ignore_errno;
+	const char *refname = refs_resolve_ref_unsafe(store, "HEAD", 0, NULL,
+						      NULL, &ignore_errno);
 
 	if (!refname)
 		die(_("No such ref: %s"), "HEAD");
@@ -48,7 +51,7 @@ static char *get_default_remote(void)
 		die(_("Expecting a full ref name, got %s"), refname);
 
 	strbuf_addf(&sb, "branch.%s.remote", refname);
-	if (git_config_get_string(sb.buf, &dest))
+	if (repo_config_get_string(repo, sb.buf, &dest))
 		ret = xstrdup("origin");
 	else
 		ret = dest;
@@ -57,6 +60,19 @@ static char *get_default_remote(void)
 	return ret;
 }
 
+static char *get_default_remote_submodule(const char *module_path)
+{
+	struct repository subrepo;
+
+	repo_submodule_init(&subrepo, the_repository, module_path, null_oid());
+	return repo_get_default_remote(&subrepo);
+}
+
+static char *get_default_remote(void)
+{
+	return repo_get_default_remote(the_repository);
+}
+
 static int print_default_remote(int argc, const char **argv, const char *prefix)
 {
 	char *remote;
@@ -1343,9 +1359,8 @@ static void sync_submodule(const char *path, const char *prefix,
 {
 	const struct submodule *sub;
 	char *remote_key = NULL;
-	char *sub_origin_url, *super_config_url, *displaypath;
+	char *sub_origin_url, *super_config_url, *displaypath, *default_remote;
 	struct strbuf sb = STRBUF_INIT;
-	struct child_process cp = CHILD_PROCESS_INIT;
 	char *sub_config_path = NULL;
 
 	if (!is_submodule_active(the_repository, path))
@@ -1384,21 +1399,15 @@ static void sync_submodule(const char *path, const char *prefix,
 	if (!is_submodule_populated_gently(path, NULL))
 		goto cleanup;
 
-	prepare_submodule_repo_env(&cp.env_array);
-	cp.git_cmd = 1;
-	cp.dir = path;
-	strvec_pushl(&cp.args, "submodule--helper",
-		     "print-default-remote", NULL);
-
 	strbuf_reset(&sb);
-	if (capture_command(&cp, &sb, 0))
+	default_remote = get_default_remote_submodule(path);
+	if (!default_remote)
 		die(_("failed to get the default remote for submodule '%s'"),
 		      path);
 
-	strbuf_strip_suffix(&sb, "\n");
-	remote_key = xstrfmt("remote.%s.url", sb.buf);
+	remote_key = xstrfmt("remote.%s.url", default_remote);
+	free(default_remote);
 
-	strbuf_reset(&sb);
 	submodule_to_gitdir(&sb, path);
 	strbuf_addstr(&sb, "/config");
 
Signature · 1 line
2.33.GIT
#7 lore raw

Re: [PATCH 06/13] submodule--helper: get remote names from any repository

Review

Quoted 4 lines · Glen Choo
(quoted diff) +4 -0
+	struct ref_store *store = get_main_ref_store(repo);
+	int ignore_errno;
+	const char *refname = refs_resolve_ref_unsafe(store, "HEAD", 0, NULL,
+						      NULL, &ignore_errno);

Given that 00/13 says this series is based on 715d08a9 (The eighth batch, 2022-02-25), which includes 03bdcfcc (Merge branch 'ab/no-errno-from-resolve-ref-unsafe', 2022-02-11), I think the above two lines are result of incorrect rebasing or something.

Have you compiled after you rebased?

It seems that after applying the band-aid below, t7406 seems to fail for two of its tests, too, but if this were not even compiled, that is to be expected X-<.

 builtin/submodule--helper.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
builtin/submodule--helper.c +1 -3
index ea88f39fb4..21401ad99e 100644
@@ -36,9 +36,7 @@ static char *repo_get_default_remote(struct repository *repo)
 	char *dest = NULL, *ret;
 	struct strbuf sb = STRBUF_INIT;
 	struct ref_store *store = get_main_ref_store(repo);
-	int ignore_errno;
-	const char *refname = refs_resolve_ref_unsafe(store, "HEAD", 0, NULL,
-						      NULL, &ignore_errno);
+	const char *refname = refs_resolve_ref_unsafe(store, "HEAD", 0, NULL, NULL);
 
 	if (!refname)
 		die(_("No such ref: %s"), "HEAD");
author
#8 lore raw

Re: [PATCH 06/13] submodule--helper: get remote names from any repository

Quoted 12 lines · Junio C Hamano

Review

Quoted 4 lines · Glen Choo
(quoted diff) +4 -0
+	struct ref_store *store = get_main_ref_store(repo);
+	int ignore_errno;
+	const char *refname = refs_resolve_ref_unsafe(store, "HEAD", 0, NULL,
+						      NULL, &ignore_errno);

Given that 00/13 says this series is based on 715d08a9 (The eighth batch, 2022-02-25), which includes 03bdcfcc (Merge branch 'ab/no-errno-from-resolve-ref-unsafe', 2022-02-11), I think the above two lines are result of incorrect rebasing or something.

Quoted 1 line

Have you compiled after you rebased?

It turns out that I didn't.. I got complacent after doing this merge a few times.

Quoted 3 lines

It seems that after applying the band-aid below, t7406 seems to fail for two of its tests, too, but if this were not even compiled, that is to be expected X-<.

This is an even sillier mistake.. patch 12 accidentally includes some tests from es/superproject-aware-submodules (probably a bad merge conflict resolution).

Quoted 21 lines


 builtin/submodule--helper.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
builtin/submodule--helper.c +1 -3
index ea88f39fb4..21401ad99e 100644
@@ -36,9 +36,7 @@ static char *repo_get_default_remote(struct repository *repo)
 	char *dest = NULL, *ret;
 	struct strbuf sb = STRBUF_INIT;
 	struct ref_store *store = get_main_ref_store(repo);
-	int ignore_errno;
-	const char *refname = refs_resolve_ref_unsafe(store, "HEAD", 0, NULL,
-						      NULL, &ignore_errno);
+	const char *refname = refs_resolve_ref_unsafe(store, "HEAD", 0, NULL, NULL);
 
 	if (!refname)
 		die(_("No such ref: %s"), "HEAD");
author
#9 lore raw

PATCH 7/13 submodule--helper: don't use bitfield indirection for parse_options()

In reply to Glen Choo

From: Ævar Arnfjörð Bjarmason <[email protected]>

Do away with the indirection of local variables added in c51f8f94e5b (submodule--helper: run update procedures from C, 2021-08-24).

These were only needed because in C you can't get a pointer to a single bit, so we were using intermediate variables instead.

---
 builtin/submodule--helper.c | 23 ++++++++++-------------
 1 file changed, 10 insertions(+), 13 deletions(-)
builtin/submodule--helper.c +10 -13
index f934e33c7e..5fbf8713db 100644
@@ -2024,10 +2024,10 @@ struct update_data {
 	struct object_id suboid;
 	struct submodule_update_strategy update_strategy;
 	int depth;
-	unsigned int force: 1;
-	unsigned int quiet: 1;
-	unsigned int nofetch: 1;
-	unsigned int just_cloned: 1;
+	unsigned int force;
+	unsigned int quiet;
+	unsigned int nofetch;
+	unsigned int just_cloned;
 };
 #define UPDATE_DATA_INIT { .update_strategy = SUBMODULE_UPDATE_STRATEGY_INIT }
 
@@ -2579,16 +2579,17 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 
 static int run_update_procedure(int argc, const char **argv, const char *prefix)
 {
-	int force = 0, quiet = 0, nofetch = 0, just_cloned = 0;
 	char *prefixed_path, *update = NULL;
 	struct update_data update_data = UPDATE_DATA_INIT;
 
 	struct option options[] = {
-		OPT__QUIET(&quiet, N_("suppress output for update by rebase or merge")),
-		OPT__FORCE(&force, N_("force checkout updates"), 0),
-		OPT_BOOL('N', "no-fetch", &nofetch,
+		OPT__QUIET(&update_data.quiet,
+			   N_("suppress output for update by rebase or merge")),
+		OPT__FORCE(&update_data.force, N_("force checkout updates"),
+			   0),
+		OPT_BOOL('N', "no-fetch", &update_data.nofetch,
 			 N_("don't fetch new objects from the remote site")),
-		OPT_BOOL(0, "just-cloned", &just_cloned,
+		OPT_BOOL(0, "just-cloned", &update_data.just_cloned,
 			 N_("overrides update mode in case the repository is a fresh clone")),
 		OPT_INTEGER(0, "depth", &update_data.depth, N_("depth for shallow fetch")),
 		OPT_STRING(0, "prefix", &prefix,
@@ -2616,10 +2617,6 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)
 	if (argc != 1)
 		usage_with_options(usage, options);
 
-	update_data.force = !!force;
-	update_data.quiet = !!quiet;
-	update_data.nofetch = !!nofetch;
-	update_data.just_cloned = !!just_cloned;
 	update_data.sm_path = argv[0];
 
 	if (update_data.recursive_prefix)
Signature · 1 line
2.33.GIT
author
#10 lore raw

PATCH 8/13 submodule--helper run-update-procedure: learn --remote

In reply to Glen Choo

Teach run-update-procedure to handle --remote instead of parsing --remote in git-submodule.sh. As a result, "git submodule--helper print-default-remote" has no more callers, so remove it.

---
 builtin/submodule--helper.c | 39 ++++++++++++++++++++++---------------
 git-submodule.sh            | 30 +---------------------------
 2 files changed, 24 insertions(+), 45 deletions(-)
builtin/submodule--helper.c +23 -16
index 5fbf8713db..1d4d9231ac 100644
@@ -73,21 +73,6 @@ static char *get_default_remote(void)
 	return repo_get_default_remote(the_repository);
 }
 
-static int print_default_remote(int argc, const char **argv, const char *prefix)
-{
-	char *remote;
-
-	if (argc != 1)
-		die(_("submodule--helper print-default-remote takes no arguments"));
-
-	remote = get_default_remote();
-	if (remote)
-		printf("%s\n", remote);
-
-	free(remote);
-	return 0;
-}
-
 static int starts_with_dot_slash(const char *str)
 {
 	return str[0] == '.' && is_dir_sep(str[1]);
@@ -2028,6 +2013,7 @@ struct update_data {
 	unsigned int quiet;
 	unsigned int nofetch;
 	unsigned int just_cloned;
+	unsigned int remote;
 };
 #define UPDATE_DATA_INIT { .update_strategy = SUBMODULE_UPDATE_STRATEGY_INIT }
 
@@ -2604,6 +2590,8 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)
 		OPT_CALLBACK_F(0, "oid", &update_data.oid, N_("sha1"),
 			       N_("SHA1 expected by superproject"), PARSE_OPT_NONEG,
 			       parse_opt_object_id),
+		OPT_BOOL(0, "remote", &update_data.remote,
+			 N_("use SHA-1 of submodule's remote tracking branch")),
 		OPT_END()
 	};
 
@@ -3034,6 +3022,25 @@ static int update_submodule2(struct update_data *update_data)
 		die(_("Unable to find current revision in submodule path '%s'"),
 			update_data->displaypath);
 
+	if (update_data->remote) {
+		char *remote_name = get_default_remote_submodule(update_data->sm_path);
+		const char *branch = remote_submodule_branch(update_data->sm_path);
+		char *remote_ref = xstrfmt("refs/remotes/%s/%s", remote_name, branch);
+
+		if (!update_data->nofetch) {
+			if (fetch_in_submodule(update_data->sm_path, update_data->depth,
+					      0, NULL))
+				die(_("Unable to fetch in submodule path '%s'"),
+				    update_data->sm_path);
+		}
+
+		if (resolve_gitlink_ref(update_data->sm_path, remote_ref, &update_data->oid))
+			die(_("Unable to find %s revision in submodule path '%s'"),
+			    remote_ref, update_data->sm_path);
+
+		free(remote_ref);
+	}
+
 	if (!oideq(&update_data->oid, &update_data->suboid) || update_data->force)
 		return do_run_update_procedure(update_data);
 
@@ -3432,10 +3439,10 @@ static struct cmd_struct commands[] = {
 	{"foreach", module_foreach, SUPPORT_SUPER_PREFIX},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
-	{"print-default-remote", print_default_remote, 0},
 	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
 	{"deinit", module_deinit, 0},
 	{"summary", module_summary, SUPPORT_SUPER_PREFIX},
+	/* NEEDSWORK: remote-branch is also obsolete */
 	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
git-submodule.sh +1 -29
index 458ce73ac6..23ebd90892 100755
@@ -247,20 +247,6 @@ cmd_deinit()
 	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper deinit ${GIT_QUIET:+--quiet} ${force:+--force} ${deinit_all:+--all} -- "$@"
 }
 
-# usage: fetch_in_submodule <module_path> [<depth>] [<sha1>]
-# Because arguments are positional, use an empty string to omit <depth>
-# but include <sha1>.
-fetch_in_submodule () (
-	sanitize_submodule_env &&
-	cd "$1" &&
-	if test $# -eq 3
-	then
-		echo "$3" | git fetch ${GIT_QUIET:+--quiet} --stdin ${2:+"$2"}
-	else
-		git fetch ${GIT_QUIET:+--quiet} ${2:+"$2"}
-	fi
-)
-
 #
 # Update each submodule path to correct revision, using clone and checkout as needed
 #
@@ -409,21 +395,6 @@ cmd_update()
 			just_cloned=
 		fi
 
-		if test -n "$remote"
-		then
-			branch=$(git submodule--helper remote-branch "$sm_path")
-			if test -z "$nofetch"
-			then
-				# Fetch remote before determining tracking $sha1
-				fetch_in_submodule "$sm_path" $depth ||
-				die "fatal: $(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
-			fi
-			remote_name=$(sanitize_submodule_env; cd "$sm_path" && git submodule--helper print-default-remote)
-			sha1=$(sanitize_submodule_env; cd "$sm_path" &&
-				git rev-parse --verify "${remote_name}/${branch}") ||
-			die "fatal: $(eval_gettext "Unable to find current \${remote_name}/\${branch} revision in submodule path '\$sm_path'")"
-		fi
-
 		out=$(git submodule--helper run-update-procedure \
 			  ${wt_prefix:+--prefix "$wt_prefix"} \
 			  ${GIT_QUIET:+--quiet} \
@@ -434,6 +405,7 @@ cmd_update()
 			  ${update:+--update "$update"} \
 			  ${prefix:+--recursive-prefix "$prefix"} \
 			  ${sha1:+--oid "$sha1"} \
+			  ${remote:+--remote} \
 			  "--" \
 			  "$sm_path")
 
Signature · 1 line
2.33.GIT
author
#11 lore raw

PATCH 9/13 submodule--helper: refactor get_submodule_displaypath()

In reply to Glen Choo

From: Atharva Raykar <[email protected]>

We create a function called do_get_submodule_displaypath() that generates the display path required by several submodule functions, and takes a custom superprefix parameter, instead of reading it from the environment.

We then redefine the existing get_submodule_displaypath() function as a call to this new function, where the superprefix is obtained from the environment.

---
 builtin/submodule--helper.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
builtin/submodule--helper.c +8 -4
index 1d4d9231ac..3598069de5 100644
@@ -248,11 +248,8 @@ static int resolve_relative_url_test(int argc, const char **argv, const char *pr
 	return 0;
 }
 
-/* the result should be freed by the caller. */
-static char *get_submodule_displaypath(const char *path, const char *prefix)
+static char *do_get_submodule_displaypath(const char *path, const char *prefix, const char *super_prefix)
 {
-	const char *super_prefix = get_super_prefix();
-
 	if (prefix && super_prefix) {
 		BUG("cannot have prefix '%s' and superprefix '%s'",
 		    prefix, super_prefix);
@@ -268,6 +265,13 @@ static char *get_submodule_displaypath(const char *path, const char *prefix)
 	}
 }
 
+/* the result should be freed by the caller. */
+static char *get_submodule_displaypath(const char *path, const char *prefix)
+{
+	const char *super_prefix = get_super_prefix();
+	return do_get_submodule_displaypath(path, prefix, super_prefix);
+}
+
 static char *compute_rev_name(const char *sub_path, const char* object_id)
 {
 	struct strbuf sb = STRBUF_INIT;
Signature · 1 line
2.33.GIT
author
#12 lore raw

PATCH 10/13 submodule--helper: allow setting superprefix for init_submodule()

In reply to Glen Choo

From: Atharva Raykar <[email protected]>

We allow callers of the init_submodule() function to optionally override the superprefix from the environment.

We need to enable this option because in our conversion of the update command that will follow, the '--init' option will be handled through this API. We will need to change the superprefix at that time to ensure the display paths show correctly in the output messages.

---
 builtin/submodule--helper.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
builtin/submodule--helper.c +7 -3
index 3598069de5..c3760f511d 100644
@@ -593,18 +593,22 @@ static int module_foreach(int argc, const char **argv, const char *prefix)
 
 struct init_cb {
 	const char *prefix;
+	const char *superprefix;
 	unsigned int flags;
 };
 #define INIT_CB_INIT { 0 }
 
 static void init_submodule(const char *path, const char *prefix,
-			   unsigned int flags)
+			   const char *superprefix, unsigned int flags)
 {
 	const struct submodule *sub;
 	struct strbuf sb = STRBUF_INIT;
 	char *upd = NULL, *url = NULL, *displaypath;
 
-	displaypath = get_submodule_displaypath(path, prefix);
+	/* try superprefix from the environment, if it is not passed explicitly */
+	if (!superprefix)
+		superprefix = get_super_prefix();
+	displaypath = do_get_submodule_displaypath(path, prefix, superprefix);
 
 	sub = submodule_from_path(the_repository, null_oid(), path);
 
@@ -678,7 +682,7 @@ static void init_submodule(const char *path, const char *prefix,
 static void init_submodule_cb(const struct cache_entry *list_item, void *cb_data)
 {
 	struct init_cb *info = cb_data;
-	init_submodule(list_item->name, info->prefix, info->flags);
+	init_submodule(list_item->name, info->prefix, info->superprefix, info->flags);
 }
 
 static int module_init(int argc, const char **argv, const char *prefix)
Signature · 1 line
2.33.GIT
author
#13 lore raw

PATCH 11/13 submodule--helper update-clone: learn --init

In reply to Glen Choo

Teach "git submodule--helper update-clone" the --init flag and remove the corresponding shell code.

When the --init flag is passed to the subcommand, we do not spawn a new subprocess and call submodule--helper init on the submodule paths, because the Git machinery is not able to pick up the configuration changes introduced by that init call. So we instead run the init_submodule_cb() callback over each submodule in the same process.

[1] https://lore.kernel.org/git/CAP8UFD0NCQ5w_3GtT_xHr35i7h8BuLX4UcHNY6VHPGREmDVObA@mail.gmail.com/

---
 builtin/submodule--helper.c | 26 ++++++++++++++++++++++++++
 git-submodule.sh            |  9 +++------
 2 files changed, 29 insertions(+), 6 deletions(-)
builtin/submodule--helper.c +26 -0
index c3760f511d..0fc07d35ae 100644
@@ -1999,6 +1999,7 @@ struct submodule_update_clone {
 	int failed_clones_nr, failed_clones_alloc;
 
 	int max_jobs;
+	unsigned int init;
 };
 #define SUBMODULE_UPDATE_CLONE_INIT { \
 	.list = MODULE_LIST_INIT, \
@@ -2508,6 +2509,8 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	int ret;
 
 	struct option module_update_clone_options[] = {
+		OPT_BOOL(0, "init", &suc.init,
+			 N_("initialize uninitialized submodules before update")),
 		OPT_STRING(0, "prefix", &prefix,
 			   N_("path"),
 			   N_("path into the working tree")),
@@ -2566,6 +2569,29 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	if (pathspec.nr)
 		suc.warn_if_uninitialized = 1;
 
+	if (suc.init) {
+		struct module_list list = MODULE_LIST_INIT;
+		struct init_cb info = INIT_CB_INIT;
+
+		if (module_list_compute(argc, argv, suc.prefix,
+					&pathspec, &list) < 0)
+			return 1;
+
+		/*
+		 * If there are no path args and submodule.active is set then,
+		 * by default, only initialize 'active' modules.
+		 */
+		if (!argc && git_config_get_value_multi("submodule.active"))
+			module_list_active(&list);
+
+		info.prefix = suc.prefix;
+		info.superprefix = suc.recursive_prefix;
+		if (suc.quiet)
+			info.flags |= OPT_QUIET;
+
+		for_each_listed_submodule(&list, init_submodule_cb, &info);
+	}
+
 	ret = update_submodules(&suc);
 	list_objects_filter_release(&filter_options);
 	return ret;
git-submodule.sh +3 -6
index 23ebd90892..51be7c7f7e 100755
@@ -361,14 +361,11 @@ cmd_update()
 		usage
 	fi
 
-	if test -n "$init"
-	then
-		cmd_init "--" "$@" || return
-	fi
-
 	{
-	git submodule--helper update-clone ${GIT_QUIET:+--quiet} \
+	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper update-clone \
+		${GIT_QUIET:+--quiet} \
 		${progress:+"--progress"} \
+		${init:+--init} \
 		${wt_prefix:+--prefix "$wt_prefix"} \
 		${prefix:+--recursive-prefix "$prefix"} \
 		${update:+--update "$update"} \
Signature · 1 line
2.33.GIT
author
#14 lore raw

PATCH 12/13 submodule update: add tests for --filter

In reply to Glen Choo

Test the "--filter" option to make sure we don't break it while refactoring "git submodule update".

---
 t/t7406-submodule-update.sh | 40 +++++++++++++++++++++++++++++++++++++
 1 file changed, 40 insertions(+)
t/t7406-submodule-update.sh +40 -0
index 7764c1c3cb..86616bf8c7 100755
@@ -1071,4 +1071,44 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s
 	)
 '
 
+test_expect_success 'submodule update adds superproject gitdir to older repos' '
+	(cd super &&
+	 git -C submodule config --unset submodule.superprojectGitdir &&
+	 git submodule update &&
+	 test-tool path-utils relative_path \
+		"$(git rev-parse --absolute-git-dir)" \
+		"$(git -C submodule rev-parse --absolute-git-dir)" >expect &&
+	 git -C submodule config submodule.superprojectGitdir >actual &&
+	 test_cmp expect actual
+	)
+'
+
+test_expect_success 'submodule update uses config.worktree if applicable' '
+	(cd super &&
+	 git -C submodule config --unset submodule.superprojectGitDir &&
+	 git -C submodule config extensions.worktreeConfig true &&
+	 git submodule update &&
+	 test-tool path-utils relative_path \
+		"$(git rev-parse --absolute-git-dir)" \
+		"$(git -C submodule rev-parse --absolute-git-dir)" >expect &&
+	 git -C submodule config submodule.superprojectGitdir >actual &&
+	 test_cmp expect actual &&
+
+	 test_file_not_empty "$(git -C submodule rev-parse --absolute-git-dir)/config.worktree"
+	)
+'
+
+test_expect_success 'submodule update --filter requires --init' '
+	test_must_fail git -C super submodule update --filter blob:none 2>err &&
+	grep "usage:" err
+'
+
+test_expect_success 'submodule update --filter sets partial clone settings' '
+	test_when_finished "rm -rf super-filter" &&
+	git clone cloned super-filter &&
+	git -C super-filter submodule update --init --filter blob:none &&
+	test_cmp_config -C super-filter/submodule true remote.origin.promisor &&
+	test_cmp_config -C super-filter/submodule blob:none remote.origin.partialclonefilter
+'
+
 test_done
Signature · 1 line
2.33.GIT
author
#15 lore raw

PATCH 13/13 submodule--helper update-clone: check for --filter and --init

In reply to Glen Choo

"git submodule update --filter" also requires the "--init" option. Teach update-clone to do this usage check in C and remove the check from git-submodule.sh.

In addition, change update-clone's usage string so that it teaches users about "git submodule update" instead of "git submodule--helper update-clone" (the string is copied from git-submodule.sh). This should be more helpful to users since they don't invoke update-clone directly.

---
Since we expect users to act upon the usage string, I've updated it to reflect "git submodule update" [1] (since that's what users actually invoke), but I feel a bit iffy about not being able to use usage_with_options() (because the options and usage string are for different commands).

This might indicate that this is work we should put off until the conversion to C is mostly complete, but on the other hand, the usage string is still more helpful than it used to be because we never presented users with the options anyway.

[1] It's not immediately obvious which command we prefer to show - some other commands use "git submodule--helper" and others use "git submodule".

 builtin/submodule--helper.c | 20 +++++++++++++++++++-
 git-submodule.sh            |  5 -----
 2 files changed, 19 insertions(+), 6 deletions(-)
builtin/submodule--helper.c +19 -1
index 0fc07d35ae..ea88f39fb4 100644
@@ -2544,7 +2544,12 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	};
 
 	const char *const git_submodule_helper_usage[] = {
-		N_("git submodule--helper update-clone [--prefix=<path>] [<path>...]"),
+		N_("git submodule [--quiet] update"
+		"[--init [--filter=<filter-spec>]] [--remote]"
+		"[-N|--no-fetch] [-f|--force]"
+		"[--checkout|--merge|--rebase]"
+		"[--[no-]recommend-shallow] [--reference <repository>]"
+		"[--recursive] [--[no-]single-branch] [--] [<path>...]"),
 		NULL
 	};
 	suc.prefix = prefix;
@@ -2555,6 +2560,19 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	memset(&filter_options, 0, sizeof(filter_options));
 	argc = parse_options(argc, argv, prefix, module_update_clone_options,
 			     git_submodule_helper_usage, 0);
+
+	if (filter_options.choice && !suc.init) {
+		/*
+		 * NEEDSWORK: Don't use usage_with_options() because the
+		 * usage string is for "git submodule update", but the
+		 * options are for "git submodule--helper update-clone".
+		 *
+		 * This will no longer be an issue when "update-clone"
+		 * is replaced by "git submodule--helper update".
+		 */
+		usage(git_submodule_helper_usage[0]);
+	}
+
 	suc.filter_options = &filter_options;
 
 	if (update)
git-submodule.sh +0 -5
index 51be7c7f7e..aa8bdfca9d 100755
@@ -356,11 +356,6 @@ cmd_update()
 		shift
 	done
 
-	if test -n "$filter" && test "$init" != "1"
-	then
-		usage
-	fi
-
 	{
 	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper update-clone \
 		${GIT_QUIET:+--quiet} \
Signature · 1 line
2.33.GIT
author
#16 lore raw
In reply to Glen Choo
Quoted 70 lines · Glen Choo

Original series: https://lore.kernel.org/git/[email protected]
(I've trimmed the cc list down to the 'most interested' parties)

= Overview

This is part 1 of 2 series that will supersede ar/submodule-update (as laid out in [1]). This series prepares for the eventual conversion of "git submodule update" to C by doing 'obvious' conversions first, and leaving more involved conversions for later.

Part 1 is a lot simpler than the original series in its entirety, and should play better with topics that Junio identified:

- This series is based off a later version of 'master' that already has 'js/apply-partial-clone-filters-recursively' merged in [2].
- There is only one, trivial, conflict with 'es/superproject-aware-submodules' (both add tests to the end of t7406) [3].

Most of these patches were originally from ar/submodule-update, but because of the new organization, some commit messages have been amended to make more sense in context. However, patches 12 and 13 are new - they were added to handle the "--filter" option introduced by 'js/apply-partial-clone-filters-recursively'.

Cc-ed Josh, who might be interested in "--filter" changes e.g. the new tests.

[1] https://lore.kernel.org/git/[email protected]
[2] This also fixes some trivial merge conflicts with 'master'.
[3] Part 2 has nontrival conflicts though. Offline, Emily mentioned that conflicts might go away in the next iteration of 'es/superproject-aware-submodules', but if not, the next round of patches will probably be based on a merge of this series + 'es/superproject-aware-submodules'.

= Patch summary

I'm not certain whether to keep patch 13, see the extra discussion in the --- description for details.

- Patch 1 adds extra tests to "git submodule update" to make sure we don't break anything
- Patch 2 removes dead code that used to be part of "git submodule update"
- Patch 3 prepares for later changes by introducing the C function that will hold most of the newly converted code
- Patch 4 moves run-update-procedure's --suboid option into C
- Patch 5 moves ensure-core-worktree into C
- Patches 6-8 move run-update-procedure's --remote option into C
- Patches 9-11 move "git submodule update"'s --init into C
- Patches 12-13 move "git submodule update"'s --filter option into C

Atharva Raykar (3):
submodule--helper: get remote names from any repository
submodule--helper: refactor get_submodule_displaypath()
submodule--helper: allow setting superprefix for init_submodule()

Glen Choo (8):
submodule--helper: remove update-module-mode
submodule--helper: reorganize code for sh to C conversion submodule--helper run-update-procedure: remove --suboid
submodule--helper: remove ensure-core-worktree submodule--helper run-update-procedure: learn --remote submodule--helper update-clone: learn --init submodule update: add tests for --filter submodule--helper update-clone: check for --filter and --init

Ævar Arnfjörð Bjarmason (2):
submodule tests: test for init and update failure output
submodule--helper: don't use bitfield indirection for parse_options()

Quoted 10 lines
 builtin/submodule--helper.c    | 230 ++++++++++++++++++++-------------
 git-submodule.sh               |  54 +-------
 t/t7406-submodule-update.sh    |  54 +++++++-
 t/t7408-submodule-reference.sh |  14 +-
 4 files changed, 211 insertions(+), 141 deletions(-)

base-commit: 715d08a9e51251ad8290b181b6ac3b9e1f9719d7

Signature · 1 line
2.33.GIT

Ugh, this version doesn't compile (I rebased it onto 'master' and forgot to take into account ce14de03db (refs API: remove "failure_errno" from refs_resolve_ref_unsafe(), 2022-01-26)).

This version is still reviewable, but I'll run this through CI again before I send out v2.

author
#17 lore raw

v2 cover submodule: convert parts of 'update' to C

In reply to Glen Choo

Original series: https://lore.kernel.org/git/[email protected]
(I've trimmed the cc list down to the 'most interested' parties)

= Overview

This is part 1 of 2 series that will supersede ar/submodule-update (as laid out in [1]). This series prepares for the eventual conversion of "git submodule update" to C by doing 'obvious' conversions first, and leaving more involved conversions for later.

Part 1 is a lot simpler than the original series in its entirety, and should play better with topics that Junio identified:

- This series is based off a later version of 'master' that already has 'js/apply-partial-clone-filters-recursively' merged in [2].
- There is only one, trivial, conflict with 'es/superproject-aware-submodules' (both add tests to the end of t7406) [3].

Most of these patches were originally from ar/submodule-update, but because of the new organization, some commit messages have been amended to make more sense in context. However, patches 12 and 13 are new - they were added to handle the "--filter" option introduced by 'js/apply-partial-clone-filters-recursively'.

Cc-ed Josh, who might be interested in "--filter" changes e.g. the new tests.

[1] https://lore.kernel.org/git/[email protected]
[2] This also fixes some trivial merge conflicts with 'master'.
[3] Part 2 has nontrival conflicts though. Offline, Emily mentioned that conflicts might go away in the next iteration of 'es/superproject-aware-submodules', but if not, the next round of patches will probably be based on a merge of this series + 'es/superproject-aware-submodules'.

= Patch summary

I'm not certain whether to keep patch 13, see the extra discussion in the --- description for details.

- Patch 1 adds extra tests to "git submodule update" to make sure we don't break anything
- Patch 2 removes dead code that used to be part of "git submodule update"
- Patch 3 prepares for later changes by introducing the C function that will hold most of the newly converted code
- Patch 4 moves run-update-procedure's --suboid option into C
- Patch 5 moves ensure-core-worktree into C
- Patches 6-8 move run-update-procedure's --remote option into C
- Patches 9-11 move "git submodule update"'s --init into C
- Patches 12-13 move "git submodule update"'s --filter option into C

= Changes

Since v1:
- Fix compilation error due to bad rebase
- Remove accidentally included tests
- Fix a NEEDSWORK (it was a leftover from ar/submodule-update)

Atharva Raykar (3):
submodule--helper: get remote names from any repository
submodule--helper: refactor get_submodule_displaypath()
submodule--helper: allow setting superprefix for init_submodule()

Glen Choo (8):
submodule--helper: remove update-module-mode
submodule--helper: reorganize code for sh to C conversion submodule--helper run-update-procedure: remove --suboid
submodule--helper: remove ensure-core-worktree submodule--helper run-update-procedure: learn --remote submodule--helper update-clone: learn --init submodule update: add tests for --filter submodule--helper update-clone: check for --filter and --init

Ævar Arnfjörð Bjarmason (2):
submodule tests: test for init and update failure output
submodule--helper: don't use bitfield indirection for parse_options()

 builtin/submodule--helper.c    | 246 +++++++++++++++++++--------------
 git-submodule.sh               |  54 +-------
 t/t7406-submodule-update.sh    |  27 +++-
 t/t7408-submodule-reference.sh |  14 +-
 4 files changed, 182 insertions(+), 159 deletions(-)

Range-diff against v1:

 1:  6138f4682c =  1:  6138f4682c submodule tests: test for init and update failure output
 2:  6c83c78819 =  2:  6c83c78819 submodule--helper: remove update-module-mode
 3:  9524986096 =  3:  9524986096 submodule--helper: reorganize code for sh to C conversion
 4:  f42f3de2b7 =  4:  f42f3de2b7 submodule--helper run-update-procedure: remove --suboid
 5:  b0a0cae633 =  5:  b0a0cae633 submodule--helper: remove ensure-core-worktree
 6:  976cfa6d24 !  6:  3bde7ccd61 submodule--helper: get remote names from any repository
    @@ builtin/submodule--helper.c
      	struct strbuf sb = STRBUF_INIT;
     -	const char *refname = resolve_ref_unsafe("HEAD", 0, NULL, NULL);
     +	struct ref_store *store = get_main_ref_store(repo);
    -+	int ignore_errno;
     +	const char *refname = refs_resolve_ref_unsafe(store, "HEAD", 0, NULL,
    -+						      NULL, &ignore_errno);
    ++						      NULL);
      
      	if (!refname)
      		die(_("No such ref: %s"), "HEAD");
 7:  4a3c07aa4f =  7:  3b2caf7a35 submodule--helper: don't use bitfield indirection for parse_options()
 8:  bbfc2278c5 !  8:  81e9da8d42 submodule--helper run-update-procedure: learn --remote
    @@ Commit message
     
         Teach run-update-procedure to handle --remote instead of parsing
         --remote in git-submodule.sh. As a result, "git submodule--helper
    -    print-default-remote" has no more callers, so remove it.
    +    [print-default-remote|remote-branch]" have no more callers, so remove
    +    them.
     
         Signed-off-by: Glen Choo <[email protected]>
     
    @@ builtin/submodule--helper.c: static int run_update_procedure(int argc, const cha
      		OPT_END()
      	};
      
    +@@ builtin/submodule--helper.c: static const char *remote_submodule_branch(const char *path)
    + 	return branch;
    + }
    + 
    +-static int resolve_remote_submodule_branch(int argc, const char **argv,
    +-		const char *prefix)
    +-{
    +-	const char *ret;
    +-	struct strbuf sb = STRBUF_INIT;
    +-	if (argc != 2)
    +-		die("submodule--helper remote-branch takes exactly one arguments, got %d", argc);
    +-
    +-	ret = remote_submodule_branch(argv[1]);
    +-	if (!ret)
    +-		die("submodule %s doesn't exist", argv[1]);
    +-
    +-	printf("%s", ret);
    +-	strbuf_release(&sb);
    +-	return 0;
    +-}
    +-
    + static int push_check(int argc, const char **argv, const char *prefix)
    + {
    + 	struct remote *remote;
     @@ builtin/submodule--helper.c: static int update_submodule2(struct update_data *update_data)
      		die(_("Unable to find current revision in submodule path '%s'"),
      			update_data->displaypath);
    @@ builtin/submodule--helper.c: static struct cmd_struct commands[] = {
      	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
      	{"deinit", module_deinit, 0},
      	{"summary", module_summary, SUPPORT_SUPER_PREFIX},
    -+	/* NEEDSWORK: remote-branch is also obsolete */
    - 	{"remote-branch", resolve_remote_submodule_branch, 0},
    +-	{"remote-branch", resolve_remote_submodule_branch, 0},
      	{"push-check", push_check, 0},
      	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
    + 	{"is-active", is_active, 0},
     
      ## git-submodule.sh ##
     @@ git-submodule.sh: cmd_deinit()
 9:  765cd21505 =  9:  122da84ce4 submodule--helper: refactor get_submodule_displaypath()
10:  0c5e922f32 = 10:  fd52d6a2c3 submodule--helper: allow setting superprefix for init_submodule()
11:  287f36122a = 11:  9422c2ecac submodule--helper update-clone: learn --init
12:  91cac6e236 ! 12:  0a3e93998d submodule update: add tests for --filter
    @@ t/t7406-submodule-update.sh: test_expect_success 'submodule update --quiet passe
      	)
      '
      
    -+test_expect_success 'submodule update adds superproject gitdir to older repos' '
    -+	(cd super &&
    -+	 git -C submodule config --unset submodule.superprojectGitdir &&
    -+	 git submodule update &&
    -+	 test-tool path-utils relative_path \
    -+		"$(git rev-parse --absolute-git-dir)" \
    -+		"$(git -C submodule rev-parse --absolute-git-dir)" >expect &&
    -+	 git -C submodule config submodule.superprojectGitdir >actual &&
    -+	 test_cmp expect actual
    -+	)
    -+'
    -+
    -+test_expect_success 'submodule update uses config.worktree if applicable' '
    -+	(cd super &&
    -+	 git -C submodule config --unset submodule.superprojectGitDir &&
    -+	 git -C submodule config extensions.worktreeConfig true &&
    -+	 git submodule update &&
    -+	 test-tool path-utils relative_path \
    -+		"$(git rev-parse --absolute-git-dir)" \
    -+		"$(git -C submodule rev-parse --absolute-git-dir)" >expect &&
    -+	 git -C submodule config submodule.superprojectGitdir >actual &&
    -+	 test_cmp expect actual &&
    -+
    -+	 test_file_not_empty "$(git -C submodule rev-parse --absolute-git-dir)/config.worktree"
    -+	)
    -+'
    -+
     +test_expect_success 'submodule update --filter requires --init' '
     +	test_must_fail git -C super submodule update --filter blob:none 2>err &&
     +	grep "usage:" err

13: f637c23a48 = 13: 6e1ef27191 submodule--helper update-clone: check for --filter and --init

base-commit: 715d08a9e51251ad8290b181b6ac3b9e1f9719d7

Signature · 1 line
2.33.GIT
author
#18 lore raw

PATCH v2 1/13 submodule tests: test for init and update failure output

From: Ævar Arnfjörð Bjarmason <[email protected]>

Amend some submodule tests to test for the failure output of "git submodule [update|init]". The lack of such tests hid a regression in an earlier version of a subsequent commit.

---
 t/t7406-submodule-update.sh    | 14 ++++++++++++--
 t/t7408-submodule-reference.sh | 14 +++++++++++++-
 2 files changed, 25 insertions(+), 3 deletions(-)
t/t7406-submodule-update.sh +12 -2
index 11cccbb333..7764c1c3cb 100755
@@ -205,8 +205,18 @@ test_expect_success 'submodule update should fail due to local changes' '
 	 (cd submodule &&
 	  compare_head
 	 ) &&
-	 test_must_fail git submodule update submodule
-	)
+	 test_must_fail git submodule update submodule 2>../actual.raw
+	) &&
+	sed "s/^> //" >expect <<-\EOF &&
+	> error: Your local changes to the following files would be overwritten by checkout:
+	> 	file
+	> Please commit your changes or stash them before you switch branches.
+	> Aborting
+	> fatal: Unable to checkout OID in submodule path '\''submodule'\''
+	EOF
+	sed -e "s/checkout $SQ[^$SQ]*$SQ/checkout OID/" <actual.raw >actual &&
+	test_cmp expect actual
+
 '
 test_expect_success 'submodule update should throw away changes with --force ' '
 	(cd super &&
t/t7408-submodule-reference.sh +13 -1
index a3892f494b..c3a4545510 100755
@@ -193,7 +193,19 @@ test_expect_success 'missing nested submodule alternate fails clone and submodul
 		cd supersuper-clone &&
 		check_that_two_of_three_alternates_are_used &&
 		# update of the submodule fails
-		test_must_fail git submodule update --init --recursive
+		cat >expect <<-\EOF &&
+		fatal: submodule '\''sub'\'' cannot add alternate: path ... does not exist
+		Failed to clone '\''sub'\''. Retry scheduled
+		fatal: submodule '\''sub-dissociate'\'' cannot add alternate: path ... does not exist
+		Failed to clone '\''sub-dissociate'\''. Retry scheduled
+		fatal: submodule '\''sub'\'' cannot add alternate: path ... does not exist
+		Failed to clone '\''sub'\'' a second time, aborting
+		fatal: Failed to recurse into submodule path ...
+		EOF
+		test_must_fail git submodule update --init --recursive 2>err &&
+		grep -e fatal: -e ^Failed err >actual.raw &&
+		sed -e "s/path $SQ[^$SQ]*$SQ/path .../" <actual.raw >actual &&
+		test_cmp expect actual
 	)
 '
 
Signature · 1 line
2.33.GIT
author
#19 lore raw

PATCH v2 2/13 submodule--helper: remove update-module-mode

In reply to Glen Choo

This is dead code - it has not been used since c51f8f94e5 (submodule--helper: run update procedures from C, 2021-08-24).

---
 builtin/submodule--helper.c | 24 ------------------------
 1 file changed, 24 deletions(-)
builtin/submodule--helper.c +0 -24
index eeacefcc38..c11ee1ea2b 100644
@@ -1957,29 +1957,6 @@ static void determine_submodule_update_strategy(struct repository *r,
 	free(key);
 }
 
-static int module_update_module_mode(int argc, const char **argv, const char *prefix)
-{
-	const char *path, *update = NULL;
-	int just_cloned;
-	struct submodule_update_strategy update_strategy = { .type = SM_UPDATE_CHECKOUT };
-
-	if (argc < 3 || argc > 4)
-		die("submodule--helper update-module-clone expects <just-cloned> <path> [<update>]");
-
-	just_cloned = git_config_int("just_cloned", argv[1]);
-	path = argv[2];
-
-	if (argc == 4)
-		update = argv[3];
-
-	determine_submodule_update_strategy(the_repository,
-					    just_cloned, path, update,
-					    &update_strategy);
-	fputs(submodule_strategy_to_string(&update_strategy), stdout);
-
-	return 0;
-}
-
 struct update_clone_data {
 	const struct submodule *sub;
 	struct object_id oid;
@@ -3430,7 +3407,6 @@ static struct cmd_struct commands[] = {
 	{"name", module_name, 0},
 	{"clone", module_clone, 0},
 	{"add", module_add, SUPPORT_SUPER_PREFIX},
-	{"update-module-mode", module_update_module_mode, 0},
 	{"update-clone", update_clone, 0},
 	{"run-update-procedure", run_update_procedure, 0},
 	{"ensure-core-worktree", ensure_core_worktree, 0},
Signature · 1 line
2.33.GIT
author
#20 lore raw

PATCH v2 3/13 submodule--helper: reorganize code for sh to C conversion

In reply to Glen Choo

Introduce a function, update_submodule2(), that will implement the functionality of run-update-procedure and its surrounding shell code in submodule.sh. This name is temporary; it will replace update_submodule() when the sh to C conversion is complete.

---
 builtin/submodule--helper.c | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)
builtin/submodule--helper.c +21 -5
index c11ee1ea2b..1b67a3887c 100644
@@ -2452,6 +2452,16 @@ static int do_run_update_procedure(struct update_data *ud)
 	return run_update_command(ud, subforce);
 }
 
+/*
+ * NEEDSWORK: As we convert "git submodule update" to C,
+ * update_submodule2() will invoke more and more functions, making it
+ * difficult to preserve the function ordering without forward
+ * declarations.
+ *
+ * When the conversion is complete, this forward declaration will be
+ * unnecessary and should be removed.
+ */
+static int update_submodule2(struct update_data *update_data);
 static void update_submodule(struct update_clone_data *ucd)
 {
 	fprintf(stdout, "dummy %s %d\t%s\n",
@@ -2618,11 +2628,7 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)
 					    &update_data.update_strategy);
 
 	free(prefixed_path);
-
-	if (!oideq(&update_data.oid, &update_data.suboid) || update_data.force)
-		return do_run_update_procedure(&update_data);
-
-	return 3;
+	return update_submodule2(&update_data);
 }
 
 static int resolve_relative_path(int argc, const char **argv, const char *prefix)
@@ -3022,6 +3028,16 @@ static int module_create_branch(int argc, const char **argv, const char *prefix)
 				    force, reflog, quiet, track, dry_run);
 	return 0;
 }
+
+/* NEEDSWORK: this is a temporary name until we delete update_submodule() */
+static int update_submodule2(struct update_data *update_data)
+{
+	if (!oideq(&update_data->oid, &update_data->suboid) || update_data->force)
+		return do_run_update_procedure(update_data);
+
+	return 3;
+}
+
 struct add_data {
 	const char *prefix;
 	const char *branch;
Signature · 1 line
2.33.GIT
author
#21 lore raw

PATCH v2 4/13 submodule--helper run-update-procedure: remove --suboid

In reply to Glen Choo

Teach run-update-procedure to determine the oid of the submodule's HEAD instead of doing it in git-subomdule.sh.

---
 builtin/submodule--helper.c | 9 ++++++---
 git-submodule.sh            | 8 +-------
 2 files changed, 7 insertions(+), 10 deletions(-)
builtin/submodule--helper.c +6 -3
index 1b67a3887c..77ca4270f4 100644
@@ -2594,9 +2594,6 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)
 		OPT_CALLBACK_F(0, "oid", &update_data.oid, N_("sha1"),
 			       N_("SHA1 expected by superproject"), PARSE_OPT_NONEG,
 			       parse_opt_object_id),
-		OPT_CALLBACK_F(0, "suboid", &update_data.suboid, N_("subsha1"),
-			       N_("SHA1 of submodule's HEAD"), PARSE_OPT_NONEG,
-			       parse_opt_object_id),
 		OPT_END()
 	};
 
@@ -3032,6 +3029,12 @@ static int module_create_branch(int argc, const char **argv, const char *prefix)
 /* NEEDSWORK: this is a temporary name until we delete update_submodule() */
 static int update_submodule2(struct update_data *update_data)
 {
+	if (update_data->just_cloned)
+		oidcpy(&update_data->suboid, null_oid());
+	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD", &update_data->suboid))
+		die(_("Unable to find current revision in submodule path '%s'"),
+			update_data->displaypath);
+
 	if (!oideq(&update_data->oid, &update_data->suboid) || update_data->force)
 		return do_run_update_procedure(update_data);
 
git-submodule.sh +1 -7
index 87772ac891..32a09302ab 100755
@@ -406,14 +406,9 @@ cmd_update()
 
 		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
 
-		if test $just_cloned -eq 1
+		if test $just_cloned -eq 0
 		then
-			subsha1=
-		else
 			just_cloned=
-			subsha1=$(sanitize_submodule_env; cd "$sm_path" &&
-				git rev-parse --verify HEAD) ||
-			die "fatal: $(eval_gettext "Unable to find current revision in submodule path '\$displaypath'")"
 		fi
 
 		if test -n "$remote"
@@ -441,7 +436,6 @@ cmd_update()
 			  ${update:+--update "$update"} \
 			  ${prefix:+--recursive-prefix "$prefix"} \
 			  ${sha1:+--oid "$sha1"} \
-			  ${subsha1:+--suboid "$subsha1"} \
 			  "--" \
 			  "$sm_path")
 
Signature · 1 line
2.33.GIT
author
#22 lore raw

PATCH v2 5/13 submodule--helper: remove ensure-core-worktree

In reply to Glen Choo

Move the logic of "git submodule--helper ensure-core-worktree" into run-update-procedure. Since the ensure-core-worktree command is obsolete, remove it.

---
 builtin/submodule--helper.c | 12 ++----------
 git-submodule.sh            |  2 --
 2 files changed, 2 insertions(+), 12 deletions(-)
builtin/submodule--helper.c +2 -10
index 77ca4270f4..6b473fc0d2 100644
@@ -2771,17 +2771,11 @@ static int push_check(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
-static int ensure_core_worktree(int argc, const char **argv, const char *prefix)
+static void ensure_core_worktree(const char *path)
 {
-	const char *path;
 	const char *cw;
 	struct repository subrepo;
 
-	if (argc != 2)
-		BUG("submodule--helper ensure-core-worktree <path>");
-
-	path = argv[1];
-
 	if (repo_submodule_init(&subrepo, the_repository, path, null_oid()))
 		die(_("could not get a repository handle for submodule '%s'"), path);
 
@@ -2801,8 +2795,6 @@ static int ensure_core_worktree(int argc, const char **argv, const char *prefix)
 		free(abs_path);
 		strbuf_release(&sb);
 	}
-
-	return 0;
 }
 
 static int absorb_git_dirs(int argc, const char **argv, const char *prefix)
@@ -3029,6 +3021,7 @@ static int module_create_branch(int argc, const char **argv, const char *prefix)
 /* NEEDSWORK: this is a temporary name until we delete update_submodule() */
 static int update_submodule2(struct update_data *update_data)
 {
+	ensure_core_worktree(update_data->sm_path);
 	if (update_data->just_cloned)
 		oidcpy(&update_data->suboid, null_oid());
 	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD", &update_data->suboid))
@@ -3428,7 +3421,6 @@ static struct cmd_struct commands[] = {
 	{"add", module_add, SUPPORT_SUPER_PREFIX},
 	{"update-clone", update_clone, 0},
 	{"run-update-procedure", run_update_procedure, 0},
-	{"ensure-core-worktree", ensure_core_worktree, 0},
 	{"relative-path", resolve_relative_path, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
 	{"foreach", module_foreach, SUPPORT_SUPER_PREFIX},
git-submodule.sh +0 -2
index 32a09302ab..458ce73ac6 100755
@@ -402,8 +402,6 @@ cmd_update()
 	do
 		die_if_unmatched "$quickabort" "$sha1"
 
-		git submodule--helper ensure-core-worktree "$sm_path" || exit 1
-
 		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
 
 		if test $just_cloned -eq 0
Signature · 1 line
2.33.GIT
author
#23 lore raw

PATCH v2 6/13 submodule--helper: get remote names from any repository

In reply to Glen Choo

From: Atharva Raykar <[email protected]>

get_default_remote() retrieves the name of a remote by resolving the refs from of the current repository's ref store.

Thus in order to use it for retrieving the remote name of a submodule, we have to start a new subprocess which runs from the submodule directory.

Let's instead introduce a function called repo_get_default_remote() which takes any repository object and retrieves the remote accordingly.

get_default_remote() is then defined as a call to repo_get_default_remote() with 'the_repository' passed to it.

Now that we have repo_get_default_remote(), we no longer have to start a subprocess that called submodule--helper get-default-remote from within the submodule directory.

So let's make a function called get_default_remote_submodule() which takes a submodule path, and returns the default remote for that submodule, all within the same process.

We can now use this function to save an unnecessary subprocess spawn in sync_submodule(), and also in the next patch, which will require this functionality.

---
 builtin/submodule--helper.c | 38 ++++++++++++++++++++++---------------
 1 file changed, 23 insertions(+), 15 deletions(-)
builtin/submodule--helper.c +23 -15
index 6b473fc0d2..a58df3e007 100644
@@ -31,11 +31,13 @@
 typedef void (*each_submodule_fn)(const struct cache_entry *list_item,
 				  void *cb_data);
 
-static char *get_default_remote(void)
+static char *repo_get_default_remote(struct repository *repo)
 {
 	char *dest = NULL, *ret;
 	struct strbuf sb = STRBUF_INIT;
-	const char *refname = resolve_ref_unsafe("HEAD", 0, NULL, NULL);
+	struct ref_store *store = get_main_ref_store(repo);
+	const char *refname = refs_resolve_ref_unsafe(store, "HEAD", 0, NULL,
+						      NULL);
 
 	if (!refname)
 		die(_("No such ref: %s"), "HEAD");
@@ -48,7 +50,7 @@ static char *get_default_remote(void)
 		die(_("Expecting a full ref name, got %s"), refname);
 
 	strbuf_addf(&sb, "branch.%s.remote", refname);
-	if (git_config_get_string(sb.buf, &dest))
+	if (repo_config_get_string(repo, sb.buf, &dest))
 		ret = xstrdup("origin");
 	else
 		ret = dest;
@@ -57,6 +59,19 @@ static char *get_default_remote(void)
 	return ret;
 }
 
+static char *get_default_remote_submodule(const char *module_path)
+{
+	struct repository subrepo;
+
+	repo_submodule_init(&subrepo, the_repository, module_path, null_oid());
+	return repo_get_default_remote(&subrepo);
+}
+
+static char *get_default_remote(void)
+{
+	return repo_get_default_remote(the_repository);
+}
+
 static int print_default_remote(int argc, const char **argv, const char *prefix)
 {
 	char *remote;
@@ -1343,9 +1358,8 @@ static void sync_submodule(const char *path, const char *prefix,
 {
 	const struct submodule *sub;
 	char *remote_key = NULL;
-	char *sub_origin_url, *super_config_url, *displaypath;
+	char *sub_origin_url, *super_config_url, *displaypath, *default_remote;
 	struct strbuf sb = STRBUF_INIT;
-	struct child_process cp = CHILD_PROCESS_INIT;
 	char *sub_config_path = NULL;
 
 	if (!is_submodule_active(the_repository, path))
@@ -1384,21 +1398,15 @@ static void sync_submodule(const char *path, const char *prefix,
 	if (!is_submodule_populated_gently(path, NULL))
 		goto cleanup;
 
-	prepare_submodule_repo_env(&cp.env_array);
-	cp.git_cmd = 1;
-	cp.dir = path;
-	strvec_pushl(&cp.args, "submodule--helper",
-		     "print-default-remote", NULL);
-
 	strbuf_reset(&sb);
-	if (capture_command(&cp, &sb, 0))
+	default_remote = get_default_remote_submodule(path);
+	if (!default_remote)
 		die(_("failed to get the default remote for submodule '%s'"),
 		      path);
 
-	strbuf_strip_suffix(&sb, "\n");
-	remote_key = xstrfmt("remote.%s.url", sb.buf);
+	remote_key = xstrfmt("remote.%s.url", default_remote);
+	free(default_remote);
 
-	strbuf_reset(&sb);
 	submodule_to_gitdir(&sb, path);
 	strbuf_addstr(&sb, "/config");
 
Signature · 1 line
2.33.GIT
#24 lore raw

Re: [PATCH v2 06/13] submodule--helper: get remote names from any repository

Quoted 5 lines · Glen Choo

From: Atharva Raykar <[email protected]>
[...]
We can now use this function to save an unnecessary subprocess spawn in sync_submodule(), and also in the next patch, which will require this functionality.

The "next patch" is now an intermediate cleanup, so narrowly: s/next/a subsequent commit/ or similar.

author
#25 lore raw

PATCH v2 7/13 submodule--helper: don't use bitfield indirection for parse_options()

In reply to Glen Choo

From: Ævar Arnfjörð Bjarmason <[email protected]>

Do away with the indirection of local variables added in c51f8f94e5b (submodule--helper: run update procedures from C, 2021-08-24).

These were only needed because in C you can't get a pointer to a single bit, so we were using intermediate variables instead.

---
 builtin/submodule--helper.c | 23 ++++++++++-------------
 1 file changed, 10 insertions(+), 13 deletions(-)
builtin/submodule--helper.c +10 -13
index a58df3e007..3a96c35b86 100644
@@ -2023,10 +2023,10 @@ struct update_data {
 	struct object_id suboid;
 	struct submodule_update_strategy update_strategy;
 	int depth;
-	unsigned int force: 1;
-	unsigned int quiet: 1;
-	unsigned int nofetch: 1;
-	unsigned int just_cloned: 1;
+	unsigned int force;
+	unsigned int quiet;
+	unsigned int nofetch;
+	unsigned int just_cloned;
 };
 #define UPDATE_DATA_INIT { .update_strategy = SUBMODULE_UPDATE_STRATEGY_INIT }
 
@@ -2578,16 +2578,17 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 
 static int run_update_procedure(int argc, const char **argv, const char *prefix)
 {
-	int force = 0, quiet = 0, nofetch = 0, just_cloned = 0;
 	char *prefixed_path, *update = NULL;
 	struct update_data update_data = UPDATE_DATA_INIT;
 
 	struct option options[] = {
-		OPT__QUIET(&quiet, N_("suppress output for update by rebase or merge")),
-		OPT__FORCE(&force, N_("force checkout updates"), 0),
-		OPT_BOOL('N', "no-fetch", &nofetch,
+		OPT__QUIET(&update_data.quiet,
+			   N_("suppress output for update by rebase or merge")),
+		OPT__FORCE(&update_data.force, N_("force checkout updates"),
+			   0),
+		OPT_BOOL('N', "no-fetch", &update_data.nofetch,
 			 N_("don't fetch new objects from the remote site")),
-		OPT_BOOL(0, "just-cloned", &just_cloned,
+		OPT_BOOL(0, "just-cloned", &update_data.just_cloned,
 			 N_("overrides update mode in case the repository is a fresh clone")),
 		OPT_INTEGER(0, "depth", &update_data.depth, N_("depth for shallow fetch")),
 		OPT_STRING(0, "prefix", &prefix,
@@ -2615,10 +2616,6 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)
 	if (argc != 1)
 		usage_with_options(usage, options);
 
-	update_data.force = !!force;
-	update_data.quiet = !!quiet;
-	update_data.nofetch = !!nofetch;
-	update_data.just_cloned = !!just_cloned;
 	update_data.sm_path = argv[0];
 
 	if (update_data.recursive_prefix)
Signature · 1 line
2.33.GIT
author
#26 lore raw

PATCH v2 8/13 submodule--helper run-update-procedure: learn --remote

In reply to Glen Choo

Teach run-update-procedure to handle --remote instead of parsing --remote in git-submodule.sh. As a result, "git submodule--helper [print-default-remote|remote-branch]" have no more callers, so remove them.

---
 builtin/submodule--helper.c | 56 +++++++++++++++----------------------
 git-submodule.sh            | 30 +-------------------
 2 files changed, 23 insertions(+), 63 deletions(-)
builtin/submodule--helper.c +22 -34
index 3a96c35b86..99341fb343 100644
@@ -72,21 +72,6 @@ static char *get_default_remote(void)
 	return repo_get_default_remote(the_repository);
 }
 
-static int print_default_remote(int argc, const char **argv, const char *prefix)
-{
-	char *remote;
-
-	if (argc != 1)
-		die(_("submodule--helper print-default-remote takes no arguments"));
-
-	remote = get_default_remote();
-	if (remote)
-		printf("%s\n", remote);
-
-	free(remote);
-	return 0;
-}
-
 static int starts_with_dot_slash(const char *str)
 {
 	return str[0] == '.' && is_dir_sep(str[1]);
@@ -2027,6 +2012,7 @@ struct update_data {
 	unsigned int quiet;
 	unsigned int nofetch;
 	unsigned int just_cloned;
+	unsigned int remote;
 };
 #define UPDATE_DATA_INIT { .update_strategy = SUBMODULE_UPDATE_STRATEGY_INIT }
 
@@ -2603,6 +2589,8 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)
 		OPT_CALLBACK_F(0, "oid", &update_data.oid, N_("sha1"),
 			       N_("SHA1 expected by superproject"), PARSE_OPT_NONEG,
 			       parse_opt_object_id),
+		OPT_BOOL(0, "remote", &update_data.remote,
+			 N_("use SHA-1 of submodule's remote tracking branch")),
 		OPT_END()
 	};
 
@@ -2682,23 +2670,6 @@ static const char *remote_submodule_branch(const char *path)
 	return branch;
 }
 
-static int resolve_remote_submodule_branch(int argc, const char **argv,
-		const char *prefix)
-{
-	const char *ret;
-	struct strbuf sb = STRBUF_INIT;
-	if (argc != 2)
-		die("submodule--helper remote-branch takes exactly one arguments, got %d", argc);
-
-	ret = remote_submodule_branch(argv[1]);
-	if (!ret)
-		die("submodule %s doesn't exist", argv[1]);
-
-	printf("%s", ret);
-	strbuf_release(&sb);
-	return 0;
-}
-
 static int push_check(int argc, const char **argv, const char *prefix)
 {
 	struct remote *remote;
@@ -3033,6 +3004,25 @@ static int update_submodule2(struct update_data *update_data)
 		die(_("Unable to find current revision in submodule path '%s'"),
 			update_data->displaypath);
 
+	if (update_data->remote) {
+		char *remote_name = get_default_remote_submodule(update_data->sm_path);
+		const char *branch = remote_submodule_branch(update_data->sm_path);
+		char *remote_ref = xstrfmt("refs/remotes/%s/%s", remote_name, branch);
+
+		if (!update_data->nofetch) {
+			if (fetch_in_submodule(update_data->sm_path, update_data->depth,
+					      0, NULL))
+				die(_("Unable to fetch in submodule path '%s'"),
+				    update_data->sm_path);
+		}
+
+		if (resolve_gitlink_ref(update_data->sm_path, remote_ref, &update_data->oid))
+			die(_("Unable to find %s revision in submodule path '%s'"),
+			    remote_ref, update_data->sm_path);
+
+		free(remote_ref);
+	}
+
 	if (!oideq(&update_data->oid, &update_data->suboid) || update_data->force)
 		return do_run_update_procedure(update_data);
 
@@ -3431,11 +3421,9 @@ static struct cmd_struct commands[] = {
 	{"foreach", module_foreach, SUPPORT_SUPER_PREFIX},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
-	{"print-default-remote", print_default_remote, 0},
 	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
 	{"deinit", module_deinit, 0},
 	{"summary", module_summary, SUPPORT_SUPER_PREFIX},
-	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
 	{"is-active", is_active, 0},
git-submodule.sh +1 -29
index 458ce73ac6..23ebd90892 100755
@@ -247,20 +247,6 @@ cmd_deinit()
 	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper deinit ${GIT_QUIET:+--quiet} ${force:+--force} ${deinit_all:+--all} -- "$@"
 }
 
-# usage: fetch_in_submodule <module_path> [<depth>] [<sha1>]
-# Because arguments are positional, use an empty string to omit <depth>
-# but include <sha1>.
-fetch_in_submodule () (
-	sanitize_submodule_env &&
-	cd "$1" &&
-	if test $# -eq 3
-	then
-		echo "$3" | git fetch ${GIT_QUIET:+--quiet} --stdin ${2:+"$2"}
-	else
-		git fetch ${GIT_QUIET:+--quiet} ${2:+"$2"}
-	fi
-)
-
 #
 # Update each submodule path to correct revision, using clone and checkout as needed
 #
@@ -409,21 +395,6 @@ cmd_update()
 			just_cloned=
 		fi
 
-		if test -n "$remote"
-		then
-			branch=$(git submodule--helper remote-branch "$sm_path")
-			if test -z "$nofetch"
-			then
-				# Fetch remote before determining tracking $sha1
-				fetch_in_submodule "$sm_path" $depth ||
-				die "fatal: $(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
-			fi
-			remote_name=$(sanitize_submodule_env; cd "$sm_path" && git submodule--helper print-default-remote)
-			sha1=$(sanitize_submodule_env; cd "$sm_path" &&
-				git rev-parse --verify "${remote_name}/${branch}") ||
-			die "fatal: $(eval_gettext "Unable to find current \${remote_name}/\${branch} revision in submodule path '\$sm_path'")"
-		fi
-
 		out=$(git submodule--helper run-update-procedure \
 			  ${wt_prefix:+--prefix "$wt_prefix"} \
 			  ${GIT_QUIET:+--quiet} \
@@ -434,6 +405,7 @@ cmd_update()
 			  ${update:+--update "$update"} \
 			  ${prefix:+--recursive-prefix "$prefix"} \
 			  ${sha1:+--oid "$sha1"} \
+			  ${remote:+--remote} \
 			  "--" \
 			  "$sm_path")
 
Signature · 1 line
2.33.GIT
author
#27 lore raw

PATCH v2 9/13 submodule--helper: refactor get_submodule_displaypath()

In reply to Glen Choo

From: Atharva Raykar <[email protected]>

We create a function called do_get_submodule_displaypath() that generates the display path required by several submodule functions, and takes a custom superprefix parameter, instead of reading it from the environment.

We then redefine the existing get_submodule_displaypath() function as a call to this new function, where the superprefix is obtained from the environment.

---
 builtin/submodule--helper.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
builtin/submodule--helper.c +8 -4
index 99341fb343..11afdeea8a 100644
@@ -247,11 +247,8 @@ static int resolve_relative_url_test(int argc, const char **argv, const char *pr
 	return 0;
 }
 
-/* the result should be freed by the caller. */
-static char *get_submodule_displaypath(const char *path, const char *prefix)
+static char *do_get_submodule_displaypath(const char *path, const char *prefix, const char *super_prefix)
 {
-	const char *super_prefix = get_super_prefix();
-
 	if (prefix && super_prefix) {
 		BUG("cannot have prefix '%s' and superprefix '%s'",
 		    prefix, super_prefix);
@@ -267,6 +264,13 @@ static char *get_submodule_displaypath(const char *path, const char *prefix)
 	}
 }
 
+/* the result should be freed by the caller. */
+static char *get_submodule_displaypath(const char *path, const char *prefix)
+{
+	const char *super_prefix = get_super_prefix();
+	return do_get_submodule_displaypath(path, prefix, super_prefix);
+}
+
 static char *compute_rev_name(const char *sub_path, const char* object_id)
 {
 	struct strbuf sb = STRBUF_INIT;
Signature · 1 line
2.33.GIT
#28 lore raw

Re: [PATCH v2 09/13] submodule--helper: refactor get_submodule_displaypath()

Quoted 25 lines · Glen Choo

From: Atharva Raykar <[email protected]>

We create a function called do_get_submodule_displaypath() that generates the display path required by several submodule functions, and takes a custom superprefix parameter, instead of reading it from the environment.

We then redefine the existing get_submodule_displaypath() function as a call to this new function, where the superprefix is obtained from the environment.

---
 builtin/submodule--helper.c | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
builtin/submodule--helper.c +0 -0
index 99341fb343..11afdeea8a 100644

builtin/submodule--helper.c static int resolve_relative_url_test(int argc, const char **argv, const char *pr · line 250

Quoted 4 lines
builtin/submodule--helper.c +0 -1
@@ -247,11 +247,8 @@ static int resolve_relative_url_test(int argc, const char **argv, const char *pr
 	return 0;
 }
 
-/* the result should be freed by the caller. */

nit: I think it's better to retain this comment here...

builtin/submodule--helper.c static char *get_submodule_displaypath(const char *path, const char *prefix) · line 267

Quoted 13 lines
builtin/submodule--helper.c +2 -3
-static char *get_submodule_displaypath(const char *path, const char *prefix)
+static char *do_get_submodule_displaypath(const char *path, const char *prefix, const char *super_prefix)
 {
-	const char *super_prefix = get_super_prefix();
-
 	if (prefix && super_prefix) {
 		BUG("cannot have prefix '%s' and superprefix '%s'",
 		    prefix, super_prefix);
@@ -267,6 +264,13 @@ static char *get_submodule_displaypath(const char *path, const char *prefix)
 	}
 }
 
+/* the result should be freed by the caller. */

..than have it here, where we don't do the xstrfmt() but just call this wrapper.:

Quoted 9 lines
builtin/submodule--helper.c +6 -0
+static char *get_submodule_displaypath(const char *path, const char *prefix)
+{
+	const char *super_prefix = get_super_prefix();
+	return do_get_submodule_displaypath(path, prefix, super_prefix);
+}
+
 static char *compute_rev_name(const char *sub_path, const char* object_id)
 {
 	struct strbuf sb = STRBUF_INIT;
author
#29 lore raw

PATCH v2 10/13 submodule--helper: allow setting superprefix for init_submodule()

In reply to Glen Choo

From: Atharva Raykar <[email protected]>

We allow callers of the init_submodule() function to optionally override the superprefix from the environment.

We need to enable this option because in our conversion of the update command that will follow, the '--init' option will be handled through this API. We will need to change the superprefix at that time to ensure the display paths show correctly in the output messages.

---
 builtin/submodule--helper.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
builtin/submodule--helper.c +7 -3
index 11afdeea8a..052b247726 100644
@@ -592,18 +592,22 @@ static int module_foreach(int argc, const char **argv, const char *prefix)
 
 struct init_cb {
 	const char *prefix;
+	const char *superprefix;
 	unsigned int flags;
 };
 #define INIT_CB_INIT { 0 }
 
 static void init_submodule(const char *path, const char *prefix,
-			   unsigned int flags)
+			   const char *superprefix, unsigned int flags)
 {
 	const struct submodule *sub;
 	struct strbuf sb = STRBUF_INIT;
 	char *upd = NULL, *url = NULL, *displaypath;
 
-	displaypath = get_submodule_displaypath(path, prefix);
+	/* try superprefix from the environment, if it is not passed explicitly */
+	if (!superprefix)
+		superprefix = get_super_prefix();
+	displaypath = do_get_submodule_displaypath(path, prefix, superprefix);
 
 	sub = submodule_from_path(the_repository, null_oid(), path);
 
@@ -677,7 +681,7 @@ static void init_submodule(const char *path, const char *prefix,
 static void init_submodule_cb(const struct cache_entry *list_item, void *cb_data)
 {
 	struct init_cb *info = cb_data;
-	init_submodule(list_item->name, info->prefix, info->flags);
+	init_submodule(list_item->name, info->prefix, info->superprefix, info->flags);
 }
 
 static int module_init(int argc, const char **argv, const char *prefix)
Signature · 1 line
2.33.GIT
author
#30 lore raw

PATCH v2 11/13 submodule--helper update-clone: learn --init

In reply to Glen Choo

Teach "git submodule--helper update-clone" the --init flag and remove the corresponding shell code.

When the --init flag is passed to the subcommand, we do not spawn a new subprocess and call submodule--helper init on the submodule paths, because the Git machinery is not able to pick up the configuration changes introduced by that init call. So we instead run the init_submodule_cb() callback over each submodule in the same process.

[1] https://lore.kernel.org/git/CAP8UFD0NCQ5w_3GtT_xHr35i7h8BuLX4UcHNY6VHPGREmDVObA@mail.gmail.com/

---
 builtin/submodule--helper.c | 26 ++++++++++++++++++++++++++
 git-submodule.sh            |  9 +++------
 2 files changed, 29 insertions(+), 6 deletions(-)
builtin/submodule--helper.c +26 -0
index 052b247726..2ffc070319 100644
@@ -1998,6 +1998,7 @@ struct submodule_update_clone {
 	int failed_clones_nr, failed_clones_alloc;
 
 	int max_jobs;
+	unsigned int init;
 };
 #define SUBMODULE_UPDATE_CLONE_INIT { \
 	.list = MODULE_LIST_INIT, \
@@ -2507,6 +2508,8 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	int ret;
 
 	struct option module_update_clone_options[] = {
+		OPT_BOOL(0, "init", &suc.init,
+			 N_("initialize uninitialized submodules before update")),
 		OPT_STRING(0, "prefix", &prefix,
 			   N_("path"),
 			   N_("path into the working tree")),
@@ -2565,6 +2568,29 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	if (pathspec.nr)
 		suc.warn_if_uninitialized = 1;
 
+	if (suc.init) {
+		struct module_list list = MODULE_LIST_INIT;
+		struct init_cb info = INIT_CB_INIT;
+
+		if (module_list_compute(argc, argv, suc.prefix,
+					&pathspec, &list) < 0)
+			return 1;
+
+		/*
+		 * If there are no path args and submodule.active is set then,
+		 * by default, only initialize 'active' modules.
+		 */
+		if (!argc && git_config_get_value_multi("submodule.active"))
+			module_list_active(&list);
+
+		info.prefix = suc.prefix;
+		info.superprefix = suc.recursive_prefix;
+		if (suc.quiet)
+			info.flags |= OPT_QUIET;
+
+		for_each_listed_submodule(&list, init_submodule_cb, &info);
+	}
+
 	ret = update_submodules(&suc);
 	list_objects_filter_release(&filter_options);
 	return ret;
git-submodule.sh +3 -6
index 23ebd90892..51be7c7f7e 100755
@@ -361,14 +361,11 @@ cmd_update()
 		usage
 	fi
 
-	if test -n "$init"
-	then
-		cmd_init "--" "$@" || return
-	fi
-
 	{
-	git submodule--helper update-clone ${GIT_QUIET:+--quiet} \
+	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper update-clone \
+		${GIT_QUIET:+--quiet} \
 		${progress:+"--progress"} \
+		${init:+--init} \
 		${wt_prefix:+--prefix "$wt_prefix"} \
 		${prefix:+--recursive-prefix "$prefix"} \
 		${update:+--update "$update"} \
Signature · 1 line
2.33.GIT
author
#31 lore raw

PATCH v2 12/13 submodule update: add tests for --filter

In reply to Glen Choo

Test the "--filter" option to make sure we don't break anything while refactoring "git submodule update".

---
 t/t7406-submodule-update.sh | 13 +++++++++++++
 1 file changed, 13 insertions(+)
t/t7406-submodule-update.sh +13 -0
index 7764c1c3cb..6cc60ff5e5 100755
@@ -1071,4 +1071,17 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s
 	)
 '
 
+test_expect_success 'submodule update --filter requires --init' '
+	test_must_fail git -C super submodule update --filter blob:none 2>err &&
+	grep "usage:" err
+'
+
+test_expect_success 'submodule update --filter sets partial clone settings' '
+	test_when_finished "rm -rf super-filter" &&
+	git clone cloned super-filter &&
+	git -C super-filter submodule update --init --filter blob:none &&
+	test_cmp_config -C super-filter/submodule true remote.origin.promisor &&
+	test_cmp_config -C super-filter/submodule blob:none remote.origin.partialclonefilter
+'
+
 test_done
Signature · 1 line
2.33.GIT
#32 lore raw

Re: [PATCH v2 12/13] submodule update: add tests for --filter

Quoted 13 lines · Glen Choo

Test the "--filter" option to make sure we don't break anything while refactoring "git submodule update".

---
 t/t7406-submodule-update.sh | 13 +++++++++++++
 1 file changed, 13 insertions(+)
t/t7406-submodule-update.sh +0 -0
index 7764c1c3cb..6cc60ff5e5 100755

t/t7406-submodule-update.sh test_expect_success 'submodule update --quiet passes quietness to fetch with a s · line 1075

Quoted 5 lines
t/t7406-submodule-update.sh +2 -0
@@ -1071,4 +1071,17 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s
 	)
 '
 
+test_expect_success 'submodule update --filter requires --init' '
+	test_must_fail git -C super submodule update --filter blob:none 2>err &&

Should be "test_expect_code 129" (presumably, or is it 128). In any case other similar "usage" test check for that:

t/t7406-submodule-update.sh test_expect_success 'submodule update --quiet passes quietness to fetch with a s · line 1076

Quoted 1 line
t/t7406-submodule-update.sh +1 -0
+	grep "usage:" err

We could retain this then, but FWIW if it's 129 other tests consider it redundant.

author
#33 lore raw

Re: [PATCH v2 12/13] submodule update: add tests for --filter

Quoted 24 lines · Ævar Arnfjörð Bjarmason
Quoted 13 lines · Glen Choo

Test the "--filter" option to make sure we don't break anything while refactoring "git submodule update".

---
 t/t7406-submodule-update.sh | 13 +++++++++++++
 1 file changed, 13 insertions(+)
t/t7406-submodule-update.sh +0 -0
index 7764c1c3cb..6cc60ff5e5 100755

t/t7406-submodule-update.sh test_expect_success 'submodule update --quiet passes quietness to fetch with a s · line 1075

Quoted 5 lines
t/t7406-submodule-update.sh +2 -0
@@ -1071,4 +1071,17 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s
 	)
 '
 
+test_expect_success 'submodule update --filter requires --init' '
+	test_must_fail git -C super submodule update --filter blob:none 2>err &&

Should be "test_expect_code 129" (presumably, or is it 128). In any case other similar "usage" test check for that:

Quoted 4 lines

t/t7406-submodule-update.sh test_expect_success 'submodule update --quiet passes quietness to fetch with a s · line 1076

Quoted 1 line
t/t7406-submodule-update.sh +1 -0
+	grep "usage:" err

We could retain this then, but FWIW if it's 129 other tests consider it redundant.

Ah, thanks for the tip. I forgot that this is how we check for usage.

author
#34 lore raw

PATCH v2 13/13 submodule--helper update-clone: check for --filter and --init

In reply to Glen Choo

"git submodule update --filter" also requires the "--init" option. Teach update-clone to do this usage check in C and remove the check from git-submodule.sh.

In addition, change update-clone's usage string so that it teaches users about "git submodule update" instead of "git submodule--helper update-clone" (the string is copied from git-submodule.sh). This should be more helpful to users since they don't invoke update-clone directly.

---
Since we expect users to act upon the usage string, I've updated it to reflect "git submodule update" [1] (since that's what users actually invoke), but I feel a bit iffy about not being able to use usage_with_options() (because the options and usage string are for different commands).

This might indicate that this is work we should put off until the conversion to C is mostly complete, but on the other hand, the usage string is still more helpful than it used to be because we never presented users with the options anyway.

[1] It's not immediately obvious which command we prefer to show - some other commands use "git submodule--helper" and others use "git submodule".

 builtin/submodule--helper.c | 20 +++++++++++++++++++-
 git-submodule.sh            |  5 -----
 2 files changed, 19 insertions(+), 6 deletions(-)
builtin/submodule--helper.c +19 -1
index 2ffc070319..3e8a05a052 100644
@@ -2543,7 +2543,12 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	};
 
 	const char *const git_submodule_helper_usage[] = {
-		N_("git submodule--helper update-clone [--prefix=<path>] [<path>...]"),
+		N_("git submodule [--quiet] update"
+		"[--init [--filter=<filter-spec>]] [--remote]"
+		"[-N|--no-fetch] [-f|--force]"
+		"[--checkout|--merge|--rebase]"
+		"[--[no-]recommend-shallow] [--reference <repository>]"
+		"[--recursive] [--[no-]single-branch] [--] [<path>...]"),
 		NULL
 	};
 	suc.prefix = prefix;
@@ -2554,6 +2559,19 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	memset(&filter_options, 0, sizeof(filter_options));
 	argc = parse_options(argc, argv, prefix, module_update_clone_options,
 			     git_submodule_helper_usage, 0);
+
+	if (filter_options.choice && !suc.init) {
+		/*
+		 * NEEDSWORK: Don't use usage_with_options() because the
+		 * usage string is for "git submodule update", but the
+		 * options are for "git submodule--helper update-clone".
+		 *
+		 * This will no longer be an issue when "update-clone"
+		 * is replaced by "git submodule--helper update".
+		 */
+		usage(git_submodule_helper_usage[0]);
+	}
+
 	suc.filter_options = &filter_options;
 
 	if (update)
git-submodule.sh +0 -5
index 51be7c7f7e..aa8bdfca9d 100755
@@ -356,11 +356,6 @@ cmd_update()
 		shift
 	done
 
-	if test -n "$filter" && test "$init" != "1"
-	then
-		usage
-	fi
-
 	{
 	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper update-clone \
 		${GIT_QUIET:+--quiet} \
Signature · 1 line
2.33.GIT
#35 lore raw

Re: [PATCH v2 13/13] submodule--helper update-clone: check for --filter and --init

Quoted 35 lines · Glen Choo

"git submodule update --filter" also requires the "--init" option. Teach update-clone to do this usage check in C and remove the check from git-submodule.sh.

In addition, change update-clone's usage string so that it teaches users about "git submodule update" instead of "git submodule--helper update-clone" (the string is copied from git-submodule.sh). This should be more helpful to users since they don't invoke update-clone directly.

---
Since we expect users to act upon the usage string, I've updated it to reflect "git submodule update" [1] (since that's what users actually invoke), but I feel a bit iffy about not being able to use usage_with_options() (because the options and usage string are for different commands).

This might indicate that this is work we should put off until the conversion to C is mostly complete, but on the other hand, the usage string is still more helpful than it used to be because we never presented users with the options anyway.

[1] It's not immediately obvious which command we prefer to show - some other commands use "git submodule--helper" and others use "git submodule".

 builtin/submodule--helper.c | 20 +++++++++++++++++++-
 git-submodule.sh            |  5 -----
 2 files changed, 19 insertions(+), 6 deletions(-)
builtin/submodule--helper.c +0 -0
index 2ffc070319..3e8a05a052 100644

builtin/submodule--helper.c static int update_clone(int argc, const char **argv, const char *prefix) · line 2551

Quoted 10 lines
builtin/submodule--helper.c +6 -1
@@ -2543,7 +2543,12 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	};
 
 	const char *const git_submodule_helper_usage[] = {
-		N_("git submodule--helper update-clone [--prefix=<path>] [<path>...]"),
+		N_("git submodule [--quiet] update"
+		"[--init [--filter=<filter-spec>]] [--remote]"
+		"[-N|--no-fetch] [-f|--force]"
+		"[--checkout|--merge|--rebase]"
+		"[--[no-]recommend-shallow] [--reference <repository>]"
+		"[--recursive] [--[no-]single-branch] [--] [<path>...]"),

Since this has <repository>, <path> etc. it should still be marked for translation with N_().

#36 lore raw

Re: [PATCH v2 13/13] submodule--helper update-clone: check for --filter and --init

Quoted 7 lines · Ævar Arnfjörð Bjarmason
Quoted 7 lines
builtin/submodule--helper.c +0 -0
index 2ffc070319..3e8a05a052 100644
@@ -2543,7 +2543,12 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	};
 
Quoted 11 lines

builtin/submodule--helper.c static int update_clone(int argc, const char **argv, const char *prefix) · line 2551

Quoted 8 lines
builtin/submodule--helper.c +6 -1
 	const char *const git_submodule_helper_usage[] = {
-		N_("git submodule--helper update-clone [--prefix=<path>] [<path>...]"),
+		N_("git submodule [--quiet] update"
+		"[--init [--filter=<filter-spec>]] [--remote]"
+		"[-N|--no-fetch] [-f|--force]"
+		"[--checkout|--merge|--rebase]"
+		"[--[no-]recommend-shallow] [--reference <repository>]"
+		"[--recursive] [--[no-]single-branch] [--] [<path>...]"),

Since this has <repository>, <path> etc. it should still be marked for translation with N_().

Yeah, that sounds like a good idea. Isn't it already inside N_()?

author
#37 lore raw

Re: [PATCH v2 13/13] submodule--helper update-clone: check for --filter and --init

Quoted 21 lines · Junio C Hamano
Quoted 7 lines · Ævar Arnfjörð Bjarmason
Quoted 7 lines
builtin/submodule--helper.c +0 -0
index 2ffc070319..3e8a05a052 100644
@@ -2543,7 +2543,12 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	};
 
Quoted 11 lines

builtin/submodule--helper.c static int update_clone(int argc, const char **argv, const char *prefix) · line 2551

Quoted 8 lines
builtin/submodule--helper.c +6 -1
 	const char *const git_submodule_helper_usage[] = {
-		N_("git submodule--helper update-clone [--prefix=<path>] [<path>...]"),
+		N_("git submodule [--quiet] update"
+		"[--init [--filter=<filter-spec>]] [--remote]"
+		"[-N|--no-fetch] [-f|--force]"
+		"[--checkout|--merge|--rebase]"
+		"[--[no-]recommend-shallow] [--reference <repository>]"
+		"[--recursive] [--[no-]single-branch] [--] [<path>...]"),

Since this has <repository>, <path> etc. it should still be marked for translation with N_().

Quoted 1 line

Yeah, that sounds like a good idea. Isn't it already inside N_()?

Did I do this correctly? e.g. an alternative interpretation is that Ævar misread this as:

  	const char *const git_submodule_helper_usage[] = {
      N_("git submodule [--quiet] update"),
      "[--init [--filter=<filter-spec>]] [--remote]",
      "[-N|--no-fetch] [-f|--force]",
      "[--checkout|--merge|--rebase]",
      "[--[no-]recommend-shallow] [--reference <repository>]",
      "[--recursive] [--[no-]single-branch] [--] [<path>...]",
#38 lore raw

Re: [PATCH v2 13/13] submodule--helper update-clone: check for --filter and --init

Quoted 28 lines · Glen Choo
Quoted 21 lines · Junio C Hamano
Quoted 7 lines · Ævar Arnfjörð Bjarmason
Quoted 7 lines
builtin/submodule--helper.c +0 -0
index 2ffc070319..3e8a05a052 100644
@@ -2543,7 +2543,12 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	};
 
Quoted 11 lines

builtin/submodule--helper.c static int update_clone(int argc, const char **argv, const char *prefix) · line 2551

Quoted 8 lines
builtin/submodule--helper.c +6 -1
 	const char *const git_submodule_helper_usage[] = {
-		N_("git submodule--helper update-clone [--prefix=<path>] [<path>...]"),
+		N_("git submodule [--quiet] update"
+		"[--init [--filter=<filter-spec>]] [--remote]"
+		"[-N|--no-fetch] [-f|--force]"
+		"[--checkout|--merge|--rebase]"
+		"[--[no-]recommend-shallow] [--reference <repository>]"
+		"[--recursive] [--[no-]single-branch] [--] [<path>...]"),

Since this has <repository>, <path> etc. it should still be marked for translation with N_().

Quoted 1 line

Yeah, that sounds like a good idea. Isn't it already inside N_()?

Did I do this correctly? e.g. an alternative interpretation is that Ævar misread this as:

Quoted 7 lines
  	const char *const git_submodule_helper_usage[] = {
      N_("git submodule [--quiet] update"),
      "[--init [--filter=<filter-spec>]] [--remote]",
      "[-N|--no-fetch] [-f|--force]",
      "[--checkout|--merge|--rebase]",
      "[--[no-]recommend-shallow] [--reference <repository>]",
      "[--recursive] [--[no-]single-branch] [--] [<path>...]",

(Even if a v3 has already been sent out). Yes, sorry. I just misread this. The pre-image is correct.

author
#39 lore raw

v3 cover submodule: convert parts of 'update' to C

In reply to Glen Choo

Original series: https://lore.kernel.org/git/[email protected]
(I've trimmed the cc list down to the 'most interested' parties)

Thanks for the input, Ævar :) This just fixes up a few small issues in the previous version.

= Patch summary

- Patch 1 adds extra tests to "git submodule update" to make sure we don't break anything
- Patch 2 removes dead code that used to be part of "git submodule update"
- Patch 3 prepares for later changes by introducing the C function that will hold most of the newly converted code
- Patch 4 moves run-update-procedure's --suboid option into C
- Patch 5 moves ensure-core-worktree into C
- Patches 6-8 move run-update-procedure's --remote option into C
- Patches 9-11 move "git submodule update"'s --init into C
- Patches 12-13 move "git submodule update"'s --filter option into C

= Changes

Since v2:
- Patch 6: Fix a stale commit message that said 'in the next patch'.
- Patch 9: Fix an overly long line (spotted by Ævar in an older iteration of ar/submodule-update)
- Patch 12: Test for usage using test_expect_code
- Patch 13: Add missing spaces to the usage string

Since v1:
- Fix compilation error due to bad rebase
- Remove accidentally included tests
- Fix a NEEDSWORK (it was a leftover from ar/submodule-update)

Atharva Raykar (3):
submodule--helper: get remote names from any repository
submodule--helper: refactor get_submodule_displaypath()
submodule--helper: allow setting superprefix for init_submodule()

Glen Choo (8):
submodule--helper: remove update-module-mode
submodule--helper: reorganize code for sh to C conversion submodule--helper run-update-procedure: remove --suboid
submodule--helper: remove ensure-core-worktree submodule--helper run-update-procedure: learn --remote submodule--helper update-clone: learn --init submodule update: add tests for --filter submodule--helper update-clone: check for --filter and --init

Ævar Arnfjörð Bjarmason (2):
submodule tests: test for init and update failure output
submodule--helper: don't use bitfield indirection for parse_options()

 builtin/submodule--helper.c    | 248 +++++++++++++++++++--------------
 git-submodule.sh               |  54 +------
 t/t7406-submodule-update.sh    |  26 +++-
 t/t7408-submodule-reference.sh |  14 +-
 4 files changed, 183 insertions(+), 159 deletions(-)

Range-diff against v2:

 1:  6138f4682c =  1:  6138f4682c submodule tests: test for init and update failure output
 2:  6c83c78819 =  2:  6c83c78819 submodule--helper: remove update-module-mode
 3:  9524986096 =  3:  9524986096 submodule--helper: reorganize code for sh to C conversion
 4:  f42f3de2b7 =  4:  f42f3de2b7 submodule--helper run-update-procedure: remove --suboid
 5:  b0a0cae633 =  5:  b0a0cae633 submodule--helper: remove ensure-core-worktree
 6:  3bde7ccd61 !  6:  8dc7bc5894 submodule--helper: get remote names from any repository

@@ Commit message
submodule, all within the same process.

We can now use this function to save an unnecessary subprocess spawn in
- sync_submodule(), and also in the next patch, which will require this
- functionality.
+ sync_submodule(), and also in a subsequent patch, which will require
+ this functionality.

 7:  3b2caf7a35 =  7:  feaf9f45d8 submodule--helper: don't use bitfield indirection for parse_options()
 8:  81e9da8d42 =  8:  91e8e1a007 submodule--helper run-update-procedure: learn --remote
 9:  122da84ce4 !  9:  aba851e71e submodule--helper: refactor get_submodule_displaypath()

@@ builtin/submodule--helper.c: static int resolve_relative_url_test(int argc, cons

-/* the result should be freed by the caller. */
-static char *get_submodule_displaypath(const char *path, const char *prefix)
-+static char *do_get_submodule_displaypath(const char *path, const char *prefix, const char *super_prefix) ++static char *do_get_submodule_displaypath(const char *path,
++ const char *prefix,
++ const char *super_prefix)
{
- const char *super_prefix = get_super_prefix();
-

10:  fd52d6a2c3 = 10:  2155c049a2 submodule--helper: allow setting superprefix for init_submodule()
11:  9422c2ecac = 11:  03bbc39a06 submodule--helper update-clone: learn --init
12:  0a3e93998d ! 12:  e49b26ad94 submodule update: add tests for --filter
    @@ t/t7406-submodule-update.sh: test_expect_success 'submodule update --quiet passe
      '
      
     +test_expect_success 'submodule update --filter requires --init' '
    -+	test_must_fail git -C super submodule update --filter blob:none 2>err &&
    -+	grep "usage:" err
    ++	test_expect_code 129 git -C super submodule update --filter blob:none
     +'
     +
     +test_expect_success 'submodule update --filter sets partial clone settings' '
13:  6e1ef27191 ! 13:  c97c97948a submodule--helper update-clone: check for --filter and --init
    @@ builtin/submodule--helper.c: static int update_clone(int argc, const char **argv
      	const char *const git_submodule_helper_usage[] = {
     -		N_("git submodule--helper update-clone [--prefix=<path>] [<path>...]"),
     +		N_("git submodule [--quiet] update"
    -+		"[--init [--filter=<filter-spec>]] [--remote]"
    -+		"[-N|--no-fetch] [-f|--force]"
    -+		"[--checkout|--merge|--rebase]"
    -+		"[--[no-]recommend-shallow] [--reference <repository>]"
    -+		"[--recursive] [--[no-]single-branch] [--] [<path>...]"),
    ++		" [--init [--filter=<filter-spec>]] [--remote]"
    ++		" [-N|--no-fetch] [-f|--force]"
    ++		" [--checkout|--merge|--rebase]"
    ++		" [--[no-]recommend-shallow] [--reference <repository>]"
    ++		" [--recursive] [--[no-]single-branch] [--] [<path>...]"),
      		NULL
      	};
      	suc.prefix = prefix;

base-commit: 715d08a9e51251ad8290b181b6ac3b9e1f9719d7

Signature · 1 line
2.33.GIT
author
#40 lore raw

PATCH v3 1/13 submodule tests: test for init and update failure output

From: Ævar Arnfjörð Bjarmason <[email protected]>

Amend some submodule tests to test for the failure output of "git submodule [update|init]". The lack of such tests hid a regression in an earlier version of a subsequent commit.

---
 t/t7406-submodule-update.sh    | 14 ++++++++++++--
 t/t7408-submodule-reference.sh | 14 +++++++++++++-
 2 files changed, 25 insertions(+), 3 deletions(-)
t/t7406-submodule-update.sh +12 -2
index 11cccbb333..7764c1c3cb 100755
@@ -205,8 +205,18 @@ test_expect_success 'submodule update should fail due to local changes' '
 	 (cd submodule &&
 	  compare_head
 	 ) &&
-	 test_must_fail git submodule update submodule
-	)
+	 test_must_fail git submodule update submodule 2>../actual.raw
+	) &&
+	sed "s/^> //" >expect <<-\EOF &&
+	> error: Your local changes to the following files would be overwritten by checkout:
+	> 	file
+	> Please commit your changes or stash them before you switch branches.
+	> Aborting
+	> fatal: Unable to checkout OID in submodule path '\''submodule'\''
+	EOF
+	sed -e "s/checkout $SQ[^$SQ]*$SQ/checkout OID/" <actual.raw >actual &&
+	test_cmp expect actual
+
 '
 test_expect_success 'submodule update should throw away changes with --force ' '
 	(cd super &&
t/t7408-submodule-reference.sh +13 -1
index a3892f494b..c3a4545510 100755
@@ -193,7 +193,19 @@ test_expect_success 'missing nested submodule alternate fails clone and submodul
 		cd supersuper-clone &&
 		check_that_two_of_three_alternates_are_used &&
 		# update of the submodule fails
-		test_must_fail git submodule update --init --recursive
+		cat >expect <<-\EOF &&
+		fatal: submodule '\''sub'\'' cannot add alternate: path ... does not exist
+		Failed to clone '\''sub'\''. Retry scheduled
+		fatal: submodule '\''sub-dissociate'\'' cannot add alternate: path ... does not exist
+		Failed to clone '\''sub-dissociate'\''. Retry scheduled
+		fatal: submodule '\''sub'\'' cannot add alternate: path ... does not exist
+		Failed to clone '\''sub'\'' a second time, aborting
+		fatal: Failed to recurse into submodule path ...
+		EOF
+		test_must_fail git submodule update --init --recursive 2>err &&
+		grep -e fatal: -e ^Failed err >actual.raw &&
+		sed -e "s/path $SQ[^$SQ]*$SQ/path .../" <actual.raw >actual &&
+		test_cmp expect actual
 	)
 '
 
Signature · 1 line
2.33.GIT
author
#41 lore raw

PATCH v3 2/13 submodule--helper: remove update-module-mode

In reply to Glen Choo

This is dead code - it has not been used since c51f8f94e5 (submodule--helper: run update procedures from C, 2021-08-24).

---
 builtin/submodule--helper.c | 24 ------------------------
 1 file changed, 24 deletions(-)
builtin/submodule--helper.c +0 -24
index eeacefcc38..c11ee1ea2b 100644
@@ -1957,29 +1957,6 @@ static void determine_submodule_update_strategy(struct repository *r,
 	free(key);
 }
 
-static int module_update_module_mode(int argc, const char **argv, const char *prefix)
-{
-	const char *path, *update = NULL;
-	int just_cloned;
-	struct submodule_update_strategy update_strategy = { .type = SM_UPDATE_CHECKOUT };
-
-	if (argc < 3 || argc > 4)
-		die("submodule--helper update-module-clone expects <just-cloned> <path> [<update>]");
-
-	just_cloned = git_config_int("just_cloned", argv[1]);
-	path = argv[2];
-
-	if (argc == 4)
-		update = argv[3];
-
-	determine_submodule_update_strategy(the_repository,
-					    just_cloned, path, update,
-					    &update_strategy);
-	fputs(submodule_strategy_to_string(&update_strategy), stdout);
-
-	return 0;
-}
-
 struct update_clone_data {
 	const struct submodule *sub;
 	struct object_id oid;
@@ -3430,7 +3407,6 @@ static struct cmd_struct commands[] = {
 	{"name", module_name, 0},
 	{"clone", module_clone, 0},
 	{"add", module_add, SUPPORT_SUPER_PREFIX},
-	{"update-module-mode", module_update_module_mode, 0},
 	{"update-clone", update_clone, 0},
 	{"run-update-procedure", run_update_procedure, 0},
 	{"ensure-core-worktree", ensure_core_worktree, 0},
Signature · 1 line
2.33.GIT
author
#42 lore raw

PATCH v3 3/13 submodule--helper: reorganize code for sh to C conversion

In reply to Glen Choo

Introduce a function, update_submodule2(), that will implement the functionality of run-update-procedure and its surrounding shell code in submodule.sh. This name is temporary; it will replace update_submodule() when the sh to C conversion is complete.

---
 builtin/submodule--helper.c | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)
builtin/submodule--helper.c +21 -5
index c11ee1ea2b..1b67a3887c 100644
@@ -2452,6 +2452,16 @@ static int do_run_update_procedure(struct update_data *ud)
 	return run_update_command(ud, subforce);
 }
 
+/*
+ * NEEDSWORK: As we convert "git submodule update" to C,
+ * update_submodule2() will invoke more and more functions, making it
+ * difficult to preserve the function ordering without forward
+ * declarations.
+ *
+ * When the conversion is complete, this forward declaration will be
+ * unnecessary and should be removed.
+ */
+static int update_submodule2(struct update_data *update_data);
 static void update_submodule(struct update_clone_data *ucd)
 {
 	fprintf(stdout, "dummy %s %d\t%s\n",
@@ -2618,11 +2628,7 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)
 					    &update_data.update_strategy);
 
 	free(prefixed_path);
-
-	if (!oideq(&update_data.oid, &update_data.suboid) || update_data.force)
-		return do_run_update_procedure(&update_data);
-
-	return 3;
+	return update_submodule2(&update_data);
 }
 
 static int resolve_relative_path(int argc, const char **argv, const char *prefix)
@@ -3022,6 +3028,16 @@ static int module_create_branch(int argc, const char **argv, const char *prefix)
 				    force, reflog, quiet, track, dry_run);
 	return 0;
 }
+
+/* NEEDSWORK: this is a temporary name until we delete update_submodule() */
+static int update_submodule2(struct update_data *update_data)
+{
+	if (!oideq(&update_data->oid, &update_data->suboid) || update_data->force)
+		return do_run_update_procedure(update_data);
+
+	return 3;
+}
+
 struct add_data {
 	const char *prefix;
 	const char *branch;
Signature · 1 line
2.33.GIT
author
#43 lore raw

PATCH v3 4/13 submodule--helper run-update-procedure: remove --suboid

In reply to Glen Choo

Teach run-update-procedure to determine the oid of the submodule's HEAD instead of doing it in git-subomdule.sh.

---
 builtin/submodule--helper.c | 9 ++++++---
 git-submodule.sh            | 8 +-------
 2 files changed, 7 insertions(+), 10 deletions(-)
builtin/submodule--helper.c +6 -3
index 1b67a3887c..77ca4270f4 100644
@@ -2594,9 +2594,6 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)
 		OPT_CALLBACK_F(0, "oid", &update_data.oid, N_("sha1"),
 			       N_("SHA1 expected by superproject"), PARSE_OPT_NONEG,
 			       parse_opt_object_id),
-		OPT_CALLBACK_F(0, "suboid", &update_data.suboid, N_("subsha1"),
-			       N_("SHA1 of submodule's HEAD"), PARSE_OPT_NONEG,
-			       parse_opt_object_id),
 		OPT_END()
 	};
 
@@ -3032,6 +3029,12 @@ static int module_create_branch(int argc, const char **argv, const char *prefix)
 /* NEEDSWORK: this is a temporary name until we delete update_submodule() */
 static int update_submodule2(struct update_data *update_data)
 {
+	if (update_data->just_cloned)
+		oidcpy(&update_data->suboid, null_oid());
+	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD", &update_data->suboid))
+		die(_("Unable to find current revision in submodule path '%s'"),
+			update_data->displaypath);
+
 	if (!oideq(&update_data->oid, &update_data->suboid) || update_data->force)
 		return do_run_update_procedure(update_data);
 
git-submodule.sh +1 -7
index 87772ac891..32a09302ab 100755
@@ -406,14 +406,9 @@ cmd_update()
 
 		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
 
-		if test $just_cloned -eq 1
+		if test $just_cloned -eq 0
 		then
-			subsha1=
-		else
 			just_cloned=
-			subsha1=$(sanitize_submodule_env; cd "$sm_path" &&
-				git rev-parse --verify HEAD) ||
-			die "fatal: $(eval_gettext "Unable to find current revision in submodule path '\$displaypath'")"
 		fi
 
 		if test -n "$remote"
@@ -441,7 +436,6 @@ cmd_update()
 			  ${update:+--update "$update"} \
 			  ${prefix:+--recursive-prefix "$prefix"} \
 			  ${sha1:+--oid "$sha1"} \
-			  ${subsha1:+--suboid "$subsha1"} \
 			  "--" \
 			  "$sm_path")
 
Signature · 1 line
2.33.GIT
#44 lore raw

Re: [PATCH v3 04/13] submodule--helper run-update-procedure: remove --suboid

Quoted 2 lines · Glen Choo

Teach run-update-procedure to determine the oid of the submodule's HEAD instead of doing it in git-subomdule.sh.

subomdule -> submodule.

builtin/submodule--helper.c static int module_create_branch(int argc, const char **argv, const char *prefix) · line 3039

Quoted 12 lines
builtin/submodule--helper.c +6 -0
@@ -3032,6 +3029,12 @@ static int module_create_branch(int argc, const char **argv, const char *prefix)
 /* NEEDSWORK: this is a temporary name until we delete update_submodule() */
 static int update_submodule2(struct update_data *update_data)
 {
+	if (update_data->just_cloned)
+		oidcpy(&update_data->suboid, null_oid());
+	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD", &update_data->suboid))
+		die(_("Unable to find current revision in submodule path '%s'"),
+			update_data->displaypath);
+
 	if (!oideq(&update_data->oid, &update_data->suboid) || update_data->force)
 		return do_run_update_procedure(update_data);

Makes sense.

author
#45 lore raw

PATCH v3 5/13 submodule--helper: remove ensure-core-worktree

In reply to Glen Choo

Move the logic of "git submodule--helper ensure-core-worktree" into run-update-procedure. Since the ensure-core-worktree command is obsolete, remove it.

---
 builtin/submodule--helper.c | 12 ++----------
 git-submodule.sh            |  2 --
 2 files changed, 2 insertions(+), 12 deletions(-)
builtin/submodule--helper.c +2 -10
index 77ca4270f4..6b473fc0d2 100644
@@ -2771,17 +2771,11 @@ static int push_check(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
-static int ensure_core_worktree(int argc, const char **argv, const char *prefix)
+static void ensure_core_worktree(const char *path)
 {
-	const char *path;
 	const char *cw;
 	struct repository subrepo;
 
-	if (argc != 2)
-		BUG("submodule--helper ensure-core-worktree <path>");
-
-	path = argv[1];
-
 	if (repo_submodule_init(&subrepo, the_repository, path, null_oid()))
 		die(_("could not get a repository handle for submodule '%s'"), path);
 
@@ -2801,8 +2795,6 @@ static int ensure_core_worktree(int argc, const char **argv, const char *prefix)
 		free(abs_path);
 		strbuf_release(&sb);
 	}
-
-	return 0;
 }
 
 static int absorb_git_dirs(int argc, const char **argv, const char *prefix)
@@ -3029,6 +3021,7 @@ static int module_create_branch(int argc, const char **argv, const char *prefix)
 /* NEEDSWORK: this is a temporary name until we delete update_submodule() */
 static int update_submodule2(struct update_data *update_data)
 {
+	ensure_core_worktree(update_data->sm_path);
 	if (update_data->just_cloned)
 		oidcpy(&update_data->suboid, null_oid());
 	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD", &update_data->suboid))
@@ -3428,7 +3421,6 @@ static struct cmd_struct commands[] = {
 	{"add", module_add, SUPPORT_SUPER_PREFIX},
 	{"update-clone", update_clone, 0},
 	{"run-update-procedure", run_update_procedure, 0},
-	{"ensure-core-worktree", ensure_core_worktree, 0},
 	{"relative-path", resolve_relative_path, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
 	{"foreach", module_foreach, SUPPORT_SUPER_PREFIX},
git-submodule.sh +0 -2
index 32a09302ab..458ce73ac6 100755
@@ -402,8 +402,6 @@ cmd_update()
 	do
 		die_if_unmatched "$quickabort" "$sha1"
 
-		git submodule--helper ensure-core-worktree "$sm_path" || exit 1
-
 		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
 
 		if test $just_cloned -eq 0
Signature · 1 line
2.33.GIT
#46 lore raw

Re: [PATCH v3 05/13] submodule--helper: remove ensure-core-worktree

Quoted 2 lines · Glen Choo

Move the logic of "git submodule--helper ensure-core-worktree" into run-update-procedure. Since the ensure-core-worktree command is

I take it as "... command is now obsolete", or "... has become obsolete"?

Quoted 7 lines

obsolete, remove it.

---
 builtin/submodule--helper.c | 12 ++----------
 git-submodule.sh            |  2 --
 2 files changed, 2 insertions(+), 12 deletions(-)

On the script side, the removed call to ensure-core-worktree used to precede these invocations of the helper

submodule--helper relative-path
submodule--helper remote-branch
submodule--helper print-default-remote

before we triggered run-update-procedure, so these helper calls were done only after we made sure we have a submodule there at the path and its configuration file has core.worktree set correctly. If we failed to do so, we wouldn't have made these calls.

Now we call them unprotected. It is not immediately obvious if that is a safe/sensible thing to do.

I would imagine that we would lose more and more code from the script in the "while" loop before run-update-procedure is triggered, and presumably the equivalent code will be added _after_ the call to ensure_core_worktree() this patch adds to the beginning of update_submodule2(), so in the end, the above will presumably become a non-issue, but the series structure still feels iffy because of it.

author
#47 lore raw

Re: [PATCH v3 05/13] submodule--helper: remove ensure-core-worktree

Quoted 7 lines · Junio C Hamano
Quoted 2 lines · Glen Choo

Move the logic of "git submodule--helper ensure-core-worktree" into run-update-procedure. Since the ensure-core-worktree command is

I take it as "... command is now obsolete", or "... has become obsolete"?

Yes, that's a better phrasing.

Quoted 15 lines

On the script side, the removed call to ensure-core-worktree used to precede these invocations of the helper

submodule--helper relative-path
submodule--helper remote-branch
submodule--helper print-default-remote

before we triggered run-update-procedure, so these helper calls were done only after we made sure we have a submodule there at the path and its configuration file has core.worktree set correctly. If we failed to do so, we wouldn't have made these calls.

Now we call them unprotected. It is not immediately obvious if that is a safe/sensible thing to do.

Quoted 6 lines

I would imagine that we would lose more and more code from the script in the "while" loop before run-update-procedure is triggered, and presumably the equivalent code will be added _after_ the call to ensure_core_worktree() this patch adds to the beginning of update_submodule2(), so in the end, the above will presumably become a non-issue, but the series structure still feels iffy because of it.

I could restructure this series so that this patch is as late as possible, so we don't have to worry about safety for 'remote-branch' and 'print-default-remote', but we still have to consider 'relative-path' because that's still around by the end of this series.

Fortunately, relative_path is just a wrapper around dir.h's relative_path(), which AFAICT is just general purpose string manipulation and has nothing to do with Git.

(I'm also certain that the other two commands don't interact with core.worktree either, but I'll just restructure the series so that the point is moot.)

author
#48 lore raw

PATCH v3 6/13 submodule--helper: get remote names from any repository

In reply to Glen Choo

From: Atharva Raykar <[email protected]>

get_default_remote() retrieves the name of a remote by resolving the refs from of the current repository's ref store.

Thus in order to use it for retrieving the remote name of a submodule, we have to start a new subprocess which runs from the submodule directory.

Let's instead introduce a function called repo_get_default_remote() which takes any repository object and retrieves the remote accordingly.

get_default_remote() is then defined as a call to repo_get_default_remote() with 'the_repository' passed to it.

Now that we have repo_get_default_remote(), we no longer have to start a subprocess that called submodule--helper get-default-remote from within the submodule directory.

So let's make a function called get_default_remote_submodule() which takes a submodule path, and returns the default remote for that submodule, all within the same process.

We can now use this function to save an unnecessary subprocess spawn in sync_submodule(), and also in a subsequent patch, which will require this functionality.

---
 builtin/submodule--helper.c | 38 ++++++++++++++++++++++---------------
 1 file changed, 23 insertions(+), 15 deletions(-)
builtin/submodule--helper.c +23 -15
index 6b473fc0d2..a58df3e007 100644
@@ -31,11 +31,13 @@
 typedef void (*each_submodule_fn)(const struct cache_entry *list_item,
 				  void *cb_data);
 
-static char *get_default_remote(void)
+static char *repo_get_default_remote(struct repository *repo)
 {
 	char *dest = NULL, *ret;
 	struct strbuf sb = STRBUF_INIT;
-	const char *refname = resolve_ref_unsafe("HEAD", 0, NULL, NULL);
+	struct ref_store *store = get_main_ref_store(repo);
+	const char *refname = refs_resolve_ref_unsafe(store, "HEAD", 0, NULL,
+						      NULL);
 
 	if (!refname)
 		die(_("No such ref: %s"), "HEAD");
@@ -48,7 +50,7 @@ static char *get_default_remote(void)
 		die(_("Expecting a full ref name, got %s"), refname);
 
 	strbuf_addf(&sb, "branch.%s.remote", refname);
-	if (git_config_get_string(sb.buf, &dest))
+	if (repo_config_get_string(repo, sb.buf, &dest))
 		ret = xstrdup("origin");
 	else
 		ret = dest;
@@ -57,6 +59,19 @@ static char *get_default_remote(void)
 	return ret;
 }
 
+static char *get_default_remote_submodule(const char *module_path)
+{
+	struct repository subrepo;
+
+	repo_submodule_init(&subrepo, the_repository, module_path, null_oid());
+	return repo_get_default_remote(&subrepo);
+}
+
+static char *get_default_remote(void)
+{
+	return repo_get_default_remote(the_repository);
+}
+
 static int print_default_remote(int argc, const char **argv, const char *prefix)
 {
 	char *remote;
@@ -1343,9 +1358,8 @@ static void sync_submodule(const char *path, const char *prefix,
 {
 	const struct submodule *sub;
 	char *remote_key = NULL;
-	char *sub_origin_url, *super_config_url, *displaypath;
+	char *sub_origin_url, *super_config_url, *displaypath, *default_remote;
 	struct strbuf sb = STRBUF_INIT;
-	struct child_process cp = CHILD_PROCESS_INIT;
 	char *sub_config_path = NULL;
 
 	if (!is_submodule_active(the_repository, path))
@@ -1384,21 +1398,15 @@ static void sync_submodule(const char *path, const char *prefix,
 	if (!is_submodule_populated_gently(path, NULL))
 		goto cleanup;
 
-	prepare_submodule_repo_env(&cp.env_array);
-	cp.git_cmd = 1;
-	cp.dir = path;
-	strvec_pushl(&cp.args, "submodule--helper",
-		     "print-default-remote", NULL);
-
 	strbuf_reset(&sb);
-	if (capture_command(&cp, &sb, 0))
+	default_remote = get_default_remote_submodule(path);
+	if (!default_remote)
 		die(_("failed to get the default remote for submodule '%s'"),
 		      path);
 
-	strbuf_strip_suffix(&sb, "\n");
-	remote_key = xstrfmt("remote.%s.url", sb.buf);
+	remote_key = xstrfmt("remote.%s.url", default_remote);
+	free(default_remote);
 
-	strbuf_reset(&sb);
 	submodule_to_gitdir(&sb, path);
 	strbuf_addstr(&sb, "/config");
 
Signature · 1 line
2.33.GIT
author
#49 lore raw

PATCH v3 7/13 submodule--helper: don't use bitfield indirection for parse_options()

In reply to Glen Choo

From: Ævar Arnfjörð Bjarmason <[email protected]>

Do away with the indirection of local variables added in c51f8f94e5b (submodule--helper: run update procedures from C, 2021-08-24).

These were only needed because in C you can't get a pointer to a single bit, so we were using intermediate variables instead.

---
 builtin/submodule--helper.c | 23 ++++++++++-------------
 1 file changed, 10 insertions(+), 13 deletions(-)
builtin/submodule--helper.c +10 -13
index a58df3e007..3a96c35b86 100644
@@ -2023,10 +2023,10 @@ struct update_data {
 	struct object_id suboid;
 	struct submodule_update_strategy update_strategy;
 	int depth;
-	unsigned int force: 1;
-	unsigned int quiet: 1;
-	unsigned int nofetch: 1;
-	unsigned int just_cloned: 1;
+	unsigned int force;
+	unsigned int quiet;
+	unsigned int nofetch;
+	unsigned int just_cloned;
 };
 #define UPDATE_DATA_INIT { .update_strategy = SUBMODULE_UPDATE_STRATEGY_INIT }
 
@@ -2578,16 +2578,17 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 
 static int run_update_procedure(int argc, const char **argv, const char *prefix)
 {
-	int force = 0, quiet = 0, nofetch = 0, just_cloned = 0;
 	char *prefixed_path, *update = NULL;
 	struct update_data update_data = UPDATE_DATA_INIT;
 
 	struct option options[] = {
-		OPT__QUIET(&quiet, N_("suppress output for update by rebase or merge")),
-		OPT__FORCE(&force, N_("force checkout updates"), 0),
-		OPT_BOOL('N', "no-fetch", &nofetch,
+		OPT__QUIET(&update_data.quiet,
+			   N_("suppress output for update by rebase or merge")),
+		OPT__FORCE(&update_data.force, N_("force checkout updates"),
+			   0),
+		OPT_BOOL('N', "no-fetch", &update_data.nofetch,
 			 N_("don't fetch new objects from the remote site")),
-		OPT_BOOL(0, "just-cloned", &just_cloned,
+		OPT_BOOL(0, "just-cloned", &update_data.just_cloned,
 			 N_("overrides update mode in case the repository is a fresh clone")),
 		OPT_INTEGER(0, "depth", &update_data.depth, N_("depth for shallow fetch")),
 		OPT_STRING(0, "prefix", &prefix,
@@ -2615,10 +2616,6 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)
 	if (argc != 1)
 		usage_with_options(usage, options);
 
-	update_data.force = !!force;
-	update_data.quiet = !!quiet;
-	update_data.nofetch = !!nofetch;
-	update_data.just_cloned = !!just_cloned;
 	update_data.sm_path = argv[0];
 
 	if (update_data.recursive_prefix)
Signature · 1 line
2.33.GIT
author
#50 lore raw

PATCH v3 8/13 submodule--helper run-update-procedure: learn --remote

In reply to Glen Choo

Teach run-update-procedure to handle --remote instead of parsing --remote in git-submodule.sh. As a result, "git submodule--helper [print-default-remote|remote-branch]" have no more callers, so remove them.

---
 builtin/submodule--helper.c | 56 +++++++++++++++----------------------
 git-submodule.sh            | 30 +-------------------
 2 files changed, 23 insertions(+), 63 deletions(-)
builtin/submodule--helper.c +22 -34
index 3a96c35b86..99341fb343 100644
@@ -72,21 +72,6 @@ static char *get_default_remote(void)
 	return repo_get_default_remote(the_repository);
 }
 
-static int print_default_remote(int argc, const char **argv, const char *prefix)
-{
-	char *remote;
-
-	if (argc != 1)
-		die(_("submodule--helper print-default-remote takes no arguments"));
-
-	remote = get_default_remote();
-	if (remote)
-		printf("%s\n", remote);
-
-	free(remote);
-	return 0;
-}
-
 static int starts_with_dot_slash(const char *str)
 {
 	return str[0] == '.' && is_dir_sep(str[1]);
@@ -2027,6 +2012,7 @@ struct update_data {
 	unsigned int quiet;
 	unsigned int nofetch;
 	unsigned int just_cloned;
+	unsigned int remote;
 };
 #define UPDATE_DATA_INIT { .update_strategy = SUBMODULE_UPDATE_STRATEGY_INIT }
 
@@ -2603,6 +2589,8 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)
 		OPT_CALLBACK_F(0, "oid", &update_data.oid, N_("sha1"),
 			       N_("SHA1 expected by superproject"), PARSE_OPT_NONEG,
 			       parse_opt_object_id),
+		OPT_BOOL(0, "remote", &update_data.remote,
+			 N_("use SHA-1 of submodule's remote tracking branch")),
 		OPT_END()
 	};
 
@@ -2682,23 +2670,6 @@ static const char *remote_submodule_branch(const char *path)
 	return branch;
 }
 
-static int resolve_remote_submodule_branch(int argc, const char **argv,
-		const char *prefix)
-{
-	const char *ret;
-	struct strbuf sb = STRBUF_INIT;
-	if (argc != 2)
-		die("submodule--helper remote-branch takes exactly one arguments, got %d", argc);
-
-	ret = remote_submodule_branch(argv[1]);
-	if (!ret)
-		die("submodule %s doesn't exist", argv[1]);
-
-	printf("%s", ret);
-	strbuf_release(&sb);
-	return 0;
-}
-
 static int push_check(int argc, const char **argv, const char *prefix)
 {
 	struct remote *remote;
@@ -3033,6 +3004,25 @@ static int update_submodule2(struct update_data *update_data)
 		die(_("Unable to find current revision in submodule path '%s'"),
 			update_data->displaypath);
 
+	if (update_data->remote) {
+		char *remote_name = get_default_remote_submodule(update_data->sm_path);
+		const char *branch = remote_submodule_branch(update_data->sm_path);
+		char *remote_ref = xstrfmt("refs/remotes/%s/%s", remote_name, branch);
+
+		if (!update_data->nofetch) {
+			if (fetch_in_submodule(update_data->sm_path, update_data->depth,
+					      0, NULL))
+				die(_("Unable to fetch in submodule path '%s'"),
+				    update_data->sm_path);
+		}
+
+		if (resolve_gitlink_ref(update_data->sm_path, remote_ref, &update_data->oid))
+			die(_("Unable to find %s revision in submodule path '%s'"),
+			    remote_ref, update_data->sm_path);
+
+		free(remote_ref);
+	}
+
 	if (!oideq(&update_data->oid, &update_data->suboid) || update_data->force)
 		return do_run_update_procedure(update_data);
 
@@ -3431,11 +3421,9 @@ static struct cmd_struct commands[] = {
 	{"foreach", module_foreach, SUPPORT_SUPER_PREFIX},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
-	{"print-default-remote", print_default_remote, 0},
 	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
 	{"deinit", module_deinit, 0},
 	{"summary", module_summary, SUPPORT_SUPER_PREFIX},
-	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
 	{"is-active", is_active, 0},
git-submodule.sh +1 -29
index 458ce73ac6..23ebd90892 100755
@@ -247,20 +247,6 @@ cmd_deinit()
 	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper deinit ${GIT_QUIET:+--quiet} ${force:+--force} ${deinit_all:+--all} -- "$@"
 }
 
-# usage: fetch_in_submodule <module_path> [<depth>] [<sha1>]
-# Because arguments are positional, use an empty string to omit <depth>
-# but include <sha1>.
-fetch_in_submodule () (
-	sanitize_submodule_env &&
-	cd "$1" &&
-	if test $# -eq 3
-	then
-		echo "$3" | git fetch ${GIT_QUIET:+--quiet} --stdin ${2:+"$2"}
-	else
-		git fetch ${GIT_QUIET:+--quiet} ${2:+"$2"}
-	fi
-)
-
 #
 # Update each submodule path to correct revision, using clone and checkout as needed
 #
@@ -409,21 +395,6 @@ cmd_update()
 			just_cloned=
 		fi
 
-		if test -n "$remote"
-		then
-			branch=$(git submodule--helper remote-branch "$sm_path")
-			if test -z "$nofetch"
-			then
-				# Fetch remote before determining tracking $sha1
-				fetch_in_submodule "$sm_path" $depth ||
-				die "fatal: $(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
-			fi
-			remote_name=$(sanitize_submodule_env; cd "$sm_path" && git submodule--helper print-default-remote)
-			sha1=$(sanitize_submodule_env; cd "$sm_path" &&
-				git rev-parse --verify "${remote_name}/${branch}") ||
-			die "fatal: $(eval_gettext "Unable to find current \${remote_name}/\${branch} revision in submodule path '\$sm_path'")"
-		fi
-
 		out=$(git submodule--helper run-update-procedure \
 			  ${wt_prefix:+--prefix "$wt_prefix"} \
 			  ${GIT_QUIET:+--quiet} \
@@ -434,6 +405,7 @@ cmd_update()
 			  ${update:+--update "$update"} \
 			  ${prefix:+--recursive-prefix "$prefix"} \
 			  ${sha1:+--oid "$sha1"} \
+			  ${remote:+--remote} \
 			  "--" \
 			  "$sm_path")
 
Signature · 1 line
2.33.GIT
#51 lore raw

Re: [PATCH v3 08/13] submodule--helper run-update-procedure: learn --remote

Quoted 12 lines · Glen Choo
builtin/submodule--helper.c +8 -0
@@ -3033,6 +3004,25 @@ static int update_submodule2(struct update_data *update_data)
 		die(_("Unable to find current revision in submodule path '%s'"),
 			update_data->displaypath);
 
+	if (update_data->remote) {
+		char *remote_name = get_default_remote_submodule(update_data->sm_path);
+		const char *branch = remote_submodule_branch(update_data->sm_path);
+		char *remote_ref = xstrfmt("refs/remotes/%s/%s", remote_name, branch);
+
+		if (!update_data->nofetch) {
+			if (fetch_in_submodule(update_data->sm_path, update_data->depth,
+					      0, NULL))

builtin/submodule--helper.c static int update_submodule2(struct update_data *update_data) · line 3024

Quoted 10 lines
builtin/submodule--helper.c +10 -0
+				die(_("Unable to fetch in submodule path '%s'"),
+				    update_data->sm_path);
+		}
+
+		if (resolve_gitlink_ref(update_data->sm_path, remote_ref, &update_data->oid))
+			die(_("Unable to find %s revision in submodule path '%s'"),
+			    remote_ref, update_data->sm_path);
+
+		free(remote_ref);
+	}

This, and ...

Quoted 8 lines
git-submodule.sh +0 -4
@@ -409,21 +395,6 @@ cmd_update()
 			just_cloned=
 		fi
 
-		if test -n "$remote"
-		then
-			branch=$(git submodule--helper remote-branch "$sm_path")
-			if test -z "$nofetch"

git-submodule.sh cmd_update() · line 425

Quoted 10 lines
git-submodule.sh +0 -10
-			then
-				# Fetch remote before determining tracking $sha1
-				fetch_in_submodule "$sm_path" $depth ||
-				die "fatal: $(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
-			fi
-			remote_name=$(sanitize_submodule_env; cd "$sm_path" && git submodule--helper print-default-remote)
-			sha1=$(sanitize_submodule_env; cd "$sm_path" &&
-				git rev-parse --verify "${remote_name}/${branch}") ||
-			die "fatal: $(eval_gettext "Unable to find current \${remote_name}/\${branch} revision in submodule path '\$sm_path'")"
-		fi

... this change would "fix" the temporary breakage [05/13] may have brought us. From the series structure's point of view, doing [05/13] after this would be more sensible. If we leave the call to "displaypath" still in the script after this series, [05/13] or its equivalent should mention that ensure-core-worktree does not matter for that particular call when it delays the call to it by moving it to the beginning of the update_submodule2() from the much earlier part of the script.

Quoted 10 lines
git-submodule.sh +1 -0
 		out=$(git submodule--helper run-update-procedure \
 			  ${wt_prefix:+--prefix "$wt_prefix"} \
 			  ${GIT_QUIET:+--quiet} \
@@ -434,6 +405,7 @@ cmd_update()
 			  ${update:+--update "$update"} \
 			  ${prefix:+--recursive-prefix "$prefix"} \
 			  ${sha1:+--oid "$sha1"} \
+			  ${remote:+--remote} \
 			  "--" \
 			  "$sm_path")
author
#52 lore raw

Re: [PATCH v3 08/13] submodule--helper run-update-procedure: learn --remote

Quoted 46 lines · Junio C Hamano
Quoted 12 lines · Glen Choo
builtin/submodule--helper.c +8 -0
@@ -3033,6 +3004,25 @@ static int update_submodule2(struct update_data *update_data)
 		die(_("Unable to find current revision in submodule path '%s'"),
 			update_data->displaypath);
 
+	if (update_data->remote) {
+		char *remote_name = get_default_remote_submodule(update_data->sm_path);
+		const char *branch = remote_submodule_branch(update_data->sm_path);
+		char *remote_ref = xstrfmt("refs/remotes/%s/%s", remote_name, branch);
+
+		if (!update_data->nofetch) {
+			if (fetch_in_submodule(update_data->sm_path, update_data->depth,
+					      0, NULL))

builtin/submodule--helper.c static int update_submodule2(struct update_data *update_data) · line 3024

Quoted 10 lines
builtin/submodule--helper.c +10 -0
+				die(_("Unable to fetch in submodule path '%s'"),
+				    update_data->sm_path);
+		}
+
+		if (resolve_gitlink_ref(update_data->sm_path, remote_ref, &update_data->oid))
+			die(_("Unable to find %s revision in submodule path '%s'"),
+			    remote_ref, update_data->sm_path);
+
+		free(remote_ref);
+	}

This, and ...

Quoted 8 lines
git-submodule.sh +0 -4
@@ -409,21 +395,6 @@ cmd_update()
 			just_cloned=
 		fi
 
-		if test -n "$remote"
-		then
-			branch=$(git submodule--helper remote-branch "$sm_path")
-			if test -z "$nofetch"

git-submodule.sh cmd_update() · line 425

Quoted 10 lines
git-submodule.sh +0 -10
-			then
-				# Fetch remote before determining tracking $sha1
-				fetch_in_submodule "$sm_path" $depth ||
-				die "fatal: $(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
-			fi
-			remote_name=$(sanitize_submodule_env; cd "$sm_path" && git submodule--helper print-default-remote)
-			sha1=$(sanitize_submodule_env; cd "$sm_path" &&
-				git rev-parse --verify "${remote_name}/${branch}") ||
-			die "fatal: $(eval_gettext "Unable to find current \${remote_name}/\${branch} revision in submodule path '\$sm_path'")"
-		fi
Quoted 9 lines

... this change would "fix" the temporary breakage [05/13] may have brought us. From the series structure's point of view, doing [05/13] after this would be more sensible. If we leave the call to "displaypath" still in the script after this series, [05/13] or its equivalent should mention that ensure-core-worktree does not matter for that particular call when it delays the call to it by moving it to the beginning of the update_submodule2() from the much earlier part of the script.

I should've read this in full before sending my reply on [05/13]. Makes sense, thanks for the pointer :)

Quoted 10 lines
Quoted 10 lines
git-submodule.sh +1 -0
 		out=$(git submodule--helper run-update-procedure \
 			  ${wt_prefix:+--prefix "$wt_prefix"} \
 			  ${GIT_QUIET:+--quiet} \
@@ -434,6 +405,7 @@ cmd_update()
 			  ${update:+--update "$update"} \
 			  ${prefix:+--recursive-prefix "$prefix"} \
 			  ${sha1:+--oid "$sha1"} \
+			  ${remote:+--remote} \
 			  "--" \
 			  "$sm_path")
author
#53 lore raw

PATCH v3 9/13 submodule--helper: refactor get_submodule_displaypath()

In reply to Glen Choo

From: Atharva Raykar <[email protected]>

We create a function called do_get_submodule_displaypath() that generates the display path required by several submodule functions, and takes a custom superprefix parameter, instead of reading it from the environment.

We then redefine the existing get_submodule_displaypath() function as a call to this new function, where the superprefix is obtained from the environment.

---
 builtin/submodule--helper.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
builtin/submodule--helper.c +10 -4
index 99341fb343..e6df99a884 100644
@@ -247,11 +247,10 @@ static int resolve_relative_url_test(int argc, const char **argv, const char *pr
 	return 0;
 }
 
-/* the result should be freed by the caller. */
-static char *get_submodule_displaypath(const char *path, const char *prefix)
+static char *do_get_submodule_displaypath(const char *path,
+					  const char *prefix,
+					  const char *super_prefix)
 {
-	const char *super_prefix = get_super_prefix();
-
 	if (prefix && super_prefix) {
 		BUG("cannot have prefix '%s' and superprefix '%s'",
 		    prefix, super_prefix);
@@ -267,6 +266,13 @@ static char *get_submodule_displaypath(const char *path, const char *prefix)
 	}
 }
 
+/* the result should be freed by the caller. */
+static char *get_submodule_displaypath(const char *path, const char *prefix)
+{
+	const char *super_prefix = get_super_prefix();
+	return do_get_submodule_displaypath(path, prefix, super_prefix);
+}
+
 static char *compute_rev_name(const char *sub_path, const char* object_id)
 {
 	struct strbuf sb = STRBUF_INIT;
Signature · 1 line
2.33.GIT
author
#54 lore raw

PATCH v3 10/13 submodule--helper: allow setting superprefix for init_submodule()

In reply to Glen Choo

From: Atharva Raykar <[email protected]>

We allow callers of the init_submodule() function to optionally override the superprefix from the environment.

We need to enable this option because in our conversion of the update command that will follow, the '--init' option will be handled through this API. We will need to change the superprefix at that time to ensure the display paths show correctly in the output messages.

---
 builtin/submodule--helper.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
builtin/submodule--helper.c +7 -3
index e6df99a884..69ada7f914 100644
@@ -594,18 +594,22 @@ static int module_foreach(int argc, const char **argv, const char *prefix)
 
 struct init_cb {
 	const char *prefix;
+	const char *superprefix;
 	unsigned int flags;
 };
 #define INIT_CB_INIT { 0 }
 
 static void init_submodule(const char *path, const char *prefix,
-			   unsigned int flags)
+			   const char *superprefix, unsigned int flags)
 {
 	const struct submodule *sub;
 	struct strbuf sb = STRBUF_INIT;
 	char *upd = NULL, *url = NULL, *displaypath;
 
-	displaypath = get_submodule_displaypath(path, prefix);
+	/* try superprefix from the environment, if it is not passed explicitly */
+	if (!superprefix)
+		superprefix = get_super_prefix();
+	displaypath = do_get_submodule_displaypath(path, prefix, superprefix);
 
 	sub = submodule_from_path(the_repository, null_oid(), path);
 
@@ -679,7 +683,7 @@ static void init_submodule(const char *path, const char *prefix,
 static void init_submodule_cb(const struct cache_entry *list_item, void *cb_data)
 {
 	struct init_cb *info = cb_data;
-	init_submodule(list_item->name, info->prefix, info->flags);
+	init_submodule(list_item->name, info->prefix, info->superprefix, info->flags);
 }
 
 static int module_init(int argc, const char **argv, const char *prefix)
Signature · 1 line
2.33.GIT
author
#55 lore raw

PATCH v3 11/13 submodule--helper update-clone: learn --init

In reply to Glen Choo

Teach "git submodule--helper update-clone" the --init flag and remove the corresponding shell code.

When the --init flag is passed to the subcommand, we do not spawn a new subprocess and call submodule--helper init on the submodule paths, because the Git machinery is not able to pick up the configuration changes introduced by that init call. So we instead run the init_submodule_cb() callback over each submodule in the same process.

[1] https://lore.kernel.org/git/CAP8UFD0NCQ5w_3GtT_xHr35i7h8BuLX4UcHNY6VHPGREmDVObA@mail.gmail.com/

---
 builtin/submodule--helper.c | 26 ++++++++++++++++++++++++++
 git-submodule.sh            |  9 +++------
 2 files changed, 29 insertions(+), 6 deletions(-)
builtin/submodule--helper.c +26 -0
index 69ada7f914..296ab80bf2 100644
@@ -2000,6 +2000,7 @@ struct submodule_update_clone {
 	int failed_clones_nr, failed_clones_alloc;
 
 	int max_jobs;
+	unsigned int init;
 };
 #define SUBMODULE_UPDATE_CLONE_INIT { \
 	.list = MODULE_LIST_INIT, \
@@ -2509,6 +2510,8 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	int ret;
 
 	struct option module_update_clone_options[] = {
+		OPT_BOOL(0, "init", &suc.init,
+			 N_("initialize uninitialized submodules before update")),
 		OPT_STRING(0, "prefix", &prefix,
 			   N_("path"),
 			   N_("path into the working tree")),
@@ -2567,6 +2570,29 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	if (pathspec.nr)
 		suc.warn_if_uninitialized = 1;
 
+	if (suc.init) {
+		struct module_list list = MODULE_LIST_INIT;
+		struct init_cb info = INIT_CB_INIT;
+
+		if (module_list_compute(argc, argv, suc.prefix,
+					&pathspec, &list) < 0)
+			return 1;
+
+		/*
+		 * If there are no path args and submodule.active is set then,
+		 * by default, only initialize 'active' modules.
+		 */
+		if (!argc && git_config_get_value_multi("submodule.active"))
+			module_list_active(&list);
+
+		info.prefix = suc.prefix;
+		info.superprefix = suc.recursive_prefix;
+		if (suc.quiet)
+			info.flags |= OPT_QUIET;
+
+		for_each_listed_submodule(&list, init_submodule_cb, &info);
+	}
+
 	ret = update_submodules(&suc);
 	list_objects_filter_release(&filter_options);
 	return ret;
git-submodule.sh +3 -6
index 23ebd90892..51be7c7f7e 100755
@@ -361,14 +361,11 @@ cmd_update()
 		usage
 	fi
 
-	if test -n "$init"
-	then
-		cmd_init "--" "$@" || return
-	fi
-
 	{
-	git submodule--helper update-clone ${GIT_QUIET:+--quiet} \
+	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper update-clone \
+		${GIT_QUIET:+--quiet} \
 		${progress:+"--progress"} \
+		${init:+--init} \
 		${wt_prefix:+--prefix "$wt_prefix"} \
 		${prefix:+--recursive-prefix "$prefix"} \
 		${update:+--update "$update"} \
Signature · 1 line
2.33.GIT
author
#56 lore raw

PATCH v3 12/13 submodule update: add tests for --filter

In reply to Glen Choo

Test the "--filter" option to make sure we don't break anything while refactoring "git submodule update".

---
 t/t7406-submodule-update.sh | 12 ++++++++++++
 1 file changed, 12 insertions(+)
t/t7406-submodule-update.sh +12 -0
index 7764c1c3cb..000e055811 100755
@@ -1071,4 +1071,16 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s
 	)
 '
 
+test_expect_success 'submodule update --filter requires --init' '
+	test_expect_code 129 git -C super submodule update --filter blob:none
+'
+
+test_expect_success 'submodule update --filter sets partial clone settings' '
+	test_when_finished "rm -rf super-filter" &&
+	git clone cloned super-filter &&
+	git -C super-filter submodule update --init --filter blob:none &&
+	test_cmp_config -C super-filter/submodule true remote.origin.promisor &&
+	test_cmp_config -C super-filter/submodule blob:none remote.origin.partialclonefilter
+'
+
 test_done
Signature · 1 line
2.33.GIT
author
#57 lore raw

PATCH v3 13/13 submodule--helper update-clone: check for --filter and --init

In reply to Glen Choo

"git submodule update --filter" also requires the "--init" option. Teach update-clone to do this usage check in C and remove the check from git-submodule.sh.

In addition, change update-clone's usage string so that it teaches users about "git submodule update" instead of "git submodule--helper update-clone" (the string is copied from git-submodule.sh). This should be more helpful to users since they don't invoke update-clone directly.

---
 builtin/submodule--helper.c | 20 +++++++++++++++++++-
 git-submodule.sh            |  5 -----
 2 files changed, 19 insertions(+), 6 deletions(-)
builtin/submodule--helper.c +19 -1
index 296ab80bf2..bef9ab22d4 100644
@@ -2545,7 +2545,12 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	};
 
 	const char *const git_submodule_helper_usage[] = {
-		N_("git submodule--helper update-clone [--prefix=<path>] [<path>...]"),
+		N_("git submodule [--quiet] update"
+		" [--init [--filter=<filter-spec>]] [--remote]"
+		" [-N|--no-fetch] [-f|--force]"
+		" [--checkout|--merge|--rebase]"
+		" [--[no-]recommend-shallow] [--reference <repository>]"
+		" [--recursive] [--[no-]single-branch] [--] [<path>...]"),
 		NULL
 	};
 	suc.prefix = prefix;
@@ -2556,6 +2561,19 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	memset(&filter_options, 0, sizeof(filter_options));
 	argc = parse_options(argc, argv, prefix, module_update_clone_options,
 			     git_submodule_helper_usage, 0);
+
+	if (filter_options.choice && !suc.init) {
+		/*
+		 * NEEDSWORK: Don't use usage_with_options() because the
+		 * usage string is for "git submodule update", but the
+		 * options are for "git submodule--helper update-clone".
+		 *
+		 * This will no longer be an issue when "update-clone"
+		 * is replaced by "git submodule--helper update".
+		 */
+		usage(git_submodule_helper_usage[0]);
+	}
+
 	suc.filter_options = &filter_options;
 
 	if (update)
git-submodule.sh +0 -5
index 51be7c7f7e..aa8bdfca9d 100755
@@ -356,11 +356,6 @@ cmd_update()
 		shift
 	done
 
-	if test -n "$filter" && test "$init" != "1"
-	then
-		usage
-	fi
-
 	{
 	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper update-clone \
 		${GIT_QUIET:+--quiet} \
Signature · 1 line
2.33.GIT
Reviewed-by on the v3 series
#58 lore raw
In reply to Glen Choo
Quoted 5 lines · Glen Choo

Original series: https://lore.kernel.org/git/[email protected]
(I've trimmed the cc list down to the 'most interested' parties)

Thanks for the input, Ævar :) This just fixes up a few small issues in the previous version.

And thanks for working on this, there was a reason I was down to nitpicking in v2, this is looking really good & I can't find any issues with this iteration at all.

So I'm happy to give this my:

Junio: It would be great to get this advancing to "next" so that the subsequent set of migrations to submodule-in-C advancing.

#59 lore raw
Quoted 8 lines · Ævar Arnfjörð Bjarmason
Quoted 5 lines · Glen Choo

Original series: https://lore.kernel.org/git/[email protected]
(I've trimmed the cc list down to the 'most interested' parties)

Thanks for the input, Ævar :) This just fixes up a few small issues in the previous version.

Quoted 7 lines

And thanks for working on this, there was a reason I was down to nitpicking in v2, this is looking really good & I can't find any issues with this iteration at all.

So I'm happy to give this my:

Thanks for reviewing.

I found that the safety of doing [05/13] that early was questionable, but other than that, it was a pleasant read.

author
#60 lore raw

v4 cover submodule: convert parts of 'update' to C

In reply to Glen Choo

Original series: https://lore.kernel.org/git/[email protected]

= Patch summary

- Patch 1 adds extra tests to "git submodule update" to make sure we don't break anything
- Patch 2 removes dead code that used to be part of "git submodule update"
- Patch 3 prepares for later changes by introducing the C function that will hold most of the newly converted code
- Patch 4 moves run-update-procedure's --suboid option into C
- Patch 5 moves ensure-core-worktree into C
- Patches 6-8 move run-update-procedure's --remote option into C
- Patches 9-11 move "git submodule update"'s --init into C
- Patches 12-13 move "git submodule update"'s --filter option into C

= Changes

Since v3:
- Move 'remove ensure-core-worktree' towards the end (see [email protected]).
- Fix typo

Since v2:
- Patch 6: Fix a stale commit message that said 'in the next patch'.
- Patch 9: Fix an overly long line (spotted by Ævar in an older iteration of ar/submodule-update)
- Patch 12: Test for usage using test_expect_code
- Patch 13: Add missing spaces to the usage string

Since v1:
- Fix compilation error due to bad rebase
- Remove accidentally included tests
- Fix a NEEDSWORK (it was a leftover from ar/submodule-update)

Atharva Raykar (3):
submodule--helper: get remote names from any repository
submodule--helper: refactor get_submodule_displaypath()
submodule--helper: allow setting superprefix for init_submodule()

Glen Choo (8):
submodule--helper: remove update-module-mode
submodule--helper: reorganize code for sh to C conversion submodule--helper run-update-procedure: remove --suboid submodule--helper run-update-procedure: learn --remote submodule--helper update-clone: learn --init
submodule--helper: remove ensure-core-worktree submodule update: add tests for --filter submodule--helper update-clone: check for --filter and --init

Ævar Arnfjörð Bjarmason (2):
submodule tests: test for init and update failure output
submodule--helper: don't use bitfield indirection for parse_options()

 builtin/submodule--helper.c    | 248 +++++++++++++++++++--------------
 git-submodule.sh               |  54 +------
 t/t7406-submodule-update.sh    |  26 +++-
 t/t7408-submodule-reference.sh |  14 +-
 4 files changed, 183 insertions(+), 159 deletions(-)

Range-diff against v3:

 1:  6138f4682c =  1:  6138f4682c submodule tests: test for init and update failure output
 2:  6c83c78819 =  2:  6c83c78819 submodule--helper: remove update-module-mode
 3:  9524986096 =  3:  9524986096 submodule--helper: reorganize code for sh to C conversion
 4:  f42f3de2b7 !  4:  533de2c787 submodule--helper run-update-procedure: remove --suboid

@@ Commit message
submodule--helper run-update-procedure: remove --suboid

Teach run-update-procedure to determine the oid of the submodule's HEAD
- instead of doing it in git-subomdule.sh.
+ instead of doing it in git-submodule.sh.

 6:  8dc7bc5894 =  5:  bcc90c332b submodule--helper: get remote names from any repository
 7:  feaf9f45d8 =  6:  93dd851882 submodule--helper: don't use bitfield indirection for parse_options()
 8:  91e8e1a007 =  7:  1aff3622bc submodule--helper run-update-procedure: learn --remote
 9:  aba851e71e =  8:  c59d6a9b3c submodule--helper: refactor get_submodule_displaypath()
10:  2155c049a2 =  9:  28bc5887ea submodule--helper: allow setting superprefix for init_submodule()
11:  03bbc39a06 = 10:  364d996481 submodule--helper update-clone: learn --init
 5:  b0a0cae633 ! 11:  3b7d961ade submodule--helper: remove ensure-core-worktree

@@ Commit message
submodule--helper: remove ensure-core-worktree

Move the logic of "git submodule--helper ensure-core-worktree" into
- run-update-procedure. Since the ensure-core-worktree command is
- obsolete, remove it.
+ run-update-procedure, and since this makes the ensure-core-worktree
+ command obsolete, remove it.
+
+ As a result, the order of two operations in git-submodule.sh is
+ reversed: 'set the value of core.worktree' now happens after the call to
+ "git submodule--helper relative-path". This is safe - "relative-path"
+ does not depend on the value of core.worktree.

12:  e49b26ad94 = 12:  70e00d294c submodule update: add tests for --filter
13:  c97c97948a = 13:  ed7f8d57dc submodule--helper update-clone: check for --filter and --init

base-commit: 715d08a9e51251ad8290b181b6ac3b9e1f9719d7

Signature · 1 line
2.33.GIT
author
#61 lore raw

PATCH v4 1/13 submodule tests: test for init and update failure output

From: Ævar Arnfjörð Bjarmason <[email protected]>

Amend some submodule tests to test for the failure output of "git submodule [update|init]". The lack of such tests hid a regression in an earlier version of a subsequent commit.

---
 t/t7406-submodule-update.sh    | 14 ++++++++++++--
 t/t7408-submodule-reference.sh | 14 +++++++++++++-
 2 files changed, 25 insertions(+), 3 deletions(-)
t/t7406-submodule-update.sh +12 -2
index 11cccbb333..7764c1c3cb 100755
@@ -205,8 +205,18 @@ test_expect_success 'submodule update should fail due to local changes' '
 	 (cd submodule &&
 	  compare_head
 	 ) &&
-	 test_must_fail git submodule update submodule
-	)
+	 test_must_fail git submodule update submodule 2>../actual.raw
+	) &&
+	sed "s/^> //" >expect <<-\EOF &&
+	> error: Your local changes to the following files would be overwritten by checkout:
+	> 	file
+	> Please commit your changes or stash them before you switch branches.
+	> Aborting
+	> fatal: Unable to checkout OID in submodule path '\''submodule'\''
+	EOF
+	sed -e "s/checkout $SQ[^$SQ]*$SQ/checkout OID/" <actual.raw >actual &&
+	test_cmp expect actual
+
 '
 test_expect_success 'submodule update should throw away changes with --force ' '
 	(cd super &&
t/t7408-submodule-reference.sh +13 -1
index a3892f494b..c3a4545510 100755
@@ -193,7 +193,19 @@ test_expect_success 'missing nested submodule alternate fails clone and submodul
 		cd supersuper-clone &&
 		check_that_two_of_three_alternates_are_used &&
 		# update of the submodule fails
-		test_must_fail git submodule update --init --recursive
+		cat >expect <<-\EOF &&
+		fatal: submodule '\''sub'\'' cannot add alternate: path ... does not exist
+		Failed to clone '\''sub'\''. Retry scheduled
+		fatal: submodule '\''sub-dissociate'\'' cannot add alternate: path ... does not exist
+		Failed to clone '\''sub-dissociate'\''. Retry scheduled
+		fatal: submodule '\''sub'\'' cannot add alternate: path ... does not exist
+		Failed to clone '\''sub'\'' a second time, aborting
+		fatal: Failed to recurse into submodule path ...
+		EOF
+		test_must_fail git submodule update --init --recursive 2>err &&
+		grep -e fatal: -e ^Failed err >actual.raw &&
+		sed -e "s/path $SQ[^$SQ]*$SQ/path .../" <actual.raw >actual &&
+		test_cmp expect actual
 	)
 '
 
Signature · 1 line
2.33.GIT
author
#62 lore raw

PATCH v4 2/13 submodule--helper: remove update-module-mode

In reply to Glen Choo

This is dead code - it has not been used since c51f8f94e5 (submodule--helper: run update procedures from C, 2021-08-24).

---
 builtin/submodule--helper.c | 24 ------------------------
 1 file changed, 24 deletions(-)
builtin/submodule--helper.c +0 -24
index eeacefcc38..c11ee1ea2b 100644
@@ -1957,29 +1957,6 @@ static void determine_submodule_update_strategy(struct repository *r,
 	free(key);
 }
 
-static int module_update_module_mode(int argc, const char **argv, const char *prefix)
-{
-	const char *path, *update = NULL;
-	int just_cloned;
-	struct submodule_update_strategy update_strategy = { .type = SM_UPDATE_CHECKOUT };
-
-	if (argc < 3 || argc > 4)
-		die("submodule--helper update-module-clone expects <just-cloned> <path> [<update>]");
-
-	just_cloned = git_config_int("just_cloned", argv[1]);
-	path = argv[2];
-
-	if (argc == 4)
-		update = argv[3];
-
-	determine_submodule_update_strategy(the_repository,
-					    just_cloned, path, update,
-					    &update_strategy);
-	fputs(submodule_strategy_to_string(&update_strategy), stdout);
-
-	return 0;
-}
-
 struct update_clone_data {
 	const struct submodule *sub;
 	struct object_id oid;
@@ -3430,7 +3407,6 @@ static struct cmd_struct commands[] = {
 	{"name", module_name, 0},
 	{"clone", module_clone, 0},
 	{"add", module_add, SUPPORT_SUPER_PREFIX},
-	{"update-module-mode", module_update_module_mode, 0},
 	{"update-clone", update_clone, 0},
 	{"run-update-procedure", run_update_procedure, 0},
 	{"ensure-core-worktree", ensure_core_worktree, 0},
Signature · 1 line
2.33.GIT
author
#63 lore raw

PATCH v4 3/13 submodule--helper: reorganize code for sh to C conversion

In reply to Glen Choo

Introduce a function, update_submodule2(), that will implement the functionality of run-update-procedure and its surrounding shell code in submodule.sh. This name is temporary; it will replace update_submodule() when the sh to C conversion is complete.

---
 builtin/submodule--helper.c | 26 +++++++++++++++++++++-----
 1 file changed, 21 insertions(+), 5 deletions(-)
builtin/submodule--helper.c +21 -5
index c11ee1ea2b..1b67a3887c 100644
@@ -2452,6 +2452,16 @@ static int do_run_update_procedure(struct update_data *ud)
 	return run_update_command(ud, subforce);
 }
 
+/*
+ * NEEDSWORK: As we convert "git submodule update" to C,
+ * update_submodule2() will invoke more and more functions, making it
+ * difficult to preserve the function ordering without forward
+ * declarations.
+ *
+ * When the conversion is complete, this forward declaration will be
+ * unnecessary and should be removed.
+ */
+static int update_submodule2(struct update_data *update_data);
 static void update_submodule(struct update_clone_data *ucd)
 {
 	fprintf(stdout, "dummy %s %d\t%s\n",
@@ -2618,11 +2628,7 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)
 					    &update_data.update_strategy);
 
 	free(prefixed_path);
-
-	if (!oideq(&update_data.oid, &update_data.suboid) || update_data.force)
-		return do_run_update_procedure(&update_data);
-
-	return 3;
+	return update_submodule2(&update_data);
 }
 
 static int resolve_relative_path(int argc, const char **argv, const char *prefix)
@@ -3022,6 +3028,16 @@ static int module_create_branch(int argc, const char **argv, const char *prefix)
 				    force, reflog, quiet, track, dry_run);
 	return 0;
 }
+
+/* NEEDSWORK: this is a temporary name until we delete update_submodule() */
+static int update_submodule2(struct update_data *update_data)
+{
+	if (!oideq(&update_data->oid, &update_data->suboid) || update_data->force)
+		return do_run_update_procedure(update_data);
+
+	return 3;
+}
+
 struct add_data {
 	const char *prefix;
 	const char *branch;
Signature · 1 line
2.33.GIT
author
#64 lore raw

PATCH v4 4/13 submodule--helper run-update-procedure: remove --suboid

In reply to Glen Choo

Teach run-update-procedure to determine the oid of the submodule's HEAD instead of doing it in git-submodule.sh.

---
 builtin/submodule--helper.c | 9 ++++++---
 git-submodule.sh            | 8 +-------
 2 files changed, 7 insertions(+), 10 deletions(-)
builtin/submodule--helper.c +6 -3
index 1b67a3887c..77ca4270f4 100644
@@ -2594,9 +2594,6 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)
 		OPT_CALLBACK_F(0, "oid", &update_data.oid, N_("sha1"),
 			       N_("SHA1 expected by superproject"), PARSE_OPT_NONEG,
 			       parse_opt_object_id),
-		OPT_CALLBACK_F(0, "suboid", &update_data.suboid, N_("subsha1"),
-			       N_("SHA1 of submodule's HEAD"), PARSE_OPT_NONEG,
-			       parse_opt_object_id),
 		OPT_END()
 	};
 
@@ -3032,6 +3029,12 @@ static int module_create_branch(int argc, const char **argv, const char *prefix)
 /* NEEDSWORK: this is a temporary name until we delete update_submodule() */
 static int update_submodule2(struct update_data *update_data)
 {
+	if (update_data->just_cloned)
+		oidcpy(&update_data->suboid, null_oid());
+	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD", &update_data->suboid))
+		die(_("Unable to find current revision in submodule path '%s'"),
+			update_data->displaypath);
+
 	if (!oideq(&update_data->oid, &update_data->suboid) || update_data->force)
 		return do_run_update_procedure(update_data);
 
git-submodule.sh +1 -7
index 87772ac891..32a09302ab 100755
@@ -406,14 +406,9 @@ cmd_update()
 
 		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
 
-		if test $just_cloned -eq 1
+		if test $just_cloned -eq 0
 		then
-			subsha1=
-		else
 			just_cloned=
-			subsha1=$(sanitize_submodule_env; cd "$sm_path" &&
-				git rev-parse --verify HEAD) ||
-			die "fatal: $(eval_gettext "Unable to find current revision in submodule path '\$displaypath'")"
 		fi
 
 		if test -n "$remote"
@@ -441,7 +436,6 @@ cmd_update()
 			  ${update:+--update "$update"} \
 			  ${prefix:+--recursive-prefix "$prefix"} \
 			  ${sha1:+--oid "$sha1"} \
-			  ${subsha1:+--suboid "$subsha1"} \
 			  "--" \
 			  "$sm_path")
 
Signature · 1 line
2.33.GIT
author
#65 lore raw

PATCH v4 5/13 submodule--helper: get remote names from any repository

In reply to Glen Choo

From: Atharva Raykar <[email protected]>

get_default_remote() retrieves the name of a remote by resolving the refs from of the current repository's ref store.

Thus in order to use it for retrieving the remote name of a submodule, we have to start a new subprocess which runs from the submodule directory.

Let's instead introduce a function called repo_get_default_remote() which takes any repository object and retrieves the remote accordingly.

get_default_remote() is then defined as a call to repo_get_default_remote() with 'the_repository' passed to it.

Now that we have repo_get_default_remote(), we no longer have to start a subprocess that called submodule--helper get-default-remote from within the submodule directory.

So let's make a function called get_default_remote_submodule() which takes a submodule path, and returns the default remote for that submodule, all within the same process.

We can now use this function to save an unnecessary subprocess spawn in sync_submodule(), and also in a subsequent patch, which will require this functionality.

---
 builtin/submodule--helper.c | 38 ++++++++++++++++++++++---------------
 1 file changed, 23 insertions(+), 15 deletions(-)
builtin/submodule--helper.c +23 -15
index 77ca4270f4..b5a2d83029 100644
@@ -31,11 +31,13 @@
 typedef void (*each_submodule_fn)(const struct cache_entry *list_item,
 				  void *cb_data);
 
-static char *get_default_remote(void)
+static char *repo_get_default_remote(struct repository *repo)
 {
 	char *dest = NULL, *ret;
 	struct strbuf sb = STRBUF_INIT;
-	const char *refname = resolve_ref_unsafe("HEAD", 0, NULL, NULL);
+	struct ref_store *store = get_main_ref_store(repo);
+	const char *refname = refs_resolve_ref_unsafe(store, "HEAD", 0, NULL,
+						      NULL);
 
 	if (!refname)
 		die(_("No such ref: %s"), "HEAD");
@@ -48,7 +50,7 @@ static char *get_default_remote(void)
 		die(_("Expecting a full ref name, got %s"), refname);
 
 	strbuf_addf(&sb, "branch.%s.remote", refname);
-	if (git_config_get_string(sb.buf, &dest))
+	if (repo_config_get_string(repo, sb.buf, &dest))
 		ret = xstrdup("origin");
 	else
 		ret = dest;
@@ -57,6 +59,19 @@ static char *get_default_remote(void)
 	return ret;
 }
 
+static char *get_default_remote_submodule(const char *module_path)
+{
+	struct repository subrepo;
+
+	repo_submodule_init(&subrepo, the_repository, module_path, null_oid());
+	return repo_get_default_remote(&subrepo);
+}
+
+static char *get_default_remote(void)
+{
+	return repo_get_default_remote(the_repository);
+}
+
 static int print_default_remote(int argc, const char **argv, const char *prefix)
 {
 	char *remote;
@@ -1343,9 +1358,8 @@ static void sync_submodule(const char *path, const char *prefix,
 {
 	const struct submodule *sub;
 	char *remote_key = NULL;
-	char *sub_origin_url, *super_config_url, *displaypath;
+	char *sub_origin_url, *super_config_url, *displaypath, *default_remote;
 	struct strbuf sb = STRBUF_INIT;
-	struct child_process cp = CHILD_PROCESS_INIT;
 	char *sub_config_path = NULL;
 
 	if (!is_submodule_active(the_repository, path))
@@ -1384,21 +1398,15 @@ static void sync_submodule(const char *path, const char *prefix,
 	if (!is_submodule_populated_gently(path, NULL))
 		goto cleanup;
 
-	prepare_submodule_repo_env(&cp.env_array);
-	cp.git_cmd = 1;
-	cp.dir = path;
-	strvec_pushl(&cp.args, "submodule--helper",
-		     "print-default-remote", NULL);
-
 	strbuf_reset(&sb);
-	if (capture_command(&cp, &sb, 0))
+	default_remote = get_default_remote_submodule(path);
+	if (!default_remote)
 		die(_("failed to get the default remote for submodule '%s'"),
 		      path);
 
-	strbuf_strip_suffix(&sb, "\n");
-	remote_key = xstrfmt("remote.%s.url", sb.buf);
+	remote_key = xstrfmt("remote.%s.url", default_remote);
+	free(default_remote);
 
-	strbuf_reset(&sb);
 	submodule_to_gitdir(&sb, path);
 	strbuf_addstr(&sb, "/config");
 
Signature · 1 line
2.33.GIT
author
#66 lore raw

PATCH v4 6/13 submodule--helper: don't use bitfield indirection for parse_options()

In reply to Glen Choo

From: Ævar Arnfjörð Bjarmason <[email protected]>

Do away with the indirection of local variables added in c51f8f94e5b (submodule--helper: run update procedures from C, 2021-08-24).

These were only needed because in C you can't get a pointer to a single bit, so we were using intermediate variables instead.

---
 builtin/submodule--helper.c | 23 ++++++++++-------------
 1 file changed, 10 insertions(+), 13 deletions(-)
builtin/submodule--helper.c +10 -13
index b5a2d83029..3832dccae5 100644
@@ -2023,10 +2023,10 @@ struct update_data {
 	struct object_id suboid;
 	struct submodule_update_strategy update_strategy;
 	int depth;
-	unsigned int force: 1;
-	unsigned int quiet: 1;
-	unsigned int nofetch: 1;
-	unsigned int just_cloned: 1;
+	unsigned int force;
+	unsigned int quiet;
+	unsigned int nofetch;
+	unsigned int just_cloned;
 };
 #define UPDATE_DATA_INIT { .update_strategy = SUBMODULE_UPDATE_STRATEGY_INIT }
 
@@ -2578,16 +2578,17 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 
 static int run_update_procedure(int argc, const char **argv, const char *prefix)
 {
-	int force = 0, quiet = 0, nofetch = 0, just_cloned = 0;
 	char *prefixed_path, *update = NULL;
 	struct update_data update_data = UPDATE_DATA_INIT;
 
 	struct option options[] = {
-		OPT__QUIET(&quiet, N_("suppress output for update by rebase or merge")),
-		OPT__FORCE(&force, N_("force checkout updates"), 0),
-		OPT_BOOL('N', "no-fetch", &nofetch,
+		OPT__QUIET(&update_data.quiet,
+			   N_("suppress output for update by rebase or merge")),
+		OPT__FORCE(&update_data.force, N_("force checkout updates"),
+			   0),
+		OPT_BOOL('N', "no-fetch", &update_data.nofetch,
 			 N_("don't fetch new objects from the remote site")),
-		OPT_BOOL(0, "just-cloned", &just_cloned,
+		OPT_BOOL(0, "just-cloned", &update_data.just_cloned,
 			 N_("overrides update mode in case the repository is a fresh clone")),
 		OPT_INTEGER(0, "depth", &update_data.depth, N_("depth for shallow fetch")),
 		OPT_STRING(0, "prefix", &prefix,
@@ -2615,10 +2616,6 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)
 	if (argc != 1)
 		usage_with_options(usage, options);
 
-	update_data.force = !!force;
-	update_data.quiet = !!quiet;
-	update_data.nofetch = !!nofetch;
-	update_data.just_cloned = !!just_cloned;
 	update_data.sm_path = argv[0];
 
 	if (update_data.recursive_prefix)
Signature · 1 line
2.33.GIT
author
#67 lore raw

PATCH v4 7/13 submodule--helper run-update-procedure: learn --remote

In reply to Glen Choo

Teach run-update-procedure to handle --remote instead of parsing --remote in git-submodule.sh. As a result, "git submodule--helper [print-default-remote|remote-branch]" have no more callers, so remove them.

---
 builtin/submodule--helper.c | 56 +++++++++++++++----------------------
 git-submodule.sh            | 30 +-------------------
 2 files changed, 23 insertions(+), 63 deletions(-)
builtin/submodule--helper.c +22 -34
index 3832dccae5..f673f7ebbf 100644
@@ -72,21 +72,6 @@ static char *get_default_remote(void)
 	return repo_get_default_remote(the_repository);
 }
 
-static int print_default_remote(int argc, const char **argv, const char *prefix)
-{
-	char *remote;
-
-	if (argc != 1)
-		die(_("submodule--helper print-default-remote takes no arguments"));
-
-	remote = get_default_remote();
-	if (remote)
-		printf("%s\n", remote);
-
-	free(remote);
-	return 0;
-}
-
 static int starts_with_dot_slash(const char *str)
 {
 	return str[0] == '.' && is_dir_sep(str[1]);
@@ -2027,6 +2012,7 @@ struct update_data {
 	unsigned int quiet;
 	unsigned int nofetch;
 	unsigned int just_cloned;
+	unsigned int remote;
 };
 #define UPDATE_DATA_INIT { .update_strategy = SUBMODULE_UPDATE_STRATEGY_INIT }
 
@@ -2603,6 +2589,8 @@ static int run_update_procedure(int argc, const char **argv, const char *prefix)
 		OPT_CALLBACK_F(0, "oid", &update_data.oid, N_("sha1"),
 			       N_("SHA1 expected by superproject"), PARSE_OPT_NONEG,
 			       parse_opt_object_id),
+		OPT_BOOL(0, "remote", &update_data.remote,
+			 N_("use SHA-1 of submodule's remote tracking branch")),
 		OPT_END()
 	};
 
@@ -2682,23 +2670,6 @@ static const char *remote_submodule_branch(const char *path)
 	return branch;
 }
 
-static int resolve_remote_submodule_branch(int argc, const char **argv,
-		const char *prefix)
-{
-	const char *ret;
-	struct strbuf sb = STRBUF_INIT;
-	if (argc != 2)
-		die("submodule--helper remote-branch takes exactly one arguments, got %d", argc);
-
-	ret = remote_submodule_branch(argv[1]);
-	if (!ret)
-		die("submodule %s doesn't exist", argv[1]);
-
-	printf("%s", ret);
-	strbuf_release(&sb);
-	return 0;
-}
-
 static int push_check(int argc, const char **argv, const char *prefix)
 {
 	struct remote *remote;
@@ -3040,6 +3011,25 @@ static int update_submodule2(struct update_data *update_data)
 		die(_("Unable to find current revision in submodule path '%s'"),
 			update_data->displaypath);
 
+	if (update_data->remote) {
+		char *remote_name = get_default_remote_submodule(update_data->sm_path);
+		const char *branch = remote_submodule_branch(update_data->sm_path);
+		char *remote_ref = xstrfmt("refs/remotes/%s/%s", remote_name, branch);
+
+		if (!update_data->nofetch) {
+			if (fetch_in_submodule(update_data->sm_path, update_data->depth,
+					      0, NULL))
+				die(_("Unable to fetch in submodule path '%s'"),
+				    update_data->sm_path);
+		}
+
+		if (resolve_gitlink_ref(update_data->sm_path, remote_ref, &update_data->oid))
+			die(_("Unable to find %s revision in submodule path '%s'"),
+			    remote_ref, update_data->sm_path);
+
+		free(remote_ref);
+	}
+
 	if (!oideq(&update_data->oid, &update_data->suboid) || update_data->force)
 		return do_run_update_procedure(update_data);
 
@@ -3439,11 +3429,9 @@ static struct cmd_struct commands[] = {
 	{"foreach", module_foreach, SUPPORT_SUPER_PREFIX},
 	{"init", module_init, SUPPORT_SUPER_PREFIX},
 	{"status", module_status, SUPPORT_SUPER_PREFIX},
-	{"print-default-remote", print_default_remote, 0},
 	{"sync", module_sync, SUPPORT_SUPER_PREFIX},
 	{"deinit", module_deinit, 0},
 	{"summary", module_summary, SUPPORT_SUPER_PREFIX},
-	{"remote-branch", resolve_remote_submodule_branch, 0},
 	{"push-check", push_check, 0},
 	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
 	{"is-active", is_active, 0},
git-submodule.sh +1 -29
index 32a09302ab..882bf097d5 100755
@@ -247,20 +247,6 @@ cmd_deinit()
 	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper deinit ${GIT_QUIET:+--quiet} ${force:+--force} ${deinit_all:+--all} -- "$@"
 }
 
-# usage: fetch_in_submodule <module_path> [<depth>] [<sha1>]
-# Because arguments are positional, use an empty string to omit <depth>
-# but include <sha1>.
-fetch_in_submodule () (
-	sanitize_submodule_env &&
-	cd "$1" &&
-	if test $# -eq 3
-	then
-		echo "$3" | git fetch ${GIT_QUIET:+--quiet} --stdin ${2:+"$2"}
-	else
-		git fetch ${GIT_QUIET:+--quiet} ${2:+"$2"}
-	fi
-)
-
 #
 # Update each submodule path to correct revision, using clone and checkout as needed
 #
@@ -411,21 +397,6 @@ cmd_update()
 			just_cloned=
 		fi
 
-		if test -n "$remote"
-		then
-			branch=$(git submodule--helper remote-branch "$sm_path")
-			if test -z "$nofetch"
-			then
-				# Fetch remote before determining tracking $sha1
-				fetch_in_submodule "$sm_path" $depth ||
-				die "fatal: $(eval_gettext "Unable to fetch in submodule path '\$sm_path'")"
-			fi
-			remote_name=$(sanitize_submodule_env; cd "$sm_path" && git submodule--helper print-default-remote)
-			sha1=$(sanitize_submodule_env; cd "$sm_path" &&
-				git rev-parse --verify "${remote_name}/${branch}") ||
-			die "fatal: $(eval_gettext "Unable to find current \${remote_name}/\${branch} revision in submodule path '\$sm_path'")"
-		fi
-
 		out=$(git submodule--helper run-update-procedure \
 			  ${wt_prefix:+--prefix "$wt_prefix"} \
 			  ${GIT_QUIET:+--quiet} \
@@ -436,6 +407,7 @@ cmd_update()
 			  ${update:+--update "$update"} \
 			  ${prefix:+--recursive-prefix "$prefix"} \
 			  ${sha1:+--oid "$sha1"} \
+			  ${remote:+--remote} \
 			  "--" \
 			  "$sm_path")
 
Signature · 1 line
2.33.GIT
author
#68 lore raw

PATCH v4 8/13 submodule--helper: refactor get_submodule_displaypath()

In reply to Glen Choo

From: Atharva Raykar <[email protected]>

We create a function called do_get_submodule_displaypath() that generates the display path required by several submodule functions, and takes a custom superprefix parameter, instead of reading it from the environment.

We then redefine the existing get_submodule_displaypath() function as a call to this new function, where the superprefix is obtained from the environment.

---
 builtin/submodule--helper.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)
builtin/submodule--helper.c +10 -4
index f673f7ebbf..52d4f47ffc 100644
@@ -247,11 +247,10 @@ static int resolve_relative_url_test(int argc, const char **argv, const char *pr
 	return 0;
 }
 
-/* the result should be freed by the caller. */
-static char *get_submodule_displaypath(const char *path, const char *prefix)
+static char *do_get_submodule_displaypath(const char *path,
+					  const char *prefix,
+					  const char *super_prefix)
 {
-	const char *super_prefix = get_super_prefix();
-
 	if (prefix && super_prefix) {
 		BUG("cannot have prefix '%s' and superprefix '%s'",
 		    prefix, super_prefix);
@@ -267,6 +266,13 @@ static char *get_submodule_displaypath(const char *path, const char *prefix)
 	}
 }
 
+/* the result should be freed by the caller. */
+static char *get_submodule_displaypath(const char *path, const char *prefix)
+{
+	const char *super_prefix = get_super_prefix();
+	return do_get_submodule_displaypath(path, prefix, super_prefix);
+}
+
 static char *compute_rev_name(const char *sub_path, const char* object_id)
 {
 	struct strbuf sb = STRBUF_INIT;
Signature · 1 line
2.33.GIT
author
#69 lore raw

PATCH v4 9/13 submodule--helper: allow setting superprefix for init_submodule()

In reply to Glen Choo

From: Atharva Raykar <[email protected]>

We allow callers of the init_submodule() function to optionally override the superprefix from the environment.

We need to enable this option because in our conversion of the update command that will follow, the '--init' option will be handled through this API. We will need to change the superprefix at that time to ensure the display paths show correctly in the output messages.

---
 builtin/submodule--helper.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)
builtin/submodule--helper.c +7 -3
index 52d4f47ffc..c6df64bf6a 100644
@@ -594,18 +594,22 @@ static int module_foreach(int argc, const char **argv, const char *prefix)
 
 struct init_cb {
 	const char *prefix;
+	const char *superprefix;
 	unsigned int flags;
 };
 #define INIT_CB_INIT { 0 }
 
 static void init_submodule(const char *path, const char *prefix,
-			   unsigned int flags)
+			   const char *superprefix, unsigned int flags)
 {
 	const struct submodule *sub;
 	struct strbuf sb = STRBUF_INIT;
 	char *upd = NULL, *url = NULL, *displaypath;
 
-	displaypath = get_submodule_displaypath(path, prefix);
+	/* try superprefix from the environment, if it is not passed explicitly */
+	if (!superprefix)
+		superprefix = get_super_prefix();
+	displaypath = do_get_submodule_displaypath(path, prefix, superprefix);
 
 	sub = submodule_from_path(the_repository, null_oid(), path);
 
@@ -679,7 +683,7 @@ static void init_submodule(const char *path, const char *prefix,
 static void init_submodule_cb(const struct cache_entry *list_item, void *cb_data)
 {
 	struct init_cb *info = cb_data;
-	init_submodule(list_item->name, info->prefix, info->flags);
+	init_submodule(list_item->name, info->prefix, info->superprefix, info->flags);
 }
 
 static int module_init(int argc, const char **argv, const char *prefix)
Signature · 1 line
2.33.GIT
author
#70 lore raw

PATCH v4 10/13 submodule--helper update-clone: learn --init

In reply to Glen Choo

Teach "git submodule--helper update-clone" the --init flag and remove the corresponding shell code.

When the --init flag is passed to the subcommand, we do not spawn a new subprocess and call submodule--helper init on the submodule paths, because the Git machinery is not able to pick up the configuration changes introduced by that init call. So we instead run the init_submodule_cb() callback over each submodule in the same process.

[1] https://lore.kernel.org/git/CAP8UFD0NCQ5w_3GtT_xHr35i7h8BuLX4UcHNY6VHPGREmDVObA@mail.gmail.com/

---
 builtin/submodule--helper.c | 26 ++++++++++++++++++++++++++
 git-submodule.sh            |  9 +++------
 2 files changed, 29 insertions(+), 6 deletions(-)
builtin/submodule--helper.c +26 -0
index c6df64bf6a..17dabf3d12 100644
@@ -2000,6 +2000,7 @@ struct submodule_update_clone {
 	int failed_clones_nr, failed_clones_alloc;
 
 	int max_jobs;
+	unsigned int init;
 };
 #define SUBMODULE_UPDATE_CLONE_INIT { \
 	.list = MODULE_LIST_INIT, \
@@ -2509,6 +2510,8 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	int ret;
 
 	struct option module_update_clone_options[] = {
+		OPT_BOOL(0, "init", &suc.init,
+			 N_("initialize uninitialized submodules before update")),
 		OPT_STRING(0, "prefix", &prefix,
 			   N_("path"),
 			   N_("path into the working tree")),
@@ -2567,6 +2570,29 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	if (pathspec.nr)
 		suc.warn_if_uninitialized = 1;
 
+	if (suc.init) {
+		struct module_list list = MODULE_LIST_INIT;
+		struct init_cb info = INIT_CB_INIT;
+
+		if (module_list_compute(argc, argv, suc.prefix,
+					&pathspec, &list) < 0)
+			return 1;
+
+		/*
+		 * If there are no path args and submodule.active is set then,
+		 * by default, only initialize 'active' modules.
+		 */
+		if (!argc && git_config_get_value_multi("submodule.active"))
+			module_list_active(&list);
+
+		info.prefix = suc.prefix;
+		info.superprefix = suc.recursive_prefix;
+		if (suc.quiet)
+			info.flags |= OPT_QUIET;
+
+		for_each_listed_submodule(&list, init_submodule_cb, &info);
+	}
+
 	ret = update_submodules(&suc);
 	list_objects_filter_release(&filter_options);
 	return ret;
git-submodule.sh +3 -6
index 882bf097d5..16dea0ca59 100755
@@ -361,14 +361,11 @@ cmd_update()
 		usage
 	fi
 
-	if test -n "$init"
-	then
-		cmd_init "--" "$@" || return
-	fi
-
 	{
-	git submodule--helper update-clone ${GIT_QUIET:+--quiet} \
+	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper update-clone \
+		${GIT_QUIET:+--quiet} \
 		${progress:+"--progress"} \
+		${init:+--init} \
 		${wt_prefix:+--prefix "$wt_prefix"} \
 		${prefix:+--recursive-prefix "$prefix"} \
 		${update:+--update "$update"} \
Signature · 1 line
2.33.GIT
author
#71 lore raw

PATCH v4 11/13 submodule--helper: remove ensure-core-worktree

In reply to Glen Choo

Move the logic of "git submodule--helper ensure-core-worktree" into run-update-procedure, and since this makes the ensure-core-worktree command obsolete, remove it.

As a result, the order of two operations in git-submodule.sh is
reversed: 'set the value of core.worktree' now happens after the call to "git submodule--helper relative-path". This is safe - "relative-path" does not depend on the value of core.worktree.

---
 builtin/submodule--helper.c | 12 ++----------
 git-submodule.sh            |  2 --
 2 files changed, 2 insertions(+), 12 deletions(-)
builtin/submodule--helper.c +2 -10
index 17dabf3d12..296ab80bf2 100644
@@ -2783,17 +2783,11 @@ static int push_check(int argc, const char **argv, const char *prefix)
 	return 0;
 }
 
-static int ensure_core_worktree(int argc, const char **argv, const char *prefix)
+static void ensure_core_worktree(const char *path)
 {
-	const char *path;
 	const char *cw;
 	struct repository subrepo;
 
-	if (argc != 2)
-		BUG("submodule--helper ensure-core-worktree <path>");
-
-	path = argv[1];
-
 	if (repo_submodule_init(&subrepo, the_repository, path, null_oid()))
 		die(_("could not get a repository handle for submodule '%s'"), path);
 
@@ -2813,8 +2807,6 @@ static int ensure_core_worktree(int argc, const char **argv, const char *prefix)
 		free(abs_path);
 		strbuf_release(&sb);
 	}
-
-	return 0;
 }
 
 static int absorb_git_dirs(int argc, const char **argv, const char *prefix)
@@ -3041,6 +3033,7 @@ static int module_create_branch(int argc, const char **argv, const char *prefix)
 /* NEEDSWORK: this is a temporary name until we delete update_submodule() */
 static int update_submodule2(struct update_data *update_data)
 {
+	ensure_core_worktree(update_data->sm_path);
 	if (update_data->just_cloned)
 		oidcpy(&update_data->suboid, null_oid());
 	else if (resolve_gitlink_ref(update_data->sm_path, "HEAD", &update_data->suboid))
@@ -3459,7 +3452,6 @@ static struct cmd_struct commands[] = {
 	{"add", module_add, SUPPORT_SUPER_PREFIX},
 	{"update-clone", update_clone, 0},
 	{"run-update-procedure", run_update_procedure, 0},
-	{"ensure-core-worktree", ensure_core_worktree, 0},
 	{"relative-path", resolve_relative_path, 0},
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
 	{"foreach", module_foreach, SUPPORT_SUPER_PREFIX},
git-submodule.sh +0 -2
index 16dea0ca59..51be7c7f7e 100755
@@ -385,8 +385,6 @@ cmd_update()
 	do
 		die_if_unmatched "$quickabort" "$sha1"
 
-		git submodule--helper ensure-core-worktree "$sm_path" || exit 1
-
 		displaypath=$(git submodule--helper relative-path "$prefix$sm_path" "$wt_prefix")
 
 		if test $just_cloned -eq 0
Signature · 1 line
2.33.GIT
author
#72 lore raw

PATCH v4 12/13 submodule update: add tests for --filter

In reply to Glen Choo

Test the "--filter" option to make sure we don't break anything while refactoring "git submodule update".

---
 t/t7406-submodule-update.sh | 12 ++++++++++++
 1 file changed, 12 insertions(+)
t/t7406-submodule-update.sh +12 -0
index 7764c1c3cb..000e055811 100755
@@ -1071,4 +1071,16 @@ test_expect_success 'submodule update --quiet passes quietness to fetch with a s
 	)
 '
 
+test_expect_success 'submodule update --filter requires --init' '
+	test_expect_code 129 git -C super submodule update --filter blob:none
+'
+
+test_expect_success 'submodule update --filter sets partial clone settings' '
+	test_when_finished "rm -rf super-filter" &&
+	git clone cloned super-filter &&
+	git -C super-filter submodule update --init --filter blob:none &&
+	test_cmp_config -C super-filter/submodule true remote.origin.promisor &&
+	test_cmp_config -C super-filter/submodule blob:none remote.origin.partialclonefilter
+'
+
 test_done
Signature · 1 line
2.33.GIT
author
#73 lore raw

PATCH v4 13/13 submodule--helper update-clone: check for --filter and --init

In reply to Glen Choo

"git submodule update --filter" also requires the "--init" option. Teach update-clone to do this usage check in C and remove the check from git-submodule.sh.

In addition, change update-clone's usage string so that it teaches users about "git submodule update" instead of "git submodule--helper update-clone" (the string is copied from git-submodule.sh). This should be more helpful to users since they don't invoke update-clone directly.

---
 builtin/submodule--helper.c | 20 +++++++++++++++++++-
 git-submodule.sh            |  5 -----
 2 files changed, 19 insertions(+), 6 deletions(-)
builtin/submodule--helper.c +19 -1
index 296ab80bf2..bef9ab22d4 100644
@@ -2545,7 +2545,12 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	};
 
 	const char *const git_submodule_helper_usage[] = {
-		N_("git submodule--helper update-clone [--prefix=<path>] [<path>...]"),
+		N_("git submodule [--quiet] update"
+		" [--init [--filter=<filter-spec>]] [--remote]"
+		" [-N|--no-fetch] [-f|--force]"
+		" [--checkout|--merge|--rebase]"
+		" [--[no-]recommend-shallow] [--reference <repository>]"
+		" [--recursive] [--[no-]single-branch] [--] [<path>...]"),
 		NULL
 	};
 	suc.prefix = prefix;
@@ -2556,6 +2561,19 @@ static int update_clone(int argc, const char **argv, const char *prefix)
 	memset(&filter_options, 0, sizeof(filter_options));
 	argc = parse_options(argc, argv, prefix, module_update_clone_options,
 			     git_submodule_helper_usage, 0);
+
+	if (filter_options.choice && !suc.init) {
+		/*
+		 * NEEDSWORK: Don't use usage_with_options() because the
+		 * usage string is for "git submodule update", but the
+		 * options are for "git submodule--helper update-clone".
+		 *
+		 * This will no longer be an issue when "update-clone"
+		 * is replaced by "git submodule--helper update".
+		 */
+		usage(git_submodule_helper_usage[0]);
+	}
+
 	suc.filter_options = &filter_options;
 
 	if (update)
git-submodule.sh +0 -5
index 51be7c7f7e..aa8bdfca9d 100755
@@ -356,11 +356,6 @@ cmd_update()
 		shift
 	done
 
-	if test -n "$filter" && test "$init" != "1"
-	then
-		usage
-	fi
-
 	{
 	git ${wt_prefix:+-C "$wt_prefix"} submodule--helper update-clone \
 		${GIT_QUIET:+--quiet} \
Signature · 1 line
2.33.GIT
#74 lore raw
In reply to Glen Choo
Quoted 4 lines · Glen Choo

Original series: https://lore.kernel.org/git/[email protected]

= Patch summary

Quoted 11 lines

- Patch 1 adds extra tests to "git submodule update" to make sure we don't break anything
- Patch 2 removes dead code that used to be part of "git submodule update"
- Patch 3 prepares for later changes by introducing the C function that will hold most of the newly converted code
- Patch 4 moves run-update-procedure's --suboid option into C
- Patch 5 moves ensure-core-worktree into C
- Patches 6-8 move run-update-procedure's --remote option into C
- Patches 9-11 move "git submodule update"'s --init into C
- Patches 12-13 move "git submodule update"'s --filter option into C

Looking good. Will queue.

Let's see if there are further comments before declaring victory and merging it down to 'next' shortly.

Thanks.