Skip to contentSkip to conversation

submodule absorbgitdirs

PATCH Superseded Stefan Beller 2 participants 10 messages Dec 8, 2016 – Dec 10, 2016 Documentation

v7 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

v7 cover submodule absorbgitdirs

v7:
* do not expose submodule_get_worktrees. The values may be wrong internally, but for our purpose we do not care about the actual values, only the count. Document the wrong values!
* more strings are _(marked up)
* renamed variables in connect_work_tree_and_git_dir
* unsigned instead of int for options in the submodule helper.
*

v6:
* renamed embedgitdirs to absorbgitdirs embedding may be interpreted as embedding the git dir into the working directory, whereas absorbing sounds more like the submodule is absorbed by the superproject, making the submodule less independent
* Worktrees API offer uses_worktrees(void) and submodule_uses_worktree(path).
* moved the printing to stderr and one layer up (out of the pure relocate_git_dir function).
* connect_... is in dir.h now.

v5:
* Add another layer of abstraction, i.e. the relocate_git_dir is only about moving a git dir of one repository. The submodule specific stuff (e.g. recursion into nested submodules) is in submodule.{c,h}

This was motivated by reviews on the series of checkout aware of submodules building on top of this series, as we want to directly call the embed-git-dirs function without the overhead of spawning a child process.

v4:
* rebuilt on top of nd/worktree-list-fixup
* fix and test behavior for un-init submodules (don't crash, rather do nothing)
* incorporated a "static" as pointed out by Ramsay
* use internal functions instead of duplicating code in worktree.c (use get_common_dir_noenv for the submodule to actually get the common dir)
* fixed a memory leak in relocate_gitdir

v3:
* have a slightly more generic function "relocate_gitdir". The recursion is strictly related to submodules, though.
* bail out if a submodule is using worktrees.
This also lays the groundwork for later doing the proper thing, as worktree.h offers a function get_submodule_worktrees(path)
* nit by duy: use git_path instead of git_common_dir

v2:
* fixed commit message for patch:
"submodule: use absolute path for computing relative path connecting"
* a new patch "submodule helper: support super prefix"
* redid the final patch with more tests and fixing bugs along the way
* "test-lib-functions.sh: teach test_commit -C <dir>" unchanged

v1:
The discussion of the submodule checkout series revealed to me that a command is needed to move the git directory from the submodules working tree to be embedded into the superprojects git directory.

So I wrote the code to intern the submodules git dir into the superproject, but whilst writing the code I realized this could be valueable for our use in testing too. So I exposed it via the submodule--helper. But as the submodule helper ought to be just an internal API, we could also offer it via the proper submodule command.

The command as it is has little value to the end user for now, but breaking it out of the submodule checkout series hopefully makes review easier.

Thanks,
Stefan
Stefan Beller (6):
submodule: use absolute path for computing relative path connecting submodule helper: support super prefix test-lib-functions.sh: teach test_commit -C <dir>
worktree: have a function to check if worktrees are in use move connect_work_tree_and_git_dir to dir.h
submodule: add absorb-git-dir function

 Documentation/git-submodule.txt    |  15 +++++
 builtin/submodule--helper.c        |  69 ++++++++++++++++----
 dir.c                              |  37 +++++++++++
 dir.h                              |   4 ++
 git-submodule.sh                   |   7 +-
 git.c                              |   2 +-
 submodule.c                        | 127 ++++++++++++++++++++++++++++++-------
 submodule.h                        |   5 +-
 t/t7412-submodule-absorbgitdirs.sh | 101 +++++++++++++++++++++++++++++
 t/test-lib-functions.sh            |  20 ++++--
 worktree.c                         |  68 +++++++++++++++++---
 worktree.h                         |   7 ++
 12 files changed, 409 insertions(+), 53 deletions(-)
 create mode 100755 t/t7412-submodule-absorbgitdirs.sh
Signature · 1 line
2.11.0.rc2.29.g7c00390.dirty
#1 lore raw

PATCH v7 1/6 submodule: use absolute path for computing relative path connecting

The current caller of connect_work_tree_and_git_dir passes
an absolute path for the git_dir parameter. In the future patch we will also pass in relative path for git_dir. Extend the functionality of connect_work_tree_and_git_dir to take relative paths for parameters.

We could work around this in the future patch by computing the absolute path for the git_dir in the calling site, however accepting relative paths for either parameter makes the API for this function much harder to misuse.

---
 submodule.c | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
submodule.c +7 -6
index 6f7d883de9..d4f7afe2f1 100644
@@ -1223,27 +1223,28 @@ int merge_submodule(unsigned char result[20], const char *path,
 }
 
 /* Update gitfile and core.worktree setting to connect work tree and git dir */
-void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir)
+void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)
 {
 	struct strbuf file_name = STRBUF_INIT;
 	struct strbuf rel_path = STRBUF_INIT;
-	const char *real_work_tree = xstrdup(real_path(work_tree));
+	char *git_dir = xstrdup(real_path(git_dir_));
+	char *work_tree = xstrdup(real_path(work_tree_));
 
 	/* Update gitfile */
 	strbuf_addf(&file_name, "%s/.git", work_tree);
 	write_file(file_name.buf, "gitdir: %s",
-		   relative_path(git_dir, real_work_tree, &rel_path));
+		   relative_path(git_dir, work_tree, &rel_path));
 
 	/* Update core.worktree setting */
 	strbuf_reset(&file_name);
 	strbuf_addf(&file_name, "%s/config", git_dir);
 	git_config_set_in_file(file_name.buf, "core.worktree",
-			       relative_path(real_work_tree, git_dir,
-					     &rel_path));
+			       relative_path(work_tree, git_dir, &rel_path));
 
 	strbuf_release(&file_name);
 	strbuf_release(&rel_path);
-	free((void *)real_work_tree);
+	free(work_tree);
+	free(git_dir);
 }
 
 int parallel_submodules(void)
Signature · 1 line
2.11.0.rc2.29.g7c00390.dirty
#2 lore raw

PATCH v7 2/6 submodule helper: support super prefix

In reply to Stefan Beller

Just like main commands in Git, the submodule helper needs access to the superproject prefix. Enable this in the git.c but have its own fuse in the helper code by having a flag to turn on the super prefix.

---
 builtin/submodule--helper.c | 31 ++++++++++++++++++++-----------
 git.c                       |  2 +-
 2 files changed, 21 insertions(+), 12 deletions(-)
builtin/submodule--helper.c +20 -11
index 4beeda5f9f..5643848667 100644
@@ -1076,21 +1076,24 @@ static int resolve_remote_submodule_branch(int argc, const char **argv,
 	return 0;
 }
 
+#define SUPPORT_SUPER_PREFIX (1<<0)
+
 struct cmd_struct {
 	const char *cmd;
 	int (*fn)(int, const char **, const char *);
+	unsigned option;
 };
 
 static struct cmd_struct commands[] = {
-	{"list", module_list},
-	{"name", module_name},
-	{"clone", module_clone},
-	{"update-clone", update_clone},
-	{"relative-path", resolve_relative_path},
-	{"resolve-relative-url", resolve_relative_url},
-	{"resolve-relative-url-test", resolve_relative_url_test},
-	{"init", module_init},
-	{"remote-branch", resolve_remote_submodule_branch}
+	{"list", module_list, 0},
+	{"name", module_name, 0},
+	{"clone", module_clone, 0},
+	{"update-clone", update_clone, 0},
+	{"relative-path", resolve_relative_path, 0},
+	{"resolve-relative-url", resolve_relative_url, 0},
+	{"resolve-relative-url-test", resolve_relative_url_test, 0},
+	{"init", module_init, 0},
+	{"remote-branch", resolve_remote_submodule_branch, 0},
 };
 
 int cmd_submodule__helper(int argc, const char **argv, const char *prefix)
@@ -1100,9 +1103,15 @@ int cmd_submodule__helper(int argc, const char **argv, const char *prefix)
 		die(_("submodule--helper subcommand must be "
 		      "called with a subcommand"));
 
-	for (i = 0; i < ARRAY_SIZE(commands); i++)
-		if (!strcmp(argv[1], commands[i].cmd))
+	for (i = 0; i < ARRAY_SIZE(commands); i++) {
+		if (!strcmp(argv[1], commands[i].cmd)) {
+			if (get_super_prefix() &&
+			    !(commands[i].option & SUPPORT_SUPER_PREFIX))
+				die(_("%s doesn't support --super-prefix"),
+				    commands[i].cmd);
 			return commands[i].fn(argc - 1, argv + 1, prefix);
+		}
+	}
 
 	die(_("'%s' is not a valid submodule--helper "
 	      "subcommand"), argv[1]);
git.c +1 -1
index efa1059fe0..98dcf6c518 100644
@@ -493,7 +493,7 @@ static struct cmd_struct commands[] = {
 	{ "stage", cmd_add, RUN_SETUP | NEED_WORK_TREE },
 	{ "status", cmd_status, RUN_SETUP | NEED_WORK_TREE },
 	{ "stripspace", cmd_stripspace },
-	{ "submodule--helper", cmd_submodule__helper, RUN_SETUP },
+	{ "submodule--helper", cmd_submodule__helper, RUN_SETUP | SUPPORT_SUPER_PREFIX},
 	{ "symbolic-ref", cmd_symbolic_ref, RUN_SETUP },
 	{ "tag", cmd_tag, RUN_SETUP },
 	{ "unpack-file", cmd_unpack_file, RUN_SETUP },
Signature · 1 line
2.11.0.rc2.29.g7c00390.dirty
#3 lore raw

PATCH v7 3/6 test-lib-functions.sh: teach test_commit -C <dir>

In reply to Stefan Beller

Specifically when setting up submodule tests, it comes in handy if we can create commits in repositories that are not at the root of the tested trash dir. Add "-C <dir>" similar to gits -C parameter that will perform the operation in the given directory.

---
 t/test-lib-functions.sh | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)
t/test-lib-functions.sh +15 -5
index fdaeb3a96b..579e812506 100644
@@ -157,16 +157,21 @@ debug () {
 	 GIT_TEST_GDB=1 "$@"
 }
 
-# Call test_commit with the arguments "<message> [<file> [<contents> [<tag>]]]"
+# Call test_commit with the arguments
+# [-C <directory>] <message> [<file> [<contents> [<tag>]]]"
 #
 # This will commit a file with the given contents and the given commit
 # message, and tag the resulting commit with the given tag name.
 #
 # <file>, <contents>, and <tag> all default to <message>.
+#
+# If the first argument is "-C", the second argument is used as a path for
+# the git invocations.
 
 test_commit () {
 	notick= &&
 	signoff= &&
+	indir= &&
 	while test $# != 0
 	do
 		case "$1" in
@@ -176,21 +181,26 @@ test_commit () {
 		--signoff)
 			signoff="$1"
 			;;
+		-C)
+			indir="$2"
+			shift
+			;;
 		*)
 			break
 			;;
 		esac
 		shift
 	done &&
+	indir=${indir:+"$indir"/} &&
 	file=${2:-"$1.t"} &&
-	echo "${3-$1}" > "$file" &&
-	git add "$file" &&
+	echo "${3-$1}" > "$indir$file" &&
+	git ${indir:+ -C "$indir"} add "$file" &&
 	if test -z "$notick"
 	then
 		test_tick
 	fi &&
-	git commit $signoff -m "$1" &&
-	git tag "${4:-$1}"
+	git ${indir:+ -C "$indir"} commit $signoff -m "$1" &&
+	git ${indir:+ -C "$indir"} tag "${4:-$1}"
 }
 
 # Call test_merge with the arguments "<message> <commit>", where <commit>
Signature · 1 line
2.11.0.rc2.29.g7c00390.dirty
#4 lore raw

PATCH v7 4/6 worktree: have a function to check if worktrees are in use

In reply to Stefan Beller

In a later patch we want to move around the the git directory of a submodule. Both submodules as well as worktrees are involved in placing git directories at unusual places, so their functionality may collide. To react appropriately to situations where worktrees in submodules are in use, offer a new function to query the a submodule if it uses the worktree feature.

This can be done cheaply (both in new code to write as well as run time) by obtaining the list of worktrees based off that submodules git directory. However as we have loaded the variables for the current repository, the values in the submodule worktree
can be wrong, e.g.
* core.ignorecase may differ between these two repositories
* the ref resolution is broken (refs/heads/branch in the submodule resolves to the sha1 value of the branch in the current repository that may not exist or have another sha1)

---
 worktree.c | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++++---------
 worktree.h |  7 +++++++
 2 files changed, 65 insertions(+), 10 deletions(-)
worktree.c +58 -10
index eb6121263b..1c46fcf25f 100644
@@ -72,7 +72,7 @@ static void add_head_info(struct strbuf *head_ref, struct worktree *worktree)
 /**
  * get the main worktree
  */
-static struct worktree *get_main_worktree(void)
+static struct worktree *get_main_worktree(const char *git_common_dir)
 {
 	struct worktree *worktree = NULL;
 	struct strbuf path = STRBUF_INIT;
@@ -81,12 +81,12 @@ static struct worktree *get_main_worktree(void)
 	int is_bare = 0;
 	int is_detached = 0;
 
-	strbuf_add_absolute_path(&worktree_path, get_git_common_dir());
+	strbuf_add_absolute_path(&worktree_path, git_common_dir);
 	is_bare = !strbuf_strip_suffix(&worktree_path, "/.git");
 	if (is_bare)
 		strbuf_strip_suffix(&worktree_path, "/.");
 
-	strbuf_addf(&path, "%s/HEAD", get_git_common_dir());
+	strbuf_addf(&path, "%s/HEAD", git_common_dir);
 
 	worktree = xcalloc(1, sizeof(*worktree));
 	worktree->path = strbuf_detach(&worktree_path, NULL);
@@ -101,7 +101,8 @@ static struct worktree *get_main_worktree(void)
 	return worktree;
 }
 
-static struct worktree *get_linked_worktree(const char *id)
+static struct worktree *get_linked_worktree(const char *git_common_dir,
+					    const char *id)
 {
 	struct worktree *worktree = NULL;
 	struct strbuf path = STRBUF_INIT;
@@ -112,7 +113,7 @@ static struct worktree *get_linked_worktree(const char *id)
 	if (!id)
 		die("Missing linked worktree name");
 
-	strbuf_git_common_path(&path, "worktrees/%s/gitdir", id);
+	strbuf_addf(&path, "%s/worktrees/%s/gitdir", git_common_dir, id);
 	if (strbuf_read_file(&worktree_path, path.buf, 0) <= 0)
 		/* invalid gitdir file */
 		goto done;
@@ -125,7 +126,7 @@ static struct worktree *get_linked_worktree(const char *id)
 	}
 
 	strbuf_reset(&path);
-	strbuf_addf(&path, "%s/worktrees/%s/HEAD", get_git_common_dir(), id);
+	strbuf_addf(&path, "%s/worktrees/%s/HEAD", git_common_dir, id);
 
 	if (parse_ref(path.buf, &head_ref, &is_detached) < 0)
 		goto done;
@@ -167,7 +168,8 @@ static int compare_worktree(const void *a_, const void *b_)
 	return fspathcmp((*a)->path, (*b)->path);
 }
 
-struct worktree **get_worktrees(unsigned flags)
+static struct worktree **get_worktrees_internal(const char *git_common_dir,
+						unsigned flags)
 {
 	struct worktree **list = NULL;
 	struct strbuf path = STRBUF_INIT;
@@ -177,9 +179,10 @@ struct worktree **get_worktrees(unsigned flags)
 
 	list = xmalloc(alloc * sizeof(struct worktree *));
 
-	list[counter++] = get_main_worktree();
+	if (!(flags & GWT_LINKED_ONLY))
+		list[counter++] = get_main_worktree(git_common_dir);
 
-	strbuf_addf(&path, "%s/worktrees", get_git_common_dir());
+	strbuf_addf(&path, "%s/worktrees", git_common_dir);
 	dir = opendir(path.buf);
 	strbuf_release(&path);
 	if (dir) {
@@ -188,7 +191,7 @@ struct worktree **get_worktrees(unsigned flags)
 			if (is_dot_or_dotdot(d->d_name))
 				continue;
 
-			if ((linked = get_linked_worktree(d->d_name))) {
+			if ((linked = get_linked_worktree(git_common_dir, d->d_name))) {
 				ALLOC_GROW(list, counter + 1, alloc);
 				list[counter++] = linked;
 			}
@@ -209,6 +212,34 @@ struct worktree **get_worktrees(unsigned flags)
 	return list;
 }
 
+struct worktree **get_worktrees(unsigned flags)
+{
+	return get_worktrees_internal(get_git_common_dir(), flags);
+}
+
+/*
+ * NEEDSWORK: The values in the returned worktrees are broken, e.g.
+ * the refs or path resolution is influenced by the current repository.
+ */
+static struct worktree **get_submodule_worktrees(const char *path, unsigned flags)
+{
+	char *submodule_gitdir;
+	struct strbuf sb = STRBUF_INIT;
+	struct worktree **ret;
+
+	submodule_gitdir = git_pathdup_submodule(path, "%s", "");
+	if (!submodule_gitdir)
+		return NULL;
+
+	/* the env would be set for the superproject */
+	get_common_dir_noenv(&sb, submodule_gitdir);
+	ret = get_worktrees_internal(sb.buf, flags);
+
+	strbuf_release(&sb);
+	free(submodule_gitdir);
+	return ret;
+}
+
 const char *get_worktree_git_dir(const struct worktree *wt)
 {
 	if (!wt)
@@ -380,3 +411,20 @@ const struct worktree *find_shared_symref(const char *symref,
 
 	return existing;
 }
+
+int uses_worktrees(void)
+{
+	struct worktree **worktrees = get_worktrees(GWT_LINKED_ONLY);
+	int retval = (worktrees != NULL && worktrees[0] != NULL);
+	free_worktrees(worktrees);
+	return retval;
+}
+
+int submodule_uses_worktrees(const char *path)
+{
+	struct worktree **worktrees =
+			get_submodule_worktrees(path, GWT_LINKED_ONLY);
+	int retval = (worktrees != NULL && worktrees[0] != NULL);
+	free_worktrees(worktrees);
+	return retval;
+}
worktree.h +7 -0
index d59ce1fee8..ebe0856330 100644
@@ -16,6 +16,7 @@ struct worktree {
 /* Functions for acting on the information about worktrees. */
 
 #define GWT_SORT_LINKED (1 << 0) /* keeps linked worktrees sorted */
+#define GWT_LINKED_ONLY (1 << 1) /* do not include the main worktree */
 
 /*
  * Get the worktrees.  The primary worktree will always be the first returned,
@@ -27,6 +28,12 @@ struct worktree {
  */
 extern struct worktree **get_worktrees(unsigned flags);
 
+/*
+ * Returns 1 if linked worktrees exist, 0 otherwise.
+ */
+extern int uses_worktrees(void);
+extern int submodule_uses_worktrees(const char *path);
+
 /*
  * Return git dir of the worktree. Note that the path may be relative.
  * If wt is NULL, git dir of current worktree is returned.
Signature · 1 line
2.11.0.rc2.29.g7c00390.dirty
#5 lore raw

Re: [PATCHv7 4/6] worktree: have a function to check if worktrees are in use

worktree.c struct worktree **get_worktrees(unsigned flags) · line 224

Quoted 5 lines · Stefan Beller
worktree.c +5 -0
+/*
+ * NEEDSWORK: The values in the returned worktrees are broken, e.g.
+ * the refs or path resolution is influenced by the current repository.
+ */
+static struct worktree **get_submodule_worktrees(const char *path, unsigned flags)

I'm ok with this (at least we know the function is half-broken). But I wonder if we could go simpler, with something like this. It's more efficient as well. And we can replace its implementation with get_worktrees() or something when we are able to.

As long as this function stays in worktree.c I think it's ok for it to peek into "worktrees" directory directly. That's what all other functions in this file do after all.

int submodule_uses_worktrees(const char *path)

{
	struct strbuf path = STRBUF_INIT;
	DIR *dir;
	struct dirent *d;
	int ret = 0;

	strbuf_addf(&path, "%s/worktrees", path);
	dir = opendir(path.buf);
	strbuf_release(&path);

	if (!dir)
		return 0;

	while ((d = readdir(dir)) != NULL) {
		if (is_dot_or_dotdot(d->d_name))
			continue;

		ret = 1;
		break;
	}
	closedir(dir);
	return ret;
}

--
Duy

#6 lore raw

Re: [PATCHv7 4/6] worktree: have a function to check if worktrees are in use

Quoted 13 lines · Duy Nguyen

int submodule_uses_worktrees(const char *path)

{
        struct strbuf path = STRBUF_INIT;
        DIR *dir;
        struct dirent *d;
        int ret = 0;

        strbuf_addf(&path, "%s/worktrees", path);
        dir = opendir(path.buf);
        strbuf_release(&path);

        if (!dir)
                return 0;

The submodule may be one of the linked worktrees, which would be caught if we use the code as I sent it out?

If this is one of the linked worktrees, we'd rather check if a file "commondir" or "gitdir" exists?

I ask that because I would not know how to relocate such a linked worktree gitdir?

#7 lore raw

Re: [PATCHv7 4/6] worktree: have a function to check if worktrees are in use

Quoted 13 lines · Stefan Beller
Quoted 11 lines · Duy Nguyen

int submodule_uses_worktrees(const char *path)

{
        struct strbuf path = STRBUF_INIT;
        DIR *dir;
        struct dirent *d;
        int ret = 0;

        strbuf_addf(&path, "%s/worktrees", path);
        dir = opendir(path.buf);
        strbuf_release(&path);
Quoted 5 lines
Quoted 2 lines
        if (!dir)
                return 0;

The submodule may be one of the linked worktrees, which would be caught if we use the code as I sent it out?

I think I simplified it too much, there should still be a git_common_dir_noenv() to retrieve the correct common dir in the submodule from gitdir. Then, if the repo in question has any linked worktrees, you probably can't handle it. It does not matter if the submodule gitdir in question is a linked or main worktree.

Quoted 2 lines

If this is one of the linked worktrees, we'd rather check if a file "commondir" or "gitdir" exists?

Well, in theory yes. But we're apparently not ready for that. If you check the files exist, but the files are not valid, then it's still not considered a worktree. Which is not much different from what we do here (if the directory exists, assuming it's a worktree). It should catch all the valid worktrees. But yes it could give some false positive. Since we're just using this function to stop the aborbing git dir operation, i think this is acceptable.

(It goes even trickier, even get_worktrees() won't detect if the linked worktree is already dead, .e.g. deleted manually, I believe. Those have to be cleaned up by 'git worktree prune' which does even more checks before it declare "this directory is garbage").

Quoted 2 lines

I ask that because I would not know how to relocate such a linked worktree gitdir?

Signature · 1 line
Duy
#8 lore raw

PATCH v7 5/6 move connect_work_tree_and_git_dir to dir.h

In reply to Stefan Beller

That function was primarily used by submodule code, but the function itself is not inherently about submodules. In the next patch we'll introduce relocate_git_dir, which can be used by worktrees as well, so find a neutral middle ground in dir.h.

---
 dir.c       | 25 +++++++++++++++++++++++++
 dir.h       |  1 +
 submodule.c | 25 -------------------------
 submodule.h |  1 -
 4 files changed, 26 insertions(+), 26 deletions(-)
dir.c +25 -0
index bfa8c8a9a5..e0efd3c2c3 100644
@@ -2748,3 +2748,28 @@ void untracked_cache_add_to_index(struct index_state *istate,
 {
 	untracked_cache_invalidate_path(istate, path);
 }
+
+/* Update gitfile and core.worktree setting to connect work tree and git dir */
+void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)
+{
+	struct strbuf file_name = STRBUF_INIT;
+	struct strbuf rel_path = STRBUF_INIT;
+	char *git_dir = xstrdup(real_path(git_dir_));
+	char *work_tree = xstrdup(real_path(work_tree_));
+
+	/* Update gitfile */
+	strbuf_addf(&file_name, "%s/.git", work_tree);
+	write_file(file_name.buf, "gitdir: %s",
+		   relative_path(git_dir, work_tree, &rel_path));
+
+	/* Update core.worktree setting */
+	strbuf_reset(&file_name);
+	strbuf_addf(&file_name, "%s/config", git_dir);
+	git_config_set_in_file(file_name.buf, "core.worktree",
+			       relative_path(work_tree, git_dir, &rel_path));
+
+	strbuf_release(&file_name);
+	strbuf_release(&rel_path);
+	free(work_tree);
+	free(git_dir);
+}
dir.h +1 -0
index 97c83bb383..051674a431 100644
@@ -335,4 +335,5 @@ struct untracked_cache *read_untracked_extension(const void *data, unsigned long
 void write_untracked_extension(struct strbuf *out, struct untracked_cache *untracked);
 void add_untracked_cache(struct index_state *istate);
 void remove_untracked_cache(struct index_state *istate);
+extern void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);
 #endif
submodule.c +0 -25
index d4f7afe2f1..0bb50b4b62 100644
@@ -1222,31 +1222,6 @@ int merge_submodule(unsigned char result[20], const char *path,
 	return 0;
 }
 
-/* Update gitfile and core.worktree setting to connect work tree and git dir */
-void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)
-{
-	struct strbuf file_name = STRBUF_INIT;
-	struct strbuf rel_path = STRBUF_INIT;
-	char *git_dir = xstrdup(real_path(git_dir_));
-	char *work_tree = xstrdup(real_path(work_tree_));
-
-	/* Update gitfile */
-	strbuf_addf(&file_name, "%s/.git", work_tree);
-	write_file(file_name.buf, "gitdir: %s",
-		   relative_path(git_dir, work_tree, &rel_path));
-
-	/* Update core.worktree setting */
-	strbuf_reset(&file_name);
-	strbuf_addf(&file_name, "%s/config", git_dir);
-	git_config_set_in_file(file_name.buf, "core.worktree",
-			       relative_path(work_tree, git_dir, &rel_path));
-
-	strbuf_release(&file_name);
-	strbuf_release(&rel_path);
-	free(work_tree);
-	free(git_dir);
-}
-
 int parallel_submodules(void)
 {
 	return parallel_jobs;
submodule.h +0 -1
index d9e197a948..4e3bf469b4 100644
@@ -65,7 +65,6 @@ int merge_submodule(unsigned char result[20], const char *path, const unsigned c
 int find_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_name,
 		struct string_list *needs_pushing);
 int push_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_name);
-void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);
 int parallel_submodules(void);
 
 /*
Signature · 1 line
2.11.0.rc2.29.g7c00390.dirty
#9 lore raw

PATCH v7 6/6 submodule: add absorb-git-dir function

In reply to Stefan Beller

When a submodule has its git dir inside the working dir, the submodule support for checkout that we plan to add in a later patch will fail.

Add functionality to migrate the git directory to be absorbed into the superprojects git directory.

The newly added code in this patch is structured such that other areas of Git can also make use of it. The code in the submodule--helper is a mere wrapper and option parser for the function absorb_git_dir_into_superproject, that takes care of embedding the submodules git directory into the superprojects git dir. That function makes use of the more abstract function for this use case relocate_gitdir, which can be used by e.g. the worktree code eventually to move around a git directory.

---
 Documentation/git-submodule.txt    |  15 ++++++
 builtin/submodule--helper.c        |  38 ++++++++++++++
 dir.c                              |  12 +++++
 dir.h                              |   3 ++
 git-submodule.sh                   |   7 ++-
 submodule.c                        | 103 +++++++++++++++++++++++++++++++++++++
 submodule.h                        |   4 ++
 t/t7412-submodule-absorbgitdirs.sh | 101 ++++++++++++++++++++++++++++++++++++
 8 files changed, 282 insertions(+), 1 deletion(-)
 create mode 100755 t/t7412-submodule-absorbgitdirs.sh
Documentation/git-submodule.txt +15 -0
index d841573475..918bd1d1bd 100644
@@ -22,6 +22,7 @@ SYNOPSIS
 	      [commit] [--] [<path>...]
 'git submodule' [--quiet] foreach [--recursive] <command>
 'git submodule' [--quiet] sync [--recursive] [--] [<path>...]
+'git submodule' [--quiet] absorbgitdirs [--] [<path>...]
 
 
 DESCRIPTION
@@ -245,6 +246,20 @@ sync::
 If `--recursive` is specified, this command will recurse into the
 registered submodules, and sync any nested submodules within.
 
+absorbgitdirs::
+	If a git directory of a submodule is inside the submodule,
+	move the git directory of the submodule into its superprojects
+	`$GIT_DIR/modules` path and then connect the git directory and
+	its working directory by setting the `core.worktree` and adding
+	a .git file pointing to the git directory embedded in the
+	superprojects git directory.
++
+A repository that was cloned independently and later added as a submodule or
+old setups have the submodules git directory inside the submodule instead of
+embedded into the superprojects git directory.
++
+This command is recursive by default.
+
 OPTIONS
 -------
 -q::
builtin/submodule--helper.c +38 -0
index 5643848667..242d9911a6 100644
@@ -1076,6 +1076,43 @@ static int resolve_remote_submodule_branch(int argc, const char **argv,
 	return 0;
 }
 
+static int absorb_git_dirs(int argc, const char **argv, const char *prefix)
+{
+	int i;
+	struct pathspec pathspec;
+	struct module_list list = MODULE_LIST_INIT;
+	unsigned flags = ABSORB_GITDIR_RECURSE_SUBMODULES;
+
+	struct option embed_gitdir_options[] = {
+		OPT_STRING(0, "prefix", &prefix,
+			   N_("path"),
+			   N_("path into the working tree")),
+		OPT_BIT(0, "--recursive", &flags, N_("recurse into submodules"),
+			ABSORB_GITDIR_RECURSE_SUBMODULES),
+		OPT_END()
+	};
+
+	const char *const git_submodule_helper_usage[] = {
+		N_("git submodule--helper embed-git-dir [<path>...]"),
+		NULL
+	};
+
+	argc = parse_options(argc, argv, prefix, embed_gitdir_options,
+			     git_submodule_helper_usage, 0);
+
+	gitmodules_config();
+	git_config(submodule_config, NULL);
+
+	if (module_list_compute(argc, argv, prefix, &pathspec, &list) < 0)
+		return 1;
+
+	for (i = 0; i < list.nr; i++)
+		absorb_git_dir_into_superproject(prefix,
+				list.entries[i]->name, flags);
+
+	return 0;
+}
+
 #define SUPPORT_SUPER_PREFIX (1<<0)
 
 struct cmd_struct {
@@ -1094,6 +1131,7 @@ static struct cmd_struct commands[] = {
 	{"resolve-relative-url-test", resolve_relative_url_test, 0},
 	{"init", module_init, 0},
 	{"remote-branch", resolve_remote_submodule_branch, 0},
+	{"absorb-git-dirs", absorb_git_dirs, SUPPORT_SUPER_PREFIX},
 };
 
 int cmd_submodule__helper(int argc, const char **argv, const char *prefix)
dir.c +12 -0
index e0efd3c2c3..d872cc1570 100644
@@ -2773,3 +2773,15 @@ void connect_work_tree_and_git_dir(const char *work_tree_, const char *git_dir_)
 	free(work_tree);
 	free(git_dir);
 }
+
+/*
+ * Migrate the git directory of the given path from old_git_dir to new_git_dir.
+ */
+void relocate_gitdir(const char *path, const char *old_git_dir, const char *new_git_dir)
+{
+	if (rename(old_git_dir, new_git_dir) < 0)
+		die_errno(_("could not migrate git directory from '%s' to '%s'"),
+			old_git_dir, new_git_dir);
+
+	connect_work_tree_and_git_dir(path, new_git_dir);
+}
dir.h +3 -0
index 051674a431..bf23a470af 100644
@@ -336,4 +336,7 @@ void write_untracked_extension(struct strbuf *out, struct untracked_cache *untra
 void add_untracked_cache(struct index_state *istate);
 void remove_untracked_cache(struct index_state *istate);
 extern void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);
+extern void relocate_gitdir(const char *path,
+			    const char *old_git_dir,
+			    const char *new_git_dir);
 #endif
git-submodule.sh +6 -1
index a024a135d6..9285b5c43d 100755
@@ -1131,6 +1131,11 @@ cmd_sync()
 	done
 }
 
+cmd_absorbgitdirs()
+{
+	git submodule--helper absorb-git-dirs --prefix "$wt_prefix" "$@"
+}
+
 # This loop parses the command line arguments to find the
 # subcommand name to dispatch.  Parsing of the subcommand specific
 # options are primarily done by the subcommand implementations.
@@ -1140,7 +1145,7 @@ cmd_sync()
 while test $# != 0 && test -z "$command"
 do
 	case "$1" in
-	add | foreach | init | deinit | update | status | summary | sync)
+	add | foreach | init | deinit | update | status | summary | sync | absorbgitdirs)
 		command=$1
 		;;
 	-q|--quiet)
submodule.c +103 -0

112 lines not on this page · raw message

submodule.h +4 -0
index 4e3bf469b4..6229054b99 100644
@@ -74,4 +74,8 @@ int parallel_submodules(void);
  */
 void prepare_submodule_repo_env(struct argv_array *out);
 
+#define ABSORB_GITDIR_RECURSE_SUBMODULES (1<<0)
+extern void absorb_git_dir_into_superproject(const char *prefix,
+					     const char *path,
+					     unsigned flags);
 #endif
t/t7412-submodule-absorbgitdirs.sh new file +101 -0

101 lines not on this page · raw message

Signature · 1 line
2.11.0.rc2.29.g7c00390.dirty