Skip to contentSkip to conversation

git checkout --recurse-submodules

PATCH In review Jonathan Nieder 3 participants 11 messages Jun 15, 2016 Documentation

lore
Share & tools
lore

Jonathan Nieder

author
#0 lore raw

cover git checkout --recurse-submodules

Hi,

This patch series comes from
https://github.com/jlehmann/git-submod-enhancements branch recursive_submodule_checkout. It needed some tiny tweaks to apply to current "master" and build without warnings, but nothing major, and I haven't sanity checked it much beyond that and letting the kind folks that use Debian experimental play with it.

I'm sending it out now to get review and ideas for what needs to happen next to get this series in shape to be included in git.git.

Thoughts of all kinds welcome.

Thanks,
Jonathan

Jens Lehmann (5):
submodule: prepare for recursive checkout of submodules
submodule: teach unpack_trees() to remove submodule contents
submodule: teach unpack_trees() to repopulate submodules
submodule: teach unpack_trees() to update submodules Teach checkout to recursively checkout submodules

 Documentation/git-checkout.txt |   8 ++
 builtin/checkout.c             |  14 +++
 entry.c                        |  19 +++-
 submodule.c                    | 217 ++++++++++++++++++++++++++++++++++++++++-
 submodule.h                    |  11 +++
 t/t2013-checkout-submodule.sh  | 215 +++++++++++++++++++++++++++++++++++++++-
 unpack-trees.c                 |  94 ++++++++++++++----
 unpack-trees.h                 |   1 +
 wrapper.c                      |   3 +
 9 files changed, 556 insertions(+), 26 deletions(-)
author
#1 lore raw

PATCH 1/5 submodule: prepare for recursive checkout of submodules

From: Jens Lehmann <[email protected]>
Date: Mon, 18 Jun 2012 22:17:59 +0200

This commit adds the functions needed for configuration, for setting the default behavior and for determining if a submodule path should be updated automatically.

Signed-off-by Jens Lehmann <[email protected]>
Signed-off-by Jonathan Nieder <[email protected]>

---
Should probably be squashed into a patch that uses and documents this configuration.

 submodule.c | 36 ++++++++++++++++++++++++++++++++++++
 submodule.h |  3 +++
 2 files changed, 39 insertions(+)
submodule.c +36 -0
index 613857e..3f18d4d 100644
@@ -16,6 +16,8 @@ static struct string_list config_name_for_path;
 static struct string_list config_fetch_recurse_submodules_for_name;
 static struct string_list config_ignore_for_name;
 static int config_fetch_recurse_submodules = RECURSE_SUBMODULES_ON_DEMAND;
+static int config_update_recurse_submodules = RECURSE_SUBMODULES_OFF;
+static int option_update_recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
 static struct string_list changed_submodule_paths;
 static int initialized_fetch_ref_tips;
 static struct sha1_array ref_tips_before_fetch;

@@ -382,6 +384,34 @@ int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg)
 	}
 }
 
+int parse_update_recurse_submodules_arg(const char *opt, const char *arg)
+{
+	switch (git_config_maybe_bool(opt, arg)) {
+	case 1:
+		return RECURSE_SUBMODULES_ON;
+	case 0:
+		return RECURSE_SUBMODULES_OFF;
+	default:
+		if (!strcmp(arg, "checkout"))
+			return RECURSE_SUBMODULES_ON;
+		die("bad %s argument: %s", opt, arg);
+	}
+}
+
+int submodule_needs_update(const char *path)
+{
+	struct string_list_item *path_option;
+	path_option = unsorted_string_list_lookup(&config_name_for_path, path);
+	if (!path_option)
+		return 0;
+
+	/* update can't be "none", "merge" or "rebase" */
+
+	if (option_update_recurse_submodules != RECURSE_SUBMODULES_DEFAULT)
+		return 1;
+	return config_update_recurse_submodules != RECURSE_SUBMODULES_OFF;
+}
+
 void show_submodule_summary(FILE *f, const char *path,
 		const char *line_prefix,
 		unsigned char one[20], unsigned char two[20],

@@ -589,6 +619,12 @@ int push_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_nam
 	return ret;
 }
 
+void set_config_update_recurse_submodules(int default_value, int option_value)
+{
+	config_update_recurse_submodules = default_value;
+	option_update_recurse_submodules = option_value;
+}
+
 static int is_submodule_commit_present(const char *path, unsigned char sha1[20])
 {
 	int is_present = 0;
submodule.h +3 -0
index 7beec48..055918c 100644
@@ -22,12 +22,15 @@ void gitmodules_config(void);
 int parse_submodule_config_option(const char *var, const char *value);
 void handle_ignore_submodules_arg(struct diff_options *diffopt, const char *);
 int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);
+int parse_update_recurse_submodules_arg(const char *opt, const char *arg);
+int submodule_needs_update(const char *path);
 void show_submodule_summary(FILE *f, const char *path,
 		const char *line_prefix,
 		unsigned char one[20], unsigned char two[20],
 		unsigned dirty_submodule, const char *meta,
 		const char *del, const char *add, const char *reset);
 void set_config_fetch_recurse_submodules(int value);
+void set_config_update_recurse_submodules(int default_value, int option_value);
 void check_for_new_submodule_commits(unsigned char new_sha1[20]);
 int fetch_populated_submodules(const struct argv_array *options,
 			       const char *prefix, int command_line_option,
Signature · 1 line
1.8.5.1
#2 lore raw

Re: [PATCH 1/5] submodule: prepare for recursive checkout of submodules

Am 26.12.2013 17:02, schrieb Jonathan Nieder:

Quoted 7 lines

From: Jens Lehmann <[email protected]>
Date: Mon, 18 Jun 2012 22:17:59 +0200

This commit adds the functions needed for configuration, for setting the default behavior and for determining if a submodule path should be updated automatically.

Quoted 5 lines
Signed-off-by Jens Lehmann <[email protected]>
Signed-off-by Jonathan Nieder <[email protected]>

---
Should probably be squashed into a patch that uses and documents this configuration.

I'm not so sure. At the end of the day at least am, bisect, checkout, checkout-index, cherry-pick, merge, pull, read-tree, rebase, reset & stash should have learned this option (and all porcelain must honor the autoupdate setting by default, plumbing only when requested by giving the "--recurse-submodules=config" option to make life easier for scripting). And I think they should learn this one command per commit, making this one - and the yet-to-be-programmed one introducing the autoupdate flag - the base for them.

Quoted 86 lines
 submodule.c | 36 ++++++++++++++++++++++++++++++++++++
 submodule.h |  3 +++
 2 files changed, 39 insertions(+)
submodule.c +36 -0
index 613857e..3f18d4d 100644
@@ -16,6 +16,8 @@ static struct string_list config_name_for_path;
 static struct string_list config_fetch_recurse_submodules_for_name;
 static struct string_list config_ignore_for_name;
 static int config_fetch_recurse_submodules = RECURSE_SUBMODULES_ON_DEMAND;
+static int config_update_recurse_submodules = RECURSE_SUBMODULES_OFF;
+static int option_update_recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
 static struct string_list changed_submodule_paths;
 static int initialized_fetch_ref_tips;
 static struct sha1_array ref_tips_before_fetch;

@@ -382,6 +384,34 @@ int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg)
 	}
 }
 
+int parse_update_recurse_submodules_arg(const char *opt, const char *arg)
+{
+	switch (git_config_maybe_bool(opt, arg)) {
+	case 1:
+		return RECURSE_SUBMODULES_ON;
+	case 0:
+		return RECURSE_SUBMODULES_OFF;
+	default:
+		if (!strcmp(arg, "checkout"))
+			return RECURSE_SUBMODULES_ON;
+		die("bad %s argument: %s", opt, arg);
+	}
+}
+
+int submodule_needs_update(const char *path)
+{
+	struct string_list_item *path_option;
+	path_option = unsorted_string_list_lookup(&config_name_for_path, path);
+	if (!path_option)
+		return 0;
+
+	/* update can't be "none", "merge" or "rebase" */
+
+	if (option_update_recurse_submodules != RECURSE_SUBMODULES_DEFAULT)
+		return 1;
+	return config_update_recurse_submodules != RECURSE_SUBMODULES_OFF;
+}
+
 void show_submodule_summary(FILE *f, const char *path,
 		const char *line_prefix,
 		unsigned char one[20], unsigned char two[20],

@@ -589,6 +619,12 @@ int push_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_nam
 	return ret;
 }
 
+void set_config_update_recurse_submodules(int default_value, int option_value)
+{
+	config_update_recurse_submodules = default_value;
+	option_update_recurse_submodules = option_value;
+}
+
 static int is_submodule_commit_present(const char *path, unsigned char sha1[20])
 {
 	int is_present = 0;
submodule.h +3 -0
index 7beec48..055918c 100644
@@ -22,12 +22,15 @@ void gitmodules_config(void);
 int parse_submodule_config_option(const char *var, const char *value);
 void handle_ignore_submodules_arg(struct diff_options *diffopt, const char *);
 int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);
+int parse_update_recurse_submodules_arg(const char *opt, const char *arg);
+int submodule_needs_update(const char *path);
 void show_submodule_summary(FILE *f, const char *path,
 		const char *line_prefix,
 		unsigned char one[20], unsigned char two[20],
 		unsigned dirty_submodule, const char *meta,
 		const char *del, const char *add, const char *reset);
 void set_config_fetch_recurse_submodules(int value);
+void set_config_update_recurse_submodules(int default_value, int option_value);
 void check_for_new_submodule_commits(unsigned char new_sha1[20]);
 int fetch_populated_submodules(const struct argv_array *options,
 			       const char *prefix, int command_line_option,
author
#3 lore raw

PATCH 2/5 submodule: teach unpack_trees() to remove submodule contents

In reply to Jonathan Nieder

From: Jens Lehmann <[email protected]>
Date: Tue, 19 Jun 2012 20:55:45 +0200

Implement the functionality needed to enable work tree manipulating commands to that a deleted submodule should not only affect the index (leaving all the files of the submodule in the work tree) but also to remove the work tree of the superproject (including any untracked files).

That will only work properly when the submodule uses a gitfile instead of a .git directory and no untracked files are present. Otherwise the removal will fail with a warning (which is just what happened until now).

Extend rmdir_or_warn() to remove the directories of those submodules which are scheduled for removal. Also teach verify_clean_submodule() to check that a submodule configured to be removed is not modified before scheduling it for removal.

Signed-off-by Jens Lehmann <[email protected]>
Signed-off-by Jonathan Nieder <[email protected]>

---
Should this share some code (or just the error message) with the 'git rm' code that checks whether a submodule is safe to remove?

rmdir_or_warn is a pretty low-level function --- it feels odd to be relying on the git repository layout here. On the other hand, that function only has two callers, so it is possible to check quickly whether it is safe.

I assume this is mostly for the sake of the caller in unpack-trees?

In builtin/apply.c, remove_file is used for deletion and rename patches. Do we want this logic take effect there, too?

 submodule.c    | 37 +++++++++++++++++++++++++++++++++++++
 submodule.h    |  1 +
 unpack-trees.c |  6 +++---
 wrapper.c      |  3 +++
 4 files changed, 44 insertions(+), 3 deletions(-)
submodule.c +37 -0
index 3f18d4d..a25db46 100644
@@ -412,6 +412,43 @@ int submodule_needs_update(const char *path)
 	return config_update_recurse_submodules != RECURSE_SUBMODULES_OFF;
 }
 
+int depopulate_submodule(const char *path)
+{
+	struct strbuf dot_git = STRBUF_INIT;
+	struct child_process cp;
+	const char *argv[] = {"rm", "-rf", path, NULL};
+
+	/* Is it populated? */
+	strbuf_addf(&dot_git, "%s/.git", path);
+	if (!resolve_gitdir(dot_git.buf)) {
+		strbuf_release(&dot_git);
+		return 0;
+	}
+	strbuf_release(&dot_git);
+
+	/* Does it have a .git directory? */
+	if (!submodule_uses_gitfile(path)) {
+		warning(_("cannot remove submodule '%s' because it (or one of "
+			  "its nested submodules) uses a .git directory"),
+			  path);
+		return -1;
+	}
+
+	/* Remove the whole submodule directory */
+	memset(&cp, 0, sizeof(cp));
+	cp.argv = argv;
+	cp.env = local_repo_env;
+	cp.git_cmd = 0;
+	cp.no_stdin = 1;
+	if (run_command(&cp)) {
+		warning("Could not remove submodule %s", path);
+		strbuf_release(&dot_git);
+		return -1;
+	}
+
+	return 0;
+}
+
 void show_submodule_summary(FILE *f, const char *path,
 		const char *line_prefix,
 		unsigned char one[20], unsigned char two[20],
submodule.h +1 -0
index 055918c..df291cf 100644
@@ -24,6 +24,7 @@ void handle_ignore_submodules_arg(struct diff_options *diffopt, const char *);
 int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);
 int parse_update_recurse_submodules_arg(const char *opt, const char *arg);
 int submodule_needs_update(const char *path);
+int depopulate_submodule(const char *path);
 void show_submodule_summary(FILE *f, const char *path,
 		const char *line_prefix,
 		unsigned char one[20], unsigned char two[20],
unpack-trees.c +3 -3
index ad3e9a0..89b506a 100644
@@ -8,6 +8,7 @@
 #include "progress.h"
 #include "refs.h"
 #include "attr.h"
+#include "submodule.h"
 
 /*
  * Error messages expected by scripts out of plumbing commands such as

@@ -1263,14 +1264,13 @@ static void invalidate_ce_path(const struct cache_entry *ce,
 /*
  * Check that checking out ce->sha1 in subdir ce->name is not
  * going to overwrite any working files.
- *
- * Currently, git does not checkout subprojects during a superproject
- * checkout, so it is not going to overwrite anything.
  */
 static int verify_clean_submodule(const struct cache_entry *ce,
 				  enum unpack_trees_error_types error_type,
 				  struct unpack_trees_options *o)
 {
+	if (submodule_needs_update(ce->name) && is_submodule_modified(ce->name, 0))
+		return 1;
 	return 0;
 }
wrapper.c +3 -0
index 0cc5636..425a3fd 100644
@@ -2,6 +2,7 @@
  * Various trivial helper wrappers around standard functions
  */
 #include "cache.h"
+#include "submodule.h"
 
 static void do_nothing(size_t size)
 {

@@ -409,6 +410,8 @@ int unlink_or_warn(const char *file)
 
 int rmdir_or_warn(const char *file)
 {
+	if (submodule_needs_update(file) && depopulate_submodule(file))
+		return -1;
 	return warn_if_unremovable("rmdir", file, rmdir(file));
 }
Signature · 1 line
1.8.5.1
#4 lore raw

Re: [PATCH 2/5] submodule: teach unpack_trees() to remove submodule contents

Am 26.12.2013 17:12, schrieb Jonathan Nieder:

Quoted 18 lines

From: Jens Lehmann <[email protected]>
Date: Tue, 19 Jun 2012 20:55:45 +0200

Implement the functionality needed to enable work tree manipulating commands to that a deleted submodule should not only affect the index (leaving all the files of the submodule in the work tree) but also to remove the work tree of the superproject (including any untracked files).

That will only work properly when the submodule uses a gitfile instead of a .git directory and no untracked files are present. Otherwise the removal will fail with a warning (which is just what happened until now).

Extend rmdir_or_warn() to remove the directories of those submodules which are scheduled for removal. Also teach verify_clean_submodule() to check that a submodule configured to be removed is not modified before scheduling it for removal.

Quoted 5 lines
Signed-off-by Jens Lehmann <[email protected]>
Signed-off-by Jonathan Nieder <[email protected]>

---
Should this share some code (or just the error message) with the 'git rm' code that checks whether a submodule is safe to remove?

Yes, that would make sense.

Quoted 6 lines

rmdir_or_warn is a pretty low-level function --- it feels odd to be relying on the git repository layout here. On the other hand, that function only has two callers, so it is possible to check quickly whether it is safe.

I assume this is mostly for the sake of the caller in unpack-trees?

Yup.

Quoted 2 lines

In builtin/apply.c, remove_file is used for deletion and rename patches. Do we want this logic take effect there, too?

I think so. Recursive update should also affect apply and am when requested via command line or configuration. But the apply documentation states that it also handles files outside a git repository, so we would have to disable this logic in that case.

Quoted 117 lines
 submodule.c    | 37 +++++++++++++++++++++++++++++++++++++
 submodule.h    |  1 +
 unpack-trees.c |  6 +++---
 wrapper.c      |  3 +++
 4 files changed, 44 insertions(+), 3 deletions(-)
submodule.c +37 -0
index 3f18d4d..a25db46 100644
@@ -412,6 +412,43 @@ int submodule_needs_update(const char *path)
 	return config_update_recurse_submodules != RECURSE_SUBMODULES_OFF;
 }
 
+int depopulate_submodule(const char *path)
+{
+	struct strbuf dot_git = STRBUF_INIT;
+	struct child_process cp;
+	const char *argv[] = {"rm", "-rf", path, NULL};
+
+	/* Is it populated? */
+	strbuf_addf(&dot_git, "%s/.git", path);
+	if (!resolve_gitdir(dot_git.buf)) {
+		strbuf_release(&dot_git);
+		return 0;
+	}
+	strbuf_release(&dot_git);
+
+	/* Does it have a .git directory? */
+	if (!submodule_uses_gitfile(path)) {
+		warning(_("cannot remove submodule '%s' because it (or one of "
+			  "its nested submodules) uses a .git directory"),
+			  path);
+		return -1;
+	}
+
+	/* Remove the whole submodule directory */
+	memset(&cp, 0, sizeof(cp));
+	cp.argv = argv;
+	cp.env = local_repo_env;
+	cp.git_cmd = 0;
+	cp.no_stdin = 1;
+	if (run_command(&cp)) {
+		warning("Could not remove submodule %s", path);
+		strbuf_release(&dot_git);
+		return -1;
+	}
+
+	return 0;
+}
+
 void show_submodule_summary(FILE *f, const char *path,
 		const char *line_prefix,
 		unsigned char one[20], unsigned char two[20],
submodule.h +1 -0
index 055918c..df291cf 100644
@@ -24,6 +24,7 @@ void handle_ignore_submodules_arg(struct diff_options *diffopt, const char *);
 int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);
 int parse_update_recurse_submodules_arg(const char *opt, const char *arg);
 int submodule_needs_update(const char *path);
+int depopulate_submodule(const char *path);
 void show_submodule_summary(FILE *f, const char *path,
 		const char *line_prefix,
 		unsigned char one[20], unsigned char two[20],
unpack-trees.c +3 -3
index ad3e9a0..89b506a 100644
@@ -8,6 +8,7 @@
 #include "progress.h"
 #include "refs.h"
 #include "attr.h"
+#include "submodule.h"
 
 /*
  * Error messages expected by scripts out of plumbing commands such as

@@ -1263,14 +1264,13 @@ static void invalidate_ce_path(const struct cache_entry *ce,
 /*
  * Check that checking out ce->sha1 in subdir ce->name is not
  * going to overwrite any working files.
- *
- * Currently, git does not checkout subprojects during a superproject
- * checkout, so it is not going to overwrite anything.
  */
 static int verify_clean_submodule(const struct cache_entry *ce,
 				  enum unpack_trees_error_types error_type,
 				  struct unpack_trees_options *o)
 {
+	if (submodule_needs_update(ce->name) && is_submodule_modified(ce->name, 0))
+		return 1;
 	return 0;
 }
wrapper.c +3 -0
index 0cc5636..425a3fd 100644
@@ -2,6 +2,7 @@
  * Various trivial helper wrappers around standard functions
  */
 #include "cache.h"
+#include "submodule.h"
 
 static void do_nothing(size_t size)
 {

@@ -409,6 +410,8 @@ int unlink_or_warn(const char *file)
 
 int rmdir_or_warn(const char *file)
 {
+	if (submodule_needs_update(file) && depopulate_submodule(file))
+		return -1;
 	return warn_if_unremovable("rmdir", file, rmdir(file));
 }
author
#5 lore raw

PATCH 3/5 submodule: teach unpack_trees() to repopulate submodules

In reply to Jonathan Nieder

From: Jens Lehmann <[email protected]>
Date: Mon, 18 Jun 2012 22:11:45 +0200

Signed-off-by Jens Lehmann <[email protected]>
Signed-off-by Jonathan Nieder <[email protected]>

---
Neat. Would probably be clearer with some of the corresponding tests squashed in to illustrate the intended behavior.

 entry.c        |  4 ++++
 submodule.c    | 44 +++++++++++++++++++++++++++++++++++++++++---
 submodule.h    |  1 +
 unpack-trees.c | 19 +++++++++++++++++++
 4 files changed, 65 insertions(+), 3 deletions(-)
entry.c +4 -0
index 7b7aa81..d1bf6ec 100644
@@ -2,6 +2,7 @@
 #include "blob.h"
 #include "dir.h"
 #include "streaming.h"
+#include "submodule.h"
 
 static void create_directories(const char *path, int path_len,
 			       const struct checkout *state)

@@ -203,6 +204,9 @@ static int write_entry(struct cache_entry *ce,
 			return error("cannot create temporary submodule %s", path);
 		if (mkdir(path, 0777) < 0)
 			return error("cannot create submodule directory %s", path);
+		if (submodule_needs_update(path) &&
+		    populate_submodule(path, ce->sha1, state->force))
+			return error("cannot checkout submodule %s", path);
 		break;
 	default:
 		return error("unknown file mode for %s in index", path);
submodule.c +41 -3

72 lines not on this page · raw message

submodule.h +1 -0
index df291cf..3657ca8 100644
@@ -24,6 +24,7 @@ void handle_ignore_submodules_arg(struct diff_options *diffopt, const char *);
 int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);
 int parse_update_recurse_submodules_arg(const char *opt, const char *arg);
 int submodule_needs_update(const char *path);
+int populate_submodule(const char *path, unsigned char sha1[20], int force);
 int depopulate_submodule(const char *path);
 void show_submodule_summary(FILE *f, const char *path,
 		const char *line_prefix,
unpack-trees.c +19 -0
index 89b506a..ed48d41 100644
@@ -1367,6 +1367,7 @@ static int check_ok_to_remove(const char *name, int len, int dtype,
 			      struct unpack_trees_options *o)
 {
 	const struct cache_entry *result;
+	char *name_copy, *separator;
 
 	/*
 	 * It may be that the 'lstat()' succeeded even though

@@ -1409,6 +1410,24 @@ static int check_ok_to_remove(const char *name, int len, int dtype,
 			return 0;
 	}
 
+	/*
+	 * If the path lies inside a to be added submodule it
+	 * is ok to remove it.
+	 */
+	name_copy = xstrdup(name);
+	while ((separator = strrchr(name_copy, '/'))) {
+		int i;
+		*separator = '\0';
+		for (i = 0; i < the_index.cache_nr; i++) {
+			struct cache_entry *ce = the_index.cache[i];
+			if (!strcmp(ce->name, name_copy) && S_ISGITLINK(ce->ce_mode)) {
+				free(name_copy);
+				return 0;
+			}
+		}
+	}
+	free(name_copy);
+
 	return o->gently ? -1 :
 		add_rejected_path(o, error_type, name);
 }
Signature · 1 line
1.8.5.1
author
#6 lore raw

PATCH 4/5 submodule: teach unpack_trees() to update submodules

In reply to Jonathan Nieder

From: Jens Lehmann <[email protected]>
Date: Fri, 5 Apr 2013 18:35:27 +0200

Signed-off-by Jens Lehmann <[email protected]>
Signed-off-by Jonathan Nieder <[email protected]>

---
Also neat, also would benefit from documentation or tests.

 entry.c        | 15 ++++++++--
 submodule.c    | 86 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 submodule.h    |  3 ++
 unpack-trees.c | 69 ++++++++++++++++++++++++++++++++++++----------
 unpack-trees.h |  1 +
 5 files changed, 157 insertions(+), 17 deletions(-)
entry.c +12 -3
index d1bf6ec..61a2767 100644
@@ -265,7 +265,7 @@ int checkout_entry(struct cache_entry *ce,
 
 	if (!check_path(path, len, &st, state->base_dir_len)) {
 		unsigned changed = ce_match_stat(ce, &st, CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE);
-		if (!changed)
+		if (!changed && (!S_ISDIR(st.st_mode) || !S_ISGITLINK(ce->ce_mode)))
 			return 0;
 		if (!state->force) {
 			if (!state->quiet)

@@ -280,9 +280,18 @@ int checkout_entry(struct cache_entry *ce,
 		 * just do the right thing)
 		 */
 		if (S_ISDIR(st.st_mode)) {
-			/* If it is a gitlink, leave it alone! */
-			if (S_ISGITLINK(ce->ce_mode))
+			if (S_ISGITLINK(ce->ce_mode)) {
+				if (submodule_needs_update(ce->name)) {
+					if (is_submodule_populated(ce->name)) {
+						if (update_submodule(ce->name, ce->sha1, state->force))
+							return error("cannot checkout submodule %s", path);
+					} else {
+						if (populate_submodule(path, ce->sha1, state->force))
+							return error("cannot populate submodule %s", path);
+					}
+				}
 				return 0;
+			}
 			if (!state->force)
 				return error("%s is a directory", path);
 			remove_subtree(path);
submodule.c +86 -0

104 lines not on this page · raw message

submodule.h +3 -0
index 3657ca8..b42ae91 100644
@@ -26,6 +26,7 @@ int parse_update_recurse_submodules_arg(const char *opt, const char *arg);
 int submodule_needs_update(const char *path);
 int populate_submodule(const char *path, unsigned char sha1[20], int force);
 int depopulate_submodule(const char *path);
+int update_submodule(const char *path, const unsigned char sha1[20], int force);
 void show_submodule_summary(FILE *f, const char *path,
 		const char *line_prefix,
 		unsigned char one[20], unsigned char two[20],

@@ -37,9 +38,11 @@ void check_for_new_submodule_commits(unsigned char new_sha1[20]);
 int fetch_populated_submodules(const struct argv_array *options,
 			       const char *prefix, int command_line_option,
 			       int quiet);
+int is_submodule_populated(const char *path);
 unsigned is_submodule_modified(const char *path, int ignore_untracked);
 int submodule_uses_gitfile(const char *path);
 int ok_to_remove_submodule(const char *path);
+unsigned is_submodule_checkout_safe(const char *path, const unsigned char sha1[20]);
 int merge_submodule(unsigned char result[20], const char *path, const unsigned char base[20],
 		    const unsigned char a[20], const unsigned char b[20], int search);
 int find_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_name,
unpack-trees.c +55 -14

99 lines not on this page · raw message

unpack-trees.h +1 -0
index 36a73a6..bee8740 100644
@@ -15,6 +15,7 @@ enum unpack_trees_error_types {
 	ERROR_WOULD_OVERWRITE = 0,
 	ERROR_NOT_UPTODATE_FILE,
 	ERROR_NOT_UPTODATE_DIR,
+	ERROR_NOT_UPTODATE_SUBMODULE,
 	ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN,
 	ERROR_WOULD_LOSE_UNTRACKED_REMOVED,
 	ERROR_BIND_OVERLAP,
Signature · 1 line
1.8.5.1
author
#7 lore raw

PATCH 5/5 Teach checkout to recursively checkout submodules

In reply to Jonathan Nieder

From: Jens Lehmann <[email protected]>
Date: Wed, 13 Jun 2012 18:50:10 +0200

Signed-off-by Jens Lehmann <[email protected]>
Signed-off-by Jonathan Nieder <[email protected]>

---
This is the patch that actually introduces the --recurse-submodules option, which makes the rest work.

The tests indicate some future directions for improving this, but it works reasonably well already. I think I'd be most comfortable applying these if they were rearranged a little to the following order:

1. First, introducing the --recurse-submodules option, perhaps with no actual effect, with tests that it is parsed correctly, the default works, etc.

2. Then, adding the desired behaviors one by one with corresponding tests (handling submodule modifications, removals, additions).

3. Finally, any needed tweaks on top.

That way, it is easy to play around with early patches without worrying about the later ones at first, and the patches are easy to review to confirm that they at least don't break anything in the --no-recurse-submodules case.

That said, Debian experimental has these applied as is already, and there haven't been any complaints yet.

Thoughts?
Jonathan

 Documentation/git-checkout.txt |   8 ++
 builtin/checkout.c             |  14 +++
 submodule.c                    |  14 +++
 submodule.h                    |   3 +
 t/t2013-checkout-submodule.sh  | 215 ++++++++++++++++++++++++++++++++++++++++-
 5 files changed, 251 insertions(+), 3 deletions(-)
Documentation/git-checkout.txt +8 -0
index 91294f8..aabcc65 100644
@@ -225,6 +225,14 @@ This means that you can use `git checkout -p` to selectively discard
 edits from your current working tree. See the ``Interactive Mode''
 section of linkgit:git-add[1] to learn how to operate the `--patch` mode.
 
+--[no-]recurse-submodules::
+	Using --recurse-submodules will update the content of all initialized
+	submodules according to the commit recorded in the superproject.If
+	local modifications in a submodule would be overwritten the checkout
+	will fail until `-f` is used. If nothing (or --no-recurse-submodules)
+	is used, the work trees of submodules will not be updated, only the
+	hash recorded in the superproject will be changed.
+
 <branch>::
 	Branch to checkout; if it refers to a branch (i.e., a name that,
 	when prepended with "refs/heads/", is a valid ref), then that
builtin/checkout.c +14 -0
index 5df3837..ac2f8d8 100644
@@ -21,6 +21,9 @@
 #include "submodule.h"
 #include "argv-array.h"
 
+static const char *recurse_submodules_default = "off";
+static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
+
 static const char * const checkout_usage[] = {
 	N_("git checkout [options] <branch>"),
 	N_("git checkout [options] [<branch>] -- <file>..."),

@@ -1111,6 +1114,12 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 			 N_("do not limit pathspecs to sparse entries only")),
 		OPT_HIDDEN_BOOL(0, "guess", &dwim_new_local_branch,
 				N_("second guess 'git checkout no-such-branch'")),
+		{ OPTION_CALLBACK, 0, "recurse-submodules", &recurse_submodules,
+			    "checkout", "control recursive updating of submodules",
+			    PARSE_OPT_OPTARG, option_parse_update_submodules },
+		{ OPTION_STRING, 0, "recurse-submodules-default",
+			   &recurse_submodules_default, NULL,
+			   "default mode for recursion", PARSE_OPT_HIDDEN },
 		OPT_END(),
 	};
 

@@ -1132,6 +1141,11 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 		git_xmerge_config("merge.conflictstyle", conflict_style, NULL);
 	}
 
+	set_config_update_recurse_submodules(
+		parse_fetch_recurse_submodules_arg("--recurse-submodules-default",
+						   recurse_submodules_default),
+		recurse_submodules);
+
 	if ((!!opts.new_branch + !!opts.new_branch_force + !!opts.new_orphan_branch) > 1)
 		die(_("-b, -B and --orphan are mutually exclusive"));
submodule.c +14 -0
index 3365987..bdce1b2 100644
@@ -398,6 +398,20 @@ int parse_update_recurse_submodules_arg(const char *opt, const char *arg)
 	}
 }
 
+int option_parse_update_submodules(const struct option *opt,
+				   const char *arg, int unset)
+{
+	if (unset) {
+		*(int *)opt->value = RECURSE_SUBMODULES_OFF;
+	} else {
+		if (arg)
+			*(int *)opt->value = parse_update_recurse_submodules_arg(opt->long_name, arg);
+		else
+			*(int *)opt->value = RECURSE_SUBMODULES_ON;
+	}
+	return 0;
+}
+
 int submodule_needs_update(const char *path)
 {
 	struct string_list_item *path_option;
submodule.h +3 -0
index b42ae91..9841da3 100644
@@ -3,6 +3,7 @@
 
 struct diff_options;
 struct argv_array;
+struct option;
 
 enum {
 	RECURSE_SUBMODULES_ON_DEMAND = -1,

@@ -23,6 +24,8 @@ int parse_submodule_config_option(const char *var, const char *value);
 void handle_ignore_submodules_arg(struct diff_options *diffopt, const char *);
 int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);
 int parse_update_recurse_submodules_arg(const char *opt, const char *arg);
+int option_parse_update_submodules(const struct option *opt,
+		const char *arg, int unset);
 int submodule_needs_update(const char *path);
 int populate_submodule(const char *path, unsigned char sha1[20], int force);
 int depopulate_submodule(const char *path);
t/t2013-checkout-submodule.sh +212 -3

240 lines not on this page · raw message

Signature · 1 line
1.8.5.1
#8 lore raw

Re: [PATCH 5/5] Teach checkout to recursively checkout submodules

Am 26.12.2013 17:22, schrieb Jonathan Nieder:

Quoted 21 lines

From: Jens Lehmann <[email protected]>
Date: Wed, 13 Jun 2012 18:50:10 +0200

Signed-off-by Jens Lehmann <[email protected]>
Signed-off-by Jonathan Nieder <[email protected]>

---
This is the patch that actually introduces the --recurse-submodules option, which makes the rest work.

The tests indicate some future directions for improving this, but it works reasonably well already. I think I'd be most comfortable applying these if they were rearranged a little to the following order:

1. First, introducing the --recurse-submodules option, perhaps with no actual effect, with tests that it is parsed correctly, the default works, etc.

2. Then, adding the desired behaviors one by one with corresponding tests (handling submodule modifications, removals, additions).

Quoted 6 lines

3. Finally, any needed tweaks on top.

That way, it is easy to play around with early patches without worrying about the later ones at first, and the patches are easy to review to confirm that they at least don't break anything in the --no-recurse-submodules case.

Makes tons of sense.

The only feature I'm aware of that is currently missing is to read the path <-> name mappings from the correct .gitmodules blob, which is needed to populate submodules that appear.

Quoted 2 lines

That said, Debian experimental has these applied as is already, and there haven't been any complaints yet.

Great!

I'm also using this code at $dayjob successfully for quite some time now. Additionally I also enable unconditional recursive checkout by putting a "return 1" in the first line of the submodule_needs_update() function (Which is a hack to emulate "autoupdate=true" while at the same time pretending to already having added "--recurse-submodules=config" to all call sites in git porcelain scripts that call plumbing themselves). Only in a few corner cases submodules aren't properly updated, but we'll weed those out while implementing the tests. And we need lots of those to cover all important combinations ...

#9 lore raw
In reply to Jonathan Nieder
Quoted 24 lines · Jonathan Nieder

Hi,

This patch series comes from
https://github.com/jlehmann/git-submod-enhancements branch recursive_submodule_checkout. It needed some tiny tweaks to apply to current "master" and build without warnings, but nothing major, and I haven't sanity checked it much beyond that and letting the kind folks that use Debian experimental play with it.

I'm sending it out now to get review and ideas for what needs to happen next to get this series in shape to be included in git.git.

Thoughts of all kinds welcome.

Thanks,
Jonathan

Jens Lehmann (5):
submodule: prepare for recursive checkout of submodules
submodule: teach unpack_trees() to remove submodule contents
submodule: teach unpack_trees() to repopulate submodules
submodule: teach unpack_trees() to update submodules Teach checkout to recursively checkout submodules

Quoted 10 lines
 Documentation/git-checkout.txt |   8 ++
 builtin/checkout.c             |  14 +++
 entry.c                        |  19 +++-
 submodule.c                    | 217 ++++++++++++++++++++++++++++++++++++++++-
 submodule.h                    |  11 +++
 t/t2013-checkout-submodule.sh  | 215 +++++++++++++++++++++++++++++++++++++++-
 unpack-trees.c                 |  94 ++++++++++++++----
 unpack-trees.h                 |   1 +
 wrapper.c                      |   3 +
 9 files changed, 556 insertions(+), 26 deletions(-)

Looks reasonably clean from a cursory read. Thanks.

#10 lore raw
In reply to Jonathan Nieder

Am 26.12.2013 16:58, schrieb Jonathan Nieder:

Quoted 6 lines

This patch series comes from
https://github.com/jlehmann/git-submod-enhancements branch recursive_submodule_checkout. It needed some tiny tweaks to apply to current "master" and build without warnings, but nothing major, and I haven't sanity checked it much beyond that and letting the kind folks that use Debian experimental play with it.

Cool! Thanks for rebasing this series and great to hear that more people are using it.

Quoted 2 lines

I'm sending it out now to get review and ideas for what needs to happen next to get this series in shape to be included in git.git.

Excellent timing, Heiko and I wanted to work on this topic in the coming days anyway.