Checkout aware of Submodules!

RFC PATCH Superseded Stefan Beller 3 participants 30 messages Dec 3, 2016 – Dec 6, 2016 Documentation

Share & tools

Stefan Beller

author

2016-12-03 00:30 UTC #0 lore raw

v2 cover Checkout aware of Submodules!

v2:
* based on top of the series sent out an hour ago
"[PATCHv4 0/5] submodule embedgitdirs"
* Try to embed a submodule if we need to remove it.
* Strictly do not change behavior if not giving the new flag.
* I think I missed some review comments from v1, but I'd like to get the current state out over the weekend, as a lot has changed so far. On Monday I'll go through the previous discussion with a comb to see if I missed something.

v1:
When working with submodules, nearly anytime after checking out a different state of the projects, that has submodules changed you'd run "git submodule update" with a current version of Git.

There are two problems with this approach:

* The "submodule update" command is dangerous as it doesn't check for work that may be lost in the submodule (e.g. a dangling commit).
* you may forget to run the command as checkout is supposed to do all the work for you.

Integrate updating the submodules into git checkout, with the same safety promises that git-checkout has, i.e. not throw away data unless asked to. This is done by first checking if the submodule is at the same sha1 as it is recorded in the superproject. If there are changes we stop proceeding the checkout just like it is when checking out a file that has local changes.

The integration happens in the code that is also used in other commands such that it will be easier in the future to make other commands aware of submodule.

This also solves d/f conflicts in case you replace a file/directory with a submodule or vice versa.

The patches are still a bit rough, but the overall series seems promising enough to me that I want to put it out here.

Any review, specifically on the design level welcome!

Thanks,
Stefan

Stefan Beller (17):
submodule.h: add extern keyword to functions
submodule: modernize ok_to_remove_submodule to use argv_array update submodules: move up prepare_submodule_repo_env update submodules: add is_submodule_populated
update submodules: add submodule config parsing
update submodules: add a config option to determine if submodules are
updated
update submodules: introduce submodule_is_interesting update submodules: add depopulate_submodule
update submodules: add scheduling to update submodules update submodules: is_submodule_checkout_safe
unpack-trees: teach verify_clean_submodule to inspect submodules
unpack-trees: remove submodule contents if interesting
entry: write_entry to write populate submodules
submodule: teach unpack_trees() to update submodules
checkout: recurse into submodules if asked to
completion: add '--recurse-submodules' to checkout
checkout: add config option to recurse into submodules by default

 Documentation/config.txt               |   6 +
 Documentation/git-checkout.txt         |   8 +
 builtin/checkout.c                     |  31 +++-
 cache.h                                |   2 +
 contrib/completion/git-completion.bash |   2 +-
 entry.c                                |  14 +-
 submodule-config.c                     |  22 +++
 submodule-config.h                     |  17 +-
 submodule.c                            | 292 +++++++++++++++++++++++++++++---
 submodule.h                            |  74 ++++++---
 t/t2013-checkout-submodule.sh          | 293 ++++++++++++++++++++++++++++++++-
 t/t9902-completion.sh                  |   1 +
 unpack-trees.c                         |  92 ++++++++---
 unpack-trees.h                         |   1 +
 14 files changed, 768 insertions(+), 87 deletions(-)

Signature · 1 line

2.11.0.rc2.28.g2673dad

Stefan Beller

author

2016-12-03 00:31 UTC #1 lore raw

RFC PATCH v2 1/17 submodule.h: add extern keyword to functions

As the upcoming series will add a lot of functions to the submodule header, let's first make the header consistent to the rest of the project by adding the extern keyword to functions.

As per the CodingGuidelines we try to stay below 80 characters per line, so adapt all those functions to stay below 80 characters that are already using more than one line. Those function using just one line are better kept in one line than breaking them up into multiple lines just for the goal of staying below the character limit as it makes grepping for functions easier if they are one liners.

Signed-off-by Stefan Beller <[email protected]>

---
 submodule.h | 57 ++++++++++++++++++++++++++++++---------------------------
 1 file changed, 30 insertions(+), 27 deletions(-)

submodule.h +30 ~~-27~~

index d9e197a948..5db0b53b3f 100644
@@ -29,50 +29,53 @@ struct submodule_update_strategy {
 };
 #define SUBMODULE_UPDATE_STRATEGY_INIT {SM_UPDATE_UNSPECIFIED, NULL}
 
-int is_staging_gitmodules_ok(void);
-int update_path_in_gitmodules(const char *oldpath, const char *newpath);
-int remove_path_from_gitmodules(const char *path);
-void stage_updated_gitmodules(void);
-void set_diffopt_flags_from_submodule_config(struct diff_options *diffopt,
+extern int is_staging_gitmodules_ok(void);
+extern int update_path_in_gitmodules(const char *oldpath, const char *newpath);
+extern int remove_path_from_gitmodules(const char *path);
+extern void stage_updated_gitmodules(void);
+extern void set_diffopt_flags_from_submodule_config(struct diff_options *,
 		const char *path);
-int submodule_config(const char *var, const char *value, void *cb);
-void gitmodules_config(void);
-int parse_submodule_update_strategy(const char *value,
+extern int submodule_config(const char *var, const char *value, void *cb);
+extern void gitmodules_config(void);
+extern int parse_submodule_update_strategy(const char *value,
 		struct submodule_update_strategy *dst);
-const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);
-void handle_ignore_submodules_arg(struct diff_options *diffopt, const char *);
-void show_submodule_summary(FILE *f, const char *path,
+extern const char *submodule_strategy_to_string(const struct submodule_update_strategy *s);
+extern void handle_ignore_submodules_arg(struct diff_options *, const char *);
+extern void show_submodule_summary(FILE *f, const char *path,
 		const char *line_prefix,
 		struct object_id *one, struct object_id *two,
 		unsigned dirty_submodule, const char *meta,
 		const char *del, const char *add, const char *reset);
-void show_submodule_inline_diff(FILE *f, const char *path,
+extern void show_submodule_inline_diff(FILE *f, const char *path,
 		const char *line_prefix,
 		struct object_id *one, struct object_id *two,
 		unsigned dirty_submodule, const char *meta,
 		const char *del, const char *add, const char *reset,
 		const struct diff_options *opt);
-void set_config_fetch_recurse_submodules(int value);
-void check_for_new_submodule_commits(unsigned char new_sha1[20]);
-int fetch_populated_submodules(const struct argv_array *options,
+extern void set_config_fetch_recurse_submodules(int value);
+extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);
+extern int fetch_populated_submodules(const struct argv_array *options,
 			       const char *prefix, int command_line_option,
 			       int quiet, int max_parallel_jobs);
-unsigned is_submodule_modified(const char *path, int ignore_untracked);
-int submodule_uses_gitfile(const char *path);
-int ok_to_remove_submodule(const char *path);
-int merge_submodule(unsigned char result[20], const char *path, const unsigned char base[20],
-		    const unsigned char a[20], const unsigned char b[20], int search);
-int find_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_name,
-		struct string_list *needs_pushing);
-int push_unpushed_submodules(unsigned char new_sha1[20], const char *remotes_name);
-void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);
-int parallel_submodules(void);
+extern unsigned is_submodule_modified(const char *path, int ignore_untracked);
+extern int submodule_uses_gitfile(const char *path);
+extern int ok_to_remove_submodule(const char *path);
+extern int merge_submodule(unsigned char result[20], const char *path,
+			   const unsigned char base[20],
+			   const unsigned char a[20],
+			   const unsigned char b[20], int search);
+extern int find_unpushed_submodules(unsigned char new_sha1[20],
+				    const char *remotes_name,
+				    struct string_list *needs_pushing);
+extern int push_unpushed_submodules(unsigned char new_sha1[20],
+				    const char *remotes_name);
+extern void connect_work_tree_and_git_dir(const char *work_tree, const char *git_dir);
+extern int parallel_submodules(void);
 
 /*
  * Prepare the "env_array" parameter of a "struct child_process" for executing
  * a submodule by clearing any repo-specific envirionment variables, but
  * retaining any config in the environment.
  */
-void prepare_submodule_repo_env(struct argv_array *out);
-
+extern void prepare_submodule_repo_env(struct argv_array *out);
 #endif

Signature · 1 line

2.11.0.rc2.28.g2673dad

Stefan Beller

author

2016-12-03 00:30 UTC #2 lore raw

RFC PATCH v2 2/17 submodule: modernize ok_to_remove_submodule to use argv_array

In reply to Stefan Beller

Instead of constructing the NULL terminated array ourselves, we should make use of the argv_array infrastructure.

Signed-off-by Stefan Beller <[email protected]>

---
 submodule.c | 14 ++++----------
 1 file changed, 4 insertions(+), 10 deletions(-)

submodule.c +4 ~~-10~~

index 66c5ce5a24..eb80b0c5ad 100644
@@ -1022,13 +1022,6 @@ int ok_to_remove_submodule(const char *path)
 {
 	ssize_t len;
 	struct child_process cp = CHILD_PROCESS_INIT;
-	const char *argv[] = {
-		"status",
-		"--porcelain",
-		"-u",
-		"--ignore-submodules=none",
-		NULL,
-	};
 	struct strbuf buf = STRBUF_INIT;
 	int ok_to_remove = 1;

@@ -1038,14 +1031,15 @@ int ok_to_remove_submodule(const char *path)
 	if (!submodule_uses_gitfile(path))
 		return 0;
 
-	cp.argv = argv;
+	argv_array_pushl(&cp.args, "status", "--porcelain", "-u",
+				   "--ignore-submodules=none", NULL);
 	prepare_submodule_repo_env(&cp.env_array);
 	cp.git_cmd = 1;
 	cp.no_stdin = 1;
 	cp.out = -1;
 	cp.dir = path;
 	if (start_command(&cp))
-		die("Could not run 'git status --porcelain -uall --ignore-submodules=none' in submodule %s", path);
+		die("Could not run 'git status --porcelain -u --ignore-submodules=none' in submodule %s", path);
 
 	len = strbuf_read(&buf, cp.out, 1024);
 	if (len > 2)

@@ -1053,7 +1047,7 @@ int ok_to_remove_submodule(const char *path)
 	close(cp.out);
 
 	if (finish_command(&cp))
-		die("'git status --porcelain -uall --ignore-submodules=none' failed in submodule %s", path);
+		die("'git status --porcelain -u --ignore-submodules=none' failed in submodule %s", path);
 
 	strbuf_release(&buf);
 	return ok_to_remove;

Signature · 1 line

2.11.0.rc2.28.g2673dad

Stefan Beller

author

2016-12-03 00:31 UTC #3 lore raw

RFC PATCH v2 3/17 update submodules: move up prepare_submodule_repo_env

In reply to Stefan Beller

In a later patch we need to prepare the submodule environment with another git directory, so split up the function.

Also move it up in the file such that we do not need to declare the function later before using it.

Signed-off-by Stefan Beller <[email protected]>

---
 submodule.c | 27 ++++++++++++++++-----------
 1 file changed, 16 insertions(+), 11 deletions(-)

submodule.c +16 ~~-11~~

index eb80b0c5ad..78b69b5a55 100644
@@ -307,6 +307,22 @@ static void print_submodule_summary(struct rev_info *rev, FILE *f,
 	strbuf_release(&sb);
 }
 
+static void prepare_submodule_repo_env_no_git_dir(struct argv_array *out)
+{
+	const char * const *var;
+
+	for (var = local_repo_env; *var; var++) {
+		if (strcmp(*var, CONFIG_DATA_ENVIRONMENT))
+			argv_array_push(out, *var);
+	}
+}
+
+void prepare_submodule_repo_env(struct argv_array *out)
+{
+	prepare_submodule_repo_env_no_git_dir(out);
+	argv_array_push(out, "GIT_DIR=.git");
+}
+
 /* Helper function to display the submodule header line prior to the full
  * summary output. If it can locate the submodule objects directory it will
  * attempt to lookup both the left and right commits and put them into the

@@ -1246,14 +1262,3 @@ int parallel_submodules(void)
 {
 	return parallel_jobs;
 }
-
-void prepare_submodule_repo_env(struct argv_array *out)
-{
-	const char * const *var;
-
-	for (var = local_repo_env; *var; var++) {
-		if (strcmp(*var, CONFIG_DATA_ENVIRONMENT))
-			argv_array_push(out, *var);
-	}
-	argv_array_push(out, "GIT_DIR=.git");
-}

Signature · 1 line

2.11.0.rc2.28.g2673dad

Stefan Beller

author

2016-12-03 00:31 UTC #4 lore raw

RFC PATCH v2 4/17 update submodules: add is_submodule_populated

In reply to Stefan Beller

See if a submodule is populated by checking if there is a .git file or directory at the given path.

Signed-off-by Stefan Beller <[email protected]>

---
 submodule.c | 11 +++++++++++
 submodule.h |  1 +
 2 files changed, 12 insertions(+)

submodule.c +11 -0

index 78b69b5a55..c29153e9ff 100644
@@ -930,6 +930,17 @@ int fetch_populated_submodules(const struct argv_array *options,
 	return spf.result;
 }
 
+int is_submodule_populated(const char *path)
+{
+	int retval = 0;
+	struct strbuf gitdir = STRBUF_INIT;
+	strbuf_addf(&gitdir, "%s/.git", path);
+	if (resolve_gitdir(gitdir.buf))
+		retval = 1;
+	strbuf_release(&gitdir);
+	return retval;
+}
+
 unsigned is_submodule_modified(const char *path, int ignore_untracked)
 {
 	ssize_t len;

submodule.h +1 -0

index 5db0b53b3f..a9eabcc3d0 100644
@@ -58,6 +58,7 @@ extern int fetch_populated_submodules(const struct argv_array *options,
 			       const char *prefix, int command_line_option,
 			       int quiet, int max_parallel_jobs);
 extern unsigned is_submodule_modified(const char *path, int ignore_untracked);
+extern int is_submodule_populated(const char *path);
 extern int submodule_uses_gitfile(const char *path);
 extern int ok_to_remove_submodule(const char *path);
 extern int merge_submodule(unsigned char result[20], const char *path,

Signature · 1 line

2.11.0.rc2.28.g2673dad

Stefan Beller

author

2016-12-03 00:31 UTC #5 lore raw

RFC PATCH v2 5/17 update submodules: add submodule config parsing

In reply to Stefan Beller

Similar to b33a15b08 (push: add recurseSubmodules config option, 2015-11-17) and 027771fcb1 (submodule: allow erroneous values for the fetchRecurseSubmodules option, 2015-08-17), we add submodule-config code that is later used to parse whether we are interested in updating submodules.

We need the die_on_error parameter to be able to call this parsing function for the config file as well, which if incorrect lets Git die.

As we're just touching the header file, also mark all functions extern.

Signed-off-by Stefan Beller <[email protected]>

---
 submodule-config.c | 22 ++++++++++++++++++++++
 submodule-config.h | 17 +++++++++--------
 2 files changed, 31 insertions(+), 8 deletions(-)

submodule-config.c +22 -0

index 098085be69..4b5297e31d 100644
@@ -234,6 +234,28 @@ int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg)
 	return parse_fetch_recurse(opt, arg, 1);
 }
 
+static int parse_update_recurse(const char *opt, const char *arg,
+				int die_on_error)
+{
+	switch (git_config_maybe_bool(opt, arg)) {
+	case 1:
+		return RECURSE_SUBMODULES_ON;
+	case 0:
+		return RECURSE_SUBMODULES_OFF;
+	default:
+		if (!strcmp(arg, "checkout"))
+			return RECURSE_SUBMODULES_ON;
+		if (die_on_error)
+			die("bad %s argument: %s", opt, arg);
+		return RECURSE_SUBMODULES_ERROR;
+	}
+}
+
+int parse_update_recurse_submodules_arg(const char *opt, const char *arg)
+{
+	return parse_update_recurse(opt, arg, 1);
+}
+
 static int parse_push_recurse(const char *opt, const char *arg,
 			       int die_on_error)
 {

submodule-config.h +9 -8

index d05c542d2c..992bfbebc0 100644
@@ -22,13 +22,14 @@ struct submodule {
 	int recommend_shallow;
 };
 
-int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);
-int parse_push_recurse_submodules_arg(const char *opt, const char *arg);
-int parse_submodule_config_option(const char *var, const char *value);
-const struct submodule *submodule_from_name(const unsigned char *commit_sha1,
-		const char *name);
-const struct submodule *submodule_from_path(const unsigned char *commit_sha1,
-		const char *path);
-void submodule_free(void);
+extern int parse_fetch_recurse_submodules_arg(const char *opt, const char *arg);
+extern int parse_update_recurse_submodules_arg(const char *opt, const char *arg);
+extern int parse_push_recurse_submodules_arg(const char *opt, const char *arg);
+extern int parse_submodule_config_option(const char *var, const char *value);
+extern const struct submodule *submodule_from_name(
+		const unsigned char *commit_sha1, const char *name);
+extern const struct submodule *submodule_from_path(
+		const unsigned char *commit_sha1, const char *path);
+extern void submodule_free(void);
 
 #endif /* SUBMODULE_CONFIG_H */

Signature · 1 line

2.11.0.rc2.28.g2673dad

Stefan Beller

author

2016-12-03 00:31 UTC #6 lore raw

RFC PATCH v2 6/17 update submodules: add a config option to determine if submodules are updated

In reply to Stefan Beller

In later patches we introduce the options and flag for commands that modify the working directory, e.g. git-checkout.

Have a central place to store such settings whether we want to update a submodule.

Signed-off-by Stefan Beller <[email protected]>

---
 submodule.c | 6 ++++++
 submodule.h | 1 +
 2 files changed, 7 insertions(+)

submodule.c +6 -0

index c29153e9ff..1ba398ba3b 100644
@@ -16,6 +16,7 @@
 #include "quote.h"
 
 static int config_fetch_recurse_submodules = RECURSE_SUBMODULES_ON_DEMAND;
+static int config_update_recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
 static int parallel_jobs = 1;
 static struct string_list changed_submodule_paths = STRING_LIST_INIT_NODUP;
 static int initialized_fetch_ref_tips;

@@ -510,6 +511,11 @@ void set_config_fetch_recurse_submodules(int value)
 	config_fetch_recurse_submodules = value;
 }
 
+void set_config_update_recurse_submodules(int value)
+{
+	config_update_recurse_submodules = value;
+}
+
 static int has_remote(const char *refname, const struct object_id *oid,
 		      int flags, void *cb_data)
 {

submodule.h +1 -0

index a9eabcc3d0..21236b095c 100644
@@ -53,6 +53,7 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,
 		const char *del, const char *add, const char *reset,
 		const struct diff_options *opt);
 extern void set_config_fetch_recurse_submodules(int value);
+extern void set_config_update_recurse_submodules(int value);
 extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);
 extern int fetch_populated_submodules(const struct argv_array *options,
 			       const char *prefix, int command_line_option,

Signature · 1 line

2.11.0.rc2.28.g2673dad

Stefan Beller

author

2016-12-03 00:31 UTC #7 lore raw

RFC PATCH v2 7/17 update submodules: introduce submodule_is_interesting

In reply to Stefan Beller

In later patches we introduce the --recurse-submodule flag for commands that modify the working directory, e.g. git-checkout.

It is potentially expensive to check if a submodule needs an update, because a common theme to interact with submodules is to spawn a child process for each interaction.

So let's introduce a function that pre checks if a submodule needs to be checked for an update.

I am not particular happy with the name submodule_is_interesting, in internal iterations I had submodule_requires_check_for_update and submodule_needs_update, but I was even less happy with those names. Maybe submodule_interesting_for_update?

Generally this is to answer "Am I allowed to touch the submodule at all?" or: "Does the user expect me to touch it?" which includes all of creation/deletion/update.

This patch is based off a prior attempt by Jens Lehmann to add submodules to checkout.

Signed-off-by Stefan Beller <[email protected]>

---
 submodule.c | 23 +++++++++++++++++++++++
 submodule.h |  9 +++++++++
 2 files changed, 32 insertions(+)

submodule.c +23 -0

index 1ba398ba3b..62e9ef3872 100644
@@ -516,6 +516,29 @@ void set_config_update_recurse_submodules(int value)
 	config_update_recurse_submodules = value;
 }
 
+int submodules_interesting_for_update(void)
+{
+	/*
+	 * Update can't be "none", "merge" or "rebase",
+	 * treat any value as OFF, except an explicit ON.
+	 */
+	return config_update_recurse_submodules == RECURSE_SUBMODULES_ON;
+}
+
+int submodule_is_interesting(const char *path)
+{
+	const struct submodule *sub;
+
+	if (!submodules_interesting_for_update())
+		return 0;
+
+	sub = submodule_from_path(null_sha1, path);
+	if (!sub)
+		return 0;
+
+	return sub->update_strategy.type != SM_UPDATE_NONE;
+}
+
 static int has_remote(const char *refname, const struct object_id *oid,
 		      int flags, void *cb_data)
 {

submodule.h +9 -0

index 21236b095c..7d890e0464 100644
@@ -54,6 +54,15 @@ extern void show_submodule_inline_diff(FILE *f, const char *path,
 		const struct diff_options *opt);
 extern void set_config_fetch_recurse_submodules(int value);
 extern void set_config_update_recurse_submodules(int value);
+
+/**
+ * When updating the working tree, we need to check if the submodule needs
+ * updating. We do not require a check if we are already sure that the
+ * submodule doesn't need updating, e.g. when we are not interested in submodules
+ * or the submodule is marked uninteresting by being not initialized.
+ */
+extern int submodule_is_interesting(const char *path);
+extern int submodules_interesting_for_update(void);
 extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);
 extern int fetch_populated_submodules(const struct argv_array *options,
 			       const char *prefix, int command_line_option,

Signature · 1 line

2.11.0.rc2.28.g2673dad

Stefan Beller

author

2016-12-03 00:31 UTC #8 lore raw

RFC PATCH v2 8/17 update submodules: add depopulate_submodule

In reply to Stefan Beller

Implement the functionality needed to enable work tree manipulating commands so that a deleted submodule should not only affect the index (leaving all the files of the submodule in the work tree) but also to remove the work tree of the superproject (including any untracked files).

To do so, we need an equivalent of "rm -rf", which is already found in entry.c, so expose that and for clarity add a suffix "_or_dir" to it.

That will only work properly when the submodule uses a gitfile instead of a .git directory and no untracked files are present. Otherwise the removal will fail with a warning (which is just what happened until now).

Signed-off-by Stefan Beller <[email protected]>

---
 cache.h     |  2 ++
 entry.c     |  5 +++++
 submodule.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 submodule.h |  1 +
 4 files changed, 54 insertions(+)

cache.h +2 -0

index a50a61a197..b645ca2f9a 100644
@@ -2018,4 +2018,6 @@ void sleep_millisec(int millisec);
  */
 void safe_create_dir(const char *dir, int share);
 
+extern void remove_directory_or_die(struct strbuf *path);
+
 #endif /* CACHE_H */

entry.c +5 -0

index c6eea240b6..02c4ac9f22 100644
@@ -73,6 +73,11 @@ static void remove_subtree(struct strbuf *path)
 		die_errno("cannot rmdir '%s'", path->buf);
 }
 
+void remove_directory_or_die(struct strbuf *path)
+{
+	remove_subtree(path);
+}
+
 static int create_file(const char *path, unsigned int mode)
 {
 	mode = (mode & 0100) ? 0777 : 0666;

submodule.c +46 -0

52 lines not on this page · raw message

submodule.h +1 -0

index 7d890e0464..d8bb1d4baf 100644
@@ -63,6 +63,7 @@ extern void set_config_update_recurse_submodules(int value);
  */
 extern int submodule_is_interesting(const char *path);
 extern int submodules_interesting_for_update(void);
+extern int depopulate_submodule(const char *path);
 extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);
 extern int fetch_populated_submodules(const struct argv_array *options,
 			       const char *prefix, int command_line_option,

Signature · 1 line

2.11.0.rc2.28.g2673dad

David Turner

2016-12-05 23:37 UTC #9 lore raw

RE: [RFC PATCHv2 08/17] update submodules: add depopulate_submodule

Quoted 8 lines

-----Original Message-----
From: Stefan Beller [mailto:[email protected]]
Sent: Friday, December 02, 2016 7:30 PM
To: [email protected]; David Turner
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Stefan Beller
Subject: [RFC PATCHv2 08/17] update submodules: add depopulate_submodule

Quoted 4 lines

Implement the functionality needed to enable work tree manipulating commands so that a deleted submodule should not only affect the index (leaving all the files of the submodule in the work tree) but also to remove the work tree of the superproject (including any untracked files).

"including any untracked files" bothers me, I think. Checkout is not usually willing to overwrite untracked files; it seems odd to me that it would be willing to do so in the submodule case. I would be OK if they were both untracked and gitignored, I think.

Quoted 72 lines

To do so, we need an equivalent of "rm -rf", which is already found in entry.c, so expose that and for clarity add a suffix "_or_dir" to it.

Signed-off-by Stefan Beller <[email protected]>

---
 cache.h     |  2 ++
 entry.c     |  5 +++++
 submodule.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++
 submodule.h |  1 +
 4 files changed, 54 insertions(+)

cache.h +2 -0

index a50a61a197..b645ca2f9a 100644
@@ -2018,4 +2018,6 @@ void sleep_millisec(int millisec);
  */
 void safe_create_dir(const char *dir, int share);
 
+extern void remove_directory_or_die(struct strbuf *path);
+
 #endif /* CACHE_H */

entry.c +4 -0

index c6eea240b6..02c4ac9f22 100644
@@ -73,6 +73,11 @@ static void remove_subtree(struct strbuf *path)
 		die_errno("cannot rmdir '%s'", path->buf);  }
 
+void remove_directory_or_die(struct strbuf *path) {
+	remove_subtree(path);
+}
+
 static int create_file(const char *path, unsigned int mode)  {
 	mode = (mode & 0100) ? 0777 : 0666;

submodule.c +20 -0

index 62e9ef3872..7bb64d6c69 100644
@@ -324,6 +324,52 @@ void prepare_submodule_repo_env(struct argv_array
 *out)
 	argv_array_push(out, "GIT_DIR=.git");
 }
 
+int depopulate_submodule(const char *path) {
+	int ret = 0;
+	struct strbuf pathbuf = STRBUF_INIT;
+	char *dot_git = xstrfmt("%s/.git", path);
+
+	/* Is it populated? */
+	if (!resolve_gitdir(dot_git))
+		goto out;
+
+	/* Does it have a .git directory? */
+	if (!submodule_uses_gitfile(path)) {
+		struct child_process cp = CHILD_PROCESS_INIT;
+
+		prepare_submodule_repo_env(&cp.env_array);
+		argv_array_pushl(&cp.args, "submodule--helper",
+				 "embed-git-dirs", path, NULL);
+		cp.git_cmd = 1;
+		if (run_command(&cp)) {
+			warning(_("Cannot remove submodule '%s'\n"
+				  "because it (or one of its nested submodules)

has a git \n"

submodule.c +1 -0

+				  "directory in the working tree, which could not

be embedded\n"

submodule.c +1 -0

+				  "the superprojects git directory

Quoted 1 line

automatically."), path);

What if instead it couldn't run the command because you're out of file descriptors or pids or memory or something?

I think this message should be in submodule--helper --embed-git-dirs instead, and we should just pass it through here. Or, perhaps, instead of shelling out here, we should just call the functions directly?

submodule.c void prepare_submodule_repo_env(struct argv_array *out) · line 356

Quoted 7 lines

submodule.c +7 -0

+			ret = -1;
+			goto out;
+		}
+
+		if (!submodule_uses_gitfile(path)) {
+			/*
+			 * We should be using a gitfile by now, let's double

Comma splice.

submodule.c void prepare_submodule_repo_env(struct argv_array *out) · line 357

Quoted 1 line

submodule.c +1 -0

+			 * check as loosing the git dir would be fatal.

s/loosing/losing/

Quoted 35 lines

submodule.c +2 -0

+			 */
+			die("BUG: \"git submodule--helper embed git-dirs '%s'\"

submodule.c +13 -0

+			    "did not embed the git-dirs recursively for '%s'",
+			    path, path);
+		}
+	}
+
+	strbuf_addstr(&pathbuf, path);
+	remove_directory_or_die(&pathbuf);
+out:
+	strbuf_release(&pathbuf);
+	free(dot_git);
+	return ret;
+}
+
 /* Helper function to display the submodule header line prior to the full
  * summary output. If it can locate the submodule objects directory it

will
* attempt to lookup both the left and right commits and put them into
the diff --git a/submodule.h b/submodule.h index 7d890e0464..d8bb1d4baf 100644

submodule.h +1 -0

@@ -63,6 +63,7 @@ extern void set_config_update_recurse_submodules(int
 value);
  */
 extern int submodule_is_interesting(const char *path);  extern int
 submodules_interesting_for_update(void);
+extern int depopulate_submodule(const char *path);
 extern void check_for_new_submodule_commits(unsigned char new_sha1[20]);
 extern int fetch_populated_submodules(const struct argv_array *options,

const char *prefix, int command_line_option,
--
2.11.0.rc2.28.g2673dad

Stefan Beller

author

2016-12-06 00:18 UTC #10 lore raw

Re: [RFC PATCHv2 08/17] update submodules: add depopulate_submodule

Quoted 8 lines · David Turner

Quoted 8 lines
-----Original Message-----
From: Stefan Beller [mailto:[email protected]]
Sent: Friday, December 02, 2016 7:30 PM
To: [email protected]; David Turner
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Stefan Beller
Subject: [RFC PATCHv2 08/17] update submodules: add depopulate_submodule

Quoted 6 lines

Quoted 4 lines
Implement the functionality needed to enable work tree manipulating commands so that a deleted submodule should not only affect the index (leaving all the files of the submodule in the work tree) but also to remove the work tree of the superproject (including any untracked files).
"including any untracked files" bothers me, I think. Checkout is not usually willing to overwrite untracked files; it seems odd to me that it would be willing to do so in the submodule case. I would be OK if they were both untracked and gitignored, I think.

I agree on being bothered, this is one of the things I thought how to solve. See the test in "checkout: recurse into submodules if asked to", which tests for untracked files and is still marked as a failure.

I think to address that issue, I'll add a flag to ok_to_remove_submodule which let's you specify which files you care about and which you don't.

Quoted 11 lines

Quoted 7 lines
(quoted diff) +2 -0
+                     warning(_("Cannot remove submodule '%s'\n"
+                               "because it (or one of its nested submodules)
has a git \n"
+ "directory in the working tree, which could not
be embedded\n"
+ "the superprojects git directory
automatically."), path);
What if instead it couldn't run the command because you're out of file descriptors or pids or memory or something?
I think this message should be in submodule--helper --embed-git-dirs instead, and we should just pass it through here. Or, perhaps, instead of shelling out here, we should just call the functions directly?

heh, good point. Will call the function directly.

Quoted 1 line

Stefan Beller

author

2016-12-03 00:31 UTC #11 lore raw

RFC PATCH v2 9/17 update submodules: add scheduling to update submodules

In reply to Stefan Beller

The walker of a tree is only expected to call schedule_submodule_for_update and once done, to run update_submodules. This avoids directory/file conflicts and later we can parallelize all submodule actions if needed.

Signed-off-by Stefan Beller <[email protected]>

---
 submodule.c | 132 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 submodule.h |   5 +++
 2 files changed, 137 insertions(+)

submodule.c +132 -0

index 7bb64d6c69..02c28ef56b 100644
@@ -1348,3 +1348,135 @@ int parallel_submodules(void)
 {
 	return parallel_jobs;
 }
+
+static struct scheduled_submodules_update_type {
+	const char *path;
+	const struct object_id *oid;
+	/*
+	 * Do we need to perform a complete checkout or just incremental
+	 * update?
+	 */
+	unsigned is_new:1;
+} *scheduled_submodules;
+#define SCHEDULED_SUBMODULES_INIT {NULL, NULL, 0}
+
+static int scheduled_submodules_nr, scheduled_submodules_alloc;
+
+void schedule_submodule_for_update(const struct cache_entry *ce, int is_new)
+{
+	struct scheduled_submodules_update_type *ssu;
+	ALLOC_GROW(scheduled_submodules,
+		   scheduled_submodules_nr + 1,
+		   scheduled_submodules_alloc);
+	ssu = &scheduled_submodules[scheduled_submodules_nr++];
+	ssu->path = ce->name;
+	ssu->oid = &ce->oid;
+	ssu->is_new = !!is_new;
+}
+
+static int update_submodule(const char *path, const struct object_id *oid,
+			    int force, int is_new)
+{
+	const char *git_dir;
+	struct child_process cp = CHILD_PROCESS_INIT;
+	const struct submodule *sub = submodule_from_path(null_sha1, path);
+
+	if (!sub || !sub->name)
+		return -1;
+
+	git_dir = resolve_gitdir(git_common_path("modules/%s", sub->name));
+
+	if (!git_dir)
+		return -1;
+
+	if (is_new)
+		connect_work_tree_and_git_dir(path, git_dir);
+
+	/* update index via `read-tree --reset sha1` */
+	argv_array_pushl(&cp.args, "read-tree",
+				   force ? "--reset" : "-m",
+				   "-u", sha1_to_hex(oid->hash), NULL);
+	prepare_submodule_repo_env(&cp.env_array);
+	cp.git_cmd = 1;
+	cp.no_stdin = 1;
+	cp.dir = path;
+	if (run_command(&cp)) {
+		warning(_("reading the index in submodule '%s' failed"), path);
+		child_process_clear(&cp);
+		return -1;
+	}
+
+	/* write index to working dir */
+	child_process_clear(&cp);
+	child_process_init(&cp);
+	argv_array_pushl(&cp.args, "checkout-index", "-a", NULL);
+	cp.git_cmd = 1;
+	cp.no_stdin = 1;
+	cp.dir = path;
+	if (force)
+		argv_array_push(&cp.args, "-f");
+
+	if (run_command(&cp)) {
+		warning(_("populating the working directory in submodule '%s' failed"), path);
+		child_process_clear(&cp);
+		return -1;
+	}
+
+	/* get the HEAD right */
+	child_process_clear(&cp);
+	child_process_init(&cp);
+	argv_array_pushl(&cp.args, "checkout", "--recurse-submodules", NULL);
+	cp.git_cmd = 1;
+	cp.no_stdin = 1;
+	cp.dir = path;
+	if (force)
+		argv_array_push(&cp.args, "-f");
+	argv_array_push(&cp.args, sha1_to_hex(oid->hash));
+
+	if (run_command(&cp)) {
+		warning(_("setting the HEAD in submodule '%s' failed"), path);
+		child_process_clear(&cp);
+		return -1;
+	}
+
+	child_process_clear(&cp);
+	return 0;
+}
+
+int update_submodules(int force)
+{
+	int i;
+	gitmodules_config();
+
+	/*
+	 * NEEDSWORK: As submodule updates can potentially take some
+	 * time each and they do not overlap (i.e. no d/f conflicts;
+	 * this can be parallelized using the run_commands.h API.
+	 */
+	for (i = 0; i < scheduled_submodules_nr; i++) {
+		struct submodule *sub;
+		struct scheduled_submodules_update_type *ssu =
+			&scheduled_submodules[i];
+
+		if (!submodule_is_interesting(ssu->path))
+			continue;
+
+		sub = submodule_from_path(null_sha1, ssu->path);
+
+		switch (sub->update_strategy) {
+		case SM_UPDATE_UNSPECIFIED: /* fall thru */
+		case SM_UPDATE_CHECKOUT:
+			update_submodule(ssu->path, ssu->oid,
+					 force, ssu->is_new);
+			break;
+		case SM_UPDATE_REBASE:
+		case SM_UPDATE_MERGE:
+		case SM_UPDATE_NONE:
+		case SM_UPDATE_COMMAND:
+			warning("update strategy for submodule '%s' not supported", ssu->path);
+		}
+	}
+
+	scheduled_submodules_nr = 0;
+	return 0;
+}

submodule.h +5 -0

index d8bb1d4baf..74df8b93d5 100644
@@ -90,4 +90,9 @@ extern int parallel_submodules(void);
  * retaining any config in the environment.
  */
 extern void prepare_submodule_repo_env(struct argv_array *out);
+
+extern void schedule_submodule_for_update(const struct cache_entry *ce,
+					  int new);
+extern int update_submodules(int force);
+
 #endif

Signature · 1 line

2.11.0.rc2.28.g2673dad

David Turner

2016-12-05 23:37 UTC #12 lore raw

RE: [RFC PATCHv2 09/17] update submodules: add scheduling to update submodules

This patch confuses me -- see below.

Quoted 8 lines

-----Original Message-----
From: Stefan Beller [mailto:[email protected]]
Sent: Friday, December 02, 2016 7:30 PM
To: [email protected]; David Turner
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Stefan Beller
Subject: [RFC PATCHv2 09/17] update submodules: add scheduling to update submodules

[snip]

Quoted 29 lines

+static int update_submodule(const char *path, const struct object_id *oid,

submodule.c +27 -0

+			    int force, int is_new)
+{
+	const char *git_dir;
+	struct child_process cp = CHILD_PROCESS_INIT;
+	const struct submodule *sub = submodule_from_path(null_sha1, path);
+
+	if (!sub || !sub->name)
+		return -1;
+
+	git_dir = resolve_gitdir(git_common_path("modules/%s", sub->name));
+
+	if (!git_dir)
+		return -1;
+
+	if (is_new)
+		connect_work_tree_and_git_dir(path, git_dir);
+
+	/* update index via `read-tree --reset sha1` */
+	argv_array_pushl(&cp.args, "read-tree",
+				   force ? "--reset" : "-m",
+				   "-u", sha1_to_hex(oid->hash), NULL);
+	prepare_submodule_repo_env(&cp.env_array);
+	cp.git_cmd = 1;
+	cp.no_stdin = 1;
+	cp.dir = path;
+	if (run_command(&cp)) {
+		warning(_("reading the index in submodule '%s' failed"),

Quoted 1 line

path);

The error is not (usually) in "reading the index" -- it's "updating the index" (or the working tree)

submodule.c int parallel_submodules(void) · line 1412

Quoted 8 lines

submodule.c +8 -0

+		child_process_clear(&cp);
+		return -1;
+	}
+
+	/* write index to working dir */
+	child_process_clear(&cp);
+	child_process_init(&cp);
+	argv_array_pushl(&cp.args, "checkout-index", "-a", NULL);

I'm confused -- doesn't read-tree -u already do this? And if not, shouldn't we back out the result of the read-tree, to leave the submodule as it was?

Quoted 17 lines

submodule.c +8 -0

+	cp.git_cmd = 1;
+	cp.no_stdin = 1;
+	cp.dir = path;
+	if (force)
+		argv_array_push(&cp.args, "-f");
+
+	if (run_command(&cp)) {
+		warning(_("populating the working directory in submodule '%s'

failed"), path);

submodule.c +8 -0

+		child_process_clear(&cp);
+		return -1;
+	}
+
+	/* get the HEAD right */
+	child_process_clear(&cp);
+	child_process_init(&cp);
+	argv_array_pushl(&cp.args, "checkout", "--recurse-submodules",

Quoted 1 line

NULL);

Why are we running checkout on the submodule when we've already done most of the checkout? The only thing left is to set HEAD and recurse, right? I must be missing something.

Stefan Beller

author

2016-12-05 23:54 UTC #13 lore raw

Re: [RFC PATCHv2 09/17] update submodules: add scheduling to update submodules

Quoted 68 lines · David Turner

This patch confuses me -- see below.

Quoted 8 lines

-----Original Message-----
From: Stefan Beller [mailto:[email protected]]
Sent: Friday, December 02, 2016 7:30 PM
To: [email protected]; David Turner
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Stefan Beller
Subject: [RFC PATCHv2 09/17] update submodules: add scheduling to update submodules

[snip]

Quoted 29 lines

+static int update_submodule(const char *path, const struct object_id *oid,

submodule.c +27 -0

+                         int force, int is_new)
+{
+     const char *git_dir;
+     struct child_process cp = CHILD_PROCESS_INIT;
+     const struct submodule *sub = submodule_from_path(null_sha1, path);
+
+     if (!sub || !sub->name)
+             return -1;
+
+     git_dir = resolve_gitdir(git_common_path("modules/%s", sub->name));
+
+     if (!git_dir)
+             return -1;
+
+     if (is_new)
+             connect_work_tree_and_git_dir(path, git_dir);
+
+     /* update index via `read-tree --reset sha1` */
+     argv_array_pushl(&cp.args, "read-tree",
+                                force ? "--reset" : "-m",
+                                "-u", sha1_to_hex(oid->hash), NULL);
+     prepare_submodule_repo_env(&cp.env_array);
+     cp.git_cmd = 1;
+     cp.no_stdin = 1;
+     cp.dir = path;
+     if (run_command(&cp)) {
+             warning(_("reading the index in submodule '%s' failed"),

Quoted 1 line

path);

The error is not (usually) in "reading the index" -- it's "updating the index" (or the working tree)

submodule.c int parallel_submodules(void) · line 1412

Quoted 8 lines

submodule.c +8 -0

+             child_process_clear(&cp);
+             return -1;
+     }
+
+     /* write index to working dir */
+     child_process_clear(&cp);
+     child_process_init(&cp);
+     argv_array_pushl(&cp.args, "checkout-index", "-a", NULL);

I'm confused -- doesn't read-tree -u already do this? And if not, shouldn't we back out the result of the read-tree, to leave the submodule as it was?

Quoted 13 lines

submodule.c +8 -0

+     cp.git_cmd = 1;
+     cp.no_stdin = 1;
+     cp.dir = path;
+     if (force)
+             argv_array_push(&cp.args, "-f");
+
+     if (run_command(&cp)) {
+             warning(_("populating the working directory in submodule '%s'

failed"), path);

submodule.c +4 -0

+             child_process_clear(&cp);
+             return -1;
+     }
+

Quoted 9 lines

Quoted 5 lines
submodule.c +4 -0
+     /* get the HEAD right */
+     child_process_clear(&cp);
+     child_process_init(&cp);
+     argv_array_pushl(&cp.args, "checkout", "--recurse-submodules",
NULL);
Why are we running checkout on the submodule when we've already done most of the checkout? The only thing left is to set HEAD and recurse, right? I must be missing something.

Yes this is only used to set the HEAD correctly and then recurse down. I tried to remove the first 2 calls to ch8ild processes at one point in time, which did not work out. I should have written in the commit message why that was a problem. So I'll redo that just to see the problem and improve the commit message.

Stefan Beller

author

2016-12-03 00:31 UTC #14 lore raw

RFC PATCH v2 10/17 update submodules: is_submodule_checkout_safe

In reply to Stefan Beller

In later patches we introduce the options and flag for commands that modify the working directory, e.g. git-checkout.

This piece of code will answer the question:
"Is it safe to change the submodule to this new state?" e.g. is it overwriting untracked files or are there local changes that would be overwritten?

Signed-off-by Stefan Beller <[email protected]>

---
 submodule.c | 37 +++++++++++++++++++++++++++++++++++++
 submodule.h |  2 ++
 2 files changed, 39 insertions(+)

submodule.c +37 -0

index 02c28ef56b..4253f7f044 100644
@@ -1155,6 +1155,43 @@ int ok_to_remove_submodule(const char *path)
 	return ok_to_remove;
 }
 
+/**
+ * Check if a submodule update to a given sha1 is safe.
+ * Return 1 if it is safe, 0 when it is not.
+ *
+ * If the submodule is not populated, we need to check
+ */
+int is_submodule_checkout_safe(const char *path,
+			       const struct object_id *new_hash)
+{
+	struct child_process cp = CHILD_PROCESS_INIT;
+
+	argv_array_pushl(&cp.args, "read-tree", "-n", "-m", "HEAD",
+			sha1_to_hex(new_hash->hash), NULL);
+
+	if (!is_submodule_populated(path)) {
+		const struct submodule *sub;
+
+		/* See if we have the submodule configured already: */
+		sub = submodule_from_path(null_sha1, path);
+
+		prepare_submodule_repo_env_no_git_dir(&cp.env_array);
+		argv_array_pushf(&cp.env_array, "GIT_DIR=%s",
+				 sub ? sub->name : path);
+		argv_array_pushf(&cp.env_array, "GIT_WORK_TREE=%s", path);
+	} else {
+		prepare_submodule_repo_env(&cp.env_array);
+	}
+
+	cp.git_cmd = 1;
+	cp.no_stdin = 1;
+	cp.no_stdout = 1;
+	cp.no_stderr = 1;
+	cp.dir = path;
+
+	return run_command(&cp) == 0;
+}
+
 static int find_first_merges(struct object_array *result, const char *path,
 		struct commit *a, struct commit *b)
 {

submodule.h +2 -0

index 74df8b93d5..1dfcd6939b 100644
@@ -72,6 +72,8 @@ extern unsigned is_submodule_modified(const char *path, int ignore_untracked);
 extern int is_submodule_populated(const char *path);
 extern int submodule_uses_gitfile(const char *path);
 extern int ok_to_remove_submodule(const char *path);
+extern int is_submodule_checkout_safe(const char *path,
+				      const struct object_id *new_hash);
 extern int merge_submodule(unsigned char result[20], const char *path,
 			   const unsigned char base[20],
 			   const unsigned char a[20],

Signature · 1 line

2.11.0.rc2.28.g2673dad

Stefan Beller

author

2016-12-03 00:31 UTC #15 lore raw

RFC PATCH v2 11/17 unpack-trees: teach verify_clean_submodule to inspect submodules

In reply to Stefan Beller

As a later patch will modify submodules, if they are interesting we need to see if the submodule is clean in case they are interesting.

If they are not interesting, then we do not care about the submodule keeping historic behavior.

Signed-off-by Stefan Beller <[email protected]>

---
 unpack-trees.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

unpack-trees.c +5 -4

index ea6bdd20e0..22e32eca96 100644
@@ -9,6 +9,7 @@
 #include "refs.h"
 #include "attr.h"
 #include "split-index.h"
+#include "submodule.h"
 #include "dir.h"
 
 /*

@@ -1361,15 +1362,15 @@ static void invalidate_ce_path(const struct cache_entry *ce,
 /*
  * Check that checking out ce->sha1 in subdir ce->name is not
  * going to overwrite any working files.
- *
- * Currently, git does not checkout subprojects during a superproject
- * checkout, so it is not going to overwrite anything.
  */
 static int verify_clean_submodule(const struct cache_entry *ce,
 				  enum unpack_trees_error_types error_type,
 				  struct unpack_trees_options *o)
 {
-	return 0;
+	if (!submodule_is_interesting(ce->name))
+		return 0;
+
+	return !is_submodule_modified(ce->name, 0);
 }
 
 static int verify_clean_subdirectory(const struct cache_entry *ce,

Signature · 1 line

2.11.0.rc2.28.g2673dad

Stefan Beller

author

2016-12-03 00:31 UTC #16 lore raw

RFC PATCH v2 12/17 unpack-trees: remove submodule contents if interesting

In reply to Stefan Beller

Currently when a unlink_entry() is called on a submodule, this fails as remove_or_warn detects it needs to delete a directory via rmdir. However rmdir only works on empty directories, such that the "_or_warn" part kicks in, and we get a warning message.

In case the submodule is of no interest we're not going to delete the submodule, so a warning like that is useful.

If the submodule is interesting then we need to depopulate it properly, there is no need to react on its return code the depopulation method handles proper warnings nor do we need to schedule the directory for removal later, such that we can return early.

Signed-off-by Stefan Beller <[email protected]>

---
 unpack-trees.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

unpack-trees.c +6 -1

index 22e32eca96..db03293347 100644
@@ -214,7 +214,12 @@ static void unlink_entry(const struct cache_entry *ce)
 {
 	if (!check_leading_path(ce->name, ce_namelen(ce)))
 		return;
-	if (remove_or_warn(ce->ce_mode, ce->name))
+
+	if (S_ISGITLINK(ce->ce_mode) &&
+	    submodule_is_interesting(ce->name)) {
+		depopulate_submodule(ce->name);
+		return;
+	} else if (remove_or_warn(ce->ce_mode, ce->name))
 		return;
 	schedule_dir_for_removal(ce->name, ce_namelen(ce));
 }

Signature · 1 line

2.11.0.rc2.28.g2673dad

Stefan Beller

author

2016-12-03 00:31 UTC #17 lore raw

RFC PATCH v2 13/17 entry: write_entry to write populate submodules

In reply to Stefan Beller

Signed-off-by Stefan Beller <[email protected]>

---
 entry.c | 2 ++
 1 file changed, 2 insertions(+)

entry.c +2 -0

index 02c4ac9f22..a668025b8e 100644
@@ -2,6 +2,7 @@
 #include "blob.h"
 #include "dir.h"
 #include "streaming.h"
+#include "submodule.h"
 
 static void create_directories(const char *path, int path_len,
 			       const struct checkout *state)

@@ -208,6 +209,7 @@ static int write_entry(struct cache_entry *ce,
 			return error("cannot create temporary submodule %s", path);
 		if (mkdir(path, 0777) < 0)
 			return error("cannot create submodule directory %s", path);
+		schedule_submodule_for_update(ce, 1);
 		break;
 	default:
 		return error("unknown file mode for %s in index", path);

Signature · 1 line

2.11.0.rc2.28.g2673dad

Stefan Beller

author

2016-12-03 00:31 UTC #18 lore raw

RFC PATCH v2 14/17 submodule: teach unpack_trees() to update submodules

In reply to Stefan Beller

Signed-off-by Stefan Beller <[email protected]>

---
 entry.c        |  7 +++---
 unpack-trees.c | 76 +++++++++++++++++++++++++++++++++++++++++++++-------------
 unpack-trees.h |  1 +
 3 files changed, 65 insertions(+), 19 deletions(-)

entry.c +4 -3

index a668025b8e..3ed885b886 100644
@@ -267,7 +267,7 @@ int checkout_entry(struct cache_entry *ce,
 
 	if (!check_path(path.buf, path.len, &st, state->base_dir_len)) {
 		unsigned changed = ce_match_stat(ce, &st, CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE);
-		if (!changed)
+		if (!changed && (!S_ISDIR(st.st_mode) || !S_ISGITLINK(ce->ce_mode)))
 			return 0;
 		if (!state->force) {
 			if (!state->quiet)

@@ -284,9 +284,10 @@ int checkout_entry(struct cache_entry *ce,
 		 * just do the right thing)
 		 */
 		if (S_ISDIR(st.st_mode)) {
-			/* If it is a gitlink, leave it alone! */
-			if (S_ISGITLINK(ce->ce_mode))
+			if (S_ISGITLINK(ce->ce_mode)) {
+				schedule_submodule_for_update(ce, 1);
 				return 0;
+			}
 			if (!state->force)
 				return error("%s is a directory", path.buf);
 			remove_subtree(&path);

unpack-trees.c +60 ~~-16~~

106 lines not on this page · raw message

unpack-trees.h +1 -0

index 36a73a6d00..bee874088a 100644
@@ -15,6 +15,7 @@ enum unpack_trees_error_types {
 	ERROR_WOULD_OVERWRITE = 0,
 	ERROR_NOT_UPTODATE_FILE,
 	ERROR_NOT_UPTODATE_DIR,
+	ERROR_NOT_UPTODATE_SUBMODULE,
 	ERROR_WOULD_LOSE_UNTRACKED_OVERWRITTEN,
 	ERROR_WOULD_LOSE_UNTRACKED_REMOVED,
 	ERROR_BIND_OVERLAP,

Signature · 1 line

2.11.0.rc2.28.g2673dad

David Turner

2016-12-05 23:37 UTC #19 lore raw

RE: [RFC PATCHv2 14/17] submodule: teach unpack_trees() to update submodules

Quoted 9 lines

-----Original Message-----
From: Stefan Beller [mailto:[email protected]]
Sent: Friday, December 02, 2016 7:30 PM
To: [email protected]; David Turner
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Stefan Beller
Subject: [RFC PATCHv2 14/17] submodule: teach unpack_trees() to update submodules

[snip]

Quoted 21 lines

(quoted diff) +0 -1

 	if (!lstat(ce->name, &st)) {
-		int flags =

CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE;

(quoted diff) +0 -1

-		unsigned changed = ie_match_stat(o->src_index, ce, &st,

flags);

unpack-trees.c +4 ~~-11~~

-		if (!changed)
-			return 0;
-		/*
-		 * NEEDSWORK: the current default policy is to allow
-		 * submodule to be out of sync wrt the superproject
-		 * index.  This needs to be tightened later for
-		 * submodules that are marked to be automatically
-		 * checked out.
-		 */
-		if (S_ISGITLINK(ce->ce_mode))
-			return 0;
+		if (S_ISGITLINK(ce->ce_mode)) {
+			if (!submodule_is_interesting(ce->name))
+				return 0;
+			if (ce_stage(ce) ? is_submodule_checkout_safe(ce->name,

&ce->oid)

unpack-trees.c static int verify_uptodate_1(const struct cache_entry *ce, · line 1333

Quoted 1 line

unpack-trees.c +1 -0

+			    : !is_submodule_modified(ce->name, 1))

This logic is too convoluted. Do a nested if or something.

unpack-trees.c static int verify_uptodate_1(const struct cache_entry *ce, · line 1338

Quoted 6 lines

unpack-trees.c +3 -0

+				return 0;
+		} else {
+			int flags = CE_MATCH_IGNORE_VALID |

CE_MATCH_IGNORE_SKIP_WORKTREE;

unpack-trees.c +2 -0

+			if (!ie_match_stat(o->src_index, ce, &st, flags))
+				return 0;

Nit: I liked the old temp var "changed" better -- it made it clear what ie_match_stat is checking.

Quoted 17 lines

unpack-trees.c +10 -0

+		}
 		errno = 0;
 	}
 	if (errno == ENOENT)

@@ -1355,6 +1359,38 @@ static int verify_uptodate_sparse(const struct
 cache_entry *ce,
 	return verify_uptodate_1(ce, o, ERROR_SPARSE_NOT_UPTODATE_FILE);  }
 
+/*
+ * When a submodule gets turned into an unmerged entry, we want it to
+be
+ * up-to-date regarding the merge changes.
+ */
+static int verify_uptodate_submodule(const struct cache_entry *old,
+				     const struct cache_entry *new,
+				     struct unpack_trees_options *o) {
+	struct stat st;

unpack-trees.c static int verify_uptodate_sparse(const struct · line 1380

Quoted 10 lines

unpack-trees.c +10 -0

+
+	if (o->index_only ||
+	    (!((old->ce_flags & CE_VALID) || ce_skip_worktree(old)) &&
+	      (o->reset || ce_uptodate(old))))
+		return 0;
+
+	if (!submodule_is_interesting(new->name))
+		return 0;
+
+	if (lstat(old->name, &st)) {

We never actually use st here. Should we? If not, is this really the right error message? And should we use access() instead of lstat?

Quoted 7 lines

unpack-trees.c +7 -0

+		if (errno == ENOENT)
+			return 0;
+		return o->gently ? -1 :
+			add_rejected_path(o, ERROR_NOT_UPTODATE_SUBMODULE,
+					  old->name);
+	}
+

Stefan Beller

author

2016-12-03 00:31 UTC #20 lore raw

RFC PATCH v2 15/17 checkout: recurse into submodules if asked to

In reply to Stefan Beller

Allow checkout to recurse into submodules via the command line option --[no-]recurse-submodules.

The flag for recurse-submodules in its current form could be an OPT_BOOL, but eventually we may want to have it as:

    git checkout --recurse-submodules=rebase|merge| \
			cherry-pick|checkout|none

which resembles the submodule.<name>.update options, so naturally a value such as "as-configured-or-X-as-fallback" would also come in handy.

Signed-off-by Stefan Beller <[email protected]>

---
 Documentation/git-checkout.txt |   7 ++
 builtin/checkout.c             |  31 ++++-
 t/t2013-checkout-submodule.sh  | 277 +++++++++++++++++++++++++++++++++++++++--
 3 files changed, 306 insertions(+), 9 deletions(-)

Documentation/git-checkout.txt +7 -0

index 8e2c0662dd..a0ea2c5651 100644
@@ -256,6 +256,13 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.
 	out anyway. In other words, the ref can be held by more than one
 	worktree.
 
+--[no-]recurse-submodules::
+	Using --recurse-submodules will update the content of all initialized
+	submodules according to the commit recorded in the superproject. If
+	local modifications in a submodule would be overwritten the checkout
+	will fail until `-f` is used. If nothing (or --no-recurse-submodules)
+	is used, the work trees of submodules will not be updated.
+
 <branch>::
 	Branch to checkout; if it refers to a branch (i.e., a name that,
 	when prepended with "refs/heads/", is a valid ref), then that

builtin/checkout.c +30 -1

index 512492aad9..5db0d933d1 100644
@@ -21,12 +21,31 @@
 #include "submodule-config.h"
 #include "submodule.h"
 
+static int recurse_submodules = RECURSE_SUBMODULES_DEFAULT;
+
 static const char * const checkout_usage[] = {
 	N_("git checkout [<options>] <branch>"),
 	N_("git checkout [<options>] [<branch>] -- <file>..."),
 	NULL,
 };
 
+int option_parse_recurse_submodules(const struct option *opt,
+				    const char *arg, int unset)
+{
+	if (unset) {
+		recurse_submodules = RECURSE_SUBMODULES_OFF;
+		return 0;
+	}
+	if (arg)
+		recurse_submodules =
+			parse_update_recurse_submodules_arg(opt->long_name,
+							    arg);
+	else
+		recurse_submodules = RECURSE_SUBMODULES_ON;
+
+	return 0;
+}
+
 struct checkout_opts {
 	int patch_mode;
 	int quiet;

@@ -826,7 +845,8 @@ static int switch_branches(const struct checkout_opts *opts,
 		parse_commit_or_die(new->commit);
 	}
 
-	ret = merge_working_tree(opts, &old, new, &writeout_error);
+	ret = merge_working_tree(opts, &old, new, &writeout_error) ||
+	      update_submodules(opts->force);
 	if (ret) {
 		free(path_to_free);
 		return ret;

@@ -1160,6 +1180,9 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 				N_("second guess 'git checkout <no-such-branch>'")),
 		OPT_BOOL(0, "ignore-other-worktrees", &opts.ignore_other_worktrees,
 			 N_("do not check if another worktree is holding the given ref")),
+		{ OPTION_CALLBACK, 0, "recurse-submodules", &recurse_submodules,
+			    "checkout", "control recursive updating of submodules",
+			    PARSE_OPT_OPTARG, option_parse_recurse_submodules },
 		OPT_BOOL(0, "progress", &opts.show_progress, N_("force progress reporting")),
 		OPT_END(),
 	};

@@ -1190,6 +1213,12 @@ int cmd_checkout(int argc, const char **argv, const char *prefix)
 		git_xmerge_config("merge.conflictstyle", conflict_style, NULL);
 	}
 
+	if (recurse_submodules != RECURSE_SUBMODULES_OFF) {
+		git_config(submodule_config, NULL);
+		if (recurse_submodules != RECURSE_SUBMODULES_DEFAULT)
+			set_config_update_recurse_submodules(recurse_submodules);
+	}
+
 	if ((!!opts.new_branch + !!opts.new_branch_force + !!opts.new_orphan_branch) > 1)
 		die(_("-b, -B and --orphan are mutually exclusive"));

t/t2013-checkout-submodule.sh +269 -8

297 lines not on this page · raw message

Signature · 1 line

2.11.0.rc2.28.g2673dad

Brandon Williams

2016-12-05 19:25 UTC #21 lore raw

Re: [RFC PATCHv2 15/17] checkout: recurse into submodules if asked to

t/t2013-checkout-submodule.sh test_expect_success '"checkout <submodule>" honors submodule.*.ignore from .git/ · line 71

Quoted 5 lines · Stefan Beller

test_expect_success '"checkout <submodule>" honors diff.ignoreSubmodules' '
t/t2013-checkout-submodule.sh +0 -0
@@ -63,6 +70,260 @@ test_expect_success '"checkout <submodule>" honors submodule.*.ignore from .git/
 	! test -s actual
 '

Should you use test_must_fail and not '!'?

Signature · 1 line

Brandon Williams

Stefan Beller

author

2016-12-05 19:31 UTC #22 lore raw

Re: [RFC PATCHv2 15/17] checkout: recurse into submodules if asked to

Quoted 8 lines · Brandon Williams

t/t2013-checkout-submodule.sh test_expect_success '"checkout <submodule>" honors submodule.*.ignore from .git/ · line 71
Quoted 5 lines · Stefan Beller
test_expect_success '"checkout <submodule>" honors diff.ignoreSubmodules' '
t/t2013-checkout-submodule.sh +0 -0
@@ -63,6 +70,260 @@ test_expect_success '"checkout <submodule>" honors submodule.*.ignore from .git/
      ! test -s actual
 '
Should you use test_must_fail and not '!'?

We use test_must_fail for git and '!' for non git thigns (test, grep etc), as the test suite is about testing git.

The test_must_fail expects the command to be run to
* not reurn 0 (success)
* not segfault
* not return some other arbitrary return codes indicating abnormal failure (125 IIRC)

So in a way test_must_fail translates to:
"I want to run this git command and it should fail gracefully because at this state, it is the best git can do"

The '!' however is just inverting the boolean expression. We assume test, grep, et al. to be flawless here. ;)

Stefan

Brandon Williams

2016-12-05 19:31 UTC #23 lore raw

Re: [RFC PATCHv2 15/17] checkout: recurse into submodules if asked to

Quoted 19 lines · Stefan Beller

Quoted 8 lines · Brandon Williams
t/t2013-checkout-submodule.sh test_expect_success '"checkout <submodule>" honors submodule.*.ignore from .git/ · line 71
Quoted 5 lines · Stefan Beller
test_expect_success '"checkout <submodule>" honors diff.ignoreSubmodules' '
t/t2013-checkout-submodule.sh +0 -0
@@ -63,6 +70,260 @@ test_expect_success '"checkout <submodule>" honors submodule.*.ignore from .git/
      ! test -s actual
 '
Should you use test_must_fail and not '!'?
We use test_must_fail for git and '!' for non git thigns (test, grep etc), as the test suite is about testing git.
The test_must_fail expects the command to be run to
* not reurn 0 (success)
* not segfault
* not return some other arbitrary return codes indicating abnormal failure (125 IIRC)

Quoted 6 lines

So in a way test_must_fail translates to:
"I want to run this git command and it should fail gracefully because at this state, it is the best git can do"
The '!' however is just inverting the boolean expression. We assume test, grep, et al. to be flawless here. ;)

Ah, alright. Thanks for the info :)

Signature · 1 line

Brandon Williams

David Turner

2016-12-05 23:36 UTC #24 lore raw

RE: [RFC PATCHv2 15/17] checkout: recurse into submodules if asked to

In reply to Stefan Beller

Quoted 10 lines

-----Original Message-----
From: Stefan Beller [mailto:[email protected]]
Sent: Friday, December 02, 2016 7:30 PM
To: [email protected]; David Turner
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Stefan Beller
Subject: [RFC PATCHv2 15/17] checkout: recurse into submodules if asked to
Allow checkout to recurse into submodules via the command line option -- [no-]recurse-submodules.

This option probably needs to precede 9/17 "update submodules: add scheduling to update submodules", since that patch uses --recurse-submodules.

Stefan Beller

author

2016-12-03 00:31 UTC #25 lore raw

RFC PATCH v2 16/17 completion: add '--recurse-submodules' to checkout

In reply to Stefan Beller

Signed-off-by Stefan Beller <[email protected]>

---
 contrib/completion/git-completion.bash | 2 +-
 t/t9902-completion.sh                  | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

contrib/completion/git-completion.bash +1 -1

index 21016bf8df..28acfdb377 100644
@@ -1068,7 +1068,7 @@ _git_checkout ()
 	--*)
 		__gitcomp "
 			--quiet --ours --theirs --track --no-track --merge
-			--conflict= --orphan --patch
+			--conflict= --orphan --patch --recurse-submodules
 			"
 		;;
 	*)

t/t9902-completion.sh +1 -0

index 2ba62fbc17..d2d1102a3d 100755
@@ -447,6 +447,7 @@ test_expect_success 'double dash "git checkout"' '
 	--conflict=
 	--orphan Z
 	--patch Z
+	--recurse-submodules Z
 	EOF
 '

Signature · 1 line

2.11.0.rc2.28.g2673dad

Stefan Beller

author

2016-12-03 00:31 UTC #26 lore raw

RFC PATCH v2 17/17 checkout: add config option to recurse into submodules by default

In reply to Stefan Beller

To make it easier for the user, who doesn't want to give the --recurse-submodules option whenever they run checkout, have an option for to set the default behavior for checkout to recurse into submodules.

Signed-off-by Stefan Beller <[email protected]>

---
 Documentation/config.txt       |  6 ++++++
 Documentation/git-checkout.txt |  5 +++--
 submodule.c                    |  6 +++---
 t/t2013-checkout-submodule.sh  | 16 ++++++++++++++++
 4 files changed, 28 insertions(+), 5 deletions(-)

Documentation/config.txt +6 -0

index a0ab66aae7..67e0714358 100644
@@ -949,6 +949,12 @@ clean.requireForce::
 	A boolean to make git-clean do nothing unless given -f,
 	-i or -n.   Defaults to true.
 
+checkout.recurseSubmodules::
+	This option can be set to a boolean value.
+	When set to true checkout will recurse into submodules and
+	update them. When set to false, which is the default, checkout
+	will leave submodules untouched.
+
 color.branch::
 	A boolean to enable/disable color in the output of
 	linkgit:git-branch[1]. May be set to `always`,

Documentation/git-checkout.txt +3 -2

index a0ea2c5651..819c430b6e 100644
@@ -260,8 +260,9 @@ section of linkgit:git-add[1] to learn how to operate the `--patch` mode.
 	Using --recurse-submodules will update the content of all initialized
 	submodules according to the commit recorded in the superproject. If
 	local modifications in a submodule would be overwritten the checkout
-	will fail until `-f` is used. If nothing (or --no-recurse-submodules)
-	is used, the work trees of submodules will not be updated.
+	will fail until `-f` is used. If `--no-recurse-submodules` is used,
+	the work trees of submodules will not be updated. If no command line
+	argument is given, `checkout.recurseSubmodules` is used as a default.
 
 <branch>::
 	Branch to checkout; if it refers to a branch (i.e., a name that,

submodule.c +3 -3

index 4253f7f044..e4e326bce4 100644
@@ -160,10 +160,10 @@ int submodule_config(const char *var, const char *value, void *cb)
 		return 0;
 	} else if (starts_with(var, "submodule."))
 		return parse_submodule_config_option(var, value);
-	else if (!strcmp(var, "fetch.recursesubmodules")) {
+	else if (!strcmp(var, "fetch.recursesubmodules"))
 		config_fetch_recurse_submodules = parse_fetch_recurse_submodules_arg(var, value);
-		return 0;
-	}
+	else if (!strcmp(var, "checkout.recursesubmodules"))
+		config_update_recurse_submodules = parse_update_recurse_submodules_arg(var, value);
 	return 0;
 }

t/t2013-checkout-submodule.sh +16 -0

index 33fb2f5de3..673021fb80 100755
@@ -137,6 +137,22 @@ test_expect_success '"checkout --recurse-submodules" repopulates submodule' '
 	submodule_creation_must_succeed delete_submodule base
 '
 
+test_expect_success 'option checkout.recurseSubmodules updates submodule' '
+	test_config checkout.recurseSubmodules 1 &&
+	git checkout base &&
+	git checkout -b advanced-base &&
+	git -C submodule commit --allow-empty -m "empty commit" &&
+	git add submodule &&
+	git commit -m "advance submodule" &&
+	git checkout base &&
+	git diff-files --quiet &&
+	git diff-index --quiet --cached base &&
+	git checkout advanced-base &&
+	git diff-files --quiet &&
+	git diff-index --quiet --cached advanced-base &&
+	git checkout --recurse-submodules base
+'
+
 test_expect_success '"checkout --recurse-submodules" repopulates submodule in existing directory' '
 	git checkout --recurse-submodules delete_submodule &&
 	mkdir submodule &&

Signature · 1 line

2.11.0.rc2.28.g2673dad

Brandon Williams

2016-12-05 19:29 UTC #27 lore raw

Re: [RFC PATCHv2 17/17] checkout: add config option to recurse into submodules by default

Quoted 6 lines · Stefan Beller

t/t2013-checkout-submodule.sh +6 -0

+test_expect_success 'option checkout.recurseSubmodules updates submodule' '
+	test_config checkout.recurseSubmodules 1 &&
+	git checkout base &&
+	git checkout -b advanced-base &&
+	git -C submodule commit --allow-empty -m "empty commit" &&
+	git add submodule &&

t/t2013-checkout-submodule.sh test_expect_success '"checkout --recurse-submodules" repopulates submodule' ' · line 155

Quoted 10 lines

t/t2013-checkout-submodule.sh +10 -0

+	git commit -m "advance submodule" &&
+	git checkout base &&
+	git diff-files --quiet &&
+	git diff-index --quiet --cached base &&
+	git checkout advanced-base &&
+	git diff-files --quiet &&
+	git diff-index --quiet --cached advanced-base &&
+	git checkout --recurse-submodules base
+'
+

This test doesn't look like it looks into the submodule to see if the submodule has indeed changed. Unless diff-index and diff-files recurse into the submodules?

Signature · 1 line

Brandon Williams

Stefan Beller

author

2016-12-05 22:23 UTC #28 lore raw

Re: [RFC PATCHv2 17/17] checkout: add config option to recurse into submodules by default

Quoted 13 lines · Brandon Williams

Quoted 12 lines · Stefan Beller

t/t2013-checkout-submodule.sh +12 -0

+test_expect_success 'option checkout.recurseSubmodules updates submodule' '
+     test_config checkout.recurseSubmodules 1 &&
+     git checkout base &&
+     git checkout -b advanced-base &&
+     git -C submodule commit --allow-empty -m "empty commit" &&
+     git add submodule &&
+     git commit -m "advance submodule" &&
+     git checkout base &&
+     git diff-files --quiet &&
+     git diff-index --quiet --cached base &&
+     git checkout advanced-base &&
+     git diff-files --quiet &&

Quoted 8 lines

t/t2013-checkout-submodule.sh test_expect_success '"checkout --recurse-submodules" repopulates submodule' ' · line 155
Quoted 4 lines
t/t2013-checkout-submodule.sh +4 -0
+     git diff-index --quiet --cached advanced-base &&
+     git checkout --recurse-submodules base
+'
+
This test doesn't look like it looks into the submodule to see if the submodule has indeed changed. Unless diff-index and diff-files recurse into the submodules?

I took the code from Jens once upon a time. Rereading the code, I agree it is not obvious how this checks the submodule state.

However git diff-files --quiet is perfectly fine, as we have submodule support by default via:

Omitting the --submodule option or specifying --submodule=short, uses the short format. This format just shows the names of the commits at the beginning and end of the range.

and then we turn it into an exit code via

--quiet
Disable all output of the program. Implies --exit-code.

--exit-code
Make the program exit with codes similar to diff(1).
That is, it exits with 1 if there were differences and 0 means no differences.

Same for diff-index.

The main purpose of this specific test is to have checkout.recurseSubmodules "to just make it work" without having to give --recurse-submodules manually. All the other tests with the manual --recurse-submodules should test for correctness of the behavior within the submodule.

So maybe I'll need to rewrite submodule_creation_must_succeed() in the previous patch to be more obvious. (Well that already has some tests for files/directories
in there, so it is a little more.)

But to be sure we can also add tests here that look more into the submodule. I am thinking of "{new,old}_sub_sha1=$(git -C submodule rev-parse HEAD)" and comparing them?

Brandon Williams

2016-12-05 22:26 UTC #29 lore raw

Re: [RFC PATCHv2 17/17] checkout: add config option to recurse into submodules by default

Quoted 49 lines · Stefan Beller

Quoted 13 lines · Brandon Williams
Quoted 12 lines · Stefan Beller
t/t2013-checkout-submodule.sh +12 -0
+test_expect_success 'option checkout.recurseSubmodules updates submodule' '
+     test_config checkout.recurseSubmodules 1 &&
+     git checkout base &&
+     git checkout -b advanced-base &&
+     git -C submodule commit --allow-empty -m "empty commit" &&
+     git add submodule &&
+     git commit -m "advance submodule" &&
+     git checkout base &&
+     git diff-files --quiet &&
+     git diff-index --quiet --cached base &&
+     git checkout advanced-base &&
+     git diff-files --quiet &&
Quoted 8 lines
t/t2013-checkout-submodule.sh test_expect_success '"checkout --recurse-submodules" repopulates submodule' ' · line 155
Quoted 4 lines
t/t2013-checkout-submodule.sh +4 -0
+     git diff-index --quiet --cached advanced-base &&
+     git checkout --recurse-submodules base
+'
+
This test doesn't look like it looks into the submodule to see if the submodule has indeed changed. Unless diff-index and diff-files recurse into the submodules?
I took the code from Jens once upon a time. Rereading the code, I agree it is not obvious how this checks the submodule state.
However git diff-files --quiet is perfectly fine, as we have submodule support by default via:
Omitting the --submodule option or specifying --submodule=short, uses the short format. This format just shows the names of the commits at the beginning and end of the range.
and then we turn it into an exit code via
--quiet
Disable all output of the program. Implies --exit-code.
--exit-code
Make the program exit with codes similar to diff(1).
That is, it exits with 1 if there were differences and 0 means no differences.
Same for diff-index.
The main purpose of this specific test is to have checkout.recurseSubmodules "to just make it work" without having to give --recurse-submodules manually. All the other tests with the manual --recurse-submodules should test for correctness of the behavior within the submodule.

Quoted 8 lines

So maybe I'll need to rewrite submodule_creation_must_succeed() in the previous patch to be more obvious. (Well that already has some tests for files/directories
in there, so it is a little more.)
But to be sure we can also add tests here that look more into the submodule. I am thinking of "{new,old}_sub_sha1=$(git -C submodule rev-parse HEAD)" and comparing them?

Ah ok, that makes sense now. Its kind of like if I run git status it would show if a submodule is at a different sha1 than the superproject has recorded.

Signature · 1 line

Brandon Williams