Skip to contentSkip to conversation

share a config between submodule and superproject

RFC PATCH Superseded Emily Shaffer 7 participants 25 messages Apr 8, 2021 – Jun 19, 2021 Documentation

lore
Share & tools
lore
#0 lore raw

cover share a config between submodule and superproject

Hi folks,

Especially after es/config-based-hooks makes its way to 'next' and 'master', we think it would be really useful to have a config that applies to a whole "project" - where "project" means "superproject and its submodules." There's a longer defense in patch 2, but that's the speedy summary.

I'm not wild about the implementation of this solution as it calls out to 'git rev-parse' and 'git ls-files' - not once, but twice! - and I understand that is considerably slower on some platforms than others. But I'm open to suggestions - I couldn't find any other examples of a repo figuring out whether it's a submodule or not. (I thought there may be some, as 'repository.submodule_prefix' exists, but it seems like that's only set in some special cases during operations initated from the superproject.)

I'm hoping to work on some other submodule-centric stuff over the coming months, and it might end up being very useful to be able to tell "am I a submodule?" and "how do I talk to my superproject?" more generally - so I'm really open to figuring out a better way than this, if folks have ideas.

Patch 1 is a small refactor that we can take or leave - I found "SCOPE_SUBMODULE" to be pretty ambiguous, especially since it seems to refer to configs from .gitmodules. Even though I decided that "superproject" was a better name than "submodule" I still wasn't super happy with the ambiguity. But we can drop it if folks don't want to rename.

Thanks a bunch,
- Emily

Emily Shaffer (2):
config: rename "submodule" scope to "gitmodules"
config: add 'config.superproject' file

 Documentation/git-config.txt   |  21 +++++-
 builtin/config.c               |  10 ++-
 config.c                       |  30 +++++++-
 config.h                       |   5 +-
 submodule-config.c             |   2 +-
 submodule.c                    |  29 ++++++++
 submodule.h                    |   6 ++
 t/t1311-superproject-config.sh | 124 +++++++++++++++++++++++++++++++++
 8 files changed, 220 insertions(+), 7 deletions(-)
 create mode 100755 t/t1311-superproject-config.sh
Signature · 1 line
2.31.1.295.g9ea45b61b8-goog
#1 lore raw

RFC PATCH 1/2 config: rename "submodule" scope to "gitmodules"

To prepare for the addition of a new config scope which only applies to submodule projects, disambiguate "CONFIG_SCOPE_SUBMODULES". This scope refers to configs gathered from the .gitmodules file in the superproject, so just call it "gitmodules."

---
 config.c           | 4 ++--
 config.h           | 2 +-
 submodule-config.c | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)
config.c +2 -2
index 6428393a41..67d9bf2238 100644
@@ -3513,8 +3513,8 @@ const char *config_scope_name(enum config_scope scope)
 		return "worktree";
 	case CONFIG_SCOPE_COMMAND:
 		return "command";
-	case CONFIG_SCOPE_SUBMODULE:
-		return "submodule";
+	case CONFIG_SCOPE_GITMODULES:
+		return "gitmodules";
 	default:
 		return "unknown";
 	}
config.h +1 -1
index 19a9adbaa9..535f5517b8 100644
@@ -42,7 +42,7 @@ enum config_scope {
 	CONFIG_SCOPE_LOCAL,
 	CONFIG_SCOPE_WORKTREE,
 	CONFIG_SCOPE_COMMAND,
-	CONFIG_SCOPE_SUBMODULE,
+	CONFIG_SCOPE_GITMODULES,
 };
 const char *config_scope_name(enum config_scope scope);
 
submodule-config.c +1 -1
index f502505566..0e435e6fdd 100644
@@ -637,7 +637,7 @@ static void config_from_gitmodules(config_fn_t fn, struct repository *repo, void
 {
 	if (repo->worktree) {
 		struct git_config_source config_source = {
-			0, .scope = CONFIG_SCOPE_SUBMODULE
+			0, .scope = CONFIG_SCOPE_GITMODULES
 		};
 		const struct config_options opts = { 0 };
 		struct object_id oid;
Signature · 1 line
2.31.1.295.g9ea45b61b8-goog
#2 lore raw

RFC PATCH 2/2 config: add 'config.superproject' file

In reply to Emily Shaffer

Some configs, such as wrapper directives like gerrit.createChangeId, or forthcoming hook configs, should apply to a superproject as well as all its submodules. It may not be appropriate to apply them globally - for example, if the user also contributes to many projects which do not use the configs necessary for one project-with-submodules - and it may be burdensome to apply them locally to the superproject and each of its submodules. Even if the user runs 'git submodule foreach "git config --local foo.bar', if a new submodule is added later on, that config is not applied to the new submodule.

It is also inappropriate to share the entire superproject config, since some items - like remote URLs or partial-clone filters - would not apply to a submodule.

To make life easier for projects with many submodules, then, create a new "config.superproject" config scope, which is included in the config parse for the superproject as well as for all the submodules of that superproject.

For the superproject, this new config file is equally local to the local config; for the submodule, the new config file is less local than the local config. So let's include it directly before the local config during the config parse.

---
 Documentation/git-config.txt   |  21 +++++-
 builtin/config.c               |  10 ++-
 config.c                       |  26 +++++++
 config.h                       |   3 +
 submodule.c                    |  29 ++++++++
 submodule.h                    |   6 ++
 t/t1311-superproject-config.sh | 124 +++++++++++++++++++++++++++++++++
 7 files changed, 216 insertions(+), 3 deletions(-)
 create mode 100755 t/t1311-superproject-config.sh
Documentation/git-config.txt +19 -2
index 4b4cc5c5e8..a33136fb08 100644
@@ -48,7 +48,7 @@ unset an existing `--type` specifier with `--no-type`.
 
 When reading, the values are read from the system, global and
 repository local configuration files by default, and options
-`--system`, `--global`, `--local`, `--worktree` and
+`--system`, `--global`, `--superproject`, `--local`, `--worktree` and
 `--file <filename>` can be used to tell the command to read from only
 that location (see <<FILES>>).
 
@@ -127,6 +127,17 @@ rather than from all available files.
 +
 See also <<FILES>>.
 
+--superproject::
+	For writing options: write to the superproject's
+	`.git/config.superproject` file, even if run from a submodule of that
+	superproject.
++
+For reading options: read only from the superproject's
+`.git/config.superproject` file, even if run from a submodule of that
+superproject, rather than from all available files.
++
+See also <<FILES>>.
+
 --local::
 	For writing options: write to the repository `.git/config` file.
 	This is the default behavior.
@@ -283,7 +294,7 @@ The default is to use a pager.
 FILES
 -----
 
-If not set explicitly with `--file`, there are four files where
+If not set explicitly with `--file`, there are five files where
 'git config' will search for configuration options:
 
 $(prefix)/etc/gitconfig::
@@ -301,6 +312,12 @@ $XDG_CONFIG_HOME/git/config::
 	User-specific configuration file. Also called "global"
 	configuration file.
 
+$GIT_DIR/config.superproject::
+	When `git config` is run from a project which is a submodule of another
+	project, that superproject's $GIT_DIR will be used. Use this config file
+	to set configurations which need to be the same across a superproject
+	and all its submodules.
+
 $GIT_DIR/config::
 	Repository specific configuration file.
 
builtin/config.c +9 -1
index f71fa39b38..f0a57a89ca 100644
@@ -26,7 +26,7 @@ static char key_delim = ' ';
 static char term = '\n';
 
 static int use_global_config, use_system_config, use_local_config;
-static int use_worktree_config;
+static int use_worktree_config, use_superproject_config;
 static struct git_config_source given_config_source;
 static int actions, type;
 static char *default_value;
@@ -130,6 +130,8 @@ static struct option builtin_config_options[] = {
 	OPT_GROUP(N_("Config file location")),
 	OPT_BOOL(0, "global", &use_global_config, N_("use global config file")),
 	OPT_BOOL(0, "system", &use_system_config, N_("use system config file")),
+	OPT_BOOL(0, "superproject",
+		 &use_superproject_config, N_("use superproject config file")),
 	OPT_BOOL(0, "local", &use_local_config, N_("use repository config file")),
 	OPT_BOOL(0, "worktree", &use_worktree_config, N_("use per-worktree config file")),
 	OPT_STRING('f', "file", &given_config_source.file, N_("file"), N_("use given config file")),
@@ -697,6 +699,12 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 	else if (use_system_config) {
 		given_config_source.file = git_etc_gitconfig();
 		given_config_source.scope = CONFIG_SCOPE_SYSTEM;
+	} else if (use_superproject_config) {
+		struct strbuf superproject_cfg = STRBUF_INIT;
+		git_config_superproject(&superproject_cfg, get_git_dir());
+		given_config_source.file = xstrdup(superproject_cfg.buf);
+		given_config_source.scope = CONFIG_SCOPE_SUPERPROJECT;
+		strbuf_release(&superproject_cfg);
 	} else if (use_local_config) {
 		given_config_source.file = git_pathdup("config");
 		given_config_source.scope = CONFIG_SCOPE_LOCAL;
config.c +26 -0
index 67d9bf2238..28bb80fd0d 100644
@@ -21,6 +21,7 @@
 #include "dir.h"
 #include "color.h"
 #include "refs.h"
+#include "submodule.h"
 
 struct config_source {
 	struct config_source *prev;
@@ -1852,6 +1853,17 @@ const char *git_etc_gitconfig(void)
 	return system_wide;
 }
 
+void git_config_superproject(struct strbuf *sb, const char *gitdir)
+{
+	if (!get_superproject_gitdir(sb)) {
+		/* not a submodule */
+		strbuf_reset(sb);
+		strbuf_addstr(sb, gitdir);
+	}
+
+	strbuf_addstr(sb, "/config.superproject");
+}
+
 /*
  * Parse environment variable 'k' as a boolean (in various
  * possible spellings); if missing, use the default value 'def'.
@@ -1909,6 +1921,17 @@ static int do_git_config_sequence(const struct config_options *opts,
 	if (user_config && !access_or_die(user_config, R_OK, ACCESS_EACCES_OK))
 		ret += git_config_from_file(fn, user_config, data);
 
+	current_parsing_scope = CONFIG_SCOPE_SUPERPROJECT;
+	if (opts->git_dir && !opts->ignore_superproject) {
+
+		struct strbuf superproject_gitdir = STRBUF_INIT;
+		git_config_superproject(&superproject_gitdir, opts->git_dir);
+		if (!access_or_die(superproject_gitdir.buf, R_OK, 0))
+			ret += git_config_from_file(fn, superproject_gitdir.buf, data);
+
+		strbuf_release(&superproject_gitdir);
+	}
+
 	current_parsing_scope = CONFIG_SCOPE_LOCAL;
 	if (!opts->ignore_repo && repo_config &&
 	    !access_or_die(repo_config, R_OK, 0))
@@ -2027,6 +2050,7 @@ void read_very_early_config(config_fn_t cb, void *data)
 
 	opts.respect_includes = 1;
 	opts.ignore_repo = 1;
+	opts.ignore_superproject = 1;
 	opts.ignore_worktree = 1;
 	opts.ignore_cmdline = 1;
 	opts.system_gently = 1;
@@ -3515,6 +3539,8 @@ const char *config_scope_name(enum config_scope scope)
 		return "command";
 	case CONFIG_SCOPE_GITMODULES:
 		return "gitmodules";
+	case CONFIG_SCOPE_SUPERPROJECT:
+		return "superproject";
 	default:
 		return "unknown";
 	}
config.h +3 -0
index 535f5517b8..b42e1d13eb 100644
@@ -43,6 +43,7 @@ enum config_scope {
 	CONFIG_SCOPE_WORKTREE,
 	CONFIG_SCOPE_COMMAND,
 	CONFIG_SCOPE_GITMODULES,
+	CONFIG_SCOPE_SUPERPROJECT,
 };
 const char *config_scope_name(enum config_scope scope);
 
@@ -84,6 +85,7 @@ typedef int (*config_parser_event_fn_t)(enum config_event_t type,
 struct config_options {
 	unsigned int respect_includes : 1;
 	unsigned int ignore_repo : 1;
+	unsigned int ignore_superproject : 1;
 	unsigned int ignore_worktree : 1;
 	unsigned int ignore_cmdline : 1;
 	unsigned int system_gently : 1;
@@ -319,6 +321,7 @@ int git_config_rename_section_in_file(const char *, const char *, const char *);
 int git_config_copy_section(const char *, const char *);
 int git_config_copy_section_in_file(const char *, const char *, const char *);
 const char *git_etc_gitconfig(void);
+void git_config_superproject(struct strbuf *, const char *);
 int git_env_bool(const char *, int);
 unsigned long git_env_ulong(const char *, unsigned long);
 int git_config_system(void);
submodule.c +29 -0
index 9767ba9893..92b00f8697 100644
@@ -2178,6 +2178,35 @@ void absorb_git_dir_into_superproject(const char *path,
 	}
 }
 
+int get_superproject_gitdir(struct strbuf *buf)
+{
+	struct strbuf sb = STRBUF_INIT;
+	struct child_process cp = CHILD_PROCESS_INIT;
+	int rc = 0;
+
+	/* NEEDSWORK: this call also calls out to a subprocess! */
+	rc = get_superproject_working_tree(&sb);
+	strbuf_release(&sb);
+
+	if (!rc)
+		return rc;
+
+	prepare_submodule_repo_env_no_git_dir(&cp.env_array);
+
+	strvec_pushl(&cp.args, "-C", "..", "rev-parse", "--absolute-git-dir", NULL);
+	cp.git_cmd = 1;
+
+	rc = capture_command(&cp, buf, 0);
+	strbuf_trim_trailing_newline(buf);
+
+	/* leave buf empty if we didn't have a superproject gitdir */
+	if (rc)
+		strbuf_reset(buf);
+
+	/* rc reflects the exit code of the rev-parse; invert into a bool */
+	return !rc;
+}
+
 int get_superproject_working_tree(struct strbuf *buf)
 {
 	struct child_process cp = CHILD_PROCESS_INIT;
submodule.h +6 -0
index 4ac6e31cf1..1308d5ae2d 100644
@@ -149,6 +149,12 @@ void prepare_submodule_repo_env(struct strvec *out);
 void absorb_git_dir_into_superproject(const char *path,
 				      unsigned flags);
 
+/*
+ * Return the gitdir of the superproject, which this project is a submodule of.
+ * If this repository is not a submodule of another repository, return 0.
+ */
+int get_superproject_gitdir(struct strbuf *buf);
+
 /*
  * Return the absolute path of the working tree of the superproject, which this
  * project is a submodule of. If this repository is not a submodule of
t/t1311-superproject-config.sh new file +124 -0

124 lines not on this page · raw message

Signature · 1 line
2.31.1.295.g9ea45b61b8-goog
#3 lore raw

Re: [RFC PATCH 2/2] config: add 'config.superproject' file

Quoted 19 lines · Emily Shaffer

Some configs, such as wrapper directives like gerrit.createChangeId, or forthcoming hook configs, should apply to a superproject as well as all its submodules. It may not be appropriate to apply them globally - for example, if the user also contributes to many projects which do not use the configs necessary for one project-with-submodules - and it may be burdensome to apply them locally to the superproject and each of its submodules. Even if the user runs 'git submodule foreach "git config --local foo.bar', if a new submodule is added later on, that config is not applied to the new submodule.

It is also inappropriate to share the entire superproject config, since some items - like remote URLs or partial-clone filters - would not apply to a submodule.

To make life easier for projects with many submodules, then, create a new "config.superproject" config scope, which is included in the config parse for the superproject as well as for all the submodules of that superproject.

Quoted 7 lines

For the superproject, this new config file is equally local to the local config; for the submodule, the new config file is less local than the local config. So let's include it directly before the local config during the config parse.

---

Does this need an update to the git config --show-origin --show-scope capability?
--
Philip

Quoted 378 lines
 Documentation/git-config.txt   |  21 +++++-
 builtin/config.c               |  10 ++-
 config.c                       |  26 +++++++
 config.h                       |   3 +
 submodule.c                    |  29 ++++++++
 submodule.h                    |   6 ++
 t/t1311-superproject-config.sh | 124 +++++++++++++++++++++++++++++++++
 7 files changed, 216 insertions(+), 3 deletions(-)
 create mode 100755 t/t1311-superproject-config.sh
Documentation/git-config.txt +19 -2
index 4b4cc5c5e8..a33136fb08 100644
@@ -48,7 +48,7 @@ unset an existing `--type` specifier with `--no-type`.
 
 When reading, the values are read from the system, global and
 repository local configuration files by default, and options
-`--system`, `--global`, `--local`, `--worktree` and
+`--system`, `--global`, `--superproject`, `--local`, `--worktree` and
 `--file <filename>` can be used to tell the command to read from only
 that location (see <<FILES>>).
 
@@ -127,6 +127,17 @@ rather than from all available files.
 +
 See also <<FILES>>.
 
+--superproject::
+	For writing options: write to the superproject's
+	`.git/config.superproject` file, even if run from a submodule of that
+	superproject.
++
+For reading options: read only from the superproject's
+`.git/config.superproject` file, even if run from a submodule of that
+superproject, rather than from all available files.
++
+See also <<FILES>>.
+
 --local::
 	For writing options: write to the repository `.git/config` file.
 	This is the default behavior.
@@ -283,7 +294,7 @@ The default is to use a pager.
 FILES
 -----
 
-If not set explicitly with `--file`, there are four files where
+If not set explicitly with `--file`, there are five files where
 'git config' will search for configuration options:
 
 $(prefix)/etc/gitconfig::
@@ -301,6 +312,12 @@ $XDG_CONFIG_HOME/git/config::
 	User-specific configuration file. Also called "global"
 	configuration file.
 
+$GIT_DIR/config.superproject::
+	When `git config` is run from a project which is a submodule of another
+	project, that superproject's $GIT_DIR will be used. Use this config file
+	to set configurations which need to be the same across a superproject
+	and all its submodules.
+
 $GIT_DIR/config::
 	Repository specific configuration file.
 
builtin/config.c +9 -1
index f71fa39b38..f0a57a89ca 100644
@@ -26,7 +26,7 @@ static char key_delim = ' ';
 static char term = '\n';
 
 static int use_global_config, use_system_config, use_local_config;
-static int use_worktree_config;
+static int use_worktree_config, use_superproject_config;
 static struct git_config_source given_config_source;
 static int actions, type;
 static char *default_value;
@@ -130,6 +130,8 @@ static struct option builtin_config_options[] = {
 	OPT_GROUP(N_("Config file location")),
 	OPT_BOOL(0, "global", &use_global_config, N_("use global config file")),
 	OPT_BOOL(0, "system", &use_system_config, N_("use system config file")),
+	OPT_BOOL(0, "superproject",
+		 &use_superproject_config, N_("use superproject config file")),
 	OPT_BOOL(0, "local", &use_local_config, N_("use repository config file")),
 	OPT_BOOL(0, "worktree", &use_worktree_config, N_("use per-worktree config file")),
 	OPT_STRING('f', "file", &given_config_source.file, N_("file"), N_("use given config file")),
@@ -697,6 +699,12 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 	else if (use_system_config) {
 		given_config_source.file = git_etc_gitconfig();
 		given_config_source.scope = CONFIG_SCOPE_SYSTEM;
+	} else if (use_superproject_config) {
+		struct strbuf superproject_cfg = STRBUF_INIT;
+		git_config_superproject(&superproject_cfg, get_git_dir());
+		given_config_source.file = xstrdup(superproject_cfg.buf);
+		given_config_source.scope = CONFIG_SCOPE_SUPERPROJECT;
+		strbuf_release(&superproject_cfg);
 	} else if (use_local_config) {
 		given_config_source.file = git_pathdup("config");
 		given_config_source.scope = CONFIG_SCOPE_LOCAL;
config.c +26 -0
index 67d9bf2238..28bb80fd0d 100644
@@ -21,6 +21,7 @@
 #include "dir.h"
 #include "color.h"
 #include "refs.h"
+#include "submodule.h"
 
 struct config_source {
 	struct config_source *prev;
@@ -1852,6 +1853,17 @@ const char *git_etc_gitconfig(void)
 	return system_wide;
 }
 
+void git_config_superproject(struct strbuf *sb, const char *gitdir)
+{
+	if (!get_superproject_gitdir(sb)) {
+		/* not a submodule */
+		strbuf_reset(sb);
+		strbuf_addstr(sb, gitdir);
+	}
+
+	strbuf_addstr(sb, "/config.superproject");
+}
+
 /*
  * Parse environment variable 'k' as a boolean (in various
  * possible spellings); if missing, use the default value 'def'.
@@ -1909,6 +1921,17 @@ static int do_git_config_sequence(const struct config_options *opts,
 	if (user_config && !access_or_die(user_config, R_OK, ACCESS_EACCES_OK))
 		ret += git_config_from_file(fn, user_config, data);
 
+	current_parsing_scope = CONFIG_SCOPE_SUPERPROJECT;
+	if (opts->git_dir && !opts->ignore_superproject) {
+
+		struct strbuf superproject_gitdir = STRBUF_INIT;
+		git_config_superproject(&superproject_gitdir, opts->git_dir);
+		if (!access_or_die(superproject_gitdir.buf, R_OK, 0))
+			ret += git_config_from_file(fn, superproject_gitdir.buf, data);
+
+		strbuf_release(&superproject_gitdir);
+	}
+
 	current_parsing_scope = CONFIG_SCOPE_LOCAL;
 	if (!opts->ignore_repo && repo_config &&
 	    !access_or_die(repo_config, R_OK, 0))
@@ -2027,6 +2050,7 @@ void read_very_early_config(config_fn_t cb, void *data)
 
 	opts.respect_includes = 1;
 	opts.ignore_repo = 1;
+	opts.ignore_superproject = 1;
 	opts.ignore_worktree = 1;
 	opts.ignore_cmdline = 1;
 	opts.system_gently = 1;
@@ -3515,6 +3539,8 @@ const char *config_scope_name(enum config_scope scope)
 		return "command";
 	case CONFIG_SCOPE_GITMODULES:
 		return "gitmodules";
+	case CONFIG_SCOPE_SUPERPROJECT:
+		return "superproject";
 	default:
 		return "unknown";
 	}
config.h +3 -0
index 535f5517b8..b42e1d13eb 100644
@@ -43,6 +43,7 @@ enum config_scope {
 	CONFIG_SCOPE_WORKTREE,
 	CONFIG_SCOPE_COMMAND,
 	CONFIG_SCOPE_GITMODULES,
+	CONFIG_SCOPE_SUPERPROJECT,
 };
 const char *config_scope_name(enum config_scope scope);
 
@@ -84,6 +85,7 @@ typedef int (*config_parser_event_fn_t)(enum config_event_t type,
 struct config_options {
 	unsigned int respect_includes : 1;
 	unsigned int ignore_repo : 1;
+	unsigned int ignore_superproject : 1;
 	unsigned int ignore_worktree : 1;
 	unsigned int ignore_cmdline : 1;
 	unsigned int system_gently : 1;
@@ -319,6 +321,7 @@ int git_config_rename_section_in_file(const char *, const char *, const char *);
 int git_config_copy_section(const char *, const char *);
 int git_config_copy_section_in_file(const char *, const char *, const char *);
 const char *git_etc_gitconfig(void);
+void git_config_superproject(struct strbuf *, const char *);
 int git_env_bool(const char *, int);
 unsigned long git_env_ulong(const char *, unsigned long);
 int git_config_system(void);
submodule.c +29 -0
index 9767ba9893..92b00f8697 100644
@@ -2178,6 +2178,35 @@ void absorb_git_dir_into_superproject(const char *path,
 	}
 }
 
+int get_superproject_gitdir(struct strbuf *buf)
+{
+	struct strbuf sb = STRBUF_INIT;
+	struct child_process cp = CHILD_PROCESS_INIT;
+	int rc = 0;
+
+	/* NEEDSWORK: this call also calls out to a subprocess! */
+	rc = get_superproject_working_tree(&sb);
+	strbuf_release(&sb);
+
+	if (!rc)
+		return rc;
+
+	prepare_submodule_repo_env_no_git_dir(&cp.env_array);
+
+	strvec_pushl(&cp.args, "-C", "..", "rev-parse", "--absolute-git-dir", NULL);
+	cp.git_cmd = 1;
+
+	rc = capture_command(&cp, buf, 0);
+	strbuf_trim_trailing_newline(buf);
+
+	/* leave buf empty if we didn't have a superproject gitdir */
+	if (rc)
+		strbuf_reset(buf);
+
+	/* rc reflects the exit code of the rev-parse; invert into a bool */
+	return !rc;
+}
+
 int get_superproject_working_tree(struct strbuf *buf)
 {
 	struct child_process cp = CHILD_PROCESS_INIT;
submodule.h +6 -0
index 4ac6e31cf1..1308d5ae2d 100644
@@ -149,6 +149,12 @@ void prepare_submodule_repo_env(struct strvec *out);
 void absorb_git_dir_into_superproject(const char *path,
 				      unsigned flags);
 
+/*
+ * Return the gitdir of the superproject, which this project is a submodule of.
+ * If this repository is not a submodule of another repository, return 0.
+ */
+int get_superproject_gitdir(struct strbuf *buf);
+
 /*
  * Return the absolute path of the working tree of the superproject, which this
  * project is a submodule of. If this repository is not a submodule of
t/t1311-superproject-config.sh new file +124 -0
new file mode 100755
index 0000000000..650c4d24c7
@@ -0,0 +1,124 @@
+#!/bin/sh
+
+test_description='Test git config --superproject in different settings'
+
+. ./test-lib.sh
+
+# follow t7400's example and use the trash dir repo as a submodule to add
+submodurl=$(pwd -P)
+
+# since only the configs are modified, set up the repo structure only once
+test_expect_success 'setup repo structure' '
+	test_commit "base" &&
+	git submodule add "${submodurl}" sub/ &&
+	git commit -m "add a submodule"
+'
+
+test_expect_success 'superproject config applies to super and submodule' '
+	cat >.git/config.superproject <<-EOF &&
+	[foo]
+		bar = baz
+	EOF
+
+	git config --get foo.bar &&
+	git -C sub config --get foo.bar &&
+
+	rm .git/config.superproject
+'
+
+test_expect_success 'can add from super or sub' '
+	git config --superproject apple.species honeycrisp &&
+	git -C sub config --superproject banana.species cavendish &&
+
+	cat >expect <<-EOF &&
+	apple.species=honeycrisp
+	banana.species=cavendish
+	EOF
+
+	git config --list >actual &&
+	grep -Ff expect actual &&
+
+	git -C sub config --list >actual &&
+	grep -Ff expect actual &&
+
+	rm .git/config.superproject
+'
+
+test_expect_success 'can --unset from super or sub' '
+	git config --superproject apple.species honeycrisp &&
+	git -C sub config --superproject banana.species cavendish &&
+
+	git config --unset --superproject banana.species &&
+	git -C sub config --unset --superproject apple.species
+'
+
+test_expect_success 'can --edit superproject config' '
+	test_config core.editor "echo [foo]bar=baz >" &&
+	git config --edit --superproject &&
+
+	git config --get foo.bar &&
+
+	rm .git/config.superproject
+'
+
+test_expect_success 'can --show-origin the superproject config' '
+	git config --superproject --add foo.bar baz &&
+
+	git config --list --show-origin >actual &&
+	grep -F "config.superproject" actual &&
+
+	rm .git/config.superproject
+'
+
+test_expect_success 'can --show-scope the superproject config' '
+	git config --superproject --add foo.bar baz &&
+
+	git config --list --show-scope >actual &&
+	grep "superproject" actual &&
+
+	rm .git/config.superproject
+'
+
+test_expect_success 'pre-existing config applies to new submodule' '
+	git config --superproject --add foo.bar baz &&
+
+	git submodule add "${submodurl}" sub2/ &&
+	git commit -m "add a second submodule" &&
+
+	git -C sub2 config --get foo.bar &&
+
+	# clean up
+	git reset HEAD^ &&
+	rm -fr sub2 &&
+	rm .git/config.superproject
+'
+
+# NEEDSWORK: submodule.c:get_superproject_working_tree doesn't support worktree
+test_expect_failure 'worktrees can still access config.superproject' '
+	git config --superproject --add foo.bar baz &&
+
+	git worktree add wt &&
+	(
+		cd wt &&
+		git config --get foo.bar
+	) &&
+
+	# clean up
+	git worktree remove wt &&
+	rm .git/config.superproject
+'
+
+# This test deletes the submodule! Keep it at the end of the test suite.
+test_expect_success 'config.submodule works even with no submodules' '
+	# get rid of the submodule
+	git reset HEAD^ &&
+	rm -fr sub &&
+
+	git config --superproject --add foo.bar baz &&
+
+	git config --get foo.bar &&
+
+	rm .git/config.superproject
+'
+
+test_done
#4 lore raw

Re: [RFC PATCH 2/2] config: add 'config.superproject' file

Quoted 26 lines · Philip Oakley
Quoted 24 lines · Emily Shaffer

Some configs, such as wrapper directives like gerrit.createChangeId, or forthcoming hook configs, should apply to a superproject as well as all its submodules. It may not be appropriate to apply them globally - for example, if the user also contributes to many projects which do not use the configs necessary for one project-with-submodules - and it may be burdensome to apply them locally to the superproject and each of its submodules. Even if the user runs 'git submodule foreach "git config --local foo.bar', if a new submodule is added later on, that config is not applied to the new submodule.

It is also inappropriate to share the entire superproject config, since some items - like remote URLs or partial-clone filters - would not apply to a submodule.

To make life easier for projects with many submodules, then, create a new "config.superproject" config scope, which is included in the config parse for the superproject as well as for all the submodules of that superproject.

For the superproject, this new config file is equally local to the local config; for the submodule, the new config file is less local than the local config. So let's include it directly before the local config during the config parse.

Quoted 5 lines
Quoted 2 lines

---

Does this need an update to the git config --show-origin --show-scope capability?

It's included:

Quoted 36 lines
Quoted 36 lines
config.c +2 -0
@@ -3515,6 +3539,8 @@ const char *config_scope_name(enum config_scope scope)
 		return "command";
 	case CONFIG_SCOPE_GITMODULES:
 		return "gitmodules";
+	case CONFIG_SCOPE_SUPERPROJECT:
+		return "superproject";
 	default:
 		return "unknown";
 	}
config.h +1 -0
index 535f5517b8..b42e1d13eb 100644
@@ -43,6 +43,7 @@ enum config_scope {
 	CONFIG_SCOPE_WORKTREE,
 	CONFIG_SCOPE_COMMAND,
 	CONFIG_SCOPE_GITMODULES,
+	CONFIG_SCOPE_SUPERPROJECT,
 };
 
t/t1311-superproject-config.sh new file +9 -0
new file mode 100755
index 0000000000..650c4d24c7
+test_expect_success 'can --show-origin the superproject config' '
+	git config --superproject --add foo.bar baz &&
+
+	git config --list --show-origin >actual &&
+	grep -F "config.superproject" actual &&
+
+	rm .git/config.superproject
+'
+
Quoted 8 lines

t/t1311-superproject-config.sh line 80

Quoted 8 lines
t/t1311-superproject-config.sh new file +8 -0
+test_expect_success 'can --show-scope the superproject config' '
+	git config --superproject --add foo.bar baz &&
+
+	git config --list --show-scope >actual &&
+	grep "superproject" actual &&
+
+	rm .git/config.superproject
+'

- Emily

#5 lore raw

Re: [RFC PATCH 2/2] config: add 'config.superproject' file

In reply to Emily Shaffer

Hi, Emily

I'm not familiar enough with this code to give a full review and I imagine you probably want comments more focused on the design level, while this is an RFC, but here are some small nitpicks I found while reading the patch. I Hope it helps :)

Quoted 15 lines · Emily Shaffer
Documentation/git-config.txt +1 -1
index 4b4cc5c5e8..a33136fb08 100644
@@ -48,7 +48,7 @@ unset an existing `--type` specifier with `--no-type`.
 
 When reading, the values are read from the system, global and
 repository local configuration files by default, and options
-`--system`, `--global`, `--local`, `--worktree` and
+`--system`, `--global`, `--superproject`, `--local`, `--worktree` and
 `--file <filename>` can be used to tell the command to read from only
 that location (see <<FILES>>).
 

Documentation/git-config.txt rather than from all available files. · line 133

Quoted 7 lines
Documentation/git-config.txt +4 -0
@@ -127,6 +127,17 @@ rather than from all available files.
 +
 See also <<FILES>>.
 
+--superproject::
+       For writing options: write to the superproject's
+       `.git/config.superproject` file, even if run from a submodule of that
+       superproject.

Hmm, I wonder what happens if a repo is both a submodule and a superproject (i.e. in case of nested submodules). Let's see:

# Create repo/sub/sub2

$ git init repo
$ cd repo
$ touch F && git add F && git commit -m F
$ git submodule add ./ sub
$ git -C sub submodule add ./sub sub2
$ git -C sub commit -m sub2
$ git commit -m sub

# Now test the config

$ git -C sub/sub2 config --superproject foo.bar 1
$ git -C sub/sub2 config --get foo.bar
1
$ git -C sub config --get foo.bar
<nothing>
$ git config --get foo.bar
<nothing>

It makes sense to me that foo.bar is not defined on repo, but shouldn't it be defined on repo/sub? Or am I doing something wrong?

(git -C sub rev-parse --git-dir gives .git/modules/sub/, where indeed there is a config.superproject with foo.bar set.)

Quoted 23 lines
builtin/config.c +3 -1
index f71fa39b38..f0a57a89ca 100644
@@ -26,7 +26,7 @@ static char key_delim = ' ';
 static char term = '\n';
 
 static int use_global_config, use_system_config, use_local_config;
-static int use_worktree_config;
+static int use_worktree_config, use_superproject_config;
 static struct git_config_source given_config_source;
 static int actions, type;
 static char *default_value;
@@ -130,6 +130,8 @@ static struct option builtin_config_options[] = {
        OPT_GROUP(N_("Config file location")),
        OPT_BOOL(0, "global", &use_global_config, N_("use global config file")),
        OPT_BOOL(0, "system", &use_system_config, N_("use system config file")),
+       OPT_BOOL(0, "superproject",
+                &use_superproject_config, N_("use superproject config file")),
        OPT_BOOL(0, "local", &use_local_config, N_("use repository config file")),
        OPT_BOOL(0, "worktree", &use_worktree_config, N_("use per-worktree config file")),
        OPT_STRING('f', "file", &given_config_source.file, N_("file"), N_("use given config file")),

builtin/config.c int cmd_config(int argc, const char **argv, const char *prefix) · line 707

Quoted 9 lines
builtin/config.c +6 -0
@@ -697,6 +699,12 @@ int cmd_config(int argc, const char **argv, const char *prefix)
        else if (use_system_config) {
                given_config_source.file = git_etc_gitconfig();
                given_config_source.scope = CONFIG_SCOPE_SYSTEM;
+       } else if (use_superproject_config) {
+               struct strbuf superproject_cfg = STRBUF_INIT;
+               git_config_superproject(&superproject_cfg, get_git_dir());
+               given_config_source.file = xstrdup(superproject_cfg.buf);
+               given_config_source.scope = CONFIG_SCOPE_SUPERPROJECT;
+               strbuf_release(&superproject_cfg);

Nit: maybe it would be a bit cleaner to replace the xstrdup() + strbuf_release() lines with a single:

    given_config_source.file = strbuf_detach(superproject_cfg, NULL);
Quoted 16 lines
builtin/config.c +0 -0
        } else if (use_local_config) {
                given_config_source.file = git_pathdup("config");
                given_config_source.scope = CONFIG_SCOPE_LOCAL;
config.c +1 -0
index 67d9bf2238..28bb80fd0d 100644
@@ -21,6 +21,7 @@
 #include "dir.h"
 #include "color.h"
 #include "refs.h"
+#include "submodule.h"
 
 struct config_source {
        struct config_source *prev;

config.c const char *git_etc_gitconfig(void) · line 1860

Quoted 8 lines
config.c +5 -0
@@ -1852,6 +1853,17 @@ const char *git_etc_gitconfig(void)
        return system_wide;
 }
 
+void git_config_superproject(struct strbuf *sb, const char *gitdir)
+{
+       if (!get_superproject_gitdir(sb)) {
+               /* not a submodule */
+               strbuf_reset(sb);

Do we have to reset sb here? It seems that get_superproject_gitdir() leaves the buffer empty when we are not inside a submodule.

submodule.h void prepare_submodule_repo_env(struct strvec *out); · line 154

Quoted 11 lines
submodule.h +3 -0
index 4ac6e31cf1..1308d5ae2d 100644
@@ -149,6 +149,12 @@ void prepare_submodule_repo_env(struct strvec *out);
 void absorb_git_dir_into_superproject(const char *path,
                                      unsigned flags);
 
+/*
+ * Return the gitdir of the superproject, which this project is a submodule of.
+ * If this repository is not a submodule of another repository, return 0.

Nit: it might be nice to say what's the state of buf on a 0 return. Perhaps also be more explicit about the return codes? Maybe something like:

"If this repository is a submodule of another repository, save the superproject's gitdir on buf and return 1. Otherwise, return 0 and leave buf empty."

Quoted 11 lines
submodule.h +2 -0
+int get_superproject_gitdir(struct strbuf *buf);
+
 /*
  * Return the absolute path of the working tree of the superproject, which this
  * project is a submodule of. If this repository is not a submodule of
t/t1311-superproject-config.sh new file +0 -0
new file mode 100755
index 0000000000..650c4d24c7
@@ -0,0 +1,124 @@

[...]

t/t1311-superproject-config.sh line 26

Quoted 10 lines
t/t1311-superproject-config.sh new file +10 -0
+test_expect_success 'superproject config applies to super and submodule' '
+       cat >.git/config.superproject <<-EOF &&
+       [foo]
+               bar = baz
+       EOF
+
+       git config --get foo.bar &&
+       git -C sub config --get foo.bar &&
+
+       rm .git/config.superproject

Hmm, if this test fails before removing the config.superproject file, couldn't it interfere with other tests (like the 'can --edit superproject config')? Perhaps this and the other similar cleanup removals could be declared inside a test_when_finished clause, to ensure they are performed even on test failure.

t/t1311-superproject-config.sh line 53

Quoted 7 lines
t/t1311-superproject-config.sh new file +7 -0
+test_expect_success 'can --unset from super or sub' '
+       git config --superproject apple.species honeycrisp &&
+       git -C sub config --superproject banana.species cavendish &&
+
+       git config --unset --superproject banana.species &&
+       git -C sub config --unset --superproject apple.species
+'

Nice "cross-setting/unsetting" test :)

[...]

t/t1311-superproject-config.sh line 112

Quoted 2 lines
t/t1311-superproject-config.sh new file +2 -0
+# This test deletes the submodule! Keep it at the end of the test suite.
+test_expect_success 'config.submodule works even with no submodules' '

s/config.submodule/config.superproject/ ?

#6 lore raw

Re: [RFC PATCH 2/2] config: add 'config.superproject' file

Quoted 26 lines · Matheus Tavares Bernardino

Hi, Emily

I'm not familiar enough with this code to give a full review and I imagine you probably want comments more focused on the design level, while this is an RFC, but here are some small nitpicks I found while reading the patch. I Hope it helps :)

Quoted 18 lines · Emily Shaffer
Documentation/git-config.txt +1 -1
index 4b4cc5c5e8..a33136fb08 100644
@@ -48,7 +48,7 @@ unset an existing `--type` specifier with `--no-type`.
 
 When reading, the values are read from the system, global and
 repository local configuration files by default, and options
-`--system`, `--global`, `--local`, `--worktree` and
+`--system`, `--global`, `--superproject`, `--local`, `--worktree` and
 `--file <filename>` can be used to tell the command to read from only
 that location (see <<FILES>>).
 
@@ -127,6 +127,17 @@ rather than from all available files.
 +
 See also <<FILES>>.
 
Quoted 7 lines

Documentation/git-config.txt rather than from all available files. · line 133

Quoted 4 lines
Documentation/git-config.txt +4 -0
+--superproject::
+       For writing options: write to the superproject's
+       `.git/config.superproject` file, even if run from a submodule of that
+       superproject.

Hmm, I wonder what happens if a repo is both a submodule and a superproject (i.e. in case of nested submodules).

Another thing I am not sure about the design is that a repository can be shared as a submodule by more than one superprojects. The superprojects may want their submodule checkouts at different submodule commits, but that is something doable by having multiple worktrees connected to a single submodule repository.

I think our design principle has been that it is perfectly OK for a superproject to be in total control if its submodules, but submodules should not even be aware of being used as a submodule by a superproject, and that allows a submodule repository to be shared by multiple superprojects. As "write to the superproject's X file" requires a submodule to know who THE superproject of itself is, this feature itself (not the implementation) feels somewhat iffy.

#7 lore raw

Re: [RFC PATCH 2/2] config: add 'config.superproject' file

Quoted 37 lines · Junio C Hamano
Quoted 26 lines · Matheus Tavares Bernardino

Hi, Emily

I'm not familiar enough with this code to give a full review and I imagine you probably want comments more focused on the design level, while this is an RFC, but here are some small nitpicks I found while reading the patch. I Hope it helps :)

Quoted 18 lines · Emily Shaffer
Documentation/git-config.txt +1 -1
index 4b4cc5c5e8..a33136fb08 100644
@@ -48,7 +48,7 @@ unset an existing `--type` specifier with `--no-type`.
 
 When reading, the values are read from the system, global and
 repository local configuration files by default, and options
-`--system`, `--global`, `--local`, `--worktree` and
+`--system`, `--global`, `--superproject`, `--local`, `--worktree` and
 `--file <filename>` can be used to tell the command to read from only
 that location (see <<FILES>>).
 
@@ -127,6 +127,17 @@ rather than from all available files.
 +
 See also <<FILES>>.
 
Quoted 7 lines

Documentation/git-config.txt rather than from all available files. · line 133

Quoted 4 lines
Documentation/git-config.txt +4 -0
+--superproject::
+       For writing options: write to the superproject's
+       `.git/config.superproject` file, even if run from a submodule of that
+       superproject.

Hmm, I wonder what happens if a repo is both a submodule and a superproject (i.e. in case of nested submodules).

Quoted 5 lines

Another thing I am not sure about the design is that a repository can be shared as a submodule by more than one superprojects. The superprojects may want their submodule checkouts at different submodule commits, but that is something doable by having multiple worktrees connected to a single submodule repository.

I think the implementation as-written actually handles this sharing-via-worktree case you describe gracefully, as it discovers the gitdir belonging to the worktree above the worktree where it is being run now:

superproject-a
-> sub-a <gitdir at superproject-a/.git/modules/sub-a/> superproject-b
-> sub-b <gitdir at superproject-a/.git/modules/sub-a/worktrees/sub-b/>

In this case running git config --list --superproject in sub-a will yield superproject-a/.git/config.superproject and running it in sub-b will yield superproject-b/.git/config.superproject; that seems logical to me. If I am adding libc as a submodule via worktree to Git as well as, say, Wireshark, just to save me on disk space, I wouldn't want my Wireshark hooks to run in Git project or vice versa.

Quoted 7 lines

I think our design principle has been that it is perfectly OK for a superproject to be in total control if its submodules, but submodules should not even be aware of being used as a submodule by a superproject, and that allows a submodule repository to be shared by multiple superprojects. As "write to the superproject's X file" requires a submodule to know who THE superproject of itself is, this feature itself (not the implementation) feels somewhat iffy.

As for this, I wonder what the reasoning is. I guess to simplify the code, and to make the behavior more predictable (for example, 'git commit' doesn't suddenly make a commit in some project that isn't this one)?

One could imagine some really nice quality-of-life improvements if the submodule is allowed to know it's a submodule (even by a config, for example):

- We could teach 'git status' to indicate the state when the submodule index is clean, but the superproject does not contain a commit pointing to the submodule's HEAD - which could still be considered a dirty state, since the change isn't associated with the larger project yet. I could imagine there might be other handy information related to submodule/superproject status we may want to display too.
- We could teach 'git log' to decorate commits which are referred to by superproject commits, perhaps?
- We could teach 'git push' to, by option or config, push the entire superproject-and-submodules package at once, to make it easier to coordinate changes across the whole superproject
- One could envision some other niceties like 'git stash --whole-superproject' or similar, where a user could operate on the entire overall project (that is, the superproject and all its submodules) without needing to switch context away

I don't think lacking these things would stop a user from doing something they want to do, but it does seem like they could make life more comfortable for a user developing in a project made up of many submodules.

- Emily

#8 lore raw

Re: [RFC PATCH 2/2] config: add 'config.superproject' file

In reply to Matheus Tavares Bernardino
Quoted 53 lines · Matheus Tavares Bernardino

Hi, Emily

I'm not familiar enough with this code to give a full review and I imagine you probably want comments more focused on the design level, while this is an RFC, but here are some small nitpicks I found while reading the patch. I Hope it helps :)

Quoted 15 lines · Emily Shaffer
Documentation/git-config.txt +1 -1
index 4b4cc5c5e8..a33136fb08 100644
@@ -48,7 +48,7 @@ unset an existing `--type` specifier with `--no-type`.
 
 When reading, the values are read from the system, global and
 repository local configuration files by default, and options
-`--system`, `--global`, `--local`, `--worktree` and
+`--system`, `--global`, `--superproject`, `--local`, `--worktree` and
 `--file <filename>` can be used to tell the command to read from only
 that location (see <<FILES>>).
 

Documentation/git-config.txt rather than from all available files. · line 133

Quoted 7 lines
Documentation/git-config.txt +4 -0
@@ -127,6 +127,17 @@ rather than from all available files.
 +
 See also <<FILES>>.
 
+--superproject::
+       For writing options: write to the superproject's
+       `.git/config.superproject` file, even if run from a submodule of that
+       superproject.

Hmm, I wonder what happens if a repo is both a submodule and a superproject (i.e. in case of nested submodules). Let's see:

# Create repo/sub/sub2

$ git init repo
$ cd repo
$ touch F && git add F && git commit -m F
$ git submodule add ./ sub
$ git -C sub submodule add ./sub sub2
$ git -C sub commit -m sub2
$ git commit -m sub

# Now test the config

$ git -C sub/sub2 config --superproject foo.bar 1
$ git -C sub/sub2 config --get foo.bar
1
$ git -C sub config --get foo.bar
<nothing>
$ git config --get foo.bar
<nothing>
Quoted 5 lines

It makes sense to me that foo.bar is not defined on repo, but shouldn't it be defined on repo/sub? Or am I doing something wrong?

(git -C sub rev-parse --git-dir gives .git/modules/sub/, where indeed there is a config.superproject with foo.bar set.)

Yeah, this isn't very surprising. The code does essentially:

  parent_gitdir = git -C ../ rev-parse --git-dir
  config_parse_order += $parent_gitdir/config.superproject

That is, in the nested submodules case, it's looking at .git/modules/sub/config.superproject - but 'sub' is getting its superproject config from .git/config.superproject and has no such file of its own.

One way I could see to solve it would be to skip the "find parent gitdir" thing entirely:

  git -C ../ config --superproject --list >parent_superproject_cfg
  config_parse_order += parent_superproject_cfg
  config_parse_order += $my_own_gitdir/config.superproject

The recursion is a little icky, I guess, but submodules are all recursive anyway, right? :) Hmm.

Quoted 37 lines
Quoted 24 lines
builtin/config.c +3 -1
index f71fa39b38..f0a57a89ca 100644
@@ -26,7 +26,7 @@ static char key_delim = ' ';
 static char term = '\n';
 
 static int use_global_config, use_system_config, use_local_config;
-static int use_worktree_config;
+static int use_worktree_config, use_superproject_config;
 static struct git_config_source given_config_source;
 static int actions, type;
 static char *default_value;
@@ -130,6 +130,8 @@ static struct option builtin_config_options[] = {
        OPT_GROUP(N_("Config file location")),
        OPT_BOOL(0, "global", &use_global_config, N_("use global config file")),
        OPT_BOOL(0, "system", &use_system_config, N_("use system config file")),
+       OPT_BOOL(0, "superproject",
+                &use_superproject_config, N_("use superproject config file")),
        OPT_BOOL(0, "local", &use_local_config, N_("use repository config file")),
        OPT_BOOL(0, "worktree", &use_worktree_config, N_("use per-worktree config file")),
        OPT_STRING('f', "file", &given_config_source.file, N_("file"), N_("use given config file")),

builtin/config.c int cmd_config(int argc, const char **argv, const char *prefix) · line 707

Quoted 9 lines
builtin/config.c +6 -0
@@ -697,6 +699,12 @@ int cmd_config(int argc, const char **argv, const char *prefix)
        else if (use_system_config) {
                given_config_source.file = git_etc_gitconfig();
                given_config_source.scope = CONFIG_SCOPE_SYSTEM;
+       } else if (use_superproject_config) {
+               struct strbuf superproject_cfg = STRBUF_INIT;
+               git_config_superproject(&superproject_cfg, get_git_dir());
+               given_config_source.file = xstrdup(superproject_cfg.buf);
+               given_config_source.scope = CONFIG_SCOPE_SUPERPROJECT;
+               strbuf_release(&superproject_cfg);

Nit: maybe it would be a bit cleaner to replace the xstrdup() + strbuf_release() lines with a single:

Quoted 1 line
    given_config_source.file = strbuf_detach(superproject_cfg, NULL);

Oh nice, thanks!

Quoted 20 lines
Quoted 20 lines
builtin/config.c +0 -0
        } else if (use_local_config) {
                given_config_source.file = git_pathdup("config");
                given_config_source.scope = CONFIG_SCOPE_LOCAL;
config.c +1 -0
index 67d9bf2238..28bb80fd0d 100644
@@ -21,6 +21,7 @@
 #include "dir.h"
 #include "color.h"
 #include "refs.h"
+#include "submodule.h"
 
 struct config_source {
        struct config_source *prev;
@@ -1852,6 +1853,17 @@ const char *git_etc_gitconfig(void)
        return system_wide;
 }
 
Quoted 8 lines

config.c const char *git_etc_gitconfig(void) · line 1860

Quoted 5 lines
config.c +5 -0
+void git_config_superproject(struct strbuf *sb, const char *gitdir)
+{
+       if (!get_superproject_gitdir(sb)) {
+               /* not a submodule */
+               strbuf_reset(sb);

Do we have to reset sb here? It seems that get_superproject_gitdir() leaves the buffer empty when we are not inside a submodule.

Since I didn't promise it in the documentation I included the reset here, but I see your comment just after this suggesting I should also promise it in the documentation ;)

Quoted 17 lines

submodule.h void prepare_submodule_repo_env(struct strvec *out); · line 154

Quoted 12 lines
submodule.h +3 -0
index 4ac6e31cf1..1308d5ae2d 100644
@@ -149,6 +149,12 @@ void prepare_submodule_repo_env(struct strvec *out);
 void absorb_git_dir_into_superproject(const char *path,
                                      unsigned flags);
 
+/*
+ * Return the gitdir of the superproject, which this project is a submodule of.
+ * If this repository is not a submodule of another repository, return 0.

Nit: it might be nice to say what's the state of buf on a 0 return. Perhaps also be more explicit about the return codes? Maybe something like:

Quoted 3 lines

"If this repository is a submodule of another repository, save the superproject's gitdir on buf and return 1. Otherwise, return 0 and leave buf empty."

Seems reasonable.

Quoted 13 lines
Quoted 12 lines
submodule.h +2 -0
+int get_superproject_gitdir(struct strbuf *buf);
+
 /*
  * Return the absolute path of the working tree of the superproject, which this
  * project is a submodule of. If this repository is not a submodule of
t/t1311-superproject-config.sh new file +0 -0
new file mode 100755
index 0000000000..650c4d24c7
@@ -0,0 +1,124 @@

[...]

Quoted 16 lines

t/t1311-superproject-config.sh line 26

Quoted 10 lines
t/t1311-superproject-config.sh new file +10 -0
+test_expect_success 'superproject config applies to super and submodule' '
+       cat >.git/config.superproject <<-EOF &&
+       [foo]
+               bar = baz
+       EOF
+
+       git config --get foo.bar &&
+       git -C sub config --get foo.bar &&
+
+       rm .git/config.superproject

Hmm, if this test fails before removing the config.superproject file, couldn't it interfere with other tests (like the 'can --edit superproject config')? Perhaps this and the other similar cleanup removals could be declared inside a test_when_finished clause, to ensure they are performed even on test failure.

Oh sure, thanks.

Quoted 11 lines

t/t1311-superproject-config.sh line 53

Quoted 8 lines
t/t1311-superproject-config.sh new file +7 -0
+test_expect_success 'can --unset from super or sub' '
+       git config --superproject apple.species honeycrisp &&
+       git -C sub config --superproject banana.species cavendish &&
+
+       git config --unset --superproject banana.species &&
+       git -C sub config --unset --superproject apple.species
+'

Nice "cross-setting/unsetting" test :)

Quoted 5 lines

[...]

t/t1311-superproject-config.sh line 112

Quoted 2 lines
t/t1311-superproject-config.sh new file +2 -0
+# This test deletes the submodule! Keep it at the end of the test suite.
+test_expect_success 'config.submodule works even with no submodules' '

s/config.submodule/config.superproject/ ?

Aaauuurgggh :)

Thanks for the nits!
- Emily

#9 lore raw

Future structure of submodules and .git/, .git/modules/* organization

In reply to Emily Shaffer
Quoted 6 lines · Emily Shaffer

I'm hoping to work on some other submodule-centric stuff over the coming months, and it might end up being very useful to be able to tell "am I a submodule?" and "how do I talk to my superproject?" more generally - so I'm really open to figuring out a better way than this, if folks have ideas.

Quoted 6 lines

Patch 1 is a small refactor that we can take or leave - I found "SCOPE_SUBMODULE" to be pretty ambiguous, especially since it seems to refer to configs from .gitmodules. Even though I decided that "superproject" was a better name than "submodule" I still wasn't super happy with the ambiguity. But we can drop it if folks don't want to rename.

This is less on your patch, and more on the larger work you're suggesting, but the two are kind of related. Skip to the paragraph starting with "But why" below for the relevance :)

I very much wish that we could eventually make the use of submodules totally transparent, i.e. (taking the example of git.git):

* You clone, and we just get objects from
https://github.com/cr-marcstevens/sha1collisiondetection.git too

* The fact that we have:

160000 commit 855827c583bc30645ba427885caa40c5b81764d2 sha1collisiondetection

Would become totally invisible to most users unless they run some gutsy ls-tree/files comand.

We used to have a full git dir at sha1collisiondetection/.git and all the UX issues that entailed (e.g. switching to an old commit without the submodule).

Now it's a stub and the actual repo is at .git/modules/sha1collisiondetection/, so we're kind of partially there.

* I would think that the next (but big) logical step would be to use some combination of delta islands, upcoming sparse indexes etc. to actually share the object stores of the parent and submodule.

Things like "git fsck" which now just punt on COMMIT would need to become smarter, but e.g. we could repack (or not, with islands) between parent and submodule.

I would think that this end goal makes more sense than the current status quo of teaching every command that needs to e.g. grep the tree to have a "--recurse-submodules". The distinction would be invisible to the likes of "git-grep".

It would mean more complexity in e.g. "git commit", but we can imagine if you wanted a cross-submodule commit it could do those commits recursively, update parent COMMIT entries etc. (and even, optionally, push out the submodule changes). That particular thing being so ad-hoc is a *very* frequent pain point in submodule use.

But why am I talking about this here when all you're suggesting is another config level?

Well, I think (but have not carefully thought about) that this CONFIG_SCOPE_GITMODULES is probably a narrow net improvement now. If you set most options in your .git/config to you that's the same logical project, why shouldn't you get your diff setting or whatever because you cd'd to a submodule "in the same project" (from the view of the user).

But I think that for a wider "improve submodules" effort it's worth someone (and right now, that sounds like it's you) thinking about where we're going with the feature. Maybe with some technical doc identifying the most common pain points, what we propose (or could envision) doing about them.

So e.g. in this case, having per-submodule config could be a step forward, but it could also be one more step of walking in a circle.

I.e. don't think any user asked for or wanted to stitch together multiple .git directories into one linked pseudo-checkout, that's ultimately something we're exposing as an implementation detail. If we no longer expose that implementation detail, would we be stuck supporting what's ultimately a workaround feature?

None of that means we shouldn't have that one step forward that solves real problems today.

But I think we should think about the end goal(s) sooner than later. E.g. in your case, do you *really* want another config level, or is it just the easiest way to get what you actually want, which is for a "git config" in the submodule dir to perhaps consider its .git/config and .git/modules/sha1collisiondetection/config as the same file for the purposes of config parsing? Sans things like the remote URLs etc.

#10 lore raw

Re: Future structure of submodules and .git/, .git/modules/* organization

Quoted 32 lines · Ævar Arnfjörð Bjarmason
Quoted 6 lines · Emily Shaffer

I'm hoping to work on some other submodule-centric stuff over the coming months, and it might end up being very useful to be able to tell "am I a submodule?" and "how do I talk to my superproject?" more generally - so I'm really open to figuring out a better way than this, if folks have ideas.

Quoted 6 lines

Patch 1 is a small refactor that we can take or leave - I found "SCOPE_SUBMODULE" to be pretty ambiguous, especially since it seems to refer to configs from .gitmodules. Even though I decided that "superproject" was a better name than "submodule" I still wasn't super happy with the ambiguity. But we can drop it if folks don't want to rename.

This is less on your patch, and more on the larger work you're suggesting, but the two are kind of related. Skip to the paragraph starting with "But why" below for the relevance :)

I very much wish that we could eventually make the use of submodules totally transparent, i.e. (taking the example of git.git):

* You clone, and we just get objects from
https://github.com/cr-marcstevens/sha1collisiondetection.git too

* The fact that we have:

160000 commit 855827c583bc30645ba427885caa40c5b81764d2 sha1collisiondetection

Would become totally invisible to most users unless they run some gutsy ls-tree/files comand.

Quoted 7 lines

We used to have a full git dir at sha1collisiondetection/.git and all the UX issues that entailed (e.g. switching to an old commit without the submodule).

Now it's a stub and the actual repo is at .git/modules/sha1collisiondetection/, so we're kind of partially there.

Side note: when I was writing the tests for patch 2 in this series I noticed it was still really easy to end up with a full git dir at e.g. sha1collisiondetection/.git, if you are trying to create a new repo to use as a submodule (easily could be the case when working on a "greenfield" project and you're the original author). There is definitely a reason that I copied the (IMO) hack from the other submodule test suite using the trash directory as a remote for my new submodule. ;) I wonder whether I was just doing it wrong, or if we need some established flow (maybe with git submodule subcommand) to create a brand new submodule, not cloned from somewhere, and put its gitdir inside of .git/modules?

Quoted 3 lines

* I would think that the next (but big) logical step would be to use some combination of delta islands, upcoming sparse indexes etc. to actually share the object stores of the parent and submodule.

Interesting - I'm trying to think of reasons not to and coming up blank, but I also don't have much firsthand experience with the area of the code that looks through the object store, so what do I know?

Quoted 8 lines

Things like "git fsck" which now just punt on COMMIT would need to become smarter, but e.g. we could repack (or not, with islands) between parent and submodule.

I would think that this end goal makes more sense than the current status quo of teaching every command that needs to e.g. grep the tree to have a "--recurse-submodules". The distinction would be invisible to the likes of "git-grep".

Yeah, I see where you're going, I think. Teaching everyone --recurse-submodules or to respect the config setting (core.recurseSubmodules? whatever it is) is inherently fragile, since it relies on human reviewers to remember to chide patch authors to think of the submodules use case. Neat.

Quoted 5 lines

It would mean more complexity in e.g. "git commit", but we can imagine if you wanted a cross-submodule commit it could do those commits recursively, update parent COMMIT entries etc. (and even, optionally, push out the submodule changes). That particular thing being so ad-hoc is a *very* frequent pain point in submodule use.

Yeah, it sounds like you're describing the approach I was hoping to use for commit-with-recursion:

- Note each submodule with staged changes, as well as the superproject
- (Optional? but might be nice) Open an editor with all the commit messages separated by scissors, so you can easily refer back to or modify the submodule commit messages while writing the superproject commit message
- Generate all the submodule commits with the supplied commit-msgs
- Take the commit IDs of all the newly created commits, stage them in the superproject, and generate the superproject commit with the supplied commit-msg

Maybe the editor bit is too much, but Jonathan Nieder at least really liked that idea :) But the bit you're talking about - generating the submodules first and then staging and committing in the superproject "on the fly" - was the approach I was hoping to take.

Quoted 25 lines

But why am I talking about this here when all you're suggesting is another config level?

Well, I think (but have not carefully thought about) that this CONFIG_SCOPE_GITMODULES is probably a narrow net improvement now. If you set most options in your .git/config to you that's the same logical project, why shouldn't you get your diff setting or whatever because you cd'd to a submodule "in the same project" (from the view of the user).

But I think that for a wider "improve submodules" effort it's worth someone (and right now, that sounds like it's you) thinking about where we're going with the feature. Maybe with some technical doc identifying the most common pain points, what we propose (or could envision) doing about them.

So e.g. in this case, having per-submodule config could be a step forward, but it could also be one more step of walking in a circle.

I.e. don't think any user asked for or wanted to stitch together multiple .git directories into one linked pseudo-checkout, that's ultimately something we're exposing as an implementation detail. If we no longer expose that implementation detail, would we be stuck supporting what's ultimately a workaround feature?

Quoted 5 lines

None of that means we shouldn't have that one step forward that solves real problems today.

But I think we should think about the end goal(s) sooner than later.

Yeah, this is actually a good nudge for me. Internally we've got a big nice doc explaining all our submodule plans for the next 6-9 months - but I should probably get to sharing that with the list ;) I'd say to look for it either this Friday or next Friday.

Quoted 5 lines

E.g. in your case, do you *really* want another config level, or is it just the easiest way to get what you actually want, which is for a "git config" in the submodule dir to perhaps consider its .git/config and .git/modules/sha1collisiondetection/config as the same file for the purposes of config parsing? Sans things like the remote URLs etc.

As for this specific case, I want what is in the patch. Using a new config file doesn't feel like a compromise to me - I actually would prefer users to be able to explicitly choose shared vs. repo-specific configs, rather than for we Git devs to implicitly decide which configs are fine to share and which aren't. (I could also see having explicitly shared or non-shared configs making it easier for wrappers to leverage the Git config infrastructure, without mirroring our own "list of configs to not share to submodule" for themselves.)

This RFC is mostly here to enable shared hooks, as you might have guessed - but even with hooks, it's easy to imagine wanting a blend of inherited vs. per-repo hooks. For example, I want to inherit a hook to create a Gerrit Change-Id footer in my superproject and all my submodules, definitely - but if my superproject is written in C and includes a submodule which is in, I dunno, Rust or Zig or Perl or whatever people are writing these days, I definitely don't want to try and run my C linter from my superproject on my 15 Rust submodules - and I definitely don't want to disable it in each one.

- Emily

#11 lore raw

Re: Future structure of submodules and .git/, .git/modules/* organization

In reply to Ævar Arnfjörð Bjarmason
Quoted 10 lines · Ævar Arnfjörð Bjarmason

I very much wish that we could eventually make the use of submodules totally transparent, i.e. (taking the example of git.git):

* You clone, and we just get objects from
https://github.com/cr-marcstevens/sha1collisiondetection.git too

* The fact that we have:

   160000 commit 855827c583bc30645ba427885caa40c5b81764d2  sha1collisiondetection

...

I am afraid that the story is not that simple (I wish it were).

There are at last two opposing ways submodules are to be used. The original motivation was to borrow an external project as part of your project, and the way we use SHA1DC is fairly close to it (but not quite). In the context of such a usage

	git commit -m "message" --recurse-submodules

would often not be an appropriate operation. A message that is suitable in the context of the entire project would not be in the context of the project that is bound to your project as a submodule, and for your changes to be reusable by the folks who own the borrowed project to make sense, your change should be defensible on its own, "it helps this project that happens to use you as a submodule" by itself is not all that convincing.

The other way submodule often gets used is to artificially split a logically single project into many subdirectories and make them into separate repositories, the top-level project binding them as submodules. An submodule in such an arrangement may not even make sense as a standalone project---this pattern was only brought into usage because without the more recent inventions like lazy/partial clones and sparse checkouts, large projects did not fit within a single repository.

With such an arrangement, of course it makes perfect sense for things like

	git commit -m "message" --recurse-submodules
	git grep --recurse-submodules

to work as if you are working inside a single repository, by definition. You are splitting a logically single project into multiple submodules as a workaround, and then still wanting to treat them as a single project, after all.

Supporting those who want to use "collection of submodules as if it were a single monolithic project" well is a worthy goal, but I do not think it is healthy to assume that is the only use and forget about use cases that would benefit from a clear boundary at submodules (e.g. not sharing commit log message, a change at the toplevel project may consist of multiple changes at the submoudle level, etc.).

Thanks.

#12 lore raw

v2 cover share a config between submodule and superproject

In reply to Emily Shaffer

Hi all,

With the second iteration, I bothered to make the tests pass (oops) - and I'm actually fairly unhappy with what I found. Patches 2 and 3 of this iteration clean up tests which were (accidentally) explicitly checking that no child processes were invoked besides the ones they expected, by way of strictly grepping traces (by test_cmp or by line counting). I don't like those tests as they were - to me, they felt like the brittle kind of white-box test - but I also got a stronger feeling that adding an additional child process or two during every Git invocation is a bad idea. I also saw a pretty significant increase in test run time:

All tests successful.
Files=927, Tests=24148, 693 wallclock secs ( 8.71 usr 2.05 sys + 3254.41 cusr 1571.78 csys = 4836.95 CPU)
Result: PASS

real    11m33.029s
user    54m23.187s
sys     26m13.857s

vs before:

All tests successful.
Files=926, Tests=24138, 144 wallclock secs ( 8.14 usr 2.03 sys + 882.29 cusr 535.61 csys = 1428.07 CPU)
Result: PASS

real    2m24.116s
user    14m50.499s
sys     8m57.649s

And that's on a Linux machine; as I understand it, invoking child processes can be even more painful on other operating systems.

If we could be assured that this extra step (finding the parent's gitdir and checking that config) was only running when we know we're in a submodule, I'd be less worried, I think. And there are a couple other pieces in the big picture submodule plan I sent[1] around which would require repos to answer "am I a submodule?"

So I think this series may need to be shelved pending an answer to that question - whether we *should* let submodules know they are submodules[2] to turn on more behavior, and if so, how we should implement that.

- Emily

[1] https://lore.kernel.org/git/[email protected]
[2] https://lore.kernel.org/git/[email protected]

Emily Shaffer (4):
config: rename "submodule" scope to "gitmodules"
t1510-repo-setup: don't use exact matching on traces t7006-pager.sh: more lenient trace checking
config: add 'config.superproject' file

 Documentation/git-config.txt   |  21 +++++-
 builtin/config.c               |   9 ++-
 config.c                       |  28 +++++++-
 config.h                       |   5 +-
 submodule-config.c             |   2 +-
 submodule.c                    |  29 +++++++++
 submodule.h                    |   8 +++
 t/t1311-superproject-config.sh | 116 +++++++++++++++++++++++++++++++++
 t/t1510-repo-setup.sh          |   2 +-
 t/t7006-pager.sh               |  24 +++++--
 10 files changed, 230 insertions(+), 14 deletions(-)
 create mode 100755 t/t1311-superproject-config.sh
Signature · 1 line
2.31.1.498.g6c1eba8ee3d-goog
#13 lore raw

RFC PATCH v2 1/4 config: rename "submodule" scope to "gitmodules"

To prepare for the addition of a new config scope which only applies to submodule projects, disambiguate "CONFIG_SCOPE_SUBMODULES". This scope refers to configs gathered from the .gitmodules file in the superproject, so just call it "gitmodules."

---
 config.c           | 4 ++--
 config.h           | 2 +-
 submodule-config.c | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)
config.c +2 -2
index 870d9534de..c8426ef3f3 100644
@@ -3499,8 +3499,8 @@ const char *config_scope_name(enum config_scope scope)
 		return "worktree";
 	case CONFIG_SCOPE_COMMAND:
 		return "command";
-	case CONFIG_SCOPE_SUBMODULE:
-		return "submodule";
+	case CONFIG_SCOPE_GITMODULES:
+		return "gitmodules";
 	default:
 		return "unknown";
 	}
config.h +1 -1
index 19a9adbaa9..535f5517b8 100644
@@ -42,7 +42,7 @@ enum config_scope {
 	CONFIG_SCOPE_LOCAL,
 	CONFIG_SCOPE_WORKTREE,
 	CONFIG_SCOPE_COMMAND,
-	CONFIG_SCOPE_SUBMODULE,
+	CONFIG_SCOPE_GITMODULES,
 };
 const char *config_scope_name(enum config_scope scope);
 
submodule-config.c +1 -1
index f502505566..0e435e6fdd 100644
@@ -637,7 +637,7 @@ static void config_from_gitmodules(config_fn_t fn, struct repository *repo, void
 {
 	if (repo->worktree) {
 		struct git_config_source config_source = {
-			0, .scope = CONFIG_SCOPE_SUBMODULE
+			0, .scope = CONFIG_SCOPE_GITMODULES
 		};
 		const struct config_options opts = { 0 };
 		struct object_id oid;
Signature · 1 line
2.31.1.498.g6c1eba8ee3d-goog
#14 lore raw

Re: [RFC PATCH v2 1/4] config: rename "submodule" scope to "gitmodules"

Hi Emily

Quoted 4 lines · Emily Shaffer

To prepare for the addition of a new config scope which only applies to submodule projects, disambiguate "CONFIG_SCOPE_SUBMODULES". This scope refers to configs gathered from the .gitmodules file in the superproject, so just call it "gitmodules."

Am I right in thinking that this changes the output of git config --show-scope? If so I'm not sure it's a good idea as it would break any scripts that are parsing that output

Best Wishes

Phillip

Quoted 49 lines
---
  config.c           | 4 ++--
  config.h           | 2 +-
  submodule-config.c | 2 +-
  3 files changed, 4 insertions(+), 4 deletions(-)
config.c +2 -2
index 870d9534de..c8426ef3f3 100644
@@ -3499,8 +3499,8 @@ const char *config_scope_name(enum config_scope scope)
  		return "worktree";
  	case CONFIG_SCOPE_COMMAND:
  		return "command";
-	case CONFIG_SCOPE_SUBMODULE:
-		return "submodule";
+	case CONFIG_SCOPE_GITMODULES:
+		return "gitmodules";
  	default:
  		return "unknown";
  	}
config.h +1 -1
index 19a9adbaa9..535f5517b8 100644
@@ -42,7 +42,7 @@ enum config_scope {
  	CONFIG_SCOPE_LOCAL,
  	CONFIG_SCOPE_WORKTREE,
  	CONFIG_SCOPE_COMMAND,
-	CONFIG_SCOPE_SUBMODULE,
+	CONFIG_SCOPE_GITMODULES,
  };
  const char *config_scope_name(enum config_scope scope);
 
submodule-config.c +1 -1
index f502505566..0e435e6fdd 100644
@@ -637,7 +637,7 @@ static void config_from_gitmodules(config_fn_t fn, struct repository *repo, void
  {
  	if (repo->worktree) {
  		struct git_config_source config_source = {
-			0, .scope = CONFIG_SCOPE_SUBMODULE
+			0, .scope = CONFIG_SCOPE_GITMODULES
  		};
  		const struct config_options opts = { 0 };
  		struct object_id oid;
#15 lore raw

RFC PATCH v2 2/4 t1510-repo-setup: don't use exact matching on traces

In reply to Emily Shaffer

Tests which interrogate the exact underlying behavior of the code under test, instead of checking for the presence of desired side effects or calls, are a known testing antipattern. They are flaky as they need to be updated every time the underlying implementation changes.

By using 'grep --fixed-strings --file <expect>' instead, we can check for the positive presence of lines we are sure should be happening, and ignore any additional things which may be happening around us (for example, additional child processes which are occurring unrelated to the code under test).

---
 t/t1510-repo-setup.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
t/t1510-repo-setup.sh +1 -1
index bbfe05b8e4..8bd4f54d03 100755
@@ -63,7 +63,7 @@ test_repo () {
 		rm -f trace &&
 		GIT_TRACE_SETUP="$(pwd)/trace" git symbolic-ref HEAD >/dev/null &&
 		grep '^setup: ' trace >result &&
-		test_cmp expected result
+		grep -Ff expected result
 	)
 }
 
Signature · 1 line
2.31.1.498.g6c1eba8ee3d-goog
#16 lore raw

Re: [RFC PATCH v2 2/4] t1510-repo-setup: don't use exact matching on traces

Hi Emily

Quoted 21 lines · Emily Shaffer

Tests which interrogate the exact underlying behavior of the code under test, instead of checking for the presence of desired side effects or calls, are a known testing antipattern. They are flaky as they need to be updated every time the underlying implementation changes.

By using 'grep --fixed-strings --file <expect>' instead, we can check for the positive presence of lines we are sure should be happening, and ignore any additional things which may be happening around us (for example, additional child processes which are occurring unrelated to the code under test).

---
  t/t1510-repo-setup.sh | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
t/t1510-repo-setup.sh +0 -0
index bbfe05b8e4..8bd4f54d03 100755

t/t1510-repo-setup.sh test_repo () { · line 66

Quoted 5 lines
t/t1510-repo-setup.sh +1 -1
@@ -63,7 +63,7 @@ test_repo () {
  		rm -f trace &&
  		GIT_TRACE_SETUP="$(pwd)/trace" git symbolic-ref HEAD >/dev/null &&
  		grep '^setup: ' trace >result &&
-		test_cmp expected result
+		grep -Ff expected result

If there is more than one line in expected (it's not clear from the limited context if there is) then grep will succeed if any of the lines match rather than requiring all the lines to match as test_cmp does.

Best wishes

Phillip

Quoted 4 lines
t/t1510-repo-setup.sh +0 -0
  	)
  }
 
#17 lore raw

RFC PATCH v2 3/4 t7006-pager.sh: more lenient trace checking

In reply to Emily Shaffer

A number of tests in t7006-pager.sh are, as a side effect, checking that 'git log' does not invoke any child processes besides the pager. There is no reason for that guarantee, and it is not explicitly the purpose of these tests, so let's make the checking more intelligent and flexible.

child_start and child_exit events share a child ID - using that child ID, we can try to disambiguate which child_exit belongs to which child_start and limit our validation to only the pager's child process.

---
 t/t7006-pager.sh | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)
t/t7006-pager.sh +18 -6
index 0e7cf75435..ac2d91d56b 100755
@@ -676,7 +676,9 @@ test_expect_success TTY 'git returns SIGPIPE on early pager exit' '
 		test_terminal git log
 	fi &&
 
-	grep child_exit trace.normal >child-exits &&
+	PAGER_CHILD_ID=$(grep pager-used trace.normal | \
+			 sed -n "s/child_start\[\([0-9]\+\)\].*/\1/p") &&
+	grep -F "child_exit["$PAGER_CHILD_ID"]" trace.normal >child-exits &&
 	test_line_count = 1 child-exits &&
 	grep " code:0 " child-exits &&
 	test_path_is_file pager-used
@@ -697,7 +699,9 @@ test_expect_success TTY 'git returns SIGPIPE on early pager non-zero exit' '
 		test_terminal git log
 	fi &&
 
-	grep child_exit trace.normal >child-exits &&
+	PAGER_CHILD_ID=$(grep pager-used trace.normal | \
+			 sed -n "s/child_start\[\([0-9]\+\)\].*/\1/p") &&
+	grep -F "child_exit["$PAGER_CHILD_ID"]" trace.normal >child-exits &&
 	test_line_count = 1 child-exits &&
 	grep " code:1 " child-exits &&
 	test_path_is_file pager-used
@@ -718,7 +722,9 @@ test_expect_success TTY 'git discards pager non-zero exit without SIGPIPE' '
 		test_terminal git log
 	fi &&
 
-	grep child_exit trace.normal >child-exits &&
+	PAGER_CHILD_ID=$(grep pager-used trace.normal | \
+			 sed -n "s/child_start\[\([0-9]\+\)\].*/\1/p") &&
+	grep -F "child_exit["$PAGER_CHILD_ID"]" trace.normal >child-exits &&
 	test_line_count = 1 child-exits &&
 	grep " code:1 " child-exits &&
 	test_path_is_file pager-used
@@ -739,7 +745,9 @@ test_expect_success TTY 'git discards nonexisting pager without SIGPIPE' '
 		test_terminal git log
 	fi &&
 
-	grep child_exit trace.normal >child-exits &&
+	PAGER_CHILD_ID=$(grep does-not-exist trace.normal | \
+			 sed -n "s/child_start\[\([0-9]\+\)\].*/\1/p") &&
+	grep -F "child_exit["$PAGER_CHILD_ID"]" trace.normal >child-exits &&
 	test_line_count = 1 child-exits &&
 	grep " code:127 " child-exits &&
 	test_path_is_file pager-used
@@ -760,7 +768,9 @@ test_expect_success TTY 'git attempts to page to nonexisting pager command, gets
 		test_terminal git log
 	fi &&
 
-	grep child_exit trace.normal >child-exits &&
+	PAGER_CHILD_ID=$(grep does-not-exist trace.normal | \
+			 sed -n "s/child_start\[\([0-9]\+\)\].*/\1/p") &&
+	grep -F "child_exit["$PAGER_CHILD_ID"]" trace.normal >child-exits &&
 	test_line_count = 1 child-exits &&
 	grep " code:-1 " child-exits
 '
@@ -780,7 +790,9 @@ test_expect_success TTY 'git returns SIGPIPE on propagated signals from pager' '
 		test_terminal git log
 	fi &&
 
-	grep child_exit trace.normal >child-exits &&
+	PAGER_CHILD_ID=$(grep pager-used trace.normal | \
+			 sed -n "s/child_start\[\([0-9]\+\)\].*/\1/p") &&
+	grep -F "child_exit["$PAGER_CHILD_ID"]" trace.normal >child-exits &&
 	test_line_count = 1 child-exits &&
 	grep " code:143 " child-exits &&
 	test_path_is_file pager-used
Signature · 1 line
2.31.1.498.g6c1eba8ee3d-goog
#18 lore raw

Re: [RFC PATCH v2 3/4] t7006-pager.sh: more lenient trace checking

Hi Emily

Quoted 8 lines · Emily Shaffer

A number of tests in t7006-pager.sh are, as a side effect, checking that 'git log' does not invoke any child processes besides the pager. There is no reason for that guarantee, and it is not explicitly the purpose of these tests, so let's make the checking more intelligent and flexible.

child_start and child_exit events share a child ID - using that child ID, we can try to disambiguate which child_exit belongs to which child_start and limit our validation to only the pager's child process.

I've not looked at this test file properly but if we want to check that the pager is invoked can we use a script that writes a file when it has been run as the pager and just cleanup that file at the end of each test case?

Quoted 10 lines
---
  t/t7006-pager.sh | 24 ++++++++++++++++++------
  1 file changed, 18 insertions(+), 6 deletions(-)
t/t7006-pager.sh +0 -0
index 0e7cf75435..ac2d91d56b 100755

t/t7006-pager.sh test_expect_success TTY 'git returns SIGPIPE on early pager exit' ' · line 680

Quoted 6 lines
t/t7006-pager.sh +2 -1
@@ -676,7 +676,9 @@ test_expect_success TTY 'git returns SIGPIPE on early pager exit' '
  		test_terminal git log
  	fi &&
 
-	grep child_exit trace.normal >child-exits &&
+	PAGER_CHILD_ID=$(grep pager-used trace.normal | \
+			 sed -n "s/child_start\[\([0-9]\+\)\].*/\1/p") &&

If you want to save a process you could use sed to do the job of the grep command. I think this should do it

sed -n -e "/child_exit/ {" -e "s/child_start\[\([0-9]\+\)\].*/\1/p" -e "}"

t/t7006-pager.sh test_expect_success TTY 'git returns SIGPIPE on early pager exit' ' · line 681

Quoted 1 line
t/t7006-pager.sh +1 -0
+	grep -F "child_exit["$PAGER_CHILD_ID"]" trace.normal >child-exits &&

Why is $PAGER_CHILD_ID unquoted?

Best Wishes

Phillip

Quoted 59 lines
t/t7006-pager.sh +15 -5
  	test_line_count = 1 child-exits &&
  	grep " code:0 " child-exits &&
  	test_path_is_file pager-used
@@ -697,7 +699,9 @@ test_expect_success TTY 'git returns SIGPIPE on early pager non-zero exit' '
  		test_terminal git log
  	fi &&
 
-	grep child_exit trace.normal >child-exits &&
+	PAGER_CHILD_ID=$(grep pager-used trace.normal | \
+			 sed -n "s/child_start\[\([0-9]\+\)\].*/\1/p") &&
+	grep -F "child_exit["$PAGER_CHILD_ID"]" trace.normal >child-exits &&
  	test_line_count = 1 child-exits &&
  	grep " code:1 " child-exits &&
  	test_path_is_file pager-used
@@ -718,7 +722,9 @@ test_expect_success TTY 'git discards pager non-zero exit without SIGPIPE' '
  		test_terminal git log
  	fi &&
 
-	grep child_exit trace.normal >child-exits &&
+	PAGER_CHILD_ID=$(grep pager-used trace.normal | \
+			 sed -n "s/child_start\[\([0-9]\+\)\].*/\1/p") &&
+	grep -F "child_exit["$PAGER_CHILD_ID"]" trace.normal >child-exits &&
  	test_line_count = 1 child-exits &&
  	grep " code:1 " child-exits &&
  	test_path_is_file pager-used
@@ -739,7 +745,9 @@ test_expect_success TTY 'git discards nonexisting pager without SIGPIPE' '
  		test_terminal git log
  	fi &&
 
-	grep child_exit trace.normal >child-exits &&
+	PAGER_CHILD_ID=$(grep does-not-exist trace.normal | \
+			 sed -n "s/child_start\[\([0-9]\+\)\].*/\1/p") &&
+	grep -F "child_exit["$PAGER_CHILD_ID"]" trace.normal >child-exits &&
  	test_line_count = 1 child-exits &&
  	grep " code:127 " child-exits &&
  	test_path_is_file pager-used
@@ -760,7 +768,9 @@ test_expect_success TTY 'git attempts to page to nonexisting pager command, gets
  		test_terminal git log
  	fi &&
 
-	grep child_exit trace.normal >child-exits &&
+	PAGER_CHILD_ID=$(grep does-not-exist trace.normal | \
+			 sed -n "s/child_start\[\([0-9]\+\)\].*/\1/p") &&
+	grep -F "child_exit["$PAGER_CHILD_ID"]" trace.normal >child-exits &&
  	test_line_count = 1 child-exits &&
  	grep " code:-1 " child-exits
  '
@@ -780,7 +790,9 @@ test_expect_success TTY 'git returns SIGPIPE on propagated signals from pager' '
  		test_terminal git log
  	fi &&
 
-	grep child_exit trace.normal >child-exits &&
+	PAGER_CHILD_ID=$(grep pager-used trace.normal | \
+			 sed -n "s/child_start\[\([0-9]\+\)\].*/\1/p") &&
+	grep -F "child_exit["$PAGER_CHILD_ID"]" trace.normal >child-exits &&
  	test_line_count = 1 child-exits &&
  	grep " code:143 " child-exits &&
  	test_path_is_file pager-used
#19 lore raw

Re: [RFC PATCH v2 3/4] t7006-pager.sh: more lenient trace checking

Hi Emily

Quoted 16 lines · Phillip Wood

Hi Emily
[...]

Quoted 11 lines
t/t7006-pager.sh +0 -0
index 0e7cf75435..ac2d91d56b 100755
@@ -676,7 +676,9 @@ test_expect_success TTY 'git returns SIGPIPE on 

early pager exit' '
          test_terminal git log
      fi &&
-    grep child_exit trace.normal >child-exits &&
+    PAGER_CHILD_ID=$(grep pager-used trace.normal | \
+             sed -n "s/child_start\[\([0-9]\+\)\].*/\1/p") &&

If you want to save a process you could use sed to do the job of the grep command. I think this should do it

Quoted 1 line

sed -n -e "/child_exit/ {" -e "s/child_start\[\([0-9]\+\)\].*/\1/p" -e "}"

I must have been half asleep when I wrote that, there is no need for the braces and the initial match is wrong it should be

sed -n "/pager-used/s/child_start\[\([0-9]\+\)\].*/\1/p"

Best Wishes

Phillip

Quoted 68 lines
Quoted 1 line

+    grep -F "child_exit["$PAGER_CHILD_ID"]" trace.normal >child-exits &&

Why is $PAGER_CHILD_ID unquoted?

Best Wishes

Phillip

Quoted 59 lines

      test_line_count = 1 child-exits &&
      grep " code:0 " child-exits &&
      test_path_is_file pager-used

t/t7006-pager.sh +0 -0
@@ -697,7 +699,9 @@ test_expect_success TTY 'git returns SIGPIPE on 

early pager non-zero exit' '
          test_terminal git log
      fi &&
-    grep child_exit trace.normal >child-exits &&
+    PAGER_CHILD_ID=$(grep pager-used trace.normal | \
+             sed -n "s/child_start\[\([0-9]\+\)\].*/\1/p") && +    grep -F "child_exit["$PAGER_CHILD_ID"]" trace.normal >child-exits && test_line_count = 1 child-exits &&
      grep " code:1 " child-exits &&
      test_path_is_file pager-used

t/t7006-pager.sh +0 -0
@@ -718,7 +722,9 @@ test_expect_success TTY 'git discards pager 

non-zero exit without SIGPIPE' '
          test_terminal git log
      fi &&
-    grep child_exit trace.normal >child-exits &&
+    PAGER_CHILD_ID=$(grep pager-used trace.normal | \
+             sed -n "s/child_start\[\([0-9]\+\)\].*/\1/p") && +    grep -F "child_exit["$PAGER_CHILD_ID"]" trace.normal >child-exits && test_line_count = 1 child-exits &&
      grep " code:1 " child-exits &&
      test_path_is_file pager-used

t/t7006-pager.sh +0 -0
@@ -739,7 +745,9 @@ test_expect_success TTY 'git discards nonexisting 

pager without SIGPIPE' '
          test_terminal git log
      fi &&
-    grep child_exit trace.normal >child-exits &&
+    PAGER_CHILD_ID=$(grep does-not-exist trace.normal | \
+             sed -n "s/child_start\[\([0-9]\+\)\].*/\1/p") && +    grep -F "child_exit["$PAGER_CHILD_ID"]" trace.normal >child-exits && test_line_count = 1 child-exits &&
      grep " code:127 " child-exits &&
      test_path_is_file pager-used

t/t7006-pager.sh +0 -0
@@ -760,7 +768,9 @@ test_expect_success TTY 'git attempts to page to 

nonexisting pager command, gets
          test_terminal git log
      fi &&
-    grep child_exit trace.normal >child-exits &&
+    PAGER_CHILD_ID=$(grep does-not-exist trace.normal | \
+             sed -n "s/child_start\[\([0-9]\+\)\].*/\1/p") && +    grep -F "child_exit["$PAGER_CHILD_ID"]" trace.normal >child-exits && test_line_count = 1 child-exits &&
      grep " code:-1 " child-exits
  '

t/t7006-pager.sh +0 -0
@@ -780,7 +790,9 @@ test_expect_success TTY 'git returns SIGPIPE on 

propagated signals from pager' '
          test_terminal git log
      fi &&
-    grep child_exit trace.normal >child-exits &&
+    PAGER_CHILD_ID=$(grep pager-used trace.normal | \
+             sed -n "s/child_start\[\([0-9]\+\)\].*/\1/p") && +    grep -F "child_exit["$PAGER_CHILD_ID"]" trace.normal >child-exits && test_line_count = 1 child-exits &&
      grep " code:143 " child-exits &&
      test_path_is_file pager-used

#20 lore raw

RFC PATCH v2 4/4 config: add 'config.superproject' file

In reply to Emily Shaffer

Some configs, such as wrapper directives like gerrit.createChangeId, or forthcoming hook configs, should apply to a superproject as well as all its submodules. It may not be appropriate to apply them globally - for example, if the user also contributes to many projects which do not use the configs necessary for one project-with-submodules - and it may be burdensome to apply them locally to the superproject and each of its submodules. Even if the user runs 'git submodule foreach "git config --local foo.bar', if a new submodule is added later on, that config is not applied to the new submodule.

It is also inappropriate to share the entire superproject config, since some items - like remote URLs or partial-clone filters - would not apply to a submodule.

To make life easier for projects with many submodules, then, create a new "config.superproject" config scope, which is included in the config parse for the superproject as well as for all the submodules of that superproject.

For the superproject, this new config file is equally local to the local config; for the submodule, the new config file is less local than the local config. So let's include it directly before the local config during the config parse.

---

Notes:
Since v1:

- Used test_when_finished liberally in tests instead of bare cleanup steps
- Fixed some nits regarding xstrdup instead of strbuf_detach.

One thing that I thought about but did not implement: rather than finding the path to the superproject's gitdir, you could imagine gathering the config by making a call out to 'git -C ../ config' - but on second thought, it seems like that will make it harder to edit. However, if we don't want to be able to edit superproject config from a submodule, that might be okay... (This approach could make 'git config --show-origin' harder to implement, though, I think.) So I did not make any changes about that.

 Documentation/git-config.txt   |  21 +++++-
 builtin/config.c               |   9 ++-
 config.c                       |  24 +++++++
 config.h                       |   3 +
 submodule.c                    |  29 +++++++++
 submodule.h                    |   8 +++
 t/t1311-superproject-config.sh | 116 +++++++++++++++++++++++++++++++++
 7 files changed, 207 insertions(+), 3 deletions(-)
 create mode 100755 t/t1311-superproject-config.sh
Documentation/git-config.txt +19 -2
index 4b4cc5c5e8..a33136fb08 100644
@@ -48,7 +48,7 @@ unset an existing `--type` specifier with `--no-type`.
 
 When reading, the values are read from the system, global and
 repository local configuration files by default, and options
-`--system`, `--global`, `--local`, `--worktree` and
+`--system`, `--global`, `--superproject`, `--local`, `--worktree` and
 `--file <filename>` can be used to tell the command to read from only
 that location (see <<FILES>>).
 
@@ -127,6 +127,17 @@ rather than from all available files.
 +
 See also <<FILES>>.
 
+--superproject::
+	For writing options: write to the superproject's
+	`.git/config.superproject` file, even if run from a submodule of that
+	superproject.
++
+For reading options: read only from the superproject's
+`.git/config.superproject` file, even if run from a submodule of that
+superproject, rather than from all available files.
++
+See also <<FILES>>.
+
 --local::
 	For writing options: write to the repository `.git/config` file.
 	This is the default behavior.
@@ -283,7 +294,7 @@ The default is to use a pager.
 FILES
 -----
 
-If not set explicitly with `--file`, there are four files where
+If not set explicitly with `--file`, there are five files where
 'git config' will search for configuration options:
 
 $(prefix)/etc/gitconfig::
@@ -301,6 +312,12 @@ $XDG_CONFIG_HOME/git/config::
 	User-specific configuration file. Also called "global"
 	configuration file.
 
+$GIT_DIR/config.superproject::
+	When `git config` is run from a project which is a submodule of another
+	project, that superproject's $GIT_DIR will be used. Use this config file
+	to set configurations which need to be the same across a superproject
+	and all its submodules.
+
 $GIT_DIR/config::
 	Repository specific configuration file.
 
builtin/config.c +8 -1
index f71fa39b38..f38a6823c7 100644
@@ -26,7 +26,7 @@ static char key_delim = ' ';
 static char term = '\n';
 
 static int use_global_config, use_system_config, use_local_config;
-static int use_worktree_config;
+static int use_worktree_config, use_superproject_config;
 static struct git_config_source given_config_source;
 static int actions, type;
 static char *default_value;
@@ -130,6 +130,8 @@ static struct option builtin_config_options[] = {
 	OPT_GROUP(N_("Config file location")),
 	OPT_BOOL(0, "global", &use_global_config, N_("use global config file")),
 	OPT_BOOL(0, "system", &use_system_config, N_("use system config file")),
+	OPT_BOOL(0, "superproject",
+		 &use_superproject_config, N_("use superproject config file")),
 	OPT_BOOL(0, "local", &use_local_config, N_("use repository config file")),
 	OPT_BOOL(0, "worktree", &use_worktree_config, N_("use per-worktree config file")),
 	OPT_STRING('f', "file", &given_config_source.file, N_("file"), N_("use given config file")),
@@ -697,6 +699,11 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 	else if (use_system_config) {
 		given_config_source.file = git_etc_gitconfig();
 		given_config_source.scope = CONFIG_SCOPE_SYSTEM;
+	} else if (use_superproject_config) {
+		struct strbuf superproject_cfg = STRBUF_INIT;
+		git_config_superproject(&superproject_cfg, get_git_dir());
+		given_config_source.file = strbuf_detach(&superproject_cfg, NULL);
+		given_config_source.scope = CONFIG_SCOPE_SUPERPROJECT;
 	} else if (use_local_config) {
 		given_config_source.file = git_pathdup("config");
 		given_config_source.scope = CONFIG_SCOPE_LOCAL;
config.c +24 -0
index c8426ef3f3..3f62cd815b 100644
@@ -21,6 +21,7 @@
 #include "dir.h"
 #include "color.h"
 #include "refs.h"
+#include "submodule.h"
 
 struct config_source {
 	struct config_source *prev;
@@ -1838,6 +1839,15 @@ const char *git_etc_gitconfig(void)
 	return system_wide;
 }
 
+void git_config_superproject(struct strbuf *sb, const char *gitdir)
+{
+	if (!get_superproject_gitdir(sb))
+		/* not a submodule */
+		strbuf_addstr(sb, gitdir);
+
+	strbuf_addstr(sb, "/config.superproject");
+}
+
 /*
  * Parse environment variable 'k' as a boolean (in various
  * possible spellings); if missing, use the default value 'def'.
@@ -1895,6 +1905,17 @@ static int do_git_config_sequence(const struct config_options *opts,
 	if (user_config && !access_or_die(user_config, R_OK, ACCESS_EACCES_OK))
 		ret += git_config_from_file(fn, user_config, data);
 
+	current_parsing_scope = CONFIG_SCOPE_SUPERPROJECT;
+	if (opts->git_dir && !opts->ignore_superproject) {
+
+		struct strbuf superproject_gitdir = STRBUF_INIT;
+		git_config_superproject(&superproject_gitdir, opts->git_dir);
+		if (!access_or_die(superproject_gitdir.buf, R_OK, 0))
+			ret += git_config_from_file(fn, superproject_gitdir.buf, data);
+
+		strbuf_release(&superproject_gitdir);
+	}
+
 	current_parsing_scope = CONFIG_SCOPE_LOCAL;
 	if (!opts->ignore_repo && repo_config &&
 	    !access_or_die(repo_config, R_OK, 0))
@@ -2013,6 +2034,7 @@ void read_very_early_config(config_fn_t cb, void *data)
 
 	opts.respect_includes = 1;
 	opts.ignore_repo = 1;
+	opts.ignore_superproject = 1;
 	opts.ignore_worktree = 1;
 	opts.ignore_cmdline = 1;
 	opts.system_gently = 1;
@@ -3501,6 +3523,8 @@ const char *config_scope_name(enum config_scope scope)
 		return "command";
 	case CONFIG_SCOPE_GITMODULES:
 		return "gitmodules";
+	case CONFIG_SCOPE_SUPERPROJECT:
+		return "superproject";
 	default:
 		return "unknown";
 	}
config.h +3 -0
index 535f5517b8..b42e1d13eb 100644
@@ -43,6 +43,7 @@ enum config_scope {
 	CONFIG_SCOPE_WORKTREE,
 	CONFIG_SCOPE_COMMAND,
 	CONFIG_SCOPE_GITMODULES,
+	CONFIG_SCOPE_SUPERPROJECT,
 };
 const char *config_scope_name(enum config_scope scope);
 
@@ -84,6 +85,7 @@ typedef int (*config_parser_event_fn_t)(enum config_event_t type,
 struct config_options {
 	unsigned int respect_includes : 1;
 	unsigned int ignore_repo : 1;
+	unsigned int ignore_superproject : 1;
 	unsigned int ignore_worktree : 1;
 	unsigned int ignore_cmdline : 1;
 	unsigned int system_gently : 1;
@@ -319,6 +321,7 @@ int git_config_rename_section_in_file(const char *, const char *, const char *);
 int git_config_copy_section(const char *, const char *);
 int git_config_copy_section_in_file(const char *, const char *, const char *);
 const char *git_etc_gitconfig(void);
+void git_config_superproject(struct strbuf *, const char *);
 int git_env_bool(const char *, int);
 unsigned long git_env_ulong(const char *, unsigned long);
 int git_config_system(void);
submodule.c +29 -0
index 9767ba9893..92b00f8697 100644
@@ -2178,6 +2178,35 @@ void absorb_git_dir_into_superproject(const char *path,
 	}
 }
 
+int get_superproject_gitdir(struct strbuf *buf)
+{
+	struct strbuf sb = STRBUF_INIT;
+	struct child_process cp = CHILD_PROCESS_INIT;
+	int rc = 0;
+
+	/* NEEDSWORK: this call also calls out to a subprocess! */
+	rc = get_superproject_working_tree(&sb);
+	strbuf_release(&sb);
+
+	if (!rc)
+		return rc;
+
+	prepare_submodule_repo_env_no_git_dir(&cp.env_array);
+
+	strvec_pushl(&cp.args, "-C", "..", "rev-parse", "--absolute-git-dir", NULL);
+	cp.git_cmd = 1;
+
+	rc = capture_command(&cp, buf, 0);
+	strbuf_trim_trailing_newline(buf);
+
+	/* leave buf empty if we didn't have a superproject gitdir */
+	if (rc)
+		strbuf_reset(buf);
+
+	/* rc reflects the exit code of the rev-parse; invert into a bool */
+	return !rc;
+}
+
 int get_superproject_working_tree(struct strbuf *buf)
 {
 	struct child_process cp = CHILD_PROCESS_INIT;
submodule.h +8 -0
index 4ac6e31cf1..cc89d5e3fa 100644
@@ -149,6 +149,14 @@ void prepare_submodule_repo_env(struct strvec *out);
 void absorb_git_dir_into_superproject(const char *path,
 				      unsigned flags);
 
+/*
+ * Return the gitdir of the superproject, which this project is a submodule of.
+ * If this repository is a submodule, return 1 and fill buf with the absolute
+ * path to the superproject's gitdir. If this repository is not a submodule of
+ * another repository, return 0 and leave buf untouched.
+ */
+int get_superproject_gitdir(struct strbuf *buf);
+
 /*
  * Return the absolute path of the working tree of the superproject, which this
  * project is a submodule of. If this repository is not a submodule of
t/t1311-superproject-config.sh new file +116 -0

116 lines not on this page · raw message

Signature · 1 line
2.31.1.498.g6c1eba8ee3d-goog
#21 lore raw

Re: [RFC PATCH v2 4/4] config: add 'config.superproject' file

Hi Emily,

Quoted 33 lines · Emily Shaffer

Some configs, such as wrapper directives like gerrit.createChangeId, or forthcoming hook configs, should apply to a superproject as well as all its submodules. It may not be appropriate to apply them globally - for example, if the user also contributes to many projects which do not use the configs necessary for one project-with-submodules - and it may be burdensome to apply them locally to the superproject and each of its submodules. Even if the user runs 'git submodule foreach "git config --local foo.bar', if a new submodule is added later on, that config is not applied to the new submodule.

It is also inappropriate to share the entire superproject config, since some items - like remote URLs or partial-clone filters - would not apply to a submodule.

To make life easier for projects with many submodules, then, create a new "config.superproject" config scope, which is included in the config parse for the superproject as well as for all the submodules of that superproject.

For the superproject, this new config file is equally local to the local config; for the submodule, the new config file is less local than the local config. So let's include it directly before the local config during the config parse.

---

Notes:
Since v1:

- Used test_when_finished liberally in tests instead of bare cleanup steps
- Fixed some nits regarding xstrdup instead of strbuf_detach.

Quoted 7 lines

One thing that I thought about but did not implement: rather than finding the path to the superproject's gitdir, you could imagine gathering the config by making a call out to 'git -C ../ config' - but on second thought, it seems like that will make it harder to edit. However, if we don't want to be able to edit superproject config from a submodule, that might be okay... (This approach could make 'git config --show-origin' harder to implement, though, I think.) So I did not make any changes about that.

Hmm. Have you thought about worktrees of subprojects that happen to be outside the superproject's directory tree?

I also wonder whether it is necessary to change Git at all, as a well-crafted [includeIf "gitdir:/path/to/superproject/**"] should do the trick, but without complicating the config code even further.

Ciao,
Johannes

#22 lore raw

v3 cover share a config between submodule and superproject

In reply to Emily Shaffer

Now based on es/superproject-aware-submodules.

By using the cached path to the superproject, I was able to drop the extra subprocess(es) to find out the superproject's gitdir. That means this change should now be relatively low overhead, and we can discuss things like naming, recursion, etc. to our hearts' content.

Since v2 of this series, I have rebased the series onto es/superproject-aware-submodules and made appropriate changes, including dropping v2 patches 2 and 3. However, I think this solution answers a handful of the comments on v2:

- dscho asked about submodules who live outside of the superproject's tree; assuming the submodule was created in one of the ways covered by superproject-aware-submodules, it should be a nonissue now.
- Phillip had comments on patches 2 and 3; they're not necessary to make the tests pass anymore, because there is no longer an additional process cluttering up the traces as a result of this series.

I did not address Phillip's comment about breaking scripts by changing the config scope name, but I think it's probably valid, so maybe patch 1 also should go (or be replaced by a robust comment).

I'm hoping this can also serve as a nice example of what
https://lore.kernel.org/git/[email protected]
can enable.

Happy CI: https://github.com/nasamuffin/git/actions/runs/951382014

Thanks,
- Emily

Emily Shaffer (2):
config: rename "submodule" scope to "gitmodules"
config: add 'config.superproject' file

 Documentation/git-config.txt   |  21 +++++-
 builtin/config.c               |   9 ++-
 config.c                       |  32 ++++++++-
 config.h                       |   5 +-
 submodule-config.c             |   2 +-
 t/t1311-superproject-config.sh | 116 +++++++++++++++++++++++++++++++++
 6 files changed, 178 insertions(+), 7 deletions(-)
 create mode 100755 t/t1311-superproject-config.sh

Range-diff against v2:

1:  785f961f4c = 1:  cddd53e33a config: rename "submodule" scope to "gitmodules"
2:  b4d853e261 ! 2:  3eaca59b65 config: add 'config.superproject' file
    @@ builtin/config.c: static struct option builtin_config_options[] = {
      	OPT_STRING('f', "file", &given_config_source.file, N_("file"), N_("use given config file")),
     @@ builtin/config.c: int cmd_config(int argc, const char **argv, const char *prefix)
      	else if (use_system_config) {
    - 		given_config_source.file = git_etc_gitconfig();
    + 		given_config_source.file = git_system_config();
      		given_config_source.scope = CONFIG_SCOPE_SYSTEM;
     +	} else if (use_superproject_config) {
     +		struct strbuf superproject_cfg = STRBUF_INIT;
    @@ config.c
      
      struct config_source {
      	struct config_source *prev;
    -@@ config.c: const char *git_etc_gitconfig(void)
    - 	return system_wide;
    +@@ config.c: void git_global_config(char **user_out, char **xdg_out)
    + 	*xdg_out = xdg_config;
      }
      
     +void git_config_superproject(struct strbuf *sb, const char *gitdir)
     +{
    -+	if (!get_superproject_gitdir(sb))
    -+		/* not a submodule */
    ++	const char *sp_gitdir;
    ++	if (git_config_get_string_tmp("submodule.superprojectGitDir", &sp_gitdir))
    ++		/* probably not a submodule */
     +		strbuf_addstr(sb, gitdir);
    ++	else
    ++		/* definitely a submodule */
    ++		strbuf_addstr(sb, sp_gitdir);
     +
     +	strbuf_addstr(sb, "/config.superproject");
     +}
    @@ config.h: typedef int (*config_parser_event_fn_t)(enum config_event_t type,
      	unsigned int ignore_worktree : 1;
      	unsigned int ignore_cmdline : 1;
      	unsigned int system_gently : 1;
    -@@ config.h: int git_config_rename_section_in_file(const char *, const char *, const char *);
    +@@ config.h: int git_config_rename_section(const char *, const char *);
    + int git_config_rename_section_in_file(const char *, const char *, const char *);
      int git_config_copy_section(const char *, const char *);
      int git_config_copy_section_in_file(const char *, const char *, const char *);
    - const char *git_etc_gitconfig(void);
     +void git_config_superproject(struct strbuf *, const char *);
      int git_env_bool(const char *, int);
      unsigned long git_env_ulong(const char *, unsigned long);
      int git_config_system(void);
     
    - ## submodule.c ##
    -@@ submodule.c: void absorb_git_dir_into_superproject(const char *path,
    - 	}
    - }
    - 
    -+int get_superproject_gitdir(struct strbuf *buf)
    -+{
    -+	struct strbuf sb = STRBUF_INIT;
    -+	struct child_process cp = CHILD_PROCESS_INIT;
    -+	int rc = 0;
    -+
    -+	/* NEEDSWORK: this call also calls out to a subprocess! */
    -+	rc = get_superproject_working_tree(&sb);
    -+	strbuf_release(&sb);
    -+
    -+	if (!rc)
    -+		return rc;
    -+
    -+	prepare_submodule_repo_env_no_git_dir(&cp.env_array);
    -+
    -+	strvec_pushl(&cp.args, "-C", "..", "rev-parse", "--absolute-git-dir", NULL);
    -+	cp.git_cmd = 1;
    -+
    -+	rc = capture_command(&cp, buf, 0);
    -+	strbuf_trim_trailing_newline(buf);
    -+
    -+	/* leave buf empty if we didn't have a superproject gitdir */
    -+	if (rc)
    -+		strbuf_reset(buf);
    -+
    -+	/* rc reflects the exit code of the rev-parse; invert into a bool */
    -+	return !rc;
    -+}
    -+
    - int get_superproject_working_tree(struct strbuf *buf)
    - {
    - 	struct child_process cp = CHILD_PROCESS_INIT;
    -
    - ## submodule.h ##
    -@@ submodule.h: void prepare_submodule_repo_env(struct strvec *out);
    - void absorb_git_dir_into_superproject(const char *path,
    - 				      unsigned flags);
    - 
    -+/*
    -+ * Return the gitdir of the superproject, which this project is a submodule of.
    -+ * If this repository is a submodule, return 1 and fill buf with the absolute
    -+ * path to the superproject's gitdir. If this repository is not a submodule of
    -+ * another repository, return 0 and leave buf untouched.
    -+ */
    -+int get_superproject_gitdir(struct strbuf *buf);
    -+
    - /*
    -  * Return the absolute path of the working tree of the superproject, which this
    -  * project is a submodule of. If this repository is not a submodule of
    -
      ## t/t1311-superproject-config.sh (new) ##
     @@
     +#!/bin/sh
Signature · 1 line
2.32.0.288.g62a8d224e6-goog
#23 lore raw

PATCH v3 1/2 config: rename "submodule" scope to "gitmodules"

To prepare for the addition of a new config scope which only applies to submodule projects, disambiguate "CONFIG_SCOPE_SUBMODULES". This scope refers to configs gathered from the .gitmodules file in the superproject, so just call it "gitmodules."

---
 config.c           | 4 ++--
 config.h           | 2 +-
 submodule-config.c | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)
config.c +2 -2
index f9c400ad30..a5fc4cacd5 100644
@@ -3516,8 +3516,8 @@ const char *config_scope_name(enum config_scope scope)
 		return "worktree";
 	case CONFIG_SCOPE_COMMAND:
 		return "command";
-	case CONFIG_SCOPE_SUBMODULE:
-		return "submodule";
+	case CONFIG_SCOPE_GITMODULES:
+		return "gitmodules";
 	default:
 		return "unknown";
 	}
config.h +1 -1
index 9038538ffd..5faa72f3f5 100644
@@ -42,7 +42,7 @@ enum config_scope {
 	CONFIG_SCOPE_LOCAL,
 	CONFIG_SCOPE_WORKTREE,
 	CONFIG_SCOPE_COMMAND,
-	CONFIG_SCOPE_SUBMODULE,
+	CONFIG_SCOPE_GITMODULES,
 };
 const char *config_scope_name(enum config_scope scope);
 
submodule-config.c +1 -1
index 2026120fb3..ed8e671d17 100644
@@ -637,7 +637,7 @@ static void config_from_gitmodules(config_fn_t fn, struct repository *repo, void
 {
 	if (repo->worktree) {
 		struct git_config_source config_source = {
-			0, .scope = CONFIG_SCOPE_SUBMODULE
+			0, .scope = CONFIG_SCOPE_GITMODULES
 		};
 		const struct config_options opts = { 0 };
 		struct object_id oid;
Signature · 1 line
2.32.0.288.g62a8d224e6-goog
#24 lore raw

PATCH v3 2/2 config: add 'config.superproject' file

In reply to Emily Shaffer

Some configs, such as wrapper directives like gerrit.createChangeId, or forthcoming hook configs, should apply to a superproject as well as all its submodules. It may not be appropriate to apply them globally - for example, if the user also contributes to many projects which do not use the configs necessary for one project-with-submodules - and it may be burdensome to apply them locally to the superproject and each of its submodules. Even if the user runs 'git submodule foreach "git config --local foo.bar', if a new submodule is added later on, that config is not applied to the new submodule.

It is also inappropriate to share the entire superproject config, since some items - like remote URLs or partial-clone filters - would not apply to a submodule.

To make life easier for projects with many submodules, then, create a new "config.superproject" config scope, which is included in the config parse for the superproject as well as for all the submodules of that superproject.

For the superproject, this new config file is equally local to the local config; for the submodule, the new config file is less local than the local config. So let's include it directly before the local config during the config parse.

---

Notes:
Since v1:

- Used test_when_finished liberally in tests instead of bare cleanup steps
- Fixed some nits regarding xstrdup instead of strbuf_detach.

One thing that I thought about but did not implement: rather than finding the path to the superproject's gitdir, you could imagine gathering the config by making a call out to 'git -C ../ config' - but on second thought, it seems like that will make it harder to edit. However, if we don't want to be able to edit superproject config from a submodule, that might be okay... (This approach could make 'git config --show-origin' harder to implement, though, I think.) So I did not make any changes about that.

 Documentation/git-config.txt   |  21 +++++-
 builtin/config.c               |   9 ++-
 config.c                       |  28 ++++++++
 config.h                       |   3 +
 t/t1311-superproject-config.sh | 116 +++++++++++++++++++++++++++++++++
 5 files changed, 174 insertions(+), 3 deletions(-)
 create mode 100755 t/t1311-superproject-config.sh
Documentation/git-config.txt +19 -2
index 5cddadafd2..312deea21d 100644
@@ -48,7 +48,7 @@ unset an existing `--type` specifier with `--no-type`.
 
 When reading, the values are read from the system, global and
 repository local configuration files by default, and options
-`--system`, `--global`, `--local`, `--worktree` and
+`--system`, `--global`, `--superproject`, `--local`, `--worktree` and
 `--file <filename>` can be used to tell the command to read from only
 that location (see <<FILES>>).
 
@@ -127,6 +127,17 @@ rather than from all available files.
 +
 See also <<FILES>>.
 
+--superproject::
+	For writing options: write to the superproject's
+	`.git/config.superproject` file, even if run from a submodule of that
+	superproject.
++
+For reading options: read only from the superproject's
+`.git/config.superproject` file, even if run from a submodule of that
+superproject, rather than from all available files.
++
+See also <<FILES>>.
+
 --local::
 	For writing options: write to the repository `.git/config` file.
 	This is the default behavior.
@@ -283,7 +294,7 @@ The default is to use a pager.
 FILES
 -----
 
-If not set explicitly with `--file`, there are four files where
+If not set explicitly with `--file`, there are five files where
 'git config' will search for configuration options:
 
 $(prefix)/etc/gitconfig::
@@ -301,6 +312,12 @@ $XDG_CONFIG_HOME/git/config::
 	User-specific configuration file. Also called "global"
 	configuration file.
 
+$GIT_DIR/config.superproject::
+	When `git config` is run from a project which is a submodule of another
+	project, that superproject's $GIT_DIR will be used. Use this config file
+	to set configurations which need to be the same across a superproject
+	and all its submodules.
+
 $GIT_DIR/config::
 	Repository specific configuration file.
 
builtin/config.c +8 -1
index 865fddd6ce..83376e0916 100644
@@ -26,7 +26,7 @@ static char key_delim = ' ';
 static char term = '\n';
 
 static int use_global_config, use_system_config, use_local_config;
-static int use_worktree_config;
+static int use_worktree_config, use_superproject_config;
 static struct git_config_source given_config_source;
 static int actions, type;
 static char *default_value;
@@ -130,6 +130,8 @@ static struct option builtin_config_options[] = {
 	OPT_GROUP(N_("Config file location")),
 	OPT_BOOL(0, "global", &use_global_config, N_("use global config file")),
 	OPT_BOOL(0, "system", &use_system_config, N_("use system config file")),
+	OPT_BOOL(0, "superproject",
+		 &use_superproject_config, N_("use superproject config file")),
 	OPT_BOOL(0, "local", &use_local_config, N_("use repository config file")),
 	OPT_BOOL(0, "worktree", &use_worktree_config, N_("use per-worktree config file")),
 	OPT_STRING('f', "file", &given_config_source.file, N_("file"), N_("use given config file")),
@@ -697,6 +699,11 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 	else if (use_system_config) {
 		given_config_source.file = git_system_config();
 		given_config_source.scope = CONFIG_SCOPE_SYSTEM;
+	} else if (use_superproject_config) {
+		struct strbuf superproject_cfg = STRBUF_INIT;
+		git_config_superproject(&superproject_cfg, get_git_dir());
+		given_config_source.file = strbuf_detach(&superproject_cfg, NULL);
+		given_config_source.scope = CONFIG_SCOPE_SUPERPROJECT;
 	} else if (use_local_config) {
 		given_config_source.file = git_pathdup("config");
 		given_config_source.scope = CONFIG_SCOPE_LOCAL;
config.c +28 -0
index a5fc4cacd5..97d63a825c 100644
@@ -21,6 +21,7 @@
 #include "dir.h"
 #include "color.h"
 #include "refs.h"
+#include "submodule.h"
 
 struct config_source {
 	struct config_source *prev;
@@ -1852,6 +1853,19 @@ void git_global_config(char **user_out, char **xdg_out)
 	*xdg_out = xdg_config;
 }
 
+void git_config_superproject(struct strbuf *sb, const char *gitdir)
+{
+	const char *sp_gitdir;
+	if (git_config_get_string_tmp("submodule.superprojectGitDir", &sp_gitdir))
+		/* probably not a submodule */
+		strbuf_addstr(sb, gitdir);
+	else
+		/* definitely a submodule */
+		strbuf_addstr(sb, sp_gitdir);
+
+	strbuf_addstr(sb, "/config.superproject");
+}
+
 /*
  * Parse environment variable 'k' as a boolean (in various
  * possible spellings); if missing, use the default value 'def'.
@@ -1911,6 +1925,17 @@ static int do_git_config_sequence(const struct config_options *opts,
 	if (user_config && !access_or_die(user_config, R_OK, ACCESS_EACCES_OK))
 		ret += git_config_from_file(fn, user_config, data);
 
+	current_parsing_scope = CONFIG_SCOPE_SUPERPROJECT;
+	if (opts->git_dir && !opts->ignore_superproject) {
+
+		struct strbuf superproject_gitdir = STRBUF_INIT;
+		git_config_superproject(&superproject_gitdir, opts->git_dir);
+		if (!access_or_die(superproject_gitdir.buf, R_OK, 0))
+			ret += git_config_from_file(fn, superproject_gitdir.buf, data);
+
+		strbuf_release(&superproject_gitdir);
+	}
+
 	current_parsing_scope = CONFIG_SCOPE_LOCAL;
 	if (!opts->ignore_repo && repo_config &&
 	    !access_or_die(repo_config, R_OK, 0))
@@ -2030,6 +2055,7 @@ void read_very_early_config(config_fn_t cb, void *data)
 
 	opts.respect_includes = 1;
 	opts.ignore_repo = 1;
+	opts.ignore_superproject = 1;
 	opts.ignore_worktree = 1;
 	opts.ignore_cmdline = 1;
 	opts.system_gently = 1;
@@ -3518,6 +3544,8 @@ const char *config_scope_name(enum config_scope scope)
 		return "command";
 	case CONFIG_SCOPE_GITMODULES:
 		return "gitmodules";
+	case CONFIG_SCOPE_SUPERPROJECT:
+		return "superproject";
 	default:
 		return "unknown";
 	}
config.h +3 -0
index 5faa72f3f5..69d7e4cb72 100644
@@ -43,6 +43,7 @@ enum config_scope {
 	CONFIG_SCOPE_WORKTREE,
 	CONFIG_SCOPE_COMMAND,
 	CONFIG_SCOPE_GITMODULES,
+	CONFIG_SCOPE_SUPERPROJECT,
 };
 const char *config_scope_name(enum config_scope scope);
 
@@ -84,6 +85,7 @@ typedef int (*config_parser_event_fn_t)(enum config_event_t type,
 struct config_options {
 	unsigned int respect_includes : 1;
 	unsigned int ignore_repo : 1;
+	unsigned int ignore_superproject : 1;
 	unsigned int ignore_worktree : 1;
 	unsigned int ignore_cmdline : 1;
 	unsigned int system_gently : 1;
@@ -318,6 +320,7 @@ int git_config_rename_section(const char *, const char *);
 int git_config_rename_section_in_file(const char *, const char *, const char *);
 int git_config_copy_section(const char *, const char *);
 int git_config_copy_section_in_file(const char *, const char *, const char *);
+void git_config_superproject(struct strbuf *, const char *);
 int git_env_bool(const char *, int);
 unsigned long git_env_ulong(const char *, unsigned long);
 int git_config_system(void);
t/t1311-superproject-config.sh new file +116 -0

116 lines not on this page · raw message

Signature · 1 line
2.32.0.288.g62a8d224e6-goog