Skip to contentSkip to conversation

includeIf breaks calling dashed externals

PATCH Superseded Bert Wesarg 4 participants 24 messages Apr 14, 2017 – Apr 18, 2017

lore
Share & tools
lore

Bert Wesarg

author
#0 lore raw

Dear Duy,

heaving an includeIf in a git config file breaks calling external git commands, most prominently git-gui.

$ git --version
git version 2.12.2.599.gcf11a6797
$ git rev-parse --is-inside-work-tree
true
$ git echo
git: 'echo' is not a git command. See 'git --help'.

Did you mean this?
fetch

$ echo '[includeIf "gitdir:does-not-exists"]path = does-not-exists'
Quoted 1 line
Quoted 1 line

.git/config

$ git rev-parse --is-inside-work-tree
true
$ git echo
fatal: BUG: setup_git_env called without repository

Best,
Bert

#1 lore raw
Quoted 12 lines · Bert Wesarg

Dear Duy,

heaving an includeIf in a git config file breaks calling external git commands, most prominently git-gui.

$ git --version
git version 2.12.2.599.gcf11a6797
$ git rev-parse --is-inside-work-tree
true
$ git echo
git: 'echo' is not a git command. See 'git --help'.
Quoted 8 lines

Did you mean this?
fetch

$ echo '[includeIf "gitdir:does-not-exists"]path = does-not-exists'
Quoted 1 line
Quoted 1 line

.git/config

$ git rev-parse --is-inside-work-tree
true
$ git echo
fatal: BUG: setup_git_env called without repository

Probably this fixes it:

config.c +3 -0
index b6e4a57b9..8d66bdf56 100644
@@ -213,6 +213,9 @@ static int include_by_gitdir(const char *cond, size_t cond_len, int icase)
 	struct strbuf pattern = STRBUF_INIT;
 	int ret = 0, prefix;
 
+	if (!have_git_dir())
+		return 0;
+
 	strbuf_add_absolute_path(&text, get_git_dir());
 	strbuf_add(&pattern, cond, cond_len);
 	prefix = prepare_include_condition_pattern(&pattern);

But it does raise a question of reading config before/after repository setup, since those will give different answers. I guess they do anyway because of $GIT_DIR/config.

-Peff

#2 lore raw
Quoted 40 lines · Jeff King
Quoted 12 lines · Bert Wesarg

Dear Duy,

heaving an includeIf in a git config file breaks calling external git commands, most prominently git-gui.

$ git --version
git version 2.12.2.599.gcf11a6797
$ git rev-parse --is-inside-work-tree
true
$ git echo
git: 'echo' is not a git command. See 'git --help'.
Quoted 8 lines

Did you mean this?
fetch

$ echo '[includeIf "gitdir:does-not-exists"]path = does-not-exists'
Quoted 1 line
Quoted 1 line

.git/config

$ git rev-parse --is-inside-work-tree
true
$ git echo
fatal: BUG: setup_git_env called without repository

Probably this fixes it:

config.c +3 -0
index b6e4a57b9..8d66bdf56 100644
@@ -213,6 +213,9 @@ static int include_by_gitdir(const char *cond, size_t cond_len, int icase)
 	struct strbuf pattern = STRBUF_INIT;
 	int ret = 0, prefix;
 
+	if (!have_git_dir())
+		return 0;
+
 	strbuf_add_absolute_path(&text, get_git_dir());
 	strbuf_add(&pattern, cond, cond_len);
 	prefix = prepare_include_condition_pattern(&pattern);
Quoted 3 lines

But it does raise a question of reading config before/after repository setup, since those will give different answers. I guess they do anyway because of $GIT_DIR/config.

This happens in execv_dased_external() -> check_pager_config() -> read_early_config(). We probably could use the same discover_git_directory trick to get .git dir (because we should find it). Maybe something like this instead?

config.c +8 -1
index 1a4d85537b..4f540ae578 100644
@@ -212,8 +212,14 @@ static int include_by_gitdir(const char *cond, size_t cond_len, int icase)
 	struct strbuf text = STRBUF_INIT;
 	struct strbuf pattern = STRBUF_INIT;
 	int ret = 0, prefix;
+	struct strbuf gitdir = STRBUF_INIT;
 
-	strbuf_add_absolute_path(&text, get_git_dir());
+	if (have_git_dir())
+		strbuf_addstr(&gitdir, get_git_dir());
+	else if (!discover_git_directory(&gitdir))
+		goto done;
+
+	strbuf_add_absolute_path(&text, gitdir.buf);
 	strbuf_add(&pattern, cond, cond_len);
 	prefix = prepare_include_condition_pattern(&pattern);
 

@@ -237,6 +243,7 @@ static int include_by_gitdir(const char *cond, size_t cond_len, int icase)
 			 icase ? WM_CASEFOLD : 0, NULL);
 
 done:
+	strbuf_release(&gitdir);
 	strbuf_release(&pattern);
 	strbuf_release(&text);
 	return ret;

--
Duy

#3 lore raw
Quoted 17 lines · Duy Nguyen
Quoted 17 lines

Probably this fixes it:

config.c +3 -0
index b6e4a57b9..8d66bdf56 100644
@@ -213,6 +213,9 @@ static int include_by_gitdir(const char *cond, size_t cond_len, int icase)
 	struct strbuf pattern = STRBUF_INIT;
 	int ret = 0, prefix;
 
+	if (!have_git_dir())
+		return 0;
+
 	strbuf_add_absolute_path(&text, get_git_dir());
 	strbuf_add(&pattern, cond, cond_len);
 	prefix = prepare_include_condition_pattern(&pattern);
Quoted 8 lines
Quoted 3 lines

But it does raise a question of reading config before/after repository setup, since those will give different answers. I guess they do anyway because of $GIT_DIR/config.

This happens in execv_dased_external() -> check_pager_config() -> read_early_config(). We probably could use the same discover_git_directory trick to get .git dir (because we should find it). Maybe something like this instead?

I know that we only kick in discover_git_directory() in certain cases when we're reading config (for the "early config"). Would it makes sense to respect the same rules here?

I.e., if we have a program that wants to look at config but explicitly _doesn't_ setup the git repository, is it right to say that we are inside that repository?

So instead of lazily doing the discovery here:

Quoted 5 lines
config.c +0 -0
index 1a4d85537b..4f540ae578 100644

config.c static int include_by_gitdir(const char *cond, size_t cond_len, int icase) · line 220

Quoted 10 lines
config.c +5 -1
@@ -212,8 +212,14 @@ static int include_by_gitdir(const char *cond, size_t cond_len, int icase)
 	struct strbuf text = STRBUF_INIT;
 	struct strbuf pattern = STRBUF_INIT;
 	int ret = 0, prefix;
+	struct strbuf gitdir = STRBUF_INIT;
 
-	strbuf_add_absolute_path(&text, get_git_dir());
+	if (have_git_dir())
+		strbuf_addstr(&gitdir, get_git_dir());
+	else if (!discover_git_directory(&gitdir))
+		goto done;

..should we record the discovered git dir in read_early_config(), and use it only if available? And if not, then we know the program is explicitly trying not to be in a repo (or we know we tried to discover one already and it didn't exist).

-Peff

#4 lore raw

PATCH 1/2 config: prepare to pass more info in git_config_with_options()

So far we can only pass one flag, respect_includes, to thie function. We need to pass some more (non-flag even), so let's make it accept a struct instead of an integer.

Signed-off-by Nguyễn Thái Ngọc Duy <[email protected]>

---
Prep patch for the bug fix in 2/2.

 builtin/config.c | 21 ++++++++++++---------
 cache.h          |  7 ++++++-
 config.c         | 16 +++++++++++-----
 3 files changed, 29 insertions(+), 15 deletions(-)
builtin/config.c +12 -9
index 4f49a0edb9..5de4a36146 100644
@@ -26,7 +26,8 @@ static int use_global_config, use_system_config, use_local_config;
 static struct git_config_source given_config_source;
 static int actions, types;
 static int end_null;
-static int respect_includes = -1;
+static int respect_includes_opt;
+static struct config_options config_options;
 static int show_origin;
 
 #define ACTION_GET (1<<0)

@@ -81,7 +82,7 @@ static struct option builtin_config_options[] = {
 	OPT_GROUP(N_("Other")),
 	OPT_BOOL('z', "null", &end_null, N_("terminate values with NUL byte")),
 	OPT_BOOL(0, "name-only", &omit_values, N_("show variable names only")),
-	OPT_BOOL(0, "includes", &respect_includes, N_("respect include directives on lookup")),
+	OPT_BOOL(0, "includes", &respect_includes_opt, N_("respect include directives on lookup")),
 	OPT_BOOL(0, "show-origin", &show_origin, N_("show origin of config (file, standard input, blob, command line)")),
 	OPT_END(),
 };

@@ -242,7 +243,7 @@ static int get_value(const char *key_, const char *regex_)
 	}
 
 	git_config_with_options(collect_config, &values,
-				&given_config_source, respect_includes);
+				&given_config_source, &config_options);
 
 	ret = !values.nr;
 

@@ -320,7 +321,7 @@ static void get_color(const char *var, const char *def_color)
 	get_color_found = 0;
 	parsed_color[0] = '\0';
 	git_config_with_options(git_get_color_config, NULL,
-				&given_config_source, respect_includes);
+				&given_config_source, &config_options);
 
 	if (!get_color_found && def_color) {
 		if (color_parse(def_color, parsed_color) < 0)

@@ -352,7 +353,7 @@ static int get_colorbool(const char *var, int print)
 	get_diff_color_found = -1;
 	get_color_ui_found = -1;
 	git_config_with_options(git_get_colorbool_config, NULL,
-				&given_config_source, respect_includes);
+				&given_config_source, &config_options);
 
 	if (get_colorbool_found < 0) {
 		if (!strcmp(get_colorbool_slot, "color.diff"))

@@ -441,7 +442,7 @@ static int get_urlmatch(const char *var, const char *url)
 	}
 
 	git_config_with_options(urlmatch_config_entry, &config,
-				&given_config_source, respect_includes);
+				&given_config_source, &config_options);
 
 	ret = !values.nr;
 

@@ -530,8 +531,10 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 				prefix_filename(prefix, given_config_source.file);
 	}
 
-	if (respect_includes == -1)
-		respect_includes = !given_config_source.file;
+	if (respect_includes_opt == -1)
+		config_options.respect_includes = !given_config_source.file;
+	else
+		config_options.respect_includes = respect_includes_opt;
 
 	if (end_null) {
 		term = '\0';

@@ -578,7 +581,7 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 		check_argc(argc, 0, 0);
 		if (git_config_with_options(show_all_config, NULL,
 					    &given_config_source,
-					    respect_includes) < 0) {
+					    &config_options) < 0) {
 			if (given_config_source.file)
 				die_errno("unable to read config file '%s'",
 					  given_config_source.file);
cache.h +6 -1
index 5c8078291c..e29a093839 100644
@@ -1882,6 +1882,10 @@ enum config_origin_type {
 	CONFIG_ORIGIN_CMDLINE
 };
 
+struct config_options {
+	unsigned int respect_includes : 1;
+};
+
 typedef int (*config_fn_t)(const char *, const char *, void *);
 extern int git_default_config(const char *, const char *, void *);
 extern int git_config_from_file(config_fn_t fn, const char *, void *);

@@ -1895,7 +1899,7 @@ extern void read_early_config(config_fn_t cb, void *data);
 extern void git_config(config_fn_t fn, void *);
 extern int git_config_with_options(config_fn_t fn, void *,
 				   struct git_config_source *config_source,
-				   int respect_includes);
+				   const struct config_options *opts);
 extern int git_parse_ulong(const char *, unsigned long *);
 extern int git_parse_maybe_bool(const char *);
 extern int git_config_int(const char *, const char *);

@@ -1947,6 +1951,7 @@ struct config_include_data {
 	int depth;
 	config_fn_t fn;
 	void *data;
+	const struct config_options *opts;
 };
 #define CONFIG_INCLUDE_INIT { 0 }
 extern int git_config_include(const char *name, const char *value, void *data);
config.c +11 -5
index 1a4d85537b..042321a3a0 100644
@@ -1530,13 +1530,14 @@ static int do_git_config_sequence(config_fn_t fn, void *data)
 
 int git_config_with_options(config_fn_t fn, void *data,
 			    struct git_config_source *config_source,
-			    int respect_includes)
+			    const struct config_options *opts)
 {
 	struct config_include_data inc = CONFIG_INCLUDE_INIT;
 
-	if (respect_includes) {
+	if (opts->respect_includes) {
 		inc.fn = fn;
 		inc.data = data;
+		inc.opts = opts;
 		fn = git_config_include;
 		data = &inc;
 	}

@@ -1557,7 +1558,10 @@ int git_config_with_options(config_fn_t fn, void *data,
 
 static void git_config_raw(config_fn_t fn, void *data)
 {
-	if (git_config_with_options(fn, data, NULL, 1) < 0)
+	struct config_options opts = {0};
+
+	opts.respect_includes = 1;
+	if (git_config_with_options(fn, data, NULL, &opts) < 0)
 		/*
 		 * git_config_with_options() normally returns only
 		 * zero, as most errors are fatal, and

@@ -1597,9 +1601,11 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 
 void read_early_config(config_fn_t cb, void *data)
 {
+	struct config_options opts = {0};
 	struct strbuf buf = STRBUF_INIT;
 
-	git_config_with_options(cb, data, NULL, 1);
+	opts.respect_includes = 1;
+	git_config_with_options(cb, data, NULL, &opts);
 
 	/*
 	 * When setup_git_directory() was not yet asked to discover the

@@ -1615,7 +1621,7 @@ void read_early_config(config_fn_t cb, void *data)
 		memset(&repo_config, 0, sizeof(repo_config));
 		strbuf_addstr(&buf, "/config");
 		repo_config.file = buf.buf;
-		git_config_with_options(cb, data, &repo_config, 1);
+		git_config_with_options(cb, data, &repo_config, &opts);
 	}
 	strbuf_release(&buf);
 }
Signature · 1 line
2.11.0.157.gd943d85
#5 lore raw

PATCH 2/2 config: handle conditional include when $GIT_DIR is not set up

If setup_git_directory() and friends have not been called, get_git_dir() (because of includeIf.gitdir:XXX) would lead to

    die("BUG: setup_git_env called without repository");

There are two cases when a config file could be read before $GIT_DIR is located. The first one is check_repository_format(), where we read just the one file $GIT_DIR/config to check if we could understand this repository. This case should be safe. The concerned variables should never be hidden away behind includes anyway.

The second one is triggered in check_pager_config() when we're about to run an external git command. We might be able to find $GIT_DIR in this case, which is exactly what read_early_config() does (and also is the what check_pager_config() uses). Conditional includes and get_git_dir() could be triggered by the first git_config_with_options() call there, before discover_git_directory() is used as a fallback $GIT_DIR detection.

Detect this special "early reading" case, pass down the $GIT_DIR, either from previous setup or detected by discover_git_directory(), and make conditional include use it.

Noticed-by Bert Wesarg <[email protected]>
Signed-off-by Nguyễn Thái Ngọc Duy <[email protected]>
---
 cache.h                   |  2 ++
 config.c                  | 35 ++++++++++++++++++++++++++---------
 t/t1305-config-include.sh | 11 +++++++++++
 3 files changed, 39 insertions(+), 9 deletions(-)
cache.h +2 -0
index e29a093839..27b7286f99 100644
@@ -1884,6 +1884,8 @@ enum config_origin_type {
 
 struct config_options {
 	unsigned int respect_includes : 1;
+	unsigned int early_config : 1;
+	const char *git_dir; /* only valid when early_config is true */
 };
 
 typedef int (*config_fn_t)(const char *, const char *, void *);
config.c +26 -9
index 042321a3a0..f323b96283 100644
@@ -207,13 +207,20 @@ static int prepare_include_condition_pattern(struct strbuf *pat)
 	return prefix;
 }
 
-static int include_by_gitdir(const char *cond, size_t cond_len, int icase)
+static int include_by_gitdir(const struct config_options *opts,
+			     const char *cond, size_t cond_len, int icase)
 {
 	struct strbuf text = STRBUF_INIT;
 	struct strbuf pattern = STRBUF_INIT;
 	int ret = 0, prefix;
 
-	strbuf_add_absolute_path(&text, get_git_dir());
+	if (!opts->early_config)
+		strbuf_add_absolute_path(&text, get_git_dir());
+	else if (opts->git_dir)
+		strbuf_add_absolute_path(&text, opts->git_dir);
+	else
+		goto done;
+
 	strbuf_add(&pattern, cond, cond_len);
 	prefix = prepare_include_condition_pattern(&pattern);
 

@@ -242,13 +249,14 @@ static int include_by_gitdir(const char *cond, size_t cond_len, int icase)
 	return ret;
 }
 
-static int include_condition_is_true(const char *cond, size_t cond_len)
+static int include_condition_is_true(const struct config_options *opts,
+				     const char *cond, size_t cond_len)
 {
 
 	if (skip_prefix_mem(cond, cond_len, "gitdir:", &cond, &cond_len))
-		return include_by_gitdir(cond, cond_len, 0);
+		return include_by_gitdir(opts, cond, cond_len, 0);
 	else if (skip_prefix_mem(cond, cond_len, "gitdir/i:", &cond, &cond_len))
-		return include_by_gitdir(cond, cond_len, 1);
+		return include_by_gitdir(opts, cond, cond_len, 1);
 
 	/* unknown conditionals are always false */
 	return 0;

@@ -273,7 +281,7 @@ int git_config_include(const char *var, const char *value, void *data)
 		ret = handle_path_include(value, inc);
 
 	if (!parse_config_key(var, "includeif", &cond, &cond_len, &key) &&
-	    (cond && include_condition_is_true(cond, cond_len)) &&
+	    (cond && include_condition_is_true(inc->opts, cond, cond_len)) &&
 	    !strcmp(key, "path"))
 		ret = handle_path_include(value, inc);
 

@@ -1603,10 +1611,13 @@ void read_early_config(config_fn_t cb, void *data)
 {
 	struct config_options opts = {0};
 	struct strbuf buf = STRBUF_INIT;
+	char *to_free = NULL;
 
 	opts.respect_includes = 1;
-	git_config_with_options(cb, data, NULL, &opts);
+	opts.early_config = 1;
 
+	if (have_git_dir())
+		opts.git_dir = get_git_dir();
 	/*
 	 * When setup_git_directory() was not yet asked to discover the
 	 * GIT_DIR, we ask discover_git_directory() to figure out whether there

@@ -1615,15 +1626,21 @@ void read_early_config(config_fn_t cb, void *data)
 	 * notably, the current working directory is still the same after the
 	 * call).
 	 */
-	if (!have_git_dir() && discover_git_directory(&buf)) {
+	else if (discover_git_directory(&buf))
+		opts.git_dir = to_free = strbuf_detach(&buf, NULL);
+
+	git_config_with_options(cb, data, NULL, &opts);
+
+	if (opts.git_dir) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));
-		strbuf_addstr(&buf, "/config");
+		strbuf_addf(&buf, "%s/config", opts.git_dir);
 		repo_config.file = buf.buf;
 		git_config_with_options(cb, data, &repo_config, &opts);
 	}
 	strbuf_release(&buf);
+	free(to_free);
 }
 
 static void git_config_check_init(void);
t/t1305-config-include.sh +11 -0
index e833939320..fddb47bafa 100755
@@ -208,6 +208,17 @@ test_expect_success 'conditional include, both unanchored, icase' '
 	)
 '
 
+test_expect_success 'conditional include, early config reading' '
+	(
+		cd foo &&
+		echo "[includeIf \"gitdir:foo/\"]path=bar6" >>.git/config &&
+		echo "[test]six=6" >.git/bar6 &&
+		echo 6 >expect &&
+		test-config read_early_config test.six >actual &&
+		test_cmp expect actual
+	)
+'
+
 test_expect_success 'include cycles are detected' '
 	cat >.gitconfig <<-\EOF &&
 	[test]value = gitconfig
Signature · 1 line
2.11.0.157.gd943d85
#6 lore raw

Re: [PATCH 2/2] config: handle conditional include when $GIT_DIR is not set up

Quoted 10 lines · Nguyễn Thái Ngọc Duy

If setup_git_directory() and friends have not been called, get_git_dir() (because of includeIf.gitdir:XXX) would lead to

    die("BUG: setup_git_env called without repository");

There are two cases when a config file could be read before $GIT_DIR is located. The first one is check_repository_format(), where we read just the one file $GIT_DIR/config to check if we could understand this repository. This case should be safe. The concerned variables should never be hidden away behind includes anyway.

Right, we should not even have respect_includes turned on for that case.

Quoted 4 lines

The second one is triggered in check_pager_config() when we're about to run an external git command. We might be able to find $GIT_DIR in this case, which is exactly what read_early_config() does (and also is the what check_pager_config() uses). Conditional includes and

s/the what/what/

cache.h enum config_origin_type { · line 1889

Quoted 11 lines
cache.h +2 -0
index e29a093839..27b7286f99 100644
@@ -1884,6 +1884,8 @@ enum config_origin_type {
 
 struct config_options {
 	unsigned int respect_includes : 1;
+	unsigned int early_config : 1;
+	const char *git_dir; /* only valid when early_config is true */
 };

Why do we need both the flag and the string? Later, you do:

Quoted 5 lines
cache.h +2 -1
-static int include_by_gitdir(const char *cond, size_t cond_len, int icase)
+static int include_by_gitdir(const struct config_options *opts,
+			     const char *cond, size_t cond_len, int icase)
 {
 	struct strbuf text = STRBUF_INIT;

cache.h enum config_origin_type {

Quoted 10 lines
cache.h +6 -1
 	struct strbuf pattern = STRBUF_INIT;
 	int ret = 0, prefix;
 
-	strbuf_add_absolute_path(&text, get_git_dir());
+	if (!opts->early_config)
+		strbuf_add_absolute_path(&text, get_git_dir());
+	else if (opts->git_dir)
+		strbuf_add_absolute_path(&text, opts->git_dir);
+	else
+		goto done;

So we call get_git_dir() always when we're not in early config. Even if we don't have a git dir! Doesn't this mean that programs operating outside of a repo will still hit the BUG? I.e.:

  git config --global includeif.gitdir:/whatever.path foo
  cd /not/a/git/dir
  git diff --no-index foo bar

?

I think instead the logic should be:

if (opts->git_dir)

	strbuf_add_absolute_path(&text, opts->git_dir);
  else if (have_git_dir())
	strbuf_add_absolute_path(&text, get_git_dir());
  else
	goto done;

I'd also be tempted to call the option field "override_git_dir" or something to indicate that it takes precedence over the normal one. With the current code it doesn't matter, because we set it only to the result of the discovered dir.

cache.h void read_early_config(config_fn_t cb, void *data) · line 1630

Quoted 7 lines
cache.h +2 -1
@@ -1615,15 +1626,21 @@ void read_early_config(config_fn_t cb, void *data)
 	 * notably, the current working directory is still the same after the
 	 * call).
 	 */
-	if (!have_git_dir() && discover_git_directory(&buf)) {
+	else if (discover_git_directory(&buf))
+		opts.git_dir = to_free = strbuf_detach(&buf, NULL);

This to_free seemed redundant to me at first; why not just hold on to the strbuf and release it later?

The answer is that we reuse the strbuf to generate the config-file path later. However, by detaching, we clear the strbuf. So later when we use it:

cache.h void read_early_config(config_fn_t cb, void *data) · line 1638

Quoted 9 lines
cache.h +2 -1
+	if (opts.git_dir) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));
-		strbuf_addstr(&buf, "/config");
+		strbuf_addf(&buf, "%s/config", opts.git_dir);
 		repo_config.file = buf.buf;
 		git_config_with_options(cb, data, &repo_config, &opts);
 	}

...we have to re-add the git_dir.

Might it be simpler to just xstrdup() to opts.git_dir, and then leave this later code alone?

That said, I actually think in the long run that git_config_with_options() should compute the repo_config itself from our git_dir parameter. That lets us fix a very subtle bug in read_early_config(). The problem is that the function works like this:

1. Run git_config_with_options(), hitting the usual sources in order

2. If we didn't have a git-dir, run it again just for our discovered repo-config

That has two implications:

- the config callback will see keys from ~/.gitconfig twice, once for each run. This is usually OK because the only early config we care about uses last-one-wins semantics (so no list-appending).

- the repo-level config is read last, so by last-one-wins it takes precedence over ~/.gitconfig. Good. But it should have _less_ precedence than command-line config, and it doesn't.

You can see the second problem with:

  # random external
  cat >git-foo <<-\EOF
  #!/bin/sh
  echo foo
  EOF
  chmod +x git-foo
  PATH=$PWD:$PATH

git init
git config pager.foo 'sed s/^/repo:/'
git -c pager.foo='sed s/^/cmdline:/' foo

That command should prefer the cmdline config, but it doesn't.

The fix is something like what's below, which is easy on top of your new options struct. I can wrap it up with a config message and test on top of your series.

config.c +11 -11
index f323b9628..5dda6e8ca 100644
@@ -1502,12 +1502,20 @@ int git_config_system(void)
 	return !git_env_bool("GIT_CONFIG_NOSYSTEM", 0);
 }
 
-static int do_git_config_sequence(config_fn_t fn, void *data)
+static int do_git_config_sequence(config_fn_t fn, void *data,
+				  const char *override_repo_dir)
 {
 	int ret = 0;
 	char *xdg_config = xdg_config_home("config");
 	char *user_config = expand_user_path("~/.gitconfig");
-	char *repo_config = have_git_dir() ? git_pathdup("config") : NULL;
+	char *repo_config;
+
+	if (override_repo_dir)
+		repo_config = mkpathdup("%s/config", override_repo_dir);
+	else if (have_git_dir())
+		repo_config = git_pathdup("config");
+	else
+		repo_config = NULL;
 
 	current_parsing_scope = CONFIG_SCOPE_SYSTEM;
 	if (git_config_system() && !access_or_die(git_etc_gitconfig(), R_OK, 0))

@@ -1561,7 +1569,7 @@ int git_config_with_options(config_fn_t fn, void *data,
 	else if (config_source && config_source->blob)
 		return git_config_from_blob_ref(fn, config_source->blob, data);
 
-	return do_git_config_sequence(fn, data);
+	return do_git_config_sequence(fn, data, opts->git_dir);
 }
 
 static void git_config_raw(config_fn_t fn, void *data)

@@ -1631,14 +1639,6 @@ void read_early_config(config_fn_t cb, void *data)
 
 	git_config_with_options(cb, data, NULL, &opts);
 
-	if (opts.git_dir) {
-		struct git_config_source repo_config;
-
-		memset(&repo_config, 0, sizeof(repo_config));
-		strbuf_addf(&buf, "%s/config", opts.git_dir);
-		repo_config.file = buf.buf;
-		git_config_with_options(cb, data, &repo_config, &opts);
-	}
 	strbuf_release(&buf);
 	free(to_free);
 }
#7 lore raw

Re: [PATCH 2/2] config: handle conditional include when $GIT_DIR is not set up

Quoted 40 lines · Jeff King

cache.h enum config_origin_type { · line 1889

Quoted 11 lines
cache.h +2 -0
index e29a093839..27b7286f99 100644
@@ -1884,6 +1884,8 @@ enum config_origin_type {
 
 struct config_options {
 	unsigned int respect_includes : 1;
+	unsigned int early_config : 1;
+	const char *git_dir; /* only valid when early_config is true */
 };

Why do we need both the flag and the string? Later, you do:

Quoted 5 lines
cache.h +2 -1
-static int include_by_gitdir(const char *cond, size_t cond_len, int icase)
+static int include_by_gitdir(const struct config_options *opts,
+			     const char *cond, size_t cond_len, int icase)
 {
 	struct strbuf text = STRBUF_INIT;

cache.h enum config_origin_type {

Quoted 10 lines
cache.h +6 -1
 	struct strbuf pattern = STRBUF_INIT;
 	int ret = 0, prefix;
 
-	strbuf_add_absolute_path(&text, get_git_dir());
+	if (!opts->early_config)
+		strbuf_add_absolute_path(&text, get_git_dir());
+	else if (opts->git_dir)
+		strbuf_add_absolute_path(&text, opts->git_dir);
+	else
+		goto done;

So we call get_git_dir() always when we're not in early config. Even if we don't have a git dir! Doesn't this mean that programs operating outside of a repo will still hit the BUG? I.e.:

  git config --global includeif.gitdir:/whatever.path foo
  cd /not/a/git/dir
  git diff --no-index foo bar

?

Quoted 8 lines

I think instead the logic should be:

if (opts->git_dir)

	strbuf_add_absolute_path(&text, opts->git_dir);
  else if (have_git_dir())
	strbuf_add_absolute_path(&text, get_git_dir());
  else
	goto done;

Do we care about the case when the override instruction is "we don't have $GIT_DIR, act as if it does not exist, even though have_git_dir() returns true"?

I'm guessing no, we won't run into that situation (and am inclined to restructure the code as you suggested). Just throwing it out there if I'm mistaken.
--
Duy

#8 lore raw

Re: [PATCH 2/2] config: handle conditional include when $GIT_DIR is not set up

Quoted 19 lines · Duy Nguyen
Quoted 10 lines

So we call get_git_dir() always when we're not in early config. Even if we don't have a git dir! Doesn't this mean that programs operating outside of a repo will still hit the BUG? I.e.:

  git config --global includeif.gitdir:/whatever.path foo
  cd /not/a/git/dir
  git diff --no-index foo bar

?

Quoted 8 lines

I think instead the logic should be:

if (opts->git_dir)

	strbuf_add_absolute_path(&text, opts->git_dir);
  else if (have_git_dir())
	strbuf_add_absolute_path(&text, get_git_dir());
  else
	goto done;
Quoted 7 lines

Do we care about the case when the override instruction is "we don't have $GIT_DIR, act as if it does not exist, even though have_git_dir() returns true"?

I'm guessing no, we won't run into that situation (and am inclined to restructure the code as you suggested). Just throwing it out there if I'm mistaken.

I can't think of a case where that would matter. We're always prodding the function to look at something it might not, and I don't think there would be any need to convince it _not_ to look at something.

-Peff

#9 lore raw

Re: [PATCH 2/2] config: handle conditional include when $GIT_DIR is not set up

In reply to Jeff King

(To Junio, this series conflicts slightly with nd/conditional-config-include, let me know if you want me to rebase this on top of that)

Quoted 10 lines · Jeff King

Review

Quoted 9 lines
(quoted diff) +2 -1
+     if (opts.git_dir) {
              struct git_config_source repo_config;
 
              memset(&repo_config, 0, sizeof(repo_config));
-             strbuf_addstr(&buf, "/config");
+             strbuf_addf(&buf, "%s/config", opts.git_dir);
              repo_config.file = buf.buf;
              git_config_with_options(cb, data, &repo_config, &opts);
      }
Quoted 4 lines

...we have to re-add the git_dir.

Might it be simpler to just xstrdup() to opts.git_dir, and then leave this later code alone?

Sure thing. But we need to restore the "if" expression too. Otherwise, if have_git_dir() is true, we may come here with an empty "buf" before that strbuf_addstr(&buf, "/config") is called. It does not matter much anyway because this block will be removed.

Quoted 10 lines

You can see the second problem with:

  # random external
  cat >git-foo <<-\EOF
  #!/bin/sh
  echo foo
  EOF
  chmod +x git-foo
  PATH=$PWD:$PATH
Quoted 5 lines

git init
git config pager.foo 'sed s/^/repo:/'
git -c pager.foo='sed s/^/cmdline:/' foo

That command should prefer the cmdline config, but it doesn't.

I actually had problems seeing the problem, for some reason it didn't work for me. I guess I made a mistake somewhere.

Quoted 3 lines

The fix is something like what's below, which is easy on top of your new options struct. I can wrap it up with a config message and test on top of your series.

... anyway I read this last sentence too late and spent too much time wrapping your changes in a patch (well most of the time was spent on writing a new test actually), so I'm sending it out too. My test uses test-config though (I have given up on dealing with pager and tty).

Off topic. Back to the pager.foo thing (because I had to fight it a bit before giving up). I find it a bit unintuitive that "--paginate" forces the pager back to less (or $PAGER) when I say "pager.foo = my-favorite-pager". Back when pager.foo is a boolean thing, it makes sense for --paginate to override the "to page or not to page" decision. But then you added a new meaning too pager.foo (what command to run). "--paginate" should respect the command pager.foo specifies when its value is a command, I think.

Signature · 1 line
Duy
#10 lore raw

PATCH v2 1/3 config: prepare to pass more info in git_config_with_options()

So far we can only pass one flag, respect_includes, to thie function. We need to pass some more (non-flag even), so let's make it accept a struct instead of an integer.

Signed-off-by Nguyễn Thái Ngọc Duy <[email protected]>
---
 builtin/config.c | 21 ++++++++++++---------
 cache.h          |  7 ++++++-
 config.c         | 16 +++++++++++-----
 3 files changed, 29 insertions(+), 15 deletions(-)
builtin/config.c +12 -9
index 4f49a0edb9..b937d175a9 100644
@@ -26,7 +26,8 @@ static int use_global_config, use_system_config, use_local_config;
 static struct git_config_source given_config_source;
 static int actions, types;
 static int end_null;
-static int respect_includes = -1;
+static int respect_includes_opt = -1;
+static struct config_options config_options;
 static int show_origin;
 
 #define ACTION_GET (1<<0)

@@ -81,7 +82,7 @@ static struct option builtin_config_options[] = {
 	OPT_GROUP(N_("Other")),
 	OPT_BOOL('z', "null", &end_null, N_("terminate values with NUL byte")),
 	OPT_BOOL(0, "name-only", &omit_values, N_("show variable names only")),
-	OPT_BOOL(0, "includes", &respect_includes, N_("respect include directives on lookup")),
+	OPT_BOOL(0, "includes", &respect_includes_opt, N_("respect include directives on lookup")),
 	OPT_BOOL(0, "show-origin", &show_origin, N_("show origin of config (file, standard input, blob, command line)")),
 	OPT_END(),
 };

@@ -242,7 +243,7 @@ static int get_value(const char *key_, const char *regex_)
 	}
 
 	git_config_with_options(collect_config, &values,
-				&given_config_source, respect_includes);
+				&given_config_source, &config_options);
 
 	ret = !values.nr;
 

@@ -320,7 +321,7 @@ static void get_color(const char *var, const char *def_color)
 	get_color_found = 0;
 	parsed_color[0] = '\0';
 	git_config_with_options(git_get_color_config, NULL,
-				&given_config_source, respect_includes);
+				&given_config_source, &config_options);
 
 	if (!get_color_found && def_color) {
 		if (color_parse(def_color, parsed_color) < 0)

@@ -352,7 +353,7 @@ static int get_colorbool(const char *var, int print)
 	get_diff_color_found = -1;
 	get_color_ui_found = -1;
 	git_config_with_options(git_get_colorbool_config, NULL,
-				&given_config_source, respect_includes);
+				&given_config_source, &config_options);
 
 	if (get_colorbool_found < 0) {
 		if (!strcmp(get_colorbool_slot, "color.diff"))

@@ -441,7 +442,7 @@ static int get_urlmatch(const char *var, const char *url)
 	}
 
 	git_config_with_options(urlmatch_config_entry, &config,
-				&given_config_source, respect_includes);
+				&given_config_source, &config_options);
 
 	ret = !values.nr;
 

@@ -530,8 +531,10 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 				prefix_filename(prefix, given_config_source.file);
 	}
 
-	if (respect_includes == -1)
-		respect_includes = !given_config_source.file;
+	if (respect_includes_opt == -1)
+		config_options.respect_includes = !given_config_source.file;
+	else
+		config_options.respect_includes = respect_includes_opt;
 
 	if (end_null) {
 		term = '\0';

@@ -578,7 +581,7 @@ int cmd_config(int argc, const char **argv, const char *prefix)
 		check_argc(argc, 0, 0);
 		if (git_config_with_options(show_all_config, NULL,
 					    &given_config_source,
-					    respect_includes) < 0) {
+					    &config_options) < 0) {
 			if (given_config_source.file)
 				die_errno("unable to read config file '%s'",
 					  given_config_source.file);
cache.h +6 -1
index 556468c25b..a6294d2573 100644
@@ -1885,6 +1885,10 @@ enum config_origin_type {
 	CONFIG_ORIGIN_CMDLINE
 };
 
+struct config_options {
+	unsigned int respect_includes : 1;
+};
+
 typedef int (*config_fn_t)(const char *, const char *, void *);
 extern int git_default_config(const char *, const char *, void *);
 extern int git_config_from_file(config_fn_t fn, const char *, void *);

@@ -1898,7 +1902,7 @@ extern void read_early_config(config_fn_t cb, void *data);
 extern void git_config(config_fn_t fn, void *);
 extern int git_config_with_options(config_fn_t fn, void *,
 				   struct git_config_source *config_source,
-				   int respect_includes);
+				   const struct config_options *opts);
 extern int git_parse_ulong(const char *, unsigned long *);
 extern int git_parse_maybe_bool(const char *);
 extern int git_config_int(const char *, const char *);

@@ -1950,6 +1954,7 @@ struct config_include_data {
 	int depth;
 	config_fn_t fn;
 	void *data;
+	const struct config_options *opts;
 };
 #define CONFIG_INCLUDE_INIT { 0 }
 extern int git_config_include(const char *name, const char *value, void *data);
config.c +11 -5
index 1a4d85537b..042321a3a0 100644
@@ -1530,13 +1530,14 @@ static int do_git_config_sequence(config_fn_t fn, void *data)
 
 int git_config_with_options(config_fn_t fn, void *data,
 			    struct git_config_source *config_source,
-			    int respect_includes)
+			    const struct config_options *opts)
 {
 	struct config_include_data inc = CONFIG_INCLUDE_INIT;
 
-	if (respect_includes) {
+	if (opts->respect_includes) {
 		inc.fn = fn;
 		inc.data = data;
+		inc.opts = opts;
 		fn = git_config_include;
 		data = &inc;
 	}

@@ -1557,7 +1558,10 @@ int git_config_with_options(config_fn_t fn, void *data,
 
 static void git_config_raw(config_fn_t fn, void *data)
 {
-	if (git_config_with_options(fn, data, NULL, 1) < 0)
+	struct config_options opts = {0};
+
+	opts.respect_includes = 1;
+	if (git_config_with_options(fn, data, NULL, &opts) < 0)
 		/*
 		 * git_config_with_options() normally returns only
 		 * zero, as most errors are fatal, and

@@ -1597,9 +1601,11 @@ static void configset_iter(struct config_set *cs, config_fn_t fn, void *data)
 
 void read_early_config(config_fn_t cb, void *data)
 {
+	struct config_options opts = {0};
 	struct strbuf buf = STRBUF_INIT;
 
-	git_config_with_options(cb, data, NULL, 1);
+	opts.respect_includes = 1;
+	git_config_with_options(cb, data, NULL, &opts);
 
 	/*
 	 * When setup_git_directory() was not yet asked to discover the

@@ -1615,7 +1621,7 @@ void read_early_config(config_fn_t cb, void *data)
 		memset(&repo_config, 0, sizeof(repo_config));
 		strbuf_addstr(&buf, "/config");
 		repo_config.file = buf.buf;
-		git_config_with_options(cb, data, &repo_config, 1);
+		git_config_with_options(cb, data, &repo_config, &opts);
 	}
 	strbuf_release(&buf);
 }
Signature · 1 line
2.11.0.157.gd943d85
#11 lore raw

PATCH v2 2/3 config: handle conditional include when $GIT_DIR is not set up

If setup_git_directory() and friends have not been called, get_git_dir() (because of includeIf.gitdir:XXX) would lead to

    die("BUG: setup_git_env called without repository");

There are two cases when a config file could be read before $GIT_DIR is located.

The first one is check_repository_format(), where we read just the one file $GIT_DIR/config to check if we could understand this repository. This case should be safe. We do not parse include directives, which can only be triggered from git_config_with_options, but setup code uses a lower-level function. The concerned variables should never be hidden away behind includes anyway.

The second one is triggered in check_pager_config() when we're about to run an external git command. We might be able to find $GIT_DIR in this case, which is exactly what read_early_config() does (and also is what check_pager_config() uses). Conditional includes and get_git_dir() could be triggered by the first git_config_with_options() call there, before discover_git_directory() is used as a fallback $GIT_DIR detection.

Detect this special "early reading" case, pass down the $GIT_DIR, either from previous setup or detected by discover_git_directory(), and make conditional include use it.

Noticed-by Bert Wesarg <[email protected]>
Signed-off-by Nguyễn Thái Ngọc Duy <[email protected]>
---
 cache.h                   |  1 +
 config.c                  | 34 ++++++++++++++++++++++++++--------
 t/t1305-config-include.sh | 11 +++++++++++
 3 files changed, 38 insertions(+), 8 deletions(-)
cache.h +1 -0
index a6294d2573..878e1d441f 100644
@@ -1887,6 +1887,7 @@ enum config_origin_type {
 
 struct config_options {
 	unsigned int respect_includes : 1;
+	const char *git_dir;
 };
 
 typedef int (*config_fn_t)(const char *, const char *, void *);
config.c +26 -8
index 042321a3a0..14f0417460 100644
@@ -207,13 +207,22 @@ static int prepare_include_condition_pattern(struct strbuf *pat)
 	return prefix;
 }
 
-static int include_by_gitdir(const char *cond, size_t cond_len, int icase)
+static int include_by_gitdir(const struct config_options *opts,
+			     const char *cond, size_t cond_len, int icase)
 {
 	struct strbuf text = STRBUF_INIT;
 	struct strbuf pattern = STRBUF_INIT;
 	int ret = 0, prefix;
+	const char *git_dir;
 
-	strbuf_add_absolute_path(&text, get_git_dir());
+	if (opts->git_dir)
+		git_dir = opts->git_dir;
+	else if (have_git_dir())
+		git_dir = get_git_dir();
+	else
+		goto done;
+
+	strbuf_add_absolute_path(&text, git_dir);
 	strbuf_add(&pattern, cond, cond_len);
 	prefix = prepare_include_condition_pattern(&pattern);
 

@@ -242,13 +251,14 @@ static int include_by_gitdir(const char *cond, size_t cond_len, int icase)
 	return ret;
 }
 
-static int include_condition_is_true(const char *cond, size_t cond_len)
+static int include_condition_is_true(const struct config_options *opts,
+				     const char *cond, size_t cond_len)
 {
 
 	if (skip_prefix_mem(cond, cond_len, "gitdir:", &cond, &cond_len))
-		return include_by_gitdir(cond, cond_len, 0);
+		return include_by_gitdir(opts, cond, cond_len, 0);
 	else if (skip_prefix_mem(cond, cond_len, "gitdir/i:", &cond, &cond_len))
-		return include_by_gitdir(cond, cond_len, 1);
+		return include_by_gitdir(opts, cond, cond_len, 1);
 
 	/* unknown conditionals are always false */
 	return 0;

@@ -273,7 +283,7 @@ int git_config_include(const char *var, const char *value, void *data)
 		ret = handle_path_include(value, inc);
 
 	if (!parse_config_key(var, "includeif", &cond, &cond_len, &key) &&
-	    (cond && include_condition_is_true(cond, cond_len)) &&
+	    (cond && include_condition_is_true(inc->opts, cond, cond_len)) &&
 	    !strcmp(key, "path"))
 		ret = handle_path_include(value, inc);
 

@@ -1603,10 +1613,12 @@ void read_early_config(config_fn_t cb, void *data)
 {
 	struct config_options opts = {0};
 	struct strbuf buf = STRBUF_INIT;
+	char *to_free = NULL;
 
 	opts.respect_includes = 1;
-	git_config_with_options(cb, data, NULL, &opts);
 
+	if (have_git_dir())
+		opts.git_dir = get_git_dir();
 	/*
 	 * When setup_git_directory() was not yet asked to discover the
 	 * GIT_DIR, we ask discover_git_directory() to figure out whether there

@@ -1615,7 +1627,12 @@ void read_early_config(config_fn_t cb, void *data)
 	 * notably, the current working directory is still the same after the
 	 * call).
 	 */
-	if (!have_git_dir() && discover_git_directory(&buf)) {
+	else if (discover_git_directory(&buf))
+		opts.git_dir = to_free = xstrdup(buf.buf);
+
+	git_config_with_options(cb, data, NULL, &opts);
+
+	if (!have_git_dir() && opts.git_dir) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));

@@ -1624,6 +1641,7 @@ void read_early_config(config_fn_t cb, void *data)
 		git_config_with_options(cb, data, &repo_config, &opts);
 	}
 	strbuf_release(&buf);
+	free(to_free);
 }
 
 static void git_config_check_init(void);
t/t1305-config-include.sh +11 -0
index e833939320..fddb47bafa 100755
@@ -208,6 +208,17 @@ test_expect_success 'conditional include, both unanchored, icase' '
 	)
 '
 
+test_expect_success 'conditional include, early config reading' '
+	(
+		cd foo &&
+		echo "[includeIf \"gitdir:foo/\"]path=bar6" >>.git/config &&
+		echo "[test]six=6" >.git/bar6 &&
+		echo 6 >expect &&
+		test-config read_early_config test.six >actual &&
+		test_cmp expect actual
+	)
+'
+
 test_expect_success 'include cycles are detected' '
 	cat >.gitconfig <<-\EOF &&
 	[test]value = gitconfig
Signature · 1 line
2.11.0.157.gd943d85
#12 lore raw

Re: [PATCH v2 2/3] config: handle conditional include when $GIT_DIR is not set up

Quoted 15 lines · Nguyễn Thái Ngọc Duy
config.c +3 -1
@@ -1603,10 +1613,12 @@ void read_early_config(config_fn_t cb, void *data)
 {
 	struct config_options opts = {0};
 	struct strbuf buf = STRBUF_INIT;
+	char *to_free = NULL;
 
 	opts.respect_includes = 1;
-	git_config_with_options(cb, data, NULL, &opts);
 
+	if (have_git_dir())
+		opts.git_dir = get_git_dir();
 	/*
 	 * When setup_git_directory() was not yet asked to discover the
 	 * GIT_DIR, we ask discover_git_directory() to figure out whether there

config.c void read_early_config(config_fn_t cb, void *data) · line 1633

Quoted 8 lines
config.c +4 -1
@@ -1615,7 +1627,12 @@ void read_early_config(config_fn_t cb, void *data)
 	 * notably, the current working directory is still the same after the
 	 * call).
 	 */
-	if (!have_git_dir() && discover_git_directory(&buf)) {
+	else if (discover_git_directory(&buf))
+		opts.git_dir = to_free = xstrdup(buf.buf);
+
+	git_config_with_options(cb, data, NULL, &opts);

This one I can understand. By having NULL for config_source, this does the usual do_git_config_sequence() dance, which knows to treat opts.git_dir is the "repository config" without necessarily being able to do git_pathdup("config").

config.c void read_early_config(config_fn_t cb, void *data) · line 1638

Quoted 4 lines
config.c +1 -0
+	if (!have_git_dir() && opts.git_dir) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));

But this one I do not quite understand. When have_git_dir() was false and asked discover_git_directory() to set opts.git_dir, we enter the body of this block and then end up doing

	git_config_with_options(cb, data &repo_config, &opts);

with repo_config set to the discovered git directory plus "/config"; we'd read the repository configuration twice, in other words.

#13 lore raw

Re: [PATCH v2 2/3] config: handle conditional include when $GIT_DIR is not set up

Quoted 9 lines · Junio C Hamano

config.c void read_early_config(config_fn_t cb, void *data) · line 1638

Quoted 4 lines
config.c +1 -0
+	if (!have_git_dir() && opts.git_dir) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));

But this one I do not quite understand. When have_git_dir() was false and asked discover_git_directory() to set opts.git_dir, we enter the body of this block and then end up doing

Quoted 4 lines
	git_config_with_options(cb, data &repo_config, &opts);

with repo_config set to the discovered git directory plus "/config"; we'd read the repository configuration twice, in other words.

Ahh, nevermind. The fix to make the "usual sequence" pay attention to the opts->git_dir comes in 3/3 and in that step this redundant reading is also removed, so all is well at the end.

Thanks.

#14 lore raw

Re: [PATCH v2 2/3] config: handle conditional include when $GIT_DIR is not set up

Quoted 11 lines · Junio C Hamano
Quoted 9 lines · Junio C Hamano

config.c void read_early_config(config_fn_t cb, void *data) · line 1638

Quoted 4 lines
config.c +1 -0
+	if (!have_git_dir() && opts.git_dir) {
 		struct git_config_source repo_config;
 
 		memset(&repo_config, 0, sizeof(repo_config));

But this one I do not quite understand. When have_git_dir() was false and asked discover_git_directory() to set opts.git_dir, we enter the body of this block and then end up doing

Quoted 8 lines
Quoted 4 lines
	git_config_with_options(cb, data &repo_config, &opts);

with repo_config set to the discovered git directory plus "/config"; we'd read the repository configuration twice, in other words.

Ahh, nevermind. The fix to make the "usual sequence" pay attention to the opts->git_dir comes in 3/3 and in that step this redundant reading is also removed, so all is well at the end.

I think 2/3 is actually OK as-is. We know we didn't read repo config in the first call to git_config_with_options() because have_git_dir() is unset.

So 2/3 doesn't make anything worse; the bug already existed before Duy's series (it goes back to my original read_early_config() hack).

-Peff

#15 lore raw

PATCH v2 3/3 config: correct file reading order in read_early_config()

In reply to Nguyễn Thái Ngọc Duy

Config file reading order is important because each file can override values in the previous files and this is expected behavior. Normally we read in this order, all in do_git_config_sequence():

1. $HOME/.gitconfig
2. $GIT_DIR/config
3. config from command line

However in read_early_config() the order may be swapped a bit if setup_git_directory() has not been called:

1. $HOME/.gitconfig
2. $GIT_DIR/config is NOT read because .git dir is not found _yet_
3. config from command line
4. $GIT_DIR/config is now READ (after discover_git_directory() call)

The reading at step 4 could override config at step 3, which is not the expectation.

Now that we could pass the .git dir around, we could feed discover_git_directory() back to step 2, so that it works again, and remove step 4.

Noticed-by Jeff King <[email protected]>
Signed-off-by Nguyễn Thái Ngọc Duy <[email protected]>
---
 config.c                | 26 ++++++++++++--------------
 t/helper/test-config.c  |  4 ++++
 t/t1309-early-config.sh | 17 +++++++++++++++++
 3 files changed, 33 insertions(+), 14 deletions(-)
config.c +12 -14
index 14f0417460..fffda653e3 100644
@@ -1504,12 +1504,20 @@ int git_config_system(void)
 	return !git_env_bool("GIT_CONFIG_NOSYSTEM", 0);
 }
 
-static int do_git_config_sequence(config_fn_t fn, void *data)
+static int do_git_config_sequence(const struct config_options *opts,
+				  config_fn_t fn, void *data)
 {
 	int ret = 0;
 	char *xdg_config = xdg_config_home("config");
 	char *user_config = expand_user_path("~/.gitconfig");
-	char *repo_config = have_git_dir() ? git_pathdup("config") : NULL;
+	char *repo_config;
+
+	if (opts->git_dir)
+		repo_config = mkpathdup("%s/config", opts->git_dir);
+	else if (have_git_dir())
+		repo_config = git_pathdup("config");
+	else
+		repo_config = NULL;
 
 	current_parsing_scope = CONFIG_SCOPE_SYSTEM;
 	if (git_config_system() && !access_or_die(git_etc_gitconfig(), R_OK, 0))

@@ -1563,7 +1571,7 @@ int git_config_with_options(config_fn_t fn, void *data,
 	else if (config_source && config_source->blob)
 		return git_config_from_blob_ref(fn, config_source->blob, data);
 
-	return do_git_config_sequence(fn, data);
+	return do_git_config_sequence(opts, fn, data);
 }
 
 static void git_config_raw(config_fn_t fn, void *data)

@@ -1613,7 +1621,6 @@ void read_early_config(config_fn_t cb, void *data)
 {
 	struct config_options opts = {0};
 	struct strbuf buf = STRBUF_INIT;
-	char *to_free = NULL;
 
 	opts.respect_includes = 1;
 

@@ -1628,20 +1635,11 @@ void read_early_config(config_fn_t cb, void *data)
 	 * call).
 	 */
 	else if (discover_git_directory(&buf))
-		opts.git_dir = to_free = xstrdup(buf.buf);
+		opts.git_dir = buf.buf;
 
 	git_config_with_options(cb, data, NULL, &opts);
 
-	if (!have_git_dir() && opts.git_dir) {
-		struct git_config_source repo_config;
-
-		memset(&repo_config, 0, sizeof(repo_config));
-		strbuf_addstr(&buf, "/config");
-		repo_config.file = buf.buf;
-		git_config_with_options(cb, data, &repo_config, &opts);
-	}
 	strbuf_release(&buf);
-	free(to_free);
 }
 
 static void git_config_check_init(void);
t/helper/test-config.c +4 -0
index 8e3ed6a76c..696d0a52fd 100644
@@ -84,6 +84,10 @@ int cmd_main(int argc, const char **argv)
 	struct config_set cs;
 
 	if (argc == 3 && !strcmp(argv[1], "read_early_config")) {
+		const char *cmdline_config = getenv("CMDL_CFG");
+
+		if (cmdline_config)
+			git_config_push_parameter(cmdline_config);
 		read_early_config(early_config_cb, (void *)argv[2]);
 		return 0;
 	}
t/t1309-early-config.sh +17 -0
index b97357b8ab..c15f18dd96 100755
@@ -47,6 +47,23 @@ test_expect_success 'ceiling #2' '
 	test xdg = "$(cat output)"
 '
 
+test_expect_success 'read config file in right order' '
+	echo "[test]source = home" >>.gitconfig &&
+	git init foo &&
+	(
+		cd foo &&
+		echo "[test]source = repo" >>.git/config &&
+		CMDL_CFG=test.source=cmdline test-config \
+			read_early_config test.source >actual &&
+		cat >expected <<-\EOF &&
+		home
+		repo
+		cmdline
+		EOF
+		test_cmp expected actual
+	)
+'
+
 test_with_config () {
 	rm -rf throwaway &&
 	git init throwaway &&
Signature · 1 line
2.11.0.157.gd943d85
#16 lore raw

Re: [PATCH v2 3/3] config: correct file reading order in read_early_config()

t/helper/test-config.c int cmd_main(int argc, const char **argv) · line 90

Quoted 12 lines · Nguyễn Thái Ngọc Duy
t/helper/test-config.c +4 -0
index 8e3ed6a76c..696d0a52fd 100644
@@ -84,6 +84,10 @@ int cmd_main(int argc, const char **argv)
 	struct config_set cs;
 
 	if (argc == 3 && !strcmp(argv[1], "read_early_config")) {
+		const char *cmdline_config = getenv("CMDL_CFG");
+
+		if (cmdline_config)
+			git_config_push_parameter(cmdline_config);

I think you can do without this hunk by just setting:

GIT_CONFIG_PARAMETERS="'foo.bar=from-cmdline'"

(note the single-quotes which must be there). See how t1308 does it.

Quoted 6 lines
t/helper/test-config.c +6 -0
+test_expect_success 'read config file in right order' '
+	echo "[test]source = home" >>.gitconfig &&
+	git init foo &&
+	(
+		cd foo &&
+		echo "[test]source = repo" >>.git/config &&

t/helper/test-config.c int cmd_main(int argc, const char **argv)

Quoted 10 lines
t/helper/test-config.c +10 -0
+		CMDL_CFG=test.source=cmdline test-config \
+			read_early_config test.source >actual &&
+		cat >expected <<-\EOF &&
+		home
+		repo
+		cmdline
+		EOF
+		test_cmp expected actual
+	)
+'

This looks good (modulo the CMDL_CFG above).

If we wanted to trigger it in a real-world test, we'd have to use pager config (since it's the only thing that uses early-config; alias lookup probably should do, but that's for another time). But I think this synthetic test is fine; it makes the output easy to verify.

-Peff

#17 lore raw

Re: [PATCH v2 1/3] config: prepare to pass more info in git_config_with_options()

In reply to Nguyễn Thái Ngọc Duy
Quoted 1 line · Nguyễn Thái Ngọc Duy

So far we can only pass one flag, respect_includes, to thie function. We

s/thie/this/

Quoted 25 lines

need to pass some more (non-flag even), so let's make it accept a struct instead of an integer.

Signed-off-by Nguyễn Thái Ngọc Duy <[email protected]>
---
 builtin/config.c | 21 ++++++++++++---------
 cache.h          |  7 ++++++-
 config.c         | 16 +++++++++++-----
 3 files changed, 29 insertions(+), 15 deletions(-)
builtin/config.c +2 -1
index 4f49a0edb9..b937d175a9 100644
@@ -26,7 +26,8 @@ static int use_global_config, use_system_config, use_local_config;
 static struct git_config_source given_config_source;
 static int actions, types;
 static int end_null;
-static int respect_includes = -1;
+static int respect_includes_opt = -1;
+static struct config_options config_options;
 static int show_origin;
 
 #define ACTION_GET (1<<0)

builtin/config.c static struct option builtin_config_options[] = { · line 85

Quoted 5 lines
builtin/config.c +1 -1
@@ -81,7 +82,7 @@ static struct option builtin_config_options[] = {
 	OPT_GROUP(N_("Other")),
 	OPT_BOOL('z', "null", &end_null, N_("terminate values with NUL byte")),
 	OPT_BOOL(0, "name-only", &omit_values, N_("show variable names only")),
-	OPT_BOOL(0, "includes", &respect_includes, N_("respect include directives on lookup")),
+	OPT_BOOL(0, "includes", &respect_includes_opt, N_("respect include directives on lookup")),

It would be more in line with what the log message advertised if you did

	static struct config_options config_options = {
		-1, /* .respect_includes: unspecified */
	};

	OPT_BOOL(0, "includes", &config_options.respect_includes, N_("...")),

no?

#18 lore raw

Re: [PATCH v2 1/3] config: prepare to pass more info in git_config_with_options()

Quoted 10 lines · Junio C Hamano

builtin/config.c static struct option builtin_config_options[] = { · line 85

Quoted 6 lines
builtin/config.c +1 -1
@@ -81,7 +82,7 @@ static struct option builtin_config_options[] = {
 	OPT_GROUP(N_("Other")),
 	OPT_BOOL('z', "null", &end_null, N_("terminate values with NUL byte")),
 	OPT_BOOL(0, "name-only", &omit_values, N_("show variable names only")),
-	OPT_BOOL(0, "includes", &respect_includes, N_("respect include directives on lookup")),
+	OPT_BOOL(0, "includes", &respect_includes_opt, N_("respect include directives on lookup")),

It would be more in line with what the log message advertised if you did

Quoted 7 lines
	static struct config_options config_options = {
		-1, /* .respect_includes: unspecified */
	};

	OPT_BOOL(0, "includes", &config_options.respect_includes, N_("...")),

no?

I think I like the split between the option-value here and the "final" value that goes into config_options.respect_includes. Because we actually munge it later based on the given-config value anyway.

So I agree this makes the diff larger than it might need to be, but I think the end result is a bit nicer.

-Peff

#19 lore raw

Re: [PATCH v2 1/3] config: prepare to pass more info in git_config_with_options()

Quoted 18 lines · Jeff King
Quoted 16 lines · Junio C Hamano

builtin/config.c static struct option builtin_config_options[] = { · line 85

Quoted 6 lines
builtin/config.c +1 -1
@@ -81,7 +82,7 @@ static struct option builtin_config_options[] = {
 	OPT_GROUP(N_("Other")),
 	OPT_BOOL('z', "null", &end_null, N_("terminate values with NUL byte")),
 	OPT_BOOL(0, "name-only", &omit_values, N_("show variable names only")),
-	OPT_BOOL(0, "includes", &respect_includes, N_("respect include directives on lookup")),
+	OPT_BOOL(0, "includes", &respect_includes_opt, N_("respect include directives on lookup")),

It would be more in line with what the log message advertised if you did

	static struct config_options config_options = {
		-1, /* .respect_includes: unspecified */
	};

	OPT_BOOL(0, "includes", &config_options.respect_includes, N_("...")),
Quoted 8 lines
Quoted 1 line

no?

I think I like the split between the option-value here and the "final" value that goes into config_options.respect_includes. Because we actually munge it later based on the given-config value anyway.

So I agree this makes the diff larger than it might need to be, but I think the end result is a bit nicer.

Yeah, I didn't see the end result was a single bit (unsigned :1). This separation is OK.

#20 lore raw

Re: [PATCH 2/2] config: handle conditional include when $GIT_DIR is not set up

In reply to Duy Nguyen
Quoted 3 lines · Duy Nguyen

(To Junio, this series conflicts slightly with nd/conditional-config-include, let me know if you want me to rebase this on top of that)

I think I can manage---having to resolve inter-topic conflict every once in a while is a good sanity check for me anyway ;-). Please holler if my resolution is faulty when I push things out tonight my time (note: I'll be operating in GMT+9 for coming weeks).

Thanks.

#21 lore raw

Re: [PATCH 2/2] config: handle conditional include when $GIT_DIR is not set up

In reply to Duy Nguyen
Quoted 10 lines · Duy Nguyen
Quoted 10 lines

You can see the second problem with:

  # random external
  cat >git-foo <<-\EOF
  #!/bin/sh
  echo foo
  EOF
  chmod +x git-foo
  PATH=$PWD:$PATH
Quoted 8 lines
Quoted 5 lines

git init
git config pager.foo 'sed s/^/repo:/'
git -c pager.foo='sed s/^/cmdline:/' foo

That command should prefer the cmdline config, but it doesn't.

I actually had problems seeing the problem, for some reason it didn't work for me. I guess I made a mistake somewhere.

I just double-checked it, and the above works for me. <shrug>

Quoted 8 lines
Quoted 3 lines

The fix is something like what's below, which is easy on top of your new options struct. I can wrap it up with a config message and test on top of your series.

... anyway I read this last sentence too late and spent too much time wrapping your changes in a patch (well most of the time was spent on writing a new test actually), so I'm sending it out too. My test uses test-config though (I have given up on dealing with pager and tty).

Thanks for doing that. I had planned to do it myself to avoid making work for you, but what you posted looks good.

I hadn't thought about tty issues. You'd have to run it under test_terminal (which isn't even available everywhere). So your test-config version is probably the best bet.

Quoted 8 lines

Off topic. Back to the pager.foo thing (because I had to fight it a bit before giving up). I find it a bit unintuitive that "--paginate" forces the pager back to less (or $PAGER) when I say "pager.foo = my-favorite-pager". Back when pager.foo is a boolean thing, it makes sense for --paginate to override the "to page or not to page" decision. But then you added a new meaning too pager.foo (what command to run). "--paginate" should respect the command pager.foo specifies when its value is a command, I think.

I never noticed that before, and I agree that it's nonsense. Using "-p" should be unnecessary if you have pager.foo defined (because it already implies "yes, use the pager"). But if you were to do it anyway, I agree it should not ignore the configured pager. One might use an extra "-p" hoping that it would override the tty check, but it doesn't. It's too late to change that now, I think, but I have been tempted to add a --paginate-me-harder option. It would be handy for tests.

-Peff

#22 lore raw

Re: [PATCH 1/2] config: prepare to pass more info in git_config_with_options()

In reply to Nguyễn Thái Ngọc Duy
Quoted 3 lines · Nguyễn Thái Ngọc Duy

So far we can only pass one flag, respect_includes, to thie function. We need to pass some more (non-flag even), so let's make it accept a struct instead of an integer.

Yeah, this makes sense. But...

builtin/config.c static int use_global_config, use_system_config, use_local_config; · line 30

Quoted 11 lines
builtin/config.c +2 -1
index 4f49a0edb9..5de4a36146 100644
@@ -26,7 +26,8 @@ static int use_global_config, use_system_config, use_local_config;
 static struct git_config_source given_config_source;
 static int actions, types;
 static int end_null;
-static int respect_includes = -1;
+static int respect_includes_opt;
+static struct config_options config_options;

It fails all the git-config "include" tests because respect_includes_opt is missing the initialization to "-1".

-Peff

#23 lore raw

Re: [PATCH 1/2] config: prepare to pass more info in git_config_with_options()

Quoted 8 lines · Jeff King
Quoted 3 lines · Nguyễn Thái Ngọc Duy

So far we can only pass one flag, respect_includes, to thie function. We need to pass some more (non-flag even), so let's make it accept a struct instead of an integer.

Yeah, this makes sense. But...

Quoted 14 lines

builtin/config.c static int use_global_config, use_system_config, use_local_config; · line 30

Quoted 11 lines
builtin/config.c +2 -1
index 4f49a0edb9..5de4a36146 100644
@@ -26,7 +26,8 @@ static int use_global_config, use_system_config, use_local_config;
 static struct git_config_source given_config_source;
 static int actions, types;
 static int end_null;
-static int respect_includes = -1;
+static int respect_includes_opt;
+static struct config_options config_options;

It fails all the git-config "include" tests because respect_includes_opt is missing the initialization to "-1".

Serves me right for doing last-minute "harmless" refactoring without rerunning the test suite :(

Quoted 2 lines

-Peff