Skip to contentSkip to conversation

rebase: understand -C again, refactor

PATCH Superseded Johannes Schindelin via GitGitGadget 14 participants 97 messages Nov 13, 2018 – Dec 4, 2018 Documentation

lore
Share & tools
lore
#0 lore raw

cover rebase: understand -C again, refactor

Phillip Wood reported a problem where the built-in rebase did not understand options like -C1, i.e. it did not expect the option argument.

While investigating how to address this best, I stumbled upon OPT_PASSTHRU_ARGV (which I was so far happily unaware of).

Instead of just fixing the -C<n> bug, I decided to simply convert all of the options intended for git am (or, eventually, for git apply). This happens to fix that bug, and does so much more: it simplifies the entire logic (and removes more lines than it adds).

Johannes Schindelin (1):
rebase: really just passthru the git am options

 builtin/rebase.c | 98 +++++++++++++++++-------------------------------
 1 file changed, 35 insertions(+), 63 deletions(-)

base-commit: 8858448bb49332d353febc078ce4a3abcc962efe
Published-As: https://github.com/gitgitgadget/git/releases/tags/pr-76%2Fdscho%2Frebase-Cn-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-76/dscho/rebase-Cn-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/76

Signature · 1 line
gitgitgadget
#1 lore raw

PATCH 1/1 rebase: really just passthru the `git am` options

From: Johannes Schindelin <[email protected]>

Currently, we parse the options intended for git am as if we wanted to handle them in git rebase, and then reconstruct them painstakingly to define the git_am_opt variable.

However, there is a much better way (that I was unaware of, at the time when I mentored Pratik to implement these options): OPT_PASSTHRU_ARGV. It is intended for exactly this use case, where command-line options want to be parsed into a separate argv_array.

Let's use this feature.

Incidentally, this also allows us to address a bug discovered by Phillip Wood, where the built-in rebase failed to understand that the -C option takes an optional argument.

---
 builtin/rebase.c | 98 +++++++++++++++++-------------------------------
 1 file changed, 35 insertions(+), 63 deletions(-)
builtin/rebase.c +35 -63
index 0ee06aa363..96ffa80b71 100644
@@ -87,7 +87,7 @@ struct rebase_options {
 		REBASE_FORCE = 1<<3,
 		REBASE_INTERACTIVE_EXPLICIT = 1<<4,
 	} flags;
-	struct strbuf git_am_opt;
+	struct argv_array git_am_opts;
 	const char *action;
 	int signoff;
 	int allow_rerere_autoupdate;
@@ -339,7 +339,7 @@ N_("Resolve all conflicts manually, mark them as resolved with\n"
 static int run_specific_rebase(struct rebase_options *opts)
 {
 	const char *argv[] = { NULL, NULL };
-	struct strbuf script_snippet = STRBUF_INIT;
+	struct strbuf script_snippet = STRBUF_INIT, buf = STRBUF_INIT;
 	int status;
 	const char *backend, *backend_func;
 
@@ -433,7 +433,9 @@ static int run_specific_rebase(struct rebase_options *opts)
 		oid_to_hex(&opts->restrict_revision->object.oid) : NULL);
 	add_var(&script_snippet, "GIT_QUIET",
 		opts->flags & REBASE_NO_QUIET ? "" : "t");
-	add_var(&script_snippet, "git_am_opt", opts->git_am_opt.buf);
+	sq_quote_argv_pretty(&buf, opts->git_am_opts.argv);
+	add_var(&script_snippet, "git_am_opt", buf.buf);
+	strbuf_release(&buf);
 	add_var(&script_snippet, "verbose",
 		opts->flags & REBASE_VERBOSE ? "t" : "");
 	add_var(&script_snippet, "diffstat",
@@ -756,7 +758,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	struct rebase_options options = {
 		.type = REBASE_UNSPECIFIED,
 		.flags = REBASE_NO_QUIET,
-		.git_am_opt = STRBUF_INIT,
+		.git_am_opts = ARGV_ARRAY_INIT,
 		.allow_rerere_autoupdate  = -1,
 		.allow_empty_message = 1,
 		.git_format_patch_opt = STRBUF_INIT,
@@ -777,12 +779,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		ACTION_EDIT_TODO,
 		ACTION_SHOW_CURRENT_PATCH,
 	} action = NO_ACTION;
-	int committer_date_is_author_date = 0;
-	int ignore_date = 0;
-	int ignore_whitespace = 0;
 	const char *gpg_sign = NULL;
-	int opt_c = -1;
-	struct string_list whitespace = STRING_LIST_INIT_NODUP;
 	struct string_list exec = STRING_LIST_INIT_NODUP;
 	const char *rebase_merges = NULL;
 	int fork_point = -1;
@@ -804,15 +801,20 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		{OPTION_NEGBIT, 'n', "no-stat", &options.flags, NULL,
 			N_("do not show diffstat of what changed upstream"),
 			PARSE_OPT_NOARG, NULL, REBASE_DIFFSTAT },
-		OPT_BOOL(0, "ignore-whitespace", &ignore_whitespace,
-			 N_("passed to 'git apply'")),
 		OPT_BOOL(0, "signoff", &options.signoff,
 			 N_("add a Signed-off-by: line to each commit")),
-		OPT_BOOL(0, "committer-date-is-author-date",
-			 &committer_date_is_author_date,
-			 N_("passed to 'git am'")),
-		OPT_BOOL(0, "ignore-date", &ignore_date,
-			 N_("passed to 'git am'")),
+		OPT_PASSTHRU_ARGV(0, "ignore-whitespace", &options.git_am_opts,
+				  NULL, N_("passed to 'git am'"),
+				  PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV(0, "committer-date-is-author-date",
+				  &options.git_am_opts, NULL,
+				  N_("passed to 'git am'"), PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV(0, "ignore-date", &options.git_am_opts, NULL,
+				  N_("passed to 'git am'"), PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV('C', NULL, &options.git_am_opts, N_("n"),
+				  N_("passed to 'git apply'"), 0),
+		OPT_PASSTHRU_ARGV(0, "whitespace", &options.git_am_opts,
+				  N_("action"), N_("passed to 'git apply'"), 0),
 		OPT_BIT('f', "force-rebase", &options.flags,
 			N_("cherry-pick all commits, even if unchanged"),
 			REBASE_FORCE),
@@ -856,10 +858,6 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		{ OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"),
 			N_("GPG-sign commits"),
 			PARSE_OPT_OPTARG, NULL, (intptr_t) "" },
-		OPT_STRING_LIST(0, "whitespace", &whitespace,
-				N_("whitespace"), N_("passed to 'git apply'")),
-		OPT_SET_INT('C', NULL, &opt_c, N_("passed to 'git apply'"),
-			    REBASE_AM),
 		OPT_BOOL(0, "autostash", &options.autostash,
 			 N_("automatically stash/stash pop before and after")),
 		OPT_STRING_LIST('x', "exec", &exec, N_("exec"),
@@ -884,6 +882,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 			 N_("rebase all reachable commits up to the root(s)")),
 		OPT_END(),
 	};
+	int i;
 
 	/*
 	 * NEEDSWORK: Once the builtin rebase has been tested enough
@@ -1064,22 +1063,17 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		    state_dir_base, cmd_live_rebase, buf.buf);
 	}
 
-	if (!(options.flags & REBASE_NO_QUIET))
-		strbuf_addstr(&options.git_am_opt, " -q");
-
-	if (committer_date_is_author_date) {
-		strbuf_addstr(&options.git_am_opt,
-			      " --committer-date-is-author-date");
-		options.flags |= REBASE_FORCE;
+	for (i = 0; i < options.git_am_opts.argc; i++) {
+		const char *option = options.git_am_opts.argv[i];
+		if (!strcmp(option, "--committer-date-is-author-date") ||
+		    !strcmp(option, "--ignore-date") ||
+		    !strcmp(option, "--whitespace=fix") ||
+		    !strcmp(option, "--whitespace=strip"))
+			options.flags |= REBASE_FORCE;
 	}
 
-	if (ignore_whitespace)
-		strbuf_addstr(&options.git_am_opt, " --ignore-whitespace");
-
-	if (ignore_date) {
-		strbuf_addstr(&options.git_am_opt, " --ignore-date");
-		options.flags |= REBASE_FORCE;
-	}
+	if (!(options.flags & REBASE_NO_QUIET))
+		argv_array_push(&options.git_am_opts, "-q");
 
 	if (options.keep_empty)
 		imply_interactive(&options, "--keep-empty");
@@ -1089,23 +1083,6 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		options.gpg_sign_opt = xstrfmt("-S%s", gpg_sign);
 	}
 
-	if (opt_c >= 0)
-		strbuf_addf(&options.git_am_opt, " -C%d", opt_c);
-
-	if (whitespace.nr) {
-		int i;
-
-		for (i = 0; i < whitespace.nr; i++) {
-			const char *item = whitespace.items[i].string;
-
-			strbuf_addf(&options.git_am_opt, " --whitespace=%s",
-				    item);
-
-			if ((!strcmp(item, "fix")) || (!strcmp(item, "strip")))
-				options.flags |= REBASE_FORCE;
-		}
-	}
-
 	if (exec.nr) {
 		int i;
 
@@ -1181,23 +1158,18 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		break;
 	}
 
-	if (options.git_am_opt.len) {
-		const char *p;
-
+	if (options.git_am_opts.argc) {
 		/* all am options except -q are compatible only with --am */
-		strbuf_reset(&buf);
-		strbuf_addbuf(&buf, &options.git_am_opt);
-		strbuf_addch(&buf, ' ');
-		while ((p = strstr(buf.buf, " -q ")))
-			strbuf_splice(&buf, p - buf.buf, 4, " ", 1);
-		strbuf_trim(&buf);
+		for (i = options.git_am_opts.argc - 1; i >= 0; i--)
+			if (strcmp(options.git_am_opts.argv[i], "-q"))
+				break;
 
-		if (is_interactive(&options) && buf.len)
+		if (is_interactive(&options) && i >= 0)
 			die(_("error: cannot combine interactive options "
 			      "(--interactive, --exec, --rebase-merges, "
 			      "--preserve-merges, --keep-empty, --root + "
 			      "--onto) with am options (%s)"), buf.buf);
-		if (options.type == REBASE_MERGE && buf.len)
+		if (options.type == REBASE_MERGE && i >= 0)
 			die(_("error: cannot combine merge options (--merge, "
 			      "--strategy, --strategy-option) with am options "
 			      "(%s)"), buf.buf);
@@ -1207,7 +1179,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		if (options.type == REBASE_PRESERVE_MERGES)
 			die("cannot combine '--signoff' with "
 			    "'--preserve-merges'");
-		strbuf_addstr(&options.git_am_opt, " --signoff");
+		argv_array_push(&options.git_am_opts, "--signoff");
 		options.flags |= REBASE_FORCE;
 	}
 
Signature · 1 line
gitgitgadget
#2 lore raw

Re: [PATCH 1/1] rebase: really just passthru the `git am` options

"Johannes Schindelin via GitGitGadget" <[email protected]>

Quoted 10 lines · writes:

However, there is a much better way (that I was unaware of, at the time when I mentored Pratik to implement these options): OPT_PASSTHRU_ARGV. It is intended for exactly this use case, where command-line options want to be parsed into a separate argv_array.

Let's use this feature.

Incidentally, this also allows us to address a bug discovered by Phillip Wood, where the built-in rebase failed to understand that the -C option takes an optional argument.

The resulting code does show what is going on more clearly. I wonder if we can do the same for -S as well? It needs to be passed to other backends, so I guess it is not such a good idea.

builtin/rebase.c int cmd_rebase(int argc, const char **argv, const char *prefix) · line 761

Quoted 2 lines
builtin/rebase.c +1 -1
-		.git_am_opt = STRBUF_INIT,
+		.git_am_opts = ARGV_ARRAY_INIT,

Yes, this is much nicer.

Quoted 12 lines
builtin/rebase.c +0 -4
@@ -856,10 +858,6 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		{ OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"),
 			N_("GPG-sign commits"),
 			PARSE_OPT_OPTARG, NULL, (intptr_t) "" },
-		OPT_STRING_LIST(0, "whitespace", &whitespace,
-				N_("whitespace"), N_("passed to 'git apply'")),
-		OPT_SET_INT('C', NULL, &opt_c, N_("passed to 'git apply'"),
-			    REBASE_AM),
 		OPT_BOOL(0, "autostash", &options.autostash,
 			 N_("automatically stash/stash pop before and after")),
 		OPT_STRING_LIST('x', "exec", &exec, N_("exec"),

builtin/rebase.c int cmd_rebase(int argc, const char **argv, const char *prefix) · line 887

Quoted 6 lines
builtin/rebase.c +2 -0
@@ -884,6 +882,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 			 N_("rebase all reachable commits up to the root(s)")),
 		OPT_END(),
 	};
+	int i;
 
+	for (i = 0; i < options.git_am_opts.argc; i++) {

Yes, this is very nice way to first collect and then inspect them separately.

builtin/rebase.c int cmd_rebase(int argc, const char **argv, const char *prefix) · line 1073

Quoted 7 lines
builtin/rebase.c +6 -0
+		const char *option = options.git_am_opts.argv[i];
+		if (!strcmp(option, "--committer-date-is-author-date") ||
+		    !strcmp(option, "--ignore-date") ||
+		    !strcmp(option, "--whitespace=fix") ||
+		    !strcmp(option, "--whitespace=strip"))
+			options.flags |= REBASE_FORCE;
 	}

Overall very nicely done. Perhaps we'll see a test or two from Philip?

Thanks.

#3 lore raw

Re: [PATCH 1/1] rebase: really just passthru the `git am` options

In reply to Johannes Schindelin via GitGitGadget

Hi Johannes

Thanks for looking at this. Unfortunately using OPT_PASSTHRU_ARGV seems to break the error reporting

Running
bin/wrappers/git rebase --onto @^^^^ @^^ -Cbad

Gives
git encountered an error while preparing the patches to replay these revisions:

67f673aa4a580b9e407b1ca505abf1f50510ec47...7c3e01a708856885e60bf4051586970e65dd326c

As a result, git cannot rebase them.

If I do

bin/wrappers/git rebase @^^ -Cbad

I get no error, it just tells me that it does not need to rebase (which is true)

Best Wishes

Phillip

Quoted 220 lines · Johannes Schindelin via GitGitGadget

From: Johannes Schindelin <[email protected]>

Currently, we parse the options intended for git am as if we wanted to handle them in git rebase, and then reconstruct them painstakingly to define the git_am_opt variable.

However, there is a much better way (that I was unaware of, at the time when I mentored Pratik to implement these options): OPT_PASSTHRU_ARGV. It is intended for exactly this use case, where command-line options want to be parsed into a separate argv_array.

Let's use this feature.

Incidentally, this also allows us to address a bug discovered by Phillip Wood, where the built-in rebase failed to understand that the -C option takes an optional argument.

---
  builtin/rebase.c | 98 +++++++++++++++++-------------------------------
  1 file changed, 35 insertions(+), 63 deletions(-)
builtin/rebase.c +35 -63
index 0ee06aa363..96ffa80b71 100644
@@ -87,7 +87,7 @@ struct rebase_options {
  		REBASE_FORCE = 1<<3,
  		REBASE_INTERACTIVE_EXPLICIT = 1<<4,
  	} flags;
-	struct strbuf git_am_opt;
+	struct argv_array git_am_opts;
  	const char *action;
  	int signoff;
  	int allow_rerere_autoupdate;
@@ -339,7 +339,7 @@ N_("Resolve all conflicts manually, mark them as resolved with\n"
  static int run_specific_rebase(struct rebase_options *opts)
  {
  	const char *argv[] = { NULL, NULL };
-	struct strbuf script_snippet = STRBUF_INIT;
+	struct strbuf script_snippet = STRBUF_INIT, buf = STRBUF_INIT;
  	int status;
  	const char *backend, *backend_func;
 
@@ -433,7 +433,9 @@ static int run_specific_rebase(struct rebase_options *opts)
  		oid_to_hex(&opts->restrict_revision->object.oid) : NULL);
  	add_var(&script_snippet, "GIT_QUIET",
  		opts->flags & REBASE_NO_QUIET ? "" : "t");
-	add_var(&script_snippet, "git_am_opt", opts->git_am_opt.buf);
+	sq_quote_argv_pretty(&buf, opts->git_am_opts.argv);
+	add_var(&script_snippet, "git_am_opt", buf.buf);
+	strbuf_release(&buf);
  	add_var(&script_snippet, "verbose",
  		opts->flags & REBASE_VERBOSE ? "t" : "");
  	add_var(&script_snippet, "diffstat",
@@ -756,7 +758,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
  	struct rebase_options options = {
  		.type = REBASE_UNSPECIFIED,
  		.flags = REBASE_NO_QUIET,
-		.git_am_opt = STRBUF_INIT,
+		.git_am_opts = ARGV_ARRAY_INIT,
  		.allow_rerere_autoupdate  = -1,
  		.allow_empty_message = 1,
  		.git_format_patch_opt = STRBUF_INIT,
@@ -777,12 +779,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
  		ACTION_EDIT_TODO,
  		ACTION_SHOW_CURRENT_PATCH,
  	} action = NO_ACTION;
-	int committer_date_is_author_date = 0;
-	int ignore_date = 0;
-	int ignore_whitespace = 0;
  	const char *gpg_sign = NULL;
-	int opt_c = -1;
-	struct string_list whitespace = STRING_LIST_INIT_NODUP;
  	struct string_list exec = STRING_LIST_INIT_NODUP;
  	const char *rebase_merges = NULL;
  	int fork_point = -1;
@@ -804,15 +801,20 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
  		{OPTION_NEGBIT, 'n', "no-stat", &options.flags, NULL,
  			N_("do not show diffstat of what changed upstream"),
  			PARSE_OPT_NOARG, NULL, REBASE_DIFFSTAT },
-		OPT_BOOL(0, "ignore-whitespace", &ignore_whitespace,
-			 N_("passed to 'git apply'")),
  		OPT_BOOL(0, "signoff", &options.signoff,
  			 N_("add a Signed-off-by: line to each commit")),
-		OPT_BOOL(0, "committer-date-is-author-date",
-			 &committer_date_is_author_date,
-			 N_("passed to 'git am'")),
-		OPT_BOOL(0, "ignore-date", &ignore_date,
-			 N_("passed to 'git am'")),
+		OPT_PASSTHRU_ARGV(0, "ignore-whitespace", &options.git_am_opts,
+				  NULL, N_("passed to 'git am'"),
+				  PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV(0, "committer-date-is-author-date",
+				  &options.git_am_opts, NULL,
+				  N_("passed to 'git am'"), PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV(0, "ignore-date", &options.git_am_opts, NULL,
+				  N_("passed to 'git am'"), PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV('C', NULL, &options.git_am_opts, N_("n"),
+				  N_("passed to 'git apply'"), 0),
+		OPT_PASSTHRU_ARGV(0, "whitespace", &options.git_am_opts,
+				  N_("action"), N_("passed to 'git apply'"), 0),
  		OPT_BIT('f', "force-rebase", &options.flags,
  			N_("cherry-pick all commits, even if unchanged"),
  			REBASE_FORCE),
@@ -856,10 +858,6 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
  		{ OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"),
  			N_("GPG-sign commits"),
  			PARSE_OPT_OPTARG, NULL, (intptr_t) "" },
-		OPT_STRING_LIST(0, "whitespace", &whitespace,
-				N_("whitespace"), N_("passed to 'git apply'")),
-		OPT_SET_INT('C', NULL, &opt_c, N_("passed to 'git apply'"),
-			    REBASE_AM),
  		OPT_BOOL(0, "autostash", &options.autostash,
  			 N_("automatically stash/stash pop before and after")),
  		OPT_STRING_LIST('x', "exec", &exec, N_("exec"),
@@ -884,6 +882,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
  			 N_("rebase all reachable commits up to the root(s)")),
  		OPT_END(),
  	};
+	int i;
 
  	/*
  	 * NEEDSWORK: Once the builtin rebase has been tested enough
@@ -1064,22 +1063,17 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
  		    state_dir_base, cmd_live_rebase, buf.buf);
  	}
 
-	if (!(options.flags & REBASE_NO_QUIET))
-		strbuf_addstr(&options.git_am_opt, " -q");
-
-	if (committer_date_is_author_date) {
-		strbuf_addstr(&options.git_am_opt,
-			      " --committer-date-is-author-date");
-		options.flags |= REBASE_FORCE;
+	for (i = 0; i < options.git_am_opts.argc; i++) {
+		const char *option = options.git_am_opts.argv[i];
+		if (!strcmp(option, "--committer-date-is-author-date") ||
+		    !strcmp(option, "--ignore-date") ||
+		    !strcmp(option, "--whitespace=fix") ||
+		    !strcmp(option, "--whitespace=strip"))
+			options.flags |= REBASE_FORCE;
  	}
 
-	if (ignore_whitespace)
-		strbuf_addstr(&options.git_am_opt, " --ignore-whitespace");
-
-	if (ignore_date) {
-		strbuf_addstr(&options.git_am_opt, " --ignore-date");
-		options.flags |= REBASE_FORCE;
-	}
+	if (!(options.flags & REBASE_NO_QUIET))
+		argv_array_push(&options.git_am_opts, "-q");
 
  	if (options.keep_empty)
  		imply_interactive(&options, "--keep-empty");
@@ -1089,23 +1083,6 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
  		options.gpg_sign_opt = xstrfmt("-S%s", gpg_sign);
  	}
 
-	if (opt_c >= 0)
-		strbuf_addf(&options.git_am_opt, " -C%d", opt_c);
-
-	if (whitespace.nr) {
-		int i;
-
-		for (i = 0; i < whitespace.nr; i++) {
-			const char *item = whitespace.items[i].string;
-
-			strbuf_addf(&options.git_am_opt, " --whitespace=%s",
-				    item);
-
-			if ((!strcmp(item, "fix")) || (!strcmp(item, "strip")))
-				options.flags |= REBASE_FORCE;
-		}
-	}
-
  	if (exec.nr) {
  		int i;
 
@@ -1181,23 +1158,18 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
  		break;
  	}
 
-	if (options.git_am_opt.len) {
-		const char *p;
-
+	if (options.git_am_opts.argc) {
  		/* all am options except -q are compatible only with --am */
-		strbuf_reset(&buf);
-		strbuf_addbuf(&buf, &options.git_am_opt);
-		strbuf_addch(&buf, ' ');
-		while ((p = strstr(buf.buf, " -q ")))
-			strbuf_splice(&buf, p - buf.buf, 4, " ", 1);
-		strbuf_trim(&buf);
+		for (i = options.git_am_opts.argc - 1; i >= 0; i--)
+			if (strcmp(options.git_am_opts.argv[i], "-q"))
+				break;
 
-		if (is_interactive(&options) && buf.len)
+		if (is_interactive(&options) && i >= 0)
  			die(_("error: cannot combine interactive options "
  			      "(--interactive, --exec, --rebase-merges, "
  			      "--preserve-merges, --keep-empty, --root + "
  			      "--onto) with am options (%s)"), buf.buf);
-		if (options.type == REBASE_MERGE && buf.len)
+		if (options.type == REBASE_MERGE && i >= 0)
  			die(_("error: cannot combine merge options (--merge, "
  			      "--strategy, --strategy-option) with am options "
  			      "(%s)"), buf.buf);
@@ -1207,7 +1179,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
  		if (options.type == REBASE_PRESERVE_MERGES)
  			die("cannot combine '--signoff' with "
  			    "'--preserve-merges'");
-		strbuf_addstr(&options.git_am_opt, " --signoff");
+		argv_array_push(&options.git_am_opts, "--signoff");
  		options.flags |= REBASE_FORCE;
  	}
 
#4 lore raw

Re: [PATCH 1/1] rebase: really just passthru the `git am` options

Hi Phillip,

Quoted 13 lines · Phillip Wood

Thanks for looking at this. Unfortunately using OPT_PASSTHRU_ARGV seems to break the error reporting

Running
bin/wrappers/git rebase --onto @^^^^ @^^ -Cbad

Gives
git encountered an error while preparing the patches to replay these revisions:

67f673aa4a580b9e407b1ca505abf1f50510ec47...7c3e01a708856885e60bf4051586970e65dd326c

Quoted 8 lines

As a result, git cannot rebase them.

If I do

bin/wrappers/git rebase @^^ -Cbad

I get no error, it just tells me that it does not need to rebase (which is true)

Hmm. Isn't this the same behavior as with the scripted version? Also: are we sure that we want to allow options to come *after* the <upstream> argument?

Ciao,
Dscho

Quoted 239 lines

Best Wishes

Phillip

Quoted 231 lines · Johannes Schindelin via GitGitGadget

From: Johannes Schindelin <[email protected]>

Currently, we parse the options intended for git am as if we wanted to handle them in git rebase, and then reconstruct them painstakingly to define the git_am_opt variable.

However, there is a much better way (that I was unaware of, at the time when I mentored Pratik to implement these options): OPT_PASSTHRU_ARGV. It is intended for exactly this use case, where command-line options want to be parsed into a separate argv_array.

Let's use this feature.

Incidentally, this also allows us to address a bug discovered by Phillip Wood, where the built-in rebase failed to understand that the -C option takes an optional argument.

---
  builtin/rebase.c | 98 +++++++++++++++++-------------------------------
  1 file changed, 35 insertions(+), 63 deletions(-)
builtin/rebase.c +1 -1
index 0ee06aa363..96ffa80b71 100644
@@ -87,7 +87,7 @@ struct rebase_options {
    REBASE_FORCE = 1<<3,
    REBASE_INTERACTIVE_EXPLICIT = 1<<4,
  	} flags;
-	struct strbuf git_am_opt;
+	struct argv_array git_am_opts;
   const char *action;
   int signoff;
   int allow_rerere_autoupdate;
@@ -339,7 +339,7 @@ N_("Resolve all conflicts manually, mark them as

resolved with\n"
static int run_specific_rebase(struct rebase_options *opts) {
const char *argv[] = { NULL, NULL };

builtin/rebase.c +1 -1
-	struct strbuf script_snippet = STRBUF_INIT;
+	struct strbuf script_snippet = STRBUF_INIT, buf = STRBUF_INIT;
   int status;
   const char *backend, *backend_func;
  @@ -433,7 +433,9 @@ static int run_specific_rebase(struct rebase_options

*opts)
oid_to_hex(&opts->restrict_revision->object.oid) : NULL);
add_var(&script_snippet, "GIT_QUIET",
opts->flags & REBASE_NO_QUIET ? "" : "t");

builtin/rebase.c +4 -2
-	add_var(&script_snippet, "git_am_opt", opts->git_am_opt.buf);
+	sq_quote_argv_pretty(&buf, opts->git_am_opts.argv);
+	add_var(&script_snippet, "git_am_opt", buf.buf);
+	strbuf_release(&buf);
   add_var(&script_snippet, "verbose",
   	opts->flags & REBASE_VERBOSE ? "t" : "");
  	add_var(&script_snippet, "diffstat",
@@ -756,7 +758,7 @@ int cmd_rebase(int argc, const char **argv, const char
 *prefix)
   struct rebase_options options = {
    .type = REBASE_UNSPECIFIED,
    .flags = REBASE_NO_QUIET,
-		.git_am_opt = STRBUF_INIT,
+		.git_am_opts = ARGV_ARRAY_INIT,
    .allow_rerere_autoupdate  = -1,
    .allow_empty_message = 1,

.git_format_patch_opt = STRBUF_INIT,

builtin/rebase.c +0 -5
@@ -777,12 +779,7 @@ int cmd_rebase(int argc, const char **argv, const
 char *prefix)
    ACTION_EDIT_TODO,
    ACTION_SHOW_CURRENT_PATCH,
  	} action = NO_ACTION;
-	int committer_date_is_author_date = 0;
-	int ignore_date = 0;
-	int ignore_whitespace = 0;
  	const char *gpg_sign = NULL;
-	int opt_c = -1;
-	struct string_list whitespace = STRING_LIST_INIT_NODUP;
   struct string_list exec = STRING_LIST_INIT_NODUP;
   const char *rebase_merges = NULL;
   int fork_point = -1;
builtin/rebase.c +7 -7
@@ -804,15 +801,20 @@ int cmd_rebase(int argc, const char **argv, const
 char *prefix)
    {OPTION_NEGBIT, 'n', "no-stat", &options.flags, NULL,
     N_("do not show diffstat of what changed upstream"),
     PARSE_OPT_NOARG, NULL, REBASE_DIFFSTAT },
-		OPT_BOOL(0, "ignore-whitespace", &ignore_whitespace,
-			 N_("passed to 'git apply'")),
    OPT_BOOL(0, "signoff", &options.signoff,
  			 N_("add a Signed-off-by: line to each commit")),
-		OPT_BOOL(0, "committer-date-is-author-date",
-			 &committer_date_is_author_date,
-			 N_("passed to 'git am'")),
-		OPT_BOOL(0, "ignore-date", &ignore_date,
-			 N_("passed to 'git am'")),
+		OPT_PASSTHRU_ARGV(0, "ignore-whitespace",
 &options.git_am_opts,
+				  NULL, N_("passed to 'git am'"),
+				  PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV(0, "committer-date-is-author-date",
+				  &options.git_am_opts, NULL,
+				  N_("passed to 'git am'"),
 PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV(0, "ignore-date", &options.git_am_opts,

NULL,

builtin/rebase.c +1 -0
+				  N_("passed to 'git am'"),

PARSE_OPT_NOARG),

builtin/rebase.c +4 -0
+		OPT_PASSTHRU_ARGV('C', NULL, &options.git_am_opts, N_("n"),
+				  N_("passed to 'git apply'"), 0),
+		OPT_PASSTHRU_ARGV(0, "whitespace", &options.git_am_opts,
+				  N_("action"), N_("passed to 'git apply'"),

0),
OPT_BIT('f', "force-rebase", &options.flags,
N_("cherry-pick all commits, even if unchanged"), REBASE_FORCE),

builtin/rebase.c +0 -4
@@ -856,10 +858,6 @@ int cmd_rebase(int argc, const char **argv, const
 char *prefix)
    { OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"),
     N_("GPG-sign commits"),
     PARSE_OPT_OPTARG, NULL, (intptr_t) "" },
-		OPT_STRING_LIST(0, "whitespace", &whitespace,
-				N_("whitespace"), N_("passed to 'git
 apply'")),
-		OPT_SET_INT('C', NULL, &opt_c, N_("passed to 'git apply'"),
-			    REBASE_AM),
    OPT_BOOL(0, "autostash", &options.autostash,

N_("automatically stash/stash pop before and after")),
OPT_STRING_LIST('x', "exec", &exec, N_("exec"),

builtin/rebase.c +1 -0
@@ -884,6 +882,7 @@ int cmd_rebase(int argc, const char **argv, const char
 *prefix)
    	 N_("rebase all reachable commits up to the root(s)")),
   	OPT_END(),
  	};
+	int i;
 
   /*

* NEEDSWORK: Once the builtin rebase has been tested enough

builtin/rebase.c +9 -28
@@ -1064,22 +1063,17 @@ int cmd_rebase(int argc, const char **argv, const
 char *prefix)
   	    state_dir_base, cmd_live_rebase, buf.buf);
   }
  -	if (!(options.flags & REBASE_NO_QUIET))
-		strbuf_addstr(&options.git_am_opt, " -q");
-
-	if (committer_date_is_author_date) {
-		strbuf_addstr(&options.git_am_opt,
-			      " --committer-date-is-author-date");
-		options.flags |= REBASE_FORCE;
+	for (i = 0; i < options.git_am_opts.argc; i++) {
+		const char *option = options.git_am_opts.argv[i];
+		if (!strcmp(option, "--committer-date-is-author-date") ||
+		    !strcmp(option, "--ignore-date") ||
+		    !strcmp(option, "--whitespace=fix") ||
+		    !strcmp(option, "--whitespace=strip"))
+			options.flags |= REBASE_FORCE;
   }
  -	if (ignore_whitespace)
-		strbuf_addstr(&options.git_am_opt, " --ignore-whitespace");
-
-	if (ignore_date) {
-		strbuf_addstr(&options.git_am_opt, " --ignore-date");
-		options.flags |= REBASE_FORCE;
-	}
+	if (!(options.flags & REBASE_NO_QUIET))
+		argv_array_push(&options.git_am_opts, "-q");
 
   if (options.keep_empty)
  		imply_interactive(&options, "--keep-empty");
@@ -1089,23 +1083,6 @@ int cmd_rebase(int argc, const char **argv, const
 char *prefix)
   	options.gpg_sign_opt = xstrfmt("-S%s", gpg_sign);
   }
  -	if (opt_c >= 0)
-		strbuf_addf(&options.git_am_opt, " -C%d", opt_c);
-
-	if (whitespace.nr) {
-		int i;
-
-		for (i = 0; i < whitespace.nr; i++) {
-			const char *item = whitespace.items[i].string;
-
-			strbuf_addf(&options.git_am_opt, " --whitespace=%s",
-				    item);
-
-			if ((!strcmp(item, "fix")) || (!strcmp(item,
 "strip")))
-				options.flags |= REBASE_FORCE;
-		}
-	}
-
   if (exec.nr) {
    int i;
  @@ -1181,23 +1158,18 @@ int cmd_rebase(int argc, const char **argv,

const char *prefix)

   	break;
   }

- if (options.git_am_opt.len) {

builtin/rebase.c +7 -10
-		const char *p;
-
+	if (options.git_am_opts.argc) {
  		/* all am options except -q are compatible only with --am */
-		strbuf_reset(&buf);
-		strbuf_addbuf(&buf, &options.git_am_opt);
-		strbuf_addch(&buf, ' ');
-		while ((p = strstr(buf.buf, " -q ")))
-			strbuf_splice(&buf, p - buf.buf, 4, " ", 1);
-		strbuf_trim(&buf);
+		for (i = options.git_am_opts.argc - 1; i >= 0; i--)
+			if (strcmp(options.git_am_opts.argv[i], "-q"))
+				break;
  -		if (is_interactive(&options) && buf.len)
+		if (is_interactive(&options) && i >= 0)
     die(_("error: cannot combine interactive options "
           "(--interactive, --exec, --rebase-merges, "
           "--preserve-merges, --keep-empty, --root + "
           "--onto) with am options (%s)"), buf.buf);
-		if (options.type == REBASE_MERGE && buf.len)
+		if (options.type == REBASE_MERGE && i >= 0)
     die(_("error: cannot combine merge options (--merge, "
           "--strategy, --strategy-option) with am options "
           "(%s)"), buf.buf);
@@ -1207,7 +1179,7 @@ int cmd_rebase(int argc, const char **argv, const
 char *prefix)
    if (options.type == REBASE_PRESERVE_MERGES)
     die("cannot combine '--signoff' with "
  			    "'--preserve-merges'");
-		strbuf_addstr(&options.git_am_opt, " --signoff");
+		argv_array_push(&options.git_am_opts, "--signoff");
   	options.flags |= REBASE_FORCE;
   }
#5 lore raw

Re: [PATCH 1/1] rebase: really just passthru the `git am` options

Hi Johannes

Quoted 19 lines · Johannes Schindelin

Hi Phillip,

Quoted 15 lines · Phillip Wood

Thanks for looking at this. Unfortunately using OPT_PASSTHRU_ARGV seems to break the error reporting

Running
bin/wrappers/git rebase --onto @^^^^ @^^ -Cbad

Gives
git encountered an error while preparing the patches to replay these revisions:

67f673aa4a580b9e407b1ca505abf1f50510ec47...7c3e01a708856885e60bf4051586970e65dd326c

As a result, git cannot rebase them.

git 2.19.1 gives

First, rewinding head to replay your work on top of it...
Applying: Ninth batch for 2.20
error: switch `C' expects a numerical value

So it has a clear message as to what the error is, this patch regresses that. It would be better if rebase detected the error before starting though.

Quoted 8 lines
Quoted 6 lines

If I do

bin/wrappers/git rebase @^^ -Cbad

I get no error, it just tells me that it does not need to rebase (which is true)

Hmm. Isn't this the same behavior as with the scripted version?

Ah you're right the script does not check if the option argument is valid.

Quoted 3 lines

Also: are
we sure that we want to allow options to come *after* the <upstream> argument?

Maybe not but the scripted version does. I'm not sure if that is a good idea or not.

Best Wishes

Phillip

Quoted 242 lines

Ciao,
Dscho

Quoted 239 lines

Best Wishes

Phillip

Quoted 231 lines · Johannes Schindelin via GitGitGadget

From: Johannes Schindelin <[email protected]>

Currently, we parse the options intended for git am as if we wanted to handle them in git rebase, and then reconstruct them painstakingly to define the git_am_opt variable.

However, there is a much better way (that I was unaware of, at the time when I mentored Pratik to implement these options): OPT_PASSTHRU_ARGV. It is intended for exactly this use case, where command-line options want to be parsed into a separate argv_array.

Let's use this feature.

Incidentally, this also allows us to address a bug discovered by Phillip Wood, where the built-in rebase failed to understand that the -C option takes an optional argument.

---
   builtin/rebase.c | 98 +++++++++++++++++-------------------------------
   1 file changed, 35 insertions(+), 63 deletions(-)
builtin/rebase.c +1 -1
index 0ee06aa363..96ffa80b71 100644
@@ -87,7 +87,7 @@ struct rebase_options {
     REBASE_FORCE = 1<<3,
     REBASE_INTERACTIVE_EXPLICIT = 1<<4,
   	} flags;
-	struct strbuf git_am_opt;
+	struct argv_array git_am_opts;
    const char *action;
    int signoff;
    int allow_rerere_autoupdate;
@@ -339,7 +339,7 @@ N_("Resolve all conflicts manually, mark them as

resolved with\n"
static int run_specific_rebase(struct rebase_options *opts)

   {
   	const char *argv[] = { NULL, NULL };
builtin/rebase.c +1 -1
-	struct strbuf script_snippet = STRBUF_INIT;
+	struct strbuf script_snippet = STRBUF_INIT, buf = STRBUF_INIT;
    int status;
    const char *backend, *backend_func;
   @@ -433,7 +433,9 @@ static int run_specific_rebase(struct rebase_options

*opts)

    	oid_to_hex(&opts->restrict_revision->object.oid) : NULL);
    add_var(&script_snippet, "GIT_QUIET",

opts->flags & REBASE_NO_QUIET ? "" : "t");

builtin/rebase.c +4 -2
-	add_var(&script_snippet, "git_am_opt", opts->git_am_opt.buf);
+	sq_quote_argv_pretty(&buf, opts->git_am_opts.argv);
+	add_var(&script_snippet, "git_am_opt", buf.buf);
+	strbuf_release(&buf);
    add_var(&script_snippet, "verbose",
    	opts->flags & REBASE_VERBOSE ? "t" : "");
   	add_var(&script_snippet, "diffstat",
@@ -756,7 +758,7 @@ int cmd_rebase(int argc, const char **argv, const char
 *prefix)
    struct rebase_options options = {
     .type = REBASE_UNSPECIFIED,
     .flags = REBASE_NO_QUIET,
-		.git_am_opt = STRBUF_INIT,
+		.git_am_opts = ARGV_ARRAY_INIT,
     .allow_rerere_autoupdate  = -1,
     .allow_empty_message = 1,

.git_format_patch_opt = STRBUF_INIT,

builtin/rebase.c +0 -5
@@ -777,12 +779,7 @@ int cmd_rebase(int argc, const char **argv, const
 char *prefix)
     ACTION_EDIT_TODO,
     ACTION_SHOW_CURRENT_PATCH,
   	} action = NO_ACTION;
-	int committer_date_is_author_date = 0;
-	int ignore_date = 0;
-	int ignore_whitespace = 0;
   	const char *gpg_sign = NULL;
-	int opt_c = -1;
-	struct string_list whitespace = STRING_LIST_INIT_NODUP;
    struct string_list exec = STRING_LIST_INIT_NODUP;
    const char *rebase_merges = NULL;
    int fork_point = -1;
builtin/rebase.c +7 -7
@@ -804,15 +801,20 @@ int cmd_rebase(int argc, const char **argv, const
 char *prefix)
     {OPTION_NEGBIT, 'n', "no-stat", &options.flags, NULL,
      N_("do not show diffstat of what changed upstream"),
      PARSE_OPT_NOARG, NULL, REBASE_DIFFSTAT },
-		OPT_BOOL(0, "ignore-whitespace", &ignore_whitespace,
-			 N_("passed to 'git apply'")),
     OPT_BOOL(0, "signoff", &options.signoff,
   			 N_("add a Signed-off-by: line to each commit")),
-		OPT_BOOL(0, "committer-date-is-author-date",
-			 &committer_date_is_author_date,
-			 N_("passed to 'git am'")),
-		OPT_BOOL(0, "ignore-date", &ignore_date,
-			 N_("passed to 'git am'")),
+		OPT_PASSTHRU_ARGV(0, "ignore-whitespace",
 &options.git_am_opts,
+				  NULL, N_("passed to 'git am'"),
+				  PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV(0, "committer-date-is-author-date",
+				  &options.git_am_opts, NULL,
+				  N_("passed to 'git am'"),
 PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV(0, "ignore-date", &options.git_am_opts,

NULL,

builtin/rebase.c +1 -0
+				  N_("passed to 'git am'"),

PARSE_OPT_NOARG),

builtin/rebase.c +4 -0
+		OPT_PASSTHRU_ARGV('C', NULL, &options.git_am_opts, N_("n"),
+				  N_("passed to 'git apply'"), 0),
+		OPT_PASSTHRU_ARGV(0, "whitespace", &options.git_am_opts,
+				  N_("action"), N_("passed to 'git apply'"),

0),
OPT_BIT('f', "force-rebase", &options.flags,
N_("cherry-pick all commits, even if unchanged"), REBASE_FORCE),

builtin/rebase.c +0 -4
@@ -856,10 +858,6 @@ int cmd_rebase(int argc, const char **argv, const
 char *prefix)
     { OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"),
      N_("GPG-sign commits"),
      PARSE_OPT_OPTARG, NULL, (intptr_t) "" },
-		OPT_STRING_LIST(0, "whitespace", &whitespace,
-				N_("whitespace"), N_("passed to 'git
 apply'")),
-		OPT_SET_INT('C', NULL, &opt_c, N_("passed to 'git apply'"),
-			    REBASE_AM),
     OPT_BOOL(0, "autostash", &options.autostash,

N_("automatically stash/stash pop before and after")),
OPT_STRING_LIST('x', "exec", &exec, N_("exec"),

builtin/rebase.c +1 -0
@@ -884,6 +882,7 @@ int cmd_rebase(int argc, const char **argv, const char
 *prefix)
     	 N_("rebase all reachable commits up to the root(s)")),
    	OPT_END(),
   	};
+	int i;
 
    /*

* NEEDSWORK: Once the builtin rebase has been tested enough

builtin/rebase.c +9 -28
@@ -1064,22 +1063,17 @@ int cmd_rebase(int argc, const char **argv, const
 char *prefix)
    	    state_dir_base, cmd_live_rebase, buf.buf);
    }
   -	if (!(options.flags & REBASE_NO_QUIET))
-		strbuf_addstr(&options.git_am_opt, " -q");
-
-	if (committer_date_is_author_date) {
-		strbuf_addstr(&options.git_am_opt,
-			      " --committer-date-is-author-date");
-		options.flags |= REBASE_FORCE;
+	for (i = 0; i < options.git_am_opts.argc; i++) {
+		const char *option = options.git_am_opts.argv[i];
+		if (!strcmp(option, "--committer-date-is-author-date") ||
+		    !strcmp(option, "--ignore-date") ||
+		    !strcmp(option, "--whitespace=fix") ||
+		    !strcmp(option, "--whitespace=strip"))
+			options.flags |= REBASE_FORCE;
    }
   -	if (ignore_whitespace)
-		strbuf_addstr(&options.git_am_opt, " --ignore-whitespace");
-
-	if (ignore_date) {
-		strbuf_addstr(&options.git_am_opt, " --ignore-date");
-		options.flags |= REBASE_FORCE;
-	}
+	if (!(options.flags & REBASE_NO_QUIET))
+		argv_array_push(&options.git_am_opts, "-q");
 
    if (options.keep_empty)
   		imply_interactive(&options, "--keep-empty");
@@ -1089,23 +1083,6 @@ int cmd_rebase(int argc, const char **argv, const
 char *prefix)
    	options.gpg_sign_opt = xstrfmt("-S%s", gpg_sign);
    }
   -	if (opt_c >= 0)
-		strbuf_addf(&options.git_am_opt, " -C%d", opt_c);
-
-	if (whitespace.nr) {
-		int i;
-
-		for (i = 0; i < whitespace.nr; i++) {
-			const char *item = whitespace.items[i].string;
-
-			strbuf_addf(&options.git_am_opt, " --whitespace=%s",
-				    item);
-
-			if ((!strcmp(item, "fix")) || (!strcmp(item,
 "strip")))
-				options.flags |= REBASE_FORCE;
-		}
-	}
-
    if (exec.nr) {
     int i;
   @@ -1181,23 +1158,18 @@ int cmd_rebase(int argc, const char **argv,

const char *prefix)

    	break;
    }

- if (options.git_am_opt.len) {

builtin/rebase.c +7 -10
-		const char *p;
-
+	if (options.git_am_opts.argc) {
   		/* all am options except -q are compatible only with --am */
-		strbuf_reset(&buf);
-		strbuf_addbuf(&buf, &options.git_am_opt);
-		strbuf_addch(&buf, ' ');
-		while ((p = strstr(buf.buf, " -q ")))
-			strbuf_splice(&buf, p - buf.buf, 4, " ", 1);
-		strbuf_trim(&buf);
+		for (i = options.git_am_opts.argc - 1; i >= 0; i--)
+			if (strcmp(options.git_am_opts.argv[i], "-q"))
+				break;
   -		if (is_interactive(&options) && buf.len)
+		if (is_interactive(&options) && i >= 0)
      die(_("error: cannot combine interactive options "
            "(--interactive, --exec, --rebase-merges, "
            "--preserve-merges, --keep-empty, --root + "
            "--onto) with am options (%s)"), buf.buf);
-		if (options.type == REBASE_MERGE && buf.len)
+		if (options.type == REBASE_MERGE && i >= 0)
      die(_("error: cannot combine merge options (--merge, "
            "--strategy, --strategy-option) with am options "
            "(%s)"), buf.buf);
@@ -1207,7 +1179,7 @@ int cmd_rebase(int argc, const char **argv, const
 char *prefix)
     if (options.type == REBASE_PRESERVE_MERGES)
      die("cannot combine '--signoff' with "
   			    "'--preserve-merges'");
-		strbuf_addstr(&options.git_am_opt, " --signoff");
+		argv_array_push(&options.git_am_opts, "--signoff");
    	options.flags |= REBASE_FORCE;
    }
#6 lore raw

rebase-in-C stability for 2.20

Quoted 42 lines · Phillip Wood

Hi Johannes

Quoted 19 lines · Johannes Schindelin

Hi Phillip,

Quoted 15 lines · Phillip Wood

Thanks for looking at this. Unfortunately using OPT_PASSTHRU_ARGV seems to break the error reporting

Running
bin/wrappers/git rebase --onto @^^^^ @^^ -Cbad

Gives
git encountered an error while preparing the patches to replay these revisions:

67f673aa4a580b9e407b1ca505abf1f50510ec47...7c3e01a708856885e60bf4051586970e65dd326c

As a result, git cannot rebase them.

git 2.19.1 gives

First, rewinding head to replay your work on top of it...
Applying: Ninth batch for 2.20
error: switch `C' expects a numerical value

So it has a clear message as to what the error is, this patch regresses that. It would be better if rebase detected the error before starting though.

Quoted 8 lines
Quoted 6 lines

If I do

bin/wrappers/git rebase @^^ -Cbad

I get no error, it just tells me that it does not need to rebase (which is true)

Hmm. Isn't this the same behavior as with the scripted version?

Quoted 8 lines

Ah you're right the script does not check if the option argument is valid.

Quoted 3 lines

Also: are
we sure that we want to allow options to come *after* the <upstream> argument?

Maybe not but the scripted version does. I'm not sure if that is a good idea or not.

According to Junio's calendar we're now 2 days from 2.20-rc0. We have the js/rebase-autostash-detach-fix bug I reported sitting in "pu" still, and then this.

I see we still have rebase.useBuiltin in the code as an escape hatch, but it's undocumented.

Given that we're still finding regressions bugs in the rebase-in-C version should we be considering reverting 5541bd5b8f ("rebase: default to using the builtin rebase", 2018-08-08)?

I love the feature, but fear that the current list of known regressions serve as a canary for a larger list which we'd discover if we held off for another major release (and would re-enable rebase.useBuiltin=true in master right after 2.20 is out the door).

But maybe I'm being overly paranoid. What do those more familiar with this think?

#7 lore raw

Re: rebase-in-C stability for 2.20

Quoted 2 lines

But maybe I'm being overly paranoid. What do those more familiar with this think?

I am not too worried,
* as rebase is a main porcelain, that is even hard to use in a script. so any failures are not deep down in some automation,
but when found exposed quickly (and hopefully reported).
* 5541bd5b8f was merged to next a month ago; internally we distribute the next branch to Googlers (on a weekly basis) and we have not had any bug reports regarding rebase. (Maybe our environment is too strict for the wide range of bugs reported)
* Johannes reported that the rebase is used in GfW
https://public-inbox.org/git/[email protected]/
https://github.com/git-for-windows/git/pull/1800
and from my cursory reading it is part of
2.19.windows, which has a large user base.

Quoted 2 lines

(and would re-enable rebase.useBuiltin=true in
master right after 2.20 is out the door).

That would be fine with me as well, but I'd rather document rebase.useBuiltin instead of flip-flopping the switch around the release.

Have there been any fixes that are only in
the C version (has the shell version already bitrotted)?

Stefan

#8 lore raw

cover rebase.useBuiltin doc & test mode

Quoted 12 lines · Stefan Beller
Quoted 2 lines

But maybe I'm being overly paranoid. What do those more familiar with this think?

I am not too worried,
* as rebase is a main porcelain, that is even hard to use in a script. so any failures are not deep down in some automation,
but when found exposed quickly (and hopefully reported).
* 5541bd5b8f was merged to next a month ago; internally we distribute the next branch to Googlers (on a weekly basis) and we have not had any bug reports regarding rebase. (Maybe our environment is too strict for the wide range of bugs reported)

I do the same at Booking.com (although at a more ad-hoc schedule) and got the report whose fix is now sitting in "pu" noted upthread.

I fear that these sorts of corporate environments, both Google's and Booking's, end up testing a relatively narrow featureset. Most people have similar enough workflows, e.g. just using "git pull --rebase", I'd be surprised if we have more than 2-3 internal users who ever use the --onto option for example.

Quoted 9 lines

* Johannes reported that the rebase is used in GfW
https://public-inbox.org/git/[email protected]/
https://github.com/git-for-windows/git/pull/1800
and from my cursory reading it is part of 2.19.windows, which has a large user base.

Quoted 2 lines

(and would re-enable rebase.useBuiltin=true in
master right after 2.20 is out the door).

Quoted 6 lines

That would be fine with me as well, but I'd rather document rebase.useBuiltin instead of flip-flopping the switch around the release.

Have there been any fixes that are only in
the C version (has the shell version already bitrotted)?

That's a good question, one which I don't think we knew the answer to before the following patches. As it turns out no, we still run the tests without failures with GIT_TEST_REBASE_USE_BUILTIN=false.

Ævar Arnfjörð Bjarmason (2):
rebase doc: document rebase.useBuiltin
tests: add a special setup where rebase.useBuiltin is off

 Documentation/config/rebase.txt | 14 ++++++++++++++
 builtin/rebase.c                |  5 ++++-
 t/README                        |  3 +++
 3 files changed, 21 insertions(+), 1 deletion(-)
Signature · 1 line
2.19.1.1182.g4ecb1133ce
#9 lore raw

Re: [PATCH 0/2] rebase.useBuiltin doc & test mode

Hi Ævar,

Quoted 37 lines · Ævar Arnfjörð Bjarmason
Quoted 12 lines · Stefan Beller
Quoted 2 lines

But maybe I'm being overly paranoid. What do those more familiar with this think?

I am not too worried,
* as rebase is a main porcelain, that is even hard to use in a script. so any failures are not deep down in some automation,
but when found exposed quickly (and hopefully reported).
* 5541bd5b8f was merged to next a month ago; internally we distribute the next branch to Googlers (on a weekly basis) and we have not had any bug reports regarding rebase. (Maybe our environment is too strict for the wide range of bugs reported)

I do the same at Booking.com (although at a more ad-hoc schedule) and got the report whose fix is now sitting in "pu" noted upthread.

I fear that these sorts of corporate environments, both Google's and Booking's, end up testing a relatively narrow featureset. Most people have similar enough workflows, e.g. just using "git pull --rebase", I'd be surprised if we have more than 2-3 internal users who ever use the --onto option for example.

Quoted 13 lines

* Johannes reported that the rebase is used in GfW
https://public-inbox.org/git/[email protected]/
https://github.com/git-for-windows/git/pull/1800
and from my cursory reading it is part of 2.19.windows, which has a large user base.

Quoted 2 lines

(and would re-enable rebase.useBuiltin=true in
master right after 2.20 is out the door).

That would be fine with me as well, but I'd rather document rebase.useBuiltin instead of flip-flopping the switch around the release.

Quoted 5 lines
Quoted 2 lines

Have there been any fixes that are only in
the C version (has the shell version already bitrotted)?

That's a good question, one which I don't think we knew the answer to before the following patches.

I pay close attention to git rebase bug reports and patches (obviously), and there have not been any changes going into the built-in rebase/rebase -i that necessitated changes also in the scripted version.

Ciao,
Dscho

Quoted 16 lines

As it turns out no, we still run the tests without failures with GIT_TEST_REBASE_USE_BUILTIN=false.

Ævar Arnfjörð Bjarmason (2):
rebase doc: document rebase.useBuiltin
tests: add a special setup where rebase.useBuiltin is off

 Documentation/config/rebase.txt | 14 ++++++++++++++
 builtin/rebase.c                |  5 ++++-
 t/README                        |  3 +++
 3 files changed, 21 insertions(+), 1 deletion(-)
Signature · 1 line
2.19.1.1182.g4ecb1133ce
#10 lore raw

PATCH 1/2 rebase doc: document rebase.useBuiltin

In reply to Stefan Beller

The rebase.useBuiltin variable introduced in 55071ea248 ("rebase:
start implementing it as a builtin", 2018-08-07) was turned on by default in 5541bd5b8f ("rebase: default to using the builtin rebase", 2018-08-08), but had no documentation.

Let's document it so that users who run into any stability issues with the C rewrite know there's an escape hatch[1], and make it clear that needing to turn off builtin rebase means you've found a bug in git.

1. https://public-inbox.org/git/[email protected]/

---
 Documentation/config/rebase.txt | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
Documentation/config/rebase.txt +14 -0
index 42e1ba7575..f079bf6b7e 100644
@@ -1,3 +1,17 @@
+rebase.useBuiltin::
+	Set to `false` to use the legacy shellscript implementation of
+	linkgit:git-rebase[1]. Is `true` by default, which means use
+	the built-in rewrite of it in C.
++
+The C rewrite is first included with Git version 2.20. This option
+serves an an escape hatch to re-enable the legacy version in case any
+bugs are found in the rewrite. This option and the shellscript version
+of linkgit:git-rebase[1] will be removed in some future release.
++
+If you find some reason to set this option to `false` other than
+one-off testing you should report the behavior difference as a bug in
+git.
+
 rebase.stat::
 	Whether to show a diffstat of what changed upstream since the last
 	rebase. False by default.
Signature · 1 line
2.19.1.1182.g4ecb1133ce
#11 lore raw

PATCH 2/2 tests: add a special setup where rebase.useBuiltin is off

In reply to Stefan Beller

Add a GIT_TEST_REBASE_USE_BUILTIN=false test mode which is equivalent to running with rebase.useBuiltin=false. This is needed to spot that we're not introducing any regressions in the legacy rebase version while we're carrying both it and the new builtin version.

---
 builtin/rebase.c | 5 ++++-
 t/README         | 3 +++
 2 files changed, 7 insertions(+), 1 deletion(-)
builtin/rebase.c +4 -1
index 0ee06aa363..68ad8c1149 100644
@@ -48,7 +48,10 @@ static int use_builtin_rebase(void)
 {
 	struct child_process cp = CHILD_PROCESS_INIT;
 	struct strbuf out = STRBUF_INIT;
-	int ret;
+	int ret, env = git_env_bool("GIT_TEST_REBASE_USE_BUILTIN", -1);
+
+	if (env != -1)
+		return env;
 
 	argv_array_pushl(&cp.args,
 			 "config", "--bool", "rebase.usebuiltin", NULL);
t/README +3 -0
index 242497455f..c719e08414 100644
@@ -348,6 +348,9 @@ GIT_TEST_MULTI_PACK_INDEX=<boolean>, when true, forces the multi-pack-
 index to be written after every 'git repack' command, and overrides the
 'core.multiPackIndex' setting to true.
 
+GIT_TEST_REBASE_USE_BUILTIN<boolean>, when false, disables the builtin
+version of git-rebase. See 'rebase.useBuiltin' in git-config(1).
+
 Naming Tests
 ------------
 
Signature · 1 line
2.19.1.1182.g4ecb1133ce
#12 lore raw

Re: rebase-in-C stability for 2.20

In reply to Ævar Arnfjörð Bjarmason
Quoted 10 lines · Ævar Arnfjörð Bjarmason

According to Junio's calendar we're now 2 days from 2.20-rc0. We have the js/rebase-autostash-detach-fix bug I reported sitting in "pu" still, and then this.

I see we still have rebase.useBuiltin in the code as an escape hatch, but it's undocumented.

Given that we're still finding regressions bugs in the rebase-in-C version should we be considering reverting 5541bd5b8f ("rebase: default to using the builtin rebase", 2018-08-08)?

That date feels slightly misleading; it has a commit date of Oct 11, and only merged to master less than two weeks ago. Given how big the two different rebase in C series were, I'd expect a couple small things to fall out after it hit master, which is what appears to be happening.

Quoted 7 lines

I love the feature, but fear that the current list of known regressions serve as a canary for a larger list which we'd discover if we held off for another major release (and would re-enable rebase.useBuiltin=true in master right after 2.20 is out the door).

But maybe I'm being overly paranoid. What do those more familiar with this think?

I probably don't qualify as more familar, but giving my $0.02 anyway... I'm happy that setting rebase.useBuiltin=false by default exists an escape hatch if things don't settle down as we get closer to the release, but I'd rather wait until further into the RC cycle before going that route, personally.

#13 lore raw

Re: rebase-in-C stability for 2.20

In reply to Ævar Arnfjörð Bjarmason
Quoted 11 lines · Ævar Arnfjörð Bjarmason

According to Junio's calendar we're now 2 days from 2.20-rc0. We have the js/rebase-autostash-detach-fix bug I reported sitting in "pu" still, and then this.

I see we still have rebase.useBuiltin in the code as an escape hatch, but it's undocumented.

Given that we're still finding regressions bugs in the rebase-in-C version should we be considering reverting 5541bd5b8f ("rebase: default to using the builtin rebase", 2018-08-08)?

Quoted 7 lines

I love the feature, but fear that the current list of known regressions serve as a canary for a larger list which we'd discover if we held off for another major release (and would re-enable rebase.useBuiltin=true in master right after 2.20 is out the door).

But maybe I'm being overly paranoid. What do those more familiar with this think?

I was hoping that having it early in GfW 2.19 would have smoked out all the remaining issues, but it seems that those building from the source and testing have usage patterns different enough to find more issues. This is a normal part of the development process, and hopefully the remaining bugs are minor and can be flushed out in the -rc testing period---this kind of thing is the whole reason why we code-freeze and test rcs.

It unfortunately is too late to depend on the rebase.useBuiltin as an escape hatch, as 'master', 'next', or anywhere else had the "rebase and rebase -i in C" series merged without it set to false and used in any seriousness. Quite honestly, I think we can trust the rest of the "rebase and rebase -i in C" series much more than its "use the scripted one even though we have built-in one" part.

So if we have to seriously consider holding back, we may be better off ripping the whole thing out. I do not offhand know how involved such a reversion would be, though, and I am *NOT* looking forward to having do such a major surgery right before the release.

#14 lore raw

Re: [PATCH 1/1] rebase: really just passthru the `git am` options

In reply to Phillip Wood

Hi Phillip,

Quoted 21 lines · Phillip Wood
Quoted 19 lines · Johannes Schindelin

Hi Phillip,

Quoted 15 lines · Phillip Wood

Thanks for looking at this. Unfortunately using OPT_PASSTHRU_ARGV seems to break the error reporting

Running
bin/wrappers/git rebase --onto @^^^^ @^^ -Cbad

Gives
git encountered an error while preparing the patches to replay these revisions:

67f673aa4a580b9e407b1ca505abf1f50510ec47...7c3e01a708856885e60bf4051586970e65dd326c

As a result, git cannot rebase them.

Quoted 8 lines

git 2.19.1 gives

First, rewinding head to replay your work on top of it...
Applying: Ninth batch for 2.20
error: switch `C' expects a numerical value

So it has a clear message as to what the error is, this patch regresses that. It would be better if rebase detected the error before starting though.

I agree that that would make most sense: why start something when you can know that it will fail later...

Let me try to add that test case that Junio wanted, and some early error handling.

Quoted 9 lines
Quoted 8 lines
Quoted 6 lines

If I do

bin/wrappers/git rebase @^^ -Cbad

I get no error, it just tells me that it does not need to rebase (which is true)

Hmm. Isn't this the same behavior as with the scripted version?

Quoted 8 lines

Ah you're right the script does not check if the option argument is valid.

Quoted 3 lines

Also: are
we sure that we want to allow options to come *after* the <upstream> argument?

Maybe not but the scripted version does. I'm not sure if that is a good idea or not.

That behavior was never documented, though, was it?

Ciao,
Dscho

Quoted 257 lines

Best Wishes

Phillip

Quoted 249 lines

Ciao,
Dscho

Quoted 246 lines

Best Wishes

Phillip

Quoted 238 lines · Johannes Schindelin via GitGitGadget

From: Johannes Schindelin <[email protected]>

Currently, we parse the options intended for git am as if we wanted to handle them in git rebase, and then reconstruct them painstakingly to define the git_am_opt variable.

However, there is a much better way (that I was unaware of, at the time when I mentored Pratik to implement these options): OPT_PASSTHRU_ARGV. It is intended for exactly this use case, where command-line options want to be parsed into a separate argv_array.

Let's use this feature.

Incidentally, this also allows us to address a bug discovered by Phillip Wood, where the built-in rebase failed to understand that the -C option takes an optional argument.

---
   builtin/rebase.c | 98
   +++++++++++++++++-------------------------------
   1 file changed, 35 insertions(+), 63 deletions(-)
builtin/rebase.c +1 -1
index 0ee06aa363..96ffa80b71 100644
@@ -87,7 +87,7 @@ struct rebase_options {
     REBASE_FORCE = 1<<3,
     REBASE_INTERACTIVE_EXPLICIT = 1<<4,
   	} flags;
-	struct strbuf git_am_opt;
+	struct argv_array git_am_opts;
    const char *action;
    int signoff;
    int allow_rerere_autoupdate;
@@ -339,7 +339,7 @@ N_("Resolve all conflicts manually, mark them as

resolved with\n"
static int run_specific_rebase(struct rebase_options *opts)

   {
   	const char *argv[] = { NULL, NULL };
builtin/rebase.c +1 -1
-	struct strbuf script_snippet = STRBUF_INIT;
+	struct strbuf script_snippet = STRBUF_INIT, buf = STRBUF_INIT;
    int status;
    const char *backend, *backend_func;
   @@ -433,7 +433,9 @@ static int run_specific_rebase(struct

rebase_options
*opts)

    	oid_to_hex(&opts->restrict_revision->object.oid) : NULL);
    add_var(&script_snippet, "GIT_QUIET",

opts->flags & REBASE_NO_QUIET ? "" : "t");

builtin/rebase.c +3 -1
-	add_var(&script_snippet, "git_am_opt", opts->git_am_opt.buf);
+	sq_quote_argv_pretty(&buf, opts->git_am_opts.argv);
+	add_var(&script_snippet, "git_am_opt", buf.buf);
+	strbuf_release(&buf);
    add_var(&script_snippet, "verbose",
    	opts->flags & REBASE_VERBOSE ? "t" : "");
   	add_var(&script_snippet, "diffstat",
@@ -756,7 +758,7 @@ int cmd_rebase(int argc, const char **argv, const

char
*prefix)
struct rebase_options options = {
.type = REBASE_UNSPECIFIED,
.flags = REBASE_NO_QUIET,

builtin/rebase.c +1 -6
-		.git_am_opt = STRBUF_INIT,
+		.git_am_opts = ARGV_ARRAY_INIT,
     .allow_rerere_autoupdate  = -1,
     .allow_empty_message = 1,
     .git_format_patch_opt = STRBUF_INIT,
@@ -777,12 +779,7 @@ int cmd_rebase(int argc, const char **argv, const
 char *prefix)
     ACTION_EDIT_TODO,
     ACTION_SHOW_CURRENT_PATCH,
   	} action = NO_ACTION;
-	int committer_date_is_author_date = 0;
-	int ignore_date = 0;
-	int ignore_whitespace = 0;
   	const char *gpg_sign = NULL;
-	int opt_c = -1;
-	struct string_list whitespace = STRING_LIST_INIT_NODUP;
    struct string_list exec = STRING_LIST_INIT_NODUP;
    const char *rebase_merges = NULL;
    int fork_point = -1;
builtin/rebase.c +6 -7
@@ -804,15 +801,20 @@ int cmd_rebase(int argc, const char **argv, const
 char *prefix)
     {OPTION_NEGBIT, 'n', "no-stat", &options.flags, NULL,
      N_("do not show diffstat of what changed upstream"),
      PARSE_OPT_NOARG, NULL, REBASE_DIFFSTAT },
-		OPT_BOOL(0, "ignore-whitespace", &ignore_whitespace,
-			 N_("passed to 'git apply'")),
     OPT_BOOL(0, "signoff", &options.signoff,
   			 N_("add a Signed-off-by: line to each
 commit")),
-		OPT_BOOL(0, "committer-date-is-author-date",
-			 &committer_date_is_author_date,
-			 N_("passed to 'git am'")),
-		OPT_BOOL(0, "ignore-date", &ignore_date,
-			 N_("passed to 'git am'")),
+		OPT_PASSTHRU_ARGV(0, "ignore-whitespace",
 &options.git_am_opts,
+				  NULL, N_("passed to 'git am'"),
+				  PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV(0, "committer-date-is-author-date",
+				  &options.git_am_opts, NULL,
+				  N_("passed to 'git am'"),

PARSE_OPT_NOARG),

builtin/rebase.c +1 -0
+		OPT_PASSTHRU_ARGV(0, "ignore-date", &options.git_am_opts,

NULL,

builtin/rebase.c +1 -0
+				  N_("passed to 'git am'"),

PARSE_OPT_NOARG),

builtin/rebase.c +4 -0
+		OPT_PASSTHRU_ARGV('C', NULL, &options.git_am_opts, N_("n"),
+				  N_("passed to 'git apply'"), 0),
+		OPT_PASSTHRU_ARGV(0, "whitespace", &options.git_am_opts,
+				  N_("action"), N_("passed to 'git apply'"),

0),
OPT_BIT('f', "force-rebase", &options.flags,
N_("cherry-pick all commits, even if unchanged"), REBASE_FORCE),

builtin/rebase.c +0 -4
@@ -856,10 +858,6 @@ int cmd_rebase(int argc, const char **argv, const
 char *prefix)
     { OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"),
      N_("GPG-sign commits"),
      PARSE_OPT_OPTARG, NULL, (intptr_t) "" },
-		OPT_STRING_LIST(0, "whitespace", &whitespace,
-				N_("whitespace"), N_("passed to 'git
 apply'")),
-		OPT_SET_INT('C', NULL, &opt_c, N_("passed to 'git apply'"),
-			    REBASE_AM),
     OPT_BOOL(0, "autostash", &options.autostash,

N_("automatically stash/stash pop before and after")),
OPT_STRING_LIST('x', "exec", &exec, N_("exec"),

builtin/rebase.c +0 -0
@@ -884,6 +882,7 @@ int cmd_rebase(int argc, const char **argv, const

char
*prefix)
N_("rebase all reachable commits up to the root(s)")),
OPT_END(),
};

builtin/rebase.c +1 -0
+	int i;
 
    /*
   	 * NEEDSWORK: Once the builtin rebase has been tested enough
@@ -1064,22 +1063,17 @@ int cmd_rebase(int argc, const char **argv,

const
char *prefix)

    	    state_dir_base, cmd_live_rebase, buf.buf);

}
- if (!(options.flags & REBASE_NO_QUIET))

builtin/rebase.c +9 -28
-		strbuf_addstr(&options.git_am_opt, " -q");
-
-	if (committer_date_is_author_date) {
-		strbuf_addstr(&options.git_am_opt,
-			      " --committer-date-is-author-date");
-		options.flags |= REBASE_FORCE;
+	for (i = 0; i < options.git_am_opts.argc; i++) {
+		const char *option = options.git_am_opts.argv[i];
+		if (!strcmp(option, "--committer-date-is-author-date") ||
+		    !strcmp(option, "--ignore-date") ||
+		    !strcmp(option, "--whitespace=fix") ||
+		    !strcmp(option, "--whitespace=strip"))
+			options.flags |= REBASE_FORCE;
   }
   -	if (ignore_whitespace)
-		strbuf_addstr(&options.git_am_opt, " --ignore-whitespace");
-
-	if (ignore_date) {
-		strbuf_addstr(&options.git_am_opt, " --ignore-date");
-		options.flags |= REBASE_FORCE;
-	}
+	if (!(options.flags & REBASE_NO_QUIET))
+		argv_array_push(&options.git_am_opts, "-q");
 
    if (options.keep_empty)
   		imply_interactive(&options, "--keep-empty");
@@ -1089,23 +1083,6 @@ int cmd_rebase(int argc, const char **argv, const
 char *prefix)
    	options.gpg_sign_opt = xstrfmt("-S%s", gpg_sign);
   }
   -	if (opt_c >= 0)
-		strbuf_addf(&options.git_am_opt, " -C%d", opt_c);
-
-	if (whitespace.nr) {
-		int i;
-
-		for (i = 0; i < whitespace.nr; i++) {
-			const char *item = whitespace.items[i].string;
-
-			strbuf_addf(&options.git_am_opt, " --whitespace=%s",
-				    item);
-
-			if ((!strcmp(item, "fix")) || (!strcmp(item,
 "strip")))
-				options.flags |= REBASE_FORCE;
-		}
-	}
-
    if (exec.nr) {
     int i;
   @@ -1181,23 +1158,18 @@ int cmd_rebase(int argc, const char **argv,

const char *prefix)

    	break;

}
- if (options.git_am_opt.len) {

builtin/rebase.c +7 -11
-		const char *p;
-
+	if (options.git_am_opts.argc) {
   		/* all am options except -q are compatible only with
--am */
-		strbuf_reset(&buf);
-		strbuf_addbuf(&buf, &options.git_am_opt);
-		strbuf_addch(&buf, ' ');
-		while ((p = strstr(buf.buf, " -q ")))
-			strbuf_splice(&buf, p - buf.buf, 4, " ", 1);
-		strbuf_trim(&buf);
+		for (i = options.git_am_opts.argc - 1; i >= 0; i--)
+			if (strcmp(options.git_am_opts.argv[i], "-q"))
+				break;
   -		if (is_interactive(&options) && buf.len)
+		if (is_interactive(&options) && i >= 0)
      die(_("error: cannot combine interactive options "
            "(--interactive, --exec, --rebase-merges, "
            "--preserve-merges, --keep-empty, --root + "
            "--onto) with am options (%s)"), buf.buf);
-		if (options.type == REBASE_MERGE && buf.len)
+		if (options.type == REBASE_MERGE && i >= 0)
      die(_("error: cannot combine merge options (--merge, "
            "--strategy, --strategy-option) with am options "
            "(%s)"), buf.buf);
@@ -1207,7 +1179,7 @@ int cmd_rebase(int argc, const char **argv, const
 char *prefix)
     if (options.type == REBASE_PRESERVE_MERGES)
      die("cannot combine '--signoff' with "
   			    "'--preserve-merges'");
-		strbuf_addstr(&options.git_am_opt, " --signoff");
+		argv_array_push(&options.git_am_opts, "--signoff");
    	options.flags |= REBASE_FORCE;
    }
#15 lore raw
In reply to Johannes Schindelin via GitGitGadget
Quoted 5 lines · Johannes Schindelin via GitGitGadget

Phillip Wood reported a problem where the built-in rebase did not understand options like -C1, i.e. it did not expect the option argument.

While investigating how to address this best, I stumbled upon OPT_PASSTHRU_ARGV (which I was so far happily unaware of).

I was unaware of it, too. Looking at the OPT_PASSTHRU and its ARGV counterpart, I think the original intent was that you'd pass through normal last-one-wins individual options with OPT_PASSTHRU, and then list-like options with OPT_PASSTHRU_ARGV. But here you've used the latter to pass sets of individual last-one-wins options.

That said, I think what you've done here is way simpler and more readable than using a bunch of OPT_PASSTHRUs would have been. And even if it was not the original intent of the ARGV variant, I can't see any reason to avoid doing it this way.

-Peff

#16 lore raw

Hi Peff,

Quoted 9 lines · Jeff King
Quoted 5 lines · Johannes Schindelin via GitGitGadget

Phillip Wood reported a problem where the built-in rebase did not understand options like -C1, i.e. it did not expect the option argument.

While investigating how to address this best, I stumbled upon OPT_PASSTHRU_ARGV (which I was so far happily unaware of).

I was unaware of it, too.

Thanks, that makes me feel better ;-)

Quoted 10 lines

Looking at the OPT_PASSTHRU and its ARGV counterpart, I think the original intent was that you'd pass through normal last-one-wins individual options with OPT_PASSTHRU, and then list-like options with OPT_PASSTHRU_ARGV. But here you've used the latter to pass sets of individual last-one-wins options.

That said, I think what you've done here is way simpler and more readable than using a bunch of OPT_PASSTHRUs would have been. And even if it was not the original intent of the ARGV variant, I can't see any reason to avoid doing it this way.

Thank you, that makes me feel *even* better ;-)

Together with the extra-early error checking for incorrect -C and --whitespace options, I think we're in a much better place now.

Ciao,
Dscho

#17 lore raw

v2 cover rebase: understand -C again, refactor

In reply to Johannes Schindelin via GitGitGadget

Phillip Wood reported a problem where the built-in rebase did not understand options like -C1, i.e. it did not expect the option argument.

While investigating how to address this best, I stumbled upon OPT_PASSTHRU_ARGV (which I was so far happily unaware of).

Instead of just fixing the -C<n> bug, I decided to simply convert all of the options intended for git am (or, eventually, for git apply). This happens to fix that bug, and does so much more: it simplifies the entire logic (and removes more lines than it adds).

Change since v1:

* Introduce early parameter validation for the options passed through to git am.

Johannes Schindelin (2):
rebase: really just passthru the git am options
rebase: validate -C<n> and --whitespace=<mode> parameters early

 builtin/rebase.c          | 108 ++++++++++++++++----------------------
 t/t3406-rebase-message.sh |   7 +++
 2 files changed, 52 insertions(+), 63 deletions(-)

base-commit: 8858448bb49332d353febc078ce4a3abcc962efe
Published-As: https://github.com/gitgitgadget/git/releases/tags/pr-76%2Fdscho%2Frebase-Cn-v2
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-76/dscho/rebase-Cn-v2
Pull-Request: https://github.com/gitgitgadget/git/pull/76

Range-diff vs v1:

 1:  dc36a45068 = 1:  dc36a45068 rebase: really just passthru the `git am` options
 -:  ---------- > 2:  4c2ba52766 rebase: validate -C<n> and --whitespace=<mode> parameters early
Signature · 1 line
gitgitgadget
#18 lore raw

PATCH v2 1/2 rebase: really just passthru the `git am` options

From: Johannes Schindelin <[email protected]>

Currently, we parse the options intended for git am as if we wanted to handle them in git rebase, and then reconstruct them painstakingly to define the git_am_opt variable.

However, there is a much better way (that I was unaware of, at the time when I mentored Pratik to implement these options): OPT_PASSTHRU_ARGV. It is intended for exactly this use case, where command-line options want to be parsed into a separate argv_array.

Let's use this feature.

Incidentally, this also allows us to address a bug discovered by Phillip Wood, where the built-in rebase failed to understand that the -C option takes an optional argument.

---
 builtin/rebase.c | 98 +++++++++++++++++-------------------------------
 1 file changed, 35 insertions(+), 63 deletions(-)
builtin/rebase.c +35 -63
index 0ee06aa363..96ffa80b71 100644
@@ -87,7 +87,7 @@ struct rebase_options {
 		REBASE_FORCE = 1<<3,
 		REBASE_INTERACTIVE_EXPLICIT = 1<<4,
 	} flags;
-	struct strbuf git_am_opt;
+	struct argv_array git_am_opts;
 	const char *action;
 	int signoff;
 	int allow_rerere_autoupdate;
@@ -339,7 +339,7 @@ N_("Resolve all conflicts manually, mark them as resolved with\n"
 static int run_specific_rebase(struct rebase_options *opts)
 {
 	const char *argv[] = { NULL, NULL };
-	struct strbuf script_snippet = STRBUF_INIT;
+	struct strbuf script_snippet = STRBUF_INIT, buf = STRBUF_INIT;
 	int status;
 	const char *backend, *backend_func;
 
@@ -433,7 +433,9 @@ static int run_specific_rebase(struct rebase_options *opts)
 		oid_to_hex(&opts->restrict_revision->object.oid) : NULL);
 	add_var(&script_snippet, "GIT_QUIET",
 		opts->flags & REBASE_NO_QUIET ? "" : "t");
-	add_var(&script_snippet, "git_am_opt", opts->git_am_opt.buf);
+	sq_quote_argv_pretty(&buf, opts->git_am_opts.argv);
+	add_var(&script_snippet, "git_am_opt", buf.buf);
+	strbuf_release(&buf);
 	add_var(&script_snippet, "verbose",
 		opts->flags & REBASE_VERBOSE ? "t" : "");
 	add_var(&script_snippet, "diffstat",
@@ -756,7 +758,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	struct rebase_options options = {
 		.type = REBASE_UNSPECIFIED,
 		.flags = REBASE_NO_QUIET,
-		.git_am_opt = STRBUF_INIT,
+		.git_am_opts = ARGV_ARRAY_INIT,
 		.allow_rerere_autoupdate  = -1,
 		.allow_empty_message = 1,
 		.git_format_patch_opt = STRBUF_INIT,
@@ -777,12 +779,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		ACTION_EDIT_TODO,
 		ACTION_SHOW_CURRENT_PATCH,
 	} action = NO_ACTION;
-	int committer_date_is_author_date = 0;
-	int ignore_date = 0;
-	int ignore_whitespace = 0;
 	const char *gpg_sign = NULL;
-	int opt_c = -1;
-	struct string_list whitespace = STRING_LIST_INIT_NODUP;
 	struct string_list exec = STRING_LIST_INIT_NODUP;
 	const char *rebase_merges = NULL;
 	int fork_point = -1;
@@ -804,15 +801,20 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		{OPTION_NEGBIT, 'n', "no-stat", &options.flags, NULL,
 			N_("do not show diffstat of what changed upstream"),
 			PARSE_OPT_NOARG, NULL, REBASE_DIFFSTAT },
-		OPT_BOOL(0, "ignore-whitespace", &ignore_whitespace,
-			 N_("passed to 'git apply'")),
 		OPT_BOOL(0, "signoff", &options.signoff,
 			 N_("add a Signed-off-by: line to each commit")),
-		OPT_BOOL(0, "committer-date-is-author-date",
-			 &committer_date_is_author_date,
-			 N_("passed to 'git am'")),
-		OPT_BOOL(0, "ignore-date", &ignore_date,
-			 N_("passed to 'git am'")),
+		OPT_PASSTHRU_ARGV(0, "ignore-whitespace", &options.git_am_opts,
+				  NULL, N_("passed to 'git am'"),
+				  PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV(0, "committer-date-is-author-date",
+				  &options.git_am_opts, NULL,
+				  N_("passed to 'git am'"), PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV(0, "ignore-date", &options.git_am_opts, NULL,
+				  N_("passed to 'git am'"), PARSE_OPT_NOARG),
+		OPT_PASSTHRU_ARGV('C', NULL, &options.git_am_opts, N_("n"),
+				  N_("passed to 'git apply'"), 0),
+		OPT_PASSTHRU_ARGV(0, "whitespace", &options.git_am_opts,
+				  N_("action"), N_("passed to 'git apply'"), 0),
 		OPT_BIT('f', "force-rebase", &options.flags,
 			N_("cherry-pick all commits, even if unchanged"),
 			REBASE_FORCE),
@@ -856,10 +858,6 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		{ OPTION_STRING, 'S', "gpg-sign", &gpg_sign, N_("key-id"),
 			N_("GPG-sign commits"),
 			PARSE_OPT_OPTARG, NULL, (intptr_t) "" },
-		OPT_STRING_LIST(0, "whitespace", &whitespace,
-				N_("whitespace"), N_("passed to 'git apply'")),
-		OPT_SET_INT('C', NULL, &opt_c, N_("passed to 'git apply'"),
-			    REBASE_AM),
 		OPT_BOOL(0, "autostash", &options.autostash,
 			 N_("automatically stash/stash pop before and after")),
 		OPT_STRING_LIST('x', "exec", &exec, N_("exec"),
@@ -884,6 +882,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 			 N_("rebase all reachable commits up to the root(s)")),
 		OPT_END(),
 	};
+	int i;
 
 	/*
 	 * NEEDSWORK: Once the builtin rebase has been tested enough
@@ -1064,22 +1063,17 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		    state_dir_base, cmd_live_rebase, buf.buf);
 	}
 
-	if (!(options.flags & REBASE_NO_QUIET))
-		strbuf_addstr(&options.git_am_opt, " -q");
-
-	if (committer_date_is_author_date) {
-		strbuf_addstr(&options.git_am_opt,
-			      " --committer-date-is-author-date");
-		options.flags |= REBASE_FORCE;
+	for (i = 0; i < options.git_am_opts.argc; i++) {
+		const char *option = options.git_am_opts.argv[i];
+		if (!strcmp(option, "--committer-date-is-author-date") ||
+		    !strcmp(option, "--ignore-date") ||
+		    !strcmp(option, "--whitespace=fix") ||
+		    !strcmp(option, "--whitespace=strip"))
+			options.flags |= REBASE_FORCE;
 	}
 
-	if (ignore_whitespace)
-		strbuf_addstr(&options.git_am_opt, " --ignore-whitespace");
-
-	if (ignore_date) {
-		strbuf_addstr(&options.git_am_opt, " --ignore-date");
-		options.flags |= REBASE_FORCE;
-	}
+	if (!(options.flags & REBASE_NO_QUIET))
+		argv_array_push(&options.git_am_opts, "-q");
 
 	if (options.keep_empty)
 		imply_interactive(&options, "--keep-empty");
@@ -1089,23 +1083,6 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		options.gpg_sign_opt = xstrfmt("-S%s", gpg_sign);
 	}
 
-	if (opt_c >= 0)
-		strbuf_addf(&options.git_am_opt, " -C%d", opt_c);
-
-	if (whitespace.nr) {
-		int i;
-
-		for (i = 0; i < whitespace.nr; i++) {
-			const char *item = whitespace.items[i].string;
-
-			strbuf_addf(&options.git_am_opt, " --whitespace=%s",
-				    item);
-
-			if ((!strcmp(item, "fix")) || (!strcmp(item, "strip")))
-				options.flags |= REBASE_FORCE;
-		}
-	}
-
 	if (exec.nr) {
 		int i;
 
@@ -1181,23 +1158,18 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		break;
 	}
 
-	if (options.git_am_opt.len) {
-		const char *p;
-
+	if (options.git_am_opts.argc) {
 		/* all am options except -q are compatible only with --am */
-		strbuf_reset(&buf);
-		strbuf_addbuf(&buf, &options.git_am_opt);
-		strbuf_addch(&buf, ' ');
-		while ((p = strstr(buf.buf, " -q ")))
-			strbuf_splice(&buf, p - buf.buf, 4, " ", 1);
-		strbuf_trim(&buf);
+		for (i = options.git_am_opts.argc - 1; i >= 0; i--)
+			if (strcmp(options.git_am_opts.argv[i], "-q"))
+				break;
 
-		if (is_interactive(&options) && buf.len)
+		if (is_interactive(&options) && i >= 0)
 			die(_("error: cannot combine interactive options "
 			      "(--interactive, --exec, --rebase-merges, "
 			      "--preserve-merges, --keep-empty, --root + "
 			      "--onto) with am options (%s)"), buf.buf);
-		if (options.type == REBASE_MERGE && buf.len)
+		if (options.type == REBASE_MERGE && i >= 0)
 			die(_("error: cannot combine merge options (--merge, "
 			      "--strategy, --strategy-option) with am options "
 			      "(%s)"), buf.buf);
@@ -1207,7 +1179,7 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 		if (options.type == REBASE_PRESERVE_MERGES)
 			die("cannot combine '--signoff' with "
 			    "'--preserve-merges'");
-		strbuf_addstr(&options.git_am_opt, " --signoff");
+		argv_array_push(&options.git_am_opts, "--signoff");
 		options.flags |= REBASE_FORCE;
 	}
 
Signature · 1 line
gitgitgadget
#19 lore raw

PATCH v2 2/2 rebase: validate -C<n> and --whitespace=<mode> parameters early

In reply to Johannes Schindelin via GitGitGadget

From: Johannes Schindelin <[email protected]>

It is a good idea to error out early upon seeing, say, -Cbad, rather than starting the rebase only to have the --am backend complain later.

Let's do this.

The only options accepting parameters which we pass through to git am (which may, or may not, forward them to git apply) are -C and --whitespace. The other options we pass through do not accept parameters, so we do not have to validate them here.

---
 builtin/rebase.c          | 12 +++++++++++-
 t/t3406-rebase-message.sh |  7 +++++++
 2 files changed, 18 insertions(+), 1 deletion(-)
builtin/rebase.c +11 -1
index 96ffa80b71..571cf899d5 100644
@@ -1064,12 +1064,22 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	}
 
 	for (i = 0; i < options.git_am_opts.argc; i++) {
-		const char *option = options.git_am_opts.argv[i];
+		const char *option = options.git_am_opts.argv[i], *p;
 		if (!strcmp(option, "--committer-date-is-author-date") ||
 		    !strcmp(option, "--ignore-date") ||
 		    !strcmp(option, "--whitespace=fix") ||
 		    !strcmp(option, "--whitespace=strip"))
 			options.flags |= REBASE_FORCE;
+		else if (skip_prefix(option, "-C", &p)) {
+			while (*p)
+				if (!isdigit(*(p++)))
+					die(_("switch `C' expects a "
+					      "numerical value"));
+		} else if (skip_prefix(option, "--whitespace=", &p)) {
+			if (*p && strcmp(p, "warn") && strcmp(p, "nowarn") &&
+			    strcmp(p, "error") && strcmp(p, "error-all"))
+				die("Invalid whitespace option: '%s'", p);
+		}
 	}
 
 	if (!(options.flags & REBASE_NO_QUIET))
t/t3406-rebase-message.sh +7 -0
index 0392e36d23..2c79eed4fe 100755
@@ -84,4 +84,11 @@ test_expect_success 'rebase --onto outputs the invalid ref' '
 	test_i18ngrep "invalid-ref" err
 '
 
+test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' '
+	test_must_fail git rebase -Cnot-a-number HEAD 2>err &&
+	test_i18ngrep "numerical value" err &&
+	test_must_fail git rebase --whitespace=bad HEAD 2>err &&
+	test_i18ngrep "Invalid whitespace option" err
+'
+
 test_done
Signature · 1 line
gitgitgadget
#20 lore raw

Re: [PATCH v2 2/2] rebase: validate -C<n> and --whitespace=<mode> parameters early

Hi Johannes

Thanks for doing this, I think this patch is good. I've not checked the first patch as I think it is the same as before judging from the covering letter.

Best Wishes

Phillip

Quoted 63 lines · Johannes Schindelin via GitGitGadget

From: Johannes Schindelin <[email protected]>

It is a good idea to error out early upon seeing, say, -Cbad, rather than starting the rebase only to have the --am backend complain later.

Let's do this.

The only options accepting parameters which we pass through to git am (which may, or may not, forward them to git apply) are -C and --whitespace. The other options we pass through do not accept parameters, so we do not have to validate them here.

---
  builtin/rebase.c          | 12 +++++++++++-
  t/t3406-rebase-message.sh |  7 +++++++
  2 files changed, 18 insertions(+), 1 deletion(-)
builtin/rebase.c +11 -1
index 96ffa80b71..571cf899d5 100644
@@ -1064,12 +1064,22 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
  	}
 
  	for (i = 0; i < options.git_am_opts.argc; i++) {
-		const char *option = options.git_am_opts.argv[i];
+		const char *option = options.git_am_opts.argv[i], *p;
  		if (!strcmp(option, "--committer-date-is-author-date") ||
  		    !strcmp(option, "--ignore-date") ||
  		    !strcmp(option, "--whitespace=fix") ||
  		    !strcmp(option, "--whitespace=strip"))
  			options.flags |= REBASE_FORCE;
+		else if (skip_prefix(option, "-C", &p)) {
+			while (*p)
+				if (!isdigit(*(p++)))
+					die(_("switch `C' expects a "
+					      "numerical value"));
+		} else if (skip_prefix(option, "--whitespace=", &p)) {
+			if (*p && strcmp(p, "warn") && strcmp(p, "nowarn") &&
+			    strcmp(p, "error") && strcmp(p, "error-all"))
+				die("Invalid whitespace option: '%s'", p);
+		}
  	}
 
  	if (!(options.flags & REBASE_NO_QUIET))
t/t3406-rebase-message.sh +7 -0
index 0392e36d23..2c79eed4fe 100755
@@ -84,4 +84,11 @@ test_expect_success 'rebase --onto outputs the invalid ref' '
  	test_i18ngrep "invalid-ref" err
  '
 
+test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' '
+	test_must_fail git rebase -Cnot-a-number HEAD 2>err &&
+	test_i18ngrep "numerical value" err &&
+	test_must_fail git rebase --whitespace=bad HEAD 2>err &&
+	test_i18ngrep "Invalid whitespace option" err
+'
+
  test_done
#21 lore raw

Re: [PATCH v2 2/2] rebase: validate -C<n> and --whitespace=<mode> parameters early

Hi Phillip,

Quoted 1 line · Phillip Wood

Thanks for doing this, I think this patch is good.

Thanks!

Quoted 2 lines

I've not checked the first patch as I think it is the same as before judging from the covering letter.

Indeed, that's what the range-diff said. Sorry for not stating this explicitly.

Thank you for your review,
Dscho

Quoted 75 lines

Best Wishes

Phillip

Quoted 67 lines · Johannes Schindelin via GitGitGadget

From: Johannes Schindelin <[email protected]>

It is a good idea to error out early upon seeing, say, -Cbad, rather than starting the rebase only to have the --am backend complain later.

Let's do this.

The only options accepting parameters which we pass through to git am (which may, or may not, forward them to git apply) are -C and --whitespace. The other options we pass through do not accept parameters, so we do not have to validate them here.

---
  builtin/rebase.c          | 12 +++++++++++-
  t/t3406-rebase-message.sh |  7 +++++++
  2 files changed, 18 insertions(+), 1 deletion(-)
builtin/rebase.c +1 -1
index 96ffa80b71..571cf899d5 100644
@@ -1064,12 +1064,22 @@ int cmd_rebase(int argc, const char **argv,
 const char *prefix)
   }
 
  	for (i = 0; i < options.git_am_opts.argc; i++) {
-		const char *option = options.git_am_opts.argv[i];
+		const char *option = options.git_am_opts.argv[i], *p;
    if (!strcmp(option, "--committer-date-is-author-date") ||

!strcmp(option, "--ignore-date") || !strcmp(option, "--whitespace=fix") || !strcmp(option, "--whitespace=strip"))
options.flags |= REBASE_FORCE;

builtin/rebase.c +7 -0
+		else if (skip_prefix(option, "-C", &p)) {
+			while (*p)
+				if (!isdigit(*(p++)))
+					die(_("switch `C' expects a "
+					      "numerical value"));
+		} else if (skip_prefix(option, "--whitespace=", &p)) {
+			if (*p && strcmp(p, "warn") && strcmp(p, "nowarn")

&&

builtin/rebase.c +3 -0
+			    strcmp(p, "error") && strcmp(p, "error-all"))
+				die("Invalid whitespace option: '%s'", p);
+		}
   }
 
  	if (!(options.flags & REBASE_NO_QUIET))
t/t3406-rebase-message.sh +0 -0
index 0392e36d23..2c79eed4fe 100755
@@ -84,4 +84,11 @@ test_expect_success 'rebase --onto outputs the

invalid ref' '
test_i18ngrep "invalid-ref" err
'

+test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' '

t/t3406-rebase-message.sh +6 -0
+	test_must_fail git rebase -Cnot-a-number HEAD 2>err &&
+	test_i18ngrep "numerical value" err &&
+	test_must_fail git rebase --whitespace=bad HEAD 2>err &&
+	test_i18ngrep "Invalid whitespace option" err
+'
+
  test_done
#22 lore raw

Re: [PATCH v2 2/2] rebase: validate -C<n> and --whitespace=<mode> parameters early

In reply to Johannes Schindelin via GitGitGadget
Quoted 51 lines · Johannes Schindelin via GitGitGadget

From: Johannes Schindelin <[email protected]>

It is a good idea to error out early upon seeing, say, -Cbad, rather than starting the rebase only to have the --am backend complain later.

Let's do this.

The only options accepting parameters which we pass through to git am (which may, or may not, forward them to git apply) are -C and --whitespace. The other options we pass through do not accept parameters, so we do not have to validate them here.

---
 builtin/rebase.c          | 12 +++++++++++-
 t/t3406-rebase-message.sh |  7 +++++++
 2 files changed, 18 insertions(+), 1 deletion(-)
builtin/rebase.c +11 -1
index 96ffa80b71..571cf899d5 100644
@@ -1064,12 +1064,22 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	}
 
 	for (i = 0; i < options.git_am_opts.argc; i++) {
-		const char *option = options.git_am_opts.argv[i];
+		const char *option = options.git_am_opts.argv[i], *p;
 		if (!strcmp(option, "--committer-date-is-author-date") ||
 		    !strcmp(option, "--ignore-date") ||
 		    !strcmp(option, "--whitespace=fix") ||
 		    !strcmp(option, "--whitespace=strip"))
 			options.flags |= REBASE_FORCE;
+		else if (skip_prefix(option, "-C", &p)) {
+			while (*p)
+				if (!isdigit(*(p++)))
+					die(_("switch `C' expects a "
+					      "numerical value"));
+		} else if (skip_prefix(option, "--whitespace=", &p)) {
+			if (*p && strcmp(p, "warn") && strcmp(p, "nowarn") &&
+			    strcmp(p, "error") && strcmp(p, "error-all"))
+				die("Invalid whitespace option: '%s'", p);
+		}
 	}
 
 	if (!(options.flags & REBASE_NO_QUIET))
t/t3406-rebase-message.sh +0 -0
index 0392e36d23..2c79eed4fe 100755

t/t3406-rebase-message.sh test_expect_success 'rebase --onto outputs the invalid ref' ' · line 93

Quoted 10 lines
t/t3406-rebase-message.sh +7 -0
@@ -84,4 +84,11 @@ test_expect_success 'rebase --onto outputs the invalid ref' '
 	test_i18ngrep "invalid-ref" err
 '
 
+test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' '
+	test_must_fail git rebase -Cnot-a-number HEAD 2>err &&
+	test_i18ngrep "numerical value" err &&
+	test_must_fail git rebase --whitespace=bad HEAD 2>err &&
+	test_i18ngrep "Invalid whitespace option" err
+'
+

The combination of this gitster/js/rebase-am-options and my gitster/ab/rebase-in-c-escape-hatch breaks tests under GIT_TEST_REBASE_USE_BUILTIN=false for obvious reasons. The C version is now more strict.

#23 lore raw

Re: [PATCH v2 2/2] rebase: validate -C<n> and --whitespace=<mode> parameters early

Hi Ævar,

Quoted 57 lines · Ævar Arnfjörð Bjarmason
Quoted 54 lines · Johannes Schindelin via GitGitGadget

From: Johannes Schindelin <[email protected]>

It is a good idea to error out early upon seeing, say, -Cbad, rather than starting the rebase only to have the --am backend complain later.

Let's do this.

The only options accepting parameters which we pass through to git am (which may, or may not, forward them to git apply) are -C and --whitespace. The other options we pass through do not accept parameters, so we do not have to validate them here.

---
 builtin/rebase.c          | 12 +++++++++++-
 t/t3406-rebase-message.sh |  7 +++++++
 2 files changed, 18 insertions(+), 1 deletion(-)
builtin/rebase.c +11 -1
index 96ffa80b71..571cf899d5 100644
@@ -1064,12 +1064,22 @@ int cmd_rebase(int argc, const char **argv, const char *prefix)
 	}
 
 	for (i = 0; i < options.git_am_opts.argc; i++) {
-		const char *option = options.git_am_opts.argv[i];
+		const char *option = options.git_am_opts.argv[i], *p;
 		if (!strcmp(option, "--committer-date-is-author-date") ||
 		    !strcmp(option, "--ignore-date") ||
 		    !strcmp(option, "--whitespace=fix") ||
 		    !strcmp(option, "--whitespace=strip"))
 			options.flags |= REBASE_FORCE;
+		else if (skip_prefix(option, "-C", &p)) {
+			while (*p)
+				if (!isdigit(*(p++)))
+					die(_("switch `C' expects a "
+					      "numerical value"));
+		} else if (skip_prefix(option, "--whitespace=", &p)) {
+			if (*p && strcmp(p, "warn") && strcmp(p, "nowarn") &&
+			    strcmp(p, "error") && strcmp(p, "error-all"))
+				die("Invalid whitespace option: '%s'", p);
+		}
 	}
 
 	if (!(options.flags & REBASE_NO_QUIET))
t/t3406-rebase-message.sh +0 -0
index 0392e36d23..2c79eed4fe 100755
@@ -84,4 +84,11 @@ test_expect_success 'rebase --onto outputs the invalid ref' '
 	test_i18ngrep "invalid-ref" err
 '
 
Quoted 12 lines

t/t3406-rebase-message.sh test_expect_success 'rebase --onto outputs the invalid ref' ' · line 93

Quoted 7 lines
t/t3406-rebase-message.sh +7 -0
+test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' '
+	test_must_fail git rebase -Cnot-a-number HEAD 2>err &&
+	test_i18ngrep "numerical value" err &&
+	test_must_fail git rebase --whitespace=bad HEAD 2>err &&
+	test_i18ngrep "Invalid whitespace option" err
+'
+

The combination of this gitster/js/rebase-am-options and my gitster/ab/rebase-in-c-escape-hatch breaks tests under GIT_TEST_REBASE_USE_BUILTIN=false for obvious reasons. The C version is now more strict.

Maybe you can concoct a prereq for this test? Something like

test_lazy_prereq BUILTIN_REBASE '

	test true = "${GIT_TEST_REBASE_USE_BUILTIN:-true}"

'

Ciao,
Dscho

#24 lore raw

PATCH 1/1 rebase: mark a test as failing with rebase.useBuiltin=false

Mark a test added in 04519d7201 ("rebase: validate -C<n> and --whitespace=<mode> parameters early", 2018-11-14) as only succeeding with the builtin version of rebase. It would be nice if the shellscript version had the same fix, but it's on its way out, and the author is not interested in fixing it[1].

This makes the entire test suite pass again with the GIT_TEST_REBASE_USE_BUILTIN=false mode added in my 62c23938fa ("tests:
add a special setup where rebase.useBuiltin is off", 2018-11-14).

1. https://public-inbox.org/git/[email protected]/

---

Quoted 6 lines · Johannes Schindelin

[...]
Maybe you can concoct a prereq for this test? Something like

test_lazy_prereq BUILTIN_REBASE '

      test true = "${GIT_TEST_REBASE_USE_BUILTIN:-true}"

'

It's better to just mark the test as needing the prereq turned off. E.g. this is what we do for the split index tests & now for the gettext tests. That way we always run the test, but just indicate that it relies on GIT_TEST_REBASE_USE_BUILTIN being unset.

 t/t3406-rebase-message.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
t/t3406-rebase-message.sh +2 -1
index 38bd876cab..77e5bbb3d5 100755
@@ -84,7 +84,8 @@ test_expect_success 'rebase --onto outputs the invalid ref' '
 	test_i18ngrep "invalid-ref" err
 '
 
-test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' '
+test_expect_success 'builtin rebase: error out early upon -C<n> or --whitespace=<bad>' '
+	sane_unset GIT_TEST_REBASE_USE_BUILTIN &&
 	test_must_fail git rebase -Cnot-a-number HEAD 2>err &&
 	test_i18ngrep "numerical value" err &&
 	test_must_fail git rebase --whitespace=bad HEAD 2>err &&
Signature · 1 line
2.20.0.rc0.387.gc7a69e6b6c
#25 lore raw

Re: [PATCH] rebase: mark a test as failing with rebase.useBuiltin=false

Hi Ævar,

Quoted 5 lines · Ævar Arnfjörð Bjarmason

Mark a test added in 04519d7201 ("rebase: validate -C<n> and --whitespace=<mode> parameters early", 2018-11-14) as only succeeding with the builtin version of rebase. It would be nice if the shellscript version had the same fix, but it's on its way out, and the author is not interested in fixing it[1].

It's not that I am not interested in fixing it. It's more like I hoped that I could work on Git for Windows v2.20.0-rc0 and rely on you to fix this bug.

Now that Git for Windows v2.20.0-rc0 is out (see
https://github.com/git-for-windows/git/releases/tag/v2.20.0-rc0.windows.1),
expect a patch soon (see https://github.com/gitgitgadget/git/pull/86).

Ciao,
Dscho

Quoted 45 lines

This makes the entire test suite pass again with the GIT_TEST_REBASE_USE_BUILTIN=false mode added in my 62c23938fa ("tests:
add a special setup where rebase.useBuiltin is off", 2018-11-14).

1. https://public-inbox.org/git/[email protected]/

---

Quoted 6 lines · Johannes Schindelin

[...]
Maybe you can concoct a prereq for this test? Something like

test_lazy_prereq BUILTIN_REBASE '

      test true = "${GIT_TEST_REBASE_USE_BUILTIN:-true}"

'

It's better to just mark the test as needing the prereq turned off. E.g. this is what we do for the split index tests & now for the gettext tests. That way we always run the test, but just indicate that it relies on GIT_TEST_REBASE_USE_BUILTIN being unset.

 t/t3406-rebase-message.sh | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
t/t3406-rebase-message.sh +2 -1
index 38bd876cab..77e5bbb3d5 100755
@@ -84,7 +84,8 @@ test_expect_success 'rebase --onto outputs the invalid ref' '
 	test_i18ngrep "invalid-ref" err
 '
 
-test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' '
+test_expect_success 'builtin rebase: error out early upon -C<n> or --whitespace=<bad>' '
+	sane_unset GIT_TEST_REBASE_USE_BUILTIN &&
 	test_must_fail git rebase -Cnot-a-number HEAD 2>err &&
 	test_i18ngrep "numerical value" err &&
 	test_must_fail git rebase --whitespace=bad HEAD 2>err &&
Signature · 1 line
2.20.0.rc0.387.gc7a69e6b6c
#26 lore raw

Git Test Coverage Report (v2.20.0-rc0)

Here is a test coverage report for the uncovered lines introduced in v2.20.0-rc0 compared to v2.19.1.

Thanks,

-Stolee

[1] https://dev.azure.com/git/git/_build/results?buildId=263&view=logs

---

apply.c
eccb5a5f3d 4071) return get_oid_hex(p->old_oid_prefix, oid); 517fe807d6 4776) BUG_ON_OPT_NEG(unset);
735ca208c5 4830) return -1;

blame.c
a470beea39  113)  !strcmp(r->index->cache[-1 - pos]->name, path)) a470beea39  272) int pos = index_name_pos(r->index, path, len); a470beea39  274) mode = r->index->cache[pos]->ce_mode;

builtin/add.c
d1664e73ad builtin/add.c 458) die(_("index file corrupt"));

builtin/am.c
2abf350385 1362) repo_init_revisions(the_repository, &rev_info, NULL); fce5664805 2117) *opt_value = PATCH_FORMAT_UNKNOWN;

builtin/blame.c
517fe807d6 builtin/blame.c    759) BUG_ON_OPT_NEG(unset);

builtin/cat-file.c
98f425b453 builtin/cat-file.c  56) die("unable to stream %s to stdout", oid_to_hex(oid)); 0eb8d3767c builtin/cat-file.c 609) return error(_("only one batch option may be specified"));

builtin/checkout.c
fa655d8411 builtin/checkout.c  539) return 0;
fa655d8411 builtin/checkout.c  953) return error(_("index file corrupt"));

builtin/difftool.c
4a7e27e957 441) if (oideq(&loid, &roid))

builtin/fast-export.c
4a7e27e957 builtin/fast-export.c  387) if (oideq(&ospec->oid, &spec->oid) &&

builtin/fetch.c
builtin/fsck.c
b29759d89a builtin/fsck.c 613) fprintf(stderr, "Checking %s link\n", head_ref_name); b29759d89a builtin/fsck.c 618) return error("Invalid %s", head_ref_name); 454ea2e4d7 builtin/fsck.c 769) for (p = get_all_packs(the_repository); p; 66ec0390e7 builtin/fsck.c 888) midx_argv[2] = "--object-dir"; 66ec0390e7 builtin/fsck.c 889) midx_argv[3] = alt->path;
66ec0390e7 builtin/fsck.c 890) if (run_command(&midx_verify)) 66ec0390e7 builtin/fsck.c 891) errors_found |= ERROR_COMMIT_GRAPH;

builtin/gc.c
3029970275 builtin/gc.c 461) ret = error_errno(_("cannot stat '%s'"), gc_log_path); 3029970275 builtin/gc.c 470) ret = error_errno(_("cannot read '%s'"), gc_log_path); fec2ed2187 builtin/gc.c 495) die(FAILED_RUN, pack_refs_cmd.argv[0]); fec2ed2187 builtin/gc.c 498) die(FAILED_RUN, reflog.argv[0]); 3029970275 builtin/gc.c 585) exit(128);
fec2ed2187 builtin/gc.c 637) die(FAILED_RUN, repack.argv[0]); fec2ed2187 builtin/gc.c 647) die(FAILED_RUN, prune.argv[0]); fec2ed2187 builtin/gc.c 654) die(FAILED_RUN, prune_worktrees.argv[0]); fec2ed2187 builtin/gc.c 658) die(FAILED_RUN, rerere.argv[0]);

builtin/grep.c
76e9bdc437 builtin/grep.c  424) grep_read_unlock();
fd6263fb73 builtin/grep.c 1051) warning(_("invalid option combination, ignoring --threads")); fd6263fb73 builtin/grep.c 1057) die(_("invalid number of threads specified (%d)"), num_threads);

builtin/help.c
e6e76baaf4 builtin/help.c 429) if (!exclude_guides || alias[0] == '!') { e6e76baaf4 builtin/help.c 430) printf_ln(_("'%s' is aliased to '%s'"), cmd, alias); e6e76baaf4 builtin/help.c 431) free(alias);
e6e76baaf4 builtin/help.c 432) exit(0);
e6e76baaf4 builtin/help.c 441) fprintf_ln(stderr, _("'%s' is aliased to '%s'"), cmd, alias); e6e76baaf4 builtin/help.c 442) count = split_cmdline(alias, &argv); e6e76baaf4 builtin/help.c 443) if (count < 0)
e6e76baaf4 builtin/help.c 444) die(_("bad alias.%s string: %s"), cmd, e6e76baaf4 builtin/help.c 446) free(argv);
e6e76baaf4 builtin/help.c 448) return alias;

builtin/log.c
517fe807d6 builtin/log.c 1196) BUG_ON_OPT_NEG(unset);
2e6fd71a52 builtin/log.c 1472) die(_("failed to infer range-diff ranges")); ee6cbf712e builtin/log.c 1818) die(_("--interdiff requires --cover-letter or single patch")); 8631bf1cdd builtin/log.c 1828) else if (!rdiff_prev)
8631bf1cdd builtin/log.c 1829) die(_("--creation-factor requires --range-diff")); 40ce41604d builtin/log.c 1833) die(_("--range-diff requires --cover-letter or single patch"));

builtin/multi-pack-index.c
6d68e6a461 35) usage_with_options(builtin_multi_pack_index_usage, 6d68e6a461 39) die(_("too many arguments"));
6d68e6a461 48) die(_("unrecognized verb: %s"), argv[0]);

builtin/pack-objects.c
6a22d52126 builtin/pack-objects.c 1091) continue;
2fa233a554 builtin/pack-objects.c 1512) hashcpy(base_oid.hash, base_sha1); 2fa233a554 builtin/pack-objects.c 1513) if (!in_same_island(&delta->idx.oid, &base_oid)) 2fa233a554 builtin/pack-objects.c 1514) return 0;
28b8a73080 builtin/pack-objects.c 2793) depth++;
108f530385 builtin/pack-objects.c 2797) oe_set_tree_depth(&to_pack, ent, depth); 454ea2e4d7 builtin/pack-objects.c 2981) p = get_all_packs(the_repository);

builtin/pack-redundant.c
454ea2e4d7 builtin/pack-redundant.c 580) struct packed_git *p = get_all_packs(the_repository); 454ea2e4d7 builtin/pack-redundant.c 595) struct packed_git *p = get_all_packs(the_repository);

builtin/pull.c
01a31f3bca 565) die(_("unable to access commit %s"),

builtin/rebase--interactive.c
53bbcfbde7 builtin/rebase--interactive2.c  24) return error(_("no HEAD?"));
53bbcfbde7 builtin/rebase--interactive2.c  51) return error_errno(_("could not create temporary %s"), path_state_dir()); 53bbcfbde7 builtin/rebase--interactive2.c  57) return error_errno(_("could not mark as interactive")); 53bbcfbde7 builtin/rebase--interactive2.c  77) return -1;
53bbcfbde7 builtin/rebase--interactive2.c  81) return -1;
53bbcfbde7 builtin/rebase--interactive2.c  87) free(revisions);
53bbcfbde7 builtin/rebase--interactive2.c  88) free(shortrevisions);
53bbcfbde7 builtin/rebase--interactive2.c  90) return -1;
53bbcfbde7 builtin/rebase--interactive2.c  98) free(revisions);
53bbcfbde7 builtin/rebase--interactive2.c  99) free(shortrevisions);
53bbcfbde7 builtin/rebase--interactive2.c 101) return error_errno(_("could not open %s"), rebase_path_todo()); 53bbcfbde7 builtin/rebase--interactive2.c 106) argv_array_push(&make_script_args, restrict_revision); 53bbcfbde7 builtin/rebase--interactive2.c 114) error(_("could not generate todo list"));
53bbcfbde7 builtin/rebase--interactive2.c 206) usage_with_options(builtin_rebase_interactive_usage, options); 53bbcfbde7 builtin/rebase--interactive2.c 220) warning(_("--[no-]rebase-cousins has no effect without " 0af129b2ed builtin/rebase--interactive2.c 226) die(_("a base commit must be provided with --upstream or --onto")); 34b47315d9 builtin/rebase--interactive.c  261) ret = rearrange_squash();
34b47315d9 builtin/rebase--interactive.c  262) break;
34b47315d9 builtin/rebase--interactive.c  264) ret = sequencer_add_exec_commands(cmd);
34b47315d9 builtin/rebase--interactive.c  265) break;

builtin/rebase.c
62c23938fa   55) return env;
55071ea248   65) strbuf_trim(&out);
55071ea248   66) ret = !strcmp("true", out.buf);
55071ea248   67) strbuf_release(&out);
002ee2fe68  119) die(_("%s requires an interactive rebase"), option); f95736288a  152) return error_errno(_("could not read '%s'"), path); f95736288a  166) return -1;
f95736288a  171) return error(_("could not get 'onto': '%s'"), buf.buf); f95736288a  182) return -1;
f95736288a  183) } else if (read_one(state_dir_path("head", opts), &buf)) f95736288a  184) return -1;
f95736288a  186) return error(_("invalid orig-head: '%s'"), buf.buf); f95736288a  190) return -1;
f95736288a  192) opts->flags &= ~REBASE_NO_QUIET;
73d51ed0a5  200) opts->signoff = 1;
73d51ed0a5  201) opts->flags |= REBASE_FORCE;
ead98c111b  208) return -1;
12026a412c  223) return -1;
ba1905a5fe  231) return -1;
ba1905a5fe  239) return -1;
6defce2b02  259) return error(_("Could not read '%s'"), path); 6defce2b02  277) res = error(_("Cannot store %s"), autostash.buf); 6defce2b02  281) return res;
bc24382c2b  379) argv_array_pushf(&child.args,
bc24382c2b  381) oid_to_hex(&opts->restrict_revision->object.oid)); ac7f467fef  515) struct strbuf dir = STRBUF_INIT;
6defce2b02  517) apply_autostash(opts);
ac7f467fef  518) strbuf_addstr(&dir, opts->state_dir);
ac7f467fef  519) remove_dir_recursively(&dir, 0);
ac7f467fef  520) strbuf_release(&dir);
ac7f467fef  521) die("Nothing to do");
3249c1251e  556) ret = -1;
3249c1251e  557) goto leave_reset_head;
bac2a1e36f  561) ret = error(_("could not determine HEAD revision")); bac2a1e36f  562) goto leave_reset_head;
3249c1251e  580) ret = error(_("could not read index"));
3249c1251e  581) goto leave_reset_head;
bac2a1e36f  585) ret = error(_("failed to find tree of %s"), oid_to_hex(oid)); bac2a1e36f  586) goto leave_reset_head;
3249c1251e  590) ret = error(_("failed to find tree of %s"), oid_to_hex(oid)); 3249c1251e  591) goto leave_reset_head;
ac7f467fef  603) ret = error(_("could not write index"));
3249c1251e  604) goto leave_reset_head;
ac7f467fef  621) } else if (old_orig)
ac7f467fef  622) delete_ref(NULL, "ORIG_HEAD", old_orig, 0); bff014dac7  655) opts->flags &= !REBASE_DIFFSTAT;
9a48a615b4  689) return 1;
9a48a615b4  705) return 0;
55071ea248  916) const char *path = mkpath("%s/git-legacy-rebase", 55071ea248  919) if (sane_execvp(path, (char **)argv) < 0) 55071ea248  920) die_errno(_("could not exec %s"), path);
0eabf4b95c  938) die(_("It looks like 'git am' is in progress. Cannot rebase.")); f28d40d3a9  975) usage_with_options(builtin_rebase_usage,
f95736288a  995) die(_("Cannot read HEAD"));
f95736288a  999) die(_("could not read index"));
f95736288a 1013) exit(1);
122420c295 1026) die(_("could not discard worktree changes")); 122420c295 1029) exit(1);
5e5d96197c 1040) exit(1);
5e5d96197c 1044) die(_("could not move back to %s"),
5a61494539 1055) die(_("could not remove '%s'"), options.state_dir); c54dacb50e 1074) const char *last_slash = strrchr(options.state_dir, '/'); c54dacb50e 1075) const char *state_dir_base =
c54dacb50e 1076) last_slash ? last_slash + 1 : options.state_dir; c54dacb50e 1077) const char *cmd_live_rebase =
c54dacb50e 1079) strbuf_reset(&buf);
c54dacb50e 1080) strbuf_addf(&buf, "rm -fr \"%s\"", options.state_dir); c54dacb50e 1081) die(_("It seems that there is already a %s directory, and\n" 3c3588c7d3 1138) else if (strcmp("no-rebase-cousins", rebase_merges)) 3c3588c7d3 1139) die(_("Unknown mode: %s"), rebase_merges); ba1905a5fe 1161) die(_("--strategy requires --merge or --interactive")); cda614e489 1179) strbuf_addstr(&options.git_format_patch_opt, " --progress"); ac7f467fef 1188) options.state_dir = apply_dir();
ac7f467fef 1189) break;
ac7f467fef 1261) die(_("invalid upstream '%s'"), options.upstream_name); 9dba809a69 1267) die(_("Could not create new root commit")); e65123a71d 1317) die(_("fatal: no such branch/commit '%s'"), ac7f467fef 1325) die(_("No such ref: %s"), "HEAD");
ac7f467fef 1337) die(_("Could not resolve HEAD to a revision")); e0333e5c63 1350) die(_("could not read index"));
6defce2b02 1377) die(_("Cannot autostash"));
6defce2b02 1380) die(_("Unexpected stash response: '%s'"), 6defce2b02 1386) die(_("Could not create directory for '%s'"), 6defce2b02 1392) die(_("could not reset --hard"));
e65123a71d 1436) ret = !!error(_("could not parse '%s'"),
e65123a71d 1438) goto cleanup;
e65123a71d 1447) ret = !!error(_("could not switch to "
1ed9c14ff2 1457)  resolve_ref_unsafe("HEAD", 0, NULL, &flag)) 1ed9c14ff2 1458) puts(_("HEAD is up to date."));
9a48a615b4 1467)  resolve_ref_unsafe("HEAD", 0, NULL, &flag)) 9a48a615b4 1468) puts(_("HEAD is up to date, rebase forced."));

builtin/reflog.c
c9ef0d95eb builtin/reflog.c 580) all_worktrees = 0; c9ef0d95eb builtin/reflog.c 616) continue;

builtin/remote.c
5025425dff builtin/remote.c  864) return error(_("No such remote: '%s'"), name);

builtin/repack.c
16d75fa48d  48) use_delta_islands = git_config_bool(var, value);
16d75fa48d  49) return 0;
2f0c9e9a9b 239) die("repack: Expecting full hex object ID lines only from pack-objects."); 2f0c9e9a9b 411) die("repack: Expecting full hex object ID lines only from pack-objects.");

builtin/rerere.c
2373b65059 builtin/rerere.c  79) warning(_("'git rerere forget' without paths is deprecated")); 2373b65059 builtin/rerere.c 111) die(_("unable to generate diff for '%s'"), rerere_path(id, NULL));

builtin/rev-list.c
7c0fe330d5 builtin/rev-list.c 227) die("unexpected missing %s object '%s'", 7c0fe330d5 builtin/rev-list.c 228)     type_name(obj->type), oid_to_hex(&obj->oid));

builtin/show-branch.c
9001dc2a74 builtin/show-branch.c 430) if (get_oid(refname + ofs, &tmp) || !oideq(&tmp, oid)) 517fe807d6 builtin/show-branch.c 607) BUG_ON_OPT_NEG(unset);

builtin/show-ref.c
517fe807d6 builtin/show-ref.c 154) BUG_ON_OPT_NEG(unset);

builtin/submodule--helper.c
ee69b2a90c 1469) die(_("Invalid update mode '%s' for submodule path '%s'"), ee69b2a90c 1473) die(_("Invalid update mode '%s' configured for submodule path '%s'"), ee69b2a90c 1476) out->type = sub->update_strategy.type;
ee69b2a90c 1477) out->command = sub->update_strategy.command;
ee69b2a90c 1497) die("submodule--helper update-module-clone expects <just-cloned> <path> [<update>]"); e0a862fdaf 1648) url = sub->url;
74d4731da1 2057) die(_("could not get a repository handle for submodule '%s'"), path);

builtin/unpack-objects.c
4a7e27e957 builtin/unpack-objects.c 306) if (oideq(&info->base_oid, &obj_list[nr].oid) ||

builtin/update-index.c
4a7e27e957 builtin/update-index.c  672) if (oideq(&ce_2->oid, &ce_3->oid) &&

builtin/worktree.c
e5353bef55  60) error_errno(_("failed to delete '%s'"), sb.buf);
e19831c94f 251)     die(_("unable to re-add worktree '%s'"), path);
68a6b3a1bd 793) die(_("cannot move a locked working tree, lock reason: %s\nuse 'move -f -f' to override or unlock first"), f4143101cb 906) die(_("cannot remove a locked working tree, lock reason: %s\nuse 'remove -f -f' to override or unlock first"),

bundle.c
2c8ee1f53c 267) error_errno(_("unable to dup bundle descriptor")); 2c8ee1f53c 268) child_process_clear(&pack_objects); 2c8ee1f53c 269) return -1;
2c8ee1f53c 478) rollback_lock_file(&lock);

cache-tree.c
combine-diff.c
0074c9110d  377) state->sline[state->nb-1].p_lno = 0074c9110d  378) xcalloc(state->num_parent, sizeof(unsigned long));

commit-graph.c
20fd6d5799   79) return 0;
6cc017431c  275) return 0;

commit-reach.c
5227c38566 134) return ret;
5227c38566 282) return 1;
5227c38566 314) return ret;
5227c38566 317) return ret;
1d614d41e5 395) return 0;
1d614d41e5 401) return 0;
1d614d41e5 405) return 0;
4fbcca4eff 538) return 1;
b67f6b26e3 559) continue;
b67f6b26e3 570) from->objects[i].item->flags |= assign_flag; b67f6b26e3 571) continue;
b67f6b26e3 577) result = 0;
b67f6b26e3 578) goto cleanup;

config.c
c780b9cfe8 2303) return val;
c780b9cfe8 2306) if (is_bool)
c780b9cfe8 2307) return val ? 0 : 1;
c780b9cfe8 2309) return val;

date.c
c27cc94fad  904) tm->tm_mon = number-1; c27cc94fad  908) else if (number > 69 && number < 100) c27cc94fad  909) tm->tm_year = number; c27cc94fad  910) else if (number < 38) c27cc94fad  911) tm->tm_year = 100 + number; c27cc94fad  952) pending_number(tm, num);

delta-islands.c
c8d521faf7  53) memcpy(b, old, size);
c8d521faf7  73) return 1;
c8d521faf7 118) return 0;
c8d521faf7 130) return 0;
c8d521faf7 187) b->refcount--;
c8d521faf7 188) b = kh_value(island_marks, pos) = island_bitmap_new(b); c8d521faf7 202) continue;
c8d521faf7 212) obj = ((struct tag *)obj)->tagged;
c8d521faf7 213) if (obj) {
c8d521faf7 214) parse_object(the_repository, &obj->oid);
c8d521faf7 215) marks = create_or_get_island_marks(obj);
c8d521faf7 216) island_bitmap_set(marks, island_counter);
c8d521faf7 248) return;
c8d521faf7 268) progress_state = start_progress(_("Propagating island marks"), nr); c8d521faf7 286) die(_("bad tree object %s"), oid_to_hex(&ent->idx.oid)); c8d521faf7 293) continue;
c8d521faf7 297) continue;
c8d521faf7 321) return config_error_nonbool(k);
c8d521faf7 330) die(_("failed to load island regex for '%s': %s"), k, re.buf); c8d521faf7 386) warning(_("island regex from config has "
c8d521faf7 397) strbuf_addch(&island_name, '-');
c8d521faf7 433) continue;
c8d521faf7 436) list[dst] = list[src];

diff-lib.c
9001dc2a74 diff-lib.c 346)     (!oideq(oid, &old_entry->oid) || !oideq(&old_entry->oid, &new_entry->oid))) {

diff.c
b78ea5fc35 4130) add_external_diff_name(o->repo, &argv, other, two);

dir.c
8a2c174677  287) name = to_free = xmemdupz(name, namelen); c46c406ae1 2282) trace_performance_leave("read directory %.*s", len, path);

entry.c
b878579ae7 402) static void mark_colliding_entries(const struct checkout *state, b878579ae7 405) int i, trust_ino = check_stat;
b878579ae7 411) ce->ce_flags |= CE_MATCHED;
b878579ae7 413) for (i = 0; i < state->istate->cache_nr; i++) { b878579ae7 414) struct cache_entry *dup = state->istate->cache[i]; b878579ae7 416) if (dup == ce)
b878579ae7 417) break;
b878579ae7 419) if (dup->ce_flags & (CE_MATCHED | CE_VALID | CE_SKIP_WORKTREE)) b878579ae7 420) continue;
b878579ae7 422) if ((trust_ino && dup->ce_stat_data.sd_ino == st->st_ino) || b878579ae7 423)     (!trust_ino && !fspathcmp(ce->name, dup->name))) { b878579ae7 424) dup->ce_flags |= CE_MATCHED;
b878579ae7 425) break;
b878579ae7 428) }
b878579ae7 488) mark_colliding_entries(state, ce, &st);

fsck.c
fb8952077d  214) die_errno("Could not read '%s'", path);

git.c
a9a60b94cc 322) fprintf_ln(stderr, _("'%s' is aliased to '%s'"), c6d75bc17a 735) string_list_clear(&cmd_list, 0);

gpg-interface.c
4de9394dcb 155) break;

help.c
26c7d06783 help.c         500) static int get_alias(const char *var, const char *value, void *data) 26c7d06783 help.c         502) struct string_list *list = data;
26c7d06783 help.c         504) if (skip_prefix(var, "alias.", &var))
26c7d06783 help.c         505) string_list_append(list, var)->util = xstrdup(value); 26c7d06783 help.c         507) return 0;
26c7d06783 help.c         530) printf("\n%s\n", _("Command aliases"));
26c7d06783 help.c         531) ALLOC_ARRAY(aliases, alias_list.nr + 1);
26c7d06783 help.c         532) for (i = 0; i < alias_list.nr; i++) {
26c7d06783 help.c         533) aliases[i].name = alias_list.items[i].string; 26c7d06783 help.c         534) aliases[i].help = alias_list.items[i].util; 26c7d06783 help.c         535) aliases[i].category = 1;
26c7d06783 help.c         537) aliases[alias_list.nr].name = NULL;
26c7d06783 help.c         538) print_command_list(aliases, 1, longest);
26c7d06783 help.c         539) free(aliases);

http.c
21084e84a4  316) free(http_ssl_backend);
21084e84a4  317) http_ssl_backend = xstrdup_or_null(value);
21084e84a4  318) return 0;
93aef7c79b  322) http_schannel_check_revoke = git_config_bool(var, value); 93aef7c79b  323) return 0;
b67d40adbb  327) http_schannel_use_ssl_cainfo = git_config_bool(var, value); b67d40adbb  328) return 0;
93aef7c79b  833)     !http_schannel_check_revoke) {
93aef7c79b  835) curl_easy_setopt(result, CURLOPT_SSL_OPTIONS, CURLSSLOPT_NO_REVOKE); b67d40adbb  883)     !http_schannel_use_ssl_cainfo) {
b67d40adbb  884) curl_easy_setopt(result, CURLOPT_CAINFO, NULL);

ident.c
501afcb8b0 172) strbuf_addstr(&git_default_email, email); 501afcb8b0 173) free((char *)email);

list-objects-filter-options.c
bc5975d24f  55) if (errbuf) {
bc5975d24f  56) strbuf_addstr(
bc5975d24f  60) return 1;
cc0b05a4cc  86) if (errbuf)

list-objects-filter.c
list-objects.c
f447a499db 200) ctx->show_object(obj, base->buf, ctx->show_data);

ll-merge.c
d64324cb60 380) marker_size = DEFAULT_CONFLICT_MARKER_SIZE;

log-tree.c
4a7e27e957 477) if (oideq(&parent->item->object.oid, oid))

mailinfo.c
3aa4d81f88  992) len--;
3aa4d81f88  998) handle_filter(mi, prev);
3aa4d81f88  999) strbuf_reset(prev);
3aa4d81f88 1090) handle_filter(mi, &prev);

midx.c
4d80560c54   58) error_errno(_("failed to read %s"), midx_name); 4d80560c54   59) goto cleanup_fail;
4d80560c54   65) error(_("multi-pack-index file %s is too small"), midx_name); 4d80560c54   66) goto cleanup_fail;
0d5b3a5ef7  146) die(_("multi-pack-index missing required OID lookup chunk")); 662148c435  148) die(_("multi-pack-index missing required object offsets chunk")); 4d80560c54  173) munmap(midx_map, midx_size);
4d80560c54  175) close(fd);
1dcd9f2043  184) return;
3715a6335c  266) return 0;
fe1ed56f5e  413) warning(_("failed to open pack-index '%s'"), fe1ed56f5e  415) close_pack(packs->list[packs->nr]);
fe1ed56f5e  416) FREE_AND_NULL(packs->list[packs->nr]);
fe1ed56f5e  417) return;
a40498a126  490) return 1;
fe1ed56f5e  507) die(_("failed to locate object %d in packfile"), cur_object); fc59e74844  769) die_errno(_("unable to create leading directories of %s"), 525e18c04b  943) die(_("failed to clear multi-pack-index at %s"), midx); 56ee7ff156  969) return 0;
cc6af73c02 1010) midx_report(_("failed to load pack-index for packfile %s"), cc6af73c02 1011)     e.p->pack_name);
cc6af73c02 1012) break;

name-hash.c
2179045fd0 532) die(_("unable to create lazy_dir thread: %s"), strerror(err)); 2179045fd0 554) die(_("unable to create lazy_name thread: %s"), strerror(err)); 2179045fd0 560) die(_("unable to join lazy_name thread: %s"), strerror(err));

oidmap.c
cc00e5ce6b 11) return !oideq(&entry_->oid,

oidset.c
8b2f8cbcb1 29) kh_del_oid(&set->set, pos);
8b2f8cbcb1 30) return 1;

pack-bitmap.c
30cdc33fba 1130) return 0;

pack-objects.c
108f530385 172) REALLOC_ARRAY(pdata->tree_depth, pdata->nr_alloc); fe0ac2fb7f 175) REALLOC_ARRAY(pdata->layer, pdata->nr_alloc); 108f530385 192) pdata->tree_depth[pdata->nr_objects - 1] = 0; fe0ac2fb7f 195) pdata->layer[pdata->nr_objects - 1] = 0;

packfile.c
1127a98cce  117) return error("index file %s is too small", path); 1127a98cce  119) return error("empty data");
fe1ed56f5e  211) if (open_pack_index(p))
fe1ed56f5e  212) return 0;
fe1ed56f5e  213) level1_ofs = p->index_data;
17c35c8969  490) break;
17c35c8969  548) return 0;

preload-index.c
ae9af12287  63) struct progress_data *pd = p->progress;
ae9af12287  65) pthread_mutex_lock(&pd->mutex);
ae9af12287  66) pd->n += last_nr - nr;
ae9af12287  67) display_progress(pd->progress, pd->n);
ae9af12287  68) pthread_mutex_unlock(&pd->mutex);
ae9af12287  69) last_nr = nr;
ae9af12287  83) struct progress_data *pd = p->progress;
ae9af12287  85) pthread_mutex_lock(&pd->mutex);
ae9af12287  86) display_progress(pd->progress, pd->n + last_nr);
ae9af12287  87) pthread_mutex_unlock(&pd->mutex);
ae9af12287 118) pd.progress = start_delayed_progress(_("Refreshing index"), index->cache_nr); ae9af12287 119) pthread_mutex_init(&pd.mutex, NULL);
ae9af12287 132) p->progress = &pd;
2179045fd0 137) die(_("unable to create threaded lstat: %s"), strerror(err));

read-cache.c
ae9af12287 1490) progress = start_delayed_progress(_("Refresh index"),
ae9af12287 1491)   istate->cache_nr);
ae9af12287 1539) display_progress(progress, i);
ae9af12287 1572) display_progress(progress, istate->cache_nr);
ae9af12287 1573) stop_progress(&progress);
252d079cbd 1784) const unsigned char *cp = (const unsigned char *)name;
252d079cbd 1788) strip_len = decode_varint(&cp);
77ff1127a4 1789) if (previous_ce) {
77ff1127a4 1790) previous_len = previous_ce->ce_namelen;
77ff1127a4 1791) if (previous_len < strip_len)
252d079cbd 1792) die(_("malformed name field in the index, near path '%s'"),
77ff1127a4 1793) previous_ce->name);
77ff1127a4 1794) copy_len = previous_len - strip_len;
252d079cbd 1796) name = (const char *)cp;
252d079cbd 1802) len += copy_len;
252d079cbd 1823) if (copy_len)
252d079cbd 1824) memcpy(ce->name, previous_ce->name, copy_len);
252d079cbd 1825) memcpy(ce->name + copy_len, name, len + 1 - copy_len);
252d079cbd 1826) *ent_size = (name - ((char *)ondisk)) + len + 1 - copy_len;
abb4bb8384 1959) munmap((void *)p->mmap, p->mmap_size);
abb4bb8384 1960) die(_("index file corrupt"));
77ff1127a4 2001) mem_pool_init(&istate->ce_mem_pool,
77ff1127a4 2039) static void *load_cache_entries_thread(void *_data)
77ff1127a4 2041) struct load_cache_entries_thread_data *p = _data;
77ff1127a4 2045) for (i = p->ieot_start; i < p->ieot_start + p->ieot_blocks; i++) {
77ff1127a4 2046) p->consumed += load_cache_entry_block(p->istate, p->ce_mem_pool,
77ff1127a4 2047) p->offset, p->ieot->entries[i].nr, p->mmap, p->ieot->entries[i].offset, NULL);
77ff1127a4 2048) p->offset += p->ieot->entries[i].nr;
77ff1127a4 2050) return NULL;
77ff1127a4 2053) static unsigned long load_cache_entries_threaded(struct index_state *istate, const char *mmap, size_t mmap_size, 77ff1127a4 2058) unsigned long consumed = 0;
77ff1127a4 2061) if (istate->name_hash_initialized)
77ff1127a4 2064) mem_pool_init(&istate->ce_mem_pool, 0);
77ff1127a4 2067) if (nr_threads > ieot->nr)
77ff1127a4 2068) nr_threads = ieot->nr;
77ff1127a4 2069) data = xcalloc(nr_threads, sizeof(*data));
77ff1127a4 2071) offset = ieot_start = 0;
77ff1127a4 2072) ieot_blocks = DIV_ROUND_UP(ieot->nr, nr_threads);
77ff1127a4 2073) for (i = 0; i < nr_threads; i++) {
77ff1127a4 2074) struct load_cache_entries_thread_data *p = &data[i];
77ff1127a4 2077) if (ieot_start + ieot_blocks > ieot->nr)
77ff1127a4 2078) ieot_blocks = ieot->nr - ieot_start;
77ff1127a4 2080) p->istate = istate;
77ff1127a4 2081) p->offset = offset;
77ff1127a4 2082) p->mmap = mmap;
77ff1127a4 2083) p->ieot = ieot;
77ff1127a4 2084) p->ieot_start = ieot_start;
77ff1127a4 2085) p->ieot_blocks = ieot_blocks;
77ff1127a4 2088) nr = 0;
77ff1127a4 2089) for (j = p->ieot_start; j < p->ieot_start + p->ieot_blocks; j++)
77ff1127a4 2090) nr += p->ieot->entries[j].nr;
77ff1127a4 2091) if (istate->version == 4) {
77ff1127a4 2092) mem_pool_init(&p->ce_mem_pool,
77ff1127a4 2095) mem_pool_init(&p->ce_mem_pool,
77ff1127a4 2099) err = pthread_create(&p->pthread, NULL, load_cache_entries_thread, p);
77ff1127a4 2100) if (err)
77ff1127a4 2101) die(_("unable to create load_cache_entries thread: %s"), strerror(err));
77ff1127a4 2104) for (j = 0; j < ieot_blocks; j++)
77ff1127a4 2105) offset += ieot->entries[ieot_start + j].nr;
77ff1127a4 2106) ieot_start += ieot_blocks;
77ff1127a4 2109) for (i = 0; i < nr_threads; i++) {
77ff1127a4 2110) struct load_cache_entries_thread_data *p = &data[i];
77ff1127a4 2112) err = pthread_join(p->pthread, NULL);
77ff1127a4 2113) if (err)
77ff1127a4 2114) die(_("unable to join load_cache_entries thread: %s"), strerror(err));
77ff1127a4 2115) mem_pool_combine(istate->ce_mem_pool, p->ce_mem_pool);
77ff1127a4 2116) consumed += p->consumed;
77ff1127a4 2119) free(data);
77ff1127a4 2121) return consumed;
abb4bb8384 2193) extension_offset = read_eoie_extension(mmap, mmap_size);
abb4bb8384 2194) if (extension_offset) {
abb4bb8384 2197) p.src_offset = extension_offset;
abb4bb8384 2198) err = pthread_create(&p.pthread, NULL, load_index_extensions, &p);
abb4bb8384 2199) if (err)
abb4bb8384 2200) die(_("unable to create load_index_extensions thread: %s"), strerror(err));
abb4bb8384 2202) nr_threads--;
77ff1127a4 2211) ieot = read_ieot_extension(mmap, mmap_size, extension_offset);
77ff1127a4 2214) src_offset += load_cache_entries_threaded(istate, mmap, mmap_size, src_offset, nr_threads, ieot); 77ff1127a4 2215) free(ieot);
abb4bb8384 2225) int ret = pthread_join(p.pthread, NULL);
abb4bb8384 2226) if (ret)
abb4bb8384 2227) die(_("unable to join load_index_extensions thread: %s"), strerror(ret));
3255089ada 2769) ieot_blocks = nr_threads;
77ff1127a4 2770) if (ieot_blocks > istate->cache_nr)
77ff1127a4 2771) ieot_blocks = istate->cache_nr;
3255089ada 2779) ieot = xcalloc(1, sizeof(struct index_entry_offset_table)
3255089ada 2780) + (ieot_blocks * sizeof(struct index_entry_offset)));
77ff1127a4 2781) ieot_entries = DIV_ROUND_UP(entries, ieot_blocks);
3255089ada 2787) free(ieot);
3b1d9e045e 2788) return -1;
3255089ada 2814) ieot->entries[ieot->nr].nr = nr;
3255089ada 2815) ieot->entries[ieot->nr].offset = offset;
3255089ada 2816) ieot->nr++;
3255089ada 2822) if (previous_name)
3255089ada 2823) previous_name->buf[0] = 0;
3255089ada 2824) nr = 0;
3255089ada 2825) offset = lseek(newfd, 0, SEEK_CUR);
3255089ada 2826) if (offset < 0) {
3255089ada 2827) free(ieot);
3255089ada 2828) return -1;
3255089ada 2830) offset += write_buffer_len;
3255089ada 2840) ieot->entries[ieot->nr].nr = nr;
3255089ada 2841) ieot->entries[ieot->nr].offset = offset;
3255089ada 2842) ieot->nr++;
3255089ada 2854) free(ieot);
3b1d9e045e 2855) return -1;
3255089ada 2868) struct strbuf sb = STRBUF_INIT;
3255089ada 2870) write_ieot_extension(&sb, ieot);
3255089ada 2871) err = write_index_ext_header(&c, &eoie_c, newfd, CACHE_EXT_INDEXENTRYOFFSETTABLE, sb.len) < 0 3255089ada 2872) || ce_write(&c, newfd, sb.buf, sb.len) < 0;
3255089ada 2873) strbuf_release(&sb);
3255089ada 2874) free(ieot);
3255089ada 2875) if (err)
3255089ada 2876) return -1;
3b1d9e045e 3363) static size_t read_eoie_extension(const char *mmap, size_t mmap_size)
3b1d9e045e 3381) if (mmap_size < sizeof(struct cache_header) + EOIE_SIZE_WITH_HEADER + the_hash_algo->rawsz) 3b1d9e045e 3382) return 0;
3b1d9e045e 3385) index = eoie = mmap + mmap_size - EOIE_SIZE_WITH_HEADER - the_hash_algo->rawsz;
3b1d9e045e 3386) if (CACHE_EXT(index) != CACHE_EXT_ENDOFINDEXENTRIES)
3b1d9e045e 3387) return 0;
3b1d9e045e 3388) index += sizeof(uint32_t);
3b1d9e045e 3391) extsize = get_be32(index);
3b1d9e045e 3392) if (extsize != EOIE_SIZE)
3b1d9e045e 3393) return 0;
3b1d9e045e 3394) index += sizeof(uint32_t);
3b1d9e045e 3400) offset = get_be32(index);
3b1d9e045e 3401) if (mmap + offset < mmap + sizeof(struct cache_header))
3b1d9e045e 3402) return 0;
3b1d9e045e 3403) if (mmap + offset >= eoie)
3b1d9e045e 3404) return 0;
3b1d9e045e 3405) index += sizeof(uint32_t);
3b1d9e045e 3416) src_offset = offset;
3b1d9e045e 3417) the_hash_algo->init_fn(&c);
3b1d9e045e 3418) while (src_offset < mmap_size - the_hash_algo->rawsz - EOIE_SIZE_WITH_HEADER) {
3b1d9e045e 3426) memcpy(&extsize, mmap + src_offset + 4, 4);
3b1d9e045e 3427) extsize = ntohl(extsize);
3b1d9e045e 3430) if (src_offset + 8 + extsize < src_offset)
3b1d9e045e 3431) return 0;
3b1d9e045e 3433) the_hash_algo->update_fn(&c, mmap + src_offset, 8);
3b1d9e045e 3435) src_offset += 8;
3b1d9e045e 3436) src_offset += extsize;
3b1d9e045e 3438) the_hash_algo->final_fn(hash, &c);
3b1d9e045e 3439) if (!hasheq(hash, (const unsigned char *)index))
3b1d9e045e 3440) return 0;
3b1d9e045e 3443) if (src_offset != mmap_size - the_hash_algo->rawsz - EOIE_SIZE_WITH_HEADER)
3b1d9e045e 3444) return 0;
3b1d9e045e 3446) return offset;
3255089ada 3465) static struct index_entry_offset_table *read_ieot_extension(const char *mmap, size_t mmap_size, size_t offset) 3255089ada 3467)        const char *index = NULL;
3255089ada 3473)        if (!offset)
3255089ada 3474)        return NULL;
3255089ada 3475)        while (offset <= mmap_size - the_hash_algo->rawsz - 8) {
3255089ada 3476)        extsize = get_be32(mmap + offset + 4);
3255089ada 3477)        if (CACHE_EXT((mmap + offset)) == CACHE_EXT_INDEXENTRYOFFSETTABLE) {
3255089ada 3478)        index = mmap + offset + 4 + 4;
3255089ada 3479)        break;
3255089ada 3481)        offset += 8;
3255089ada 3482)        offset += extsize;
3255089ada 3484)        if (!index)
3255089ada 3485)        return NULL;
3255089ada 3488)        ext_version = get_be32(index);
3255089ada 3489)        if (ext_version != IEOT_VERSION) {
3255089ada 3490)        error("invalid IEOT version %d", ext_version);
3255089ada 3491)        return NULL;
3255089ada 3493)        index += sizeof(uint32_t);
3255089ada 3496)        nr = (extsize - sizeof(uint32_t)) / (sizeof(uint32_t) + sizeof(uint32_t)); 3255089ada 3497)        if (!nr) {
3255089ada 3498)        error("invalid number of IEOT entries %d", nr);
3255089ada 3499)        return NULL;
3255089ada 3501)        ieot = xmalloc(sizeof(struct index_entry_offset_table)
3255089ada 3502)        + (nr * sizeof(struct index_entry_offset)));
3255089ada 3503)        ieot->nr = nr;
3255089ada 3504)        for (i = 0; i < nr; i++) {
3255089ada 3505)        ieot->entries[i].offset = get_be32(index);
3255089ada 3506)        index += sizeof(uint32_t);
3255089ada 3507)        ieot->entries[i].nr = get_be32(index);
3255089ada 3508)        index += sizeof(uint32_t);
3255089ada 3511)        return ieot;
3255089ada 3514) static void write_ieot_extension(struct strbuf *sb, struct index_entry_offset_table *ieot) 3255089ada 3520)        put_be32(&buffer, IEOT_VERSION);
3255089ada 3521)        strbuf_add(sb, &buffer, sizeof(uint32_t));
3255089ada 3524)        for (i = 0; i < ieot->nr; i++) {
3255089ada 3527)        put_be32(&buffer, ieot->entries[i].offset);
3255089ada 3528)        strbuf_add(sb, &buffer, sizeof(uint32_t));
3255089ada 3531)        put_be32(&buffer, ieot->entries[i].nr);
3255089ada 3532)        strbuf_add(sb, &buffer, sizeof(uint32_t));
3255089ada 3534) }

rebase-interactive.c
64a43cbd5d 62) return error_errno(_("could not read '%s'."), todo_file); 64a43cbd5d 66) strbuf_release(&buf);
64a43cbd5d 67) return -1;
a9f5476fbc 75) return error_errno(_("could not read '%s'."), todo_file); a9f5476fbc 79) strbuf_release(&buf);
a9f5476fbc 80) return -1;
64a43cbd5d 86) return -1;

ref-filter.c
f0062d3b74 1039) v->s = xstrdup("");
f0062d3b74 1302) free((char *)to_free);
f0062d3b74 1303) return xstrdup("");
f0062d3b74 1340) free((char *)to_free);
f0062d3b74 1341) return xstrdup("");
f0062d3b74 1391) *s = xstrdup("=");
f0062d3b74 1393) *s = xstrdup("<");
f0062d3b74 1518) ref->symref = xstrdup("");
f0062d3b74 1587) v->s = xstrdup("");

refs.c
3a3b9d8cde  661) return 0;
4a6067cda5 1431) return 0;

refs/files-backend.c
refs/packed-backend.c
9001dc2a74 1163) } else if (!oideq(&update->old_oid, iter->oid)) {

refs/ref-cache.c
9001dc2a74 275) if (!oideq(&ref1->u.value.oid, &ref2->u.value.oid))

remote.c
85daa01f6b 1219) continue;
85daa01f6b 1225) continue;

rerere.c
2373b65059  217) die(_("corrupt MERGE_RR"));
2373b65059  226) die(_("corrupt MERGE_RR"));
2373b65059  229) die(_("corrupt MERGE_RR"));
2373b65059  264) die(_("unable to write rerere record")); 2373b65059  269) die(_("unable to write rerere record")); 4af32207bc  376) break;
4af32207bc  380) strbuf_addbuf(&two, &conflict);
c0f16f8e14  384) break;
c0f16f8e14  388) break;
c0f16f8e14  392) break;
2373b65059  480) return error_errno(_("could not open '%s'"), path); 2373b65059  485) error_errno(_("could not write '%s'"), output); 2373b65059  495) error(_("there were errors while writing '%s' (%s)"), 2373b65059  498) io.io.wrerror = error_errno(_("failed to flush '%s'"), path); 2373b65059  565) return error(_("index file corrupt")); 2373b65059  599) return error(_("index file corrupt")); 2373b65059  684) warning_errno(_("failed utime() on '%s'"), 2373b65059  690) return error_errno(_("could not open '%s'"), path); 2373b65059  692) error_errno(_("could not write '%s'"), path); 2373b65059  694) return error_errno(_("writing '%s' failed"), path); 2373b65059  720) die(_("unable to write new index file")); 2373b65059  803) die_errno(_("cannot unlink stray '%s'"), path); 2373b65059 1057) error(_("failed to update conflicted state in '%s'"), path); 2373b65059 1075) error(_("no remembered resolution for '%s'"), path); 2373b65059 1077) error_errno(_("cannot unlink '%s'"), filename); 2373b65059 1111) return error(_("index file corrupt")); 2373b65059 1199) die_errno(_("unable to open rr-cache directory"));

revision.c
2abf350385 1538) if (ce_path_match(istate, ce, &revs->prune_data, NULL)) { 2abf350385 1544) while ((i+1 < istate->cache_nr) &&
2abf350385 1545)        ce_same_name(ce, istate->cache[i+1])) b45424181e 2942) return;
b45424181e 2945) return;
b45424181e 2951) c->object.flags |= UNINTERESTING;
b45424181e 2954) return;
b45424181e 2957) mark_parents_uninteresting(c);
b45424181e 2980) return;
b45424181e 2983) return;
b45424181e 3048) continue;
f0d9cc4196 3097) if (!revs->ignore_missing_links)
f0d9cc4196 3098) die("Failed to traverse parents of commit %s", f0d9cc4196 3099)     oid_to_hex(&commit->object.oid)); b45424181e 3107) continue;
4a7e27e957 3473)     oideq(&p->item->object.oid, &commit->object.oid))

run-command.c
2179045fd0 1229) error(_("cannot create async thread: %s"), strerror(err));

send-pack.c
c0e40a2d66 207) close(fd[1]);

sequencer.c
bcd33ec25f  683) np = strchrnul(buf, '\n');
bcd33ec25f  684) return error(_("no key present in '%.*s'"), bcd33ec25f  695) return error(_("unable to dequote value of '%s'"), bcd33ec25f  737) goto finish;
bcd33ec25f  742) name_i = error(_("'GIT_AUTHOR_NAME' already given")); bcd33ec25f  747) email_i = error(_("'GIT_AUTHOR_EMAIL' already given")); bcd33ec25f  752) date_i = error(_("'GIT_AUTHOR_DATE' already given")); bcd33ec25f  756) err = error(_("unknown variable '%s'"), bcd33ec25f  761) error(_("missing 'GIT_AUTHOR_NAME'")); bcd33ec25f  763) error(_("missing 'GIT_AUTHOR_EMAIL'")); bcd33ec25f  765) error(_("missing 'GIT_AUTHOR_DATE'")); 65850686cf 2329) return;
65850686cf 2426) write_file(rebase_path_quiet(), "%s\n", quiet); 2c58483a59 3427) return error(_("could not checkout %s"), commit); 4df66c40b0 3441) return error(_("%s: not a valid OID"), orig_head); 71f82465b1 3461) fprintf(stderr, _("Stopped at HEAD\n")); b97e187364 4827) return -1;
b97e187364 4830) return -1;
b97e187364 4836) return error_errno(_("could not read '%s'."), todo_file); b97e187364 4839) todo_list_release(&todo_list);
b97e187364 4840) return error(_("unusable todo list: '%s'"), todo_file); b97e187364 4859) todo_list_release(&todo_list);
b97e187364 4860) return -1;
b97e187364 4864) return error(_("could not copy '%s' to '%s'."), todo_file, b97e187364 4868) return error(_("could not transform the todo list")); b97e187364 4897) return error(_("could not transform the todo list")); b97e187364 4900) return error(_("could not skip unnecessary pick commands")); b97e187364 4906) return -1;

setup.c
58b284a2e9  413) return config_error_nonbool(var);

sha1-file.c
67947c34ae sha1-file.c 2225) if (!hasheq(expected_sha1, real_sha1)) {

sha1-name.c
8aac67a174 sha1-name.c  162) return;

split-index.c
e3d837989e 335) ce->ce_flags |= CE_UPDATE_IN_BASE;

strbuf.c
f95736288a  127) --sb->len;

submodule-config.c
bcbc780d14 739) return CONFIG_INVALID_KEY;
45f5ef3d77 754) warning(_("Could not update .gitmodules entry %s"), key);

trace.c
c46c406ae1 189) now = getnanotime();
c46c406ae1 190) perf_start_times[perf_indent] = now;
c46c406ae1 191) if (perf_indent + 1 < ARRAY_SIZE(perf_start_times)) c46c406ae1 192) perf_indent++;
c46c406ae1 195) return now;
c46c406ae1 211) if (perf_indent >= strlen(space))
c46c406ae1 214) strbuf_addf(&buf, ":%.*s ", perf_indent, space); c46c406ae1 317) void trace_performance_leave_fl(const char *file, int line, c46c406ae1 323) if (perf_indent)
c46c406ae1 324) perf_indent--;
c46c406ae1 326) if (!format) /* Allow callers to leave without tracing anything */ c46c406ae1 327) return;
c46c406ae1 329) since = perf_start_times[perf_indent];
c46c406ae1 330) va_start(ap, format);
c46c406ae1 331) trace_performance_vprintf_fl(file, line, nanos - since, format, ap); c46c406ae1 332) va_end(ap);
c46c406ae1 477) trace_performance_leave("git command:%s", command_line.buf); c46c406ae1 485) if (!command_line.len)
c46c406ae1 490) trace_performance_enter();

transport.c
unpack-trees.c
b878579ae7  360) string_list_append(&list, ce->name);
b878579ae7  361) ce->ce_flags &= ~CE_MATCHED;
b878579ae7  368) warning(_("the following paths have collided (e.g. case-sensitive paths\n" b878579ae7  372) for (i = 0; i < list.nr; i++)
b878579ae7  373) fprintf(stderr, "  '%s'\n", list.items[i].string); f1e11c6510  777) free(tree_ce);
b4da37380b  778) return rc;
b4da37380b  785) printf("Unpacked %d entries from %s to %s using cache-tree\n", b4da37380b  787)        o->src_index->cache[pos]->name,
b4da37380b  788)        o->src_index->cache[pos + nr_entries - 1]->name);

upload-pack.c
1d1243fe63 1403) deepen(INFINITE_DEPTH, data->deepen_relative, &data->shallows,

worktree.c
3a3b9d8cde 495) return -1;
3a3b9d8cde 508) return -1;
3a3b9d8cde 517) return -1;
ab3e1f78ae 537) break;

wt-status.c
f3bd35fa0d  671) s->committable = 1;
73ba5d78b4 1958) if (s->state.rebase_in_progress || 73ba5d78b4 1959)     s->state.rebase_interactive_in_progress) 73ba5d78b4 1960) branch_name = s->state.onto; 73ba5d78b4 1961) else if (s->state.detached_from) 73ba5d78b4 1962) branch_name = s->state.detached_from;

xdiff-interface.c
xdiff/xutils.c
611e42a598 405) return -1;

Commits introducing uncovered code:
Ævar Arnfjörð Bjarmason      62c23938f: tests: add a special setup where rebase.useBuiltin is off Alban Gruin      0af129b2e: rebase--interactive2: rewrite the submodes of interactive rebase in C Alban Gruin      2c58483a5: rebase -i: rewrite setup_reflog_action() in C
Alban Gruin      34b47315d: rebase -i: move rebase--helper modes to rebase--interactive Alban Gruin      4df66c40b: rebase -i: rewrite checkout_onto() in C
Alban Gruin      53bbcfbde: rebase -i: implement the main part of interactive rebase as a builtin Alban Gruin      64a43cbd5: rebase -i: rewrite the edit-todo functionality in C
Alban Gruin      65850686c: rebase -i: rewrite write_basic_state() in C
Alban Gruin      a9f5476fb: sequencer: refactor append_todo_help() to write its message to a buffer Alban Gruin      b97e18736: rebase -i: rewrite complete_action() in C
Antonio Ospite      45f5ef3d7: submodule: factor out a config_set_in_gitmodules_file_gently function Antonio Ospite      76e9bdc43: submodule: support reading .gitmodules when it's not in the working tree Antonio Ospite      bcbc780d1: submodule: add a print_config_from_gitmodules() helper
Ben Peart      3255089ad: ieot: add Index Entry Offset Table (IEOT) extension
Ben Peart      3b1d9e045: eoie: add End of Index Entry (EOIE) extension
Ben Peart      77ff1127a: read-cache: load cache entries on worker threads
Ben Peart      abb4bb838: read-cache: load cache extensions on a worker thread
Ben Peart      c780b9cfe: config: add new index.threads config setting
Ben Peart      d1664e73a: add: speed up cmd_add() by utilizing read_cache_preload()
Ben Peart      fa655d841: checkout: optimize "git checkout -b <new_branch>"
Brendan Forster      93aef7c79: http: add support for disabling SSL revocation checks in cURL brian m. carlson      2f0c9e9a9: builtin/repack: replace hard-coded constants
brian m. carlson      eccb5a5f3: apply: rename new_sha1_prefix and old_sha1_prefix Christian Couder      108f53038: pack-objects: move tree_depth into 'struct packing_data' Christian Couder      fe0ac2fb7: pack-objects: move 'layer' into 'struct packing_data' Derrick Stolee      0d5b3a5ef: midx: write object ids in a chunk
Derrick Stolee      17c35c896: packfile: skip loading index if in multi-pack-index
Derrick Stolee      1d614d41e: commit-reach: move ref_newer from remote.c
Derrick Stolee      1dcd9f204: midx: close multi-pack-index on repack
Derrick Stolee      20fd6d579: commit-graph: not compatible with grafts
Derrick Stolee      3715a6335: midx: read objects from multi-pack-index
Derrick Stolee      454ea2e4d: treewide: use get_all_packs
Derrick Stolee      4d80560c5: multi-pack-index: load into memory
Derrick Stolee      4fbcca4ef: commit-reach: make can_all_from_reach... linear
Derrick Stolee      5227c3856: commit-reach: move walk methods from commit.c
Derrick Stolee      525e18c04: midx: clear midx on repack
Derrick Stolee      56ee7ff15: multi-pack-index: add 'verify' verb
Derrick Stolee      662148c43: midx: write object offsets
Derrick Stolee      66ec0390e: fsck: verify multi-pack-index
Derrick Stolee      6a22d5212: pack-objects: consider packs in multi-pack-index
Derrick Stolee      6cc017431: commit-reach: use can_all_from_reach
Derrick Stolee      6d68e6a46: multi-pack-index: provide more helpful usage info
Derrick Stolee      85daa01f6: remote: make add_missing_tags() linear
Derrick Stolee      8aac67a17: midx: use midx in abbreviation calculations
Derrick Stolee      a40498a12: midx: use existing midx when writing new one
Derrick Stolee      b45424181: revision.c: generation-based topo-order algorithm
Derrick Stolee      b67f6b26e: commit-reach: properly peel tags
Derrick Stolee      cc6af73c0: multi-pack-index: verify object offsets
Derrick Stolee      f0d9cc419: revision.c: begin refactoring --topo-order logic
Derrick Stolee      fc59e7484: midx: write header information to lockfile
Derrick Stolee      fe1ed56f5: midx: sort and deduplicate objects from packfiles
Duy Nguyen      b878579ae: clone: report duplicate entries on case-insensitive filesystems Eric Sunshine      2e6fd71a5: format-patch: extend --range-diff to accept revision range Eric Sunshine      40ce41604: format-patch: allow --range-diff to apply to a lone-patch Eric Sunshine      68a6b3a1b: worktree: teach 'move' to override lock when --force given twice Eric Sunshine      8631bf1cd: format-patch: add --creation-factor tweak for --range-diff Eric Sunshine      e19831c94: worktree: teach 'add' to respect --force for registered but missing path Eric Sunshine      e5353bef5: worktree: move delete_git_dir() earlier in file for upcoming new callers Eric Sunshine      ee6cbf712: format-patch: allow --interdiff to apply to a lone-patch
Eric Sunshine      f4143101c: worktree: teach 'remove' to override lock when --force given twice Jeff King      0074c9110: combine-diff: use an xdiff hunk callback
Jeff King      01a31f3bc: pull: handle --verify-signatures for unborn branch
Jeff King      0eb8d3767: cat-file: report an error on multiple --batch options
Jeff King      16d75fa48: repack: add delta-islands support
Jeff King      28b8a7308: pack-objects: add delta-islands support
Jeff King      2c8ee1f53: bundle: dup() output descriptor closer to point-of-use
Jeff King      2fa233a55: pack-objects: handle island check for "external" delta base
Jeff King      30cdc33fb: pack-bitmap: save "have" bitmap from walk
Jeff King      4a7e27e95: convert "oidcmp() == 0" to oideq()
Jeff King      517fe807d: assert NOARG/NONEG behavior of parse-options callbacks
Jeff King      611e42a59: xdiff: provide a separate emit callback for hunks
Jeff King      67947c34a: convert "hashcmp() != 0" to "!hasheq()"
Jeff King      735ca208c: apply: return -1 from option callback instead of calling exit(1) Jeff King      8a2c17467: pathspec: handle non-terminated strings with :(attr)
Jeff King      9001dc2a7: convert "oidcmp() != 0" to "!oideq()"
Jeff King      98f425b45: cat-file: handle streaming failures consistently
Jeff King      c27cc94fa: approxidate: handle pending number for "specials"
Jeff King      c8d521faf: Add delta-islands.{c,h}
Jeff King      cc00e5ce6: convert hashmap comparison functions to oideq()
Jeff King      fce566480: am: handle --no-patch-format option
Johannes Schindelin      21084e84a: http: add support for selecting SSL backends at runtime Johannes Schindelin      3249c1251: rebase: consolidate clean-up code before leaving reset_head() Johannes Schindelin      501afcb8b: mingw: use domain information for default email Johannes Schindelin      71f82465b: rebase -i: introduce the 'break' command
Johannes Schindelin      b67d40adb: http: when using Secure Channel, ignore sslCAInfo by default Johannes Schindelin      bac2a1e36: built-in rebase: reinstate checkout -q behavior where appropriate Johannes Schindelin      bc24382c2: builtin rebase: prepare for builtin rebase -i
Jonathan Nieder      302997027: gc: do not return error for prior errors in daemonized mode Jonathan Nieder      fec2ed218: gc: exit with status 128 on failure
Jonathan Tan      1d1243fe6: upload-pack: make want_obj not global
Josh Steadmon      1127a98cc: fuzz: add fuzz testing for packfile indices.
Matthew DeVore      7c0fe330d: rev-list: handle missing tree objects properly
Matthew DeVore      bc5975d24: list-objects-filter: implement filter tree:0
Matthew DeVore      cc0b05a4c: list-objects-filter-options: do not over-strbuf_init
Matthew DeVore      f447a499d: list-objects: store common func args in struct
Michał Górny      4de9394dc: gpg-interface.c: obtain primary key fingerprint as well
Nguyễn Thái Ngọc Duy      2179045fd: Clean up pthread_create() error handling
Nguyễn Thái Ngọc Duy      252d079cb: read-cache.c: optimize reading index format v4
Nguyễn Thái Ngọc Duy      26c7d0678: help -a: improve and make --verbose default
Nguyễn Thái Ngọc Duy      2abf35038: revision.c: remove implicit dependency on the_index Nguyễn Thái Ngọc Duy      3a3b9d8cd: refs: new ref types to make per-worktree refs visible to all worktrees Nguyễn Thái Ngọc Duy      58b284a2e: worktree: add per-worktree config files
Nguyễn Thái Ngọc Duy      a470beea3: blame.c: rename "repo" argument to "r"
Nguyễn Thái Ngọc Duy      ab3e1f78a: revision.c: better error reporting on ref from different worktrees Nguyễn Thái Ngọc Duy      ae9af1228: status: show progress bar if refreshing the index takes too long Nguyễn Thái Ngọc Duy      b29759d89: fsck: check HEAD and reflog from other worktrees Nguyễn Thái Ngọc Duy      b4da37380: unpack-trees: optimize walking same trees with cache-tree Nguyễn Thái Ngọc Duy      b78ea5fc3: diff.c: reduce implicit dependency on the_index
Nguyễn Thái Ngọc Duy      c0e40a2d6: send-pack.c: move async's #ifdef NO_PTHREADS back to run-command.c Nguyễn Thái Ngọc Duy      c46c406ae: trace.h: support nested performance tracing
Nguyễn Thái Ngọc Duy      c9ef0d95e: reflog expire: cover reflog from all worktrees
Nguyễn Thái Ngọc Duy      f1e11c651: unpack-trees: reduce malloc in cache-tree walk
Nguyễn Thái Ngọc Duy      fd6263fb7: grep: clean up num_threads handling
Olga Telezhnaya      f0062d3b7: ref-filter: free item->value and item->value->s
Phillip Wood      bcd33ec25: add read_author_script() to libgit
Pratik Karki      002ee2fe6: builtin rebase: support keep-empty option
Pratik Karki      0eabf4b95: builtin rebase: stop if git am is in progress
Pratik Karki      12026a412: builtin rebase: support --gpg-sign option
Pratik Karki      122420c29: builtin rebase: support --skip
Pratik Karki      1ed9c14ff: builtin rebase: support --force-rebase
Pratik Karki      3c3588c7d: builtin rebase: support --rebase-merges[=[no-]rebase-cousins] Pratik Karki      55071ea24: rebase: start implementing it as a builtin
Pratik Karki      5a6149453: builtin rebase: support --quit
Pratik Karki      5e5d96197: builtin rebase: support --abort
Pratik Karki      6defce2b0: builtin rebase: support --autostash option
Pratik Karki      73d51ed0a: builtin rebase: support --signoff
Pratik Karki      9a48a615b: builtin rebase: try to fast forward when possible
Pratik Karki      9dba809a6: builtin rebase: support --root
Pratik Karki      ac7f467fe: builtin/rebase: support running "git rebase <upstream>"
Pratik Karki      ba1905a5f: builtin rebase: add support for custom merge strategies
Pratik Karki      bff014dac: builtin rebase: support the verbose and diffstat options Pratik Karki      c54dacb50: builtin rebase: start a new rebase only if none is in progress Pratik Karki      cda614e48: builtin rebase: show progress when connected to a terminal Pratik Karki      e0333e5c6: builtin rebase: require a clean worktree
Pratik Karki      e65123a71: builtin rebase: support git rebase <upstream> <switch-to> Pratik Karki      ead98c111: builtin rebase: support --rerere-autoupdate
Pratik Karki      f28d40d3a: builtin rebase: support --onto
Pratik Karki      f95736288: builtin rebase: support --continue
Rasmus Villemoes      a9a60b94c: git.c: handle_alias: prepend alias info when first argument is -h Rasmus Villemoes      e6e76baaf: help: redirect to aliased commands for "git cmd --help" René Scharfe      3aa4d81f8: mailinfo: support format=flowed
René Scharfe      8b2f8cbcb: oidset: use khash
René Scharfe      fb8952077: fsck: use strbuf_getline() to read skiplist file
Shulhan      5025425df: builtin/remote: quote remote name on error to display empty name Stefan Beller      4a6067cda: refs.c: migrate internal ref iteration to pass thru repository argument Stefan Beller      74d4731da: submodule--helper: replace connect-gitdir-workingtree by ensure-core-worktree Stefan Beller      e0a862fda: submodule helper: convert relative URL to absolute URL if needed Stefan Beller      ee69b2a90: submodule--helper: introduce new update-module-mode helper Stephen P. Smith      73ba5d78b: roll wt_status_state into wt_status and populate in the collect phase Stephen P. Smith      f3bd35fa0: wt-status.c: set the committable flag in the collect phase SZEDER Gábor      e3d837989: split-index: don't compare cached data of entries already marked for split index Thomas Gummerer      2373b6505: rerere: mark strings for translation
Thomas Gummerer      4af32207b: rerere: teach rerere to handle nested conflicts
Thomas Gummerer      c0f16f8e1: rerere: factor out handle_conflict function
Tim Schumacher      c6d75bc17: alias: add support for aliases of an alias
Torsten Bögershausen      d64324cb6: Make git_check_attr() a void function

#27 lore raw

Re: Git Test Coverage Report (v2.20.0-rc0)

The test coverage reports started mid-way through this release cycle, so I thought it would be good to do a full review of the new uncovered code since the last release.

I eliminated most of the uncovered code due to the following cases:

1. Code was only moved or refactored.
2. Code was related to unusual error conditions (e.g. open_pack_index() fails)

The comments below are intended only to point out potential directions to improve test coverage. Some of it is for me to do!

Thanks,
-Stolee

Quoted 5 lines · Derrick Stolee

66ec0390e7 builtin/fsck.c 888) midx_argv[2] = "--object-dir"; 66ec0390e7 builtin/fsck.c 889) midx_argv[3] = alt->path; 66ec0390e7 builtin/fsck.c 890) if (run_command(&midx_verify)) 66ec0390e7 builtin/fsck.c 891) errors_found |= ERROR_COMMIT_GRAPH;

There are two things wrong here:

1. Not properly covering multi-pack-index fsck with alternates.
2. the ERROR_COMMIT_GRAPH flag when the multi-pack-index is being checked.

I'll submit a patch to fix this.

Quoted 3 lines

2fa233a554 builtin/pack-objects.c 1512) hashcpy(base_oid.hash, base_sha1); 2fa233a554 builtin/pack-objects.c 1513) if (!in_same_island(&delta->idx.oid, &base_oid)) 2fa233a554 builtin/pack-objects.c 1514) return 0;

These lines are inside a block for the following if statement:

+       /*
+        * Otherwise, reachability bitmaps may tell us if the receiver has it, +        * even if it was buried too deep in history to make it into the +        * packing list.
+        */
+       if (thin && bitmap_has_sha1_in_uninteresting(bitmap_git, base_sha1)) {

Peff: is this difficult to test?

Quoted 2 lines

28b8a73080 builtin/pack-objects.c 2793) depth++;
108f530385 builtin/pack-objects.c 2797) oe_set_tree_depth(&to_pack, ent, depth);

This 'depth' variable is incremented as part of a for loop in this patch:

 static void show_object(struct object *obj, const char *name, void *data)
@@ -2686,6 +2706,19 @@ static void show_object(struct object *obj, const char *name, void *data) add_preferred_base_object(name);
        add_object_entry(&obj->oid, obj->type, name, 0);
        obj->flags |= OBJECT_ADDED;
+
+       if (use_delta_islands) {
+               const char *p;
+               unsigned depth = 0;
+               struct object_entry *ent;
+
+               for (p = strchr(name, '/'); p; p = strchr(p + 1, '/'))
+                       depth++;
+
+               ent = packlist_find(&to_pack, obj->oid.hash, NULL);
+               if (ent && depth > ent->tree_depth)
+                       ent->tree_depth = depth;
+       }
 }

And that 'ent->tree_depth = depth;' line is replaced with the oe_set_tree_depth() call in the report.

Since depth is never incremented, we are not covering this block. Is it possible to test?

Quoted 3 lines

builtin/repack.c
16d75fa48d  48) use_delta_islands = git_config_bool(var, value); 16d75fa48d  49) return 0;

This is a simple config option check for "repack.useDeltaIslands". The logic it enables is tested, so this is an OK gap, in my opinion.

Quoted 3 lines

builtin/submodule--helper.c
ee69b2a90c 1476) out->type = sub->update_strategy.type; ee69b2a90c 1477) out->command = sub->update_strategy.command;

This block was introduced by this part of the patch:

+       } else if (sub->update_strategy.type != SM_UPDATE_UNSPECIFIED) { +               trace_printf("loaded thing");
+               out->type = sub->update_strategy.type;
+               out->command = sub->update_strategy.command;

Which seems to be an important case, as the other SM_UPDATE_* types seem like interesting cases.

Stefan: what actions would trigger this block? Is it easy to test?

Quoted 2 lines

delta-islands.c
c8d521faf7  53) memcpy(b, old, size);

This memcpy happens when the 'old' island_bitmap is passed to island_bitmap_new(), but...

Quoted 2 lines

c8d521faf7 187) b->refcount--;
c8d521faf7 188) b = kh_value(island_marks, pos) = island_bitmap_new(b);

This block has the only non-NULL caller to island_bitmap_new().

Quoted 5 lines

c8d521faf7 212) obj = ((struct tag *)obj)->tagged; c8d521faf7 213) if (obj) {
c8d521faf7 214) parse_object(the_repository, &obj->oid); c8d521faf7 215) marks = create_or_get_island_marks(obj); c8d521faf7 216) island_bitmap_set(marks, island_counter);

It appears that this block would happen if we placed a tag in the delta island.

Quoted 1 line

c8d521faf7 397) strbuf_addch(&island_name, '-');

This block is inside the following patch:

+       if (matches[ARRAY_SIZE(matches) - 1].rm_so != -1)
+               warning(_("island regex from config has "
+                         "too many capture groups (max=%d)"),
+                       (int)ARRAY_SIZE(matches) - 2);
+
+       for (m = 1; m < ARRAY_SIZE(matches); m++) {
+               regmatch_t *match = &matches[m];
+
+               if (match->rm_so == -1)
+                       continue;
+
+               if (island_name.len)
+                       strbuf_addch(&island_name, '-');
+
+               strbuf_add(&island_name, refname + match->rm_so, match->rm_eo - match->rm_so); +       }

This likely means that ARRAY_SIZE(matches) is never more than two.

Quoted 2 lines

c8d521faf7 433) continue;
c8d521faf7 436) list[dst] = list[src];

These blocks are inside the following nested loop in deduplicate_islands():

+       for (ref = 0; ref + 1 < island_count; ref++) {
+               for (src = ref + 1, dst = src; src < island_count; src++) { +                       if (list[ref]->hash == list[src]->hash) +                               continue;
+
+                       if (src != dst)
+                               list[dst] = list[src];
+
+                       dst++;
+               }
+               island_count = dst;
+       }

This means that our "deduplication" logic is never actually doing anything meaningful.

Quoted 2 lines

entry.c
b878579ae7 402) static void mark_colliding_entries(const struct checkout *state,

(there is interesting logic in this method, but it is only enabled on case-insensitive filesystems. This run was done on a case-sensitive file system. Related changes happen in unpack-trees.c.)

Quoted 7 lines

help.c
26c7d06783 help.c         500) static int get_alias(const char *var, const char *value, void *data) 26c7d06783 help.c         502) struct string_list *list = data;
26c7d06783 help.c         504) if (skip_prefix(var, "alias.", &var))
26c7d06783 help.c         505) string_list_append(list, var)->util = xstrdup(value); 26c7d06783 help.c         507) return 0;
26c7d06783 help.c         530) printf("\n%s\n", _("Command aliases"));

Quoted 8 lines

26c7d06783 help.c         531) ALLOC_ARRAY(aliases, alias_list.nr + 1); 26c7d06783 help.c         532) for (i = 0; i < alias_list.nr; i++) { 26c7d06783 help.c         533) aliases[i].name = alias_list.items[i].string; 26c7d06783 help.c         534) aliases[i].help = alias_list.items[i].util; 26c7d06783 help.c         535) aliases[i].category = 1; 26c7d06783 help.c         537) aliases[alias_list.nr].name = NULL; 26c7d06783 help.c         538) print_command_list(aliases, 1, longest); 26c7d06783 help.c         539) free(aliases);

This logic introduces alias help in 'git help -a'. This seems like a simple thing for adding a test to ensure that this works now and in the future.

Quoted 2 lines

http.c

The code in here seems to be logic for Windows-specific SSL backends, so is not covered by this report.

Quoted 6 lines

preload-index.c
ae9af12287  63) struct progress_data *pd = p->progress; ae9af12287  65) pthread_mutex_lock(&pd->mutex); ae9af12287  66) pd->n += last_nr - nr; ae9af12287  67) display_progress(pd->progress, pd->n); ae9af12287  68) pthread_mutex_unlock(&pd->mutex);

Quoted 8 lines

ae9af12287  69) last_nr = nr;
ae9af12287  83) struct progress_data *pd = p->progress;
ae9af12287  85) pthread_mutex_lock(&pd->mutex);
ae9af12287  86) display_progress(pd->progress, pd->n + last_nr);
ae9af12287  87) pthread_mutex_unlock(&pd->mutex);
ae9af12287 118) pd.progress = start_delayed_progress(_("Refreshing index"), index->cache_nr); ae9af12287 119) pthread_mutex_init(&pd.mutex, NULL);
ae9af12287 132) p->progress = &pd;

There's a lot of stuff going on with showing progress on index writes. While the commit message states the progress doesn't show up for 3 seconds, perhaps that can be tweaked to be in the millisecond range for a test?

Quoted 1 line

read-cache.c

(There's a lot of progress stuff here, too.)

There are a lot of lines introduced by the IEOT extension in these commits:

Quoted 11 lines

Ben Peart      3255089ad: ieot: add Index Entry Offset Table (IEOT) extension Ben Peart      3b1d9e045: eoie: add End of Index Entry (EOIE) extension Ben Peart      77ff1127a: read-cache: load cache entries on worker threads Ben Peart      abb4bb838: read-cache: load cache extensions on a worker thread Ben Peart      c780b9cfe: config: add new index.threads config setting Ben Peart      d1664e73a: add: speed up cmd_add() by utilizing read_cache_preload() Ben Peart      fa655d841: checkout: optimize "git checkout -b <new_branch>"

revision.c
b45424181e 2951) c->object.flags |= UNINTERESTING; b45424181e 2957) mark_parents_uninteresting(c);

These blocks are currently unreachable because we do not use the new topo-order logic when there are UNINTERESTING commits. (This will be replaced after we have generation numbers v2.) I could force using this logic in a git log --topo-order A..B query when GIT_TEST_COMMIT_GRAPH is enabled.

#28 lore raw

Re: Git Test Coverage Report (v2.20.0-rc0)

Quoted 17 lines · Derrick Stolee
Quoted 5 lines

2fa233a554 builtin/pack-objects.c 1512) hashcpy(base_oid.hash, base_sha1);
2fa233a554 builtin/pack-objects.c 1513) if (!in_same_island(&delta->idx.oid, &base_oid)) 2fa233a554 builtin/pack-objects.c 1514) return 0;

These lines are inside a block for the following if statement:

+       /*
+        * Otherwise, reachability bitmaps may tell us if the receiver has it,
+        * even if it was buried too deep in history to make it into the +        * packing list.
+        */
+       if (thin && bitmap_has_sha1_in_uninteresting(bitmap_git, base_sha1)) {

Quoted 1 line

Peff: is this difficult to test?

A bit.

The two features (thin-packs and delta-islands) would not generally be used together (one is for serving fetches, the other is for optimizing on-disk packs to serve fetches). Something like:

echo HEAD^ | git pack-objects --revs --thin --delta-islands

would probably exercise it (assuming there's a delta in HEAD^ against something in HEAD), but you'd need to construct a very specific scenario if you wanted to do any kind of checks no the output.

Quoted 30 lines
Quoted 3 lines

28b8a73080 builtin/pack-objects.c 2793) depth++;
108f530385 builtin/pack-objects.c 2797) oe_set_tree_depth(&to_pack, ent, depth);

This 'depth' variable is incremented as part of a for loop in this patch:

 static void show_object(struct object *obj, const char *name, void *data)

(quoted diff) +13 -0
@@ -2686,6 +2706,19 @@ static void show_object(struct object *obj, const
 char *name, void *data)
         add_preferred_base_object(name);
         add_object_entry(&obj->oid, obj->type, name, 0);
         obj->flags |= OBJECT_ADDED;
+
+       if (use_delta_islands) {
+               const char *p;
+               unsigned depth = 0;
+               struct object_entry *ent;
+
+               for (p = strchr(name, '/'); p; p = strchr(p + 1, '/'))
+                       depth++;
+
+               ent = packlist_find(&to_pack, obj->oid.hash, NULL);
+               if (ent && depth > ent->tree_depth)
+                       ent->tree_depth = depth;
+       }
  }
 

And that 'ent->tree_depth = depth;' line is replaced with the oe_set_tree_depth() call in the report.

Quoted 2 lines

Since depth is never incremented, we are not covering this block. Is it possible to test?

Looks like t5320 only has single-level trees. We probably just need to add a deeper tree. A more interesting case is when an object really is found at multiple depths, but constructing a case that cared about that would be quite complicated.

That said, there is much bigger problem with this code, which is that 108f530385 (pack-objects: move tree_depth into 'struct packing_data', 2018-08-16) is totally broken. It works on the trivial repository in the test, but try this (especially under valgrind or ASan) on a real repository:

git repack -adi

which will crash immediately. It doesn't correctly maintain the invariant that if tree_depth is not NULL, it is the same size as the main object array.

I'll see if I can come up with a fix.

-Peff

#29 lore raw

Re: Git Test Coverage Report (v2.20.0-rc0)

Quoted 31 lines · Jeff King
Quoted 14 lines

+       if (use_delta_islands) {
+               const char *p;
+               unsigned depth = 0;
+               struct object_entry *ent;

(quoted diff) +8 -0
+
+               for (p = strchr(name, '/'); p; p = strchr(p + 1, '/'))
+                       depth++;
+
+               ent = packlist_find(&to_pack, obj->oid.hash, NULL);
+               if (ent && depth > ent->tree_depth)
+                       ent->tree_depth = depth;
+       }

 }

Quoted 5 lines

And that 'ent->tree_depth = depth;' line is replaced with the oe_set_tree_depth() call in the report.

Since depth is never incremented, we are not covering this block. Is it possible to test?

Looks like t5320 only has single-level trees. We probably just need to add a deeper tree. A more interesting case is when an object really is found at multiple depths, but constructing a case that cared about that would be quite complicated.

That said, there is much bigger problem with this code, which is that 108f530385 (pack-objects: move tree_depth into 'struct packing_data', 2018-08-16) is totally broken. It works on the trivial repository in the test, but try this (especially under valgrind or ASan) on a real repository:

Quoted 7 lines

git repack -adi

which will crash immediately. It doesn't correctly maintain the invariant that if tree_depth is not NULL, it is the same size as the main object array.

I'll see if I can come up with a fix.

Just a quick update to prevent anyone else looking at it: I have a fix for this (and another related issue with that commit).

There's an edge case in that depth computation that I think is unhandled, as well. I _think_ it doesn't trigger very often, but I'm running some experiments to verify it. That's S-L-O-W, so I probably won't have results until tomorrow.

-Peff

#30 lore raw

Re: Git Test Coverage Report (v2.20.0-rc0)

In reply to Derrick Stolee
Quoted 20 lines · Derrick Stolee

The test coverage reports started mid-way through this release cycle, so I thought it would be good to do a full review of the new uncovered code since the last release.

I eliminated most of the uncovered code due to the following cases:

1. Code was only moved or refactored.
2. Code was related to unusual error conditions (e.g. open_pack_index() fails)

The comments below are intended only to point out potential directions to improve test coverage. Some of it is for me to do!

Thanks,
-Stolee

On 11/18/2018 9:54 PM, Derrick Stolee wrote:

There are a lot of lines introduced by the IEOT extension in these commits:

Quoted 1 line

Ben Peart      3255089ad: ieot: add Index Entry Offset Table (IEOT)

extension

Quoted 2 lines

Ben Peart      3b1d9e045: eoie: add End of Index Entry (EOIE) extension Ben Peart      77ff1127a: read-cache: load cache entries on worker

Quoted 9 lines

threads

Quoted 1 line

Ben Peart      abb4bb838: read-cache: load cache extensions on a

worker thread

Quoted 2 lines

Ben Peart      c780b9cfe: config: add new index.threads config setting Ben Peart      d1664e73a: add: speed up cmd_add() by utilizing

read_cache_preload()

Quoted 1 line

Ben Peart      fa655d841: checkout: optimize "git checkout -b

<new_branch>"

These should be hit if you run the test suite with GIT_TEST_INDEX_THREADS=2. Without that, the indexes for the various tests are too small to trigger multi-threaded index reads/writes.

From t/README:

GIT_TEST_INDEX_THREADS=<n> enables exercising the multi-threaded loading of the index for the whole test suite by bypassing the default number of cache entries and thread minimums. Setting this to 1 will make the index loading single threaded.

#31 lore raw

Re: Git Test Coverage Report (v2.20.0-rc0)

Quoted 15 lines · Ben Peart
Quoted 11 lines · Derrick Stolee

There are a lot of lines introduced by the IEOT extension in these commits:

 > Ben Peart      3255089ad: ieot: add Index Entry Offset Table (IEOT) extension > Ben Peart      3b1d9e045: eoie: add End of Index Entry (EOIE) extension > Ben Peart      77ff1127a: read-cache: load cache entries on worker threads > Ben Peart      abb4bb838: read-cache: load cache extensions on a worker thread > Ben Peart      c780b9cfe: config: add new index.threads config setting > Ben Peart      d1664e73a: add: speed up cmd_add() by utilizing read_cache_preload() > Ben Peart      fa655d841: checkout: optimize "git checkout -b <new_branch>"

Quoted 8 lines

These should be hit if you run the test suite with GIT_TEST_INDEX_THREADS=2.  Without that, the indexes for the various tests are too small to trigger multi-threaded index reads/writes.

From t/README:

GIT_TEST_INDEX_THREADS=<n> enables exercising the multi-threaded loading of the index for the whole test suite by bypassing the default number of cache entries and thread minimums. Setting this to 1 will make the index loading single threaded.

I updated my build to add GIT_TEST_INDEX_THREADS=2 and I still see lots of uncovered stuff, including that load_cache_entries_threaded() is never run.

I added the following diff to my repo and ran the test suite manually with GIT_TEST_INDEX_THREADS=2 and it didn't fail:

read-cache.c +0 -0
index 4ca81286c0..36502586a2 100644
@@ -2057,6 +2057,9 @@ static unsigned long load_cache_entries_threaded(struct index_state *istate, con

        struct load_cache_entries_thread_data *data; unsigned long consumed = 0;

+       fprintf(stderr, "load_cache_entries_threaded\n"); +       exit(1);
+
        /* a little sanity checking */
        if (istate->name_hash_initialized) BUG("the name hash isn't thread safe");

Am I missing something? Is there another variable I should add?

When I look for where the GIT_TEST_INDEX_THREADS variable is checked, I see that the calls to git_config_get_index_threads() are followed by a check for NO_PTHREADS (setting the number of threads to 1 again). Is it possible that my compiler environment is not allowing me to even compile with threads?

Thanks,
-Stolee

#32 lore raw

Re: Git Test Coverage Report (v2.20.0-rc0)

In reply to Derrick Stolee
Quoted 30 lines · Derrick Stolee
Quoted 3 lines

28b8a73080 builtin/pack-objects.c 2793) depth++;
108f530385 builtin/pack-objects.c 2797) oe_set_tree_depth(&to_pack, ent, depth);

This 'depth' variable is incremented as part of a for loop in this patch:

 static void show_object(struct object *obj, const char *name, void *data)

(quoted diff) +13 -0
@@ -2686,6 +2706,19 @@ static void show_object(struct object *obj, const
 char *name, void *data)
         add_preferred_base_object(name);
         add_object_entry(&obj->oid, obj->type, name, 0);
         obj->flags |= OBJECT_ADDED;
+
+       if (use_delta_islands) {
+               const char *p;
+               unsigned depth = 0;
+               struct object_entry *ent;
+
+               for (p = strchr(name, '/'); p; p = strchr(p + 1, '/'))
+                       depth++;
+
+               ent = packlist_find(&to_pack, obj->oid.hash, NULL);
+               if (ent && depth > ent->tree_depth)
+                       ent->tree_depth = depth;
+       }
  }
 

And that 'ent->tree_depth = depth;' line is replaced with the oe_set_tree_depth() call in the report.

Quoted 2 lines

Since depth is never incremented, we are not covering this block. Is it possible to test?

This should be covered by the fix in:

https://public-inbox.org/git/[email protected]/

because now entries at the top-level are depth "1". The "depth++" line is still never executed in our test suite. I'm not sure how much that matters.

Quoted 10 lines
Quoted 2 lines

delta-islands.c
c8d521faf7  53) memcpy(b, old, size);

This memcpy happens when the 'old' island_bitmap is passed to island_bitmap_new(), but...

Quoted 2 lines

c8d521faf7 187) b->refcount--;
c8d521faf7 188) b = kh_value(island_marks, pos) = island_bitmap_new(b);

This block has the only non-NULL caller to island_bitmap_new().

This is another case where it triggers a lot for a reasonably-sized repo, but it's hard to construct a small case. This code implements a copy-on-write of the bitmap, which means the same objects have to be accessible from two different paths through the reachability graph, each with different island marks. And then a test would I guess make sure that the correct subsets of objects never become deltas, which gets complicated.

And I think that's a pattern with the delta-island code. What we really care about most is that if we throw a real fork-network repository at it, it produces faster clones with fewer un-reusable deltas. So I think a much more interesting approach here would be perf tests. But:

- we'd want to count those as coverage, and that likely makes your coverage tests prohibitively expensive

- it requires a specialized repo to demonstrate, which most people aren't going to have handy

Quoted 8 lines
Quoted 5 lines

c8d521faf7 212) obj = ((struct tag *)obj)->tagged; c8d521faf7 213) if (obj) {
c8d521faf7 214) parse_object(the_repository, &obj->oid); c8d521faf7 215) marks = create_or_get_island_marks(obj); c8d521faf7 216) island_bitmap_set(marks, island_counter);

It appears that this block would happen if we placed a tag in the delta island.

Yep. Again, exercised by real repositories.

I'm not sure how far we want to go in the blind pursuit of coverage. Certainly we could add a tag to the repo in t5320, and this code would get executed. But verifying that it's doing the right thing is much harder (and is more easily done with a perf test).

Quoted 4 lines
Quoted 1 line

c8d521faf7 397) strbuf_addch(&island_name, '-');

This block is inside the following patch:
[...]

Yeah, this triggers if your regex has more than one capture group (and likewise, we almost certainly don't run the "you have too many groups" warning).

Quoted 18 lines
Quoted 2 lines

c8d521faf7 433) continue;
c8d521faf7 436) list[dst] = list[src];

These blocks are inside the following nested loop in deduplicate_islands():

+       for (ref = 0; ref + 1 < island_count; ref++) {
+               for (src = ref + 1, dst = src; src < island_count; src++) { +                       if (list[ref]->hash == list[src]->hash) +                               continue;

(quoted diff) +8 -0
+
+                       if (src != dst)
+                               list[dst] = list[src];
+
+                       dst++;
+               }
+               island_count = dst;
+       }
Quoted 2 lines

This means that our "deduplication" logic is never actually doing anything meaningful.

Sorry, I don't even remember what this code is trying to do. The island code is 5+ years old, and just recently ported to upstream Git by Christian. And that's perhaps part of my laziness in the above tests; it would be a significant effort to re-figure out all these corner cases. It's a big part of why I hadn't been sending the patches upstream myself.

-Peff

#33 lore raw

Re: Git Test Coverage Report (v2.20.0-rc0)

Quoted 7 lines · Jeff King

On Mon, Nov 19, 2018 at 10:40:53AM -0500, Derrick Stolee wrote:

Since depth is never incremented, we are not covering this block. Is it possible to test?
This should be covered by the fix in:

Quoted 5 lines
   https://public-inbox.org/git/[email protected]/

because now entries at the top-level are depth "1". The "depth++" line is still never executed in our test suite. I'm not sure how much that matters.

Thanks! I'll go take a look at your patch.

Quoted 21 lines
Quoted 8 lines
Quoted 2 lines

delta-islands.c
c8d521faf7  53) memcpy(b, old, size);

This memcpy happens when the 'old' island_bitmap is passed to island_bitmap_new(), but...

Quoted 2 lines

c8d521faf7 187) b->refcount--;
c8d521faf7 188) b = kh_value(island_marks, pos) = island_bitmap_new(b);

This block has the only non-NULL caller to island_bitmap_new().

This is another case where it triggers a lot for a reasonably-sized repo, but it's hard to construct a small case. This code implements a copy-on-write of the bitmap, which means the same objects have to be accessible from two different paths through the reachability graph, each with different island marks. And then a test would I guess make sure that the correct subsets of objects never become deltas, which gets complicated.

And I think that's a pattern with the delta-island code. What we really care about most is that if we throw a real fork-network repository at it, it produces faster clones with fewer un-reusable deltas. So I think a much more interesting approach here would be perf tests. But:

Quoted 5 lines

- we'd want to count those as coverage, and that likely makes your coverage tests prohibitively expensive

- it requires a specialized repo to demonstrate, which most people aren't going to have handy

Do you have regularly-running tests that check this in your infrastructure? As long as someone would notice if this code starts failing, that would be enough.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

c8d521faf7 212) obj = ((struct tag *)obj)->tagged; c8d521faf7 213) if (obj) {
c8d521faf7 214) parse_object(the_repository, &obj->oid); c8d521faf7 215) marks = create_or_get_island_marks(obj); c8d521faf7 216) island_bitmap_set(marks, island_counter);

Quoted 8 lines
Quoted 2 lines

It appears that this block would happen if we placed a tag in the delta island.

Yep. Again, exercised by real repositories.

I'm not sure how far we want to go in the blind pursuit of coverage. Certainly we could add a tag to the repo in t5320, and this code would get executed. But verifying that it's doing the right thing is much harder (and is more easily done with a perf test).

Blind coverage goals are definitely not worth the effort. My goal here was to re-check all of the new code (since last release) that is not covered, because it's easier to hide a bug there.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

c8d521faf7 397) strbuf_addch(&island_name, '-');

This block is inside the following patch:
[...]

Yeah, this triggers if your regex has more than one capture group (and likewise, we almost certainly don't run the "you have too many groups" warning).

Did you know that regexes are notoriously under-tested [1]? When looking at this code, I didn't even know regexes were involved (but I didn't look enough at the context).

[1] https://dl.acm.org/citation.cfm?id=3236072

Quoted 18 lines
Quoted 18 lines
Quoted 3 lines

c8d521faf7 433) continue;
c8d521faf7 436) list[dst] = list[src];

These blocks are inside the following nested loop in deduplicate_islands():

+       for (ref = 0; ref + 1 < island_count; ref++) {
+               for (src = ref + 1, dst = src; src < island_count; src++) { +                       if (list[ref]->hash == list[src]->hash) +                               continue;

(quoted diff) +8 -0
+
+                       if (src != dst)
+                               list[dst] = list[src];
+
+                       dst++;
+               }
+               island_count = dst;
+       }
Quoted 8 lines
Quoted 2 lines

This means that our "deduplication" logic is never actually doing anything meaningful.

Sorry, I don't even remember what this code is trying to do. The island code is 5+ years old, and just recently ported to upstream Git by Christian. And that's perhaps part of my laziness in the above tests; it would be a significant effort to re-figure out all these corner cases. It's a big part of why I hadn't been sending the patches upstream myself.

Sure. Hopefully pointing out these blocks gives us more motivation to manually inspect them and avoid silly bugs.

Thanks,
-Stolee

#34 lore raw

Re: Git Test Coverage Report (v2.20.0-rc0)

Quoted 8 lines · Derrick Stolee
Quoted 8 lines

And I think that's a pattern with the delta-island code. What we really care about most is that if we throw a real fork-network repository at it, it produces faster clones with fewer un-reusable deltas. So I think a much more interesting approach here would be perf tests. But:

- we'd want to count those as coverage, and that likely makes your coverage tests prohibitively expensive

Quoted 6 lines
Quoted 2 lines

- it requires a specialized repo to demonstrate, which most people aren't going to have handy

Do you have regularly-running tests that check this in your infrastructure? As long as someone would notice if this code starts failing, that would be enough.

We do integration tests, and this feature gets exercised quite a bit in production at GitHub. But none of that caught the breakage I fixed this morning for the simple fact that we don't keep up with upstream master in real-time. We're running the delta-island code I wrote years ago, and the bug was introduced by the upstreaming. Probably in a month or three I'll merge v2.20 into our fork, and we would have noticed then.

I've had dreams of following upstream more closely, but there are two blockers:

- we have enough custom bits that the merges are time-consuming (and introduce the possibility of funny interactions or regressions). I'd like to reduce our overall diff from upstream, but it's slow going and time is finite. (And ironically, it's made harder in some ways by the lag, because many of our topics are backports of things I send upstream).

- deploying the ".0" release from master can be scary. For instance, I was really happy to have a fix for 9ac3f0e5b3 (pack-objects: fix performance issues on packing large deltas, 2018-07-22) before putting the bug into production.

Quoted 9 lines
Quoted 3 lines

Yeah, this triggers if your regex has more than one capture group (and likewise, we almost certainly don't run the "you have too many groups" warning).

Did you know that regexes are notoriously under-tested [1]? When looking at this code, I didn't even know regexes were involved (but I didn't look enough at the context).

[1] https://dl.acm.org/citation.cfm?id=3236072

Heh. Well, fortunately we are compiling regexes from the user's config, so it's not Git's fault if the regexes are bad. ;)

(Of course on our production servers I'm on the hook for both sides!)

-Peff

#35 lore raw

Re: Git Test Coverage Report (v2.20.0-rc0)

In reply to Derrick Stolee
Quoted 2 lines · Derrick Stolee

Here is a test coverage report for the uncovered lines introduced in v2.20.0-rc0 compared to v2.19.1.

Thanks a lot for this.

Quoted 6 lines

[...]
builtin/rebase.c
62c23938fa 55) return env;
[...]
Ævar Arnfjörð Bjarmason 62c23938f: tests: add a special setup where rebase.useBuiltin is off

This one would be covered with
GIT_TEST_REBASE_USE_BUILTIN=false. Obviously trivial, but I wonder if
the rest of the coverage would look different when passed through the various GIT_TEST_* options.

That would make it take at least 12x as long if you did them one at a time. Probably just 2-3x as long in practice since most can be combined in some way, and somewhat computationally infeasible if you're going to try all possible combinations.

Quoted 6 lines

[...]
fsck.c
fb8952077d 214) die_errno("Could not read '%s'", path);

René Scharfe fb8952077: fsck: use strbuf_getline() to read skiplist file

The fault of a patch I submitted. Couldn't find a sane & portable way to emulate cases where ferror() would trigger.

#36 lore raw

Re: [PATCH] tests: add a special setup where rebase.useBuiltin is off (Re: Git Test Coverage Report (v2.20.0-rc0))

Quoted 9 lines · Ævar Arnfjörð Bjarmason
Quoted 6 lines · Derrick Stolee

[...]
builtin/rebase.c
62c23938fa 55) return env;
[...]
Ævar Arnfjörð Bjarmason 62c23938f: tests: add a special setup where rebase.useBuiltin is off

Quoted 3 lines

This one would be covered with
GIT_TEST_REBASE_USE_BUILTIN=false. Obviously trivial, but I wonder if
the rest of the coverage would look different when passed through the various GIT_TEST_* options.

I wonder if we can do better for this kind of thing.

When I do routine development, I am not running tests with any non-default flags. So why should tests run with non-default flags count toward coverage? Is there a way to make the default test settings dip their feet into some non-default configurations, without running the full battery of tests and slowing tests down accordingly? E.g. is there some kind of smoke test that rebase with useBuiltin=false works at all that could run, even if I am not running the full battery of rebase tests?

That's a bit of a non sequitor for this example, which is actual code to handle GIT_TEST_REBASE_USE_BUILTIN, though. For it, I wonder why we need rebase.c to understand the envvar --- couldn't test-lib.sh take care of setting rebase.useBuiltin to false when it's set?

Thanks,
Jonathan

#37 lore raw

Re: [PATCH] tests: add a special setup where rebase.useBuiltin is off (Re: Git Test Coverage Report (v2.20.0-rc0))

Quoted 16 lines · Jonathan Nieder
Quoted 9 lines · Ævar Arnfjörð Bjarmason
Quoted 6 lines · Derrick Stolee

[...]
builtin/rebase.c
62c23938fa 55) return env;
[...]
Ævar Arnfjörð Bjarmason 62c23938f: tests: add a special setup where rebase.useBuiltin is off

Quoted 3 lines

This one would be covered with
GIT_TEST_REBASE_USE_BUILTIN=false. Obviously trivial, but I wonder if
the rest of the coverage would look different when passed through the various GIT_TEST_* options.

I wonder if we can do better for this kind of thing.

Quoted 8 lines

When I do routine development, I am not running tests with any non-default flags. So why should tests run with non-default flags count toward coverage? Is there a way to make the default test settings dip their feet into some non-default configurations, without running the full battery of tests and slowing tests down accordingly? E.g. is there some kind of smoke test that rebase with useBuiltin=false works at all that could run, even if I am not running the full battery of rebase tests?

Yeah, definitely. Just pointing out that it would smoke out coverage we don't have at all v.s. cases where we just don't have coverage with the default tests without any special modes.

Derrick: I think it would be useful to produce some delta report showing covered lines without any GIT_TEST_* variables v.s. when they're set. As Jonathan points out those should ideally be tested with the normal test suite, leaving GIT_TEST_* just for stress testing to find new bugs.

Quoted 4 lines

That's a bit of a non sequitor for this example, which is actual code to handle GIT_TEST_REBASE_USE_BUILTIN, though. For it, I wonder why we need rebase.c to understand the envvar --- couldn't test-lib.sh take care of setting rebase.useBuiltin to false when it's set?

I guess the test-lib.sh could pass down things like "GIT_CONFIG_PARAMETERS='x.y=z'". Maybe that's a better way to do it.

#38 lore raw

Re: Git Test Coverage Report (v2.20.0-rc0)

In reply to Ævar Arnfjörð Bjarmason
Quoted 12 lines · Ævar Arnfjörð Bjarmason
Quoted 2 lines · Derrick Stolee

Here is a test coverage report for the uncovered lines introduced in v2.20.0-rc0 compared to v2.19.1.

Thanks a lot for this.

Quoted 6 lines

[...]
builtin/rebase.c
62c23938fa 55) return env;
[...]
Ævar Arnfjörð Bjarmason 62c23938f: tests: add a special setup where rebase.useBuiltin is off

Quoted 3 lines

This one would be covered with
GIT_TEST_REBASE_USE_BUILTIN=false. Obviously trivial, but I wonder if
the rest of the coverage would look different when passed through the various GIT_TEST_* options.

The coverage report has been using the following:

export GIT_TEST_MULTI_PACK_INDEX=1
export GIT_TEST_COMMIT_GRAPH=1
export GIT_TEST_INDEX_VERION=4
export GIT_TEST_SPLIT_INDEX=yes
export GIT_TEST_OE_SIZE=10
export GIT_TEST_OE_DELTA_SIZE=5

I need to add GIT_TEST_INDEX_THREADS=2 and GIT_TEST_REBASE_USE_BUILTIN=false

Thanks!
-Stolee

#39 lore raw

Re: Git Test Coverage Report (v2.20.0-rc0)

Quoted 19 lines · Derrick Stolee
Quoted 12 lines · Ævar Arnfjörð Bjarmason
Quoted 2 lines · Derrick Stolee

Here is a test coverage report for the uncovered lines introduced in v2.20.0-rc0 compared to v2.19.1.

Thanks a lot for this.

Quoted 6 lines

[...]
builtin/rebase.c
62c23938fa 55) return env;
[...]
Ævar Arnfjörð Bjarmason 62c23938f: tests: add a special setup where rebase.useBuiltin is off

Quoted 3 lines

This one would be covered with
GIT_TEST_REBASE_USE_BUILTIN=false. Obviously trivial, but I wonder if
the rest of the coverage would look different when passed through the various GIT_TEST_* options.

The coverage report has been using the following:

Quoted 8 lines

export GIT_TEST_MULTI_PACK_INDEX=1
export GIT_TEST_COMMIT_GRAPH=1
export GIT_TEST_INDEX_VERION=4
export GIT_TEST_SPLIT_INDEX=yes
export GIT_TEST_OE_SIZE=10
export GIT_TEST_OE_DELTA_SIZE=5

I need to add GIT_TEST_INDEX_THREADS=2 and GIT_TEST_REBASE_USE_BUILTIN=false

...although note you'll need to also test without GIT_TEST_REBASE_USE_BUILTIN=false, otherwise a lot of the new C code won't have coverage.

#40 lore raw

Re: Git Test Coverage Report (v2.20.0-rc0)

Quoted 11 lines · Ævar Arnfjörð Bjarmason
Quoted 9 lines · Derrick Stolee

The coverage report has been using the following:

export GIT_TEST_MULTI_PACK_INDEX=1
export GIT_TEST_COMMIT_GRAPH=1
export GIT_TEST_INDEX_VERION=4
export GIT_TEST_SPLIT_INDEX=yes
export GIT_TEST_OE_SIZE=10
export GIT_TEST_OE_DELTA_SIZE=5

Quoted 4 lines
Quoted 1 line

I need to add GIT_TEST_INDEX_THREADS=2 and GIT_TEST_REBASE_USE_BUILTIN=false

...although note you'll need to also test without GIT_TEST_REBASE_USE_BUILTIN=false, otherwise a lot of the new C code won't have coverage.

Sorry for lack of clarity: I first run 'make coverage-test' with no GIT_TEST_* variables, then run the test suite again with the optional variables.

Thanks,
-Stolee

#41 lore raw

Re: Git Test Coverage Report (v2.20.0-rc0)

In reply to Ævar Arnfjörð Bjarmason
Quoted 8 lines · Ævar Arnfjörð Bjarmason
Quoted 6 lines · Derrick Stolee

[...]
builtin/rebase.c
62c23938fa 55) return env;
[...]
Ævar Arnfjörð Bjarmason 62c23938f: tests: add a special setup where rebase.useBuiltin is off

Quoted 4 lines

This one would be covered with
GIT_TEST_REBASE_USE_BUILTIN=false. Obviously trivial, but I wonder if
the rest of the coverage would look different when passed through the various GIT_TEST_* options.

Thanks for pointing out this GIT_TEST_* variable to me. I had been running builds with some of them enabled, but didn't know about this one.

Unfortunately, t3406-rebase-message.sh fails with GIT_TEST_REBASE_USE_BUILTIN=false and it bisects to 4520c2337: Merge branch 'ab/rebase-in-c-escape-hatch'.

The issue is that the commit 04519d72 "rebase: validate -C<n> and --whitespace=<mode> parameters early" introduced the following test that cares about error messages:

+test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' ' +       test_must_fail git rebase -Cnot-a-number HEAD 2>err && +       test_i18ngrep "numerical value" err &&
+       test_must_fail git rebase --whitespace=bad HEAD 2>err && +       test_i18ngrep "Invalid whitespace option" err
+'

The merge commit then was the first place where this test could run with that variable.

What's the correct fix here? Force the builtin rebase in this test? Unify the error message in the non-builtin case?

Thanks,
-Stolee

#42 lore raw

Re: Git Test Coverage Report (v2.20.0-rc0)

Quoted 33 lines · Derrick Stolee
Quoted 8 lines · Ævar Arnfjörð Bjarmason
Quoted 6 lines · Derrick Stolee

[...]
builtin/rebase.c
62c23938fa 55) return env;
[...]
Ævar Arnfjörð Bjarmason 62c23938f: tests: add a special setup where rebase.useBuiltin is off

Quoted 4 lines

This one would be covered with
GIT_TEST_REBASE_USE_BUILTIN=false. Obviously trivial, but I wonder if
the rest of the coverage would look different when passed through the various GIT_TEST_* options.

Thanks for pointing out this GIT_TEST_* variable to me. I had been running builds with some of them enabled, but didn't know about this one.

Unfortunately, t3406-rebase-message.sh fails with GIT_TEST_REBASE_USE_BUILTIN=false and it bisects to 4520c2337: Merge branch 'ab/rebase-in-c-escape-hatch'.

The issue is that the commit 04519d72 "rebase: validate -C<n> and --whitespace=<mode> parameters early" introduced the following test that cares about error messages:

+test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' '
+ test_must_fail git rebase -Cnot-a-number HEAD 2>err &&
+ test_i18ngrep "numerical value" err &&
+ test_must_fail git rebase --whitespace=bad HEAD 2>err &&
+ test_i18ngrep "Invalid whitespace option" err
+'

Quoted 2 lines

The merge commit then was the first place where this test could run with that variable.

Yup. Sorry should have mentioned that, it's broken in master. Reported it earlier today:
https://public-inbox.org/git/[email protected]/

Quoted 2 lines

What's the correct fix here? Force the builtin rebase in this test? Unify the error message in the non-builtin case?

Probably to just force the builtin, unless Johannes wants to also fix the bug for the shellscript version. I don't know if for 2.20 we're trying to maintain 100% compatibility.

#43 lore raw

Re: Git Test Coverage Report (v2.20.0-rc0)

In reply to Derrick Stolee

Hi Stolee,

Quoted 33 lines · Derrick Stolee
Quoted 8 lines · Ævar Arnfjörð Bjarmason
Quoted 6 lines · Derrick Stolee

[...]
builtin/rebase.c
62c23938fa 55) return env;
[...]
Ævar Arnfjörð Bjarmason 62c23938f: tests: add a special setup where rebase.useBuiltin is off

Quoted 5 lines

This one would be covered with GIT_TEST_REBASE_USE_BUILTIN=false. Obviously trivial, but I wonder if the rest of the coverage would look different when passed through the various GIT_TEST_* options.

Thanks for pointing out this GIT_TEST_* variable to me. I had been running builds with some of them enabled, but didn't know about this one.

Unfortunately, t3406-rebase-message.sh fails with GIT_TEST_REBASE_USE_BUILTIN=false and it bisects to 4520c2337: Merge branch 'ab/rebase-in-c-escape-hatch'.

The issue is that the commit 04519d72 "rebase: validate -C<n> and --whitespace=<mode> parameters early" introduced the following test that cares about error messages:

+test_expect_success 'error out early upon -C<n> or --whitespace=<bad>' ' +       test_must_fail git rebase -Cnot-a-number HEAD 2>err && +       test_i18ngrep "numerical value" err &&
+       test_must_fail git rebase --whitespace=bad HEAD 2>err && +       test_i18ngrep "Invalid whitespace option" err
+'

Quoted 5 lines

The merge commit then was the first place where this test could run with that variable.

What's the correct fix here? Force the builtin rebase in this test? Unify the error message in the non-builtin case?

I am obviously biased, but my take is that the correct fix is in
https://public-inbox.org/git/[email protected]

Ciao,
Dscho

#44 lore raw

[ANNOUNCE] Git v2.20.0-rc1

A release candidate Git v2.20.0-rc1 is now available for testing at the usual places. It is comprised of 915 non-merge commits since v2.19.0, contributed by 73 people, 24 of which are new faces.

The tarballs are found at:

https://www.kernel.org/pub/software/scm/git/testing/

The following public repositories all have a copy of the 'v2.20.0-rc1' tag and the 'master' branch that the tag points at:

url = https://kernel.googlesource.com/pub/scm/git/git
url = git://repo.or.cz/alt-git.git
url = https://github.com/gitster/git

New contributors whose contributions weren't in v2.19.0 are as follows. Welcome to the Git development community!

Aaron Lindsay, Alexander Pyhalov, Anton Serbulov, Brendan Forster, Carlo Marcelo Arenas Belón, Daniels Umanovskis, David Zych, Đoàn Trần Công Danh, Frederick Eaton, James Knight, Jann Horn, Joshua Watt, Loo Rong Jie, Lucas De Marchi, Matthew DeVore, Mihir Mehta, Nickolai Belakovski, Roger Strain, Sam McKelvie, Saulius Gurklys, Shulhan, Steven Fernandez, Strain, Roger L, and Tim Schumacher.

Returning contributors who helped this release are as follows. Thanks for your continued support.

Ævar Arnfjörð Bjarmason, Alban Gruin, Andreas Gruenbacher, Andreas Heiduk, Antonio Ospite, Ben Peart, Brandon Williams, brian m. carlson, Christian Couder, Christian Hesse, Denton Liu, Derrick Stolee, Elijah Newren, Eric Sunshine, Jeff Hostetler, Jeff King, Johannes Schindelin, Johannes Sixt, Jonathan Nieder, Jonathan Tan, Josh Steadmon, Junio C Hamano, Karsten Blees, Luke Diamand, Martin Ågren, Max Kirillov, Michael Witten, Michał Górny, Nguyễn Thái Ngọc Duy, Noam Postavsky, Olga Telezhnaya, Phillip Wood, Pratik Karki, Rafael Ascensão, Ralf Thielow, Ramsay Jones, Rasmus Villemoes, René Scharfe, Sebastian Staudt, Stefan Beller, Stephen P. Smith, Steve Hoelzer, SZEDER Gábor, Tao Qingyun, Taylor Blau, Thomas Gummerer, Todd Zullinger, Torsten Bögershausen, and Uwe Kleine-König.

----------------------------------------------------------------

Git 2.20 Release Notes (draft)
==============================

Backward Compatibility Notes
----------------------------

* "git branch -l <foo>" used to be a way to ask a reflog to be created while creating a new branch, but that is no longer the case. It is a short-hand for "git branch --list <foo>" now.

* "git push" into refs/tags/* hierarchy is rejected without getting forced, but "git fetch" (misguidedly) used the "fast forwarding" rule used for the refs/heads/* hierarchy; this has been corrected, which means some fetches of tags that did not fail with older version of Git will fail without "--force" with this version.

* "git help -a" now gives verbose output (same as "git help -av"). Those who want the old output may say "git help --no-verbose -a"..

* "git cpn --help", when "cpn" is an alias to, say, "cherry-pick -n", reported only the alias expansion of "cpn" in earlier versions of Git. It now runs "git cherry-pick --help" to show the manual page of the command, while sending the alias expansion to the standard error stream.

* "git send-email" learned to grab address-looking string on any trailer whose name ends with "-by". This is a backward-incompatible change. Adding "--suppress-cc=misc-by" on the command line, or setting sendemail.suppresscc configuration variable to "misc-by", can be used to disable this behaviour.

Updates since v2.19
-------------------

UI, Workflows & Features

* Running "git clone" against a project that contain two files with pathnames that differ only in cases on a case insensitive filesystem would result in one of the files lost because the underlying filesystem is incapable of holding both at the same time. An attempt is made to detect such a case and warn.

* "git checkout -b newbranch [HEAD]" should not have to do as much as checking out a commit different from HEAD. An attempt is made to optimize this special case.

* "git rev-list --stdin </dev/null" used to be an error; it now shows no output without an error. "git rev-list --stdin --default HEAD" still falls back to the given default when nothing is given on the standard input.

* Lift code from GitHub to restrict delta computation so that an object that exists in one fork is not made into a delta against another object that does not appear in the same forked repository.

* "git format-patch" learned new "--interdiff" and "--range-diff" options to explain the difference between this version and the previous attempt in the cover letter (or after the three-dashes as a comment).

* "git mailinfo" used in "git am" learned to make a best-effort recovery of a patch corrupted by MUA that sends text/plain with format=flawed option.
(merge 3aa4d81f88 rs/mailinfo-format-flowed later to maint).

* The rules used by "git push" and "git fetch" to determine if a ref can or cannot be updated were inconsistent; specifically, fetching to update existing tags were allowed even though tags are supposed to be unmoving anchoring points. "git fetch" was taught to forbid updates to existing tags without the "--force" option.

* "git multi-pack-index" learned to detect corruption in the .midx file it uses, and this feature has been integrated into "git fsck".

* Generation of (experimental) commit-graph files have so far been fairly silent, even though it takes noticeable amount of time in a meaningfully large repository. The users will now see progress output.

* The minimum version of Windows supported by Windows port of Git is now set to Vista.

* The completion script (in contrib/) learned to complete a handful of options "git stash list" command takes.

* The completion script (in contrib/) learned that "git fetch --multiple" only takes remote names as arguments and no refspecs.

* "git status" learns to show progress bar when refreshing the index takes a long time.
(merge ae9af12287 nd/status-refresh-progress later to maint).

* "git help -a" and "git help -av" give different pieces of information, and generally the "verbose" version is more friendly to the new users. "git help -a" by default now uses the more verbose output (with "--no-verbose", you can go back to the original). Also "git help -av" now lists aliases and external commands, which it did not used to.

* Unlike "grep", "git grep" by default recurses to the whole tree. The command learned "git grep --recursive" option, so that "git grep --no-recursive" can serve as a synonym to setting the max-depth to 0.

* When pushing into a repository that borrows its objects from an alternate object store, "git receive-pack" that responds to the push request on the other side lists the tips of refs in the alternate to reduce the amount of objects transferred. This sometimes is detrimental when the number of refs in the alternate is absurdly large, in which case the bandwidth saved in potentially fewer objects transferred is wasted in excessively large ref advertisement. The alternate refs that are advertised are now configurable with a pair of configuration variables.

* "git cmd --help" when "cmd" is aliased used to only say "cmd is aliased to ...". Now it shows that to the standard error stream and runs "git $cmd --help" where $cmd is the first word of the alias expansion.

* The documentation of "git gc" has been updated to mention that it is no longer limited to "pruning away crufts" but also updates ancillary files like commit-graph as a part of repository optimization.

* "git p4 unshelve" improvements.

* The logic to select the default user name and e-mail on Windows has been improved.
(merge 501afcb8b0 js/mingw-default-ident later to maint).

* The "rev-list --filter" feature learned to exclude all trees via "tree:0" filter.

* "git send-email" learned to grab address-looking string on any trailer whose name ends with "-by"; --suppress-cc=misc-by on the command line, or setting sendemail.suppresscc configuration variable to "misc-by", can be used to disable this behaviour.

* Developer builds now uses -Wunused-function compilation option.

* One of our CI tests to run with "unusual/experimental/random" settings now also uses commit-graph and midx.

* "git mergetool" learned to take the "--[no-]gui" option, just like "git difftool" does.

* "git rebase -i" learned a new insn, 'break', that the user can insert in the to-do list. Upon hitting it, the command returns control back to the user.

* New "--pretty=format:" placeholders %GF and %GP that show the GPG key fingerprints have been invented.

* On platforms with recent cURL library, http.sslBackend configuration variable can be used to choose a different SSL backend at runtime. The Windows port uses this mechanism to switch between OpenSSL and Secure Channel while talking over the HTTPS protocol.

* "git send-email" learned to disable SMTP authentication via the "--smtp-auth=none" option, even when the smtp username is given (which turns the authentication on by default).

* A fourth class of configuration files (in addition to the traditional "system wide", "per user in the $HOME directory" and "per repository in the $GIT_DIR/config") has been introduced so that different worktrees that share the same repository (hence the same $GIT_DIR/config file) can use different customization.

* A pattern with '**' that does not have a slash on either side used to be an invalid one, but the code now treats such double-asterisks the same way as two normal asterisks that happen to be adjacent to each other.
(merge e5bbe09e88 nd/wildmatch-double-asterisk later to maint).

* The "--no-patch" option, which can be used to get a high-level overview without the actual line-by-line patch difference shown, of the "range-diff" command was earlier broken, which has been corrected.

* The recently merged "rebase in C" has an escape hatch to use the scripted version when necessary, but it hasn't been documented, which has been corrected.

Performance, Internal Implementation, Development Support etc.

* When there are too many packfiles in a repository (which is not recommended), looking up an object in these would require consulting many pack .idx files; a new mechanism to have a single file that consolidates all of these .idx files is introduced.

* "git submodule update" is getting rewritten piece-by-piece into C.

* The code for computing history reachability has been shuffled, obtained a bunch of new tests to cover them, and then being improved.

* The unpack_trees() API used in checking out a branch and merging walks one or more trees along with the index. When the cache-tree in the index tells us that we are walking a tree whose flattened contents is known (i.e. matches a span in the index), as linearly scanning a span in the index is much more efficient than having to open tree objects recursively and listing their entries, the walk can be optimized, which has been done.

* When creating a thin pack, which allows objects to be made into a delta against another object that is not in the resulting pack but is known to be present on the receiving end, the code learned to take advantage of the reachability bitmap; this allows the server to send a delta against a base beyond the "boundary" commit.

* spatch transformation to replace boolean uses of !hashcmp() to newly introduced oideq() is added, and applied, to regain performance lost due to support of multiple hash algorithms.

* Fix a bug in which the same path could be registered under multiple worktree entries if the path was missing (for instance, was removed manually). Also, as a convenience, expand the number of cases in which --force is applicable.

* Split Documentation/config.txt for easier maintenance. (merge 6014363f0b nd/config-split later to maint).

* Test helper binaries clean-up.
(merge c9a1f4161f nd/test-tool later to maint).

* Various tests have been updated to make it easier to swap the hash function used for object identification. (merge ae0c89d41b bc/hash-independent-tests later to maint).

* Update fsck.skipList implementation and documentation. (merge 371a655074 ab/fsck-skiplist later to maint).

* An alias that expands to another alias has so far been forbidden, but now it is allowed to create such an alias.

* Various test scripts have been updated for style and also correct handling of exit status of various commands.

* "gc --auto" ended up calling exit(-1) upon error, which has been corrected to use exit(1). Also the error reporting behaviour when daemonized has been updated to exit with zero status when stopping due to a previously discovered error (which implies there is no point running gc to improve the situation); we used to exit with failure in such a case.

* Various codepaths in the core-ish part learned to work on an arbitrary in-core index structure, not necessarily the default instance "the_index".
(merge b3c7eef9b0 nd/the-index later to maint).

* Code clean-up in the internal machinery used by "git status" and "git commit --dry-run".
(merge 73ba5d78b4 ss/wt-status-committable later to maint).

* Some environment variables that control the runtime options of Git used during tests are getting renamed for consistency. (merge 4231d1ba99 bp/rename-test-env-var later to maint).

* A pair of new extensions to the index file have been introduced. They allow the index file to be read in parallel for performance.

* The oidset API was built on top of the oidmap API which in turn is on the hashmap API. Replace the implementation to build on top of the khash API and gain performance.

* Over some transports, fetching objects with an exact commit object name can be done without first seeing the ref advertisements. The code has been optimized to exploit this.

* In a partial clone that will lazily be hydrated from the originating repository, we generally want to avoid "does this object exist (locally)?" on objects that we deliberately omitted when we created the clone. The cache-tree codepath (which is used to write a tree object out of the index) however insisted that the object exists, even for paths that are outside of the partial checkout area. The code has been updated to avoid such a check.

* To help developers, an EditorConfig file that attempts to follow the project convention has been added.
(merge b548d698a0 bc/editorconfig later to maint).

* The result of coverage test can be combined with "git blame" to check the test coverage of code introduced recently with a new 'coverage-diff' tool (in contrib/).
(merge 783faedd65 ds/coverage-diff later to maint).

* An experiment to fuzz test a few areas, hopefully we can gain more coverage to various areas.

* More codepaths are moving away from hardcoded hash sizes.

* The way the Windows port figures out the current directory has been improved.

* The way DLLs are loaded on the Windows port has been improved.

* Some tests have been reorganized and renamed; "ls t/" now gives a better overview of what is tested for these scripts than before.

* "git rebase" and "git rebase -i" have been reimplemented in C.

* Windows port learned to use nano-second resolution file timestamps.

* The overly large Documentation/config.txt file have been split into million little pieces. This potentially allows each individual piece included into the manual page of the command it affects more easily.

* Replace three string-list instances used as look-up tables in "git fetch" with hashmaps.

* Unify code to read the author-script used in "git am" and the commands that use the sequencer machinery, e.g. "git rebase -i".

* In preparation to the day when we can deprecate and remove the "rebase -p", make sure we can skip and later remove tests for it.

* The history traversal used to implement the tag-following has been optimized by introducing a new helper.

* The helper function to refresh the cached stat information in the in-core index has learned to perform the lstat() part of the operation in parallel on multi-core platforms.

* The code to traverse objects for reachability, used to decide what objects are unreferenced and expendable, have been taught to also consider per-worktree refs of other worktrees as starting points to prevent data loss.

* "git add" needs to internally run "diff-files" equivalent, and the codepath learned the same optimization as "diff-files" has to run lstat(2) in parallel to find which paths have been updated in the working tree.

* The procedure to install dependencies before testing at Travis CI is getting revamped for both simplicity and flexibility, taking advantage of the recent move to the vm-based environment.

* The support for format-patch (and send-email) by the command-line completion script (in contrib/) has been simplified a bit.

* The revision walker machinery learned to take advantage of the commit generation numbers stored in the commit-graph file.

* The codebase has been cleaned up to reduce "#ifndef NO_PTHREADS".

* The way -lcurl library gets linked has been simplified by taking advantage of the fact that we can just ask curl-config command how.

* Various functions have been audited for "-Wunused-parameter" warnings and bugs in them got fixed.

* A sanity check for start-up sequence has been added in the config API codepath.

* The build procedure to link for fuzzing test has been made customizable with a new Makefile variable.

* The way "git rebase" parses and forwards the command line options meant for underlying "git am" has been revamped, which fixed for options with parameters that were not passed correctly.

* Our testing framework uses a special i18n "poisoned localization" feature to find messages that ought to stay constant but are incorrectly marked to be translated. This feature has been made into a runtime option (it used to be a compile-time option).

* "git push" used to check ambiguities between object-names and refnames while processing the list of refs' old and new values, which was unnecessary (as it knew that it is feeding raw object names). This has been optimized out.

* The xcurl_off_t() helper function is used to cast size_t to curl_off_t, but some compilers gave warnings against the code to ensure the casting is done without wraparound, when size_t is narrower than curl_off_t. This warning has been squelched.

* Code preparation to replace ulong vars with size_t vars where appropriate continues.

* The "test installed Git" mode of our test suite has been updated to work better.

* A coding convention around the Coccinelle semantic patches to have two classes to ease code migration process has been proposed and its support has been added to the Makefile.

Fixes since v2.19
-----------------

* "git interpret-trailers" and its underlying machinery had a buggy code that attempted to ignore patch text after commit log message, which triggered in various codepaths that will always get the log message alone and never get such an input.
(merge 66e83d9b41 jk/trailer-fixes later to maint).

* Malformed or crafted data in packstream can make our code attempt to read or write past the allocated buffer and abort, instead of reporting an error, which has been fixed.

* "git rebase -i" did not clear the state files correctly when a run of "squash/fixup" is aborted and then the user manually amended the commit instead, which has been corrected.
(merge 10d2f35436 js/rebase-i-autosquash-fix later to maint).

* When fsmonitor is in use, after operation on submodules updates .gitmodules, we lost track of the fact that we did so and relied on stale fsmonitor data.
(merge 43f1180814 bp/mv-submodules-with-fsmonitor later to maint).

* Fix for a long-standing bug that leaves the index file corrupt when it shrinks during a partial commit.
(merge 6c003d6ffb jk/reopen-tempfile-truncate later to maint).

* Further fix for O_APPEND emulation on Windows (merge eeaf7ddac7 js/mingw-o-append later to maint).

* A corner case bugfix in "git rerere" code.
(merge ad2bf0d9b4 en/rerere-multi-stage-1-fix later to maint).

* "git add ':(attr:foo)'" is not supported and is supposed to be rejected while the command line arguments are parsed, but we fail to reject such a command line upfront.
(merge 84d938b732 nd/attr-pathspec-fix later to maint).

* Recent update broke the reachability algorithm when refs (e.g. tags) that point at objects that are not commit were involved, which has been fixed.

* "git rebase" etc. in Git 2.19 fails to abort when given an empty commit log message as result of editing, which has been corrected. (merge a3ec9eaf38 en/sequencer-empty-edit-result-aborts later to maint).

* The code to backfill objects in lazily cloned repository did not work correctly, which has been corrected.
(merge e68302011c jt/lazy-object-fetch-fix later to maint).

* Update error messages given by "git remote" and make them consistent. (merge 5025425dff ms/remote-error-message-update later to maint).

* "git update-ref" learned to make both "--no-deref" and "--stdin" work at the same time.
(merge d345e9fbe7 en/update-ref-no-deref-stdin later to maint).

* Recently added "range-diff" had a corner-case bug to cause it segfault, which has been corrected.
(merge e467a90c7a tg/range-diff-corner-case-fix later to maint).

* The recently introduced commit-graph auxiliary data is incompatible with mechanisms such as replace & grafts that "breaks" immutable nature of the object reference relationship. Disable optimizations based on its use (and updating existing commit-graph) when these incompatible features are in use in the repository. (merge 829a321569 ds/commit-graph-with-grafts later to maint).

* The mailmap file update.
(merge 255eb03edf jn/mailmap-update later to maint).

* The code in "git status" sometimes hit an assertion failure. This
was caused by a structure that was reused without cleaning the data used for the first run, which has been corrected.
(merge 3e73cc62c0 en/status-multiple-renames-to-the-same-target-fix later to maint).

* "git fetch $repo $object" in a partial clone did not correctly fetch the asked-for object that is referenced by an object in promisor packfile, which has been fixed.

* A corner-case bugfix.
(merge c5cbb27cb5 sm/show-superproject-while-conflicted later to maint).

* Various fixes to "diff --color-moved-ws".

* A partial clone that is configured to lazily fetch missing objects will on-demand issue a "git fetch" request to the originating repository to fill not-yet-obtained objects. The request has been optimized for requesting a tree object (and not the leaf blob objects contained in it) by telling the originating repository that no blobs are needed.
(merge 4c7f9567ea jt/non-blob-lazy-fetch later to maint).

* The codepath to support the experimental split-index mode had remaining "racily clean" issues fixed.
(merge 4c490f3d32 sg/split-index-racefix later to maint).

* "git log --graph" showing an octopus merge sometimes miscounted the number of display columns it is consuming to show the merge and its parent commits, which has been corrected.
(merge 04005834ed np/log-graph-octopus-fix later to maint).

* "git range-diff" did not work well when the compared ranges had changes in submodules and the "--submodule=log" was used.

* The implementation of run_command() API on the UNIX platforms had a bug that caused a command not on $PATH to be found in the current directory.
(merge f67b980771 jk/run-command-notdot later to maint).

* A mutex used in "git pack-objects" were not correctly initialized and this caused "git repack" to dump core on Windows. (merge 34204c8166 js/pack-objects-mutex-init-fix later to maint).

* Under certain circumstances, "git diff D:/a/b/c D:/a/b/d" on Windows would strip initial parts from the paths because they were not recognized as absolute, which has been corrected. (merge ffd04e92e2 js/diff-notice-has-drive-prefix later to maint).

* The receive.denyCurrentBranch=updateInstead codepath kicked in even when the push should have been rejected due to other reasons, such as it does not fast-forward or the update-hook rejects it, which has been corrected.
(merge b072a25fad jc/receive-deny-current-branch-fix later to maint).

* The logic to determine the archive type "git archive" uses did not correctly kick in for "git archive --remote", which has been corrected.

* "git repack" in a shallow clone did not correctly update the shallow points in the repository, leading to a repository that does not pass fsck.
(merge 5dcfbf564c js/shallow-and-fetch-prune later to maint).

* Some codepaths failed to form a proper URL when .gitmodules record the URL to a submodule repository as relative to the repository of superproject, which has been corrected.
(merge e0a862fdaf sb/submodule-url-to-absolute later to maint).

* "git fetch" over protocol v2 into a shallow repository failed to fetch full history behind a new tip of history that was diverged before the cut-off point of the history that was previously fetched shallowly.

* The command line completion machinery (in contrib/) has been updated to allow the completion script to tweak the list of options that are reported by the parse-options machinery correctly. (merge 276b49ff34 nd/completion-negation later to maint).

* Operations on promisor objects make sense in the context of only a small subset of the commands that internally use the revisions machinery, but the "--exclude-promisor-objects" option were taken and led to nonsense results by commands like "log", to which it didn't make much sense. This has been corrected. (merge 669b1d2aae md/exclude-promisor-objects-fix later to maint).

* The "container" mode of TravisCI is going away. Our .travis.yml file is getting prepared for the transition.
(merge 32ee384be8 ss/travis-ci-force-vm-mode later to maint).

* Our test scripts can now take the '-V' option as a synonym for the '--verbose-log' option.
(merge a5f52c6dab sg/test-verbose-log later to maint).

* A regression in Git 2.12 era made "git fsck" fall into an infinite loop while processing truncated loose objects.
(merge 18ad13e5b2 jk/detect-truncated-zlib-input later to maint).

* "git ls-remote $there foo" was broken by recent update for the protocol v2 and stopped showing refs that match 'foo' that are not refs/{heads,tags}/foo, which has been fixed.
(merge 6a139cdd74 jk/proto-v2-ref-prefix-fix later to maint).

* Additional comment on a tricky piece of code to help developers. (merge 0afbe3e806 jk/stream-pack-non-delta-clarification later to maint).

* A couple of tests used to leave the repository in a state that is deliberately corrupt, which have been corrected. (merge aa984dbe5e ab/pack-tests-cleanup later to maint).

* The submodule support has been updated to read from the blob at HEAD:.gitmodules when the .gitmodules file is missing from the working tree.
(merge 2b1257e463 ao/submodule-wo-gitmodules-checked-out later to maint).

* "git fetch" was a bit loose in parsing responses from the other side when talking over the protocol v2.

* "git rev-parse --exclude=* --branches --branches" (i.e. first saying "add only things that do not match '*' out of all branches" and then adding all branches, without any exclusion this time") worked as expected, but "--exclude=* --all --all" did not work the same way, which has been fixed.
(merge 5221048092 ag/rev-parse-all-exclude-fix later to maint).

* "git send-email --transfer-encoding=..." in recent versions of Git sometimes produced an empty "Content-Transfer-Encoding:" header, which has been corrected.
(merge 3c88e46f1a al/send-email-auto-cte-fixup later to maint).

* The interface into "xdiff" library used to discover the offset and size of a generated patch hunk by first formatting it into the textual hunk header "@@ -n,m +k,l @@" and then parsing the numbers out. A new interface has been introduced to allow callers a more direct access to them.
(merge 5eade0746e jk/xdiff-interface later to maint).

* Pathspec matching against a tree object were buggy when negative pathspec elements were involved, which has been fixed. (merge b7845cebc0 nd/tree-walk-path-exclusion later to maint).

* "git merge" and "git pull" that merges into an unborn branch used to completely ignore "--verify-signatures", which has been corrected.
(merge 01a31f3bca jk/verify-sig-merge-into-void later to maint).

* "git rebase --autostash" did not correctly re-attach the HEAD at times.

* "rev-parse --exclude=<pattern> --branches=<pattern>" etc. did not quite work, which has been corrected.
(merge 9ab9b5df0e ra/rev-parse-exclude-glob later to maint).

* When editing a patch in a "git add -i" session, a hunk could be made to no-op. The "git apply" program used to reject a patch with such a no-op hunk to catch user mistakes, but it is now updated to explicitly allow a no-op hunk in an edited patch.
(merge 22cb3835b9 js/apply-recount-allow-noop later to maint).

* The URL to an MSDN page in a comment has been updated. (merge 2ef2ae2917 js/mingw-msdn-url later to maint).

* "git ls-remote --sort=<thing>" can feed an object that is not yet available into the comparison machinery and segfault, which has been corrected to check such a request upfront and reject it.

* When "git bundle" aborts due to an empty commit ranges
(i.e. resulting in an empty pack), it left a file descriptor to an lockfile open, which resulted in leftover lockfile on Windows where you cannot remove a file with an open file descriptor. This has been corrected.
(merge 2c8ee1f53c jk/close-duped-fd-before-unlock-for-bundle later to maint).

* "git format-patch --stat=<width>" can be used to specify the width used by the diffstat (shown in the cover letter).
(merge 284aeb7e60 nd/format-patch-cover-letter-stat-width later to maint).

* The way .git/index and .git/sharedindex* files were initially created gave these files different perm bits until they were adjusted for shared repository settings. This was made consistent. (merge c9d6c78870 cc/shared-index-permbits later to maint).

* Code cleanup, docfix, build fix, etc.
(merge 96a7501aad ts/doc-build-manpage-xsl-quietly later to maint). (merge b9b07efdb2 tg/conflict-marker-size later to maint).
(merge fa0aeea770 sg/doc-trace-appends later to maint).
(merge d64324cb60 tb/void-check-attr later to maint).
(merge c3b9bc94b9 en/double-semicolon-fix later to maint).
(merge 79336116f5 sg/t3701-tighten-trace later to maint).
(merge 801fa63a90 jk/dev-build-format-security later to maint). (merge 0597dd62ba sb/string-list-remove-unused later to maint). (merge db2d36fad8 bw/protocol-v2 later to maint).
(merge 456d7cd3a9 sg/split-index-test later to maint).
(merge 7b6057c852 tq/refs-internal-comment-fix later to maint). (merge 29e8dc50ad tg/t5551-with-curl-7.61.1 later to maint). (merge 55f6bce2c9 fe/doc-updates later to maint).
(merge 7987d2232d jk/check-everything-connected-is-long-gone later to maint). (merge 4ba3c9be47 dz/credential-doc-url-matching-rules later to maint). (merge 4c399442f7 ma/commit-graph-docs later to maint).
(merge fc0503b04e ma/t1400-undebug-test later to maint).
(merge e56b53553a nd/packobjectshook-doc-fix later to maint). (merge c56170a0c4 ma/mailing-list-address-in-git-help later to maint). (merge 6e8fc70fce rs/sequencer-oidset-insert-avoids-dups later to maint). (merge ad0b8f9575 mw/doc-typofixes later to maint).
(merge d9f079ad1a jc/how-to-document-api later to maint).
(merge b1492bf315 ma/t7005-bash-workaround later to maint).
(merge ac1f98a0df du/rev-parse-is-plumbing later to maint).
(merge ca8ed443a5 mm/doc-no-dashed-git later to maint).
(merge ce366a8144 du/get-tar-commit-id-is-plumbing later to maint). (merge 61018fe9e0 du/cherry-is-plumbing later to maint).
(merge c7e5fe79b9 sb/strbuf-h-update later to maint).
(merge 8d2008196b tq/branch-create-wo-branch-get later to maint). (merge 2e3c894f4b tq/branch-style-fix later to maint).
(merge c5d844af9c sg/doc-show-branch-typofix later to maint). (merge 081d91618b ah/doc-updates later to maint).
(merge b84c783882 jc/cocci-preincr later to maint).
(merge 5e495f8122 uk/merge-subtree-doc-update later to maint). (merge aaaa881822 jk/uploadpack-packobjectshook-fix later to maint). (merge 3063477445 tb/char-may-be-unsigned later to maint).
(merge 8c64bc9420 sg/test-rebase-editor-fix later to maint). (merge 71571cd7d6 ma/sequencer-do-reset-saner-loop-termination later to maint). (merge 9a4cb8781e cb/notes-freeing-always-null-fix later to maint).

----------------------------------------------------------------

Changes since v2.19.0 are as follows:

Aaron Lindsay (1):
send-email: avoid empty transfer encoding header

Alban Gruin (21):
sequencer: make three functions and an enum from sequencer.c public rebase -i: rewrite append_todo_help() in C
editor: add a function to launch the sequence editor
rebase -i: rewrite the edit-todo functionality in C
sequencer: add a new function to silence a command, except if it fails rebase -i: rewrite setup_reflog_action() in C
rebase -i: rewrite checkout_onto() in C
sequencer: refactor append_todo_help() to write its message to a buffer
sequencer: change the way skip_unnecessary_picks() returns its result
t3404: todo list with commented-out commands only aborts rebase -i: rewrite complete_action() in C
rebase -i: remove unused modes and functions
rebase -i: implement the logic to initialize $revisions in C rebase -i: rewrite the rest of init_revisions_and_shortrevisions() in C rebase -i: rewrite write_basic_state() in C
rebase -i: rewrite init_basic_state() in C
rebase -i: implement the main part of interactive rebase as a builtin
rebase--interactive2: rewrite the submodes of interactive rebase in C rebase -i: remove git-rebase--interactive.sh
rebase -i: move rebase--helper modes to rebase--interactive
p3400: replace calls to git checkout -b' by git checkout -B'

Alexander Pyhalov (1):
t7005-editor: quote filename to fix whitespace-issue

Andreas Gruenbacher (1):

      rev-parse: clear --exclude list after 'git rev-parse --all'

Andreas Heiduk (6):
doc: clarify boundaries of 'git worktree list --porcelain'
doc: fix ASCII art tab spacing
doc: fix inappropriate monospace formatting
doc: fix descripion for 'git tag --format'
doc: fix indentation of listing blocks in gitweb.conf.txt
doc: fix formatting in git-update-ref

Anton Serbulov (1):
mingw: fix getcwd when the parent directory cannot be queried

Antonio Ospite (10):
submodule: add a print_config_from_gitmodules() helper
submodule: factor out a config_set_in_gitmodules_file_gently function
t7411: merge tests 5 and 6
t7411: be nicer to future tests and really clean things up
submodule--helper: add a new 'config' subcommand
submodule: use the 'submodule--helper config' command
t7506: clean up .gitmodules properly before setting up new scenario
submodule: add a helper to check if it is safe to write to .gitmodules
submodule: support reading .gitmodules when it's not in the working tree t/helper: add test-submodule-nested-repo-config

Ben Peart (19):
checkout: optimize "git checkout -b <new_branch>"
git-mv: allow submodules and fsmonitor to work together t/README: correct spelling of "uncommon"
preload-index: use git_env_bool() not getenv() for customization
fsmonitor: update GIT_TEST_FSMONITOR support
read-cache: update TEST_GIT_INDEX_VERSION support
preload-index: update GIT_FORCE_PRELOAD_TEST support
read-cache: clean up casting and byte decoding
eoie: add End of Index Entry (EOIE) extension
config: add new index.threads config setting
read-cache: load cache extensions on a worker thread
ieot: add Index Entry Offset Table (IEOT) extension
read-cache: load cache entries on worker threads
reset: don't compute unstaged changes after reset when --quiet
reset: add new reset.quiet config setting
reset: warn when refresh_index() takes more than 2 seconds speed up refresh_index() by utilizing preload_index()
add: speed up cmd_add() by utilizing read_cache_preload()
refresh_index: remove unnecessary calls to preload_index()

Brandon Williams (1):
config: document value 2 for protocol.version

Brendan Forster (1):
http: add support for disabling SSL revocation checks in cURL

Carlo Marcelo Arenas Belón (8):
unpack-trees: avoid dead store for struct progress
multi-pack-index: avoid dead store for struct progress
read-cache: use of memory after it is freed
commit-slabs: move MAYBE_UNUSED out
khash: silence -Wunused-function for delta-islands
compat: make sure git_mmap is not expected to write
sequencer: cleanup for gcc warning in non developer mode builtin/notes: remove unnecessary free

Christian Couder (3):
pack-objects: refactor code into compute_layer_order()
pack-objects: move tree_depth into 'struct packing_data'
pack-objects: move 'layer' into 'struct packing_data'

Christian Hesse (2):
subtree: add build targets 'man' and 'html'
subtree: make install targets depend on build targets

Daniels Umanovskis (3):
doc: move git-rev-parse from porcelain to plumbing
doc: move git-get-tar-commit-id to plumbing
doc: move git-cherry to plumbing

David Zych (1):
doc: clarify gitcredentials path component matching

Denton Liu (3):
mergetool: accept -g/--[no-]gui as arguments
completion: support git mergetool --[no-]gui
doc: document diff/merge.guitool config keys

Derrick Stolee (93):
multi-pack-index: add design document
multi-pack-index: add format details
multi-pack-index: add builtin
multi-pack-index: add 'write' verb
midx: write header information to lockfile
multi-pack-index: load into memory
t5319: expand test data
packfile: generalize pack directory list
multi-pack-index: read packfile list
multi-pack-index: write pack names in chunk
midx: read pack names into array
midx: sort and deduplicate objects from packfiles
midx: write object ids in a chunk
midx: write object id fanout chunk
midx: write object offsets
config: create core.multiPackIndex setting
midx: read objects from multi-pack-index
midx: use midx in abbreviation calculations
midx: use existing midx when writing new one
midx: use midx in approximate_object_count
midx: prevent duplicate packfile loads
packfile: skip loading index if in multi-pack-index
midx: clear midx on repack
commit-reach: move walk methods from commit.c commit.h: remove method declarations
commit-reach: move ref_newer from remote.c
commit-reach: move commit_contains from ref-filter
upload-pack: make reachable() more generic
upload-pack: refactor ok_to_give_up()
upload-pack: generalize commit date cutoff
commit-reach: move can_all_from_reach_with_flags
test-reach: create new test tool for ref_newer
test-reach: test in_merge_bases
test-reach: test is_descendant_of
test-reach: test get_merge_bases_many
test-reach: test reduce_heads
test-reach: test can_all_from_reach_with_flags
test-reach: test commit_contains
commit-reach: replace ref_newer logic
commit-reach: make can_all_from_reach... linear
commit-reach: use can_all_from_reach
multi-pack-index: provide more helpful usage info
multi-pack-index: store local property
midx: mark bad packed objects
midx: stop reporting garbage
midx: fix bug that skips midx with alternates
packfile: add all_packs list
treewide: use get_all_packs
midx: test a few commands that use get_all_packs
pack-objects: consider packs in multi-pack-index
commit-graph: update design document
test-repository: properly init repo
commit-graph: not compatible with replace objects
commit-graph: not compatible with grafts
commit-graph: not compatible with uninitialized repo
commit-graph: close_commit_graph before shallow walk
commit-graph: define GIT_TEST_COMMIT_GRAPH t3206-range-diff.sh: cover single-patch case
t5318: use test_oid for HASH_LEN
multi-pack-index: add 'verify' verb
multi-pack-index: verify bad header
multi-pack-index: verify corrupt chunk lookup table
multi-pack-index: verify packname order
multi-pack-index: verify missing pack
multi-pack-index: verify oid fanout order
multi-pack-index: verify oid lookup order
multi-pack-index: fix 32-bit vs 64-bit size check
multi-pack-index: verify object offsets
multi-pack-index: report progress during 'verify'
fsck: verify multi-pack-index
commit-reach: properly peel tags
commit-reach: fix memory and flag leaks
commit-reach: cleanups in can_all_from_reach...
commit-graph: clean up leaked memory during write
commit-graph: reduce initial oid allocation
midx: fix broken free() in close_midx()
contrib: add coverage-diff script
ci: add optional test variables
commit-reach: fix first-parent heuristic
midx: close multi-pack-index on repack
multi-pack-index: define GIT_TEST_MULTI_PACK_INDEX
packfile: close multi-pack-index in close_all_packs
prio-queue: add 'peek' operation
test-reach: add run_three_modes method
test-reach: add rev-list tests
revision.c: begin refactoring --topo-order logic commit/revisions: bookkeeping before refactoring revision.c: generation-based topo-order algorithm
t6012: make rev-list tests more interesting
commit-reach: implement get_reachable_subset
test-reach: test get_reachable_subset
remote: make add_missing_tags() linear
pack-objects: ignore ambiguous object warnings

Elijah Newren (14):
Remove superfluous trailing semicolons
t4200: demonstrate rerere segfault on specially crafted merge
rerere: avoid buffer overrun
update-ref: fix type of update_flags variable to match its usage
update-ref: allow --no-deref with --stdin
sequencer: fix --allow-empty-message behavior, make it smarter
merge-recursive: set paths correctly when three-way merging content
merge-recursive: avoid wrapper function when unnecessary and wasteful
merge-recursive: remove final remaining caller of merge_file_one()
merge-recursive: rename merge_file_1() and merge_content()
commit: fix erroneous BUG, 'multiple renames on the same target? how?'
merge-recursive: improve auto-merging messages with path collisions
merge-recursive: avoid showing conflicts with merge branch before HEAD
fsck: move fsck_head_link() to get_default_heads() to avoid some globals

Eric Sunshine (26):
format-patch: allow additional generated content in make_cover_letter()
format-patch: add --interdiff option to embed diff in cover letter
format-patch: teach --interdiff to respect -v/--reroll-count
interdiff: teach show_interdiff() to indent interdiff
log-tree: show_log: make commentary block delimiting reusable
format-patch: allow --interdiff to apply to a lone-patch
range-diff: respect diff_option.file rather than assuming 'stdout'
range-diff: publish default creation factor
range-diff: relieve callers of low-level configuration burden
format-patch: add --range-diff option to embed diff in cover letter
format-patch: extend --range-diff to accept revision range
format-patch: teach --range-diff to respect -v/--reroll-count
format-patch: add --creation-factor tweak for --range-diff
format-patch: allow --range-diff to apply to a lone-patch
worktree: don't die() in library function find_worktree()
worktree: move delete_git_dir() earlier in file for upcoming new callers
worktree: generalize delete_git_dir() to reduce code duplication
worktree: prepare for more checks of whether path can become worktree
worktree: disallow adding same path multiple times
worktree: teach 'add' to respect --force for registered but missing path
worktree: teach 'move' to override lock when --force given twice
worktree: teach 'remove' to override lock when --force given twice
worktree: delete .git/worktrees if empty after 'remove'
doc-diff: fix non-portable 'man' invocation
doc-diff: add --clean mode to remove temporary working gunk doc/Makefile: drop doc-diff worktree and temporary files on "make clean"

Frederick Eaton (3):
git-archimport.1: specify what kind of Arch we're talking about git-column.1: clarify initial description, provide examples git-describe.1: clarify that "human readable" is also git-readable

James Knight (1):
build: link with curl-defined linker flags

Jann Horn (2):
patch-delta: fix oob read
patch-delta: consistently report corruption

Jeff Hostetler (2):
t0051: test GIT_TRACE to a windows named pipe
mingw: fix mingw_open_append to work with named pipes

Jeff King (97):
branch: make "-l" a synonym for "--list"
Add delta-islands.{c,h}
pack-objects: add delta-islands support
repack: add delta-islands support
t5320: tests for delta islands
t/perf: factor boilerplate out of test_perf
t/perf: factor out percent calculations
t/perf: add infrastructure for measuring sizes
t/perf: add perf tests for fetches from a bitmapped server
pack-bitmap: save "have" bitmap from walk
pack-objects: reuse on-disk deltas for thin "have" objects
SubmittingPatches: mention doc-diff
rev-list: make empty --stdin not an error
trailer: use size_t for string offsets
trailer: use size_t for iterating trailer list
trailer: pass process_trailer_opts to trailer_info_get()
interpret-trailers: tighten check for "---" patch boundary
interpret-trailers: allow suppressing "---" divider pretty, ref-filter: format %(trailers) with no_divider option
sequencer: ignore "---" divider when parsing trailers
append_signoff: use size_t for string offsets
coccinelle: use <...> for function exclusion
introduce hasheq() and oideq()
convert "oidcmp() == 0" to oideq()
convert "hashcmp() == 0" to hasheq()
convert "oidcmp() != 0" to "!oideq()"
convert "hashcmp() != 0" to "!hasheq()"
convert hashmap comparison functions to oideq()
read-cache: use oideq() in ce_compare functions
show_dirstat: simplify same-content check
doc-diff: always use oids inside worktree
test-delta: read input into a heap buffer
t5303: test some corrupt deltas
patch-delta: handle truncated copy parameters
t5303: use printf to generate delta bases doc/git-branch: remove obsolete "-l" references bitmap_has_sha1_in_uninteresting(): drop BUG check
t5310: test delta reuse with bitmaps traverse_bitmap_commit_list(): don't free result
pack-bitmap: drop "loaded" flag
reopen_tempfile(): truncate opened file
doc-diff: force worktree add
config.mak.dev: add -Wformat-security
pack-objects: handle island check for "external" delta base
receive-pack: update comment with check_everything_connected
submodule--helper: use "--" to signal end of clone options
submodule-config: ban submodule urls that start with dash
submodule-config: ban submodule paths that start with a dash
fsck: detect submodule urls starting with dash
fsck: detect submodule paths starting with dash
more oideq/hasheq conversions
transport: drop refnames from for_each_alternate_ref
test-tool: show tool list on error
config.mak.dev: enable -Wunused-function
run-command: mark path lookup errors with ENOENT
t5410: use longer path for sample script
upload-pack: fix broken if/else chain in config callback
t1450: check large blob in trailing-garbage test check_stream_sha1(): handle input underflow
cat-file: handle streaming failures consistently
ls-remote: do not send ref prefixes for patterns
ls-remote: pass heads/tags prefixes to transport read_istream_pack_non_delta(): document input handling
xdiff: provide a separate emit callback for hunks
xdiff-interface: provide a separate consume callback for hunks
rev-list: handle flags for --indexed-objects
approxidate: handle pending number for "specials"
pathspec: handle non-terminated strings with :(attr)
diff: avoid generating unused hunk header lines
diff: discard hunk headers for patch-ids earlier
diff: use hunk callback for word-diff
combine-diff: use an xdiff hunk callback
diff: convert --check to use a hunk callback
range-diff: use a hunk callback
xdiff-interface: drop parse_hunk_header()
apply: mark include/exclude options as NONEG
am: handle --no-patch-format option
ls-files: mark exclude options as NONEG
pack-objects: mark index-version option as NONEG
cat-file: mark batch options with NONEG
status: mark --find-renames option with NONEG
format-patch: mark "--no-numbered" option with NONEG
show-branch: mark --reflog option as NONEG
tag: mark "--message" option with NONEG
cat-file: report an error on multiple --batch options
apply: return -1 from option callback instead of calling exit(1)
parse-options: drop OPT_DATE()
assert NOARG/NONEG behavior of parse-options callbacks
midx: double-check large object write loop
merge: extract verify_merge_signature() helper
merge: handle --verify-signatures for unborn branch
pull: handle --verify-signatures for unborn branch
approxidate: fix NULL dereference in date_time()
bundle: dup() output descriptor closer to point-of-use
pack-objects: fix tree_depth and layer invariants
pack-objects: zero-initialize tree_depth/layer arrays
pack-objects: fix off-by-one in delta-island tree-depth computation

Johannes Schindelin (62):
rebase -i --autosquash: demonstrate a problem skipping the last squash rebase -i: be careful to wrap up fixup/squash chains compat/poll: prepare for targeting Windows Vista
mingw: set _WIN32_WINNT explicitly for Git for Windows
mingw: bump the minimum Windows version to Vista
builtin rebase: prepare for builtin rebase -i
rebase -i: clarify what happens on a failed exec
rebase -i: introduce the 'break' command
getpwuid(mingw): initialize the structure only once getpwuid(mingw): provide a better default for the user name
mingw: use domain information for default email
http: add support for selecting SSL backends at runtime
pack-objects: fix typo 'detla' -> 'delta'
pack-objects (mingw): demonstrate a segmentation fault with large deltas pack-objects (mingw): initialize packing_data mutex in the correct spot rebase (autostash): avoid duplicate call to state_dir_path() rebase (autostash): store the full OID in <state-dir>/autostash rebase (autostash): use an explicit OID to apply the stash
mingw: factor out code to set stat() data
rebase --autostash: demonstrate a problem with dirty submodules rebase --autostash: fix issue with dirty submodules
mingw: load system libraries the recommended way
mingw: ensure getcwd() reports the correct case
repack: point out a bug handling stale shallow info
shallow: offer to prune only non-existing entries
repack -ad: prune the list of shallow commits
http: when using Secure Channel, ignore sslCAInfo by default
t7800: fix quoting
mingw: reencode environment variables on the fly (UTF-16 <-> UTF-8)
config: rename dummy parameter to cb in git_default_config()
config: allow for platform-specific core.* config settings
config: move Windows-specific config settings into compat/mingw.c
mingw: unset PERL5LIB by default
mingw: fix isatty() after dup2()
t3404: decouple some test cases from outcomes of previous test cases
t3418: decouple test cases from a previous rebase -p test case
tests: optionally skip git rebase -p tests
Windows: force-recompile git.res for differing architectures built-in rebase: demonstrate regression with --autostash built-in rebase --autostash: leave the current branch alone if possible Update .mailmap
rebase -r: demonstrate bug with conflicting merges
rebase -r: do not write MERGE_HEAD unless needed
rebase -i: include MERGE_HEAD into files to clean up built-in rebase --skip/--abort: clean up stale .git/<name> files
status: rebase and merge can be in progress at the same time apply --recount: allow "no-op hunks"
rebase: consolidate clean-up code before leaving reset_head()
rebase: prepare reset_head() for more flags
built-in rebase: reinstate checkout -q behavior where appropriate
tests: fix GIT_TEST_INSTALLED's PATH to include t/helper/
tests: respect GIT_TEST_INSTALLED when initializing repositories t/lib-gettext: test installed git-sh-i18n if GIT_TEST_INSTALLED is set
mingw: use CreateHardLink() directly
rebase: really just passthru the git am options
rebase: validate -C<n> and --whitespace=<mode> parameters early
config: report a bug if git_dir exists without commondir
tests: do not require Git to be built when testing an installed Git
tests: explicitly use git.exe on Windows
mingw: replace an obsolete link with the superseding one
legacy-rebase: backport -C<n> and --whitespace=<option> checks
rebase: warn about the correct tree's OID

Johannes Sixt (3):
diff: don't attempt to strip prefix from absolute Windows paths rebase -i: recognize short commands without arguments
t3404-rebase-interactive: test abbreviated commands

Jonathan Nieder (9):
gc: improve handling of errors reading gc.log
gc: exit with status 128 on failure
gc: do not return error for prior errors in daemonized mode
commit-reach: correct accidental #include of C file
mailmap: consistently normalize brian m. carlson's name git doc: direct bug reporters to mailing list archive
eoie: default to not writing EOIE section
ieot: default to not writing IEOT section
index: make index.threads=true enable ieot and eoie

Jonathan Tan (15):
fetch-object: unify fetch_object[s] functions
fetch-object: set exact_oid when fetching
connected: document connectivity in partial clones
fetch: in partial clone, check presence of targets
fetch-pack: avoid object flags if no_dependents
fetch-pack: exclude blobs when lazy-fetching trees
transport: allow skipping of ref listing
transport: do not list refs if possible
transport: list refs before fetch if necessary
fetch: do not list refs if fetching only hashes
cache-tree: skip some blob checks in partial clone
upload-pack: make have_obj not global
upload-pack: make want_obj not global
upload-pack: clear flags before each v2 request
fetch-pack: be more precise in parsing v2 response

Josh Steadmon (4):
fuzz: add basic fuzz testing target.
fuzz: add fuzz testing for packfile indices.
archive: initialize archivers earlier
Makefile: use FUZZ_CXXFLAGS for linking fuzzers

Joshua Watt (1):
send-email: explicitly disable authentication

Junio C Hamano (34):
Revert "doc/Makefile: drop doc-diff worktree and temporary files on "make clean"" Initial batch post 2.19
Second batch post 2.19
Git 2.14.5
Git 2.15.3
Git 2.16.5
Git 2.17.2
Git 2.18.1
Git 2.19.1
t0000: do not get self-test disrupted by environment warnings
CodingGuidelines: document the API in *.h files
Declare that the next one will be named 2.20
Third batch for 2.20
rebase: fix typoes in error messages
Fourth batch for 2.20
Revert "subtree: make install targets depend on build targets" Fifth batch for 2.20
receive: denyCurrentBranch=updateinstead should not blindly update
cocci: simplify "if (++u > 1)" to "if (u++)"
fsck: s/++i > 1/i++/
http: give curl version warnings consistently
Sixth batch for 2.20
Seventh batch for 2.20
fetch: replace string-list used as a look-up table with a hashmap
rebase: apply cocci patch
Eighth batch for 2.20
Ninth batch for 2.20
Makefile: ease dynamic-gettext-poison transition
Tenth batch for 2.20
Git 2.20-rc0
RelNotes: name the release properly
Prepare for 2.20-rc1
Git 2.19.2
Git 2.20-rc1

Karsten Blees (2):
mingw: replace MSVCRT's fstat() with a Win32-based implementation
mingw: implement nanosecond-precision file times

Loo Rong Jie (1):
win32: replace pthread_cond_*() with much simpler code

Lucas De Marchi (1):
range-diff: allow to diff files regardless of submodule config

Luke Diamand (3):
git-p4: do not fail in verbose mode for missing 'fileSize' key
git-p4: unshelve into refs/remotes/p4-unshelved, not refs/remotes/p4/unshelved
git-p4: fully support unshelving changelists

Martin Ågren (9):
Doc: use --type=bool instead of --bool git-config.txt: fix 'see: above' note git-commit-graph.txt: fix bullet lists git-commit-graph.txt: typeset more in monospace git-commit-graph.txt: refer to "*commit*-graph file"
Doc: refer to the "commit-graph file" with dash
t1400: drop debug echo to actually execute test builtin/commit-graph.c: UNLEAK variables
sequencer: break out of loop explicitly

Matthew DeVore (19):
list-objects: store common func args in struct
list-objects: refactor to process_tree_contents
list-objects: always parse trees gently
t/README: reformat Do, Don't, Keep in mind lists
Documentation: add shell guidelines
tests: standardize pipe placement
t/*: fix ordering of expected/observed arguments
tests: don't swallow Git errors upstream of pipes
t9109: don't swallow Git errors upstream of pipes
tests: order arguments to git-rev-list properly
rev-list: handle missing tree objects properly
revision: mark non-user-given objects instead
list-objects-filter: use BUG rather than die list-objects-filter-options: do not over-strbuf_init
list-objects-filter: implement filter tree:0
filter-trees: code clean-up of tests
list-objects: support for skipping tree traversal Documentation/git-log.txt: do not show --exclude-promisor-objects
exclude-promisor-objects: declare when option is allowed

Max Kirillov (1):
http-backend test: make empty CONTENT_LENGTH test more realistic

Michael Witten (3):
docs: typo: s/go/to/
docs: graph: remove unnecessary `graph_update()' call
docs: typo: s/isimilar/similar/

Michał Górny (6):
gpg-interface.c: detect and reject multiple signatures on commits gpg-interface.c: use flags to determine key/signer info presence gpg-interface.c: support getting key fingerprint via %GF format gpg-interface.c: obtain primary key fingerprint as well t/t7510-signed-commit.sh: Add %GP to custom format checks t/t7510-signed-commit.sh: add signing subkey to Eris Discordia key

Mihir Mehta (1):
doc: fix a typo and clarify a sentence

Nguyễn Thái Ngọc Duy (168):
clone: report duplicate entries on case-insensitive filesystems trace.h: support nested performance tracing
unpack-trees: add performance tracing
unpack-trees: optimize walking same trees with cache-tree
unpack-trees: reduce malloc in cache-tree walk
unpack-trees: reuse (still valid) cache-tree from src_index
unpack-trees: add missing cache invalidation
cache-tree: verify valid cache-tree in the test suite Document update for nd/unpack-trees-with-cache-tree bisect.c: make show_list() build again
t/helper: keep test-tool command list sorted
t/helper: merge test-dump-untracked-cache into test-tool t/helper: merge test-pkt-line into test-tool
t/helper: merge test-parse-options into test-tool t/helper: merge test-dump-fsmonitor into test-tool
Makefile: add a hint about TEST_BUILTINS_OBJS
config.txt: follow camelCase naming
config.txt: move fetch part out to a separate file config.txt: move format part out to a separate file config.txt: move gitcvs part out to a separate file config.txt: move gui part out to a separate file config.txt: move pull part out to a separate file config.txt: move push part out to a separate file config.txt: move receive part out to a separate file config.txt: move sendemail part out to a separate file config.txt: move sequence.editor out of "core" part config.txt: move submodule part out to a separate file archive.c: remove implicit dependency the_repository
status: show progress bar if refreshing the index takes too long
add: do not accept pathspec magic 'attr'
completion: support "git fetch --multiple"
read-cache.c: remove 'const' from index_has_changes() diff.c: reduce implicit dependency on the_index combine-diff.c: remove implicit dependency on the_index blame.c: rename "repo" argument to "r"
diff.c: remove the_index dependency in textconv() functions grep.c: remove implicit dependency on the_index
diff.c: remove implicit dependency on the_index read-cache.c: remove implicit dependency on the_index diff-lib.c: remove implicit dependency on the_index ll-merge.c: remove implicit dependency on the_index merge-blobs.c: remove implicit dependency on the_index merge.c: remove implicit dependency on the_index patch-ids.c: remove implicit dependency on the_index sha1-file.c: remove implicit dependency on the_index rerere.c: remove implicit dependency on the_index userdiff.c: remove implicit dependency on the_index line-range.c: remove implicit dependency on the_index submodule.c: remove implicit dependency on the_index tree-diff.c: remove implicit dependency on the_index
ws.c: remove implicit dependency on the_index
revision.c: remove implicit dependency on the_index revision.c: reduce implicit dependency the_repository read-cache.c: optimize reading index format v4
config.txt: correct the note about uploadpack.packObjectsHook help -a: improve and make --verbose default
refs.c: indent with tabs, not spaces
Add a place for (not) sharing stuff between worktrees submodule.c: remove some of the_repository references
completion: fix __gitcomp_builtin no longer consider extra options
t1300: extract and use test_cmp_config()
worktree: add per-worktree config files
refs: new ref types to make per-worktree refs visible to all worktrees revision.c: correct a parameter name
revision.c: better error reporting on ref from different worktrees
fsck: check HEAD and reflog from other worktrees
reflog expire: cover reflog from all worktrees
Update makefile in preparation for Documentation/config/*.txt config.txt: move advice.* to a separate file
config.txt: move core.* to a separate file
config.txt: move add.* to a separate file
config.txt: move alias.* to a separate file
config.txt: move am.* to a separate file
config.txt: move apply.* to a separate file
config.txt: move blame.* to a separate file
config.txt: move branch.* to a separate file
config.txt: move browser.* to a separate file
config.txt: move checkout.* to a separate file
config.txt: move clean.* to a separate file
config.txt: move color.* to a separate file
config.txt: move column.* to a separate file
config.txt: move commit.* to a separate file
config.txt: move credential.* to a separate file config.txt: move completion.* to a separate file config.txt: move diff-config.txt to config/
config.txt: move difftool.* to a separate file
config.txt: move fastimport.* to a separate file config.txt: move fetch-config.txt to config/
config.txt: move filter.* to a separate file
config.txt: move format-config.txt to config/
config.txt: move fmt-merge-msg-config.txt to config/ config.txt: move fsck.* to a separate file
config.txt: move gc.* to a separate file
config.txt: move gitcvs-config.txt to config/
config.txt: move gitweb.* to a separate file
config.txt: move grep.* to a separate file
config.txt: move gpg.* to a separate file
config.txt: move gui-config.txt to config/
config.txt: move guitool.* to a separate file
config.txt: move help.* to a separate file
config.txt: move ssh.* to a separate file
config.txt: move http.* to a separate file
config.txt: move i18n.* to a separate file git-imap-send.txt: move imap.* to a separate file config.txt: move index.* to a separate file
config.txt: move init.* to a separate file
config.txt: move instaweb.* to a separate file
config.txt: move interactive.* to a separate file config.txt: move log.* to a separate file
config.txt: move mailinfo.* to a separate file
config.txt: move mailmap.* to a separate file
config.txt: move man.* to a separate file
config.txt: move merge-config.txt to config/
config.txt: move mergetool.* to a separate file config.txt: move notes.* to a separate file
config.txt: move pack.* to a separate file
config.txt: move pager.* to a separate file
config.txt: move pretty.* to a separate file
config.txt: move protocol.* to a separate file
config.txt: move pull-config.txt to config/
config.txt: move push-config.txt to config/
config.txt: move rebase-config.txt to config/
config.txt: move receive-config.txt to config/
config.txt: move remote.* to a separate file
config.txt: move remotes.* to a separate file
config.txt: move repack.* to a separate file
config.txt: move rerere.* to a separate file
config.txt: move reset.* to a separate file
config.txt: move sendemail-config.txt to config/ config.txt: move sequencer.* to a separate file config.txt: move showBranch.* to a separate file config.txt: move splitIndex.* to a separate file config.txt: move status.* to a separate file
config.txt: move stash.* to a separate file
config.txt: move submodule.* to a separate file config.txt: move tag.* to a separate file
config.txt: move transfer.* to a separate file
config.txt: move uploadarchive.* to a separate file config.txt: move uploadpack.* to a separate file config.txt: move url.* to a separate file
config.txt: move user.* to a separate file
config.txt: move versionsort.* to a separate file config.txt: move web.* to a separate file
config.txt: move worktree.* to a separate file
config.txt: remove config/dummy.txt
thread-utils: macros to unconditionally compile pthreads API
wildmatch: change behavior of "foo**bar" in WM_PATHNAME mode git-worktree.txt: correct linkgit command name sequencer.c: remove a stray semicolon
tree-walk.c: fix overoptimistic inclusion in :(exclude) matching run-command.h: include thread-utils.h instead of pthread.h send-pack.c: move async's #ifdef NO_PTHREADS back to run-command.c
index-pack: remove #ifdef NO_PTHREADS
name-hash.c: remove #ifdef NO_PTHREADS
attr.c: remove #ifdef NO_PTHREADS
grep: remove #ifdef NO_PTHREADS
grep: clean up num_threads handling
preload-index.c: remove #ifdef NO_PTHREADS
pack-objects: remove #ifdef NO_PTHREADS
read-cache.c: remove #ifdef NO_PTHREADS
read-cache.c: reduce branching based on HAVE_THREADS read-cache.c: initialize copy_len to shut up gcc 8
Clean up pthread_create() error handling
completion: use __gitcomp_builtin for format-patch
build: fix broken command-list.h generation with core.autocrlf
format-patch: respect --stat in cover letter's diffstat
doc: move extensions.worktreeConfig to the right place
clone: fix colliding file detection on APFS

Nickolai Belakovski (2):
worktree: update documentation for lock_reason and lock_reason_valid
worktree: rename is_worktree_locked to worktree_lock_reason

Noam Postavsky (1):
log: fix coloring of certain octopus merge shapes

Olga Telezhnaya (3):
ref-filter: free memory from used_atom
ls-remote: release memory instead of UNLEAK
ref-filter: free item->value and item->value->s

Phillip Wood (11):
diff: fix --color-moved-ws=allow-indentation-change diff --color-moved-ws: fix double free crash diff --color-moved-ws: fix out of bounds string access diff --color-moved-ws: fix a memory leak diff --color-moved-ws: fix another memory leak diff --color-moved: fix a memory leak
am: don't die in read_author_script()
am: improve author-script error reporting
am: rename read_author_script()
add read_author_script() to libgit
sequencer: use read_author_script()

Pratik Karki (46):
rebase: start implementing it as a builtin
rebase: refactor common shell functions into their own file builtin/rebase: support running "git rebase <upstream>" builtin rebase: support --onto
builtin rebase: support git rebase --onto A...B
builtin rebase: handle the pre-rebase hook and --no-verify builtin rebase: support --quiet
builtin rebase: support the verbose and diffstat options builtin rebase: require a clean worktree
builtin rebase: try to fast forward when possible
builtin rebase: support --force-rebase
builtin rebase: start a new rebase only if none is in progress builtin rebase: only store fully-qualified refs in options.head_name builtin rebase: support git rebase <upstream> <switch-to> builtin rebase: support --continue
builtin rebase: support --skip
builtin rebase: support --abort
builtin rebase: support --quit
builtin rebase: support --edit-todo and --show-current-patch builtin rebase: actions require a rebase in progress builtin rebase: stop if git am is in progress
builtin rebase: allow selecting the rebase "backend" builtin rebase: support --signoff
builtin rebase: support --rerere-autoupdate
builtin rebase: support --committer-date-is-author-date builtin rebase: support ignore-whitespace option
builtin rebase: support ignore-date option
builtin rebase: support keep-empty option
builtin rebase: support --autosquash
builtin rebase: support --gpg-sign option
builtin rebase: support -C and --whitespace=<type> builtin rebase: support --autostash option
builtin rebase: support --exec
builtin rebase: support --allow-empty-message option builtin rebase: support --rebase-merges[=[no-]rebase-cousins] merge-base --fork-point: extract libified function
builtin rebase: support fork-point option
builtin rebase: add support for custom merge strategies builtin rebase: support --root
builtin rebase: optionally auto-detect the upstream builtin rebase: optionally pass custom reflogs to reset_head() builtin rebase: fast-forward to onto if it is a proper descendant builtin rebase: show progress when connected to a terminal builtin rebase: use no-op editor when interactive is "implied" builtin rebase: error out on incompatible option/mode combinations
rebase: default to using the builtin rebase

Rafael Ascensão (2):
refs: show --exclude failure with --branches/tags/remotes=glob
refs: fix some exclude patterns being ignored

Ralf Thielow (1):
git-rebase.sh: fix typos in error messages

Ramsay Jones (12):
Makefile: add a hdr-check target
json-writer.h: add missing include (hdr-check) ewah/ewok_rlw.h: add missing include (hdr-check) refs/ref-cache.h: add missing declarations (hdr-check) refs/packed-backend.h: add missing declaration (hdr-check) refs/refs-internal.h: add missing declarations (hdr-check) midx.h: add missing forward declarations (hdr-check) delta-islands.h: add missing forward declarations (hdr-check)
headers: normalize the spelling of some header guards fetch-object.h: add missing declaration (hdr-check) ewok_rlw.h: add missing 'inline' to function definition commit-reach.h: add missing declarations (hdr-check)

Rasmus Villemoes (6):
help: redirect to aliased commands for "git cmd --help" git.c: handle_alias: prepend alias info when first argument is -h git-help.txt: document "git help cmd" vs "git cmd --help" for aliases Documentation/git-send-email.txt: style fixes
send-email: only consider lines containing @ or <> for automatic Cc'ing
send-email: also pick up cc addresses from -by trailers

René Scharfe (12):
mailinfo: support format=flowed
fsck: add a performance test for skipList
fsck: use strbuf_getline() to read skiplist file
fsck: use oidset instead of oid_array for skipList
sequencer: use return value of oidset_insert()
grep: add -r/--[no-]recursive
fetch-pack: factor out is_unmatched_ref()
fetch-pack: load tip_oids eagerly iff needed
khash: factor out kh_release_*
oidset: use khash
oidset: uninline oidset_init()
commit-reach: fix cast in compare_commits_by_gen()

Roger Strain (1):
subtree: performance improvement for finding unexpected parent commits

SZEDER Gábor (17):
t1404: increase core.packedRefsTimeout to avoid occasional test failure Documentation/git.txt: clarify that GIT_TRACE=/path appends
t3701-add-interactive: tighten the check of trace output
t1700-split-index: drop unnecessary 'grep'
t0090: disable GIT_TEST_SPLIT_INDEX for the test checking split index
t1700-split-index: document why FSMONITOR is disabled in this test script
split-index: add tests to demonstrate the racy split index problem
t1700-split-index: date back files to avoid racy situations
split-index: count the number of deleted entries
split-index: don't compare cached data of entries already marked for split index
split-index: smudge and add racily clean cache entries to split index
split-index: BUG() when cache entry refers to non-existing shared entry object_id.cocci: match only expressions of type 'struct object_id'
test-lib: introduce the '-V' short option for '--verbose-log'
travis-ci: install packages in 'ci/install-dependencies.sh'
coccicheck: introduce 'pending' semantic patches
ref-filter: don't look for objects when outside of a repository

Sam McKelvie (1):
rev-parse: --show-superproject-working-tree should work during a merge

Saulius Gurklys (1):
doc: fix small typo in git show-branch

Sebastian Staudt (1):
travis-ci: no longer use containers

Shulhan (1):
builtin/remote: quote remote name on error to display empty name

Stefan Beller (25):
git-submodule.sh: align error reporting for update mode to use path git-submodule.sh: rename unused variables builtin/submodule--helper: factor out submodule updating builtin/submodule--helper: store update_clone information in a struct builtin/submodule--helper: factor out method to update a single submodule
submodule--helper: replace connect-gitdir-workingtree by ensure-core-worktree
submodule--helper: introduce new update-module-mode helper
test_decode_color: understand FAINT and ITALIC
t3206: add color test for range-diff --dual-color
diff.c: simplify caller of emit_line_0
diff.c: reorder arguments for emit_line_ws_markup
diff.c: add set_sign to emit_line_0
diff: use emit_line_0 once per line
diff.c: omit check for line prefix in emit_line_0
diff.c: rewrite emit_line_0 more understandably
diff.c: add --output-indicator-{new, old, context}
range-diff: make use of different output indicators
range-diff: indent special lines as context
refs.c: migrate internal ref iteration to pass thru repository argument refs.c: upgrade for_each_replace_ref to be a each_repo_ref_fn callback
string-list: remove unused function print_string_list
strbuf.h: format according to coding guidelines
diff.c: pass sign_index to emit_line_ws_markup
submodule helper: convert relative URL to absolute URL if needed builtin/submodule--helper: remove debugging leftover tracing

Stephen P. Smith (10):
wt-status.c: move has_unmerged earlier in the file
wt-status: rename commitable to committable
t7501: add test of "commit --dry-run --short" wt-status.c: set the committable flag in the collect phase roll wt_status_state into wt_status and populate in the collect phase
t2000: rename and combine checkout clash tests
t7509: cleanup description and filename
t7502: rename commit test script to comply with naming convention
t7500: rename commit tests script to comply with naming convention
t7501: rename commit test to comply with naming convention

Steve Hoelzer (1):
poll: use GetTickCount64() to avoid wrap-around issues

Steven Fernandez (1):
git-completion.bash: add completion for stash list

Strain, Roger L (4):
subtree: refactor split of a commit into standalone method
subtree: make --ignore-joins pay attention to adds
subtree: use commits before rejoins for splits
subtree: improve decision on merges kept in split

Tao Qingyun (3):
refs: docstring typo
builtin/branch.c: remove useless branch_get
branch: trivial style fix

Taylor Blau (4):
transport.c: extract 'fill_alternate_refs_command' transport.c: introduce core.alternateRefsCommand transport.c: introduce core.alternateRefsPrefixes Documentation/config.txt: fix typo in core.alternateRefsCommand

Thomas Gummerer (17):
rerere: unify error messages when read_cache fails
rerere: lowercase error messages
rerere: wrap paths in output in sq
rerere: mark strings for translation
rerere: add documentation for conflict normalization
rerere: fix crash with files rerere can't handle
rerere: only return whether a path has conflicts or not
rerere: factor out handle_conflict function
rerere: return strbuf from handle path
rerere: teach rerere to handle nested conflicts
rerere: recalculate conflict ID when unresolved conflict is committed
rerere: mention caveat about unmatched conflict markers
rerere: add note about files with existing conflict markers .gitattributes: add conflict-marker-size for relevant files
linear-assignment: fix potential out of bounds memory access
t5551: move setup code inside test_expect blocks
t5551: compare sorted cookies files

Tim Schumacher (4):
Documentation/Makefile: make manpage-base-url.xsl generation quieter
alias: add support for aliases of an alias
alias: show the call history when an alias is looping
t0014: introduce an alias testing suite

Todd Zullinger (1):
Documentation: build technical/multi-pack-index

Torsten Bögershausen (4):
Make git_check_attr() a void function
path.c: char is not (always) signed
Upcast size_t variables to uintmax_t when printing remote-curl.c: xcurl_off_t is not portable (on 32 bit platfoms)

Uwe Kleine-König (1):

      howto/using-merge-subtree: mention --allow-unrelated-histories

brian m. carlson (26):
t: add test functions to translate hash-related values
t0000: use hash translation table
t0000: update tests for SHA-256
t0002: abstract away SHA-1 specific constants
t0064: make hash size independent
t1006: make hash size independent
t1400: switch hard-coded object ID to variable
t1405: make hash size independent
t1406: make hash-size independent
t1407: make hash size independent
editorconfig: provide editor settings for Git developers
editorconfig: indicate settings should be kept in sync
pack-bitmap-write: use GIT_MAX_RAWSZ for allocation builtin/repack: replace hard-coded constants builtin/mktree: remove hard-coded constant builtin/fetch-pack: remove constants with parse_oid_hex
pack-revindex: express constants in terms of the_hash_algo
packfile: express constants in terms of the_hash_algo refs/packed-backend: express constants using the_hash_algo
upload-pack: express constants in terms of the_hash_algo
transport: use parse_oid_hex instead of a constant
tag: express constant in terms of the_hash_algo
apply: replace hard-coded constants
apply: rename new_sha1_prefix and old_sha1_prefix
submodule: make zero-oid comparison hash function agnostic
rerere: convert to use the_hash_algo

Ævar Arnfjörð Bjarmason (33):
fetch: change "branch" to "reference" in --force -h output push tests: make use of unused $1 in test description push tests: use spaces in interpolated string
fetch tests: add a test for clobbering tag behavior
push doc: remove confusing mention of remote merger
push doc: move mention of "tag <tag>" later in the prose push doc: correct lies about how push refspecs work
fetch: document local ref updates with/without --force
fetch: stop clobbering existing tags without --force
fsck tests: setup of bogus commit object
fsck tests: add a test for no skipList input
fsck: document and test sorted skipList input
fsck: document and test commented & empty line skipList input
fsck: document that skipList input must be unabbreviated
fsck: add a performance test
fsck: support comments & empty lines in skipList commit-graph write: add progress output
commit-graph verify: add progress output
config doc: add missing list separator for checkout.optimizeNewBranch push doc: add spacing between two words
fetch doc: correct grammar in --force docs
gc: fix regression in 7b0f229222 impacting --quiet
gc doc: mention the commit-graph in the intro pack-objects test: modernize style
pack-objects tests: don't leave test .git corrupt at end index-pack tests: don't leave test repo dirty at end
i18n: make GETTEXT_POISON a runtime option
range-diff doc: add a section about output stability
range-diff: fix regression in passing along diff options
range-diff: make diff option behavior (e.g. --stat) consistent rebase doc: document rebase.useBuiltin
tests: add a special setup where rebase.useBuiltin is off
read-cache: make the split index obey umask settings

Đoàn Trần Công Danh (1):
git-compat-util: prefer poll.h to sys/poll.h

#45 lore raw

Re: [ANNOUNCE] Git v2.20.0-rc1

Quoted 4 lines · Junio C Hamano

* The "--no-patch" option, which can be used to get a high-level overview without the actual line-by-line patch difference shown, of the "range-diff" command was earlier broken, which has been corrected.

There's a regression related to this that I wanted to send a headsup for, but don't have time to fix today. Now range-diff in format-patch includes --stat output. See e.g. my
https://public-inbox.org/git/[email protected]/

Preliminary patch:

 builtin/log.c         | 3 +++
 t/t3206-range-diff.sh | 2 ++
 2 files changed, 5 insertions(+)
builtin/log.c +3 -0
index 0fe6f9ba1e..fdaba480d2 100644
@@ -1094,9 +1094,12 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
 	}
 
 	if (rev->rdiff1) {
+		const int oldfmt = rev->diffopt.output_format;
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
+		rev->diffopt.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
 				rev->creation_factor, 1, &rev->diffopt);
+		rev->diffopt.output_format = oldfmt;
 	}
 }
 
t/t3206-range-diff.sh +2 -0
index e497c1358f..2e913542f3 100755
@@ -248,8 +248,10 @@ test_expect_success 'dual-coloring' '
 for prev in topic master..topic
 do
 	test_expect_success "format-patch --range-diff=$prev" '
+		cat actual &&
 		git format-patch --stdout --cover-letter --range-diff=$prev \
 			master..unmodified >actual &&
+		! grep "a => b" actual &&
 		grep "= 1: .* s/5/A" actual &&
 		grep "= 2: .* s/4/A" actual &&
 		grep "= 3: .* s/11/B" actual &&
#46 lore raw

Re: [ANNOUNCE] Git v2.20.0-rc1

Quoted 4 lines · Ævar Arnfjörð Bjarmason

There's a regression related to this that I wanted to send a headsup for, but don't have time to fix today. Now range-diff in format-patch includes --stat output. See e.g. my
https://public-inbox.org/git/[email protected]/

Umf. Unfortunate fallout from [1].

builtin/log.c static void make_cover_letter(struct rev_info *rev, int use_stdout, · line 1102

Quoted 11 lines
builtin/log.c +3 -0
@@ -1094,9 +1094,12 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
        if (rev->rdiff1) {
+               const int oldfmt = rev->diffopt.output_format;
                fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
+               rev->diffopt.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
                show_range_diff(rev->rdiff1, rev->rdiff2,
                                rev->creation_factor, 1, &rev->diffopt);
+               rev->diffopt.output_format = oldfmt;
        }
 }

A few questions/observations:

Does this same "fix" need to be applied to the --interdiff case just above this --range-diff block?

Does the same "fix" need to be applied to the --interdiff and --range-diff cases in log-tree.c:show_log()?

Aside from fixing the broken --no-patches option[2], a goal of the series was to some day make --stat do something useful. Doesn't this "fix" go against that goal?

The way this change needs to be spread around at various locations is making it feel like a BandAid "fix" rather than a proper solution. It seems like it should be fixed at a different level, though I'm not sure yet if that level is higher (where the options get set) or lower (where they actually affect the operation).

Until we figure out the answers to these questions, I wonder if a more sensible short-term solution would be to back out [1] and just keep [2], which fixed the --no-patches regression.

t/t3206-range-diff.sh test_expect_success 'dual-coloring' ' · line 251

Quoted 6 lines
t/t3206-range-diff.sh +1 -0
@@ -248,8 +248,10 @@ test_expect_success 'dual-coloring' '
 for prev in topic master..topic
 do
        test_expect_success "format-patch --range-diff=$prev" '
+               cat actual &&

Debugging session gunk?

t/t3206-range-diff.sh test_expect_success 'dual-coloring' ' · line 255

Quoted 4 lines
t/t3206-range-diff.sh +1 -0
                git format-patch --stdout --cover-letter --range-diff=$prev \
                        master..unmodified >actual &&
+               ! grep "a => b" actual &&
                grep "= 1: .* s/5/A" actual &&

[1]: https://public-inbox.org/git/[email protected]/
[2]: https://public-inbox.org/git/[email protected]/

#47 lore raw

cover format-patch: pre-2.20 range-diff regression fix

[Dropping LKML & git-packagers from CC]

Quoted 23 lines · Eric Sunshine
Quoted 4 lines · Ævar Arnfjörð Bjarmason

There's a regression related to this that I wanted to send a headsup for, but don't have time to fix today. Now range-diff in format-patch includes --stat output. See e.g. my
https://public-inbox.org/git/[email protected]/

Umf. Unfortunate fallout from [1].

builtin/log.c static void make_cover_letter(struct rev_info *rev, int use_stdout, · line 1102

Quoted 11 lines
builtin/log.c +3 -0
@@ -1094,9 +1094,12 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
        if (rev->rdiff1) {
+               const int oldfmt = rev->diffopt.output_format;
                fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
+               rev->diffopt.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
                show_range_diff(rev->rdiff1, rev->rdiff2,
                                rev->creation_factor, 1, &rev->diffopt);
+               rev->diffopt.output_format = oldfmt;
        }
 }

A few questions/observations:

Quoted 5 lines

Does this same "fix" need to be applied to the --interdiff case just above this --range-diff block?

Does the same "fix" need to be applied to the --interdiff and --range-diff cases in log-tree.c:show_log()?

No, that seems to do the right thing, but perhaps tests are lacking there too. I haven't looked.

Quoted 3 lines

Aside from fixing the broken --no-patches option[2], a goal of the series was to some day make --stat do something useful. Doesn't this "fix" go against that goal?

The goal was to fix the regression introduced in 73a834e9e2 ("range-diff: relieve callers of low-level configuration burden", 2018-07-22). One aspect of having fixed that is we might in the future do stuff with --stat.

Quoted 9 lines

The way this change needs to be spread around at various locations is making it feel like a BandAid "fix" rather than a proper solution. It seems like it should be fixed at a different level, though I'm not sure yet if that level is higher (where the options get set) or lower (where they actually affect the operation).

Until we figure out the answers to these questions, I wonder if a more sensible short-term solution would be to back out [1] and just keep [2], which fixed the --no-patches regression.

I think that patch leaves range-diff itself in a good state without any bugs, and it would be a mistake to revert it.

But this interaction with format-patch --range-diff is another matter. As noted in 2/2 I think in practice this series sweeps the current bugs under the rug.

But as also noted there I think re-using what we get from setup_revisions() in format-patch for the range-diff was a mistake, and is always going to lead to some confusion. It should be split up so we can supply those diff options independently.

Quoted 3 lines

[...]
[1]: https://public-inbox.org/git/[email protected]/
[2]: https://public-inbox.org/git/[email protected]/

Ævar Arnfjörð Bjarmason (2):
format-patch: add a more exhaustive --range-diff test
format-patch: don't include --stat with --range-diff output

 builtin/log.c         |  7 ++++++-
 t/t3206-range-diff.sh | 15 ++++++++++-----
 2 files changed, 16 insertions(+), 6 deletions(-)
Signature · 1 line
2.20.0.rc0.387.gc7a69e6b6c
#48 lore raw

PATCH 1/2 format-patch: add a more exhaustive --range-diff test

In reply to Eric Sunshine

Change the narrow test added in 31e2617a5f ("format-patch: add --range-diff option to embed diff in cover letter", 2018-07-22) to test the full output. This test would have spotted a regression in the output if it wasn't beating around the bush and tested the full output, let's do that.

---
 t/t3206-range-diff.sh | 27 ++++++++++++++++++++++-----
 1 file changed, 22 insertions(+), 5 deletions(-)
t/t3206-range-diff.sh +22 -5
index e497c1358f..0235c038be 100755
@@ -249,11 +249,28 @@ for prev in topic master..topic
 do
 	test_expect_success "format-patch --range-diff=$prev" '
 		git format-patch --stdout --cover-letter --range-diff=$prev \
-			master..unmodified >actual &&
-		grep "= 1: .* s/5/A" actual &&
-		grep "= 2: .* s/4/A" actual &&
-		grep "= 3: .* s/11/B" actual &&
-		grep "= 4: .* s/12/B" actual
+			master..unmodified >actual.raw &&
+		sed -e "s|^:||" -e "s|:$||" >expect <<-\EOF &&
+		:1:  4de457d = 1:  35b9b25 s/5/A/
+		:     a => b | 0
+		:     1 file changed, 0 insertions(+), 0 deletions(-)
+		:    :
+		:2:  fccce22 = 2:  de345ab s/4/A/
+		:     a => b | 0
+		:     1 file changed, 0 insertions(+), 0 deletions(-)
+		:    :
+		:3:  147e64e = 3:  9af6654 s/11/B/
+		:     a => b | 0
+		:     1 file changed, 0 insertions(+), 0 deletions(-)
+		:    :
+		:4:  a63e992 = 4:  2901f77 s/12/B/
+		:     a => b | 0
+		:     1 file changed, 0 insertions(+), 0 deletions(-)
+		:    :
+		:-- :
+		EOF
+		sed -ne "/^1:/,/^--/p" <actual.raw >actual &&
+		test_cmp expect actual
 	'
 done
 
Signature · 1 line
2.20.0.rc0.387.gc7a69e6b6c
#49 lore raw

Re: [PATCH 1/2] format-patch: add a more exhaustive --range-diff test

Quoted 5 lines · Ævar Arnfjörð Bjarmason

Change the narrow test added in 31e2617a5f ("format-patch: add --range-diff option to embed diff in cover letter", 2018-07-22) to test the full output. This test would have spotted a regression in the output if it wasn't beating around the bush and tested the full output, let's do that.

This looks wrong, given that we are declaring that showing the broken --stat in the output is wrong. It makes us to expect a known-wrong output from the command.

If the theme of the topic were "what we are giving by default is a sensible output when --stat is asked for, but it is rather too noisy and our default should be quieter---let's tone it down", then this patch in 1/2 and then a behaviour-change patch with updated expectation would make sense, but I do not think that is what you are aiming for with this series.

Perhaps squash this into the real "fix" to show what the desired output should look like with the same patch?

Quoted 42 lines
---
 t/t3206-range-diff.sh | 27 ++++++++++++++++++++++-----
 1 file changed, 22 insertions(+), 5 deletions(-)
t/t3206-range-diff.sh +22 -5
index e497c1358f..0235c038be 100755
@@ -249,11 +249,28 @@ for prev in topic master..topic
 do
 	test_expect_success "format-patch --range-diff=$prev" '
 		git format-patch --stdout --cover-letter --range-diff=$prev \
-			master..unmodified >actual &&
-		grep "= 1: .* s/5/A" actual &&
-		grep "= 2: .* s/4/A" actual &&
-		grep "= 3: .* s/11/B" actual &&
-		grep "= 4: .* s/12/B" actual
+			master..unmodified >actual.raw &&
+		sed -e "s|^:||" -e "s|:$||" >expect <<-\EOF &&
+		:1:  4de457d = 1:  35b9b25 s/5/A/
+		:     a => b | 0
+		:     1 file changed, 0 insertions(+), 0 deletions(-)
+		:    :
+		:2:  fccce22 = 2:  de345ab s/4/A/
+		:     a => b | 0
+		:     1 file changed, 0 insertions(+), 0 deletions(-)
+		:    :
+		:3:  147e64e = 3:  9af6654 s/11/B/
+		:     a => b | 0
+		:     1 file changed, 0 insertions(+), 0 deletions(-)
+		:    :
+		:4:  a63e992 = 4:  2901f77 s/12/B/
+		:     a => b | 0
+		:     1 file changed, 0 insertions(+), 0 deletions(-)
+		:    :
+		:-- :
+		EOF
+		sed -ne "/^1:/,/^--/p" <actual.raw >actual &&
+		test_cmp expect actual
 	'
 done
#50 lore raw

Re: [PATCH 1/2] format-patch: add a more exhaustive --range-diff test

Quoted 19 lines · Junio C Hamano
Quoted 5 lines · Ævar Arnfjörð Bjarmason

Change the narrow test added in 31e2617a5f ("format-patch: add --range-diff option to embed diff in cover letter", 2018-07-22) to test the full output. This test would have spotted a regression in the output if it wasn't beating around the bush and tested the full output, let's do that.

This looks wrong, given that we are declaring that showing the broken --stat in the output is wrong. It makes us to expect a known-wrong output from the command.

If the theme of the topic were "what we are giving by default is a sensible output when --stat is asked for, but it is rather too noisy and our default should be quieter---let's tone it down", then this patch in 1/2 and then a behaviour-change patch with updated expectation would make sense, but I do not think that is what you are aiming for with this series.

Quoted 2 lines

Perhaps squash this into the real "fix" to show what the desired output should look like with the same patch?

I see you did that in 'pu'. I don't mind, looks good to me.

FWIW I split this up for ease of review, i.e. showing what the output was before and what effect the code change had, but maybe that was overdoing it and it's simpler just to have this all in one go.

Quoted 42 lines
Quoted 42 lines
---
 t/t3206-range-diff.sh | 27 ++++++++++++++++++++++-----
 1 file changed, 22 insertions(+), 5 deletions(-)
t/t3206-range-diff.sh +22 -5
index e497c1358f..0235c038be 100755
@@ -249,11 +249,28 @@ for prev in topic master..topic
 do
 	test_expect_success "format-patch --range-diff=$prev" '
 		git format-patch --stdout --cover-letter --range-diff=$prev \
-			master..unmodified >actual &&
-		grep "= 1: .* s/5/A" actual &&
-		grep "= 2: .* s/4/A" actual &&
-		grep "= 3: .* s/11/B" actual &&
-		grep "= 4: .* s/12/B" actual
+			master..unmodified >actual.raw &&
+		sed -e "s|^:||" -e "s|:$||" >expect <<-\EOF &&
+		:1:  4de457d = 1:  35b9b25 s/5/A/
+		:     a => b | 0
+		:     1 file changed, 0 insertions(+), 0 deletions(-)
+		:    :
+		:2:  fccce22 = 2:  de345ab s/4/A/
+		:     a => b | 0
+		:     1 file changed, 0 insertions(+), 0 deletions(-)
+		:    :
+		:3:  147e64e = 3:  9af6654 s/11/B/
+		:     a => b | 0
+		:     1 file changed, 0 insertions(+), 0 deletions(-)
+		:    :
+		:4:  a63e992 = 4:  2901f77 s/12/B/
+		:     a => b | 0
+		:     1 file changed, 0 insertions(+), 0 deletions(-)
+		:    :
+		:-- :
+		EOF
+		sed -ne "/^1:/,/^--/p" <actual.raw >actual &&
+		test_cmp expect actual
 	'
 done
#51 lore raw

PATCH 2/2 format-patch: don't include --stat with --range-diff output

In reply to Eric Sunshine

Fix a regression introduced in my a48e12ef7a ("range-diff: make diff option behavior (e.g. --stat) consistent", 2018-11-13). Since the format-patch setup code implicitly sets --stat --summary by default, we started emitting the --stat output in the cover letter's range-diff.

As noted in df569c3f31 ("range-diff doc: add a section about output stability", 2018-11-09) the --stat output is currently rather useless, and just adds noise.

Perhaps we should detect if --stat or --summary were implicitly passed to format-patch, and then pass them along, but I think fixing it this way is fine. If our --stat output ever becomes useful in range-diff we can revisit this.

There's still cases like e.g. --numstat triggering rather useless range-diff output, but I think it's OK to just handle the default case. Users are unlikely to produce a formatted patch with the likes of --numstat, or indeed any other custom diff option except -U<n> or maybe -W. If they need weirder combinations of options they can always manually produce the range-diff.

This whole situation comes about because we're assuming that when the user passes along e.g. -U10 that they want that some 10-line context for the range-diff as for the patches themselves. As noted in [1] I think it's worth re-visiting this and making -U10 just apply to the patches, and e.g. --range-diff-U10 to the range-diff. But that's left as a topic for another series less close to a rc2.

1. https://public-inbox.org/git/[email protected]/

---
 builtin/log.c         |  7 ++++++-
 t/t3206-range-diff.sh | 12 ------------
 2 files changed, 6 insertions(+), 13 deletions(-)
builtin/log.c +6 -1
index 0fe6f9ba1e..7cd2db0be9 100644
@@ -1094,9 +1094,13 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
 	}
 
 	if (rev->rdiff1) {
+		struct diff_options opts;
+		memcpy(&opts, &rev->diffopt, sizeof(opts));
+		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
+
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
-				rev->creation_factor, 1, &rev->diffopt);
+				rev->creation_factor, 1, &opts);
 	}
 }
 
@@ -1697,6 +1701,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	if (!use_patch_format &&
 		(!rev.diffopt.output_format ||
 		 rev.diffopt.output_format == DIFF_FORMAT_PATCH))
+		/* Needs to be mirrored in show_range_diff() invocation */
 		rev.diffopt.output_format = DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY;
 	if (!rev.diffopt.stat_width)
 		rev.diffopt.stat_width = MAIL_DEFAULT_WRAP;
t/t3206-range-diff.sh +0 -12
index 0235c038be..90def330bd 100755
@@ -252,21 +252,9 @@ do
 			master..unmodified >actual.raw &&
 		sed -e "s|^:||" -e "s|:$||" >expect <<-\EOF &&
 		:1:  4de457d = 1:  35b9b25 s/5/A/
-		:     a => b | 0
-		:     1 file changed, 0 insertions(+), 0 deletions(-)
-		:    :
 		:2:  fccce22 = 2:  de345ab s/4/A/
-		:     a => b | 0
-		:     1 file changed, 0 insertions(+), 0 deletions(-)
-		:    :
 		:3:  147e64e = 3:  9af6654 s/11/B/
-		:     a => b | 0
-		:     1 file changed, 0 insertions(+), 0 deletions(-)
-		:    :
 		:4:  a63e992 = 4:  2901f77 s/12/B/
-		:     a => b | 0
-		:     1 file changed, 0 insertions(+), 0 deletions(-)
-		:    :
 		:-- :
 		EOF
 		sed -ne "/^1:/,/^--/p" <actual.raw >actual &&
Signature · 1 line
2.20.0.rc0.387.gc7a69e6b6c
#52 lore raw

Re: [PATCH 2/2] format-patch: don't include --stat with --range-diff output

builtin/log.c static void make_cover_letter(struct rev_info *rev, int use_stdout, · line 1103

Quoted 9 lines · Ævar Arnfjörð Bjarmason
builtin/log.c +5 -1
 	if (rev->rdiff1) {
+		struct diff_options opts;
+		memcpy(&opts, &rev->diffopt, sizeof(opts));
+		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
+
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
-				rev->creation_factor, 1, &rev->diffopt);
+				rev->creation_factor, 1, &opts);

I am not quite convinced if this shallow copy is a safe thing to do. Quite honestly at this late in the release cycle, as a band-aid, I'd rather see a simpler revert than a change like this that we have to worry about what happens if/when show_range_diff() _thinks_ it is done with the opts and ends up discarding resources (e.g. "FILE *") that are shared with rev->diffopt that would still have to be used later.

#53 lore raw

Re: [PATCH 2/2] format-patch: don't include --stat with --range-diff output

Quoted 12 lines · Junio C Hamano

builtin/log.c static void make_cover_letter(struct rev_info *rev, int use_stdout, · line 1103

Quoted 9 lines · Ævar Arnfjörð Bjarmason
builtin/log.c +5 -1
 	if (rev->rdiff1) {
+		struct diff_options opts;
+		memcpy(&opts, &rev->diffopt, sizeof(opts));
+		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
+
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
-				rev->creation_factor, 1, &rev->diffopt);
+				rev->creation_factor, 1, &opts);
Quoted 7 lines

I am not quite convinced if this shallow copy is a safe thing to do. Quite honestly at this late in the release cycle, as a band-aid, I'd rather see a simpler revert than a change like this that we have to worry about what happens if/when show_range_diff() _thinks_ it is done with the opts and ends up discarding resources (e.g. "FILE *") that are shared with rev->diffopt that would still have to be used later.

Well, let's take it anyway as-is, at least for today, as I notice show_range_diff() itself does another shallow copy, so we are not making anything dramatically worse.

Thanks.

#54 lore raw

PATCH 2/2 format-patch: allow for independent diff & range-diff options

Change the semantics of the "--range-diff" option so that the regular diff options can be provided separately for the range-diff and the patch. This allows for supplying e.g. --range-diff-U0 and -U1 to "format-patch" to provide different context for the range-diff and the patch. This wasn't possible before.

Ever since the "--range-diff" option was added in 31e2617a5f ("format-patch: add --range-diff option to embed diff in cover letter", 2018-07-22) the "rev->diffopt" we pass down to the diff machinery has been the one we get from "format-patch"'s own setup_revisions().

This sort of thing is unique among the log-like commands in builtin/log.c, no command than format-patch will embed the output of another log-like command. Since the "rev->diffopt" is reused we need to munge it before we pass it to show_range_diff(). See 43dafc4172 ("format-patch: don't include --stat with --range-diff output", 2018-11-22) for a related regression fix which is being mostly reverted here.

Implementation notes: 1) We're not bothering with the full teardown around die() and will leak memory, but it's too much boilerplate to do all the frees with/without the die() and not worth it. 2) We call repo_init_revisions() for "rd_rev" even though we could get away with a shallow copy like the code we're replacing (and which show_range_diff() itself does). This is to make this code more easily understood.

---
 Documentation/git-format-patch.txt | 10 ++++++-
 builtin/log.c                      | 42 +++++++++++++++++++++++-------
 t/t3206-range-diff.sh              | 41 +++++++++++++++++++++++++++++
 3 files changed, 82 insertions(+), 11 deletions(-)
Documentation/git-format-patch.txt +9 -1
index aba4c5febe..6c048f415f 100644
@@ -24,7 +24,8 @@ SYNOPSIS
 		   [--to=<email>] [--cc=<email>]
 		   [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
 		   [--interdiff=<previous>]
-		   [--range-diff=<previous> [--creation-factor=<percent>]]
+		   [--range-diff=<previous> [--creation-factor=<percent>]
+		      [--range-diff<common diff option>]]
 		   [--progress]
 		   [<common diff options>]
 		   [ <since> | <revision range> ]
@@ -257,6 +258,13 @@ feeding the result to `git send-email`.
 	creation/deletion cost fudge factor. See linkgit:git-range-diff[1])
 	for details.
 
+--range-diff<common diff option>::
+	Other options prefixed with `--range-diff` are stripped of
+	that prefix and passed as-is to the diff machinery used to
+	generate the range-diff, e.g. `--range-diff-U0` and
+	`--range-diff--no-color`. This allows for adjusting the format
+	of the range-diff independently from the patch itself.
+
 --notes[=<ref>]::
 	Append the notes (see linkgit:git-notes[1]) for the commit
 	after the three-dash line.
builtin/log.c +32 -10

86 lines not on this page · raw message

t/t3206-range-diff.sh +41 -0

51 lines not on this page · raw message

Signature · 1 line
2.20.0.rc1.387.gf8505762e3
#55 lore raw

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

Documentation/git-format-patch.txt SYNOPSIS · line 28

Quoted 1 line · Ævar Arnfjörð Bjarmason
Documentation/git-format-patch.txt +1 -0
+		      [--range-diff<common diff option>]]

Let's make sure a random string thrown at this mechanism will properly get noticed and diagnosed.

Documentation/git-format-patch.txt feeding the result to `git send-email`. · line 266

Quoted 10 lines
Documentation/git-format-patch.txt +6 -0
@@ -257,6 +258,13 @@ feeding the result to `git send-email`.
 	creation/deletion cost fudge factor. See linkgit:git-range-diff[1])
 	for details.
 
+--range-diff<common diff option>::
+	Other options prefixed with `--range-diff` are stripped of
+	that prefix and passed as-is to the diff machinery used to
+	generate the range-diff, e.g. `--range-diff-U0` and
+	`--range-diff--no-color`. This allows for adjusting the format
+	of the range-diff independently from the patch itself.

Taking anything is of course the most general, but I am afraid if this backfires if there are some options that do not make sense to be different between the invocations of range-diff and format-patch.

builtin/log.c int cmd_format_patch(int argc, const char **argv, const char *prefix) · line 1693

Quoted 9 lines
builtin/log.c +3 -2
@@ -1689,8 +1688,32 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	rev.preserve_subject = keep_subject;
 
 	argc = setup_revisions(argc, argv, &rev, &s_r_opt);
-	if (argc > 1)
-		die(_("unrecognized argument: %s"), argv[1]);
+	if (argc > 1) {
+		struct argv_array args = ARGV_ARRAY_INIT;
+		const char *prefix = "--range-diff";

Please call that anything but "prefix" that hides the parameter to the function.

	const char *range_diff_opt = "--range-diff";

might work OK, or it might not. Let's read on.

Quoted 7 lines
builtin/log.c +7 -0
+		int have_prefix = 0;
+
+		for (i = 0; i < argc; i++) {
+			struct strbuf sb = STRBUF_INIT;
+			char *str;
+
+			strbuf_addstr(&sb, argv[i]);

builtin/log.c int cmd_format_patch(int argc, const char **argv, const char *prefix) · line 1710

Quoted 10 lines
builtin/log.c +10 -0
+			if (starts_with(argv[i], prefix)) {
+				have_prefix = 1;
+				strbuf_remove(&sb, 0, strlen(prefix));
+			}
+			str = strbuf_detach(&sb, NULL);
+			strbuf_release(&sb);
+
+			argv_array_push(&args, str);
+		}
+

Is the body of the loop essentially this?

			char *passopt = argv[i];
			if (!skip_prefix(passopt, range_diff_opt, &passopt))
				saw_range_diff_opt = 1;
			argv_array_push(&args, xstrdup(passopt));

We only use that "prefix" thing once, so we may not even need the variable.

Review int cmd_format_patch(int argc, const char **argv, const char *prefix) · line 1712

Quoted 2 lines
(quoted diff) +2 -0
+		if (!have_prefix)
+			die(_("unrecognized argument: %s"), argv[1]);

So we take normal options and check the leftover args; if there is no --range-diff<whatever> among the leftover bits, we pretend that we stumbled while reading the first such leftover arg.

builtin/log.c int cmd_format_patch(int argc, const char **argv, const char *prefix) · line 1716

Quoted 4 lines
builtin/log.c +4 -0
+		argc = setup_revisions(args.argc, args.argv, &rd_rev, NULL);
+		if (argc > 1)
+			die(_("unrecognized argument: %s"), argv[1]);
+	}

Otherwise, we pass all the leftover bits, which is a random mixture but guaranteed to have at least one meant for range-diff, to another setup_revisions(). If it leaves a leftover arg, then that is diagnosed here, so we'd be OK (iow, this is not a new "attack vector" to inject random string to command line parser).

One minor glitch I can see is "format-patch --range-diffSilly" would report "unrecognised arg: Silly". As we are pretending to be and reporting errors as format-patch, it would be better if we report that --range-diffSilly was what we did not understand.

Quoted 3 lines

Junio: I know it's late, but unless Eric has objections to this UI change I'd really like to have this in 2.20 since this is a change to a new command-line UI that's newly added in 2.20.

Quite honestly, I'd rather document "driving range-diff from format-patch is experimental and does silly things when given non-standard options in this release" and not touch the code at this late stage in the game. Would it be less intrusive a change to *not* support the --range-diff<whatever> option, still use rd_rev that is separate from the main rev, and use a reasonable hardcoded default settings when preparing rd_rev?

#56 lore raw

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

In reply to Ævar Arnfjörð Bjarmason

Hi Ævar,

Quoted 5 lines · Ævar Arnfjörð Bjarmason

Change the semantics of the "--range-diff" option so that the regular diff options can be provided separately for the range-diff and the patch. This allows for supplying e.g. --range-diff-U0 and -U1 to "format-patch" to provide different context for the range-diff and the patch. This wasn't possible before.

I really, really dislike the --range-diff-<random-thing>. We have precedent for passing optional arguments that are passed to some other command, so a much more logical and consistent convention would be to use --range-diff[=<diff-option>..], allowing all of the diff options that you might want to pass to the outer diff in one go rather than having a lengthy string of --range-diff-this and --range-diff-that options.

I only had time to skim the patch, and I have to wonder why you pass around full-blown rev_info structs for range diff (and with that really awful name rd_rev) rather than just the diff_options that you *actually* care about?

Ciao,
Dscho

Quoted 216 lines

Ever since the "--range-diff" option was added in 31e2617a5f ("format-patch: add --range-diff option to embed diff in cover letter", 2018-07-22) the "rev->diffopt" we pass down to the diff machinery has been the one we get from "format-patch"'s own setup_revisions().

This sort of thing is unique among the log-like commands in builtin/log.c, no command than format-patch will embed the output of another log-like command. Since the "rev->diffopt" is reused we need to munge it before we pass it to show_range_diff(). See 43dafc4172 ("format-patch: don't include --stat with --range-diff output", 2018-11-22) for a related regression fix which is being mostly reverted here.

Implementation notes: 1) We're not bothering with the full teardown around die() and will leak memory, but it's too much boilerplate to do all the frees with/without the die() and not worth it. 2) We call repo_init_revisions() for "rd_rev" even though we could get away with a shallow copy like the code we're replacing (and which show_range_diff() itself does). This is to make this code more easily understood.

---
 Documentation/git-format-patch.txt | 10 ++++++-
 builtin/log.c                      | 42 +++++++++++++++++++++++-------
 t/t3206-range-diff.sh              | 41 +++++++++++++++++++++++++++++
 3 files changed, 82 insertions(+), 11 deletions(-)
Documentation/git-format-patch.txt +9 -1
index aba4c5febe..6c048f415f 100644
@@ -24,7 +24,8 @@ SYNOPSIS
 		   [--to=<email>] [--cc=<email>]
 		   [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
 		   [--interdiff=<previous>]
-		   [--range-diff=<previous> [--creation-factor=<percent>]]
+		   [--range-diff=<previous> [--creation-factor=<percent>]
+		      [--range-diff<common diff option>]]
 		   [--progress]
 		   [<common diff options>]
 		   [ <since> | <revision range> ]
@@ -257,6 +258,13 @@ feeding the result to `git send-email`.
 	creation/deletion cost fudge factor. See linkgit:git-range-diff[1])
 	for details.
 
+--range-diff<common diff option>::
+	Other options prefixed with `--range-diff` are stripped of
+	that prefix and passed as-is to the diff machinery used to
+	generate the range-diff, e.g. `--range-diff-U0` and
+	`--range-diff--no-color`. This allows for adjusting the format
+	of the range-diff independently from the patch itself.
+
 --notes[=<ref>]::
 	Append the notes (see linkgit:git-notes[1]) for the commit
 	after the three-dash line.
builtin/log.c +32 -10
index 02d88fa233..7658e56ecc 100644
@@ -1023,7 +1023,8 @@ static void show_diffstat(struct rev_info *rev,
 	fprintf(rev->diffopt.file, "\n");
 }
 
-static void make_cover_letter(struct rev_info *rev, int use_stdout,
+static void make_cover_letter(struct rev_info *rev, struct rev_info *rd_rev,
+			      int use_stdout,
 			      struct commit *origin,
 			      int nr, struct commit **list,
 			      const char *branch_name,
@@ -1095,13 +1096,9 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
 	}
 
 	if (rev->rdiff1) {
-		struct diff_options opts;
-		memcpy(&opts, &rev->diffopt, sizeof(opts));
-		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
-
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
-				rev->creation_factor, 1, &opts);
+				rev->creation_factor, 1, &rd_rev->diffopt);
 	}
 }
 
@@ -1485,6 +1482,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	struct commit *commit;
 	struct commit **list = NULL;
 	struct rev_info rev;
+	struct rev_info rd_rev;
 	struct setup_revision_opt s_r_opt;
 	int nr = 0, total, i;
 	int use_stdout = 0;
@@ -1603,6 +1601,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	init_log_defaults();
 	git_config(git_format_config, NULL);
 	repo_init_revisions(the_repository, &rev, prefix);
+	repo_init_revisions(the_repository, &rd_rev, prefix);
 	rev.commit_format = CMIT_FMT_EMAIL;
 	rev.expand_tabs_in_log_default = 0;
 	rev.verbose_header = 1;
@@ -1689,8 +1688,32 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	rev.preserve_subject = keep_subject;
 
 	argc = setup_revisions(argc, argv, &rev, &s_r_opt);
-	if (argc > 1)
-		die(_("unrecognized argument: %s"), argv[1]);
+	if (argc > 1) {
+		struct argv_array args = ARGV_ARRAY_INIT;
+		const char *prefix = "--range-diff";
+		int have_prefix = 0;
+
+		for (i = 0; i < argc; i++) {
+			struct strbuf sb = STRBUF_INIT;
+			char *str;
+
+			strbuf_addstr(&sb, argv[i]);
+			if (starts_with(argv[i], prefix)) {
+				have_prefix = 1;
+				strbuf_remove(&sb, 0, strlen(prefix));
+			}
+			str = strbuf_detach(&sb, NULL);
+			strbuf_release(&sb);
+
+			argv_array_push(&args, str);
+		}
+
+		if (!have_prefix)
+			die(_("unrecognized argument: %s"), argv[1]);
+		argc = setup_revisions(args.argc, args.argv, &rd_rev, NULL);
+		if (argc > 1)
+			die(_("unrecognized argument: %s"), argv[1]);
+	}
 
 	if (rev.diffopt.output_format & DIFF_FORMAT_NAME)
 		die(_("--name-only does not make sense"));
@@ -1702,7 +1725,6 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	if (!use_patch_format &&
 		(!rev.diffopt.output_format ||
 		 rev.diffopt.output_format == DIFF_FORMAT_PATCH))
-		/* Needs to be mirrored in show_range_diff() invocation */
 		rev.diffopt.output_format = DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY;
 	if (!rev.diffopt.stat_width)
 		rev.diffopt.stat_width = MAIL_DEFAULT_WRAP;
@@ -1877,7 +1899,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	if (cover_letter) {
 		if (thread)
 			gen_message_id(&rev, "cover");
-		make_cover_letter(&rev, use_stdout,
+		make_cover_letter(&rev, &rd_rev, use_stdout,
 				  origin, nr, list, branch_name, quiet);
 		print_bases(&bases, rev.diffopt.file);
 		print_signature(rev.diffopt.file);
t/t3206-range-diff.sh +41 -0
index bc5facc1cd..6916103888 100755
@@ -308,6 +308,35 @@ test_expect_success 'format-patch with <common diff option>' '
 		--range-diff=topic~..topic changed~..changed >actual.raw &&
 	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
 	sed -e "s|:$||" >expect <<-\EOF &&
+	1:  a63e992 ! 1:  d966c5c s/12/B/
+	    @@ -8,7 +8,7 @@
+	     @@
+	      9
+	      10
+	    - B
+	    + BB
+	     -12
+	     +B
+	      13
+	-- :
+	EOF
+	test_cmp expect actual.range-diff &&
+	sed -ne "/^--- /,/^--/p" <actual.raw >actual.diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
+	--- a/file
+	+++ b/file
+	@@ -12 +12 @@ BB
+	-12
+	+B
+	-- :
+	EOF
+	test_cmp expect actual.diff &&
+
+	# -U0 & --range-diff-U0
+	git format-patch --cover-letter --stdout -U0 --range-diff-U0 \
+		--range-diff=topic~..topic changed~..changed >actual.raw &&
+	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
 	1:  a63e992 ! 1:  d966c5c s/12/B/
 	    @@ -11 +11 @@
 	    - B
@@ -327,4 +356,16 @@ test_expect_success 'format-patch with <common diff option>' '
 	test_cmp expect actual.diff
 '
 
+test_expect_success 'format-patch option parsing with --range-diff-*' '
+	test_must_fail git format-patch --stdout --unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --unknown" stderr &&
+	test_must_fail git format-patch --stdout --range-diff-unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --range-diff-unknown" stderr &&
+	test_must_fail git format-patch --stdout --unknown --range-diff-unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --unknown" stderr
+'
+
 test_done
Signature · 1 line
2.20.0.rc1.387.gf8505762e3
#57 lore raw

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

Quoted 10 lines · Johannes Schindelin

Hi Ævar,

Quoted 5 lines · Ævar Arnfjörð Bjarmason

Change the semantics of the "--range-diff" option so that the regular diff options can be provided separately for the range-diff and the patch. This allows for supplying e.g. --range-diff-U0 and -U1 to "format-patch" to provide different context for the range-diff and the patch. This wasn't possible before.

Quoted 6 lines

I really, really dislike the --range-diff-<random-thing>. We have precedent for passing optional arguments that are passed to some other command, so a much more logical and consistent convention would be to use --range-diff[=<diff-option>..], allowing all of the diff options that you might want to pass to the outer diff in one go rather than having a lengthy string of --range-diff-this and --range-diff-that options.

Where do we pass those sorts of arguments?

Reasons I did it this way:

a) Passing it as one option will require the user to double-quote those options that take quoted arguments (e.g. --word-diff-regex), which I thought sucked more than the prefix. On the implementation side we couldn't leave the parsing of the command-line to the shell anymore.

b) I think people will want to tweak this very rarely, much more rarely than e.g. -U10 in format-patch itself, so having something long-ish doesn't sound bad.

Quoted 4 lines

I only had time to skim the patch, and I have to wonder why you pass around full-blown rev_info structs for range diff (and with that really awful name rd_rev) rather than just the diff_options that you *actually* care about?

Because setup_revisions() which does all the command-line parsing needs a rev_info, so even if we only need the diffopt in the end we need to initiate the whole thing.

Suggestions for a better varibale name most welcome.

Quoted 219 lines

Ciao,
Dscho

Quoted 216 lines

Ever since the "--range-diff" option was added in 31e2617a5f ("format-patch: add --range-diff option to embed diff in cover letter", 2018-07-22) the "rev->diffopt" we pass down to the diff machinery has been the one we get from "format-patch"'s own setup_revisions().

This sort of thing is unique among the log-like commands in builtin/log.c, no command than format-patch will embed the output of another log-like command. Since the "rev->diffopt" is reused we need to munge it before we pass it to show_range_diff(). See 43dafc4172 ("format-patch: don't include --stat with --range-diff output", 2018-11-22) for a related regression fix which is being mostly reverted here.

Implementation notes: 1) We're not bothering with the full teardown around die() and will leak memory, but it's too much boilerplate to do all the frees with/without the die() and not worth it. 2) We call repo_init_revisions() for "rd_rev" even though we could get away with a shallow copy like the code we're replacing (and which show_range_diff() itself does). This is to make this code more easily understood.

---
 Documentation/git-format-patch.txt | 10 ++++++-
 builtin/log.c                      | 42 +++++++++++++++++++++++-------
 t/t3206-range-diff.sh              | 41 +++++++++++++++++++++++++++++
 3 files changed, 82 insertions(+), 11 deletions(-)
Documentation/git-format-patch.txt +9 -1
index aba4c5febe..6c048f415f 100644
@@ -24,7 +24,8 @@ SYNOPSIS
 		   [--to=<email>] [--cc=<email>]
 		   [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
 		   [--interdiff=<previous>]
-		   [--range-diff=<previous> [--creation-factor=<percent>]]
+		   [--range-diff=<previous> [--creation-factor=<percent>]
+		      [--range-diff<common diff option>]]
 		   [--progress]
 		   [<common diff options>]
 		   [ <since> | <revision range> ]
@@ -257,6 +258,13 @@ feeding the result to `git send-email`.
 	creation/deletion cost fudge factor. See linkgit:git-range-diff[1])
 	for details.
 
+--range-diff<common diff option>::
+	Other options prefixed with `--range-diff` are stripped of
+	that prefix and passed as-is to the diff machinery used to
+	generate the range-diff, e.g. `--range-diff-U0` and
+	`--range-diff--no-color`. This allows for adjusting the format
+	of the range-diff independently from the patch itself.
+
 --notes[=<ref>]::
 	Append the notes (see linkgit:git-notes[1]) for the commit
 	after the three-dash line.
builtin/log.c +32 -10
index 02d88fa233..7658e56ecc 100644
@@ -1023,7 +1023,8 @@ static void show_diffstat(struct rev_info *rev,
 	fprintf(rev->diffopt.file, "\n");
 }
 
-static void make_cover_letter(struct rev_info *rev, int use_stdout,
+static void make_cover_letter(struct rev_info *rev, struct rev_info *rd_rev,
+			      int use_stdout,
 			      struct commit *origin,
 			      int nr, struct commit **list,
 			      const char *branch_name,
@@ -1095,13 +1096,9 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
 	}
 
 	if (rev->rdiff1) {
-		struct diff_options opts;
-		memcpy(&opts, &rev->diffopt, sizeof(opts));
-		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
-
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
-				rev->creation_factor, 1, &opts);
+				rev->creation_factor, 1, &rd_rev->diffopt);
 	}
 }
 
@@ -1485,6 +1482,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	struct commit *commit;
 	struct commit **list = NULL;
 	struct rev_info rev;
+	struct rev_info rd_rev;
 	struct setup_revision_opt s_r_opt;
 	int nr = 0, total, i;
 	int use_stdout = 0;
@@ -1603,6 +1601,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	init_log_defaults();
 	git_config(git_format_config, NULL);
 	repo_init_revisions(the_repository, &rev, prefix);
+	repo_init_revisions(the_repository, &rd_rev, prefix);
 	rev.commit_format = CMIT_FMT_EMAIL;
 	rev.expand_tabs_in_log_default = 0;
 	rev.verbose_header = 1;
@@ -1689,8 +1688,32 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	rev.preserve_subject = keep_subject;
 
 	argc = setup_revisions(argc, argv, &rev, &s_r_opt);
-	if (argc > 1)
-		die(_("unrecognized argument: %s"), argv[1]);
+	if (argc > 1) {
+		struct argv_array args = ARGV_ARRAY_INIT;
+		const char *prefix = "--range-diff";
+		int have_prefix = 0;
+
+		for (i = 0; i < argc; i++) {
+			struct strbuf sb = STRBUF_INIT;
+			char *str;
+
+			strbuf_addstr(&sb, argv[i]);
+			if (starts_with(argv[i], prefix)) {
+				have_prefix = 1;
+				strbuf_remove(&sb, 0, strlen(prefix));
+			}
+			str = strbuf_detach(&sb, NULL);
+			strbuf_release(&sb);
+
+			argv_array_push(&args, str);
+		}
+
+		if (!have_prefix)
+			die(_("unrecognized argument: %s"), argv[1]);
+		argc = setup_revisions(args.argc, args.argv, &rd_rev, NULL);
+		if (argc > 1)
+			die(_("unrecognized argument: %s"), argv[1]);
+	}
 
 	if (rev.diffopt.output_format & DIFF_FORMAT_NAME)
 		die(_("--name-only does not make sense"));
@@ -1702,7 +1725,6 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	if (!use_patch_format &&
 		(!rev.diffopt.output_format ||
 		 rev.diffopt.output_format == DIFF_FORMAT_PATCH))
-		/* Needs to be mirrored in show_range_diff() invocation */
 		rev.diffopt.output_format = DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY;
 	if (!rev.diffopt.stat_width)
 		rev.diffopt.stat_width = MAIL_DEFAULT_WRAP;
@@ -1877,7 +1899,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	if (cover_letter) {
 		if (thread)
 			gen_message_id(&rev, "cover");
-		make_cover_letter(&rev, use_stdout,
+		make_cover_letter(&rev, &rd_rev, use_stdout,
 				  origin, nr, list, branch_name, quiet);
 		print_bases(&bases, rev.diffopt.file);
 		print_signature(rev.diffopt.file);
t/t3206-range-diff.sh +41 -1
index bc5facc1cd..6916103888 100755
@@ -308,6 +308,35 @@ test_expect_success 'format-patch with <common diff option>' '
 		--range-diff=topic~..topic changed~..changed >actual.raw &&
 	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
 	sed -e "s|:$||" >expect <<-\EOF &&
+	1:  a63e992 ! 1:  d966c5c s/12/B/
+	    @@ -8,7 +8,7 @@
+	     @@
+	      9
+	      10
+	    - B
+	    + BB
+	     -12
+	     +B
+	      13
+	-- :
+	EOF
+	test_cmp expect actual.range-diff &&
+	sed -ne "/^--- /,/^--/p" <actual.raw >actual.diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
+	--- a/file
+	+++ b/file
+	@@ -12 +12 @@ BB
+	-12
+	+B
+	-- :
+	EOF
+	test_cmp expect actual.diff &&
+
+	# -U0 & --range-diff-U0
+	git format-patch --cover-letter --stdout -U0 --range-diff-U0 \
+		--range-diff=topic~..topic changed~..changed >actual.raw &&
+	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
 	1:  a63e992 ! 1:  d966c5c s/12/B/
 	    @@ -11 +11 @@
 	    - B
@@ -327,4 +356,16 @@ test_expect_success 'format-patch with <common diff option>' '
 	test_cmp expect actual.diff
 '
 
+test_expect_success 'format-patch option parsing with --range-diff-*' '
+	test_must_fail git format-patch --stdout --unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --unknown" stderr &&
+	test_must_fail git format-patch --stdout --range-diff-unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --range-diff-unknown" stderr &&
+	test_must_fail git format-patch --stdout --unknown --range-diff-unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --unknown" stderr
+'
+
 test_done
--

2.20.0.rc1.387.gf8505762e3

#58 lore raw

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

Hi Ævar,

Quoted 21 lines · Ævar Arnfjörð Bjarmason
Quoted 8 lines · Johannes Schindelin
Quoted 5 lines · Ævar Arnfjörð Bjarmason

Change the semantics of the "--range-diff" option so that the regular diff options can be provided separately for the range-diff and the patch. This allows for supplying e.g. --range-diff-U0 and -U1 to "format-patch" to provide different context for the range-diff and the patch. This wasn't possible before.

Quoted 6 lines

I really, really dislike the --range-diff-<random-thing>. We have precedent for passing optional arguments that are passed to some other command, so a much more logical and consistent convention would be to use --range-diff[=<diff-option>..], allowing all of the diff options that you might want to pass to the outer diff in one go rather than having a lengthy string of --range-diff-this and --range-diff-that options.

Where do we pass those sorts of arguments?

Reasons I did it this way:

Quoted 8 lines

a) Passing it as one option will require the user to double-quote those options that take quoted arguments (e.g. --word-diff-regex), which I thought sucked more than the prefix. On the implementation side we couldn't leave the parsing of the command-line to the shell anymore.

b) I think people will want to tweak this very rarely, much more rarely than e.g. -U10 in format-patch itself, so having something long-ish doesn't sound bad.

Hmm. I still don't like it. It sets a precedent, and we simply do not do it that way in other circumstances (most obvious would be the -X merge options). The more divergent user interfaces for the same sort of thing are, the more brain cycles you force users to spend on navigating said interfaces.

Quoted 10 lines
Quoted 4 lines

I only had time to skim the patch, and I have to wonder why you pass around full-blown rev_info structs for range diff (and with that really awful name rd_rev) rather than just the diff_options that you *actually* care about?

Because setup_revisions() which does all the command-line parsing needs a rev_info, so even if we only need the diffopt in the end we need to initiate the whole thing.

Suggestions for a better varibale name most welcome.

range_diff_revs

And you do not need to pass around the whole thing. You can easily pass &range_diff_revs.diffopt.

Don't pass around what you do not need to pass around.

Ciao,
Dscho

Quoted 221 lines
Quoted 220 lines

Ciao,
Dscho

Quoted 216 lines

Ever since the "--range-diff" option was added in 31e2617a5f ("format-patch: add --range-diff option to embed diff in cover letter", 2018-07-22) the "rev->diffopt" we pass down to the diff machinery has been the one we get from "format-patch"'s own setup_revisions().

This sort of thing is unique among the log-like commands in builtin/log.c, no command than format-patch will embed the output of another log-like command. Since the "rev->diffopt" is reused we need to munge it before we pass it to show_range_diff(). See 43dafc4172 ("format-patch: don't include --stat with --range-diff output", 2018-11-22) for a related regression fix which is being mostly reverted here.

Implementation notes: 1) We're not bothering with the full teardown around die() and will leak memory, but it's too much boilerplate to do all the frees with/without the die() and not worth it. 2) We call repo_init_revisions() for "rd_rev" even though we could get away with a shallow copy like the code we're replacing (and which show_range_diff() itself does). This is to make this code more easily understood.

---
 Documentation/git-format-patch.txt | 10 ++++++-
 builtin/log.c                      | 42 +++++++++++++++++++++++-------
 t/t3206-range-diff.sh              | 41 +++++++++++++++++++++++++++++
 3 files changed, 82 insertions(+), 11 deletions(-)
Documentation/git-format-patch.txt +9 -1
index aba4c5febe..6c048f415f 100644
@@ -24,7 +24,8 @@ SYNOPSIS
 		   [--to=<email>] [--cc=<email>]
 		   [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
 		   [--interdiff=<previous>]
-		   [--range-diff=<previous> [--creation-factor=<percent>]]
+		   [--range-diff=<previous> [--creation-factor=<percent>]
+		      [--range-diff<common diff option>]]
 		   [--progress]
 		   [<common diff options>]
 		   [ <since> | <revision range> ]
@@ -257,6 +258,13 @@ feeding the result to `git send-email`.
 	creation/deletion cost fudge factor. See linkgit:git-range-diff[1])
 	for details.
 
+--range-diff<common diff option>::
+	Other options prefixed with `--range-diff` are stripped of
+	that prefix and passed as-is to the diff machinery used to
+	generate the range-diff, e.g. `--range-diff-U0` and
+	`--range-diff--no-color`. This allows for adjusting the format
+	of the range-diff independently from the patch itself.
+
 --notes[=<ref>]::
 	Append the notes (see linkgit:git-notes[1]) for the commit
 	after the three-dash line.
builtin/log.c +32 -10
index 02d88fa233..7658e56ecc 100644
@@ -1023,7 +1023,8 @@ static void show_diffstat(struct rev_info *rev,
 	fprintf(rev->diffopt.file, "\n");
 }
 
-static void make_cover_letter(struct rev_info *rev, int use_stdout,
+static void make_cover_letter(struct rev_info *rev, struct rev_info *rd_rev,
+			      int use_stdout,
 			      struct commit *origin,
 			      int nr, struct commit **list,
 			      const char *branch_name,
@@ -1095,13 +1096,9 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
 	}
 
 	if (rev->rdiff1) {
-		struct diff_options opts;
-		memcpy(&opts, &rev->diffopt, sizeof(opts));
-		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
-
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
-				rev->creation_factor, 1, &opts);
+				rev->creation_factor, 1, &rd_rev->diffopt);
 	}
 }
 
@@ -1485,6 +1482,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	struct commit *commit;
 	struct commit **list = NULL;
 	struct rev_info rev;
+	struct rev_info rd_rev;
 	struct setup_revision_opt s_r_opt;
 	int nr = 0, total, i;
 	int use_stdout = 0;
@@ -1603,6 +1601,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	init_log_defaults();
 	git_config(git_format_config, NULL);
 	repo_init_revisions(the_repository, &rev, prefix);
+	repo_init_revisions(the_repository, &rd_rev, prefix);
 	rev.commit_format = CMIT_FMT_EMAIL;
 	rev.expand_tabs_in_log_default = 0;
 	rev.verbose_header = 1;
@@ -1689,8 +1688,32 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	rev.preserve_subject = keep_subject;
 
 	argc = setup_revisions(argc, argv, &rev, &s_r_opt);
-	if (argc > 1)
-		die(_("unrecognized argument: %s"), argv[1]);
+	if (argc > 1) {
+		struct argv_array args = ARGV_ARRAY_INIT;
+		const char *prefix = "--range-diff";
+		int have_prefix = 0;
+
+		for (i = 0; i < argc; i++) {
+			struct strbuf sb = STRBUF_INIT;
+			char *str;
+
+			strbuf_addstr(&sb, argv[i]);
+			if (starts_with(argv[i], prefix)) {
+				have_prefix = 1;
+				strbuf_remove(&sb, 0, strlen(prefix));
+			}
+			str = strbuf_detach(&sb, NULL);
+			strbuf_release(&sb);
+
+			argv_array_push(&args, str);
+		}
+
+		if (!have_prefix)
+			die(_("unrecognized argument: %s"), argv[1]);
+		argc = setup_revisions(args.argc, args.argv, &rd_rev, NULL);
+		if (argc > 1)
+			die(_("unrecognized argument: %s"), argv[1]);
+	}
 
 	if (rev.diffopt.output_format & DIFF_FORMAT_NAME)
 		die(_("--name-only does not make sense"));
@@ -1702,7 +1725,6 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	if (!use_patch_format &&
 		(!rev.diffopt.output_format ||
 		 rev.diffopt.output_format == DIFF_FORMAT_PATCH))
-		/* Needs to be mirrored in show_range_diff() invocation */
 		rev.diffopt.output_format = DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY;
 	if (!rev.diffopt.stat_width)
 		rev.diffopt.stat_width = MAIL_DEFAULT_WRAP;
@@ -1877,7 +1899,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	if (cover_letter) {
 		if (thread)
 			gen_message_id(&rev, "cover");
-		make_cover_letter(&rev, use_stdout,
+		make_cover_letter(&rev, &rd_rev, use_stdout,
 				  origin, nr, list, branch_name, quiet);
 		print_bases(&bases, rev.diffopt.file);
 		print_signature(rev.diffopt.file);
t/t3206-range-diff.sh +41 -1
index bc5facc1cd..6916103888 100755
@@ -308,6 +308,35 @@ test_expect_success 'format-patch with <common diff option>' '
 		--range-diff=topic~..topic changed~..changed >actual.raw &&
 	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
 	sed -e "s|:$||" >expect <<-\EOF &&
+	1:  a63e992 ! 1:  d966c5c s/12/B/
+	    @@ -8,7 +8,7 @@
+	     @@
+	      9
+	      10
+	    - B
+	    + BB
+	     -12
+	     +B
+	      13
+	-- :
+	EOF
+	test_cmp expect actual.range-diff &&
+	sed -ne "/^--- /,/^--/p" <actual.raw >actual.diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
+	--- a/file
+	+++ b/file
+	@@ -12 +12 @@ BB
+	-12
+	+B
+	-- :
+	EOF
+	test_cmp expect actual.diff &&
+
+	# -U0 & --range-diff-U0
+	git format-patch --cover-letter --stdout -U0 --range-diff-U0 \
+		--range-diff=topic~..topic changed~..changed >actual.raw &&
+	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
 	1:  a63e992 ! 1:  d966c5c s/12/B/
 	    @@ -11 +11 @@
 	    - B
@@ -327,4 +356,16 @@ test_expect_success 'format-patch with <common diff option>' '
 	test_cmp expect actual.diff
 '
 
+test_expect_success 'format-patch option parsing with --range-diff-*' '
+	test_must_fail git format-patch --stdout --unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --unknown" stderr &&
+	test_must_fail git format-patch --stdout --range-diff-unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --range-diff-unknown" stderr &&
+	test_must_fail git format-patch --stdout --unknown --range-diff-unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --unknown" stderr
+'
+
 test_done
--

2.20.0.rc1.387.gf8505762e3

#59 lore raw

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

Quoted 34 lines · Johannes Schindelin

Hi Ævar,

Quoted 21 lines · Ævar Arnfjörð Bjarmason
Quoted 8 lines · Johannes Schindelin
Quoted 5 lines · Ævar Arnfjörð Bjarmason

Change the semantics of the "--range-diff" option so that the regular diff options can be provided separately for the range-diff and the patch. This allows for supplying e.g. --range-diff-U0 and -U1 to "format-patch" to provide different context for the range-diff and the patch. This wasn't possible before.

Quoted 6 lines

I really, really dislike the --range-diff-<random-thing>. We have precedent for passing optional arguments that are passed to some other command, so a much more logical and consistent convention would be to use --range-diff[=<diff-option>..], allowing all of the diff options that you might want to pass to the outer diff in one go rather than having a lengthy string of --range-diff-this and --range-diff-that options.

Where do we pass those sorts of arguments?

Reasons I did it this way:

Quoted 8 lines

a) Passing it as one option will require the user to double-quote those options that take quoted arguments (e.g. --word-diff-regex), which I thought sucked more than the prefix. On the implementation side we couldn't leave the parsing of the command-line to the shell anymore.

b) I think people will want to tweak this very rarely, much more rarely than e.g. -U10 in format-patch itself, so having something long-ish doesn't sound bad.

Quoted 5 lines

Hmm. I still don't like it. It sets a precedent, and we simply do not do it that way in other circumstances (most obvious would be the -X merge options). The more divergent user interfaces for the same sort of thing are, the more brain cycles you force users to spend on navigating said interfaces.

Yeah it sucks, I just think it sucks less than the alternative :) I.e. I'm not picky about --range-diff-* prefix the name, but I think doing our own shell parsing would be nasty.

Quoted 9 lines
Quoted 9 lines
Quoted 4 lines

I only had time to skim the patch, and I have to wonder why you pass around full-blown rev_info structs for range diff (and with that really awful name rd_rev) rather than just the diff_options that you *actually* care about?

Because setup_revisions() which does all the command-line parsing needs a rev_info, so even if we only need the diffopt in the end we need to initiate the whole thing.

Quoted 8 lines
Quoted 1 line

Suggestions for a better varibale name most welcome.

range_diff_revs

And you do not need to pass around the whole thing. You can easily pass &range_diff_revs.diffopt.

Don't pass around what you do not need to pass around.

Ah, you mean internally in log.c, yes that makes sense. I thought you meant just pass diffopt to setup_revisions() (which needs the containing struct). Willdo.

Quoted 224 lines

Ciao,
Dscho

Quoted 221 lines
Quoted 220 lines

Ciao,
Dscho

Quoted 216 lines

Ever since the "--range-diff" option was added in 31e2617a5f ("format-patch: add --range-diff option to embed diff in cover letter", 2018-07-22) the "rev->diffopt" we pass down to the diff machinery has been the one we get from "format-patch"'s own setup_revisions().

This sort of thing is unique among the log-like commands in builtin/log.c, no command than format-patch will embed the output of another log-like command. Since the "rev->diffopt" is reused we need to munge it before we pass it to show_range_diff(). See 43dafc4172 ("format-patch: don't include --stat with --range-diff output", 2018-11-22) for a related regression fix which is being mostly reverted here.

Implementation notes: 1) We're not bothering with the full teardown around die() and will leak memory, but it's too much boilerplate to do all the frees with/without the die() and not worth it. 2) We call repo_init_revisions() for "rd_rev" even though we could get away with a shallow copy like the code we're replacing (and which show_range_diff() itself does). This is to make this code more easily understood.

---
 Documentation/git-format-patch.txt | 10 ++++++-
 builtin/log.c                      | 42 +++++++++++++++++++++++-------
 t/t3206-range-diff.sh              | 41 +++++++++++++++++++++++++++++
 3 files changed, 82 insertions(+), 11 deletions(-)
Documentation/git-format-patch.txt +9 -1
index aba4c5febe..6c048f415f 100644
@@ -24,7 +24,8 @@ SYNOPSIS
 		   [--to=<email>] [--cc=<email>]
 		   [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
 		   [--interdiff=<previous>]
-		   [--range-diff=<previous> [--creation-factor=<percent>]]
+		   [--range-diff=<previous> [--creation-factor=<percent>]
+		      [--range-diff<common diff option>]]
 		   [--progress]
 		   [<common diff options>]
 		   [ <since> | <revision range> ]
@@ -257,6 +258,13 @@ feeding the result to `git send-email`.
 	creation/deletion cost fudge factor. See linkgit:git-range-diff[1])
 	for details.
 
+--range-diff<common diff option>::
+	Other options prefixed with `--range-diff` are stripped of
+	that prefix and passed as-is to the diff machinery used to
+	generate the range-diff, e.g. `--range-diff-U0` and
+	`--range-diff--no-color`. This allows for adjusting the format
+	of the range-diff independently from the patch itself.
+
 --notes[=<ref>]::
 	Append the notes (see linkgit:git-notes[1]) for the commit
 	after the three-dash line.
builtin/log.c +32 -10
index 02d88fa233..7658e56ecc 100644
@@ -1023,7 +1023,8 @@ static void show_diffstat(struct rev_info *rev,
 	fprintf(rev->diffopt.file, "\n");
 }
 
-static void make_cover_letter(struct rev_info *rev, int use_stdout,
+static void make_cover_letter(struct rev_info *rev, struct rev_info *rd_rev,
+			      int use_stdout,
 			      struct commit *origin,
 			      int nr, struct commit **list,
 			      const char *branch_name,
@@ -1095,13 +1096,9 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
 	}
 
 	if (rev->rdiff1) {
-		struct diff_options opts;
-		memcpy(&opts, &rev->diffopt, sizeof(opts));
-		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
-
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
-				rev->creation_factor, 1, &opts);
+				rev->creation_factor, 1, &rd_rev->diffopt);
 	}
 }
 
@@ -1485,6 +1482,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	struct commit *commit;
 	struct commit **list = NULL;
 	struct rev_info rev;
+	struct rev_info rd_rev;
 	struct setup_revision_opt s_r_opt;
 	int nr = 0, total, i;
 	int use_stdout = 0;
@@ -1603,6 +1601,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	init_log_defaults();
 	git_config(git_format_config, NULL);
 	repo_init_revisions(the_repository, &rev, prefix);
+	repo_init_revisions(the_repository, &rd_rev, prefix);
 	rev.commit_format = CMIT_FMT_EMAIL;
 	rev.expand_tabs_in_log_default = 0;
 	rev.verbose_header = 1;
@@ -1689,8 +1688,32 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	rev.preserve_subject = keep_subject;
 
 	argc = setup_revisions(argc, argv, &rev, &s_r_opt);
-	if (argc > 1)
-		die(_("unrecognized argument: %s"), argv[1]);
+	if (argc > 1) {
+		struct argv_array args = ARGV_ARRAY_INIT;
+		const char *prefix = "--range-diff";
+		int have_prefix = 0;
+
+		for (i = 0; i < argc; i++) {
+			struct strbuf sb = STRBUF_INIT;
+			char *str;
+
+			strbuf_addstr(&sb, argv[i]);
+			if (starts_with(argv[i], prefix)) {
+				have_prefix = 1;
+				strbuf_remove(&sb, 0, strlen(prefix));
+			}
+			str = strbuf_detach(&sb, NULL);
+			strbuf_release(&sb);
+
+			argv_array_push(&args, str);
+		}
+
+		if (!have_prefix)
+			die(_("unrecognized argument: %s"), argv[1]);
+		argc = setup_revisions(args.argc, args.argv, &rd_rev, NULL);
+		if (argc > 1)
+			die(_("unrecognized argument: %s"), argv[1]);
+	}
 
 	if (rev.diffopt.output_format & DIFF_FORMAT_NAME)
 		die(_("--name-only does not make sense"));
@@ -1702,7 +1725,6 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	if (!use_patch_format &&
 		(!rev.diffopt.output_format ||
 		 rev.diffopt.output_format == DIFF_FORMAT_PATCH))
-		/* Needs to be mirrored in show_range_diff() invocation */
 		rev.diffopt.output_format = DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY;
 	if (!rev.diffopt.stat_width)
 		rev.diffopt.stat_width = MAIL_DEFAULT_WRAP;
@@ -1877,7 +1899,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	if (cover_letter) {
 		if (thread)
 			gen_message_id(&rev, "cover");
-		make_cover_letter(&rev, use_stdout,
+		make_cover_letter(&rev, &rd_rev, use_stdout,
 				  origin, nr, list, branch_name, quiet);
 		print_bases(&bases, rev.diffopt.file);
 		print_signature(rev.diffopt.file);
t/t3206-range-diff.sh +41 -1
index bc5facc1cd..6916103888 100755
@@ -308,6 +308,35 @@ test_expect_success 'format-patch with <common diff option>' '
 		--range-diff=topic~..topic changed~..changed >actual.raw &&
 	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
 	sed -e "s|:$||" >expect <<-\EOF &&
+	1:  a63e992 ! 1:  d966c5c s/12/B/
+	    @@ -8,7 +8,7 @@
+	     @@
+	      9
+	      10
+	    - B
+	    + BB
+	     -12
+	     +B
+	      13
+	-- :
+	EOF
+	test_cmp expect actual.range-diff &&
+	sed -ne "/^--- /,/^--/p" <actual.raw >actual.diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
+	--- a/file
+	+++ b/file
+	@@ -12 +12 @@ BB
+	-12
+	+B
+	-- :
+	EOF
+	test_cmp expect actual.diff &&
+
+	# -U0 & --range-diff-U0
+	git format-patch --cover-letter --stdout -U0 --range-diff-U0 \
+		--range-diff=topic~..topic changed~..changed >actual.raw &&
+	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
 	1:  a63e992 ! 1:  d966c5c s/12/B/
 	    @@ -11 +11 @@
 	    - B
@@ -327,4 +356,16 @@ test_expect_success 'format-patch with <common diff option>' '
 	test_cmp expect actual.diff
 '
 
+test_expect_success 'format-patch option parsing with --range-diff-*' '
+	test_must_fail git format-patch --stdout --unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --unknown" stderr &&
+	test_must_fail git format-patch --stdout --range-diff-unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --range-diff-unknown" stderr &&
+	test_must_fail git format-patch --stdout --unknown --range-diff-unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --unknown" stderr
+'
+
 test_done
--

2.20.0.rc1.387.gf8505762e3

#60 lore raw

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

Hi Ævar,

Quoted 40 lines · Ævar Arnfjörð Bjarmason
Quoted 32 lines · Johannes Schindelin
Quoted 21 lines · Ævar Arnfjörð Bjarmason
Quoted 8 lines · Johannes Schindelin
Quoted 5 lines · Ævar Arnfjörð Bjarmason

Change the semantics of the "--range-diff" option so that the regular diff options can be provided separately for the range-diff and the patch. This allows for supplying e.g. --range-diff-U0 and -U1 to "format-patch" to provide different context for the range-diff and the patch. This wasn't possible before.

Quoted 6 lines

I really, really dislike the --range-diff-<random-thing>. We have precedent for passing optional arguments that are passed to some other command, so a much more logical and consistent convention would be to use --range-diff[=<diff-option>..], allowing all of the diff options that you might want to pass to the outer diff in one go rather than having a lengthy string of --range-diff-this and --range-diff-that options.

Where do we pass those sorts of arguments?

Reasons I did it this way:

Quoted 8 lines

a) Passing it as one option will require the user to double-quote those options that take quoted arguments (e.g. --word-diff-regex), which I thought sucked more than the prefix. On the implementation side we couldn't leave the parsing of the command-line to the shell anymore.

b) I think people will want to tweak this very rarely, much more rarely than e.g. -U10 in format-patch itself, so having something long-ish doesn't sound bad.

Quoted 5 lines

Hmm. I still don't like it. It sets a precedent, and we simply do not do it that way in other circumstances (most obvious would be the -X merge options). The more divergent user interfaces for the same sort of thing are, the more brain cycles you force users to spend on navigating said interfaces.

Quoted 3 lines

Yeah it sucks, I just think it sucks less than the alternative :) I.e. I'm not picky about --range-diff-* prefix the name, but I think doing our own shell parsing would be nasty.

What prevents you from using sq_dequote_to_argv()?

Quoted 13 lines
Quoted 13 lines
Quoted 10 lines
Quoted 4 lines

I only had time to skim the patch, and I have to wonder why you pass around full-blown rev_info structs for range diff (and with that really awful name rd_rev) rather than just the diff_options that you *actually* care about?

Because setup_revisions() which does all the command-line parsing needs a rev_info, so even if we only need the diffopt in the end we need to initiate the whole thing.

Suggestions for a better varibale name most welcome.

range_diff_revs

Quoted 8 lines
Quoted 4 lines

And you do not need to pass around the whole thing. You can easily pass &range_diff_revs.diffopt.

Don't pass around what you do not need to pass around.

Ah, you mean internally in log.c, yes that makes sense. I thought you meant just pass diffopt to setup_revisions() (which needs the containing struct). Willdo.

Thanks,
Dscho

Quoted 226 lines
Quoted 225 lines

Ciao,
Dscho

Quoted 221 lines
Quoted 220 lines

Ciao,
Dscho

Quoted 216 lines

Ever since the "--range-diff" option was added in 31e2617a5f ("format-patch: add --range-diff option to embed diff in cover letter", 2018-07-22) the "rev->diffopt" we pass down to the diff machinery has been the one we get from "format-patch"'s own setup_revisions().

This sort of thing is unique among the log-like commands in builtin/log.c, no command than format-patch will embed the output of another log-like command. Since the "rev->diffopt" is reused we need to munge it before we pass it to show_range_diff(). See 43dafc4172 ("format-patch: don't include --stat with --range-diff output", 2018-11-22) for a related regression fix which is being mostly reverted here.

Implementation notes: 1) We're not bothering with the full teardown around die() and will leak memory, but it's too much boilerplate to do all the frees with/without the die() and not worth it. 2) We call repo_init_revisions() for "rd_rev" even though we could get away with a shallow copy like the code we're replacing (and which show_range_diff() itself does). This is to make this code more easily understood.

---
 Documentation/git-format-patch.txt | 10 ++++++-
 builtin/log.c                      | 42 +++++++++++++++++++++++-------
 t/t3206-range-diff.sh              | 41 +++++++++++++++++++++++++++++
 3 files changed, 82 insertions(+), 11 deletions(-)
Documentation/git-format-patch.txt +9 -1
index aba4c5febe..6c048f415f 100644
@@ -24,7 +24,8 @@ SYNOPSIS
 		   [--to=<email>] [--cc=<email>]
 		   [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
 		   [--interdiff=<previous>]
-		   [--range-diff=<previous> [--creation-factor=<percent>]]
+		   [--range-diff=<previous> [--creation-factor=<percent>]
+		      [--range-diff<common diff option>]]
 		   [--progress]
 		   [<common diff options>]
 		   [ <since> | <revision range> ]
@@ -257,6 +258,13 @@ feeding the result to `git send-email`.
 	creation/deletion cost fudge factor. See linkgit:git-range-diff[1])
 	for details.
 
+--range-diff<common diff option>::
+	Other options prefixed with `--range-diff` are stripped of
+	that prefix and passed as-is to the diff machinery used to
+	generate the range-diff, e.g. `--range-diff-U0` and
+	`--range-diff--no-color`. This allows for adjusting the format
+	of the range-diff independently from the patch itself.
+
 --notes[=<ref>]::
 	Append the notes (see linkgit:git-notes[1]) for the commit
 	after the three-dash line.
builtin/log.c +32 -10
index 02d88fa233..7658e56ecc 100644
@@ -1023,7 +1023,8 @@ static void show_diffstat(struct rev_info *rev,
 	fprintf(rev->diffopt.file, "\n");
 }
 
-static void make_cover_letter(struct rev_info *rev, int use_stdout,
+static void make_cover_letter(struct rev_info *rev, struct rev_info *rd_rev,
+			      int use_stdout,
 			      struct commit *origin,
 			      int nr, struct commit **list,
 			      const char *branch_name,
@@ -1095,13 +1096,9 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
 	}
 
 	if (rev->rdiff1) {
-		struct diff_options opts;
-		memcpy(&opts, &rev->diffopt, sizeof(opts));
-		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
-
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
-				rev->creation_factor, 1, &opts);
+				rev->creation_factor, 1, &rd_rev->diffopt);
 	}
 }
 
@@ -1485,6 +1482,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	struct commit *commit;
 	struct commit **list = NULL;
 	struct rev_info rev;
+	struct rev_info rd_rev;
 	struct setup_revision_opt s_r_opt;
 	int nr = 0, total, i;
 	int use_stdout = 0;
@@ -1603,6 +1601,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	init_log_defaults();
 	git_config(git_format_config, NULL);
 	repo_init_revisions(the_repository, &rev, prefix);
+	repo_init_revisions(the_repository, &rd_rev, prefix);
 	rev.commit_format = CMIT_FMT_EMAIL;
 	rev.expand_tabs_in_log_default = 0;
 	rev.verbose_header = 1;
@@ -1689,8 +1688,32 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	rev.preserve_subject = keep_subject;
 
 	argc = setup_revisions(argc, argv, &rev, &s_r_opt);
-	if (argc > 1)
-		die(_("unrecognized argument: %s"), argv[1]);
+	if (argc > 1) {
+		struct argv_array args = ARGV_ARRAY_INIT;
+		const char *prefix = "--range-diff";
+		int have_prefix = 0;
+
+		for (i = 0; i < argc; i++) {
+			struct strbuf sb = STRBUF_INIT;
+			char *str;
+
+			strbuf_addstr(&sb, argv[i]);
+			if (starts_with(argv[i], prefix)) {
+				have_prefix = 1;
+				strbuf_remove(&sb, 0, strlen(prefix));
+			}
+			str = strbuf_detach(&sb, NULL);
+			strbuf_release(&sb);
+
+			argv_array_push(&args, str);
+		}
+
+		if (!have_prefix)
+			die(_("unrecognized argument: %s"), argv[1]);
+		argc = setup_revisions(args.argc, args.argv, &rd_rev, NULL);
+		if (argc > 1)
+			die(_("unrecognized argument: %s"), argv[1]);
+	}
 
 	if (rev.diffopt.output_format & DIFF_FORMAT_NAME)
 		die(_("--name-only does not make sense"));
@@ -1702,7 +1725,6 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	if (!use_patch_format &&
 		(!rev.diffopt.output_format ||
 		 rev.diffopt.output_format == DIFF_FORMAT_PATCH))
-		/* Needs to be mirrored in show_range_diff() invocation */
 		rev.diffopt.output_format = DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY;
 	if (!rev.diffopt.stat_width)
 		rev.diffopt.stat_width = MAIL_DEFAULT_WRAP;
@@ -1877,7 +1899,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	if (cover_letter) {
 		if (thread)
 			gen_message_id(&rev, "cover");
-		make_cover_letter(&rev, use_stdout,
+		make_cover_letter(&rev, &rd_rev, use_stdout,
 				  origin, nr, list, branch_name, quiet);
 		print_bases(&bases, rev.diffopt.file);
 		print_signature(rev.diffopt.file);
t/t3206-range-diff.sh +41 -1
index bc5facc1cd..6916103888 100755
@@ -308,6 +308,35 @@ test_expect_success 'format-patch with <common diff option>' '
 		--range-diff=topic~..topic changed~..changed >actual.raw &&
 	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
 	sed -e "s|:$||" >expect <<-\EOF &&
+	1:  a63e992 ! 1:  d966c5c s/12/B/
+	    @@ -8,7 +8,7 @@
+	     @@
+	      9
+	      10
+	    - B
+	    + BB
+	     -12
+	     +B
+	      13
+	-- :
+	EOF
+	test_cmp expect actual.range-diff &&
+	sed -ne "/^--- /,/^--/p" <actual.raw >actual.diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
+	--- a/file
+	+++ b/file
+	@@ -12 +12 @@ BB
+	-12
+	+B
+	-- :
+	EOF
+	test_cmp expect actual.diff &&
+
+	# -U0 & --range-diff-U0
+	git format-patch --cover-letter --stdout -U0 --range-diff-U0 \
+		--range-diff=topic~..topic changed~..changed >actual.raw &&
+	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
 	1:  a63e992 ! 1:  d966c5c s/12/B/
 	    @@ -11 +11 @@
 	    - B
@@ -327,4 +356,16 @@ test_expect_success 'format-patch with <common diff option>' '
 	test_cmp expect actual.diff
 '
 
+test_expect_success 'format-patch option parsing with --range-diff-*' '
+	test_must_fail git format-patch --stdout --unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --unknown" stderr &&
+	test_must_fail git format-patch --stdout --range-diff-unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --range-diff-unknown" stderr &&
+	test_must_fail git format-patch --stdout --unknown --range-diff-unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --unknown" stderr
+'
+
 test_done
--

2.20.0.rc1.387.gf8505762e3

#61 lore raw

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

Quoted 44 lines · Johannes Schindelin

Hi Ævar,

Quoted 40 lines · Ævar Arnfjörð Bjarmason
Quoted 37 lines · Johannes Schindelin
Quoted 21 lines · Ævar Arnfjörð Bjarmason
Quoted 8 lines · Johannes Schindelin
Quoted 5 lines · Ævar Arnfjörð Bjarmason

Change the semantics of the "--range-diff" option so that the regular diff options can be provided separately for the range-diff and the patch. This allows for supplying e.g. --range-diff-U0 and -U1 to "format-patch" to provide different context for the range-diff and the patch. This wasn't possible before.

Quoted 6 lines

I really, really dislike the --range-diff-<random-thing>. We have precedent for passing optional arguments that are passed to some other command, so a much more logical and consistent convention would be to use --range-diff[=<diff-option>..], allowing all of the diff options that you might want to pass to the outer diff in one go rather than having a lengthy string of --range-diff-this and --range-diff-that options.

Where do we pass those sorts of arguments?

Reasons I did it this way:

Quoted 8 lines

a) Passing it as one option will require the user to double-quote those options that take quoted arguments (e.g. --word-diff-regex), which I thought sucked more than the prefix. On the implementation side we couldn't leave the parsing of the command-line to the shell anymore.

b) I think people will want to tweak this very rarely, much more rarely than e.g. -U10 in format-patch itself, so having something long-ish doesn't sound bad.

Hmm. I still don't like it. It sets a precedent, and we simply do not do it that way in other circumstances (most obvious would be the -X merge options). The more divergent user interfaces for the same sort of thing are, the more brain cycles you force users to spend on navigating said interfaces.

Quoted 5 lines
Quoted 3 lines

Yeah it sucks, I just think it sucks less than the alternative :) I.e. I'm not picky about --range-diff-* prefix the name, but I think doing our own shell parsing would be nasty.

What prevents you from using sq_dequote_to_argv()?

I mean not just nasty in terms of implementation, yeah we could do it, but also a nasty UX for things like --word-diff-regex. I.e. instead of:

    --range-diff-word-diff-regex='[0-9"]'

You need:

    --range-diff-opts="--word-diff-regex='[0-9\"]'"

Now admittedly that in itself isn't very painful *in this case*, but in terms of precedent I really dislike that option, i.e. git having some mode where I need to work to escape input to pass to another command.

Not saying that this --range-diff-* thing is what we should go for, but surely we can find some way to do deal with this that doesn't involve the user needing to escape stuff like this.

It also has other downstream effects in the UI, e.g. it's presumably easy to teach the bash completion that a --foo=XYZ option is also called --some-prefix--foo=XYZ and to enable completion for that, less so for making it smart enough to complete "--some-prefix-opts="--foo=<TAB>".

Quoted 251 lines
Quoted 13 lines
Quoted 13 lines
Quoted 10 lines
Quoted 4 lines

I only had time to skim the patch, and I have to wonder why you pass around full-blown rev_info structs for range diff (and with that really awful name rd_rev) rather than just the diff_options that you *actually* care about?

Because setup_revisions() which does all the command-line parsing needs a rev_info, so even if we only need the diffopt in the end we need to initiate the whole thing.

Suggestions for a better varibale name most welcome.

range_diff_revs

Quoted 8 lines
Quoted 4 lines

And you do not need to pass around the whole thing. You can easily pass &range_diff_revs.diffopt.

Don't pass around what you do not need to pass around.

Ah, you mean internally in log.c, yes that makes sense. I thought you meant just pass diffopt to setup_revisions() (which needs the containing struct). Willdo.

Thanks,
Dscho

Quoted 226 lines
Quoted 225 lines

Ciao,
Dscho

Quoted 221 lines
Quoted 220 lines

Ciao,
Dscho

Quoted 216 lines

Ever since the "--range-diff" option was added in 31e2617a5f ("format-patch: add --range-diff option to embed diff in cover letter", 2018-07-22) the "rev->diffopt" we pass down to the diff machinery has been the one we get from "format-patch"'s own setup_revisions().

This sort of thing is unique among the log-like commands in builtin/log.c, no command than format-patch will embed the output of another log-like command. Since the "rev->diffopt" is reused we need to munge it before we pass it to show_range_diff(). See 43dafc4172 ("format-patch: don't include --stat with --range-diff output", 2018-11-22) for a related regression fix which is being mostly reverted here.

Implementation notes: 1) We're not bothering with the full teardown around die() and will leak memory, but it's too much boilerplate to do all the frees with/without the die() and not worth it. 2) We call repo_init_revisions() for "rd_rev" even though we could get away with a shallow copy like the code we're replacing (and which show_range_diff() itself does). This is to make this code more easily understood.

---
 Documentation/git-format-patch.txt | 10 ++++++-
 builtin/log.c                      | 42 +++++++++++++++++++++++-------
 t/t3206-range-diff.sh              | 41 +++++++++++++++++++++++++++++
 3 files changed, 82 insertions(+), 11 deletions(-)
Documentation/git-format-patch.txt +9 -1
index aba4c5febe..6c048f415f 100644
@@ -24,7 +24,8 @@ SYNOPSIS
 		   [--to=<email>] [--cc=<email>]
 		   [--[no-]cover-letter] [--quiet] [--notes[=<ref>]]
 		   [--interdiff=<previous>]
-		   [--range-diff=<previous> [--creation-factor=<percent>]]
+		   [--range-diff=<previous> [--creation-factor=<percent>]
+		      [--range-diff<common diff option>]]
 		   [--progress]
 		   [<common diff options>]
 		   [ <since> | <revision range> ]
@@ -257,6 +258,13 @@ feeding the result to `git send-email`.
 	creation/deletion cost fudge factor. See linkgit:git-range-diff[1])
 	for details.
 
+--range-diff<common diff option>::
+	Other options prefixed with `--range-diff` are stripped of
+	that prefix and passed as-is to the diff machinery used to
+	generate the range-diff, e.g. `--range-diff-U0` and
+	`--range-diff--no-color`. This allows for adjusting the format
+	of the range-diff independently from the patch itself.
+
 --notes[=<ref>]::
 	Append the notes (see linkgit:git-notes[1]) for the commit
 	after the three-dash line.
builtin/log.c +32 -10
index 02d88fa233..7658e56ecc 100644
@@ -1023,7 +1023,8 @@ static void show_diffstat(struct rev_info *rev,
 	fprintf(rev->diffopt.file, "\n");
 }
 
-static void make_cover_letter(struct rev_info *rev, int use_stdout,
+static void make_cover_letter(struct rev_info *rev, struct rev_info *rd_rev,
+			      int use_stdout,
 			      struct commit *origin,
 			      int nr, struct commit **list,
 			      const char *branch_name,
@@ -1095,13 +1096,9 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
 	}
 
 	if (rev->rdiff1) {
-		struct diff_options opts;
-		memcpy(&opts, &rev->diffopt, sizeof(opts));
-		opts.output_format &= ~(DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY);
-
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
-				rev->creation_factor, 1, &opts);
+				rev->creation_factor, 1, &rd_rev->diffopt);
 	}
 }
 
@@ -1485,6 +1482,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	struct commit *commit;
 	struct commit **list = NULL;
 	struct rev_info rev;
+	struct rev_info rd_rev;
 	struct setup_revision_opt s_r_opt;
 	int nr = 0, total, i;
 	int use_stdout = 0;
@@ -1603,6 +1601,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	init_log_defaults();
 	git_config(git_format_config, NULL);
 	repo_init_revisions(the_repository, &rev, prefix);
+	repo_init_revisions(the_repository, &rd_rev, prefix);
 	rev.commit_format = CMIT_FMT_EMAIL;
 	rev.expand_tabs_in_log_default = 0;
 	rev.verbose_header = 1;
@@ -1689,8 +1688,32 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	rev.preserve_subject = keep_subject;
 
 	argc = setup_revisions(argc, argv, &rev, &s_r_opt);
-	if (argc > 1)
-		die(_("unrecognized argument: %s"), argv[1]);
+	if (argc > 1) {
+		struct argv_array args = ARGV_ARRAY_INIT;
+		const char *prefix = "--range-diff";
+		int have_prefix = 0;
+
+		for (i = 0; i < argc; i++) {
+			struct strbuf sb = STRBUF_INIT;
+			char *str;
+
+			strbuf_addstr(&sb, argv[i]);
+			if (starts_with(argv[i], prefix)) {
+				have_prefix = 1;
+				strbuf_remove(&sb, 0, strlen(prefix));
+			}
+			str = strbuf_detach(&sb, NULL);
+			strbuf_release(&sb);
+
+			argv_array_push(&args, str);
+		}
+
+		if (!have_prefix)
+			die(_("unrecognized argument: %s"), argv[1]);
+		argc = setup_revisions(args.argc, args.argv, &rd_rev, NULL);
+		if (argc > 1)
+			die(_("unrecognized argument: %s"), argv[1]);
+	}
 
 	if (rev.diffopt.output_format & DIFF_FORMAT_NAME)
 		die(_("--name-only does not make sense"));
@@ -1702,7 +1725,6 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	if (!use_patch_format &&
 		(!rev.diffopt.output_format ||
 		 rev.diffopt.output_format == DIFF_FORMAT_PATCH))
-		/* Needs to be mirrored in show_range_diff() invocation */
 		rev.diffopt.output_format = DIFF_FORMAT_DIFFSTAT | DIFF_FORMAT_SUMMARY;
 	if (!rev.diffopt.stat_width)
 		rev.diffopt.stat_width = MAIL_DEFAULT_WRAP;
@@ -1877,7 +1899,7 @@ int cmd_format_patch(int argc, const char **argv, const char *prefix)
 	if (cover_letter) {
 		if (thread)
 			gen_message_id(&rev, "cover");
-		make_cover_letter(&rev, use_stdout,
+		make_cover_letter(&rev, &rd_rev, use_stdout,
 				  origin, nr, list, branch_name, quiet);
 		print_bases(&bases, rev.diffopt.file);
 		print_signature(rev.diffopt.file);
t/t3206-range-diff.sh +41 -1
index bc5facc1cd..6916103888 100755
@@ -308,6 +308,35 @@ test_expect_success 'format-patch with <common diff option>' '
 		--range-diff=topic~..topic changed~..changed >actual.raw &&
 	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
 	sed -e "s|:$||" >expect <<-\EOF &&
+	1:  a63e992 ! 1:  d966c5c s/12/B/
+	    @@ -8,7 +8,7 @@
+	     @@
+	      9
+	      10
+	    - B
+	    + BB
+	     -12
+	     +B
+	      13
+	-- :
+	EOF
+	test_cmp expect actual.range-diff &&
+	sed -ne "/^--- /,/^--/p" <actual.raw >actual.diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
+	--- a/file
+	+++ b/file
+	@@ -12 +12 @@ BB
+	-12
+	+B
+	-- :
+	EOF
+	test_cmp expect actual.diff &&
+
+	# -U0 & --range-diff-U0
+	git format-patch --cover-letter --stdout -U0 --range-diff-U0 \
+		--range-diff=topic~..topic changed~..changed >actual.raw &&
+	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
 	1:  a63e992 ! 1:  d966c5c s/12/B/
 	    @@ -11 +11 @@
 	    - B
@@ -327,4 +356,16 @@ test_expect_success 'format-patch with <common diff option>' '
 	test_cmp expect actual.diff
 '
 
+test_expect_success 'format-patch option parsing with --range-diff-*' '
+	test_must_fail git format-patch --stdout --unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --unknown" stderr &&
+	test_must_fail git format-patch --stdout --range-diff-unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --range-diff-unknown" stderr &&
+	test_must_fail git format-patch --stdout --unknown --range-diff-unknown \
+		master..unmodified 2>stderr &&
+	test_i18ngrep "unrecognized argument: --unknown" stderr
+'
+
 test_done
--

2.20.0.rc1.387.gf8505762e3

#62 lore raw

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

Hi Ævar,

Quoted 50 lines · Ævar Arnfjörð Bjarmason
Quoted 42 lines · Johannes Schindelin
Quoted 40 lines · Ævar Arnfjörð Bjarmason
Quoted 37 lines · Johannes Schindelin
Quoted 21 lines · Ævar Arnfjörð Bjarmason
Quoted 8 lines · Johannes Schindelin
Quoted 5 lines · Ævar Arnfjörð Bjarmason

Change the semantics of the "--range-diff" option so that the regular diff options can be provided separately for the range-diff and the patch. This allows for supplying e.g. --range-diff-U0 and -U1 to "format-patch" to provide different context for the range-diff and the patch. This wasn't possible before.

Quoted 6 lines

I really, really dislike the --range-diff-<random-thing>. We have precedent for passing optional arguments that are passed to some other command, so a much more logical and consistent convention would be to use --range-diff[=<diff-option>..], allowing all of the diff options that you might want to pass to the outer diff in one go rather than having a lengthy string of --range-diff-this and --range-diff-that options.

Where do we pass those sorts of arguments?

Reasons I did it this way:

Quoted 8 lines

a) Passing it as one option will require the user to double-quote those options that take quoted arguments (e.g. --word-diff-regex), which I thought sucked more than the prefix. On the implementation side we couldn't leave the parsing of the command-line to the shell anymore.

b) I think people will want to tweak this very rarely, much more rarely than e.g. -U10 in format-patch itself, so having something long-ish doesn't sound bad.

Hmm. I still don't like it. It sets a precedent, and we simply do not do it that way in other circumstances (most obvious would be the -X merge options). The more divergent user interfaces for the same sort of thing are, the more brain cycles you force users to spend on navigating said interfaces.

Quoted 5 lines
Quoted 3 lines

Yeah it sucks, I just think it sucks less than the alternative :) I.e. I'm not picky about --range-diff-* prefix the name, but I think doing our own shell parsing would be nasty.

What prevents you from using sq_dequote_to_argv()?

Quoted 8 lines

I mean not just nasty in terms of implementation, yeah we could do it, but also a nasty UX for things like --word-diff-regex. I.e. instead of:

    --range-diff-word-diff-regex='[0-9"]'

You need:

    --range-diff-opts="--word-diff-regex='[0-9\"]'"

Really? I think that would not work. It would pass the single quotes as part of the regex to the diff machinery.

Or maybe not. But the extra quotes do not strike me as necessary, as there is no shell script involved (thank deity!) after git range-diff parsed the options.

Quoted 4 lines

Now admittedly that in itself isn't very painful *in this case*, but in terms of precedent I really dislike that option, i.e. git having some mode where I need to work to escape input to pass to another command.

Quoted 8 lines

Not saying that this --range-diff-* thing is what we should go for, but surely we can find some way to do deal with this that doesn't involve the user needing to escape stuff like this.

It also has other downstream effects in the UI, e.g. it's presumably easy to teach the bash completion that a --foo=XYZ option is also called --some-prefix--foo=XYZ and to enable completion for that, less so for making it smart enough to complete "--some-prefix-opts="--foo=<TAB>".

These are all good points, and need proper discussion.

Sadly, all that time needed for a proper discussion is not left before v2.20.0 is supposed to come out.

Quite honestly, I think what we will have to do is to describe in the documentation of format-patch's --range-diff option that the exact user interface how to pass diff options down to range-diff is in flux and not final.

That way, we can give your design the proper treatment, and work together on making a user interface we all can be happy with.

Ciao,
Dscho

#63 lore raw

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

In reply to Ævar Arnfjörð Bjarmason
Quoted 7 lines · Ævar Arnfjörð Bjarmason
Quoted 1 line

What prevents you from using sq_dequote_to_argv()?

I mean not just nasty in terms of implementation, yeah we could do it, but also a nasty UX for things like --word-diff-regex. I.e. instead of:

    --range-diff-word-diff-regex='[0-9"]'
Quoted 7 lines

You need:

    --range-diff-opts="--word-diff-regex='[0-9\"]'"

Now admittedly that in itself isn't very painful *in this case*, but in terms of precedent I really dislike that option, i.e. git having some mode where I need to work to escape input to pass to another command.

In addition, sq_dequote are meant to be used on quoted string we internally produce; I do not think we want to promise that it is safe to use on a random string that comes from end users.

In any case, I tend to agree with the conclusion in the downthread by Dscho that we should just clearly mark that invocations of the "format-patch --range-diff" command with additional diff options is an experimental feature that may not do anything sensible in the upcoming release, and declare that the UI to pass diff options to affect only the range-diff part may later be invented. IOW, I am coming a bit stronger than Dscho's suggestion in that we should not even pretend that we aimed to make the options used for range-diff customizable when driven from format-patch in the upcoming release, or aimed to make --range-diff option compatible with other diff options given to the format-patch command.

I had to delay -rc2 to see these last minute tweaks come to some reasonable place to stop at, and I do not think we want to delay the final any longer or destablizing it further by piling last minute undercooked changes on top.

#64 lore raw

PATCH 1/1 format-patch: do not let its diff-options affect --range-diff (was Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options)

Quoted 12 lines · Junio C Hamano

In any case, I tend to agree with the conclusion in the downthread by Dscho that we should just clearly mark that invocations of the "format-patch --range-diff" command with additional diff options is an experimental feature that may not do anything sensible in the upcoming release, and declare that the UI to pass diff options to affect only the range-diff part may later be invented. IOW, I am coming a bit stronger than Dscho's suggestion in that we should not even pretend that we aimed to make the options used for range-diff customizable when driven from format-patch in the upcoming release, or aimed to make --range-diff option compatible with other diff options given to the format-patch command.

Quoted 4 lines

I had to delay -rc2 to see these last minute tweaks come to some reasonable place to stop at, and I do not think we want to delay the final any longer or destablizing it further by piling last minute undercooked changes on top.

So how about doing this on top of 'master' instead? As this leaks *no* information wrt how range-diff machinery should behave from the format-patch side by not passing any diffopt, as long as the new code I added to show_range_diff() comes up with a reasonable default diffopts (for which I really would appreciate extra sets of eyes to make sure), this change by definition cannot be wrong (famous last words).

-- >8 --
Subject: format-patch: do not let its diff-options affect --range-diff

Stop leaking how the primary output of format-patch is customized to the range-diff machinery and instead let the latter use its own "reasonable default", in order to correct the breakage introduced by a5170794 ("Merge branch 'ab/range-diff-no-patch'", 2018-11-18) on the 'master' front. "git format-patch --range-diff..." without any weird diff option started to include the "range-diff --stat" output, which is rather useless right now, that made the whole thing unusable and this is probably the least disruptive way to whip the codebase into a shippable shape.

We may want to later make the range-diff driven by format-patch more configurable, but that would have to wait until we have a good design.

---
 Documentation/git-format-patch.txt | 5 +++++
 builtin/log.c                      | 2 +-
 log-tree.c                         | 2 +-
 range-diff.c                       | 6 +++++-
 range-diff.h                       | 5 +++++
 5 files changed, 17 insertions(+), 3 deletions(-)
Documentation/git-format-patch.txt +5 -0
index aba4c5febe..27304428a1 100644
@@ -250,6 +250,11 @@ feeding the result to `git send-email`.
 	feature/v2`), or a revision range if the two versions of the series are
 	disjoint (for example `git format-patch --cover-letter
 	--range-diff=feature/v1~3..feature/v1 -3 feature/v2`).
++
+Note that diff options passed to the command affect how the primary
+product of `format-patch` is generated, and they are not passed to
+the underlying `range-diff` machinery used to generate the cover-letter
+material (this may change in the future).
 
 --creation-factor=<percent>::
 	Used with `--range-diff`, tweak the heuristic which matches up commits
builtin/log.c +1 -1
index 0fe6f9ba1e..5ac18e2848 100644
@@ -1096,7 +1096,7 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
 	if (rev->rdiff1) {
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
-				rev->creation_factor, 1, &rev->diffopt);
+				rev->creation_factor, 1, NULL);
 	}
 }
 
log-tree.c +1 -1
index 7a83e99250..b243779a0b 100644
@@ -762,7 +762,7 @@ void show_log(struct rev_info *opt)
 		next_commentary_block(opt, NULL);
 		fprintf_ln(opt->diffopt.file, "%s", opt->rdiff_title);
 		show_range_diff(opt->rdiff1, opt->rdiff2,
-				opt->creation_factor, 1, &opt->diffopt);
+				opt->creation_factor, 1, NULL);
 
 		memcpy(&diff_queued_diff, &dq, sizeof(diff_queued_diff));
 	}
range-diff.c +5 -1
index 767af8c5bb..8e52a85c19 100644
@@ -460,7 +460,11 @@ int show_range_diff(const char *range1, const char *range2,
 		struct diff_options opts;
 		struct strbuf indent = STRBUF_INIT;
 
-		memcpy(&opts, diffopt, sizeof(opts));
+		if (diffopt)
+			memcpy(&opts, diffopt, sizeof(opts));
+		else
+			repo_diff_setup(the_repository, &opts);
+
 		if (!opts.output_format)
 			opts.output_format = DIFF_FORMAT_PATCH;
 		opts.flags.suppress_diff_headers = 1;
range-diff.h +5 -0
index 190593f0c7..08a50b6e98 100644
@@ -5,6 +5,11 @@
 
 #define RANGE_DIFF_CREATION_FACTOR_DEFAULT 60
 
+/*
+ * Compare series of commmits in RANGE1 and RANGE2, and emit to the
+ * standard output.  NULL can be passed to DIFFOPT to use the built-in
+ * default.
+ */
 int show_range_diff(const char *range1, const char *range2,
 		    int creation_factor, int dual_color,
 		    struct diff_options *diffopt);
#65 lore raw

Re: [PATCH] format-patch: do not let its diff-options affect --range-diff (was Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options)

Quoted 5 lines · Junio C Hamano
Quoted 4 lines

I had to delay -rc2 to see these last minute tweaks come to some reasonable place to stop at, and I do not think we want to delay the final any longer or destablizing it further by piling last minute undercooked changes on top.

Quoted 7 lines

So how about doing this on top of 'master' instead? As this leaks *no* information wrt how range-diff machinery should behave from the format-patch side by not passing any diffopt, as long as the new code I added to show_range_diff() comes up with a reasonable default diffopts (for which I really would appreciate extra sets of eyes to make sure), this change by definition cannot be wrong (famous last words).

As listed in today's "What's cooking" report, I've merged this to 'next' in today's pushout and planning to have it in the -rc2. I am not married to this exact implementation, and I'd welcome to have an even simpler and less disruptive solution if exists, but I am hoping that this is a good-enough interim measure for the upcoming release, until we decide what to do with the customizability of range-diff driven by format-patch.

In addition to this, I am planning the "rebase --stat" and "reflog that does not say 'rebase -i' but 'rebase'" fixes merged to 'master' before cutting -rc2.

#66 lore raw

Re: [PATCH] format-patch: do not let its diff-options affect --range-diff (was Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options)

Quoted 23 lines · Junio C Hamano
Quoted 5 lines · Junio C Hamano
Quoted 4 lines

I had to delay -rc2 to see these last minute tweaks come to some reasonable place to stop at, and I do not think we want to delay the final any longer or destablizing it further by piling last minute undercooked changes on top.

Quoted 7 lines

So how about doing this on top of 'master' instead? As this leaks *no* information wrt how range-diff machinery should behave from the format-patch side by not passing any diffopt, as long as the new code I added to show_range_diff() comes up with a reasonable default diffopts (for which I really would appreciate extra sets of eyes to make sure), this change by definition cannot be wrong (famous last words).

As listed in today's "What's cooking" report, I've merged this to 'next' in today's pushout and planning to have it in the -rc2. I am not married to this exact implementation, and I'd welcome to have an even simpler and less disruptive solution if exists, but I am hoping that this is a good-enough interim measure for the upcoming release, until we decide what to do with the customizability of range-diff driven by format-patch.

Quoted 3 lines

In addition to this, I am planning the "rebase --stat" and "reflog that does not say 'rebase -i' but 'rebase'" fixes merged to 'master' before cutting -rc2.

Thanks a lot, yeah having this wait looks good to me.

#67 lore raw

Re: [PATCH] format-patch: do not let its diff-options affect --range-diff (was Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options)

In reply to Junio C Hamano

Hi Junio,

Quoted 23 lines · Junio C Hamano
Quoted 5 lines · Junio C Hamano
Quoted 4 lines

I had to delay -rc2 to see these last minute tweaks come to some reasonable place to stop at, and I do not think we want to delay the final any longer or destablizing it further by piling last minute undercooked changes on top.

Quoted 7 lines

So how about doing this on top of 'master' instead? As this leaks *no* information wrt how range-diff machinery should behave from the format-patch side by not passing any diffopt, as long as the new code I added to show_range_diff() comes up with a reasonable default diffopts (for which I really would appreciate extra sets of eyes to make sure), this change by definition cannot be wrong (famous last words).

As listed in today's "What's cooking" report, I've merged this to 'next' in today's pushout and planning to have it in the -rc2. I am not married to this exact implementation, and I'd welcome to have an even simpler and less disruptive solution if exists, but I am hoping that this is a good-enough interim measure for the upcoming release, until we decide what to do with the customizability of range-diff driven by format-patch.

Quoted 3 lines

In addition to this, I am planning the "rebase --stat" and "reflog that does not say 'rebase -i' but 'rebase'" fixes merged to 'master' before cutting -rc2.

Thank you for integrating them. That way, we have an -rc2 with no issues in the built-in rebase/rebase -i that we know of.

Ciao,
Dscho

#68 lore raw

Re: [PATCH] format-patch: do not let its diff-options affect --range-diff (was Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options)

In reply to Junio C Hamano
Quoted 12 lines · Junio C Hamano
Quoted 11 lines · Junio C Hamano

In any case, I tend to agree with the conclusion in the downthread by Dscho that we should just clearly mark that invocations of the "format-patch --range-diff" command with additional diff options is an experimental feature that may not do anything sensible in the upcoming release, and declare that the UI to pass diff options to affect only the range-diff part may later be invented. IOW, I am coming a bit stronger than Dscho's suggestion in that we should not even pretend that we aimed to make the options used for range-diff customizable when driven from format-patch in the upcoming release, or aimed to make --range-diff option compatible with other diff options given to the format-patch command.

I agree that this way forward makes sense. It's clear that I overlooked how there could be unexpected interactions from passing git-format-patch's own diff_options to show_range_diff(), so taking time to think it through without the pressure of a looming release is preferable to rushing out some "fixes".

Quoted 7 lines

So how about doing this on top of 'master' instead? As this leaks *no* information wrt how range-diff machinery should behave from the format-patch side by not passing any diffopt, as long as the new code I added to show_range_diff() comes up with a reasonable default diffopts (for which I really would appreciate extra sets of eyes to make sure), this change by definition cannot be wrong (famous last words).

I, myself, was going to suggest this approach of leaking none of the git-format-patch's options into range_diff(), so I think it is a good one. Later, we can selectively pass certain _sensible_ options into show_range_diff() once we figure out the correct UI (for instance, --range-diff-opts=nopatch,creation-factor:60).

A couple comments on the patch itself...

range-diff.c int show_range_diff(const char *range1, const char *range2, · line 463

Quoted 7 lines
range-diff.c +4 -1
@@ -460,7 +460,11 @@ int show_range_diff(const char *range1, const char *range2,
-               memcpy(&opts, diffopt, sizeof(opts));
+               if (diffopt)
+                       memcpy(&opts, diffopt, sizeof(opts));
+               else
+                       repo_diff_setup(the_repository, &opts);

The first attempt at adding --range-diff to git-format-patch invoked the git-range-diff command, so no diff_options were passed at all. After Dscho libified the range-diff machinery in one of his major re-rolls, I took advantage of that to avoid the subprocess invocation. Another benefit of calling show_range_diff() directly is that when "git format-patch --stdout --range-diff=..." is sent to the terminal, the range-diff gets colored output for free. I'm pleased to see that that still works after this change.

range-diff.h line 9

Quoted 7 lines
range-diff.h +5 -0
@@ -5,6 +5,11 @@
+/*
+ * Compare series of commmits in RANGE1 and RANGE2, and emit to the
+ * standard output.  NULL can be passed to DIFFOPT to use the built-in
+ * default.
+ */

It is more correct to say that the range-diff is emitted to diffopt->file (which may be stdout).

Thanks for working on this.

#69 lore raw

Re: [PATCH] format-patch: do not let its diff-options affect --range-diff (was Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options)

Quoted 11 lines · Eric Sunshine
Quoted 8 lines · Junio C Hamano

Junio C Hamano <[email protected]> writes:
So how about doing this on top of 'master' instead? As this leaks *no* information wrt how range-diff machinery should behave from the format-patch side by not passing any diffopt, as long as the new code I added to show_range_diff() comes up with a reasonable default diffopts (for which I really would appreciate extra sets of eyes to make sure), this change by definition cannot be wrong (famous last words).

Quoted 4 lines

Another benefit of calling show_range_diff() directly is that when "git format-patch --stdout --range-diff=..." is sent to the terminal, the range-diff gets colored output for free. I'm pleased to see that that still works after this change.

(If the patch below makes any sense to you and you know more about this diff/color thing, the fourth bullet in the log message below might interest you.)

Quoted 10 lines

range-diff.h line 9

Quoted 7 lines
range-diff.h +5 -0
@@ -5,6 +5,11 @@
+/*
+ * Compare series of commmits in RANGE1 and RANGE2, and emit to the
+ * standard output.  NULL can be passed to DIFFOPT to use the built-in
+ * default.
+ */

It is more correct to say that the range-diff is emitted to diffopt->file (which may be stdout).

This seems to be an important remark. There's a pretty bad regression here since when diffopt is NULL, we've lost our original, intended diffopt->file and the range-diff ends up on stdout.

Here's my whitespace-damaged WIP. I would be able to pick this up again in about 6h, but anyone is more than welcome to pick this up and run with it in the meantime.

This is not a corner of the code that I'm particularly familiar with...

-->8--
Subject: [PATCH] range-diff: always pass at least minimal diff options
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Commit d8981c3f88 ("format-patch: do not let its diff-options affect --range-diff", 2018-11-30) taught show_range_diff() to accept a NULL-pointer as an indication that it should use its own "reasonable default". That fixed a regression from a5170794 ("Merge branch 'ab/range-diff-no-patch'", 2018-11-18), but unfortunately it introduced a regression of its own.

In particular, it means we forget the file member of the diff options, so rather than placing a range-diff in the cover-letter, we write it to stdout. In order to fix this, rewrite the two callers adjusted by d8981c3f88 to create a "dummy" set of diff options where they only fill in which file to use.

A couple of remarks about this commit:

* No tests. The change in builtin/log.c has been tested manually, the one in log-tree.c not at all, other than by running existing tests.

* I have not convinced myself 100% that there aren't other things that are just as important as file to pass down.

* show_range_diff() can still take NULL, although that is now dead code. If something like this here commit is deemed the proper fix for this, that code path could also go, either as part of this commit, or separately, once we've cut 2.20.

* The range-diff is written colored regardless of destination, i.e., when written to a file, it contains garbage.

---
 builtin/log.c | 12 +++++++++++-
 log-tree.c    | 12 +++++++++++-
 2 files changed, 22 insertions(+), 2 deletions(-)
builtin/log.c +0 -0
index 5ac18e2848..0609e41ae5 100644
@@ -1094,9 +1094,19 @@ static void make_cover_letter(struct rev_info

*rev, int use_stdout,

     }

     if (rev->rdiff1) {
+        /**
+         * (At least for now) we only want to pass down
+         * the file handle where we want the range-diff
+         * to appear. Avoid any other diff options until
+         * we know how we want to handle them.
+         */
+        struct diff_options opts;
+        diff_setup(&opts);
+        opts.file = rev->diffopt.file;
+        diff_setup_done(&opts);
         fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
         show_range_diff(rev->rdiff1, rev->rdiff2,
-                rev->creation_factor, 1, NULL);
+                rev->creation_factor, 1, &opts);
     }

}

log-tree.c +11 -1
index b243779a0b..bc355a4e91 100644
@@ -755,14 +755,24 @@ void show_log(struct rev_info *opt)
 
     if (cmit_fmt_is_mail(ctx.fmt) && opt->rdiff1) {
         struct diff_queue_struct dq;
+        struct diff_options opts;
 
         memcpy(&dq, &diff_queued_diff, sizeof(diff_queued_diff));
         DIFF_QUEUE_CLEAR(&diff_queued_diff);
 
         next_commentary_block(opt, NULL);
         fprintf_ln(opt->diffopt.file, "%s", opt->rdiff_title);
+        /**
+         * (At least for now) we only want to pass down
+         * the file handle where we want the range-diff
+         * to appear. Avoid any other diff options until
+         * we know how we want to handle them.
+        */
+        diff_setup(&opts);
+        opts.file = opt->diffopt.file;
+        diff_setup_done(&opts);
         show_range_diff(opt->rdiff1, opt->rdiff2,
-                opt->creation_factor, 1, NULL);
+                opt->creation_factor, 1, &opts);
 
         memcpy(&diff_queued_diff, &dq, sizeof(diff_queued_diff));
     }
Signature · 1 line
2.20.0.rc2
#70 lore raw

PATCH v2 1/1 range-diff: always pass at least minimal diff options

Commit d8981c3f88 ("format-patch: do not let its diff-options affect --range-diff", 2018-11-30) taught show_range_diff() to accept a NULL-pointer as an indication that it should use its own "reasonable default". That fixed a regression from a5170794 ("Merge branch 'ab/range-diff-no-patch'", 2018-11-18), but unfortunately it introduced a regression of its own.

In particular, it means we forget the file member of the diff options, so rather than placing a range-diff in the cover-letter, we write it to stdout. In order to fix this, rewrite the two callers adjusted by d8981c3f88 to instead create a "dummy" set of diff options where they only fill in which file to use.

Plus, turn off coloring to make sure we don't write any color codes. Maybe we could do opts.use_color = opts.file != stdout, but for now, I'd much rather always write uncolored output than write color codes where there shouldn't be any.

Modify and extend the existing tests to try and verify that the right contents end up in the right place.

Don't revert show_range_diff(), i.e., let it keep accepting NULL. Rather than removing what is dead code and figuring out it isn't actually dead and we've broken 2.20, just leave it for now.

---
Here's another attempt at fixing this recent regression.

 t/t3206-range-diff.sh | 20 +++++++++++++-------
 builtin/log.c         | 13 ++++++++++++-
 log-tree.c            | 13 ++++++++++++-
 3 files changed, 37 insertions(+), 9 deletions(-)
t/t3206-range-diff.sh +13 -7
index e497c1358f..048feaf6dd 100755
@@ -248,18 +248,24 @@ test_expect_success 'dual-coloring' '
 for prev in topic master..topic
 do
 	test_expect_success "format-patch --range-diff=$prev" '
-		git format-patch --stdout --cover-letter --range-diff=$prev \
+		git format-patch --cover-letter --range-diff=$prev \
 			master..unmodified >actual &&
-		grep "= 1: .* s/5/A" actual &&
-		grep "= 2: .* s/4/A" actual &&
-		grep "= 3: .* s/11/B" actual &&
-		grep "= 4: .* s/12/B" actual
+		test_when_finished "rm 000?-*" &&
+		test_line_count = 5 actual &&
+		test_i18ngrep "^Range-diff:$" 0000-* &&
+		grep "= 1: .* s/5/A" 0000-* &&
+		grep "= 2: .* s/4/A" 0000-* &&
+		grep "= 3: .* s/11/B" 0000-* &&
+		grep "= 4: .* s/12/B" 0000-*
 	'
 done
 
 test_expect_success 'format-patch --range-diff as commentary' '
-	git format-patch --stdout --range-diff=HEAD~1 HEAD~1 >actual &&
-	test_i18ngrep "^Range-diff:$" actual
+	git format-patch --range-diff=HEAD~1 HEAD~1 >actual &&
+	test_when_finished "rm 0001-*" &&
+	test_line_count = 1 actual &&
+	test_i18ngrep "^Range-diff:$" 0001-* &&
+	grep "> 1: .* new message" 0001-*
 '
 
 test_done
builtin/log.c +12 -1
index 5ac18e2848..e42487b46d 100644
@@ -1094,9 +1094,20 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
 	}
 
 	if (rev->rdiff1) {
+		/*
+		 * (At least for now) we only want to pass down
+		 * the file handle where we want the range-diff
+		 * to appear. Avoid any other diff options until
+		 * we know how we want to handle them.
+		 */
+		struct diff_options opts;
+		diff_setup(&opts);
+		opts.file = rev->diffopt.file;
+		opts.use_color = 0;
+		diff_setup_done(&opts);
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
-				rev->creation_factor, 1, NULL);
+				rev->creation_factor, 1, &opts);
 	}
 }
 
log-tree.c +12 -1
index b243779a0b..fd79a3ec37 100644
@@ -755,14 +755,25 @@ void show_log(struct rev_info *opt)
 
 	if (cmit_fmt_is_mail(ctx.fmt) && opt->rdiff1) {
 		struct diff_queue_struct dq;
+		struct diff_options opts;
 
 		memcpy(&dq, &diff_queued_diff, sizeof(diff_queued_diff));
 		DIFF_QUEUE_CLEAR(&diff_queued_diff);
 
 		next_commentary_block(opt, NULL);
 		fprintf_ln(opt->diffopt.file, "%s", opt->rdiff_title);
+		/*
+		 * (At least for now) we only want to pass down
+		 * the file handle where we want the range-diff
+		 * to appear. Avoid any other diff options until
+		 * we know how we want to handle them.
+		 */
+		diff_setup(&opts);
+		opts.file = opt->diffopt.file;
+		opts.use_color = 0;
+		diff_setup_done(&opts);
 		show_range_diff(opt->rdiff1, opt->rdiff2,
-				opt->creation_factor, 1, NULL);
+				opt->creation_factor, 1, &opts);
 
 		memcpy(&diff_queued_diff, &dq, sizeof(diff_queued_diff));
 	}
Signature · 1 line
2.20.0.rc2.1.gfcc5f94f1e
#71 lore raw

PATCH v3 1/1 range-diff: always pass at least minimal diff options

From: Martin Ågren <[email protected]>

Commit d8981c3f88 ("format-patch: do not let its diff-options affect --range-diff", 2018-11-30) taught show_range_diff() to accept a NULL-pointer as an indication that it should use its own "reasonable default". That fixed a regression from a5170794 ("Merge branch 'ab/range-diff-no-patch'", 2018-11-18), but unfortunately it introduced a regression of its own.

In particular, it means we forget the file member of the diff options, so rather than placing a range-diff in the cover-letter, we write it to stdout. In order to fix this, rewrite the two callers adjusted by d8981c3f88 to instead create a "dummy" set of diff options where they only fill in the fields we absolutely require, such as output file and color.

Modify and extend the existing tests to try and verify that the right contents end up in the right place.

Don't revert show_range_diff(), i.e., let it keep accepting NULL. Rather than removing what is dead code and figuring out it isn't actually dead and we've broken 2.20, just leave it for now.

[es: retain diff coloring when going to stdout]

---

This is a re-roll of Martin's v2[1]. The only difference from v2 is that it retains coloring when emitting to the terminal (plus an in-code comment was simplified).

The regression introduced by d8981c3f88, in which the range-diff only ever gets emitted to the terminal, and never to the cover letter or commentary section of a standalone patch, makes the --range-diff option rather useless, so this fix probably ought to be fast-tracked. An alternative would be to rip out all the recent "--range-diff"-related changes and go with the --range-diff implementation which has been in use for a few months, even if it is not perfect.

[1]: https://public-inbox.org/git/[email protected]/

builtin/log.c | 11 ++++++++++-

 log-tree.c            | 11 ++++++++++-
 t/t3206-range-diff.sh | 20 +++++++++++++-------
 3 files changed, 33 insertions(+), 9 deletions(-)
builtin/log.c +10 -1
index 5ac18e2848..e8e51068bd 100644
@@ -1094,9 +1094,18 @@ static void make_cover_letter(struct rev_info *rev, int use_stdout,
 	}
 
 	if (rev->rdiff1) {
+		/*
+		 * Pass minimum required diff-options to range-diff; others
+		 * can be added later if deemed desirable.
+		 */
+		struct diff_options opts;
+		diff_setup(&opts);
+		opts.file = rev->diffopt.file;
+		opts.use_color = rev->diffopt.use_color;
+		diff_setup_done(&opts);
 		fprintf_ln(rev->diffopt.file, "%s", rev->rdiff_title);
 		show_range_diff(rev->rdiff1, rev->rdiff2,
-				rev->creation_factor, 1, NULL);
+				rev->creation_factor, 1, &opts);
 	}
 }
 
log-tree.c +10 -1
index b243779a0b..10680c139e 100644
@@ -755,14 +755,23 @@ void show_log(struct rev_info *opt)
 
 	if (cmit_fmt_is_mail(ctx.fmt) && opt->rdiff1) {
 		struct diff_queue_struct dq;
+		struct diff_options opts;
 
 		memcpy(&dq, &diff_queued_diff, sizeof(diff_queued_diff));
 		DIFF_QUEUE_CLEAR(&diff_queued_diff);
 
 		next_commentary_block(opt, NULL);
 		fprintf_ln(opt->diffopt.file, "%s", opt->rdiff_title);
+		/*
+		 * Pass minimum required diff-options to range-diff; others
+		 * can be added later if deemed desirable.
+		 */
+		diff_setup(&opts);
+		opts.file = opt->diffopt.file;
+		opts.use_color = opt->diffopt.use_color;
+		diff_setup_done(&opts);
 		show_range_diff(opt->rdiff1, opt->rdiff2,
-				opt->creation_factor, 1, NULL);
+				opt->creation_factor, 1, &opts);
 
 		memcpy(&diff_queued_diff, &dq, sizeof(diff_queued_diff));
 	}
t/t3206-range-diff.sh +13 -7
index e497c1358f..048feaf6dd 100755
@@ -248,18 +248,24 @@ test_expect_success 'dual-coloring' '
 for prev in topic master..topic
 do
 	test_expect_success "format-patch --range-diff=$prev" '
-		git format-patch --stdout --cover-letter --range-diff=$prev \
+		git format-patch --cover-letter --range-diff=$prev \
 			master..unmodified >actual &&
-		grep "= 1: .* s/5/A" actual &&
-		grep "= 2: .* s/4/A" actual &&
-		grep "= 3: .* s/11/B" actual &&
-		grep "= 4: .* s/12/B" actual
+		test_when_finished "rm 000?-*" &&
+		test_line_count = 5 actual &&
+		test_i18ngrep "^Range-diff:$" 0000-* &&
+		grep "= 1: .* s/5/A" 0000-* &&
+		grep "= 2: .* s/4/A" 0000-* &&
+		grep "= 3: .* s/11/B" 0000-* &&
+		grep "= 4: .* s/12/B" 0000-*
 	'
 done
 
 test_expect_success 'format-patch --range-diff as commentary' '
-	git format-patch --stdout --range-diff=HEAD~1 HEAD~1 >actual &&
-	test_i18ngrep "^Range-diff:$" actual
+	git format-patch --range-diff=HEAD~1 HEAD~1 >actual &&
+	test_when_finished "rm 0001-*" &&
+	test_line_count = 1 actual &&
+	test_i18ngrep "^Range-diff:$" 0001-* &&
+	grep "> 1: .* new message" 0001-*
 '
 
 test_done
Signature · 1 line
2.20.0.rc2.403.gdbc3b29805
#72 lore raw

Re: [PATCH v3] range-diff: always pass at least minimal diff options

Quoted 8 lines · Eric Sunshine

This is a re-roll of Martin's v2[1]. The only difference from v2 is that it retains coloring when emitting to the terminal (plus an in-code comment was simplified).

The regression introduced by d8981c3f88, in which the range-diff only ever gets emitted to the terminal, and never to the cover letter or commentary section of a standalone patch, makes the --range-diff option rather useless, so this fix probably ought to be fast-tracked.

Yup. Thanks. The only thing that makes me wonder is why any of the existing tests (among which I think I saw range-diff driven from format-patch) did not catch this rather obvious glitch.

t/t3206-range-diff.sh test_expect_success 'dual-coloring' ' · line 252

Quoted 11 lines
t/t3206-range-diff.sh +1 -1
index e497c1358f..048feaf6dd 100755
@@ -248,18 +248,24 @@ test_expect_success 'dual-coloring' '
 for prev in topic master..topic
 do
 	test_expect_success "format-patch --range-diff=$prev" '
-		git format-patch --stdout --cover-letter --range-diff=$prev \
+		git format-patch --cover-letter --range-diff=$prev \
 			master..unmodified >actual &&

Ah, of course. Then the "actual" file gets the names of the output files; we expect to see 5 of them.

But now we have lost all the range-diff tests run under "--stdout", so next time some other change regresses only that codepath, we will not notice (which is fine for now but would want to be addressed before the end of the year around which time we certainly will all forget).

Thanks.

Quoted 25 lines
t/t3206-range-diff.sh +12 -6
-		grep "= 1: .* s/5/A" actual &&
-		grep "= 2: .* s/4/A" actual &&
-		grep "= 3: .* s/11/B" actual &&
-		grep "= 4: .* s/12/B" actual
+		test_when_finished "rm 000?-*" &&
+		test_line_count = 5 actual &&
+		test_i18ngrep "^Range-diff:$" 0000-* &&
+		grep "= 1: .* s/5/A" 0000-* &&
+		grep "= 2: .* s/4/A" 0000-* &&
+		grep "= 3: .* s/11/B" 0000-* &&
+		grep "= 4: .* s/12/B" 0000-*
 	'
 done
 
 test_expect_success 'format-patch --range-diff as commentary' '
-	git format-patch --stdout --range-diff=HEAD~1 HEAD~1 >actual &&
-	test_i18ngrep "^Range-diff:$" actual
+	git format-patch --range-diff=HEAD~1 HEAD~1 >actual &&
+	test_when_finished "rm 0001-*" &&
+	test_line_count = 1 actual &&
+	test_i18ngrep "^Range-diff:$" 0001-* &&
+	grep "> 1: .* new message" 0001-*
 '
 
 test_done
#73 lore raw

Re: [PATCH v3] range-diff: always pass at least minimal diff options

In reply to Eric Sunshine
Quoted 9 lines · Eric Sunshine

[es: retain diff coloring when going to stdout]

---

This is a re-roll of Martin's v2[1]. The only difference from v2 is that it retains coloring when emitting to the terminal (plus an in-code comment was simplified).

Thank you so much for this.

builtin/log.c static void make_cover_letter(struct rev_info *rev, int use_stdout, · line 1100

Quoted 5 lines
        if (rev->rdiff1) {
builtin/log.c +4 -0
+               /*
+                * Pass minimum required diff-options to range-diff; others
+                * can be added later if deemed desirable.
+                */

Agreed.

builtin/log.c static void make_cover_letter(struct rev_info *rev, int use_stdout, · line 1104

Quoted 4 lines
builtin/log.c +4 -0
+               struct diff_options opts;
+               diff_setup(&opts);
+               opts.file = rev->diffopt.file;
+               opts.use_color = rev->diffopt.use_color;

Ah, s/0/rev->diffopt.use_color/, well that's obvious.

Thanks!

Martin

#74 lore raw

Re: [PATCH 2/2] format-patch: allow for independent diff & range-diff options

In reply to Ævar Arnfjörð Bjarmason
Quoted 9 lines · Ævar Arnfjörð Bjarmason

I mean not just nasty in terms of implementation, yeah we could do it, but also a nasty UX for things like --word-diff-regex. I.e. instead of:

    --range-diff-word-diff-regex='[0-9"]'

You need:

    --range-diff-opts="--word-diff-regex='[0-9\"]'"
Quoted 7 lines

Now admittedly that in itself isn't very painful *in this case*, but in terms of precedent I really dislike that option, i.e. git having some mode where I need to work to escape input to pass to another command.

Not saying that this --range-diff-* thing is what we should go for, but surely we can find some way to do deal with this that doesn't involve the user needing to escape stuff like this.

I should mention that it was never the intention that git-format-patch's --range-diff option would allow passing _all_ possible options to git-range-diff, but only those most likely to be tweaked by the user (such as --creation-factor). It was understood from the start (and stated by me either in the cover letter to that series or in discussion) that the user always has the escape hatch of running git-range-diff manually and copy/pasting its output into the cover letter.

So, I'm not convinced that we need this sort of flexibility in git-format-patch's --range-diff option, but we certainly can add convenience options (such as I did with --creation-factor) as people complain that their favorite option is missing. For a UI, I think we already have sufficient precedent for "--range-diff-opts=nopatch,creation-factor:60" as a possibility.

#75 lore raw

cover format-patch: fix root cause of recent regression

In reply to Junio C Hamano

As noted in 2/2 this fixes the root cause of the bug I plastered over in
https://public-inbox.org/git/[email protected]/
(that patch is sitting in 'next').

1/2 is a test for existing behavior, to make it more easily understood what's being changed.

Junio: I know it's late, but unless Eric has objections to this UI change I'd really like to have this in 2.20 since this is a change to a new command-line UI that's newly added in 2.20.

As noted in 2/2 the current implementation is inherently limited, you can't tweak diff options for the range-diff in the cover-letter and the patch independently, now you can, and the implementation is much less nasty now that we're not having to share diffopts across two different modes of operation.

Ævar Arnfjörð Bjarmason (2):
format-patch: add test for --range-diff diff output
format-patch: allow for independent diff & range-diff options

 Documentation/git-format-patch.txt |  10 ++-
 builtin/log.c                      |  42 +++++++++---
 t/t3206-range-diff.sh              | 101 +++++++++++++++++++++++++++++
 3 files changed, 142 insertions(+), 11 deletions(-)
Signature · 1 line
2.20.0.rc1.387.gf8505762e3
#76 lore raw

PATCH 1/2 format-patch: add test for --range-diff diff output

In reply to Junio C Hamano

As noted in 43dafc4172 ("format-patch: don't include --stat with --range-diff output", 2018-11-22) the diff options provided on the command-line currently affect both the range-diff and the patch output, but there was no test for checking this with output where we'd show a patch diff. Let's add one.

---
 t/t3206-range-diff.sh | 60 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 60 insertions(+)
t/t3206-range-diff.sh +60 -0
index 90def330bd..bc5facc1cd 100755
@@ -267,4 +267,64 @@ test_expect_success 'format-patch --range-diff as commentary' '
 	test_i18ngrep "^Range-diff:$" actual
 '
 
+test_expect_success 'format-patch with <common diff option>' '
+	# No diff options
+	git format-patch --cover-letter --stdout --range-diff=topic~..topic \
+		changed~..changed >actual.raw &&
+	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
+	1:  a63e992 ! 1:  d966c5c s/12/B/
+	    @@ -8,7 +8,7 @@
+	     @@
+	      9
+	      10
+	    - B
+	    + BB
+	     -12
+	     +B
+	      13
+	-- :
+	EOF
+	test_cmp expect actual.range-diff &&
+	sed -ne "/^--- /,/^--/p" <actual.raw >actual.diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
+	--- a/file
+	+++ b/file
+	@@ -9,7 +9,7 @@ A
+	 9
+	 10
+	 BB
+	-12
+	+B
+	 13
+	 14
+	 15
+	-- :
+	EOF
+	test_cmp expect actual.diff &&
+
+	# -U0
+	git format-patch --cover-letter --stdout -U0 \
+		--range-diff=topic~..topic changed~..changed >actual.raw &&
+	sed -ne "/^1:/,/^--/p" <actual.raw >actual.range-diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
+	1:  a63e992 ! 1:  d966c5c s/12/B/
+	    @@ -11 +11 @@
+	    - B
+	    + BB
+	-- :
+	EOF
+	test_cmp expect actual.range-diff &&
+	sed -ne "/^--- /,/^--/p" <actual.raw >actual.diff &&
+	sed -e "s|:$||" >expect <<-\EOF &&
+	--- a/file
+	+++ b/file
+	@@ -12 +12 @@ BB
+	-12
+	+B
+	-- :
+	EOF
+	test_cmp expect actual.diff
+'
+
 test_done
Signature · 1 line
2.20.0.rc1.387.gf8505762e3
#77 lore raw

Re: [ANNOUNCE] Git v2.20.0-rc1

In reply to Junio C Hamano
Quoted 1 line · Junio C Hamano

* "git rebase" and "git rebase -i" have been reimplemented in C.

Here's another regression in the C version (and rc1), note: the sha1collisiondetection is just a stand in for "some repo":

    (
        rm -rf /tmp/repo &&
        git init /tmp/repo &&
        cd /tmp/repo &&
        for c in 1 2
        do
            touch $c &&
            git add $c &&
            git commit -m"add $c"
        done &&
        git remote add origin https://github.com/cr-marcstevens/sha1collisiondetection.git &&
        git fetch &&
        git branch --set-upstream-to origin/master &&
        git rebase -i
    )

The C version will die with "fatal: unable to read tree 0000000000000000000000000000000000000000". Running this with rebase.useBuiltin=false does the right thing and rebases as of the merge base of the two (which here is the root of the history).

I wasn't trying to stress test rebase. I was just wanting to rebase a history I was about to force-push after cleaning it up, hardly an obscure use-case. So [repeat last transmission in
https://public-inbox.org/git/[email protected]/ ]

#78 lore raw

Re: [ANNOUNCE] Git v2.20.0-rc1

Quoted 7 lines · Ævar Arnfjörð Bjarmason
Quoted 1 line

* "git rebase" and "git rebase -i" have been reimplemented in C.

Here's another regression in the C version (and rc1),...
I wasn't trying to stress test rebase. I was just wanting to rebase a history I was about to force-push after cleaning it up, hardly an obscure use-case. So [repeat last transmission in
https://public-inbox.org/git/[email protected]/ ]

which, to those who are reading from sidelines:

Given that we're still finding regressions bugs in the rebase-in-C version should we be considering reverting 5541bd5b8f ("rebase: default to using the builtin rebase", 2018-08-08)?

I love the feature, but fear that the current list of known regressions serve as a canary for a larger list which we'd discover if we held off for another major release (and would re-enable rebase.useBuiltin=true in master right after 2.20 is out the door).

I am fine with the proposed flip, but I'll have to see the extent of damage this late in the game so that I won't miss anything. In addition to the one-liner below, we'd need to update the quoted release notes entry, and possibly adjust some tests (even though the "reimplementation" ought to be bug-to-bug compatible, it may not be).

builtin/rebase.c +1 -1
index 9dc8475cd3..60e357c735 100644
@@ -54,7 +54,7 @@ static int use_builtin_rebase(void)
 	cp.git_cmd = 1;
 	if (capture_command(&cp, &out, 6)) {
 		strbuf_release(&out);
-		return 1;
+		return 0;
 	}
 
 	strbuf_trim(&out);
#79 lore raw

PATCH 1/1 rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

Quoted 21 lines · Junio C Hamano
Quoted 7 lines · Ævar Arnfjörð Bjarmason
Quoted 1 line

* "git rebase" and "git rebase -i" have been reimplemented in C.

Here's another regression in the C version (and rc1),...
I wasn't trying to stress test rebase. I was just wanting to rebase a history I was about to force-push after cleaning it up, hardly an obscure use-case. So [repeat last transmission in
https://public-inbox.org/git/[email protected]/ ]

which, to those who are reading from sidelines:

Given that we're still finding regressions bugs in the rebase-in-C version should we be considering reverting 5541bd5b8f ("rebase: default to using the builtin rebase", 2018-08-08)?

I love the feature, but fear that the current list of known regressions serve as a canary for a larger list which we'd discover if we held off for another major release (and would re-enable rebase.useBuiltin=true in master right after 2.20 is out the door).

Quoted 5 lines

I am fine with the proposed flip, but I'll have to see the extent of damage this late in the game so that I won't miss anything. In addition to the one-liner below, we'd need to update the quoted release notes entry, and possibly adjust some tests (even though the "reimplementation" ought to be bug-to-bug compatible, it may not be).

So, in a more concrete form, what you want to see is something like this in -rc2 and later?

-- >8 --
Subject: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature

It turns out to be a bit too early to unleash the reimplementation to the general public. Let's rewrite some documentation and make it an opt-in feature.

---
 Documentation/config/rebase.txt | 16 ++++++----------
 builtin/rebase.c                |  2 +-
 t/README                        |  4 ++--
 3 files changed, 9 insertions(+), 13 deletions(-)
Documentation/config/rebase.txt +6 -10
index f079bf6b7e..af12623151 100644
@@ -1,16 +1,12 @@
 rebase.useBuiltin::
-	Set to `false` to use the legacy shellscript implementation of
-	linkgit:git-rebase[1]. Is `true` by default, which means use
-	the built-in rewrite of it in C.
+	Set to `true` to use the experimental reimplementation of
+	linkgit:git-rebase[1] in C.  Defaults to `false`.
 +
 The C rewrite is first included with Git version 2.20. This option
-serves an an escape hatch to re-enable the legacy version in case any
-bugs are found in the rewrite. This option and the shellscript version
-of linkgit:git-rebase[1] will be removed in some future release.
-+
-If you find some reason to set this option to `false` other than
-one-off testing you should report the behavior difference as a bug in
-git.
+allows early adopters to opt into the experimental version to find
+bugs in the rewritten version. This option and the shellscript version
+of linkgit:git-rebase[1] will be removed in some future release once
+the reimplementation becomes more stable.
 
 rebase.stat::
 	Whether to show a diffstat of what changed upstream since the last
builtin/rebase.c +1 -1
index 5b3e5baec8..19ad97b177 100644
@@ -59,7 +59,7 @@ static int use_builtin_rebase(void)
 	cp.git_cmd = 1;
 	if (capture_command(&cp, &out, 6)) {
 		strbuf_release(&out);
-		return 1;
+		return 0;
 	}
 
 	strbuf_trim(&out);
t/README +2 -2
index 28711cc508..7e925e5fea 100644
@@ -345,8 +345,8 @@ for the index version specified.  Can be set to any valid version
 GIT_TEST_PRELOAD_INDEX=<boolean> exercises the preload-index code path
 by overriding the minimum number of cache entries required per thread.
 
-GIT_TEST_REBASE_USE_BUILTIN=<boolean>, when false, disables the
-builtin version of git-rebase. See 'rebase.useBuiltin' in
+GIT_TEST_REBASE_USE_BUILTIN=<boolean>, when true, forces the use of
+builtin version of git-rebase in the test. See 'rebase.useBuiltin' in
 git-config(1).
 
 GIT_TEST_INDEX_THREADS=<n> enables exercising the multi-threaded loading
Signature · 1 line
2.20.0-rc1
#80 lore raw

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

Hi,

Quoted 10 lines · Junio C Hamano
Quoted 10 lines
Quoted 8 lines · Ævar Arnfjörð Bjarmason

Given that we're still finding regressions bugs in the rebase-in-C version should we be considering reverting 5541bd5b8f ("rebase: default to using the builtin rebase", 2018-08-08)?

I love the feature, but fear that the current list of known regressions serve as a canary for a larger list which we'd discover if we held off for another major release (and would re-enable rebase.useBuiltin=true in master right after 2.20 is out the door).

[...]

Quoted 10 lines

So, in a more concrete form, what you want to see is something like this in -rc2 and later?

-- >8 --
Subject: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature

It turns out to be a bit too early to unleash the reimplementation to the general public. Let's rewrite some documentation and make it an opt-in feature.

Quoted 6 lines
---
 Documentation/config/rebase.txt | 16 ++++++----------
 builtin/rebase.c                |  2 +-
 t/README                        |  4 ++--
 3 files changed, 9 insertions(+), 13 deletions(-)

I thought I should weigh in on how this would affect Debian's and Google's deployments.

First of all, I've looked over the revert patch carefully and it is well written and does what it says on the tin.

At https://bugs.debian.org/914695 is a report of a test regression in an outside project that is very likely to have been triggered by the new faster rebase code. The issue has not been triaged, so I don't know yet whether it's a problem in rebase-in-c or a manifestation of a bug in the test.

That said, Google has been running with the new rebase since ~1 month ago when it became the default, with no issues reported by users. As a result, I am confident that it can cope with what most users of "next" throw at it, which means that if we are to find more issues to polish it better, it will need all the exposure it can get.

In the Google deployment, we will keep using rebase-in-c even if it gets disabled by default, in order to help with that.

From the Debian point of view, it's only a matter of time before rebase-in-c becomes the default: even if it's not the default in 2.20, it would presumably be so in 2.21 or 2.22. That means the community's attention when resolving security and reliability bugs would be on the rebase-in-c implementation. As a result, the Debian package will most likely enable rebase-in-c by default even if upstream disables it, in order to increase the package's shelf life (i.e. to ease the maintenance burden of supporting whichever version of the package ends up in the next Debian stable).

So with either hat on, it doesn't matter whether you apply this patch upstream.

Having two pretty different deployments end up with the same conclusion leads me to suspect that it's best for upstream not to apply the revert patch, unless either

(a) we have a concrete regression to address and then try again, or
(b) we have a test or other plan to follow before trying again.

Thanks and hope that helps,
Jonathan

#81 lore raw

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

Hi Jonathan,

Quoted 3 lines · Jonathan Nieder

At https://bugs.debian.org/914695 is a report of a test regression in an outside project that is very likely to have been triggered by the new faster rebase code.

From looking through that log.gz (without having a clue where the test code lives, so I cannot say what it is supposed to do, and also: this is the first time I hear about dgit...), it would appear that this must be a regression in the reflog messages produced by git rebase.

Quoted 2 lines

The issue has not been triaged, so I don't know yet whether it's a problem in rebase-in-c or a manifestation of a bug in the test.

It ends thusly:

-- snip --
[...]
+ git reflog
+ egrep 'debrebase new-upstream.*checkout'
+ test 1 = 0
+ t-report-failure
+ set +x
TEST FAILED
-- snap --

Which makes me think that the reflog we produce in *some* code path that originally called git checkout differs from the scripted rebase's generated reflog.

Quoted 5 lines

That said, Google has been running with the new rebase since ~1 month ago when it became the default, with no issues reported by users. As a result, I am confident that it can cope with what most users of "next" throw at it, which means that if we are to find more issues to polish it better, it will need all the exposure it can get.

Right. And there are a few weeks before the holidays, which should give me time to fix whatever bugs are discovered (I only half mind being the only one who fixes these bugs).

Quoted 16 lines

In the Google deployment, we will keep using rebase-in-c even if it gets disabled by default, in order to help with that.

From the Debian point of view, it's only a matter of time before rebase-in-c becomes the default: even if it's not the default in 2.20, it would presumably be so in 2.21 or 2.22. That means the community's attention when resolving security and reliability bugs would be on the rebase-in-c implementation. As a result, the Debian package will most likely enable rebase-in-c by default even if upstream disables it, in order to increase the package's shelf life (i.e. to ease the maintenance burden of supporting whichever version of the package ends up in the next Debian stable).

So with either hat on, it doesn't matter whether you apply this patch upstream.

Quoted 6 lines

Having two pretty different deployments end up with the same conclusion leads me to suspect that it's best for upstream not to apply the revert patch, unless either

(a) we have a concrete regression to address and then try again, or
(b) we have a test or other plan to follow before trying again.

In this instance, I am more a fan of the "let's move fast and break things, then move even faster fixing them" approach.

Besides, the bug that Ævar discovered was a bug already in the scripted rebase, but hidden by yet another bug (the missing error checking).

I get the pretty firm impression that the common code paths are now pretty robust, and only lesser-exercised features may expose a bug (or regression, as in the case of the reflogs, where one could argue that the exact reflog message is not something we promise not to fiddle with).

Ciao,
Dscho

#82 lore raw

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

Quoted 68 lines · Johannes Schindelin

Hi Jonathan,

Quoted 3 lines · Jonathan Nieder

At https://bugs.debian.org/914695 is a report of a test regression in an outside project that is very likely to have been triggered by the new faster rebase code.

From looking through that log.gz (without having a clue where the test code lives, so I cannot say what it is supposed to do, and also: this is the first time I hear about dgit...), it would appear that this must be a regression in the reflog messages produced by git rebase.

Quoted 2 lines

The issue has not been triaged, so I don't know yet whether it's a problem in rebase-in-c or a manifestation of a bug in the test.

It ends thusly:

-- snip --
[...]
+ git reflog
+ egrep 'debrebase new-upstream.*checkout'
+ test 1 = 0
+ t-report-failure
+ set +x
TEST FAILED
-- snap --

Which makes me think that the reflog we produce in *some* code path that originally called git checkout differs from the scripted rebase's generated reflog.

Quoted 5 lines

That said, Google has been running with the new rebase since ~1 month ago when it became the default, with no issues reported by users. As a result, I am confident that it can cope with what most users of "next" throw at it, which means that if we are to find more issues to polish it better, it will need all the exposure it can get.

Right. And there are a few weeks before the holidays, which should give me time to fix whatever bugs are discovered (I only half mind being the only one who fixes these bugs).

Quoted 16 lines

In the Google deployment, we will keep using rebase-in-c even if it gets disabled by default, in order to help with that.

From the Debian point of view, it's only a matter of time before rebase-in-c becomes the default: even if it's not the default in 2.20, it would presumably be so in 2.21 or 2.22. That means the community's attention when resolving security and reliability bugs would be on the rebase-in-c implementation. As a result, the Debian package will most likely enable rebase-in-c by default even if upstream disables it, in order to increase the package's shelf life (i.e. to ease the maintenance burden of supporting whichever version of the package ends up in the next Debian stable).

So with either hat on, it doesn't matter whether you apply this patch upstream.

Quoted 6 lines

Having two pretty different deployments end up with the same conclusion leads me to suspect that it's best for upstream not to apply the revert patch, unless either

(a) we have a concrete regression to address and then try again, or
(b) we have a test or other plan to follow before trying again.

In this instance, I am more a fan of the "let's move fast and break things, then move even faster fixing them" approach.

Quoted 7 lines

Besides, the bug that Ævar discovered was a bug already in the scripted rebase, but hidden by yet another bug (the missing error checking).

I get the pretty firm impression that the common code paths are now pretty robust, and only lesser-exercised features may expose a bug (or regression, as in the case of the reflogs, where one could argue that the exact reflog message is not something we promise not to fiddle with).

Since I raised this 'should we hold off?' I thought I'd chime in and say that I'm fine with going along with what you suggest and having the builtin as the default in the final. IOW not merge jc/postpone-rebase-in-c down.

#83 lore raw

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

Quoted 4 lines · Ævar Arnfjörð Bjarmason

Since I raised this 'should we hold off?' I thought I'd chime in and say that I'm fine with going along with what you suggest and having the builtin as the default in the final. IOW not merge jc/postpone-rebase-in-c down.

OK.

#84 lore raw

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

In reply to Johannes Schindelin

Hi Jonathan,

if you could pry more information (or better information) out of that bug reporter, that would be nice. Apparently my email address is blacklisted by his mail provider, so he is unlikely to have received my previous mail (nor will he receive this one, I am sure).

Thanks,
Dscho

Quoted 78 lines · Johannes Schindelin

Hi Jonathan,

Quoted 3 lines · Jonathan Nieder

At https://bugs.debian.org/914695 is a report of a test regression in an outside project that is very likely to have been triggered by the new faster rebase code.

From looking through that log.gz (without having a clue where the test code lives, so I cannot say what it is supposed to do, and also: this is the first time I hear about dgit...), it would appear that this must be a regression in the reflog messages produced by git rebase.

Quoted 2 lines

The issue has not been triaged, so I don't know yet whether it's a problem in rebase-in-c or a manifestation of a bug in the test.

It ends thusly:

-- snip --
[...]
+ git reflog
+ egrep 'debrebase new-upstream.*checkout'
+ test 1 = 0
+ t-report-failure
+ set +x
TEST FAILED
-- snap --

Which makes me think that the reflog we produce in *some* code path that originally called git checkout differs from the scripted rebase's generated reflog.

Quoted 5 lines

That said, Google has been running with the new rebase since ~1 month ago when it became the default, with no issues reported by users. As a result, I am confident that it can cope with what most users of "next" throw at it, which means that if we are to find more issues to polish it better, it will need all the exposure it can get.

Right. And there are a few weeks before the holidays, which should give me time to fix whatever bugs are discovered (I only half mind being the only one who fixes these bugs).

Quoted 16 lines

In the Google deployment, we will keep using rebase-in-c even if it gets disabled by default, in order to help with that.

From the Debian point of view, it's only a matter of time before rebase-in-c becomes the default: even if it's not the default in 2.20, it would presumably be so in 2.21 or 2.22. That means the community's attention when resolving security and reliability bugs would be on the rebase-in-c implementation. As a result, the Debian package will most likely enable rebase-in-c by default even if upstream disables it, in order to increase the package's shelf life (i.e. to ease the maintenance burden of supporting whichever version of the package ends up in the next Debian stable).

So with either hat on, it doesn't matter whether you apply this patch upstream.

Quoted 6 lines

Having two pretty different deployments end up with the same conclusion leads me to suspect that it's best for upstream not to apply the revert patch, unless either

(a) we have a concrete regression to address and then try again, or
(b) we have a test or other plan to follow before trying again.

In this instance, I am more a fan of the "let's move fast and break things, then move even faster fixing them" approach.

Besides, the bug that Ævar discovered was a bug already in the scripted rebase, but hidden by yet another bug (the missing error checking).

I get the pretty firm impression that the common code paths are now pretty robust, and only lesser-exercised features may expose a bug (or regression, as in the case of the reflogs, where one could argue that the exact reflog message is not something we promise not to fiddle with).

Ciao,
Dscho

#85 lore raw

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

Johannes Schindelin writes ("Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)"):

Quoted 4 lines

if you could pry more information (or better information) out of that bug reporter, that would be nice. Apparently my email address is blacklisted by his mail provider, so he is unlikely to have received my previous mail (nor will he receive this one, I am sure).

(I did receive this mail. Sorry for the inconvenience, which sadly is inevitable occasionally in the modern email world. FTR in future feel free to send the bounce to postmaster@chiark and I will make a you-shaped hole in my spamfilter. Also with Debian bugs you can launder your messages by, eg, emailing 914695-submitter@bugs.)

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

At https://bugs.debian.org/914695 is a report of a test regression in an outside project that is very likely to have been triggered by the new faster rebase code.

As I wrote in the bug report last night:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=914695#15

I have investigated and the bug seems to be that git-rebase --onto now fails to honour GIT_REFLOG_ACTION for the initial checkout.

In a successful run with older git I get a reflog like this:

4833d74 HEAD@{0}: rebase finished: returning to refs/heads/with-preexisting
4833d74 HEAD@{1}: debrebase new-upstream 2.1-1: rebase: Add another new upstream file cabd5ec HEAD@{2}: debrebase new-upstream 2.1-1: rebase: Edit the .c file
0b362ce HEAD@{3}: debrebase new-upstream 2.1-1: rebase: Add a new upstream file
29653e5 HEAD@{4}: debrebase new-upstream 2.1-1: rebase: checkout 29653e5a17bee4ac23a68bba3e12bc1f52858ac3 85e0c46 HEAD@{5}: debrebase: launder for new upstream

With a newer git I get this:

6d3fb91 HEAD@{0}: rebase finished: returning to refs/heads/master 6d3fb91 HEAD@{1}: debrebase new-upstream 2.1-1: rebase: Add another new upstream file 86c0721 HEAD@{2}: debrebase new-upstream 2.1-1: rebase: Edit the .c file 50ba56c HEAD@{3}: debrebase new-upstream 2.1-1: rebase: Add a new upstream file 8272825 HEAD@{4}: rebase: checkout 8272825bb4ff6eba89afa936e32b6460f963a36a c78db55 HEAD@{5}: debrebase: launder for new upstream

This breaks the test because my test suite is checking that I set GIT_REFLOG_ACTION appropriately.

If you want I can provide a minimal test case but this should suffice to see the bug I hope...

Regards
Ian.

Signature · 4 lines
Ian Jackson <[email protected]>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
#86 lore raw

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

Hi Ian,

Quoted 11 lines · Ian Jackson

Johannes Schindelin writes ("Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)"):

Quoted 4 lines

if you could pry more information (or better information) out of that bug reporter, that would be nice. Apparently my email address is blacklisted by his mail provider, so he is unlikely to have received my previous mail (nor will he receive this one, I am sure).

(I did receive this mail. Sorry for the inconvenience, which sadly is inevitable occasionally in the modern email world. FTR in future feel free to send the bounce to postmaster@chiark and I will make a you-shaped hole in my spamfilter. Also with Debian bugs you can launder your messages by, eg, emailing 914695-submitter@bugs.)

Right. I myself have plenty of email-related problems that seem to crop up this year in particular.

Quoted 29 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

At https://bugs.debian.org/914695 is a report of a test regression in an outside project that is very likely to have been triggered by the new faster rebase code.

As I wrote in the bug report last night:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=914695#15

I have investigated and the bug seems to be that git-rebase --onto now fails to honour GIT_REFLOG_ACTION for the initial checkout.

In a successful run with older git I get a reflog like this:

4833d74 HEAD@{0}: rebase finished: returning to refs/heads/with-preexisting
4833d74 HEAD@{1}: debrebase new-upstream 2.1-1: rebase: Add another new upstream file cabd5ec HEAD@{2}: debrebase new-upstream 2.1-1: rebase: Edit the .c file
0b362ce HEAD@{3}: debrebase new-upstream 2.1-1: rebase: Add a new upstream file
29653e5 HEAD@{4}: debrebase new-upstream 2.1-1: rebase: checkout 29653e5a17bee4ac23a68bba3e12bc1f52858ac3 85e0c46 HEAD@{5}: debrebase: launder for new upstream

With a newer git I get this:

6d3fb91 HEAD@{0}: rebase finished: returning to refs/heads/master 6d3fb91 HEAD@{1}: debrebase new-upstream 2.1-1: rebase: Add another new upstream file 86c0721 HEAD@{2}: debrebase new-upstream 2.1-1: rebase: Edit the .c file 50ba56c HEAD@{3}: debrebase new-upstream 2.1-1: rebase: Add a new upstream file 8272825 HEAD@{4}: rebase: checkout 8272825bb4ff6eba89afa936e32b6460f963a36a c78db55 HEAD@{5}: debrebase: launder for new upstream

Quoted 5 lines

This breaks the test because my test suite is checking that I set GIT_REFLOG_ACTION appropriately.

If you want I can provide a minimal test case but this should suffice to see the bug I hope...

This should be plenty for me to get going. Thank you!

Ciao,
Johannes

Quoted 10 lines

Regards
Ian.

Signature · 4 lines
Ian Jackson <[email protected]>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
#87 lore raw

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

Johannes Schindelin writes ("Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)"):

Quoted 8 lines
Quoted 8 lines

In a successful run with older git I get a reflog like this:

4833d74 HEAD@{0}: rebase finished: returning to refs/heads/with-preexisting
4833d74 HEAD@{1}: debrebase new-upstream 2.1-1: rebase: Add another new upstream file cabd5ec HEAD@{2}: debrebase new-upstream 2.1-1: rebase: Edit the .c file
0b362ce HEAD@{3}: debrebase new-upstream 2.1-1: rebase: Add a new upstream file
29653e5 HEAD@{4}: debrebase new-upstream 2.1-1: rebase: checkout 29653e5a17bee4ac23a68bba3e12bc1f52858ac3 85e0c46 HEAD@{5}: debrebase: launder for new upstream

...

Quoted 7 lines
Quoted 5 lines

This breaks the test because my test suite is checking that I set GIT_REFLOG_ACTION appropriately.

If you want I can provide a minimal test case but this should suffice to see the bug I hope...

This should be plenty for me to get going. Thank you!

Happy hunting.

While you're looking at this, I observe that the fact that the `rebase finished' message also does not honour GIT_REFLOG_ACTION appears to be a pre-existing bug.

(In general one often can't rely on GIT_REFLOG_ACTION still being set because the rebase might have been interrupted and restarted, which I think is why my test case looks for it in the initial `checkout' message.)

Regards,
Ian.

Signature · 4 lines
Ian Jackson <[email protected]>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
#88 lore raw

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

Hi Ian,

Quoted 13 lines · Ian Jackson

Johannes Schindelin writes ("Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)"):

Quoted 8 lines
Quoted 8 lines

In a successful run with older git I get a reflog like this:

4833d74 HEAD@{0}: rebase finished: returning to refs/heads/with-preexisting
4833d74 HEAD@{1}: debrebase new-upstream 2.1-1: rebase: Add another new upstream file cabd5ec HEAD@{2}: debrebase new-upstream 2.1-1: rebase: Edit the .c file
0b362ce HEAD@{3}: debrebase new-upstream 2.1-1: rebase: Add a new upstream file
29653e5 HEAD@{4}: debrebase new-upstream 2.1-1: rebase: checkout 29653e5a17bee4ac23a68bba3e12bc1f52858ac3 85e0c46 HEAD@{5}: debrebase: launder for new upstream

...

Quoted 3 lines
Quoted 3 lines

This breaks the test because my test suite is checking that I set GIT_REFLOG_ACTION appropriately.

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

If you want I can provide a minimal test case but this should suffice to see the bug I hope...

This should be plenty for me to get going. Thank you!

Happy hunting.

I'll have to take a (lengthy) dinner break now, but this is what I have so
far: a regression test that verifies the breakage (see the fix-reflog-action branch at https://github.com/dscho/git). I'll continue after dinner and am confident that this bug will be fixed within the next four hours.

Quoted 3 lines

While you're looking at this, I observe that the fact that the `rebase finished' message also does not honour GIT_REFLOG_ACTION appears to be a pre-existing bug.

I noticed that, too, but at this point I am only fixing regressions. We can try to fix this long-standing bug in the v2.20 cycle.

Ciao,
Johannes

Quoted 14 lines

(In general one often can't rely on GIT_REFLOG_ACTION still being set because the rebase might have been interrupted and restarted, which I think is why my test case looks for it in the initial `checkout' message.)

Regards,
Ian.

Signature · 4 lines
Ian Jackson <[email protected]>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
#89 lore raw

Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)

Johannes Schindelin writes ("Re: [PATCH] rebase: mark the C reimplementation as an experimental opt-in feature (was Re: [ANNOUNCE] Git v2.20.0-rc1)"):

Quoted 5 lines

I'll have to take a (lengthy) dinner break now, but this is what I have so
far: a regression test that verifies the breakage (see the fix-reflog-action branch at https://github.com/dscho/git). I'll continue after dinner and am confident that this bug will be fixed within the next four hours.

That seems super speedy to me!

When you have a fix I will leave it up to the Debian git maintainers to decide whether they want to cherry pick your fix into their package, or await an updated upstream branch with rc, or what.

Quoted 7 lines

[Ian:]

Quoted 3 lines

While you're looking at this, I observe that the fact that the `rebase finished' message also does not honour GIT_REFLOG_ACTION appears to be a pre-existing bug.

I noticed that, too, but at this point I am only fixing regressions. We can try to fix this long-standing bug in the v2.20 cycle.

Right.

Thanks,
Ian.

Signature · 4 lines
Ian Jackson <[email protected]>   These opinions are my own.

If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.
#90 lore raw

Re: [ANNOUNCE] Git v2.20.0-rc1

In reply to Ævar Arnfjörð Bjarmason

Hi Ævar,

Quoted 23 lines · Ævar Arnfjörð Bjarmason
Quoted 1 line · Junio C Hamano

* "git rebase" and "git rebase -i" have been reimplemented in C.

Here's another regression in the C version (and rc1), note: the sha1collisiondetection is just a stand in for "some repo":

    (
        rm -rf /tmp/repo &&
        git init /tmp/repo &&
        cd /tmp/repo &&
        for c in 1 2
        do
            touch $c &&
            git add $c &&
            git commit -m"add $c"
        done &&
        git remote add origin https://github.com/cr-marcstevens/sha1collisiondetection.git &&
        git fetch &&
        git branch --set-upstream-to origin/master &&
        git rebase -i
    )
Quoted 4 lines

The C version will die with "fatal: unable to read tree 0000000000000000000000000000000000000000". Running this with rebase.useBuiltin=false does the right thing and rebases as of the merge base of the two (which here is the root of the history).

Sorry, this bug does not reproduce here:

$ git rebase -i
Successfully rebased and updated refs/heads/master.
Quoted 4 lines

I wasn't trying to stress test rebase. I was just wanting to rebase a history I was about to force-push after cleaning it up, hardly an obscure use-case. So [repeat last transmission in
https://public-inbox.org/git/[email protected]/ ]

Maybe you can give me the full details so that I can verify that this is indeed a bug in the builtin C and not just a regression caused by some random branches being merged together?

In short: please provide me with the exact URL and branch of your git.git fork to test. Then please make sure to specify the precise revision of the sha1collisiondetection/master rev, just in case that it matters.

Ideally, you would reduce the problem to a proper test case, say, for t3412 (it seems that you try to rebase onto an unrelated history, so it is *vaguely* related to "rebase-root").

Ciao,
Dscho

#91 lore raw

Re: [ANNOUNCE] Git v2.20.0-rc1

Hi Ævar,

Quoted 44 lines · Johannes Schindelin
Quoted 23 lines · Ævar Arnfjörð Bjarmason
Quoted 1 line · Junio C Hamano

* "git rebase" and "git rebase -i" have been reimplemented in C.

Here's another regression in the C version (and rc1), note: the sha1collisiondetection is just a stand in for "some repo":

    (
        rm -rf /tmp/repo &&
        git init /tmp/repo &&
        cd /tmp/repo &&
        for c in 1 2
        do
            touch $c &&
            git add $c &&
            git commit -m"add $c"
        done &&
        git remote add origin https://github.com/cr-marcstevens/sha1collisiondetection.git &&
        git fetch &&
        git branch --set-upstream-to origin/master &&
        git rebase -i
    )
Quoted 4 lines

The C version will die with "fatal: unable to read tree 0000000000000000000000000000000000000000". Running this with rebase.useBuiltin=false does the right thing and rebases as of the merge base of the two (which here is the root of the history).

Sorry, this bug does not reproduce here:

$ git rebase -i
Successfully rebased and updated refs/heads/master.
Quoted 4 lines

I wasn't trying to stress test rebase. I was just wanting to rebase a history I was about to force-push after cleaning it up, hardly an obscure use-case. So [repeat last transmission in
https://public-inbox.org/git/[email protected]/ ]

Maybe you can give me the full details so that I can verify that this is indeed a bug in the builtin C and not just a regression caused by some random branches being merged together?

Quoted 7 lines

In short: please provide me with the exact URL and branch of your git.git fork to test. Then please make sure to specify the precise revision of the sha1collisiondetection/master rev, just in case that it matters.

Ideally, you would reduce the problem to a proper test case, say, for t3412 (it seems that you try to rebase onto an unrelated history, so it is *vaguely* related to "rebase-root").

So I was getting spooked enough by your half-complete bug report that I did more digging (it is really quite a bit frustrating to have so little hard evidence to go on, a wild goose chase is definitely not what I was looking forward to after a day of fighting other fires, but you know, built-in rebase is dear to me).

The error message you copied clearly comes from tree-walk.c, from fill_tree_descriptor() (the other "unable to read tree" messages enclose the hash in parentheses).

There are exactly 3 calls to said function in the built-in rebase/rebase -i in the current master, a1598010f775 (Merge branch 'nd/per-worktree-ref-iteration', 2018-11-26):

$ git grep fill_tree_descriptor -- builtin/rebase*.c sequencer.[ch] rebase-interactive.[ch]
builtin/rebase.c:       if (!reset_hard && !fill_tree_descriptor(&desc[nr++], &head_oid)) {
builtin/rebase.c:       if (!fill_tree_descriptor(&desc[nr++], oid)) {
sequencer.c:    if (!fill_tree_descriptor(&desc, &oid)) {

The last one of these is in do_reset(), i.e. handling a reset command which you did not ask for, as you passed -i to git rebase, not -ir.

The first two *both* are in reset_head(). The first of them uses head_oid, which is read directly via get_oid("HEAD", &head_oid), so if this is all zeroes for you, then it's not rebase's fault.

The second one uses the parameter oid passed into reset_head(). The only calls to that function that do not pass NULL as oid (which would trigger oid to be replaced by &head_oid, i.e again not all zeroes unless your setup is broken) are:

- in the --abort code path
- in the --autostash code path
- in the fast-forwarding code path
- just after the "First, rewinding head" message in the *non*-interactive rebase

None of these apply to your script snippet.

Under the assumption that you might have forgotten to talk about rebase.autostash=true and some dirty file, I tried to augment the script snippet accordingly, but the built-in rebase as of current master still works for me, plus: reading the autostash code path, it is hard to imagine that the lookup_commit_reference() would return a pointer to a commit object whose oid is all zeroes.

In short, even a thorough study of the code (keeping in mind the few tidbits of information provided by you) leaves me really wondering which code you run, because it sure does not look like current master to me.

And if it is not master, then I have to ask why you keep suggesting to turn off the built-in rebase by default in master.

Ciao,
Johannes

P.S.: Maybe you have a hook you forgot to mention?

#92 lore raw

Re: [ANNOUNCE] Git v2.20.0-rc1

Quoted 5 lines · Johannes Schindelin

...
In short, even a thorough study of the code (keeping in mind the few tidbits of information provided by you) leaves me really wondering which code you run, because it sure does not look like current master to me.

Quoted 7 lines

And if it is not master, then I have to ask why you keep suggesting to turn off the built-in rebase by default in master.

Ciao,
Johannes

P.S.: Maybe you have a hook you forgot to mention?

Any response? Or can I retract jc/postpone-rebase-in-c that was prepared as a reaction to this?

#93 lore raw

Re: [ANNOUNCE] Git v2.20.0-rc1

Hi Junio,

Quoted 10 lines · Junio C Hamano
Quoted 8 lines · Johannes Schindelin

...
In short, even a thorough study of the code (keeping in mind the few tidbits of information provided by you) leaves me really wondering which code you run, because it sure does not look like current master to me.

And if it is not master, then I have to ask why you keep suggesting to turn off the built-in rebase by default in master.

Quoted 7 lines
Quoted 4 lines

Ciao,
Johannes

P.S.: Maybe you have a hook you forgot to mention?

Any response? Or can I retract jc/postpone-rebase-in-c that was prepared as a reaction to this?

I worked with Ævar via IRC and we figured out the root cause and I submitted https://public-inbox.org/git/[email protected]/
to fix it.

Given that this is a really obscure edge case (git rebase --stat -v -i onto an unrelated commit history, if you take away one of these, the bug does not trigger), and that it was only discovered to be a bug *because* of the built-in rebase (the scripted version had the same bug, but simply forgot to do proper error checking), I would not think that the reported bug is a strong argument in favor of turning off the built-in rebase by defauly.

In other words, after understanding the bug I am even more confident than before that the built-in rebase is actually in a pretty good shape.

I do not expect any major regressions, and if any happen: we do have that escape hatch for corner cases while I fix those bugs.

Ciao,
Dscho

#94 lore raw

Re: [ANNOUNCE] Git v2.20.0-rc1

In reply to Junio C Hamano

Unless I hear otherwise in the next 24 hours, I am planning to merge the following topics to 'master' before cutting -rc2. Please stop me on any of these topics.

- jc/postpone-rebase-in-c

This may be the most controversial. It demotes the C reimplementation of "git rebase" to an experimental opt-in feature that can only be enabled by setting rebase.useBuiltIn configuration that defaults to false.

cf. <[email protected]>

- ab/format-patch-rangediff-not-stat

The "--rangediff" option recently added to "format-patch" interspersed a bogus and useless "--stat" information by mistake, which is being corrected.

cf. <[email protected]>

The following three topics I do not need help in deciding; they are all good and will be merged before -rc2.

- jk/t5562-perl-path-fix

This is to invoke a perl scriptlet with "$PERL_PATH" in one of the new tests, instead of relying on (an incorrect) she-bang line in the script file.

- tb/clone-case-smashing-warning-test

This enables the test to see the behaviour of "git clone" after cloning a project that has paths that are different only in case on MINGW (earlier it wanted CASE_INSENSITIVE_FS prerequisite and in addition not on MINGW).

- nd/per-worktree-ref-iteration

This fixes a "return function_that_returns_void(...)" in a function that returns void.

Thanks.

#95 lore raw

Re: [ANNOUNCE] Git v2.20.0-rc1

Quoted 5 lines · Junio C Hamano

Unless I hear otherwise in the next 24 hours, I am planning to merge the following topics to 'master' before cutting -rc2. Please stop me on any of these topics.

Quoted 8 lines

- jc/postpone-rebase-in-c

This may be the most controversial. It demotes the C reimplementation of "git rebase" to an experimental opt-in feature that can only be enabled by setting rebase.useBuiltIn configuration that defaults to false.

   cf. <[email protected]>

If we don't set rebase.useBuiltin to false, then there is also a minor regression in the error message printed by the built-in rebase we may want to try to address. I have a patch for it at <[email protected]>, but I don't know if that patch is acceptable as-is this close to a release since that'd not give translators much time to update.

#96 lore raw

Re: [ANNOUNCE] Git v2.20.0-rc1

Quoted 6 lines · Elijah Newren

If we don't set rebase.useBuiltin to false, then there is also a minor regression in the error message printed by the built-in rebase we may want to try to address. I have a patch for it at <[email protected]>, but I don't know if that patch is acceptable as-is this close to a release since that'd not give translators much time to update.

For this particular one, I'd rather ship "rebase in C" with known message glitch, with or without the "mark it experimental and make it opt-in" last-time change. Of course, turning it off by default would let us worry about these message glitches even less, but at this point, I'd be worried more about bugs that can affect the actual operation and outcome recorded in the resulting history.