Skip to contentSkip to conversation

git-diff: must --exit-code work with --ignore* options?

DISCUSSION Jim Meyering 3 participants 9 messages Jun 15, 2016

lore
Share & tools
lore
#0 lore raw

git-diff's --quiet option works how I'd expect with --ignore-space-at-eol as long as I'm also using --no-index:

    $ echo>b; echo \ >c; git diff --no-index --quiet --ignore-space-at-eol b c \
      && echo good
    good

But in what I think of as normal operation (i.e., without --no-index), --exit-code (or --quiet) makes git-diff say there were differences, even when they have been ignored:

    # do this in an empty directory
    $ git init -q; echo>k; git add .; git commit -q -m. .; echo \ >k
    $ git diff --ignore-space-at-eol --quiet || echo bad
    bad

Same problem with --ignore-space-change.

-------------------
In the surprising case, builtin-diff.c's builtin_diff_files calls diff_result_code, which returns nonzero due to this:

          if (diff_queued_diff.nr)
                  DIFF_OPT_SET(options, HAS_CHANGES);
          else
                  DIFF_OPT_CLR(options, HAS_CHANGES);

However, the queued diffs may contain only ignorable changes.

With --no-index, it takes a different code path and uses diffopt.found_changes to produce the desired exit status.

#1 lore raw
Quoted 7 lines · Jim Meyering

git-diff's --quiet option works how I'd expect with --ignore-space-at-eol as long as I'm also using --no-index:

    $ echo>b; echo \ >c; git diff --no-index --quiet --ignore-space-at-eol b c \
      && echo good
    good
Quoted 8 lines

But in what I think of as normal operation (i.e., without --no-index), --exit-code (or --quiet) makes git-diff say there were differences, even when they have been ignored:

    # do this in an empty directory
    $ git init -q; echo>k; git add .; git commit -q -m. .; echo \ >k
    $ git diff --ignore-space-at-eol --quiet || echo bad
    bad

I am slightly torn about this, in that I can picture myself saying that this is unintuitive on some different days, but not today ;-)

If you look at the output (i.e. no --quiet), you would see that the blob changes are still reported for the path. E.g. you would see something like...

	$ git diff --ignore-space-at-eol
        diff --git a/k b/k
        index 8b13789..8d1c8b6 100644

The "index" line is still showing that there _is_ a difference.

The --ignore-* options are there merely to tell git what changes are not worth _showing_ in the textual part of the patch, in order to cut down the amount of the output. It never affects the outcome.

So if anything, I think --no-index codepath is what's buggy; if it does not report the blob difference that is a different matter, though.

#2 lore raw
Quoted 12 lines · Junio C Hamano
Quoted 11 lines · Jim Meyering

git-diff's --quiet option works how I'd expect with --ignore-space-at-eol as long as I'm also using --no-index:

    $ echo>b; echo \ >c; git diff --no-index --quiet --ignore-space-at-eol b c \
      && echo good
    good

But in what I think of as normal operation (i.e., without --no-index), --exit-code (or --quiet) makes git-diff say there were differences, even when they have been ignored:

Quoted 7 lines
Quoted 4 lines
    # do this in an empty directory
    $ git init -q; echo>k; git add .; git commit -q -m. .; echo \ >k
    $ git diff --ignore-space-at-eol --quiet || echo bad
    bad

I am slightly torn about this, in that I can picture myself saying that this is unintuitive on some different days, but not today ;-)

Thanks for the quick reply. Here's why I noticed:

I wanted to ensure that the only changes induced by commit C were to trailing blanks. I wrote something like this, expecting to be able to deal with the exception:

    git --quiet --ignore-space-at-eol --quiet C^..C || handle_unexpected

But handle_unexpected was always being invoked.
I was surprised because GNU diff's --ignore-all-space (-w) option does work the way I expected:

    $ echo>b; echo \ >c; diff -w b c && echo $?
    0
Quoted 9 lines

If you look at the output (i.e. no --quiet), you would see that the blob changes are still reported for the path. E.g. you would see something like...

	$ git diff --ignore-space-at-eol
        diff --git a/k b/k
        index 8b13789..8d1c8b6 100644

The "index" line is still showing that there _is_ a difference.

I did see that, to my chagrin:
if using a --ignore-... option had also suppressed those, I could have tested for empty output instead of exit status.

Quoted 6 lines

The --ignore-* options are there merely to tell git what changes are not worth _showing_ in the textual part of the patch, in order to cut down the amount of the output. It never affects the outcome.

So if anything, I think --no-index codepath is what's buggy; if it does not report the blob difference that is a different matter, though.

If need be, I can work around it.

#3 lore raw
Quoted 8 lines · Jim Meyering
Quoted 7 lines · Junio C Hamano
Quoted 5 lines · Jim Meyering
    # do this in an empty directory
    $ git init -q; echo>k; git add .; git commit -q -m. .; echo \ >k
    $ git diff --ignore-space-at-eol --quiet || echo bad
    bad
Quoted 5 lines
Quoted 2 lines

I am slightly torn about this, in that I can picture myself saying that this is unintuitive on some different days, but not today ;-)

Thanks for the quick reply. Here's why I noticed:
...

It seems that today is already "some different day" ;-) We could do something like this patch.

While in the longer term I think it may make the world a better place by being more consistent with what users expect, I am not sure at what revision boundary we should introduce such a semantic change.

We could always declare this a bug and apply the "fix" at any time. It's all perception ;-).

-- >8 --
Subject: [PATCH] diff --quiet: special case "ignore whitespace" options

The option "QUIET" primarily meant "find if we have _any_ difference as quick as possible and report", which means we often do not even have to look at blobs if we know the trees are different by looking at the higher level (e.g. "diff-tree A B"). As a side effect, because there is no point showing one change that we happened to have found first, it also enables NO_OUTPUT and EXIT_WITH_STATUS options, making the end result look quiet.

Traditionally, the --ignore-whitespace* options have merely meant to tell the diff output routine that some class of differences are not worth showing in the textual diff output, so that the end user has easier time to review the remaining (presumably more meaningful) changes. These options never affected the outcome of the command, given as the exit status when the --exit-code option was in effect (either directly or indirectly).

These two classes of options are incompatible. When you have only whitespace changes, you would expect:

	git diff -b --quiet

to report that there is _no_ change. This is unfortunately not the case, however, if there are differences to be reported if the command was run without --quiet; there _is_ a change, and the command still exits with non-zero status.

And that is wrong.

Change the semantics of --ignore-whitespace* options to mean more than "omit showing the difference in text". When these options are used, the internal "quick" optimization is turned off, and the status reported with the --exit-code option will now match if any the textual diff output is actually produced.

Also rename the internal option "QUIET" to "QUICK" to better reflect what its true purpose is.

---
 builtin-log.c      |    2 +-
 builtin-rev-list.c |    2 +-
 diff-lib.c         |    4 ++--
 diff.c             |   39 ++++++++++++++++++++++++++++++++++++---
 diff.h             |    3 ++-
 revision.c         |    2 +-
 tree-diff.c        |    2 +-
 7 files changed, 44 insertions(+), 10 deletions(-)
builtin-log.c +1 -1
index 5eaec5d..80624f5 100644
@@ -536,7 +536,7 @@ static int reopen_stdout(struct commit *commit, struct rev_info *rev)
 
 	get_patch_filename(commit, rev->nr, fmt_patch_suffix, &filename);
 
-	if (!DIFF_OPT_TST(&rev->diffopt, QUIET))
+	if (!DIFF_OPT_TST(&rev->diffopt, QUICK))
 		fprintf(realstdout, "%s\n", filename.buf + outdir_offset);
 
 	if (freopen(filename.buf, "w", stdout) == NULL)
builtin-rev-list.c +1 -1
index 38a8f23..61d3126 100644
@@ -315,7 +315,7 @@ int cmd_rev_list(int argc, const char **argv, const char *prefix)
 	memset(&info, 0, sizeof(info));
 	info.revs = &revs;
 
-	quiet = DIFF_OPT_TST(&revs.diffopt, QUIET);
+	quiet = DIFF_OPT_TST(&revs.diffopt, QUICK);
 	for (i = 1 ; i < argc; i++) {
 		const char *arg = argv[i];
 
diff-lib.c +2 -2
index a310fb2..a549ee6 100644
@@ -73,7 +73,7 @@ int run_diff_files(struct rev_info *revs, unsigned int option)
 		struct cache_entry *ce = active_cache[i];
 		int changed;
 
-		if (DIFF_OPT_TST(&revs->diffopt, QUIET) &&
+		if (DIFF_OPT_TST(&revs->diffopt, QUICK) &&
 			DIFF_OPT_TST(&revs->diffopt, HAS_CHANGES))
 			break;
 
@@ -520,7 +520,7 @@ int index_differs_from(const char *def, int diff_flags)
 
 	init_revisions(&rev, NULL);
 	setup_revisions(0, NULL, &rev, def);
-	DIFF_OPT_SET(&rev.diffopt, QUIET);
+	DIFF_OPT_SET(&rev.diffopt, QUICK);
 	DIFF_OPT_SET(&rev.diffopt, EXIT_WITH_STATUS);
 	rev.diffopt.flags |= diff_flags;
 	run_diff_index(&rev, 1);
diff.c +36 -3
index f06876b..f2ed2ac 100644
@@ -2370,6 +2370,21 @@ int diff_setup_done(struct diff_options *options)
 	if (count > 1)
 		die("--name-only, --name-status, --check and -s are mutually exclusive");
 
+	/*
+	 * Most of the time we can say "there are changes"
+	 * only by checking if there are changed paths, but
+	 * --ignore-whitespace* options force us to look
+	 * inside contets.
+	 */
+
+	if ((XDF_IGNORE_WHITESPACE|
+	     XDF_IGNORE_WHITESPACE_CHANGE|
+	     XDF_IGNORE_WHITESPACE_AT_EOL) & options->xdl_opts) {
+		DIFF_OPT_SET(options, DIFF_FROM_CONTENTS);
+	} else {
+		DIFF_OPT_CLR(options, DIFF_FROM_CONTENTS);
+	}
+
 	if (DIFF_OPT_TST(options, FIND_COPIES_HARDER))
 		options->detect_rename = DIFF_DETECT_COPY;
 
@@ -2430,9 +2445,19 @@ int diff_setup_done(struct diff_options *options)
 	 * to have found.  It does not make sense not to return with
 	 * exit code in such a case either.
 	 */
-	if (DIFF_OPT_TST(options, QUIET)) {
+	if (DIFF_OPT_TST(options, QUICK)) {
 		options->output_format = DIFF_FORMAT_NO_OUTPUT;
 		DIFF_OPT_SET(options, EXIT_WITH_STATUS);
+
+		/*
+		 * QUICK means "if we find any difference return early
+		 * and say 'there is a difference'", and we often do
+		 * not even look at the blobs.  Some options would not
+		 * be compatible with this optimization, so we turn it
+		 * off, make it into "no output but exit with status".
+		 */
+		if (DIFF_OPT_TST(options, DIFF_FROM_CONTENTS))
+			DIFF_OPT_CLR(options, QUICK);
 	}
 
 	return 0;
@@ -2621,7 +2646,8 @@ int diff_opt_parse(struct diff_options *options, const char **av, int ac)
 	else if (!strcmp(arg, "--exit-code"))
 		DIFF_OPT_SET(options, EXIT_WITH_STATUS);
 	else if (!strcmp(arg, "--quiet"))
-		DIFF_OPT_SET(options, QUIET);
+		/* see postprocessing in diff_setup_done() */
+		DIFF_OPT_SET(options, QUICK);
 	else if (!strcmp(arg, "--ext-diff"))
 		DIFF_OPT_SET(options, ALLOW_EXTERNAL);
 	else if (!strcmp(arg, "--no-ext-diff"))
@@ -3322,6 +3348,13 @@ free_queue:
 	q->nr = q->alloc = 0;
 	if (options->close_file)
 		fclose(options->file);
+
+	if (DIFF_OPT_TST(options, DIFF_FROM_CONTENTS)) {
+		if (options->found_changes)
+			DIFF_OPT_SET(options, HAS_CHANGES);
+		else
+			DIFF_OPT_CLR(options, HAS_CHANGES);
+	}
 }
 
 static void diffcore_apply_filter(const char *filter)
@@ -3458,7 +3491,7 @@ void diffcore_std(struct diff_options *options)
 	diff_resolve_rename_copy();
 	diffcore_apply_filter(options->filter);
 
-	if (diff_queued_diff.nr)
+	if (diff_queued_diff.nr && !DIFF_OPT_TST(options, DIFF_FROM_CONTENTS))
 		DIFF_OPT_SET(options, HAS_CHANGES);
 	else
 		DIFF_OPT_CLR(options, HAS_CHANGES);
diff.h +2 -1
index 6616877..a7e7ccb 100644
@@ -55,7 +55,7 @@ typedef void (*diff_format_fn_t)(struct diff_queue_struct *q,
 #define DIFF_OPT_COLOR_DIFF          (1 <<  8)
 #define DIFF_OPT_COLOR_DIFF_WORDS    (1 <<  9)
 #define DIFF_OPT_HAS_CHANGES         (1 << 10)
-#define DIFF_OPT_QUIET               (1 << 11)
+#define DIFF_OPT_QUICK               (1 << 11)
 #define DIFF_OPT_NO_INDEX            (1 << 12)
 #define DIFF_OPT_ALLOW_EXTERNAL      (1 << 13)
 #define DIFF_OPT_EXIT_WITH_STATUS    (1 << 14)
@@ -66,6 +66,7 @@ typedef void (*diff_format_fn_t)(struct diff_queue_struct *q,
 #define DIFF_OPT_DIRSTAT_CUMULATIVE  (1 << 19)
 #define DIFF_OPT_DIRSTAT_BY_FILE     (1 << 20)
 #define DIFF_OPT_ALLOW_TEXTCONV      (1 << 21)
+#define DIFF_OPT_DIFF_FROM_CONTENTS  (1 << 22)
 #define DIFF_OPT_TST(opts, flag)    ((opts)->flags & DIFF_OPT_##flag)
 #define DIFF_OPT_SET(opts, flag)    ((opts)->flags |= DIFF_OPT_##flag)
 #define DIFF_OPT_CLR(opts, flag)    ((opts)->flags &= ~DIFF_OPT_##flag)
revision.c +1 -1
index 18b7ebb..1c114ab 100644
@@ -800,7 +800,7 @@ void init_revisions(struct rev_info *revs, const char *prefix)
 	revs->ignore_merges = 1;
 	revs->simplify_history = 1;
 	DIFF_OPT_SET(&revs->pruning, RECURSIVE);
-	DIFF_OPT_SET(&revs->pruning, QUIET);
+	DIFF_OPT_SET(&revs->pruning, QUICK);
 	revs->pruning.add_remove = file_add_remove;
 	revs->pruning.change = file_change;
 	revs->lifo = 1;
tree-diff.c +1 -1
index edd8394..ac85a55 100644
@@ -280,7 +280,7 @@ int diff_tree(struct tree_desc *t1, struct tree_desc *t2, const char *base, stru
 	int baselen = strlen(base);
 
 	for (;;) {
-		if (DIFF_OPT_TST(opt, QUIET) && DIFF_OPT_TST(opt, HAS_CHANGES))
+		if (DIFF_OPT_TST(opt, QUICK) && DIFF_OPT_TST(opt, HAS_CHANGES))
 			break;
 		if (opt->nr_paths) {
 			skip_uninteresting(t1, base, baselen, opt);
Signature · 1 line
1.6.3.1.70.ga80aa
#4 lore raw
Quoted 18 lines · Junio C Hamano
Quoted 8 lines · Jim Meyering
Quoted 7 lines · Junio C Hamano
Quoted 5 lines · Jim Meyering
    # do this in an empty directory
    $ git init -q; echo>k; git add .; git commit -q -m. .; echo \ >k
    $ git diff --ignore-space-at-eol --quiet || echo bad
    bad
Quoted 5 lines
Quoted 2 lines

I am slightly torn about this, in that I can picture myself saying that this is unintuitive on some different days, but not today ;-)

Thanks for the quick reply. Here's why I noticed:
...

It seems that today is already "some different day" ;-) We could do something like this patch.

Quoted 7 lines

While in the longer term I think it may make the world a better place by being more consistent with what users expect, I am not sure at what revision boundary we should introduce such a semantic change.

-- >8 --
Subject: [PATCH] diff --quiet: special case "ignore whitespace" options
...

Wow. And now a patch. Service with style ;-)

Quoted 2 lines

We could always declare this a bug and apply the "fix" at any time. It's all perception ;-).

The declare-it-a-bug option sounds sensible, since I doubt anyone even noticed, much less relied on, the changing behavior.

Thank you!

#5 lore raw
In reply to Junio C Hamano
Quoted 55 lines · Junio C Hamano
Quoted 8 lines · Jim Meyering
Quoted 7 lines · Junio C Hamano
Quoted 5 lines · Jim Meyering
    # do this in an empty directory
    $ git init -q; echo>k; git add .; git commit -q -m. .; echo \ >k
    $ git diff --ignore-space-at-eol --quiet || echo bad
    bad
Quoted 5 lines
Quoted 2 lines

I am slightly torn about this, in that I can picture myself saying that this is unintuitive on some different days, but not today ;-)

Thanks for the quick reply. Here's why I noticed:
...

It seems that today is already "some different day" ;-) We could do something like this patch.

While in the longer term I think it may make the world a better place by being more consistent with what users expect, I am not sure at what revision boundary we should introduce such a semantic change.

We could always declare this a bug and apply the "fix" at any time. It's all perception ;-).

-- >8 --
Subject: [PATCH] diff --quiet: special case "ignore whitespace" options

The option "QUIET" primarily meant "find if we have _any_ difference as quick as possible and report", which means we often do not even have to look at blobs if we know the trees are different by looking at the higher level (e.g. "diff-tree A B"). As a side effect, because there is no point showing one change that we happened to have found first, it also enables NO_OUTPUT and EXIT_WITH_STATUS options, making the end result look quiet.

Traditionally, the --ignore-whitespace* options have merely meant to tell the diff output routine that some class of differences are not worth showing in the textual diff output, so that the end user has easier time to review the remaining (presumably more meaningful) changes. These options never affected the outcome of the command, given as the exit status when the --exit-code option was in effect (either directly or indirectly).

These two classes of options are incompatible. When you have only whitespace changes, you would expect:

	git diff -b --quiet

to report that there is _no_ change. This is unfortunately not the case, however, if there are differences to be reported if the command was run without --quiet; there _is_ a change, and the command still exits with non-zero status.

And that is wrong.

Quoted 8 lines

Change the semantics of --ignore-whitespace* options to mean more than "omit showing the difference in text". When these options are used, the internal "quick" optimization is turned off, and the status reported with the --exit-code option will now match if any the textual diff output is actually produced.

Also rename the internal option "QUIET" to "QUICK" to better reflect what its true purpose is.

Thanks again.
If there's anything I can to do help (add a test?), let me know.

#6 lore raw
Quoted 10 lines · Jim Meyering

Junio C Hamano wrote:
...

Quoted 8 lines

Subject: [PATCH] diff --quiet: special case "ignore whitespace" options
...
Change the semantics of --ignore-whitespace* options to mean more than "omit showing the difference in text". When these options are used, the internal "quick" optimization is turned off, and the status reported with the --exit-code option will now match if any the textual diff output is actually produced.

Quoted 5 lines
Quoted 2 lines

Also rename the internal option "QUIET" to "QUICK" to better reflect what its true purpose is.

Thanks again.
If there's anything I can to do help (add a test?), let me know.

The change has been cooking in 'next' and hopefully be in 1.7.0. I think the updated series adds its own test script, too.

Using it in every day scenario, and reporting any breakage you notice before 1.7.0 happens, would be greatly appreciated.

Thanks.

#7 lore raw
Quoted 14 lines · Junio C Hamano
Quoted 13 lines · Jim Meyering

Junio C Hamano wrote:
...

Quoted 10 lines

Subject: [PATCH] diff --quiet: special case "ignore whitespace" options
...
Change the semantics of --ignore-whitespace* options to mean more than "omit showing the difference in text". When these options are used, the internal "quick" optimization is turned off, and the status reported with the --exit-code option will now match if any the textual diff output is actually produced.

Also rename the internal option "QUIET" to "QUICK" to better reflect what its true purpose is.

Quoted 8 lines
Quoted 2 lines

Thanks again.
If there's anything I can to do help (add a test?), let me know.

The change has been cooking in 'next' and hopefully be in 1.7.0. I think the updated series adds its own test script, too.

Using it in every day scenario, and reporting any breakage you notice before 1.7.0 happens, would be greatly appreciated.

Oh! I am using next (will test!), and even searched log summary output, but obviously my search was too cursory or just inaccurate.

I glanced through it and it looks fine (of course!).
I spotted one typo, and suggest a second change that's barely worth mentioning, both in comments:

diff.c +2 -2
index 91d6ea2..24bd3fc 100644
@@ -2382,7 +2382,7 @@ int diff_setup_done(struct diff_options *options)
 	 * Most of the time we can say "there are changes"
 	 * only by checking if there are changed paths, but
 	 * --ignore-whitespace* options force us to look
-	 * inside contets.
+	 * inside contents.
 	 */
 
 	if (DIFF_XDL_TST(options, IGNORE_WHITESPACE) ||
@@ -3346,7 +3346,7 @@ free_queue:
 		fclose(options->file);
 
 	/*
-	 * Report the contents level differences with HAS_CHANGES;
+	 * Report the content-level differences with HAS_CHANGES;
 	 * diff_addremove/diff_change does not set the bit when
 	 * DIFF_FROM_CONTENTS is in effect (e.g. with -w).
 	 */
#8 lore raw
In reply to Junio C Hamano
Quoted 18 lines · Junio C Hamano
Quoted 10 lines · Jim Meyering

Junio C Hamano wrote:
...

Quoted 8 lines

Subject: [PATCH] diff --quiet: special case "ignore whitespace" options
...
Change the semantics of --ignore-whitespace* options to mean more than "omit showing the difference in text". When these options are used, the internal "quick" optimization is turned off, and the status reported with the --exit-code option will now match if any the textual diff output is actually produced.

Quoted 5 lines
Quoted 2 lines

Also rename the internal option "QUIET" to "QUICK" to better reflect what its true purpose is.

Thanks again.
If there's anything I can to do help (add a test?), let me know.

Quoted 7 lines

The change has been cooking in 'next' and hopefully be in 1.7.0. I think the updated series adds its own test script, too.

Using it in every day scenario, and reporting any breakage you notice before 1.7.0 happens, would be greatly appreciated.

Thanks.

Perhaps I'm expected something different than what I _should_ be expecting, but shouldn't --quiet always return the same as --exit-code?

# Cut/Paste example
mkdir test_ws_quiet && cd test_ws_quiet && git init printf "foo bar \n\n" >f1.txt
git add .
git commit -m 'f text'
printf "foo bar\n\n" >f1.txt
git commit -a -m 'f with diff white-space in middle & end' git diff -w --exit-code HEAD^ >/dev/null
echo $?

# returns '0' which it should
git diff -w --quiet HEAD^
echo $?
# returns '0' which it should
git diff -b --exit-code HEAD^ >/dev/null
echo $?
# returns '0' which it should
git diff -b --quiet HEAD^ >/dev/null
echo $?
# returns '0' which it should
git diff --ignore-space-at-eol --exit-code HEAD^ >/dev/null
echo $?
# returns '1' which it should
git diff --ignore-space-at-eol --quiet HEAD^
echo $?
#returns '0' <=== Unexpected.

#
# Next phase
#
printf "foobar\n\n">f1.txt
git commit -a -m 'f without any spaces'
git diff -w --exit-code HEAD^ >/dev/null
echo $?

# returns '0' which it should
git diff -w --quiet HEAD^
echo $?
# returns '0' which it should
git diff -b --exit-code HEAD^ >/dev/null
echo $?
# returns '1' which it should
git diff -b --quiet HEAD^ >/dev/null
echo $?
# returns '0' <=== Unexpected
git diff --ignore-space-at-eol --exit-code HEAD^ >/dev/null
echo $?
# returns '1' which it should
git diff --ignore-space-at-eol --quiet HEAD^
echo $?
#returns '0' <=== Unexpected.
Signature · 1 line
Thell