Skip to contentSkip to conversation

color-moved: ignore all space changes by default

PATCH In review Stefan Beller 3 participants 6 messages Oct 25, 2017 – Oct 26, 2017 Documentation

lore
Share & tools
lore
#0 lore raw

cover color-moved: ignore all space changes by default

On Mon, Oct 23, 2017 at 7:52 PM, Stefan Beller <[email protected]> wrote[1]:

Quoted 9 lines · Junio C Hamano
Quoted 4 lines

* As moved-lines display is mostly a presentation thing, I wonder if it makes sense to always match loosely wrt whitespace differences.

Well, sometimes the user wants to know if it is byte-for-byte identical (unlikely to be code, but maybe column oriented data for input; think of all our FORTRAN users. ;)

... and this is the implementation and the flip of the default setting to ignore all white space for the move detection.

It applies on "v3 (x)diff cleanup: remove duplicate code" [2].

Thanks,
Stefan

[1] https://public-inbox.org/git/CAGZ79kbba9KuDX7HsxhW3jXs7ocWfZg=KSHE-GsXjntnT4PWdg@mail.gmail.com
[2] https://public-inbox.org/git/[email protected]/

Stefan Beller (2):
diff: decouple white space treatment for move detection from generic
option
diff.c: ignore all white space changes by default in the move
detection

 Documentation/diff-options.txt |  15 ++++
 diff.c                         |  18 ++++-
 diff.h                         |   1 +
 t/t4015-diff-whitespace.sh     | 156 +++++++++++++++++++++++++++++++++++++++--
 4 files changed, 181 insertions(+), 9 deletions(-)
Signature · 1 line
2.15.0.rc2.6.g953226eb5f
#1 lore raw

PATCH 1/2 diff: decouple white space treatment for move detection from generic option

In the original implementation of the move dection logic we assumed that the choice for ignoring white space changes is the same for the move detection as it is for the generic diff.

It turns out, users want to have different choices regarding white spaces for the move detection and the generic diff. For example when reviewing a patch that is sent to the mailing list (which doesn't ignore white space changes at all; it is an exact patch) refactors some nested code out into a separate function, we want to have the move detection kick in despite different indentation levels of the old and new code.

This patch enables the user to have a different choice for ignoring white spaces in the move detection.

The example given is covered in a test. Convert the existing tests to be more explicit on their choice of white space behavior in the move detection as we'll change the default shortly.

---
 Documentation/diff-options.txt |  13 ++++
 diff.c                         |  17 ++++-
 diff.h                         |   1 +
 t/t4015-diff-whitespace.sh     | 150 +++++++++++++++++++++++++++++++++++++++--
 4 files changed, 172 insertions(+), 9 deletions(-)
Documentation/diff-options.txt +13 -0
index a88c76741e..1000b53b84 100644
@@ -265,6 +265,19 @@ dimmed_zebra::
 	blocks are considered interesting, the rest is uninteresting.
 --
 
+--color-moved-[no-]ignore-all-space::
+	Ignore whitespace when comparing lines when performing the move
+	detection for --color-moved.  This ignores differences even if
+	one line has whitespace where the other line has none.
+--color-moved-[no-]ignore-space-change::
+	Ignore changes in amount of whitespace when performing the move
+	detection for --color-moved.  This ignores whitespace
+	at line end, and considers all other sequences of one or
+	more whitespace characters to be equivalent.
+--color-moved-[no-]ignore-space-at-eol::
+	Ignore changes in whitespace at EOL when performing the move
+	detection for --color-moved.
+
 --word-diff[=<mode>]::
 	Show a word diff, using the <mode> to delimit changed words.
 	By default, words are delimited by whitespace; see
diff.c +15 -2
index e6814b9e9c..2580315ab9 100644
@@ -714,7 +714,7 @@ static int moved_entry_cmp(const struct diff_options *diffopt,
 {
 	return !xdiff_compare_lines(a->es->line, a->es->len,
 				    b->es->line, b->es->len,
-				    diffopt->xdl_opts);
+				    diffopt->color_moved_ignore_space);
 }
 
 static struct moved_entry *prepare_entry(struct diff_options *o,
@@ -723,7 +723,8 @@ static struct moved_entry *prepare_entry(struct diff_options *o,
 	struct moved_entry *ret = xmalloc(sizeof(*ret));
 	struct emitted_diff_symbol *l = &o->emitted_symbols->buf[line_no];
 
-	ret->ent.hash = xdiff_hash_string(l->line, l->len, o->xdl_opts);
+	ret->ent.hash = xdiff_hash_string(l->line, l->len,
+					  o->color_moved_ignore_space);
 	ret->es = l;
 	ret->next_line = NULL;
 
@@ -4592,6 +4593,18 @@ int diff_opt_parse(struct diff_options *options,
 		DIFF_XDL_SET(options, IGNORE_WHITESPACE_AT_EOL);
 	else if (!strcmp(arg, "--ignore-blank-lines"))
 		DIFF_XDL_SET(options, IGNORE_BLANK_LINES);
+	else if (!strcmp(arg, "--color-moved-no-ignore-all-space"))
+		options->color_moved_ignore_space &= ~XDF_IGNORE_WHITESPACE;
+	else if (!strcmp(arg, "--color-moved-no-ignore-space-change"))
+		options->color_moved_ignore_space &= ~XDF_IGNORE_WHITESPACE_CHANGE;
+	else if (!strcmp(arg, "--color-moved-no-ignore-space-at-eol"))
+		options->color_moved_ignore_space &= ~XDF_IGNORE_WHITESPACE_AT_EOL;
+	else if (!strcmp(arg, "--color-moved-ignore-all-space"))
+		options->color_moved_ignore_space |= XDF_IGNORE_WHITESPACE;
+	else if (!strcmp(arg, "--color-moved-ignore-space-change"))
+		options->color_moved_ignore_space |= XDF_IGNORE_WHITESPACE_CHANGE;
+	else if (!strcmp(arg, "--color-moved-ignore-space-at-eol"))
+		options->color_moved_ignore_space |= XDF_IGNORE_WHITESPACE_AT_EOL;
 	else if (!strcmp(arg, "--indent-heuristic"))
 		DIFF_XDL_SET(options, INDENT_HEURISTIC);
 	else if (!strcmp(arg, "--no-indent-heuristic"))
diff.h +1 -0
index aca150ba2e..6ba3f53bbd 100644
@@ -196,6 +196,7 @@ struct diff_options {
 	} color_moved;
 	#define COLOR_MOVED_DEFAULT COLOR_MOVED_ZEBRA
 	#define COLOR_MOVED_MIN_ALNUM_COUNT 20
+	int color_moved_ignore_space;
 };
 
 void diff_emit_submodule_del(struct diff_options *o, const char *line);
t/t4015-diff-whitespace.sh +143 -7

192 lines not on this page · raw message

Signature · 1 line
2.15.0.rc2.6.g953226eb5f
#2 lore raw

PATCH 2/2 diff.c: ignore all white space changes by default in the move detection

In reply to Stefan Beller

The coloring of moved lines is currently only related to the presentation, as there are no options to export the move detection information into the patch format. Hence we can be very loose about the default, so let's ignore any white space change for the move detection. If a user really cares about move detection detecting only lines moved byte-for-byte, they can exercise the --color-moved-no-ignore-* options.

---
 Documentation/diff-options.txt | 4 +++-
 diff.c                         | 1 +
 t/t4015-diff-whitespace.sh     | 6 ++++++
 3 files changed, 10 insertions(+), 1 deletion(-)
Documentation/diff-options.txt +3 -1
index 1000b53b84..fdf40cbefc 100644
@@ -269,14 +269,16 @@ dimmed_zebra::
 	Ignore whitespace when comparing lines when performing the move
 	detection for --color-moved.  This ignores differences even if
 	one line has whitespace where the other line has none.
+	This is on by default.
 --color-moved-[no-]ignore-space-change::
 	Ignore changes in amount of whitespace when performing the move
 	detection for --color-moved.  This ignores whitespace
 	at line end, and considers all other sequences of one or
 	more whitespace characters to be equivalent.
+	This is on by default.
 --color-moved-[no-]ignore-space-at-eol::
 	Ignore changes in whitespace at EOL when performing the move
-	detection for --color-moved.
+	detection for --color-moved. This is on by default.
 
 --word-diff[=<mode>]::
 	Show a word diff, using the <mode> to delimit changed words.
diff.c +1 -0
index 2580315ab9..fef4874169 100644
@@ -4105,6 +4105,7 @@ void diff_setup(struct diff_options *options)
 	}
 
 	options->color_moved = diff_color_moved_default;
+	options->color_moved_ignore_space = XDF_WHITESPACE_FLAGS;
 }
 
 void diff_setup_done(struct diff_options *options)
t/t4015-diff-whitespace.sh +6 -0
index 4ef4d6934a..a4ffc84f41 100755
@@ -1813,6 +1813,12 @@ test_expect_success 'move detection only ignores white spaces' '
 	 Qbar();<RESET>
 	 Q// more unrelated stuff<RESET>
 	EOF
+	test_cmp expected actual &&
+
+	# test that this is the default:
+	git diff --color --color-moved |
+		grep -v "index" |
+		test_decode_color >actual &&
 	test_cmp expected actual
 '
 
Signature · 1 line
2.15.0.rc2.6.g953226eb5f
#3 lore raw
In reply to Stefan Beller
Quoted 7 lines · Stefan Beller

On Mon, Oct 23, 2017 at 7:52 PM, Stefan Beller wrote[1]:

Quoted 6 lines · Junio C Hamano
Quoted 4 lines

* As moved-lines display is mostly a presentation thing, I wonder if it makes sense to always match loosely wrt whitespace differences.

Quoted 6 lines
Quoted 3 lines

Well, sometimes the user wants to know if it is byte-for-byte identical (unlikely to be code, but maybe column oriented data for input; think of all our FORTRAN users. ;)

... and this is the implementation and the flip of the default setting to ignore all white space for the move detection.

Hello,

I'm not sure if this is a good default. I think it's not obvious that moved code gets treated differently than regular changes. I wouldn't expect git diff to ignore whitespace changes (without me telling it to) and so when I see moved code I expect they were moved as is.

And there are languages where indentation is relevant (e.g. Python, YAML) and as color-moved is also treated as review tool to detect unwanted changes this new default can be dangerous.

The new options sound like a good addition but I don't think the defaults should change. However unrelated to this decision, please add config settings in addition to these new options so users can globally configure the behavior they want.

Regards
Simon

Signature · 3 lines
+ privacy is necessary
+ using gnupg http://gnupg.org
+ public key id: 0x92FEFDB7E44C32F9
#4 lore raw
Quoted 27 lines · Simon Ruderich
Quoted 7 lines · Stefan Beller

On Mon, Oct 23, 2017 at 7:52 PM, Stefan Beller wrote[1]:

Quoted 6 lines · Junio C Hamano
Quoted 4 lines

* As moved-lines display is mostly a presentation thing, I wonder if it makes sense to always match loosely wrt whitespace differences.

Quoted 6 lines
Quoted 3 lines

Well, sometimes the user wants to know if it is byte-for-byte identical (unlikely to be code, but maybe column oriented data for input; think of all our FORTRAN users. ;)

... and this is the implementation and the flip of the default setting to ignore all white space for the move detection.

Hello,

I'm not sure if this is a good default. I think it's not obvious that moved code gets treated differently than regular changes. I wouldn't expect git diff to ignore whitespace changes (without me telling it to) and so when I see moved code I expect they were moved as is.

And there are languages where indentation is relevant (e.g. Python, YAML) and as color-moved is also treated as review tool to detect unwanted changes this new default can be dangerous.

Quoted 8 lines

The new options sound like a good addition but I don't think the defaults should change. However unrelated to this decision, please add config settings in addition to these new options so users can globally configure the behavior they want.

Regards
Simon
--

Even languages which are indentation sensitive often move blocks of lines between indentation levels a lot. I personally think the default could change.

However, I would suspect the best path forward is leave the default "exact match" and allow users who care and know about the feature to change their config settings.

Thanks,
Jake

#5 lore raw
Quoted 9 lines · Jacob Keller
Quoted 9 lines

Hello,

I'm not sure if this is a good default. I think it's not obvious that moved code gets treated differently than regular changes. I wouldn't expect git diff to ignore whitespace changes (without me telling it to) and so when I see moved code I expect they were moved as is.

Quoted 3 lines
Quoted 3 lines

And there are languages where indentation is relevant (e.g. Python, YAML) and as color-moved is also treated as review tool to detect unwanted changes this new default can be dangerous.

That makes sense.

Quoted 6 lines
Quoted 6 lines

The new options sound like a good addition but I don't think the defaults should change. However unrelated to this decision, please add config settings in addition to these new options so users can globally configure the behavior they want.

Quoted 7 lines
Quoted 3 lines

Regards
Simon
--

Even languages which are indentation sensitive often move blocks of lines between indentation levels a lot. I personally think the default could change.

A safe default for such languages would be when the
change in whitespace across lines is taking into account, i.e. when lots of lines are copied, but all of them miss two tab indentation, then it is probably fine to color it all the same. However if there would be a couple of lines in that moved block of code that have
a different indentation than most other lines, this could indeed change the program flow in python.

So ideally we'd compare the whitespace delta across lines. Note sure if that is easy. Maybe instead of ignoring whitespaces completely we'd rather make up a "white space delta", e.g. by using word-diff between the old and new line, and then keeping a hash of that space delta for each line. The move detection would then also compare the hashes of adjacent moved lines.

Quoted 3 lines

However, I would suspect the best path forward is leave the default "exact match" and allow users who care and know about the feature to change their config settings.

I think that is what I'll do for now as it seems simple enough.

Thanks,
Stefan