Fixes for undefined behaviour

I've been playing with Clang's undefined behaviour sanitizer, which points out a few potential issues in Git when running the test suite (it's a runtime analysis that is compiled in by setting suitable CFLAGS).

These patches fix one issue that I think we need to worry about and one that's trivial to fix.

The remaining warnings are:

refs.c:2426:17: runtime error: index -1 out of bounds for type 'char [8192]'

Caused by a loop walking backwards over the reflog which sets its scan pointer to be one before the start of the buffer in order to break out of the loop. It seems unlikely that the (stack allocated) buffer will be at address zero so I don't think any sane compiler will cause us problems here.

tag.c:104:40: runtime error: member access within null pointer of type 'struct commit'

This does "&lookup_commit(sha1)->object" which ends up being okay because "object" is the first item in struct commit. I'm not sure it's worth the churn to change this.

xdiff/xutils.c:308:7: runtime error: load of misaligned address for type 'unsigned long', which requires 8 byte alignment

This is in the XDL_FAST_HASH code, which should only be used on architectures where this is likely to be reasonably fast. The commit introducing this code points at an LKML thread[1] discussing a similar implementation in the kernel, which discusses the impact of the unaligned access, the conclusion being that it's faster than any alternative.

[1] https://lkml.org/lkml/2012/3/2/452

John Keeping (2):
diffcore-break: don't divide by zero
bisect: avoid signed integer overflow

 bisect.c         | 2 +-
 diffcore-break.c | 3 ++-
 2 files changed, 3 insertions(+), 2 deletions(-)

1.8.2.540.gf023cfe

When the source file is empty, the calculation of the merge score results in a division by zero. Since the merge score is initialized to zero, it makes sense to just leave it as it is if the source size is zero. This means that we still use the extent of damage metric to decide whether to break the filepair.

---
 diffcore-break.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

index 44f8678..37d8d05 100644
@@ -90,7 +90,8 @@ static int should_break(struct diff_filespec *src,
 	 * merge the surviving pair together if the score is
 	 * less than the minimum, after rename/copy runs.
 	 */
-	*merge_score_p = (int)(src_removed * MAX_SCORE / src->size);
+	if (src->size)
+		*merge_score_p = (int)(src_removed * MAX_SCORE / src->size);
 	if (*merge_score_p > break_score)
 		return 1;
 

1.8.2.540.gf023cfe

Well spotted. An empty blob to another blob that is larger than 400 bytes will trigger div0, and it makes sense to leave merge-score to 0 (i.e. do not show as whole-delete-then-whole-add after rename detection is done and the broken filepair is merged back).

Actually, if src->size is 0, we probably shouldn't break the filepair in the first place. That is, if your preimage and postimage looked like these:

== preimage == == postimage ==

F (empty file) F (a large file)
E (a new empty file)

do we want to see F renamed to E and then a new file created as F while running "git diff -B -M"? I doubt it.

So in that sense, this might be a better solution. I dunno.

 diffcore-break.c | 2 ++
 1 file changed, 2 insertions(+)

index 44f8678..eabafd5 100644
@@ -67,6 +67,8 @@ static int should_break(struct diff_filespec *src,
 	max_size = ((src->size > dst->size) ? src->size : dst->size);
 	if (max_size < MINIMUM_BREAK_SIZE)
 		return 0; /* we do not break too small filepair */
+	if (src->size == 0)
+		return 0; /* we do not let empty files get renamed */
 
 	if (diffcore_count_changes(src, dst,
 				   &src->cnt_data, &dst->cnt_data,

I considered this, but didn't quite understand the code well enough to be sure that was the right thing to do. Does it make sense to bail out early if dst->size is zero as well?

The message for commit 6dd4b66 (Fix diffcore-break total breakage) indicates that "don't bother to break small files" is wrong in some cases, but it I wonder if "don't bother to break empty files" is okay.

This has a rather subtle ramifications, and we would need to think carefully. "break" does two very different things, and the criteria you would want to use to decide to "break" a filepair depends on which one you are interested in.

The very original motivation of "break" was to show a patch that rewrites an existing file completely in a way different from the usual "diff -u" output, which will try to minimize the output by finding (rare) lines that happen to be the same between the preimage and postimage, intersparsed in many lines of "-" (deletion) and "+" (addition). Such a change is often easier to understand when shown as a single block of "-" (deletion) of the entire original contents, followed by a single block of "+" (addition) of the entire new contents.

A totally separate motivation of "break" is the one Linus talks about in the log message of the said commit. A path filename.h was moved to filename_32.h, and a new (and much smaller) filename.h was introduced, that "#include"s filename_32.h. "diff -M" that pairs deleted files with created files to compute renames in such a case would not consider the original filename.h as a possible source of filename_32.h that was created. You want to break modification of filename.h into (virtual) deletion and addition of filename.h.

For the purpose of the former, you would want not to break a file too aggressively. If you started from a file with 1000 lines in it, and deleted 910 lines and added 10 lines to result in a file with 100 lines, you still have 90 lines of shared contents between the preimage and the postimage, and you do not want to show it as "delete 1000 lines and add 100 lines". You would want to base your decision on how much common material exists between the two.

For the purpose of the latter, however, it is better to err on the side of breaking than not breaking. After breaking a suspicious modification into addition and deletion, if rename comparison does not find a suitable destination for the virtual deletion, the broken halves will be merged back, so breaking too little can hurt by missing potential renames, but breaking too much will not. You do want to break the "900 deleted out of 1000 and then added 10" case, as that 900 lines may have gone to another file that was created in the same commit. "But we have 90 lines of common material" does not matter for the decision to break.

In today's code, the return value of should_break() is about the latter, while the score it stores in *merge_score is about the former.

When the source file is empty, the calculation of the merge score results in a division by zero. In the situation:

== preimage == == postimage ==

F (empty file) F (a large file)
E (a new empty file)

it does not make sense to consider F->E as a rename, so it is better not to break the pre- and post-image of F.

Bail out early in this case to avoid hitting the divide-by-zero. This causes the merge score to be left at zero.

---

Thanks for this explanation. Following it through, it seems that bailing out early when the destination is empty will do the wrong thing, whereas letting the code carry on should result in a merge score of MAX_SCORE and the function returning 1.

So it seems that the patch you proposed is the best way to handle this.

 diffcore-break.c | 3 +++
 1 file changed, 3 insertions(+)

index 44f8678..1d9e530 100644
@@ -68,6 +68,9 @@ static int should_break(struct diff_filespec *src,
 	if (max_size < MINIMUM_BREAK_SIZE)
 		return 0; /* we do not break too small filepair */
 
+	if (!src->size)
+		return 0; /* we do not let empty files get renamed */
+
 	if (diffcore_count_changes(src, dst,
 				   &src->cnt_data, &dst->cnt_data,
 				   0,

1.8.2.540.gf023cfe

---
 bisect.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index bd1b7b5..0d33c6f 100644
@@ -526,7 +526,7 @@ struct commit_list *filter_skipped(struct commit_list *list,
  * for this application.
  */
 static int get_prn(int count) {
-	count = count * 1103515245 + 12345;
+	count = ((unsigned) count) * 1103515245 + 12345;
 	return ((unsigned)(count/65536) % PRN_MODULO);
 }
 

1.8.2.540.gf023cfe

---
Changes since v1:
- Change the function signature instead of casting a value in the function.
- This lets us remove an existing cast.

 bisect.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

index bd1b7b5..374d9e2 100644
@@ -525,9 +525,9 @@ struct commit_list *filter_skipped(struct commit_list *list,
  * is increased by one between each call, but that should not matter
  * for this application.
  */
-static int get_prn(int count) {
+static unsigned get_prn(unsigned count) {
 	count = count * 1103515245 + 12345;
-	return ((unsigned)(count/65536) % PRN_MODULO);
+	return (count/65536) % PRN_MODULO;
 }
 
 /*

1.8.2.540.gf023cfe