Skip to contentSkip to conversation

t0025 flakey?

PATCH In review Johannes Schindelin 6 participants 14 messages Feb 6, 2019 – Feb 8, 2019

lore
Share & tools
lore
#0 lore raw

Hi Torsten,

at first I thought that those intermittent test failures were limited to Windows, but they are not: I can see it now in a build on 32-bit Linux. Full logs here:

https://dev.azure.com/gitgitgadget/git/_build/results?buildId=1032&_a=summary&view=ms.vss-test-web.build-test-results-tab

Excerpt from the failing test case:

-- snip --
not ok 2 - renormalize CRLF in repo
expecting success:

	echo "*.txt text=auto" >.gitattributes &&
	git add --renormalize "*.txt" &&
	cat >expect <<-\EOF &&
	i/lf w/crlf attr/text=auto CRLF.txt
	i/lf w/lf attr/text=auto LF.txt
	i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
	EOF
	git ls-files --eol |
	sed -e "s/	/ /g" -e "s/  */ /g" |
	sort >actual &&
	test_cmp expect actual

+ echo *.txt text=auto
+ git add --renormalize *.txt
+ cat
+ sort
+ sed -e s/ / /g -e s/ */ /g
+ git ls-files --eol
+ test_cmp expect actual
+ diff -u expect actual

actual renamed from expect +2 -2
@@ -1,3 +1,3 @@
-i/lf w/crlf attr/text=auto CRLF.txt
+i/crlf w/crlf attr/text=auto CRLF.txt
 i/lf w/lf attr/text=auto LF.txt
-i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
+i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt

error: last command exited with $?=1
-- snap --

Any ideas?
Johannes

#1 lore raw
Quoted 44 lines · Johannes Schindelin

Hi Torsten,

at first I thought that those intermittent test failures were limited to Windows, but they are not: I can see it now in a build on 32-bit Linux. Full logs here:

https://dev.azure.com/gitgitgadget/git/_build/results?buildId=1032&_a=summary&view=ms.vss-test-web.build-test-results-tab

Excerpt from the failing test case:

-- snip --
not ok 2 - renormalize CRLF in repo
expecting success:

	echo "*.txt text=auto" >.gitattributes &&
	git add --renormalize "*.txt" &&
	cat >expect <<-\EOF &&
	i/lf w/crlf attr/text=auto CRLF.txt
	i/lf w/lf attr/text=auto LF.txt
	i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
	EOF
	git ls-files --eol |
	sed -e "s/	/ /g" -e "s/  */ /g" |
	sort >actual &&
	test_cmp expect actual

+ echo *.txt text=auto
+ git add --renormalize *.txt
+ cat
+ sort
+ sed -e s/ / /g -e s/ */ /g
+ git ls-files --eol
+ test_cmp expect actual
+ diff -u expect actual

actual renamed from expect +2 -2
@@ -1,3 +1,3 @@
-i/lf w/crlf attr/text=auto CRLF.txt
+i/crlf w/crlf attr/text=auto CRLF.txt
 i/lf w/lf attr/text=auto LF.txt
-i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
+i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt

error: last command exited with $?=1
-- snap --

Quoted 1 line

Any ideas?

I reported this and Peff looked into it on the way to Git Merge, but not working solution yet.

https://public-inbox.org/git/[email protected]/T/#u

#2 lore raw

Hi Gábor,

Quoted 45 lines · SZEDER Gábor
Quoted 43 lines · Johannes Schindelin

at first I thought that those intermittent test failures were limited to Windows, but they are not: I can see it now in a build on 32-bit Linux.
Full logs here:

https://dev.azure.com/gitgitgadget/git/_build/results?buildId=1032&_a=summary&view=ms.vss-test-web.build-test-results-tab

Excerpt from the failing test case:

-- snip --
not ok 2 - renormalize CRLF in repo
expecting success:

	echo "*.txt text=auto" >.gitattributes &&
	git add --renormalize "*.txt" &&
	cat >expect <<-\EOF &&
	i/lf w/crlf attr/text=auto CRLF.txt
	i/lf w/lf attr/text=auto LF.txt
	i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
	EOF
	git ls-files --eol |
	sed -e "s/	/ /g" -e "s/  */ /g" |
	sort >actual &&
	test_cmp expect actual

+ echo *.txt text=auto
+ git add --renormalize *.txt
+ cat
+ sort
+ sed -e s/ / /g -e s/ */ /g
+ git ls-files --eol
+ test_cmp expect actual
+ diff -u expect actual

actual renamed from expect +2 -2
@@ -1,3 +1,3 @@
-i/lf w/crlf attr/text=auto CRLF.txt
+i/crlf w/crlf attr/text=auto CRLF.txt
 i/lf w/lf attr/text=auto LF.txt
-i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
+i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt

error: last command exited with $?=1
-- snap --

Quoted 6 lines
Quoted 1 line

Any ideas?

I reported this and Peff looked into it on the way to Git Merge, but not working solution yet.

https://public-inbox.org/git/[email protected]/T/#u

Thank you!
Dscho

#3 lore raw
Quoted 54 lines · Johannes Schindelin

Hi Gábor,

Quoted 50 lines · SZEDER Gábor
Quoted 43 lines · Johannes Schindelin

at first I thought that those intermittent test failures were limited to Windows, but they are not: I can see it now in a build on 32-bit Linux.
Full logs here:

https://dev.azure.com/gitgitgadget/git/_build/results?buildId=1032&_a=summary&view=ms.vss-test-web.build-test-results-tab

Excerpt from the failing test case:

-- snip --
not ok 2 - renormalize CRLF in repo
expecting success:

	echo "*.txt text=auto" >.gitattributes &&
	git add --renormalize "*.txt" &&
	cat >expect <<-\EOF &&
	i/lf w/crlf attr/text=auto CRLF.txt
	i/lf w/lf attr/text=auto LF.txt
	i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
	EOF
	git ls-files --eol |
	sed -e "s/	/ /g" -e "s/  */ /g" |
	sort >actual &&
	test_cmp expect actual

+ echo *.txt text=auto
+ git add --renormalize *.txt
+ cat
+ sort
+ sed -e s/ / /g -e s/ */ /g
+ git ls-files --eol
+ test_cmp expect actual
+ diff -u expect actual

actual renamed from expect +2 -2
@@ -1,3 +1,3 @@
-i/lf w/crlf attr/text=auto CRLF.txt
+i/crlf w/crlf attr/text=auto CRLF.txt
 i/lf w/lf attr/text=auto LF.txt
-i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
+i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt

error: last command exited with $?=1
-- snap --

Quoted 1 line

Any ideas?

I reported this and Peff looked into it on the way to Git Merge, but not working solution yet.

Quoted 4 lines
Quoted 1 line

https://public-inbox.org/git/[email protected]/T/#u

Thank you!
Dscho

I shortly looked into the pointers here -
Is t0025 flaky after the fix from Peff:

[PATCH] add: use separate ADD_CACHE_RENORMALIZE flag

Or has it always been shaky ?
Does anybody know ?

#4 lore raw
Quoted 60 lines · Torsten Bögershausen
Quoted 54 lines · Johannes Schindelin

Hi Gábor,

Quoted 50 lines · SZEDER Gábor
Quoted 43 lines · Johannes Schindelin

at first I thought that those intermittent test failures were limited to Windows, but they are not: I can see it now in a build on 32-bit Linux.
Full logs here:

https://dev.azure.com/gitgitgadget/git/_build/results?buildId=1032&_a=summary&view=ms.vss-test-web.build-test-results-tab

Excerpt from the failing test case:

-- snip --
not ok 2 - renormalize CRLF in repo
expecting success:

	echo "*.txt text=auto" >.gitattributes &&
	git add --renormalize "*.txt" &&
	cat >expect <<-\EOF &&
	i/lf w/crlf attr/text=auto CRLF.txt
	i/lf w/lf attr/text=auto LF.txt
	i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
	EOF
	git ls-files --eol |
	sed -e "s/	/ /g" -e "s/  */ /g" |
	sort >actual &&
	test_cmp expect actual

+ echo *.txt text=auto
+ git add --renormalize *.txt
+ cat
+ sort
+ sed -e s/ / /g -e s/ */ /g
+ git ls-files --eol
+ test_cmp expect actual
+ diff -u expect actual

actual renamed from expect +2 -2
@@ -1,3 +1,3 @@
-i/lf w/crlf attr/text=auto CRLF.txt
+i/crlf w/crlf attr/text=auto CRLF.txt
 i/lf w/lf attr/text=auto LF.txt
-i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
+i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt

error: last command exited with $?=1
-- snap --

Quoted 1 line

Any ideas?

I reported this and Peff looked into it on the way to Git Merge, but not working solution yet.

Quoted 4 lines
Quoted 1 line

https://public-inbox.org/git/[email protected]/T/#u

Thank you!
Dscho

Quoted 7 lines

I shortly looked into the pointers here -
Is t0025 flaky after the fix from Peff:

[PATCH] add: use separate ADD_CACHE_RENORMALIZE flag

Or has it always been shaky ?
Does anybody know ?

I sort-of bisected it, and it pointed to Peff's fix.

Running 't0025 --stress' in the merge of 'sg/stress-test' and 'jk/add-ignore-errors-bit-assignment-fix' results in a failure within seconds or <20 repetitions. In the merge with 'jk/add-ignore-errors-bit-assignment-fix^' it runs successfully for thousands of repetitions.

#5 lore raw
In reply to Torsten Bögershausen
Quoted 60 lines · Torsten Bögershausen

To: Johannes Schindelin <[email protected]>
Cc: SZEDER Gábor <[email protected]>; Jeff King <[email protected]>; [email protected]
Subject: Re: t0025 flakey?

Quoted 54 lines · Johannes Schindelin

Hi Gábor,

Quoted 50 lines · SZEDER Gábor
Quoted 43 lines · Johannes Schindelin

at first I thought that those intermittent test failures were limited to Windows, but they are not: I can see it now in a build on 32-bit Linux.
Full logs here:

https://dev.azure.com/gitgitgadget/git/_build/results?buildId=1032
&_a=summary&view=ms.vss-test-web.build-test-results-tab

Excerpt from the failing test case:

-- snip --
not ok 2 - renormalize CRLF in repo expecting success:

	echo "*.txt text=auto" >.gitattributes &&
	git add --renormalize "*.txt" &&
	cat >expect <<-\EOF &&
	i/lf w/crlf attr/text=auto CRLF.txt
	i/lf w/lf attr/text=auto LF.txt
	i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
	EOF
	git ls-files --eol |
	sed -e "s/	/ /g" -e "s/  */ /g" |
	sort >actual &&
	test_cmp expect actual

+ echo *.txt text=auto
+ git add --renormalize *.txt
+ cat
+ sort
+ sed -e s/ / /g -e s/ */ /g
+ git ls-files --eol
+ test_cmp expect actual
+ diff -u expect actual

actual renamed from expect +2 -2
@@ -1,3 +1,3 @@
-i/lf w/crlf attr/text=auto CRLF.txt
+i/crlf w/crlf attr/text=auto CRLF.txt
 i/lf w/lf attr/text=auto LF.txt
-i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
+i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt

error: last command exited with $?=1
-- snap --

Quoted 1 line

Any ideas?

I reported this and Peff looked into it on the way to Git Merge, but not working solution yet.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

https://public-inbox.org/git/[email protected]
.net/T/#u

Thank you!
Dscho

I shortly looked into the pointers here - Is t0025 flaky after the fix

from Peff:

Quoted 5 lines

[PATCH] add: use separate ADD_CACHE_RENORMALIZE flag

Or has it always been shaky ?
Does anybody know ?

The NonStop port has traditionally had issues with t0025, which we tended to ignore because things did work. We wrote those off as bash issues in t0025 since they seemed to be corrected when we picked up a new bash version about a year ago. I will keep monitoring this, particularly when 2.21 comes out.

Cheers,
Randall

-- Brief whoami:
NonStop developer since approximately 211288444200000000 UNIX developer since approximately 421664400
-- In my real life, I talk too much.

#6 lore raw
In reply to Torsten Bögershausen
Quoted 62 lines · On February 6, 2019 13:01, I wrote:
Quoted 11 lines · Torsten Bögershausen

To: Johannes Schindelin <[email protected]>
Cc: SZEDER Gábor <[email protected]>; Jeff King <[email protected]>; [email protected]
Subject: Re: t0025 flakey?

Quoted 5 lines · Johannes Schindelin

Hi Gábor,

Quoted 1 line · SZEDER Gábor

On Wed, Feb 06, 2019 at 11:25:38AM +0100, Johannes Schindelin

Quoted 49 lines · wrote:
Quoted 49 lines
Quoted 49 lines
Quoted 44 lines

at first I thought that those intermittent test failures were limited to Windows, but they are not: I can see it now in a build on 32-bit Linux.
Full logs here:

https://dev.azure.com/gitgitgadget/git/_build/results?buildId=10
32 &_a=summary&view=ms.vss-test-web.build-test-results-tab

Excerpt from the failing test case:

-- snip --
not ok 2 - renormalize CRLF in repo expecting success:

	echo "*.txt text=auto" >.gitattributes &&
	git add --renormalize "*.txt" &&
	cat >expect <<-\EOF &&
	i/lf w/crlf attr/text=auto CRLF.txt
	i/lf w/lf attr/text=auto LF.txt
	i/lf w/mixed attr/text=auto CRLF_mix_LF.txt
	EOF
	git ls-files --eol |
	sed -e "s/	/ /g" -e "s/  */ /g" |
	sort >actual &&
	test_cmp expect actual

+ echo *.txt text=auto
+ git add --renormalize *.txt
+ cat
+ sort
+ sed -e s/ / /g -e s/ */ /g
+ git ls-files --eol
+ test_cmp expect actual
+ diff -u expect actual

actual renamed from expect +1 -1
@@ -1,3 +1,3 @@
-i/lf w/crlf attr/text=auto CRLF.txt
+i/crlf w/crlf attr/text=auto CRLF.txt
 i/lf w/lf attr/text=auto LF.txt -i/lf w/mixed attr/text=auto

CRLF_mix_LF.txt
+i/mixed w/mixed attr/text=auto CRLF_mix_LF.txt
error: last command exited with $?=1
-- snap --

Quoted 1 line

Any ideas?

I reported this and Peff looked into it on the way to Git Merge, but not working solution yet.

Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

https://public-inbox.org/git/[email protected]
ff
.net/T/#u

Thank you!
Dscho

I shortly looked into the pointers here - Is t0025 flaky after the fix

from

Quoted 11 lines

Peff:

Quoted 5 lines

[PATCH] add: use separate ADD_CACHE_RENORMALIZE flag

Or has it always been shaky ?
Does anybody know ?

The NonStop port has traditionally had issues with t0025, which we tended to ignore because things did work. We wrote those off as bash issues in t0025 since they seemed to be corrected when we picked up a new bash version about a year ago. I will keep monitoring this, particularly when

2.21

Quoted 1 line

comes out.

FYI: t0020-t0027 all passed on the NonStop port for 2.21.0-rc0 - so no issues for us on this one.

Cheers,
Randall

-- Brief whoami:
NonStop developer since approximately 211288444200000000 UNIX developer since approximately 421664400
-- In my real life, I talk too much.

#7 lore raw
Quoted 13 lines · Randall S. Becker
Quoted 5 lines · On February 6, 2019 13:01, I wrote:
Quoted 4 lines · Torsten Bögershausen

To: Johannes Schindelin <[email protected]>
...
[PATCH] add: use separate ADD_CACHE_RENORMALIZE flag

Quoted 7 lines
Quoted 2 lines

Or has it always been shaky ?
Does anybody know ?

The NonStop port has traditionally had issues with t0025, which we tended to ignore because things did work. We wrote those off as bash issues in t0025 since they seemed to be corrected when we picked up a new bash version about a year ago. I will keep monitoring this, particularly when

Quoted 5 lines

2.21

Quoted 1 line

comes out.

FYI: t0020-t0027 all passed on the NonStop port for 2.21.0-rc0 - so no issues for us on this one.

Yup, the preview has both the renormalize fix and another hotfix for it Peff made. Thanks all for quick fixes and confirmations.

#8 lore raw
In reply to Randall S. Becker
Quoted 9 lines · Randall S. Becker
Quoted 4 lines

The NonStop port has traditionally had issues with t0025, which we tended to ignore because things did work. We wrote those off as bash issues in t0025 since they seemed to be corrected when we picked up a new bash version about a year ago. I will keep monitoring this, particularly when

2.21

Quoted 1 line

comes out.

FYI: t0020-t0027 all passed on the NonStop port for 2.21.0-rc0 - so no issues for us on this one.

Note that t0021 is very likely flaky on NonStop, too:

https://public-inbox.org/git/[email protected]/T/#u

#9 lore raw
Quoted 8 lines · SZEDER Gábor
Quoted 7 lines · Randall S. Becker
Quoted 6 lines

The NonStop port has traditionally had issues with t0025, which we tended to ignore because things did work. We wrote those off as bash issues in
t0025 since they seemed to be corrected when we picked up a new bash version about a year ago. I will keep monitoring this, particularly when

2.21

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

comes out.

FYI: t0020-t0027 all passed on the NonStop port for 2.21.0-rc0 - so no issues for us on this one.

Note that t0021 is very likely flaky on NonStop, too:

https://public-inbox.org/git/[email protected]/T/#u

We will keep a watch on it, thanks. t0021 has been stable on this platform for at least a year and passes for 2.21.0-rc0 as well as 2.20.0.

Cheers,
Randall

#10 lore raw

PATCH 1/1 add_to_index(): convert forgotten HASH_RENORMALIZE check

In reply to SZEDER Gábor
Quoted 4 lines · SZEDER Gábor

I reported this and Peff looked into it on the way to Git Merge, but not working solution yet.

https://public-inbox.org/git/[email protected]/T/#u

Oof. Well, now I know why my attempts to fix the test failed. It was not my new test that was failing at all, but rather the existing test. Which implies that I severely bungled the actual code change.

Armed with that knowledge, it was pretty easy to find said bungling. The fix is below.

Junio, this should go on top of jk/add-ignore-errors-bit-assignment-fix as soon as possible, as the regression is already in master. And I'll go find a brown paper bag. ;)

-- >8 --
Subject: [PATCH] add_to_index(): convert forgotten HASH_RENORMALIZE check

Commit 9e5da3d055 (add: use separate ADD_CACHE_RENORMALIZE flag, 2019-01-17) switched out using HASH_RENORMALIZE in our flags field for a new ADD_CACHE_RENORMALIZE flag. However, it forgot to convert one of the checks for HASH_RENORMALIZE into the new flag, which totally broke "git add --renormalize".

To make matters even more confusing, the resulting code would racily pass the tests! The forgotten check was responsible for defeating the up-to-date check of the index entry. That meant that "git add --renormalize" would refuse to renormalize things that appeared stat-clean. But most of the time the test commands run fast enough that the file mtime is the same as the index mtime. And thus we err on the conservative side and re-hash the file, which is what "--renormalize" would have wanted.

But if you're unlucky and cross that one-second boundary between writing the file and writing the index (which is more likely to happen on a slow or heavily-loaded system), then the file appears stat-clean. And "--renormalize" would effectively do nothing.

The fix is straightforward: convert this check to use the right flag.

---
 read-cache.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
read-cache.c +1 -1
index 9783c493a3..accc059951 100644
@@ -746,7 +746,7 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,
 	if (ignore_case) {
 		adjust_dirname_case(istate, ce->name);
 	}
-	if (!(flags & HASH_RENORMALIZE)) {
+	if (!(flags & ADD_CACHE_RENORMALIZE)) {
 		alias = index_file_exists(istate, ce->name,
 					  ce_namelen(ce), ignore_case);
 		if (alias &&
Signature · 1 line
2.20.1.1122.g2972e48916
#11 lore raw

Re: [PATCH] add_to_index(): convert forgotten HASH_RENORMALIZE check

Quoted 63 lines · Jeff King
Quoted 4 lines · SZEDER Gábor

I reported this and Peff looked into it on the way to Git Merge, but not working solution yet.

https://public-inbox.org/git/[email protected]/T/#u

Oof. Well, now I know why my attempts to fix the test failed. It was not my new test that was failing at all, but rather the existing test. Which implies that I severely bungled the actual code change.

Armed with that knowledge, it was pretty easy to find said bungling. The fix is below.

Junio, this should go on top of jk/add-ignore-errors-bit-assignment-fix as soon as possible, as the regression is already in master. And I'll go find a brown paper bag. ;)

-- >8 --
Subject: [PATCH] add_to_index(): convert forgotten HASH_RENORMALIZE check

Commit 9e5da3d055 (add: use separate ADD_CACHE_RENORMALIZE flag, 2019-01-17) switched out using HASH_RENORMALIZE in our flags field for a new ADD_CACHE_RENORMALIZE flag. However, it forgot to convert one of the checks for HASH_RENORMALIZE into the new flag, which totally broke "git add --renormalize".

To make matters even more confusing, the resulting code would racily pass the tests! The forgotten check was responsible for defeating the up-to-date check of the index entry. That meant that "git add --renormalize" would refuse to renormalize things that appeared stat-clean. But most of the time the test commands run fast enough that the file mtime is the same as the index mtime. And thus we err on the conservative side and re-hash the file, which is what "--renormalize" would have wanted.

But if you're unlucky and cross that one-second boundary between writing the file and writing the index (which is more likely to happen on a slow or heavily-loaded system), then the file appears stat-clean. And "--renormalize" would effectively do nothing.

The fix is straightforward: convert this check to use the right flag.

---
 read-cache.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
read-cache.c +1 -2
index 9783c493a3..accc059951 100644
@@ -746,7 +746,7 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,
 	if (ignore_case) {
 		adjust_dirname_case(istate, ce->name);
 	}
-	if (!(flags & HASH_RENORMALIZE)) {
+	if (!(flags & ADD_CACHE_RENORMALIZE)) {
 		alias = index_file_exists(istate, ce->name,
 					  ce_namelen(ce), ignore_case);
 		if (alias &&
--
Quoted 2 lines

2.20.1.1122.g2972e48916

Ack, of course.

And trying to answer an older question:

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

The reason appears to be wrong bit mask usage
#define ADD_CACHE_IGNORE_ERRORS 4
and
#define HASH_RENORMALIZE 4

What if we had renamed "flags" like this ?

read-cache.c +6 -6
index 5096e395ee..f93d291613 100644
@@ -696,20 +696,20 @@ void set_object_name_for_intent_to_add_entry(struct cache_entry *ce)
 	oidcpy(&ce->oid, &oid);
 }
 
-int add_to_index(struct index_state *istate, const char *path, struct stat *st, int flags)
+int add_to_index(struct index_state *istate, const char *path, struct stat *st, int add_cache_flags)
 {
 	int namelen, was_same;
 	mode_t st_mode = st->st_mode;
 	struct cache_entry *ce, *alias = NULL;
 	unsigned ce_option = CE_MATCH_IGNORE_VALID|CE_MATCH_IGNORE_SKIP_WORKTREE|CE_MATCH_RACY_IS_DIRTY;
-	int verbose = flags & (ADD_CACHE_VERBOSE | ADD_CACHE_PRETEND);
-	int pretend = flags & ADD_CACHE_PRETEND;
-	int intent_only = flags & ADD_CACHE_INTENT;
+	int verbose = add_cache_flags & (ADD_CACHE_VERBOSE | ADD_CACHE_PRETEND);
+	int pretend = add_cache_flags & ADD_CACHE_PRETEND;
+	int intent_only = add_cache_flags & ADD_CACHE_INTENT;
 	int add_option = (ADD_CACHE_OK_TO_ADD|ADD_CACHE_OK_TO_REPLACE|
 			  (intent_only ? ADD_CACHE_NEW_ONLY : 0));
 	int hash_flags = HASH_WRITE_OBJECT;
 
-	if (flags & ADD_CACHE_RENORMALIZE)
+	if (add_cache_flags & ADD_CACHE_RENORMALIZE)
 		hash_flags |= HASH_RENORMALIZE;
 
 	if (!S_ISREG(st_mode) && !S_ISLNK(st_mode) && !S_ISDIR(st_mode))
@@ -750,7 +750,7 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,
 	if (ignore_case) {
 		adjust_dirname_case(istate, ce->name);
 	}
-	if (!(flags & HASH_RENORMALIZE)) {
+	if (!(add_cache_flags & HASH_RENORMALIZE)) {
 		alias = index_file_exists(istate, ce->name,
 					  ce_namelen(ce), ignore_case);
 		if (alias &&
#12 lore raw

Re: [PATCH] add_to_index(): convert forgotten HASH_RENORMALIZE check

read-cache.c void set_object_name_for_intent_to_add_entry(struct cache_entry *ce) · line 699

Quoted 11 lines · Torsten Bögershausen

And trying to answer an older question:

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

The reason appears to be wrong bit mask usage
#define ADD_CACHE_IGNORE_ERRORS 4
and
#define HASH_RENORMALIZE 4

What if we had renamed "flags" like this ?
[...]

read-cache.c +1 -1
-int add_to_index(struct index_state *istate, const char *path, struct stat *st, int flags)
+int add_to_index(struct index_state *istate, const char *path, struct stat *st, int add_cache_flags)

Yes, changing the name of the variable in the original patch would have caught this case. I don't know if it is worth doing now or not (the code as it is now seems pretty clear to me, but of course I've looked at it a lot lately).

-Peff

#13 lore raw

Re: [PATCH] add_to_index(): convert forgotten HASH_RENORMALIZE check

In reply to Jeff King

Hi Peff,

Quoted 11 lines · Jeff King
Quoted 4 lines · SZEDER Gábor

I reported this and Peff looked into it on the way to Git Merge, but not working solution yet.

https://public-inbox.org/git/[email protected]/T/#u

Oof. Well, now I know why my attempts to fix the test failed. It was not my new test that was failing at all, but rather the existing test. Which implies that I severely bungled the actual code change.

Quoted 6 lines

Armed with that knowledge, it was pretty easy to find said bungling. The fix is below.

Junio, this should go on top of jk/add-ignore-errors-bit-assignment-fix as soon as possible, as the regression is already in master. And I'll go find a brown paper bag. ;)

Thank you *so* much for the quick fix!
Dscho

Quoted 48 lines

-- >8 --
Subject: [PATCH] add_to_index(): convert forgotten HASH_RENORMALIZE check

Commit 9e5da3d055 (add: use separate ADD_CACHE_RENORMALIZE flag, 2019-01-17) switched out using HASH_RENORMALIZE in our flags field for a new ADD_CACHE_RENORMALIZE flag. However, it forgot to convert one of the checks for HASH_RENORMALIZE into the new flag, which totally broke "git add --renormalize".

To make matters even more confusing, the resulting code would racily pass the tests! The forgotten check was responsible for defeating the up-to-date check of the index entry. That meant that "git add --renormalize" would refuse to renormalize things that appeared stat-clean. But most of the time the test commands run fast enough that the file mtime is the same as the index mtime. And thus we err on the conservative side and re-hash the file, which is what "--renormalize" would have wanted.

But if you're unlucky and cross that one-second boundary between writing the file and writing the index (which is more likely to happen on a slow or heavily-loaded system), then the file appears stat-clean. And "--renormalize" would effectively do nothing.

The fix is straightforward: convert this check to use the right flag.

---
 read-cache.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
read-cache.c +1 -1
index 9783c493a3..accc059951 100644
@@ -746,7 +746,7 @@ int add_to_index(struct index_state *istate, const char *path, struct stat *st,
 	if (ignore_case) {
 		adjust_dirname_case(istate, ce->name);
 	}
-	if (!(flags & HASH_RENORMALIZE)) {
+	if (!(flags & ADD_CACHE_RENORMALIZE)) {
 		alias = index_file_exists(istate, ce->name,
 					  ce_namelen(ce), ignore_case);
 		if (alias &&
Signature · 1 line
2.20.1.1122.g2972e48916