A couple of CI fixes regarding the built-in add --patch
cover A couple of CI fixes regarding the built-in add --patch
This is an attempt to address the concern Junio raised in
https://lore.kernel.org/git/[email protected]/: originally motivated by a test suite failure when running Git's test suite in Visual Studio, this pulls out the signed vs unsigned fix whose implications are potentially much wider than Visual C.
While in that space, I spent the time (which took almost as long as I expected
[https://lore.kernel.org/git/[email protected]/])
to craft a semantic patch to scrutinize Git's source code for similar issues (narrator's voice: there were no other instances, what did ya expect?).
To verify the fix, I then worked on a patch to exercise the built-in git add -p in the test suite even when NO_PERL is set, and while developing this patch and validating it, I got really puzzled that the add -p test case in t6132 did not need to be guarded behind a PERL prereq. So this patch series also includes a fix for that.
The story arc that binds all of these patches together is that they all revolve around NO_PERL and CI issues that involve git add --patch.
Note: This patch series is based on ds/github-actions-use-newer-ubuntu (but probably applies cleanly even on maint) because I tried to develop a semantic patch to fix similar issues in the code base. However, I've since run into what looks like a bug in Coccinelle
[https://github.com/coccinelle/coccinelle/issues/284]. My latest version of that semantic patch looks like this, but I stopped when running it on Git's source code triggered the bug for 66 of Git's .c files:
@@
type T = { unsigned int };
T:n b;
type S != { unsigned int, size_t };
S s;
binary operator o != { &&, || };
@@
-s o b
+s o (S)b
@@
type T = { unsigned int };
T:n b;
type S != { unsigned int, size_t };
S s;
binary operator o != { &&, || };
@@
-b o s
+(S)b o s
Johannes Schindelin (3):
add -p: avoid ambiguous signed/unsigned comparison
t3701: test the built-in add -i regardless of NO_PERL t6132(NO_PERL): do not run the scripted add -p
add-patch.c | 2 +- t/t3701-add-interactive.sh | 4 ++-- t/t6132-pathspec-exclude.sh | 6 +++++- 3 files changed, 8 insertions(+), 4 deletions(-)
base-commit: ef46584831268a83591412a33783caf866867482
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1340%2Fdscho%2Fbuilt-in-add-i-does-not-need-perl-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1340/dscho/built-in-add-i-does-not-need-perl-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1340
Signature · 1 line
gitgitgadget
PATCH 1/3 add -p: avoid ambiguous signed/unsigned comparison
From: Johannes Schindelin <[email protected]>
In the interactive add operation, users can choose to jump to specific hunks, and Git will present the hunk list in that case. To avoid showing too many lines at once, only a maximum of 21 hunks are shown, skipping the "mode change" pseudo hunk.
The comparison performed to skip the "mode change" pseudo hunk (if any) compares a signed integer i to the unsigned value mode_change (which can be 0 or 1 because it is a 1-bit type).
According to section 6.3.1.8 of the C99 standard (see e.g.
https://www.open-std.org/jtc1/sc22/WG14/www/docs/n1256.pdf), what should happen is an automatic conversion of the "lesser" type to the "greater" type, but since the types differ in signedness, it is ill-defined what is the correct "usual arithmetic conversion".
Which means that Visual C's behavior can (and does) differ from GCC's:
When compiling Git using the latter, add -p's goto command shows no hunks by default because it casts a negative start offset to a pretty large unsigned value, breaking the "goto hunk" test case in t3701-add-interactive.sh.
Let's avoid that by converting the unsigned bit explicitly to a signed integer.
Note: This is a long-standing bug in the Visual C build of Git, but it has never been caught because t3701 is skipped when NO_PERL is set, which is the case in the vs-test jobs of Git's CI runs.
--- add-patch.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
add-patch.c +1 -1
@@ -1547,7 +1547,7 @@ soft_increment:strbuf_remove (&s->answer,0 ,1 );strbuf_trim (&s->answer); i = hunk_index -DISPLAY_HUNKS_LINES /2 ; -if (i < file_diff->mode_change) +if (i < (int )file_diff->mode_change) i = file_diff->mode_change;while (s->answer.len ==0 ) { i =display_hunks (s, file_diff, i);
Signature · 1 line
gitgitgadget
Re: [PATCH 1/3] add -p: avoid ambiguous signed/unsigned comparison
"Johannes Schindelin via GitGitGadget" <[email protected]>
Quoted 6 lines · writes:
From: Johannes Schindelin <[email protected]>
In the interactive
addoperation, users can choose to jump to specific hunks, and Git will present the hunk list in that case. To avoid showing too many lines at once, only a maximum of 21 hunks are shown, skipping the "mode change" pseudo hunk.
Wow.
While the whole "add -i" was what I started, I admit that everything more recent than the support of the "edit" action in "add -p" is foreign to me. I didn't remember the existence of, and the justification for, the 20-line limit. It seems that this came from 3f6aff68 (Add subroutine to display one-line summary of hunks, 2008-12-04).
Quoted 3 lines
The comparison performed to skip the "mode change" pseudo hunk (if any) compares a signed integer
ito the unsigned valuemode_change(which can be 0 or 1 because it is a 1-bit type).
OK, that warrants a casting to signed, surely.
Quoted 3 lines
Note: This is a long-standing bug in the Visual C build of Git, but it has never been caught because t3701 is skipped when
NO_PERLis set, which is the case in thevs-testjobs of Git's CI runs.
Good finding.
Quoted 18 lines
Signed-off-by Johannes Schindelin <[email protected]>--- add-patch.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)add-patch.c +1
-1@@ -1547,7 +1547,7 @@ soft_increment:strbuf_remove (&s->answer,0 ,1 );strbuf_trim (&s->answer); i = hunk_index -DISPLAY_HUNKS_LINES /2 ; -if (i < file_diff->mode_change) +if (i < (int )file_diff->mode_change) i = file_diff->mode_change;while (s->answer.len ==0 ) { i =display_hunks (s, file_diff, i);
PATCH 2/3 t3701: test the built-in `add -i` regardless of NO_PERL
In reply to Johannes Schindelin via GitGitGadgetFrom: Johannes Schindelin <[email protected]>
The built-in git add --interactive does not require Perl, therefore we can safely run these tests even when building with NO_PERL=LetsDoThat.
--- t/t3701-add-interactive.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
t/t3701-add-interactive.sh +2 -2
@@ -7,9 +7,9 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME . ./test-lib.sh . "$TEST_DIRECTORY"/lib-terminal.sh -if ! test_have_prereq PERL +if test_have_prereq !ADD_I_USE_BUILTIN,!PERL then - skip_all='skipping add -i tests, perl not available' + skip_all='skipping add -i (scripted) tests, perl not available' test_done fi
Signature · 1 line
gitgitgadget
Re: [PATCH 2/3] t3701: test the built-in `add -i` regardless of NO_PERL
Quoted 4 lines · Johannes Schindelin via GitGitGadget
From: Johannes Schindelin <[email protected]>
The built-in
git add --interactivedoes not require Perl, therefore we can safely run these tests even when building withNO_PERL=LetsDoThat.
Make sense. The patch is small enough that it is certainly worth doing in the meantime, but is it time to start thinking about dropping the perl implementation (and hence this prereq) entirely?
-Peff
Re: [PATCH 2/3] t3701: test the built-in `add -i` regardless of NO_PERL
Hi Peff,
Quoted 10 lines · Jeff King
Quoted 4 lines · Johannes Schindelin via GitGitGadget
From: Johannes Schindelin <[email protected]>
The built-in
git add --interactivedoes not require Perl, therefore we can safely run these tests even when building withNO_PERL=LetsDoThat.Make sense. The patch is small enough that it is certainly worth doing in the meantime, but is it time to start thinking about dropping the perl implementation (and hence this prereq) entirely?
A couple of months ago, I would have said the same (and you seemed to suggest something similar in [*1*], too), but since we flipped the default to the built-in version in 0527ccb1b55 (add -i: default to the built-in implementation, 2021-11-30), we discovered the need for several fixes:
- pw/add-p-hunk-split-fix
- js/add-i-delete
- js/add-p-diff-parsing-fix
Therefore I consider it premature to drop git-add--interactive.perl just yet.
Ciao,
Dscho
Footnote *1*:
https://lore.kernel.org/git/YaaP%[email protected]/
Re: [PATCH 2/3] t3701: test the built-in `add -i` regardless of NO_PERL
In reply to Johannes Schindelin via GitGitGadget"Johannes Schindelin via GitGitGadget" <[email protected]>
Quoted 9 lines · writes:
From: Johannes Schindelin <[email protected]>
The built-in
git add --interactivedoes not require Perl, therefore we can safely run these tests even when building withNO_PERL=LetsDoThat.Signed-off-by Johannes Schindelin <[email protected]>--- t/t3701-add-interactive.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
A companion to the previous step that is obvious and very much to the point. Looking good.
Quoted 15 lines
t/t3701-add-interactive.sh +2
-2@@ -7,9 +7,9 @@ export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME . ./test-lib.sh . "$TEST_DIRECTORY"/lib-terminal.sh -if ! test_have_prereq PERL +if test_have_prereq !ADD_I_USE_BUILTIN,!PERL then - skip_all='skipping add -i tests, perl not available' + skip_all='skipping add -i (scripted) tests, perl not available' test_done fi
PATCH 3/3 t6132(NO_PERL): do not run the scripted `add -p`
In reply to Johannes Schindelin via GitGitGadgetFrom: Johannes Schindelin <[email protected]>
When using the non-built-in version of git add -p in a NO_PERL build, we expect that invocation to fail.
However, when b02fdbc80a4 (pathspec: correct an empty string used as a pathspec element, 2022-05-29) added a test case to t6132 to exercise git add -p, it did not add appropriate prereqs (which admittedly did not exist back then).
Let's specify the appropriate prereqs.
--- t/t6132-pathspec-exclude.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
t/t6132-pathspec-exclude.sh +5 -1
@@ -293,7 +293,11 @@ test_expect_success 'add with all negative' ' test_cmp expect actual ' -test_expect_success 'add -p with all negative' ' +test_lazy_prereq ADD_I_USE_BUILTIN_OR_PERL ' + test_have_prereq ADD_I_USE_BUILTIN || test_have_prereq PERL +' + +test_expect_success ADD_I_USE_BUILTIN_OR_PERL 'add -p with all negative' ' H=$(git rev-parse HEAD) && git reset --hard $H && git clean -f &&
Signature · 1 line
gitgitgadget
validating signed/unsigned comparisons with Coccinelle, was Re: [PATCH 0/3] A couple of CI fixes regarding the built-in add --patch
In reply to Johannes Schindelin via GitGitGadgetHi,
Quoted 25 lines · Johannes Schindelin via GitGitGadget
Note: This patch series is based on ds/github-actions-use-newer-ubuntu (but probably applies cleanly even on maint) because I tried to develop a semantic patch to fix similar issues in the code base. However, I've since run into what looks like a bug in Coccinelle
[https://github.com/coccinelle/coccinelle/issues/284]. My latest version of that semantic patch looks like this, but I stopped when running it on Git's source code triggered the bug for 66 of Git's .c files:@@
type T = { unsigned int };
T:n b;
type S != { unsigned int,size_t};
S s;
binary operator o != { &&, || };
@@(quoted diff) +1
-1-s o b +s o (S)b@@
type T = { unsigned int };
T:n b;
type S != { unsigned int,size_t};
S s;
binary operator o != { &&, || };
@@
Review
Quoted 2 lines
(quoted diff) +1
-1-b o s +(S)b o s
The bug in Coccinelle is already fixed (are you impressed? I certainly am at the incredible speed and at the wonderful conversation I had!), and I verified with this semantic patch that our code is clean:
-- snip --
@@
type T = { unsigned int };
T:n b;
type S != { unsigned int, size_t, float, double }; S s;
binary operator o != { &&, || };
@@
s o
-b
+(S)b
@@
type T = { unsigned int };
T:n b;
type S != { unsigned int, size_t, float, double }; S s;
binary operator o != { &&, || };
@@
-b
+(S)b
o s
-- snap --
I do not currently plan on integrating this into contrib/coccinelle/, though, because it will take a while until we can benefit from the fix in Git's CI/PR runs.
Ciao,
Dscho
Re: validating signed/unsigned comparisons with Coccinelle, was Re: [PATCH 0/3] A couple of CI fixes regarding the built-in add --patch
Hi Dscho
Quoted 22 lines · Johannes Schindelin
Hi,
Quoted 18 lines · Johannes Schindelin via GitGitGadget
Note: This patch series is based on ds/github-actions-use-newer-ubuntu (but probably applies cleanly even on maint) because I tried to develop a semantic patch to fix similar issues in the code base. However, I've since run into what looks like a bug in Coccinelle
[https://github.com/coccinelle/coccinelle/issues/284]. My latest version of that semantic patch looks like this, but I stopped when running it on Git's source code triggered the bug for 66 of Git's .c files:@@
type T = { unsigned int };
T:n b;
type S != { unsigned int,size_t};
S s;
binary operator o != { &&, || };
@@(quoted diff) +1
-1-s o b +s o (S)b
Quoted 13 lines
Review
Quoted 9 lines
@@
type T = { unsigned int };
T:n b;
type S != { unsigned int,size_t};
S s;
binary operator o != { &&, || };
@@(quoted diff) +1
-1-b o s +(S)b o sThe bug in Coccinelle is already fixed (are you impressed? I certainly am at the incredible speed and at the wonderful conversation I had!), and I verified with this semantic patch that our code is clean:
Wow that's fast, I wonder if they would be interested in fixing the parsing bug we found with Peff's UNUSED() series. It's good news that the patch does not find any other problems.
Best Wishes
Phillip
Quoted 31 lines
-- snip --
@@
type T = { unsigned int };
T:n b;
type S != { unsigned int,size_t, float, double }; S s;
binary operator o != { &&, || };
@@
s o(quoted diff) +1
-1-b +(S)b@@
type T = { unsigned int };
T:n b;
type S != { unsigned int,size_t, float, double }; S s;
binary operator o != { &&, || };
@@(quoted diff) +1
-2-b +(S)b o s -- snap --I do not currently plan on integrating this into
contrib/coccinelle/, though, because it will take a while until we can benefit from the fix in Git's CI/PR runs.Ciao,
Dscho
Re: validating signed/unsigned comparisons with Coccinelle, was Re: [PATCH 0/3] A couple of CI fixes regarding the built-in add --patch
Hi Phillip,
Quoted 2 lines · Phillip Wood
I wonder if they would be interested in fixing the parsing bug we found with Peff's
UNUSED()series.
Could you point me to the relevant mail? I am sure that I can come up with an MCVE that will help them pinpoint the bug and fix it as quickly as they did with this here bug.
Still amazed at the turn-around time,
Dscho
Re: validating signed/unsigned comparisons with Coccinelle, was Re: [PATCH 0/3] A couple of CI fixes regarding the built-in add --patch
Quoted 7 lines · Johannes Schindelin
Hi Phillip,
Quoted 2 lines · Phillip Wood
I wonder if they would be interested in fixing the parsing bug we found with Peff's
UNUSED()series.
Quoted 6 lines
Could you point me to the relevant mail? I am sure that I can come up with an MCVE that will help them pinpoint the bug and fix it as quickly as they did with this here bug.
Still amazed at the turn-around time,
Dscho
Re: validating signed/unsigned comparisons with Coccinelle, was Re: [PATCH 0/3] A couple of CI fixes regarding the built-in add --patch
Quoted 4 lines · Junio C Hamano
Quoted 2 lines · Johannes Schindelin
Hi Phillip,
Quoted 8 lines
Quoted 8 lines · Phillip Wood
Quoted 3 lines
I wonder if they would be interested in fixing the parsing bug we found with Peff's
UNUSED()series.Could you point me to the relevant mail? I am sure that I can come up with an MCVE that will help them pinpoint the bug and fix it as quickly as they did with this here bug.
I think the reference Peff posted is much closer to the description of the "parsing bug". Sorry for the noise.
Quoted 2 lines
Quoted 2 lines
Still amazed at the turn-around time,
Dscho
Yes, indeed.
Thanks.
Re: validating signed/unsigned comparisons with Coccinelle, was Re: [PATCH 0/3] A couple of CI fixes regarding the built-in add --patch
In reply to Johannes SchindelinQuoted 8 lines · Johannes Schindelin
Quoted 2 lines · Phillip Wood
I wonder if they would be interested in fixing the parsing bug we found with Peff's
UNUSED()series.Could you point me to the relevant mail? I am sure that I can come up with an MCVE that will help them pinpoint the bug and fix it as quickly as they did with this here bug.
It's the sub-thread starting here:
https://lore.kernel.org/git/[email protected]/
I tried to put together a minimal example, which would be something like:
$ cat foo.c void old_function(int); void new_function(int);
#if defined(__GNUC__)
#define UNUSED(var) UNUSED_##var __attribute__((unused))
#else
#define UNUSED(var) UNUSED_##var
#endif
void foo(int UNUSED(bar), int x)
{
old_function(x);
}$ cat foo.cocci @@ expression E; @@ -old_function(E) +new_function(E)
but to my frustration, it actually works! If you add --verbose-parsing to the spatch command, you can see it complaining about some of the parsing. And it produces wrong outputs in the real-world with our actual unused.cocci file. I suspect the example semantic patch might just need to be a little more complicated.
-Peff
Hi Dscho
All three patches look good to me, thanks for working on them.
Phillip
Quoted 65 lines · Johannes Schindelin via GitGitGadget
This is an attempt to address the concern Junio raised in
https://lore.kernel.org/git/[email protected]/: originally motivated by a test suite failure when running Git's test suite in Visual Studio, this pulls out the signed vs unsigned fix whose implications are potentially much wider than Visual C.While in that space, I spent the time (which took almost as long as I expected
[https://lore.kernel.org/git/[email protected]/])
to craft a semantic patch to scrutinize Git's source code for similar issues (narrator's voice: there were no other instances, what did ya expect?).To verify the fix, I then worked on a patch to exercise the built-in git add -p in the test suite even when
NO_PERLis set, and while developing this patch and validating it, I got really puzzled that the add -p test case in t6132 did not need to be guarded behind a PERL prereq. So this patch series also includes a fix for that.The story arc that binds all of these patches together is that they all revolve around
NO_PERLand CI issues that involve git add --patch.Note: This patch series is based on ds/github-actions-use-newer-ubuntu (but probably applies cleanly even on maint) because I tried to develop a semantic patch to fix similar issues in the code base. However, I've since run into what looks like a bug in Coccinelle
[https://github.com/coccinelle/coccinelle/issues/284]. My latest version of that semantic patch looks like this, but I stopped when running it on Git's source code triggered the bug for 66 of Git's .c files:@@
type T = { unsigned int };
T:n b;
type S != { unsigned int,size_t};
S s;
binary operator o != { &&, || };
@@(quoted diff) +1
-1-s o b +s o (S)b@@
type T = { unsigned int };
T:n b;
type S != { unsigned int,size_t};
S s;
binary operator o != { &&, || };
@@(quoted diff) +1
-1-b o s +(S)b o sJohannes Schindelin (3):
add -p: avoid ambiguous signed/unsigned comparison
t3701: test the built-inadd -iregardless ofNO_PERLt6132(NO_PERL): do not run the scriptedadd -padd-patch.c | 2 +- t/t3701-add-interactive.sh | 4 ++-- t/t6132-pathspec-exclude.sh | 6 +++++- 3 files changed, 8 insertions(+), 4 deletions(-)base-commit:
ef46584831268a83591412a33783caf866867482
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-1340%2Fdscho%2Fbuilt-in-add-i-does-not-need-perl-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-1340/dscho/built-in-add-i-does-not-need-perl-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/1340