Skip to contentSkip to conversation

checkpatch: add new cases to commit handling

RFC PATCH In review Dwaipayan Ray 2 participants 4 messages Nov 13, 2020 Checkpatch

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 checkpatch: add new cases to commit handling

Commit extraction in checkpatch fails in some cases. One of the most common false positives is a split line between "commit" and the git SHA of the commit.

Improve commit handling to reduce false positives.

Improvements:
- handle split line between commit and git SHA of commit.
- fix handling of split commit description.

A quick evaluation of 50k commits from v5.4 showed that the GIT_COMMIT_ID errors dropped from 1032 to 897. Most of these were split lines between commit and its hash.

---
 scripts/checkpatch.pl | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
scripts/checkpatch.pl +13 -1
index 024514946bed..f5ba2beac008 100755
@@ -2990,6 +2990,16 @@ sub process {
 			if ($line =~ /\bcommit\s+[0-9a-f]{5,}\s+\("([^"]+)"\)/i) {
 				$orig_desc = $1;
 				$hasparens = 1;
+			} elsif ($line =~ /^\s*[0-9a-f]{5,}\s+\("([^"]+)"\)/i &&
+				 defined $rawlines[$linenr-2] &&
+				 $rawlines[$linenr-2] =~ /\bcommit\s*$/i) {
+				$line =~ /^\s*[0-9a-f]{5,}\s+\("([^"]+)"\)/i;
+				$orig_desc = $1;
+				$hasparens = 1;
+				$space = 0;
+				$short = 0 if ($line =~ /\b[0-9a-f]{12,40}/i);
+				$long = 1 if ($line =~ /\b[0-9a-f]{41,}/i);
+				$case = 0 if ($line =~ /\b[0-9a-f]{5,40}[^A-F]/ && $rawlines[$linenr-2] =~ /\b[Cc]ommit\s*$/);
 			} elsif ($line =~ /\bcommit\s+[0-9a-f]{5,}\s*$/i &&
 				 defined $rawlines[$linenr] &&
 				 $rawlines[$linenr] =~ /^\s*\("([^"]+)"\)/) {
@@ -3001,7 +3011,9 @@ sub process {
 				$line =~ /\bcommit\s+[0-9a-f]{5,}\s+\("([^"]+)$/i;
 				$orig_desc = $1;
 				$rawlines[$linenr] =~ /^\s*([^"]+)"\)/;
-				$orig_desc .= " " . $1;
+				my $split_desc = $1;
+				$split_desc = " $split_desc" if ($line =~ /[\w\,\.]$/);
+				$orig_desc .= $split_desc;
 				$hasparens = 1;
 			}
 
Signature · 6 lines
2.27.0

_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees
#1 lore raw
Quoted 7 lines · Dwaipayan Ray

Commit extraction in checkpatch fails in some cases. One of the most common false positives is a split line between "commit" and the git SHA of the commit.

Improve commit handling to reduce false positives.

Quoted 8 lines

Improvements:
- handle split line between commit and git SHA of commit.
- fix handling of split commit description.

A quick evaluation of 50k commits from v5.4 showed that the GIT_COMMIT_ID errors dropped from 1032 to 897. Most of these were split lines between commit and its hash.

Can you send me the file of the evaluation, e.g., all contexts (two lines above and two lines below) around the warned line in the commits where the GIT_COMMIT_ID dropped?

Then, I can do a quick sanity check as well.

Lukas

Quoted 38 lines
---
 scripts/checkpatch.pl | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
scripts/checkpatch.pl +13 -2
index 024514946bed..f5ba2beac008 100755
@@ -2990,6 +2990,16 @@ sub process {
                        if ($line =~ /\bcommit\s+[0-9a-f]{5,}\s+\("([^"]+)"\)/i) {
                                $orig_desc = $1;
                                $hasparens = 1;
+                       } elsif ($line =~ /^\s*[0-9a-f]{5,}\s+\("([^"]+)"\)/i &&
+                                defined $rawlines[$linenr-2] &&
+                                $rawlines[$linenr-2] =~ /\bcommit\s*$/i) {
+                               $line =~ /^\s*[0-9a-f]{5,}\s+\("([^"]+)"\)/i;
+                               $orig_desc = $1;
+                               $hasparens = 1;
+                               $space = 0;
+                               $short = 0 if ($line =~ /\b[0-9a-f]{12,40}/i);
+                               $long = 1 if ($line =~ /\b[0-9a-f]{41,}/i);
+                               $case = 0 if ($line =~ /\b[0-9a-f]{5,40}[^A-F]/ && $rawlines[$linenr-2] =~ /\b[Cc]ommit\s*$/);
                        } elsif ($line =~ /\bcommit\s+[0-9a-f]{5,}\s*$/i &&
                                 defined $rawlines[$linenr] &&
                                 $rawlines[$linenr] =~ /^\s*\("([^"]+)"\)/) {
@@ -3001,7 +3011,9 @@ sub process {
                                $line =~ /\bcommit\s+[0-9a-f]{5,}\s+\("([^"]+)$/i;
                                $orig_desc = $1;
                                $rawlines[$linenr] =~ /^\s*([^"]+)"\)/;
-                               $orig_desc .= " " . $1;
+                               my $split_desc = $1;
+                               $split_desc = " $split_desc" if ($line =~ /[\w\,\.]$/);
+                               $orig_desc .= $split_desc;
                                $hasparens = 1;
                        }
 
--
Quoted 2 lines

2.27.0

_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

#2 lore raw
Quoted 18 lines · Lukas Bulwahn
Quoted 7 lines · Dwaipayan Ray

Commit extraction in checkpatch fails in some cases. One of the most common false positives is a split line between "commit" and the git SHA of the commit.

Improve commit handling to reduce false positives.

Quoted 8 lines

Improvements:
- handle split line between commit and git SHA of commit.
- fix handling of split commit description.

A quick evaluation of 50k commits from v5.4 showed that the GIT_COMMIT_ID errors dropped from 1032 to 897. Most of these were split lines between commit and its hash.

Quoted 6 lines

Can you send me the file of the evaluation, e.g., all contexts (two lines above and two lines below) around the warned line in the commits where the GIT_COMMIT_ID dropped?

Then, I can do a quick sanity check as well.

Thanks, Dwaipayan; I checked your file sent off-list and it looks good to not report on those cases.

Maybe we can now check the remaining 900 cases once again; are they all true positives or is there still a big false positive class?

Lukas
_______________________________________________
Linux-kernel-mentees mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/linux-kernel-mentees

#3 lore raw
Quoted 27 lines · Lukas Bulwahn
Quoted 18 lines · Lukas Bulwahn
Quoted 7 lines · Dwaipayan Ray

Commit extraction in checkpatch fails in some cases. One of the most common false positives is a split line between "commit" and the git SHA of the commit.

Improve commit handling to reduce false positives.

Quoted 8 lines

Improvements:
- handle split line between commit and git SHA of commit.
- fix handling of split commit description.

A quick evaluation of 50k commits from v5.4 showed that the GIT_COMMIT_ID errors dropped from 1032 to 897. Most of these were split lines between commit and its hash.

Quoted 6 lines

Can you send me the file of the evaluation, e.g., all contexts (two lines above and two lines below) around the warned line in the commits where the GIT_COMMIT_ID dropped?

Then, I can do a quick sanity check as well.

Quoted 7 lines

Thanks, Dwaipayan; I checked your file sent off-list and it looks good to not report on those cases.

Maybe we can now check the remaining 900 cases once again; are they all true positives or is there still a big false positive class?

Lukas

Hi,
I had roughly gone through the list and most of them are true positives. But there are two particular cases which may be false:

1) References: tag. (I don't know if it is a proper convention). There were about 50 of these:

References: 22b7a426bbe1 ("drm/i915/execlists: Preempt-to-busy")

But it is non uniform. Some commits use this tag to refer to links also.

2) Quotes inside commit title. (apart from the main enclosing quotes) I think by design checkpatch doesn't handle this case.

Thanks,
Dwaipayan.