checkpatch: fix false positives in REPEATED_WORD warning
PATCH v4 1/1 checkpatch: fix false positives in REPEATED_WORD warning
Presence of hexadecimal address or symbol results in false warning message by checkpatch.pl.
For example, running checkpatch on commit b8ad540dd4e4 ("mptcp: fix memory leak in mptcp_subflow_create_socket()") results in warning:
WARNING:REPEATED_WORD: Possible repeated word: 'ff'
00 00 00 00 00 00 00 00 00 2f 30 0a 81 88 ff ff ........./0.....
Similarly, the presence of list command output in commit results in an unnecessary warning.
For example, running checkpatch on commit 899e5ffbf246 ("perf record:
Introduce --switch-output-event") gives:
WARNING:REPEATED_WORD: Possible repeated word: 'root'
dr-xr-x---. 12 root root 4096 Apr 27 17:46 ..
Here, it reports 'ff' and 'root to be repeated, but it is in fact part of some address or code, where it has to be repeated.
In these cases, the intent of the warning to find stylistic issues in commit messages is not met and the warning is just completely wrong in this case.
To avoid these warnings, add additional regex check for the directory permission pattern and avoid checking the line for this class of warning. Similarly, to avoid hex pattern, check if the word consists of hex symbols and skip this warning if it is not among the common english words formed using hex letters.
A quick evaluation on v5.6..v5.8 showed that this fix reduces REPEATED_WORD warnings from 2797 to 907.
A quick manual check found all cases are related to hex output or list command outputs in commit messages.
--- scripts/checkpatch.pl | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
scripts/checkpatch.pl +14 -1
@@ -3049,7 +3049,9 @@ sub process { } # check for repeated words separated by a single space - if ($rawline =~ /^\+/ || $in_commit_log) { +# avoid false positive from list command eg, '-rw-r--r-- 1 root root' + if (($rawline =~ /^\+/ || $in_commit_log) && + $rawline !~ /[bcCdDlMnpPs\?-][rwxsStT-]{9}/) { pos($rawline) = 1 if (!$in_commit_log); while ($rawline =~ /\b($word_pattern) (?=($word_pattern))/g) {
@@ -3074,6 +3076,17 @@ sub process { next if ($start_char =~ /^\S$/); next if (index(" \t.,;?!", $end_char) == -1); + # avoid repeating hex occurrences like 'ff ff fe 09 ...' + my %allow_repeated_words = ( + add => '', + added => '', + bad => '', + be => '', + ); + if ($first =~ /\b[0-9a-f]{2,}\b/) { + next if (!exists($allow_repeated_words{lc($first)})); + } + if (WARN("REPEATED_WORD", "Possible repeated word: '$first'\n" . $herecurr) && $fix) {
Signature · 1 line
2.17.1
Quoted 66 lines · Aditya Srivastava
Presence of hexadecimal address or symbol results in false warning message by checkpatch.pl.
For example, running checkpatch on commit
b8ad540dd4e4("mptcp: fix memory leak inmptcp_subflow_create_socket()") results in warning:WARNING:REPEATED_WORD: Possible repeated word: 'ff'
00 00 00 00 00 00 00 00 00 2f 30 0a 81 88 ff ff ........./0.....Similarly, the presence of list command output in commit results in an unnecessary warning.
For example, running checkpatch on commit
899e5ffbf246("perf record:
Introduce --switch-output-event") gives:WARNING:REPEATED_WORD: Possible repeated word: 'root'
dr-xr-x---. 12 root root 4096 Apr 27 17:46 ..Here, it reports 'ff' and 'root to be repeated, but it is in fact part of some address or code, where it has to be repeated.
In these cases, the intent of the warning to find stylistic issues in commit messages is not met and the warning is just completely wrong in this case.
To avoid these warnings, add additional regex check for the directory permission pattern and avoid checking the line for this class of warning. Similarly, to avoid hex pattern, check if the word consists of hex symbols and skip this warning if it is not among the common english words formed using hex letters.
A quick evaluation on v5.6..v5.8 showed that this fix reduces
REPEATED_WORDwarnings from 2797 to 907.A quick manual check found all cases are related to hex output or list command outputs in commit messages.
Signed-off-by Aditya Srivastava <[email protected]>--- scripts/checkpatch.pl | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)scripts/checkpatch.pl +7
-1@@ -3049,7 +3049,9 @@ sub process { } # check for repeated words separated by a single space - if ($rawline =~ /^\+/ || $in_commit_log) { +# avoid false positive from list command eg, '-rw-r--r-- 1 root root' + if (($rawline =~ /^\+/ || $in_commit_log) && + $rawline !~ /[bcCdDlMnpPs\?-][rwxsStT-]{9}/) { pos($rawline) = 1 if (!$in_commit_log); while ($rawline =~ /\b($word_pattern) (?=($word_pattern))/g) {@@ -3074,6 +3076,17 @@ sub process { next if ($start_char =~ /^\S$/); next if (index(" \t.,;?!", $end_char) == -1); + # avoid repeating hex occurrences like 'ff ff fe 09 ...' + my %allow_repeated_words = ( + add => '', + added => '',
scripts/checkpatch.pl sub process { · line 3092
Quoted 10 lines
scripts/checkpatch.pl +7
-0+ bad => '', + be => '', + ); + if ($first =~ /\b[0-9a-f]{2,}\b/) { + next if (!exists($allow_repeated_words{lc($first)})); + } + if (WARN("REPEATED_WORD", "Possible repeated word: '$first'\n" . $herecurr) && $fix) {
Changes made in v4:
- Fix indentation
- Reduce the dictionary size for allowed words
- Apply changes over Dwaipayan's patch
(https://lore.kernel.org/linux-kernel-mentees/[email protected]/)
Joe Perches
maintainerQuoted 12 lines · Aditya Srivastava
Presence of hexadecimal address or symbol results in false warning message by checkpatch.pl.
For example, running checkpatch on commit
b8ad540dd4e4("mptcp: fix memory leak inmptcp_subflow_create_socket()") results in warning:WARNING:REPEATED_WORD: Possible repeated word: 'ff'
00 00 00 00 00 00 00 00 00 2f 30 0a 81 88 ff ff ........./0.....Similarly, the presence of list command output in commit results in an unnecessary warning.
Quoted 7 lines
For example, running checkpatch on commit
899e5ffbf246("perf record:
Introduce --switch-output-event") gives:WARNING:REPEATED_WORD: Possible repeated word: 'root'
dr-xr-x---. 12 root root 4096 Apr 27 17:46 ..Here, it reports 'ff' and 'root to be repeated, but it is in fact part
'root'
Quoted 7 lines
of some address or code, where it has to be repeated.
In these cases, the intent of the warning to find stylistic issues in commit messages is not met and the warning is just completely wrong in this case.
To avoid these warnings, add additional regex check for the
add an
Quoted 7 lines
directory permission pattern and avoid checking the line for this class of warning. Similarly, to avoid hex pattern, check if the word consists of hex symbols and skip this warning if it is not among the common english words formed using hex letters.
A quick evaluation on v5.6..v5.8 showed that this fix reduces
REPEATED_WORDwarnings from 2797 to 907.
How many of these 907 remaining are still false positive?
Quoted 2 lines
A quick manual check found all cases are related to hex output or list command outputs in commit messages.
You mean 1890 of the 2797 are now no longer reported and all 1890 were false positives yes?
scripts/checkpatch.pl
Quoted 1 line
scripts/checkpatch.pl +0
-0
[]
scripts/checkpatch.pl sub process { · line 3054
Quoted 9 lines
scripts/checkpatch.pl +0
-0@@ -3049,7 +3049,9 @@ sub process { }# check for repeated words separated by a single spacescripts/checkpatch.pl +3
-1- if ($rawline =~ /^\+/ || $in_commit_log) { +# avoid false positive from list command eg, '-rw-r--r-- 1 root root' + if (($rawline =~ /^\+/ || $in_commit_log) && + $rawline !~ /[bcCdDlMnpPs\?-][rwxsStT-]{9}/) {
Use maximal tab indentation and spaces to align please. 2 tabs, 4 spaces
Quoted 9 lines
pos($rawline) = 1 if (!$in_commit_log);
while ($rawline =~ /\b($word_pattern) (?=($word_pattern))/g) {scripts/checkpatch.pl +0
-0@@ -3074,6 +3076,17 @@ sub process { next if ($start_char =~ /^\S$/); next if (index(" \t.,;?!", $end_char) == -1);
scripts/checkpatch.pl sub process { · line 3085
Quoted 7 lines
scripts/checkpatch.pl +7
-0+ # avoid repeating hex occurrences like 'ff ff fe 09 ...' + my %allow_repeated_words = ( + add => '', + added => '', + bad => '', + be => '', + );
If perl caches this local hash declaration, fine, but I think it better to use 'our %allow_repeated_words' and move it so it's only declared using the file scope.
Quoted 1 line
+ if ($first =~ /\b[0-9a-f]{2,}\b/) {
This regex matches only lower case so it wouldn't match "Add".
I think this regex would be clearer using
/^[0-9a-f]+$/i or /^[A-Fa-f0-9]+$/
Quoted 4 lines · Joe Perches
Quoted 2 lines · Aditya Srivastava
A quick evaluation on v5.6..v5.8 showed that this fix reduces
REPEATED_WORDwarnings from 2797 to 907.
Quoted 8 lines
How many of these 907 remaining are still false positive?
Quoted 2 lines
A quick manual check found all cases are related to hex output or list command outputs in commit messages.
You mean 1890 of the 2797 are now no longer reported and all 1890 were false positives yes?
Yes. In v5.6..5.8, there were 2797 warnings for REPEATED_WORD, after these changes, they are reduced to 907.
However, many among these 907 must have been fixed by Dwaipayan's patch. I'll replace it with 1890 instead, for the better.
Quoted 9 lines
Quoted 9 lines
pos($rawline) = 1 if (!$in_commit_log);
while ($rawline =~ /\b($word_pattern) (?=($word_pattern))/g) {scripts/checkpatch.pl +0
-0@@ -3074,6 +3076,17 @@ sub process { next if ($start_char =~ /^\S$/); next if (index(" \t.,;?!", $end_char) == -1);
Quoted 12 lines
scripts/checkpatch.pl sub process { · line 3085
Quoted 7 lines
scripts/checkpatch.pl +7
-0+ # avoid repeating hex occurrences like 'ff ff fe 09 ...' + my %allow_repeated_words = ( + add => '', + added => '', + bad => '', + be => '', + );If perl caches this local hash declaration, fine, but I think it better to use 'our %allow_repeated_words' and move it so it's only declared using the file scope.
I ran checkpatch over few commits, it was working fine. But I'll move it to file scope, using 'our'. That should do as well.
Quoted 8 lines
Quoted 1 line
+ if ($first =~ /\b[0-9a-f]{2,}\b/) {
This regex matches only lower case so it wouldn't match "Add".
I think this regex would be clearer using
/^[0-9a-f]+$/i or /^[A-Fa-f0-9]+$/
Missed it. Will do.
Thanks
Aditya