Skip to contentSkip to conversation

checkpatch: add new exception to repeated word check

PATCH Superseded Dwaipayan Ray 2 participants 3 messages Oct 17, 2020 Checkpatch

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/1 checkpatch: add new exception to repeated word check

Recently, commit 4f6ad8aa1eac ("checkpatch: move repeated word test") moved the repeated word test to check for more file types. But after this, if checkpatch.pl is run on MAINTAINERS, it generates several new warnings of the type:

WARNING: Possible repeated word: 'git'

For example:
WARNING: Possible repeated word: 'git'
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/rw/uml.git

So, the pattern "git git://..." is a false positive in this case.

There are several other combinations which may produce a wrong warning message, such as "@size size", ":Begin begin", etc.

Extend repeated word check to compare the characters before and after the word matches. If the preceding or succeeding character belongs to the exception list, the warning is avoided.

---
 scripts/checkpatch.pl | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)
scripts/checkpatch.pl +13 -2
index f1a4e61917eb..5dc5bf75c6e7 100755
@@ -3052,19 +3052,30 @@ sub process {
 
 # check for repeated words separated by a single space
 		if ($rawline =~ /^\+/ || $in_commit_log) {
+			pos($rawline) = 1 if (!$in_commit_log);
 			while ($rawline =~ /\b($word_pattern) (?=($word_pattern))/g) {
 
 				my $first = $1;
 				my $second = $2;
-
+				my $start_pos = $-[1];
+				my $end_pos = $+[2];
 				if ($first =~ /(?:struct|union|enum)/) {
 					pos($rawline) += length($first) + length($second) + 1;
 					next;
 				}
 
-				next if ($first ne $second);
+				next if (lc($first) ne lc($second));
 				next if ($first eq 'long');
 
+				# check for character before and after the word matches
+				my $start_char = '';
+				my $end_char = '';
+				$start_char = substr($rawline, $start_pos - 1, 1) if ($start_pos > ($in_commit_log? 0 : 1));
+				$end_char = substr($rawline, $end_pos, 1) if ($end_pos < length($rawline));
+
+				next if ($start_char =~ /^\S$/);
+				next if ($end_char !~ /^[\.\,\;\?\!\s]?$/);
+
 				if (WARN("REPEATED_WORD",
 					 "Possible repeated word: '$first'\n" . $herecurr) &&
 				    $fix) {
Signature · 1 line
2.27.0
maintainer
#1 lore raw
Quoted 11 lines · Dwaipayan Ray

Recently, commit 4f6ad8aa1eac ("checkpatch: move repeated word test") moved the repeated word test to check for more file types. But after this, if checkpatch.pl is run on MAINTAINERS, it generates several new warnings of the type:

WARNING: Possible repeated word: 'git'

For example:
WARNING: Possible repeated word: 'git'
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/rw/uml.git

Quoted 8 lines

So, the pattern "git git://..." is a false positive in this case.

There are several other combinations which may produce a wrong warning message, such as "@size size", ":Begin begin", etc.

Extend repeated word check to compare the characters before and after the word matches. If the preceding or succeeding character belongs to the exception list, the warning is avoided.

Not true.

This excludes any non-space character before the first word and excludes space or punctuation after the second word.

This also adds case insensitive word matching.

scripts/checkpatch.pl

Quoted 1 line
scripts/checkpatch.pl +0 -0

[]

Quoted 9 lines
scripts/checkpatch.pl +1 -0
@@ -3052,19 +3052,30 @@ sub process {
 
 # check for repeated words separated by a single space
 		if ($rawline =~ /^\+/ || $in_commit_log) {
+			pos($rawline) = 1 if (!$in_commit_log);
 			while ($rawline =~ /\b($word_pattern) (?=($word_pattern))/g) {
 
 				my $first = $1;
 				my $second = $2;

scripts/checkpatch.pl sub process { · line 3067

Quoted 10 lines
scripts/checkpatch.pl +3 -2
-
+				my $start_pos = $-[1];
+				my $end_pos = $+[2];
 				if ($first =~ /(?:struct|union|enum)/) {
 					pos($rawline) += length($first) + length($second) + 1;
 					next;
 				}
 
-				next if ($first ne $second);
+				next if (lc($first) ne lc($second));

case-insensitive matching

scripts/checkpatch.pl sub process { · line 3076

Quoted 9 lines
scripts/checkpatch.pl +7 -0
 				next if ($first eq 'long');
 
+				# check for character before and after the word matches
+				my $start_char = '';
+				my $end_char = '';
+				$start_char = substr($rawline, $start_pos - 1, 1) if ($start_pos > ($in_commit_log? 0 : 1));
+				$end_char = substr($rawline, $end_pos, 1) if ($end_pos < length($rawline));
+
+				next if ($start_char =~ /^\S$/);

non-space

scripts/checkpatch.pl sub process { · line 3077

Quoted 1 line
scripts/checkpatch.pl +1 -0
+				next if ($end_char !~ /^[\.\,\;\?\!\s]?$/);

space or punctuation.

trivia:

I believe using index would be ~50% faster than !~ here Perhaps more readable too.

				next if (index(" \t.,;?!", $end_char) >= 0);
Quoted 4 lines
scripts/checkpatch.pl +1 -0
+
 				if (WARN("REPEATED_WORD",
 					 "Possible repeated word: '$first'\n" . $herecurr) &&
 				    $fix) {
#2 lore raw
Quoted 22 lines · Joe Perches
Quoted 11 lines · Dwaipayan Ray

Recently, commit 4f6ad8aa1eac ("checkpatch: move repeated word test") moved the repeated word test to check for more file types. But after this, if checkpatch.pl is run on MAINTAINERS, it generates several new warnings of the type:

WARNING: Possible repeated word: 'git'

For example:
WARNING: Possible repeated word: 'git'
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/rw/uml.git

Quoted 8 lines

So, the pattern "git git://..." is a false positive in this case.

There are several other combinations which may produce a wrong warning message, such as "@size size", ":Begin begin", etc.

Extend repeated word check to compare the characters before and after the word matches. If the preceding or succeeding character belongs to the exception list, the warning is avoided.

Quoted 7 lines

Not true.

This excludes any non-space character before the first word and excludes space or punctuation after the second word.

This also adds case insensitive word matching.

Right, I didn't change the commit message. I will update it properly.

Quoted 38 lines

scripts/checkpatch.pl

Quoted 1 line
scripts/checkpatch.pl +0 -0

[]

Quoted 9 lines
scripts/checkpatch.pl +1 -0
@@ -3052,19 +3052,30 @@ sub process {
 
 # check for repeated words separated by a single space
              if ($rawline =~ /^\+/ || $in_commit_log) {
+                     pos($rawline) = 1 if (!$in_commit_log);
                      while ($rawline =~ /\b($word_pattern) (?=($word_pattern))/g) {
 
                              my $first = $1;
                              my $second = $2;

scripts/checkpatch.pl sub process { · line 3067

Quoted 10 lines
scripts/checkpatch.pl +3 -2
-
+                             my $start_pos = $-[1];
+                             my $end_pos = $+[2];
                              if ($first =~ /(?:struct|union|enum)/) {
                                      pos($rawline) += length($first) + length($second) + 1;
                                      next;
                              }
 
-                             next if ($first ne $second);
+                             next if (lc($first) ne lc($second));

case-insensitive matching

scripts/checkpatch.pl sub process { · line 3076

Quoted 9 lines
scripts/checkpatch.pl +7 -0
                              next if ($first eq 'long');
 
+                             # check for character before and after the word matches
+                             my $start_char = '';
+                             my $end_char = '';
+                             $start_char = substr($rawline, $start_pos - 1, 1) if ($start_pos > ($in_commit_log? 0 : 1));
+                             $end_char = substr($rawline, $end_pos, 1) if ($end_pos < length($rawline));
+
+                             next if ($start_char =~ /^\S$/);

non-space

scripts/checkpatch.pl sub process { · line 3077

Quoted 1 line
scripts/checkpatch.pl +1 -0
+                             next if ($end_char !~ /^[\.\,\;\?\!\s]?$/);
Quoted 9 lines

space or punctuation.

trivia:

I believe using index would be ~50% faster than !~ here Perhaps more readable too.

                                next if (index(" \t.,;?!", $end_char) >= 0);

Yes, it looks better. Thanks for the suggestion.

I will send in a new iteration after all of this is incorporated.

Thank you,
Dwaipayan.