update checkpatch.pl to version 0.10

This version brings a number of new checks, and a number of bug fixes. Of note:

- better categorisation and space checks for dual use unary/binary operators
- warn on deprecated use of {SPIN,RW}_LOCK_UNLOCKED
- check if/for/while with trailing ';' for hanging statements
- detect DOS line endings
- detect redundant casts for kalloc()

Andy Whitcroft (18):
Version: 0.10
asmlinkage is also a storage type
pull out inline specifiers
allow only some operators before a unary operator parenthesised values may span line ends
add additional attribute matching
handle sparse annotations within pointer type space checks support alternative function definition syntax for typedefs check if/for/while with trailing ';' for hanging statements fix output format for case checks
deprecate SPIN_LOCK_UNLOCKED and RW_LOCK_UNLOCKED allow complex macros with bracketing braces detect and report DOS line endings
fastcall is a valid function attribute
bracket spacing is ok for 'for'
categorise operators into unary/binary/definitions add heuristic to pick up on unannotated types remove spurious warnings from cat_vet

Dave Jones (1):
Make checkpatch warn about pointless casting of kalloc returns.

---
 scripts/checkpatch.pl |  254 +++++++++++++++++++++++++++++++++++++-----------
 1 files changed, 196 insertions(+), 58 deletions(-)

index dae7d30..59ad83c 100755
@@ -9,7 +9,7 @@ use strict;
 my $P = $0;
 $P =~ s@.*/@@g;
 
-my $V = '0.09';
+my $V = '0.10';
 
 use Getopt::Long qw(:config no_auto_abbrev);
 

@@ -212,6 +212,11 @@ sub ctx_block_level {
 
 	return ctx_block_get($linenr, $remain, 0, '{', '}', 0);
 }
+sub ctx_statement_level {
+	my ($linenr, $remain, $off) = @_;
+
+	return ctx_block_get($linenr, $remain, 0, '(', ')', $off);
+}
 
 sub ctx_locate_comment {
 	my ($first_line, $end_line) = @_;

@@ -255,13 +260,42 @@ sub ctx_has_comment {
 	return ($cmt ne '');
 }
 
+sub ctx_expr_before {
+	my ($line) = @_;
+
+	##print "CHECK<$line>\n";
+
+	my $pos = length($line) - 1;
+	my $count = 0;
+	my $c;
+
+	for (; $pos >= 0; $pos--) {
+		$c = substr($line, $pos, 1);
+		##print "CHECK: c<$c> count<$count>\n";
+		if ($c eq ')') {
+			$count++;
+		} elsif ($c eq '(') {
+			last if (--$count == 0);
+		}
+	}
+
+	##print "CHECK: result<" . substr($line, 0, $pos) . ">\n";
+
+	return substr($line, 0, $pos);
+}
+
 sub cat_vet {
 	my ($vet) = @_;
+	my ($res, $coded);
 
-	$vet =~ s/\t/^I/;
-	$vet =~ s/$/\$/;
+	$res = '';
+	while ($vet =~ /([^[:cntrl:]]*)([[:cntrl:]])/g) {
+		$coded = sprintf("^%c", unpack('C', $2) + 64);
+		$res .= $1 . $coded;
+	}
+	$res =~ s/$/\$/;
 
-	return $vet;
+	return $res;
 }
 
 my @report = ();

@@ -310,8 +344,17 @@ sub process {
 	my $first_line = 0;
 
 	my $Ident	= qr{[A-Za-z\d_]+};
-	my $Storage	= qr{extern|static};
-	my $Sparse	= qr{__user|__kernel|__force|__iomem|__must_check|__init_refok};
+	my $Storage	= qr{extern|static|asmlinkage};
+	my $Sparse	= qr{
+				__user|
+				__kernel|
+				__force|
+				__iomem|
+				__must_check|
+				__init_refok|
+				fastcall
+			}x;
+	my $Inline	= qr{inline|__always_inline|noinline};
 	my $NonptrType	= qr{
 				\b
 				(?:const\s+)?

@@ -345,11 +388,18 @@ sub process {
 				(?:\s+$Sparse)*
 			  }x;
 	my $Declare	= qr{(?:$Storage\s+)?$Type};
-	my $Attribute	= qr{const|__read_mostly|__init|__initdata|__meminit};
-
+	my $Attribute	= qr{
+				const|
+				__read_mostly|
+				__(?:mem|cpu|dev|)(?:initdata|init)
+			  }x;
 	my $Member	= qr{->$Ident|\.$Ident|\[[^]]*\]};
 	my $Lval	= qr{$Ident(?:$Member)*};
 
+	# Possible bare types.
+	my @bare = ();
+	my $Bare = $NonptrType;
+
 	# Pre-scan the patch looking for any __setup documentation.
 	my @setup_docs = ();
 	my $setup_docs = 0;

@@ -477,7 +527,11 @@ sub process {
 		next if ($realfile !~ /\.(h|c|s|S|pl|sh)$/);
 
 #trailing whitespace
-		if ($line =~ /^\+.*\S\s+$/ || $line =~ /^\+\s+$/) {
+		if ($line =~ /^\+.*\015/) {
+			my $herevet = "$here\n" . cat_vet($line) . "\n";
+			ERROR("DOS line endings\n" . $herevet);
+
+		} elsif ($line =~ /^\+.*\S\s+$/ || $line =~ /^\+\s+$/) {
 			my $herevet = "$here\n" . cat_vet($line) . "\n";
 			ERROR("trailing whitespace\n" . $herevet);
 		}

@@ -509,6 +563,30 @@ sub process {
 # Standardise the strings and chars within the input to simplify matching.
 		$line = sanitise_line($line);
 
+# Check for potential 'bare' types
+		if ($realcnt &&
+		    $line !~ /^.\s*(?:$Storage\s+)?(?:$Inline\s+)?$Type\b/ &&
+		    $line !~ /$Ident:\s*$/ &&
+		    $line !~ /^.\s*$Ident\s*\(/ &&
+		    ($line =~ /^.(?:$Storage\s+)?(?:$Inline\s+)?($Ident)\b/ ||
+		     $line =~ /^.\s*(?:$Storage\s+)?($Ident)\b\s*\**\s*$Ident\s*(?:;|=)/)) {
+			my $possible = $1;
+			if ($possible !~ /^(?:$Storage|$Type|DEFINE_\S+)$/ &&
+			    $possible ne 'goto' && $possible ne 'return' &&
+			    $possible ne 'struct' && $possible ne 'enum' &&
+			    $possible ne 'case' && $possible ne 'else' &&
+			    $possible ne 'typedef') {
+				#print "POSSIBLE<$possible>\n";
+				push(@bare, $possible);
+				my $bare = join("|", @bare);
+				$Bare	= qr{
+						\b(?:$bare)\b
+						(?:\s*\*+\s*const|\s*\*+|(?:\s*\[\s*\])+)?
+						(?:\s+$Sparse)*
+					  }x;
+			}
+		}
+
 #
 # Checks which may be anchored in the context.
 #

@@ -531,18 +609,19 @@ sub process {
 				}
 			}
 			if ($err ne '') {
-				ERROR("switch and case should be at the same indent\n$hereline\n$err\n");
+				ERROR("switch and case should be at the same indent\n$hereline$err");
 			}
 		}
 
 # if/while/etc brace do not go on next line, unless defining a do while loop,
 # or if that brace on the next line is for something else
 		if ($line =~ /\b(?:(if|while|for|switch)\s*\(|do\b|else\b)/ && $line !~ /^.#/) {
-			my @ctx = ctx_statement($linenr, $realcnt, 0);
+			my ($level, @ctx) = ctx_statement_level($linenr, $realcnt, 0);
 			my $ctx_ln = $linenr + $#ctx + 1;
 			my $ctx_cnt = $realcnt - $#ctx - 1;
 			my $ctx = join("\n", @ctx);
 
+			# Skip over any removed lines in the context following statement.
 			while ($ctx_cnt > 0 && $lines[$ctx_ln - 1] =~ /^-/) {
 				$ctx_ln++;
 				$ctx_cnt--;

@@ -553,6 +632,13 @@ sub process {
 				ERROR("That open brace { should be on the previous line\n" .
 					"$here\n$ctx\n$lines[$ctx_ln - 1]");
 			}
+			if ($level == 0 && $ctx =~ /\)\s*\;\s*$/ && defined $lines[$ctx_ln - 1]) {
+				my ($nlength, $nindent) = line_stats($lines[$ctx_ln - 1]);
+				if ($nindent > $indent) {
+					WARN("Trailing semicolon indicates no statements, indent implies otherwise\n" .
+						"$here\n$ctx\n$lines[$ctx_ln - 1]");
+				}
+			}
 		}
 
 #ignore lines not being added

@@ -619,7 +705,7 @@ sub process {
 # check for new typedefs, only function parameters and sparse annotations
 # make sense.
 		if ($line =~ /\btypedef\s/ &&
-		    $line !~ /\btypedef\s+$Type\s+\(\s*\*$Ident\s*\)\s*\(/ &&
+		    $line !~ /\btypedef\s+$Type\s+\(\s*\*?$Ident\s*\)\s*\(/ &&
 		    $line !~ /\b__bitwise(?:__|)\b/) {
 			WARN("do not add new typedefs\n" . $herecurr);
 		}

@@ -633,11 +719,11 @@ sub process {
 			ERROR("\"(foo $1 )\" should be \"(foo $1)\"\n" .
 				$herecurr);
 
-		} elsif ($line =~ m{$NonptrType(\*+)(?:\s+$Attribute)?\s+[A-Za-z\d_]+}) {
+		} elsif ($line =~ m{$NonptrType(\*+)(?:\s+(?:$Attribute|$Sparse))?\s+[A-Za-z\d_]+}) {
 			ERROR("\"foo$1 bar\" should be \"foo $1bar\"\n" .
 				$herecurr);
 
-		} elsif ($line =~ m{$NonptrType\s+(\*+)(?!\s+$Attribute)\s+[A-Za-z\d_]+}) {
+		} elsif ($line =~ m{$NonptrType\s+(\*+)(?!\s+(?:$Attribute|$Sparse))\s+[A-Za-z\d_]+}) {
 			ERROR("\"foo $1 bar\" should be \"foo $1bar\"\n" .
 				$herecurr);
 		}

@@ -693,7 +779,13 @@ sub process {
 		$opline = expand_tabs($opline);
 		$opline =~ s/^./ /;
 		if (!($line=~/\#\s*include/)) {
-			my @elements = split(/(<<=|>>=|<=|>=|==|!=|\+=|-=|\*=|\/=|%=|\^=|\|=|&=|=>|->|<<|>>|<|>|=|!|~|&&|\|\||,|\^|\+\+|--|;|&|\||\+|-|\*|\/\/|\/)/, $opline);
+			my $ops = qr{
+				<<=|>>=|<=|>=|==|!=|
+				\+=|-=|\*=|\/=|%=|\^=|\|=|&=|
+				=>|->|<<|>>|<|>|=|!|~|
+				&&|\|\||,|\^|\+\+|--|&|\||\+|-|\*|\/
+			}x;
+			my @elements = split(/($ops|;)/, $opline);
 			my $off = 0;
 			for (my $n = 0; $n < $#elements; $n += 2) {
 				$off += length($elements[$n]);

@@ -733,7 +825,60 @@ sub process {
 				my $ptr = (" " x $off) . "^";
 				my $hereptr = "$hereline$ptr\n";
 
-				##print "<$s1:$op:$s2> <$elements[$n]:$elements[$n + 1]:$elements[$n + 2]>\n";
+				# Classify operators into binary, unary, or
+				# definitions (* only) where they have more
+				# than one mode.
+				my $unary_ctx = $prevline . $ca;
+				$unary_ctx =~ s/^./ /;
+				my $is_unary = 0;
+				my $Unary = qr{
+					(?:
+						^|;|,|$ops|\(|\?|:|
+						\(\s*$Type\s*\)|
+						$Type|
+						return|case|else|
+						\{|\}|
+						\[|
+						^.\#\s*define\s+$Ident\s*(?:\([^\)]*\))?|
+						^.\#\s*else|
+						^.\#\s*endif|
+						^.\#\s*(?:if|ifndef|ifdef)\b.*
+					)\s*(?:|\\)\s*$
+				}x;
+				my $UnaryFalse = qr{
+					sizeof\s*\(\s*$Type\s*\)\s*$
+				}x;
+				my $UnaryDefine = qr{
+					 (?:$Type|$Bare)\s*|
+					 (?:$Type|$Bare).*,\s*\**
+				}x;
+				if ($op eq '-' || $op eq '&' || $op eq '*') {
+					# An operator is binary if the left hand
+					# side is a value.  Pick out the known
+					# non-values.
+					if ($unary_ctx =~ /$Unary$/s &&
+					    $unary_ctx !~ /$UnaryFalse$/s) {
+						$is_unary = 1;
+
+					# Special handling for ')' check if this
+					# brace represents a conditional, if so
+					# we are unary.
+					} elsif ($unary_ctx =~ /\)\s*$/) {
+						my $before = ctx_expr_before($unary_ctx);
+						if ($before =~ /(?:for|if|while)\s*$/) {
+							$is_unary = 1;
+						}
+					}
+
+					# Check for type definition for of '*'.
+					if ($op eq '*' && $unary_ctx =~ /$UnaryDefine$/) {
+						$is_unary = 2;
+					}
+				}
+
+				#if ($op eq '-' || $op eq '&' || $op eq '*') {
+				#	print "UNARY: <$is_unary $a:$op:$c> <$ca:$op:$cc> <$unary_ctx>\n";
+				#}
 
 				# ; should have either the end of line or a space or \ after it
 				if ($op eq ';') {

@@ -757,9 +902,16 @@ sub process {
 						ERROR("need space after that '$op' $at\n" . $hereptr);
 					}
 
-				# unary ! and unary ~ are allowed no space on the right
-				} elsif ($op eq '!' or $op eq '~') {
-					if ($ctx !~ /[WOEB]x./) {
+				# '*' as part of a type definition -- reported already.
+				} elsif ($op eq '*' && $is_unary == 2) {
+					#warn "'*' is part of type\n";
+
+				# unary operators should have a space before and
+				# none after.  May be left adjacent to another
+				# unary operator, or a cast
+				} elsif ($op eq '!' || $op eq '~' ||
+				         ($is_unary && ($op eq '*' || $op eq '-' || $op eq '&'))) {
+					if ($ctx !~ /[WEB]x./ && $ca !~ /(?:\)|!|~|\*|-|\&|\||\+\+|\-\-|\{)$/) {
 						ERROR("need space before that '$op' $at\n" . $hereptr);
 					}
 					if ($ctx =~ /.xW/) {

@@ -775,39 +927,13 @@ sub process {
 						ERROR("no space before that '$op' $at\n" . $hereptr);
 					}
 
-				# & is both unary and binary
-				# unary:
-				# 	a &b
-				# binary (consistent spacing):
-				#	a&b		OK
-				#	a & b		OK
-				#
-				# boiling down to: if there is a space on the right then there
-				# should be one on the left.
-				#
-				# - is the same
-				#
-				} elsif ($op eq '&' or $op eq '-') {
-					if ($ctx !~ /VxV|[EW]x[WE]|[EWB]x[VO]/) {
-						ERROR("need space before that '$op' $at\n" . $hereptr);
-					}
-
-				# * is the same as & only adding:
-				# type:
-				# 	(foo *)
-				#	(foo **)
-				#
-				} elsif ($op eq '*') {
-					if ($ca !~ /$Type$/ && $cb !~ /(\*$;|$;\*)/ &&
-					    $ctx !~ /VxV|[EW]x[WE]|[EWB]x[VO]|OxV|WxB|BxB/) {
-						ERROR("need space before that '$op' $at\n" . $hereptr);
-					}
-
 				# << and >> may either have or not have spaces both sides
-				} elsif ($op eq '<<' or $op eq '>>' or $op eq '+' or $op eq '/' or
-					 $op eq '^' or $op eq '|')
+				} elsif ($op eq '<<' or $op eq '>>' or
+					 $op eq '&' or $op eq '^' or $op eq '|' or
+					 $op eq '+' or $op eq '-' or
+					 $op eq '*' or $op eq '/')
 				{
-					if ($ctx !~ /VxV|WxW|VxE|WxE/) {
+					if ($ctx !~ /VxV|WxW|VxE|WxE|VxO/) {
 						ERROR("need consistent spacing around '$op' $at\n" .
 							$hereptr);
 					}

@@ -865,10 +991,12 @@ sub process {
 		}
 
 # check spacing on paretheses
-		if ($line =~ /\(\s/ && $line !~ /\(\s*$/) {
+		if ($line =~ /\(\s/ && $line !~ /\(\s*(?:\\)?$/ &&
+		    $line !~ /for\s*\(\s+;/) {
 			ERROR("no space after that open parenthesis '('\n" . $herecurr);
 		}
-		if ($line =~ /\s\)/) {
+		if ($line =~ /\s\)/ && $line !~ /^.\s*\)/ &&
+		    $line !~ /for\s*\(.*;\s+\)/) {
 			ERROR("no space before that close parenthesis ')'\n" . $herecurr);
 		}
 

@@ -926,10 +1054,10 @@ sub process {
 # multi-statement macros should be enclosed in a do while loop, grab the
 # first statement and ensure its the whole macro if its not enclosed
 # in a known goot container
-		if (($prevline=~/\#define.*\\/) and
-		   !($prevline=~/do\s+{/) and !($prevline=~/\(\{/) and
-		   !($line=~/do.*{/) and !($line=~/\(\{/) and
-		   !($line=~/^.\s*$Declare\s/)) {
+		if ($prevline =~ /\#define.*\\/ &&
+		   $prevline !~/(?:do\s+{|\(\{|\{)/ &&
+		   $line !~ /(?:do\s+{|\(\{|\{)/ &&
+		   $line !~ /^.\s*$Declare\s/) {
 			# Grab the first statement, if that is the entire macro
 			# its ok.  This may start either on the #define line
 			# or the one below.

@@ -1027,6 +1155,11 @@ sub process {
 			WARN("Use of volatile is usually wrong: see Documentation/volatile-considered-harmful.txt\n" . $herecurr);
 		}
 
+# SPIN_LOCK_UNLOCKED & RW_LOCK_UNLOCKED are deprecated
+		if ($line =~ /\b(SPIN_LOCK_UNLOCKED|RW_LOCK_UNLOCKED)/) {
+			ERROR("Use of $1 is deprecated: see Documentation/spinlocks.txt\n" . $herecurr);
+		}
+
 # warn about #if 0
 		if ($line =~ /^.#\s*if\s+0\b/) {
 			CHK("if this code is redundant consider removing it\n" .

@@ -1073,8 +1206,8 @@ sub process {
 
 # check the location of the inline attribute, that it is between
 # storage class and type.
-		if ($line =~ /$Type\s+(?:inline|__always_inline|noinline)\b/ ||
-		    $line =~ /\b(?:inline|__always_inline|noinline)\s+$Storage/) {
+		if ($line =~ /\b$Type\s+$Inline\b/ ||
+		    $line =~ /\b$Inline\s+$Storage\b/) {
 			ERROR("inline keyword should sit between storage class and type\n" . $herecurr);
 		}
 

@@ -1091,6 +1224,11 @@ sub process {
 				CHK("__setup appears un-documented -- check Documentation/kernel-parameters.txt\n" . $herecurr);
 			}
 		}
+
+# check for pointless casting of kmalloc return
+		if ($line =~ /\*\s*\)\s*k[czm]alloc\b/) {
+			WARN("unnecessary cast may hide bugs, see http://c-faq.com/malloc/mallocnocast.html\n" . $herecurr);
+		}
 	}
 
 	if ($chk_patch && !$is_patch) {

your checkpatch patch itself produces 22 warnings ...

i ran it over kernel/sched.c and there are many bogus warnings that i reported to you earlier:

WARNING: multiple assignments should be avoided
#2319:
+ max_load = this_load = total_load = total_pwr = 0;

and new bogus ones:

ERROR: need consistent spacing around '*' (ctx:WxV) #5287:
+ mode_t mode, proc_handler *proc_handler)

ERROR: need consistent spacing around '*' (ctx:WxV) #5328:
+static ctl_table *sd_alloc_ctl_cpu_table(int cpu)

ERROR: need space before that '*' (ctx:VxV)
#209:
+# define INIT_TASK_GRP_LOAD 2*NICE_0_LOAD

why did you ignore my feedback? Ever since v8 the quality of checkpatch.pl has been getting worse and worse as there are way too many false positives. I'm still stuck on v8 for my own use, v9 and v10 is unusable.

Ingo

--------------->
WARNING: line over 80 characters
#174: FILE: scripts/checkpatch.pl:572:
+ $line =~ /^.\s*(?:$Storage\s+)?($Ident)\b\s*\**\s*$Ident\s*(?:;|=)/)) {

WARNING: line over 80 characters
#186: FILE: scripts/checkpatch.pl:584:
+ (?:\s*\*+\s*const|\s*\*+|(?:\s*\[\s*\])+)?

WARNING: line over 80 characters
#200: FILE: scripts/checkpatch.pl:612:
+ ERROR("switch and case should be at the same indent\n$hereline$err");

WARNING: line over 80 characters
#208: FILE: scripts/checkpatch.pl:619:
+ my ($level, @ctx) = ctx_statement_level($linenr, $realcnt, 0);

WARNING: line over 80 characters
#213: FILE: scripts/checkpatch.pl:624:
+ # Skip over any removed lines in the context following statement.

WARNING: line over 80 characters
#221: FILE: scripts/checkpatch.pl:635:
+ if ($level == 0 && $ctx =~ /\)\s*\;\s*$/ && defined $lines[$ctx_ln - 1]) {

WARNING: line over 80 characters
#222: FILE: scripts/checkpatch.pl:636:
+ my ($nlength, $nindent) = line_stats($lines[$ctx_ln - 1]);

WARNING: line over 80 characters
#224: FILE: scripts/checkpatch.pl:638:
+ WARN("Trailing semicolon indicates no statements, indent implies otherwise\n" .

WARNING: line over 80 characters
#225: FILE: scripts/checkpatch.pl:639:
+ "$here\n$ctx\n$lines[$ctx_ln - 1]");

WARNING: line over 80 characters
#245: FILE: scripts/checkpatch.pl:722:
+ } elsif ($line =~ m{$NonptrType(\*+)(?:\s+(?:$Attribute|$Sparse))?\s+[A-Za-z\d_]+}) {

WARNING: line over 80 characters
#250: FILE: scripts/checkpatch.pl:726:
+ } elsif ($line =~ m{$NonptrType\s+(\*+)(?!\s+(?:$Attribute|$Sparse))\s+[A-Za-z\d_]+}) {

WARNING: line over 80 characters
#288: FILE: scripts/checkpatch.pl:842:
+ ^.\#\s*define\s+$Ident\s*(?:\([^\)]*\))?|

WARNING: line over 80 characters
#313: FILE: scripts/checkpatch.pl:867:
+ my $before = ctx_expr_before($unary_ctx);

WARNING: line over 80 characters
#314: FILE: scripts/checkpatch.pl:868:
+ if ($before =~ /(?:for|if|while)\s*$/) {

WARNING: line over 80 characters
#320: FILE: scripts/checkpatch.pl:874:
+ if ($op eq '*' && $unary_ctx =~ /$UnaryDefine$/) {

WARNING: line over 80 characters
#326: FILE: scripts/checkpatch.pl:880:
+ # print "UNARY: <$is_unary $a:$op:$c> <$ca:$op:$cc> <$unary_ctx>\n";

WARNING: line over 80 characters
#338: FILE: scripts/checkpatch.pl:905:
+ # '*' as part of a type definition -- reported already.

WARNING: line over 80 characters
#346: FILE: scripts/checkpatch.pl:913:
+ ($is_unary && ($op eq '*' || $op eq '-' || $op eq '&'))) {

WARNING: line over 80 characters
#347: FILE: scripts/checkpatch.pl:914:
+ if ($ctx !~ /[WEB]x./ && $ca !~ /(?:\)|!|~|\*|-|\&|\||\+\+|\-\-|\{)$/) {

WARNING: line over 80 characters
#387: FILE: scripts/checkpatch.pl:932:
+ $op eq '&' or $op eq '^' or $op eq '|' or

WARNING: line over 80 characters
#432: FILE: scripts/checkpatch.pl:1160:
+ ERROR("Use of $1 is deprecated: see Documentation/spinlocks.txt\n" . $herecurr);

WARNING: line over 80 characters
#456: FILE: scripts/checkpatch.pl:1230:
+ WARN("unnecessary cast may hide bugs, see http://c-faq.com/malloc/mallocnocast.html\n" . $herecurr);

Your patch has style problems, please review. If any of these errors are false positives report them to the maintainer, see CHECKPATCH in MAINTAINERS.

That warning is non-bogus, although this is one of the bogosities which I personally don't bother fixing or making a fuss about.

But I do think it detracts from the readability of the code, and from patches which later alter that code. A bit.

I tend to agree, watching the automatic replies from checkpatch, the majority of these are "justifiable" in their context. I think I'll lower this one to a CHECK in the next release.

-apw

what matters is that only items should be displayed that i _can_ fix. With v8 i was able to make kernel/sched*.c almost noise-free, but with v9 and v10 that's not possible anymore. And the moment the default output of the tool cannot be made 'empty', we've lost the biggest battle. Seeing the same bogus (or borderline) warnings again and again destroys the biggest dynamic that could get people to use this tool more
often: the gratification of having a 'perfectly clean' file/patch.

And this is not about any particular false positive. I dont mind an "advanced mode" non-default opt-in option for the script, if someone is interested in borderline or hard to judge warnings too, but these default false positives are _lethal_ for a tool like this. (and i made this point before.) This is a _fundamental_ thing, and i'm still not sure whether you accept and understand that point. This is very basic and very important, and this isnt the first (or second) time i raised this.

Ingo

You are striving for a level of perfection that is simply not achieveable. v8 is silent about sched.c because it is not checking very much of it, the logical extension of your position is to run 0.1 as that didn't check anything. v9 and 10 carry checks for most of the binary operators which were not there before. Due as I have mentioned before to the complexity of handling the unary/binary scism that C has for those operators and our different spacing requirement for each mode.

Whilst I can see that it is gratifying that a patch or file is violations free where there are stylistic aberations in it they should be reported. Where those are questionble showing those as "CHECK" is not unreasonable. I will add a "--no-check" for you so that those are suppressed based on the assumption you know what you are doing.

I think it is clear that we differ on what should and should not be output by default. Clever people are able to opt out of the warnings, of things they think they dissagree with. It is the people with little experience who need the most guidance and those people who the tool should target by default. You cannot expect someone with no experience to know they need to add '--i-need-more-help' whereas _you_ I can expect to say '--leave-me-alone' or indeed to make the call that the output is plain wrong and _you_ know you should ignore it.

Fundamentally I am not trying to help the people who are careful but those who do not know better. As for the false positives, those I am always interested in and always striving to remove, as they annoy me as much as the next man.

-apw

Am Freitag, 28. September 2007 schrieb Andy Whitcroft:

I dont think Ingo is looking for perfection. Its about a different optimization goals.

Let me put it this way:

checkpatch in advanced mode:
- I want to be able to see as many possible problems (this is the optimization goal)
- I accept that I get false positives
- not useful for git and mail traffic

checkpatch in safe mode:
- I never want a false positive (different optimization goal!)
- I accept that I will miss several real bugs because several tricky tests are disabled
- useful for git and mail traffic

Christian

Maybe checkpatch.pl needs an option '-W' to turn on/off those vexed "noise". (It seems that 'q|quiet' doesn't do as much as what it hints.)

;-)

"Bill, look, we understand that you're interested in selling us this
operating system, but compare it to ours.  We can't possibly take such
a retrograde step."

Make checkpatch.pl a C language parser, then it can handle all the whitespace violations without false positives. :-)

Well, yes, that's the root problem.
checkpatch is a regex tool, not a C parser.

And like Andy commented in another mail in this thread, (esp. since it's not a C parser), it's just a guide.

---
~Randy

you _still_ fail to understand (let alone address), the fundamental issues i raised many times. (see below for details)

the most usable checkpatch.pl version was v6/v7. It went downhill after that. Look at the script size versus the number of complaints about kernel/sched.c:

size # warnings
----------------------------------------

  25383  checkpatch.pl.v6       5
  26038  checkpatch.pl.v7       6
  29603  checkpatch.pl.v8       65
  31160  checkpatch.pl.v9       24
  34950  checkpatch.pl.v10      28

i.e. size did not increase all that significantly, still the number of warnings has increased significantly - with little benefit.

and here's the breakdown for v10: out of 28 warnings, only 6 are legitimate! So the signal to noise ratio has worsened significantly since v6. One warning per 300 lines of sched.c is _not manageable_. It translates to _over 25 thousand false positives_ for the whole kernel.

every false warning has additive costs: i will have to ignore it from that point on, forever - and i frequently have to re-check the same thing again, and again - just to discover that the warning is still bogus.

the mails from me in your mbox during the past few months should be proof that i'm fully constructive regarding this matter and that i'm not striving for 100% level of perfection. I'm simply stating the obvious:
your tool is less and less useful with every new version and more troubling than that is your apparent inability to realize what this whole issue is about. (see below for details)

you still dont get the point of this. This isnt about writing a tool that 'can' find something to complain about. This whole kernel source code quality task is about:

_making kernel source code quality better_

not more, not less. If your tool outputs way too many false positives and way too many unimportant borderline cases, people will ignore the tool, and consequently YOU MAKE THE KERNEL'S SOURCE CODE WORSE than it could be. How hard is that to understand?? Having zero (or near zero) output from a checker mechanism is FUNDAMENTAL.

( and note that i was a goddamn early adopter of your tool, i'm a tester
and i gave you feedback, and i'm one of the few kernel hackers who actually use your script as a mandatory component of their patch-toolchain here and today. But your unchanged fundamentalistic attitude has turned me from a happy supporter of checkpatch.pl into an almost-opponent. If anything, that should be a warning shot for you. )

then remove most of those 22 false positives as a starter. I dont care if it's "hard/impossible" or not. If it cannot be coded like that, DONT output that particular type of check by default. Let people OPT IN if there's a reasonable chance for false positives.

( the same is true for gcc warnings: false positives are a huge PITA and
they _CAUSE_ bugs because people have learned to ignore gcc warnings and will accidentally ignore real bugs too. )

Ingo

That is unfair. Every time we discuss it I state that I disagree that hiding mostly useful tests is a good thing. I would love the tests to be 100% accurate, but if I removed all the tests that can false positive I would literally have none. There is a balance to be struck and we have significantly different ideas on where the balance is.

That is just stupid, I am no fundamentalist. I personally care not what tests there are or indeed if they are enabled by default. I personally feel you are more capable of turning off things you think are wrong, and as such the default should be to offer all of the tests. You disagree, that doesn't make either of us fundamentalist.

Yes, but dispite that we don't just turn warnings off. Even though they produce false positives, as having them has some benefit. My contention is that checkpatch is following that same approach, risking some false positives to try and catch problems.

Anyhow. I have already added a --check/--no-check option which controls the more subjective tests which will be in the next release; though its likely the option name will be something more useful by then.

The only question is whether this should default to on. You are voting off. I personally think on.

Andrew? Randy? Joel?

-apw

Hi Andy,

Are you disagreeing with the numbers Ingo posted? 25,000 false positives for the kernel is beyond silly... Existing conventions should matter a lot and the default configuration for a static code checker should really be 100%. So why not hide the potentially useful warnings under -Wtoo-strict or similar command line option?

I have not run across the whole kernel to find out, his estimation is likely high as his sample (mm/sched.c) includes a particular construct (multiple assignment) which is reported and overly common in that piece of code. If I take mm/signal.c (also big) I get 1/1000 files, and those two are easily fixed. I should note it shows some 62 actual real violations in that file.

I do receive automated checks of every patch posted to lkml and I work to remove the false positives from them. The false positive ratio is very low in those reports and it those which drive my development effort.

checkpatch is a work in progress and likely will be for many years to come.

I have propose we 'gate' those subjective tests, and have asked for input on that thread on the default for those tests.

-apw

The main audience of this is new contributors, who should have more verbose output, including nitpicky things like multiple assignments per line. The default should target them. More advanced users can certainly use a flag that says "give me only the real errors".

It might be a good idea to have three levels.
--really-errors
--really-picky
--really-experimental

Only with better names. --really-picky would be default, but would only include tests that have a very very high ratio of hits to false positives, but would still call out things like multiple assignments per line. --really-errors we could call the Igno level. --really-experimental would call out all issues, even on checks that generate a fair number of false positives.

-Joel

I think that if someone wants --verbose mode (--noisy, --beginner), they should have to ask for it.

---
~Randy

oy, knock it off.

-strict?

off, I'd say. That way people are more likely to use it. Or, more accurately, will have less excuses to not use it.

Really not trying to have a bunfight, honest.

Ok, then I think thats 2 for on and 3 for off. So off it is.

I was tending towards --subjective for the tests which are err more subjective. --strict is good too. Perhaps I'll put both of those in as aliases.

I will also review the tests which are warnings and checks (subjective) and see if any are now miss-categorised.

-apw

here is an update wrt. the latest checkpatch.pl-next version (v11-to-be), about kernel/sched.c warnings:

    35948  checkpatch.pl.v11pre   11

so things are heading in the right direction :)

of those 11 warnings, 6 are correct warnings (4 will be solved via KERN_CONT, 1 will be solved via a proper include file, and 1 is an overlength line), 4 are borderline warnings (easily fixed) and only one is a false positive! So v11-to-be gets the "best checkpatch.pl ever" badge from me :)

The false positive is:

ERROR: need consistent spacing around '*' (ctx:WxV) #5322:
+static ctl_table *sd_alloc_ctl_cpu_table(int cpu)

                    ^

i think checkpatch.pl mistook this function definition as an arithmetic expression?

But, there's a cleanliness bug underlying this false positive:
'ctl_table' is a typedef, and it would be cleaner to use 'struct ctl_table' thoughout the kernel. When running checkpatch.pl over include/linux/sysctl.h, it warns about the typedef:

WARNING: do not add new typedefs
#944:
+typedef struct ctl_table ctl_table;

(but mistaking that function for an arithmetic expression is still a bug i think.)

nice work Andy!

Ingo

Hi Andy.

The original purpose was to catch the most tivial coding style errors. But then people started to be far too innovative and we now ended up with a checkpatch that try to check for every lillte glory detail.

It would be much preferable to have a checkpatch - and that may be an incarnation of the current one or a new one.
What it should do would be to catch the trivialities that we see happens in many patch submissions like:

a) wrong patch format (-p1, unified diff)
b) Whitespace versus tabs
c) PascalCasing in new functions
d) excessive line length
e) C99 comments '//' (not that I see why they are banned)

and maybe a few more trivial chacks.

This would benefit from a very low false-positive rate and it could then be used as a patch-bot.

As it is now where it tries to check for everything and a bit more it generates too much noise so a patch-bot usage is not possible. And users get annoyed by the excessive output.

If we then have the same script that in a -pedantic mode generate more noise - fine. But leave the default simple.

Sam

well, the two variants is:

	max_load = this_load = total_load = total_pwr = 0;

	max_load = 0;
	this_load = 0;
	total_load = 0;
	total_pwr = 0;

and the first one is more readable and more compact. (as long as the conceptual 'type' of the variables is the same - which it is in this case.)

anyway, this is something where reasonable people might disagree, and a tool should not force it one way or another. And this is the second time i raised this very example and Andy ignored my feedback and failed to notice the structural problem behind it (that a tool should only warn by default if it is _sure_ that there is a problem - otherwise the tool cannot be used for effective [i.e. automated] quality control), so i'm raising this point again, in a slightly more irritated tone ;-)

Ingo

I think if you read your incoming email you will see nothing of the sort. I have discussed this with you and in public. The multiple assignment check you dissagree with and we have softened it in direct response to that dislike. However, the main proponent of this existing wanted that check. Therefore it has stayed. The other false positives you report are real. Some are fixed in my development version, others are not. They come from the fact that I was asked for better checks on '*' and the like in its binary mode. To get that I had to actually start telling unary and binary uses of the same operator appart. That is hard in the face of typedef'd types. I am working to make it better.

However, the key here is that it will never be 100%, not without becoming a try C parser. The output is a _guide_ if you don't like its output ignore the reports you dislike. I for one send out patches with style violations where I deem that the code is better that way.

-apw