checkpatch: warn when patch exceeds a maximum message size

PATCH In review Bruce Allan 2 participants 9 messages Jun 6, 2014 Checkpatch

Share & tools

Bruce Allan

author

2014-06-06 00:01 UTC #0 lore raw

PATCH 1/1 checkpatch: warn when patch exceeds a maximum message size

Patch submissions which exceed a maximum message size imposed by email list servers can be silently blocked. Warn when a patch message exceeds the maximum message size (100,000 characters) allowed by Majordomo at vger.kernel.org since that is where most Linux email lists are served. Also provide a command-line option to specify a different maximum size.

Signed-off-by Bruce Allan <[email protected]>

---

 scripts/checkpatch.pl |   15 +++++++++++++++
 1 file changed, 15 insertions(+)

scripts/checkpatch.pl +15 -0

index 34eb216..79bd7fd6 100755
@@ -40,6 +40,7 @@ my @ignore = ();
 my $help = 0;
 my $configuration_file = ".checkpatch.conf";
 my $max_line_length = 80;
+my $max_msg_size = 100000;	# Max allowed by Majordomo at vger.kernel.org
 my $ignore_perl_version = 0;
 my $minimum_perl_version = 5.10.0;

@@ -62,6 +63,7 @@ Options:
   --types TYPE(,TYPE2...)    show only these comma separated message types
   --ignore TYPE(,TYPE2...)   ignore various comma separated message types
   --max-line-length=n        set the maximum line length, if exceeded, warn
+  --max-msg-size=n           set the maximum message size, if exceeded, warn
   --show-types               show the message "types" in the output
   --root=PATH                PATH to the kernel tree root
   --no-summary               suppress the per-file summary

@@ -130,6 +132,7 @@ GetOptions(
 	'types=s'	=> \@use,
 	'show-types!'	=> \$show_types,
 	'max-line-length=i' => \$max_line_length,
+	'max-msg-size=i' => \$max_msg_size,
 	'root=s'	=> \$root,
 	'summary!'	=> \$summary,
 	'mailback!'	=> \$mailback,

@@ -4600,6 +4603,18 @@ sub process {
 		exit(0);
 	}
 
+	# Warn if patch exceeds the maximum message size
+	if (!$file) {
+		my $cnt_chars = 0;
+		foreach my $rawline (@rawlines) {
+			$cnt_chars += length($rawline) + 1;
+		}
+		if ($cnt_chars > $max_msg_size) {
+			WARN("PATCH_TOO_LARGE",
+			     "patch exceeds maximum message size ($max_msg_size) and might be blocked by email list server(s).\n");
+		}
+	}
+
 	if (!$is_patch) {
 		ERROR("NOT_UNIFIED_DIFF",
 		      "Does not appear to be a unified-diff format patch\n");

Joe Perches

maintainer

2014-06-06 01:36 UTC #1 lore raw

Quoted 2 lines · Bruce Allan

the maximum message size (100,000 characters) allowed by Majordomo at vger.kernel.org since that is where most Linux email lists are served.

I believe this is incorrect and the patch is unnecessary.

It'd maybe more interesting if the email header sizes were calculated and if > 1024, then some bleating message were emitted.

Quoted 56 lines

Also provide a command-line option to specify a different maximum size.

Signed-off-by Bruce Allan <[email protected]>

---

 scripts/checkpatch.pl |   15 +++++++++++++++
 1 file changed, 15 insertions(+)

scripts/checkpatch.pl +15 -0

index 34eb216..79bd7fd6 100755
@@ -40,6 +40,7 @@ my @ignore = ();
 my $help = 0;
 my $configuration_file = ".checkpatch.conf";
 my $max_line_length = 80;
+my $max_msg_size = 100000;	# Max allowed by Majordomo at vger.kernel.org
 my $ignore_perl_version = 0;
 my $minimum_perl_version = 5.10.0;

@@ -62,6 +63,7 @@ Options:
   --types TYPE(,TYPE2...)    show only these comma separated message types
   --ignore TYPE(,TYPE2...)   ignore various comma separated message types
   --max-line-length=n        set the maximum line length, if exceeded, warn
+  --max-msg-size=n           set the maximum message size, if exceeded, warn
   --show-types               show the message "types" in the output
   --root=PATH                PATH to the kernel tree root
   --no-summary               suppress the per-file summary

@@ -130,6 +132,7 @@ GetOptions(
 	'types=s'	=> \@use,
 	'show-types!'	=> \$show_types,
 	'max-line-length=i' => \$max_line_length,
+	'max-msg-size=i' => \$max_msg_size,
 	'root=s'	=> \$root,
 	'summary!'	=> \$summary,
 	'mailback!'	=> \$mailback,

@@ -4600,6 +4603,18 @@ sub process {
 		exit(0);
 	}
 
+	# Warn if patch exceeds the maximum message size
+	if (!$file) {
+		my $cnt_chars = 0;
+		foreach my $rawline (@rawlines) {
+			$cnt_chars += length($rawline) + 1;
+		}
+		if ($cnt_chars > $max_msg_size) {
+			WARN("PATCH_TOO_LARGE",
+			     "patch exceeds maximum message size ($max_msg_size) and might be blocked by email list server(s).\n");
+		}
+	}
+
 	if (!$is_patch) {
 		ERROR("NOT_UNIFIED_DIFF",
 		      "Does not appear to be a unified-diff format patch\n");

Allan, Bruce W

author

2014-06-06 15:08 UTC #2 lore raw

Quoted 8 lines

-----Original Message-----
From: Joe Perches [mailto:[email protected]]
Sent: Thursday, June 05, 2014 6:36 PM
To: Allan, Bruce W
Cc: [email protected]; [email protected]; postmaster
Subject: Re: [PATCH] checkpatch: warn when patch exceeds a maximum message size

Quoted 5 lines

Quoted 2 lines · Bruce Allan
the maximum message size (100,000 characters) allowed by Majordomo at vger.kernel.org since that is where most Linux email lists are served.
I believe this is incorrect and the patch is unnecessary.

Actually, it is correct (it even says so at http://vger.kernel.org/majordomo-info.html).
We just experienced 2 out of 4 patches in the initial RFC submittal of the Intel(R) QuickAssist Technology (QAT) driver being blocked by Majordomo from getting out on the linux-crypto mailing list. It took a while to figure out the problem and had to break up those two blocked patches into smaller ones which went through fine. I submitted this checkpatch patch in hopes to prevent others from having to figure out the same.

Quoted 68 lines

It'd maybe more interesting if the email header sizes were calculated and if > 1024, then some bleating message were emitted.
Quoted 13 lines
Also provide a command-line option to specify a different maximum size.
Signed-off-by Bruce Allan <[email protected]>
---
 scripts/checkpatch.pl |   15 +++++++++++++++
 1 file changed, 15 insertions(+)
scripts/checkpatch.pl +0 -0
index 34eb216..79bd7fd6 100755
scripts/checkpatch.pl my @ignore = (); · line 43
Quoted 4 lines
scripts/checkpatch.pl +1 -0
@@ -40,6 +40,7 @@ my @ignore = ();
 my $help = 0;
 my $configuration_file = ".checkpatch.conf";
 my $max_line_length = 80;
+my $max_msg_size = 100000;	# Max allowed by Majordomo at
vger.kernel.org
scripts/checkpatch.pl Options: · line 63
Quoted 5 lines
scripts/checkpatch.pl +0 -0
 my $ignore_perl_version = 0;
 my $minimum_perl_version = 5.10.0;
 
@@ -62,6 +63,7 @@ Options:
   --types TYPE(,TYPE2...)    show only these comma separated message
types
scripts/checkpatch.pl Options: · line 64
Quoted 1 line
scripts/checkpatch.pl +0 -0
   --ignore TYPE(,TYPE2...)   ignore various comma separated message
types
scripts/checkpatch.pl Options: · line 66
Quoted 2 lines
scripts/checkpatch.pl +1 -0
   --max-line-length=n        set the maximum line length, if exceeded, warn
+  --max-msg-size=n           set the maximum message size, if exceeded,
warn
Quoted 14 lines
scripts/checkpatch.pl +1 -0
   --show-types               show the message "types" in the output
   --root=PATH                PATH to the kernel tree root
   --no-summary               suppress the per-file summary
@@ -130,6 +132,7 @@ GetOptions(
 	'types=s'	=> \@use,
 	'show-types!'	=> \$show_types,
 	'max-line-length=i' => \$max_line_length,
+	'max-msg-size=i' => \$max_msg_size,
 	'root=s'	=> \$root,
 	'summary!'	=> \$summary,
 	'mailback!'	=> \$mailback,
@@ -4600,6 +4603,18 @@ sub process {
 		exit(0);
 	}
scripts/checkpatch.pl sub process { · line 4614
Quoted 10 lines
scripts/checkpatch.pl +9 -0
 
+	# Warn if patch exceeds the maximum message size
+	if (!$file) {
+		my $cnt_chars = 0;
+		foreach my $rawline (@rawlines) {
+			$cnt_chars += length($rawline) + 1;
+		}
+		if ($cnt_chars > $max_msg_size) {
+			WARN("PATCH_TOO_LARGE",
+			     "patch exceeds maximum message size
($max_msg_size) and might be blocked by email list server(s).\n");
Quoted 7 lines
scripts/checkpatch.pl +3 -0
+		}
+	}
+
 	if (!$is_patch) {
 		ERROR("NOT_UNIFIED_DIFF",
 		      "Does not appear to be a unified-diff format patch\n");

Joe Perches

maintainer

2014-06-06 15:26 UTC #3 lore raw

Quoted 8 lines · Bruce W

Quoted 8 lines
-----Original Message-----
From: Joe Perches [mailto:[email protected]]
Sent: Thursday, June 05, 2014 6:36 PM
To: Allan, Bruce W
Cc: [email protected]; [email protected]; postmaster
Subject: Re: [PATCH] checkpatch: warn when patch exceeds a maximum message size

Quoted 7 lines

Quoted 5 lines · Bruce Allan
Quoted 2 lines
the maximum message size (100,000 characters) allowed by Majordomo at vger.kernel.org since that is where most Linux email lists are served.
I believe this is incorrect and the patch is unnecessary.
Actually, it is correct (it even says so at http://vger.kernel.org/majordomo-info.html).

where?

All it says there that's relevant is:

------------------------------------------
All email sent to there must be TEXT/PLAIN, there can be no multipart messages, no VCARDs, nothing ``fancy''. In presence of such things, Majordomo will very likely do the wrong thing.

When you send there email, do make sure that all of the email headers, both visible and transport level, have same addresses in them. People experience problems when for example ``From:'', ``Sender:'' and possible ``Reply-To:'' headers present different addresses. The most common manifestation is complete silence from VGER!
------------------------------------------

Quoted 3 lines

We just experienced 2 out of 4 patches in the initial RFC submittal of the Intel(R) QuickAssist Technology (QAT) driver being blocked by Majordomo from getting out on the linux-crypto mailing list.

Possible, but the largest patch I've seen recently on linux-kernel is about .5 Meg, 5 times your limit.

Quoted 4 lines

It took a while to figure out the problem and had to
break up those two blocked patches into smaller ones which went through fine. I submitted this checkpatch patch in hopes to prevent others from having to figure out the same.

Maybe specific to crypto?

Quoted 3 lines

Quoted 3 lines
It'd maybe more interesting if the email header sizes were calculated and if > 1024, then some bleating message were emitted.

This specific bit, email headers > 1K in total size, is an actual failure mode btw.

Joe Perches

maintainer

2014-06-06 15:34 UTC #4 lore raw

Quoted 9 lines · Joe Perches

Quoted 7 lines · Bruce W
From: Joe Perches [mailto:[email protected]]
Sent: Thursday, June 05, 2014 6:36 PM
Quoted 4 lines · Bruce Allan
Quoted 2 lines
the maximum message size (100,000 characters) allowed by Majordomo at vger.kernel.org since that is where most Linux email lists are served.
I believe this is incorrect and the patch is unnecessary.
Actually, it is correct (it even says so at http://vger.kernel.org/majordomo-info.html).
where?

Duh, near the very end it does have:

Message size exceeding 100 000 characters causes blocking.

But I believe that info is out of date.

Allan, Bruce W

author

2014-06-06 15:36 UTC #5 lore raw

Quoted 18 lines

-----Original Message-----
From: Joe Perches [mailto:[email protected]]
Sent: Friday, June 06, 2014 8:34 AM
To: Allan, Bruce W
Cc: [email protected]; [email protected]; postmaster
Subject: Re: [PATCH] checkpatch: warn when patch exceeds a maximum message size
Quoted 5 lines · Joe Perches
Quoted 4 lines · Bruce W
From: Joe Perches [mailto:[email protected]]
Sent: Thursday, June 05, 2014 6:36 PM
Quoted 2 lines · Bruce Allan
Quoted 1 line
the maximum message size (100,000 characters) allowed by
Majordomo at
Quoted 3 lines
Quoted 3 lines
Quoted 2 lines
Quoted 1 line
vger.kernel.org since that is where most Linux email lists are served.
I believe this is incorrect and the patch is unnecessary.
Actually, it is correct (it even says so at

Quoted 9 lines

http://vger.kernel.org/majordomo-info.html).
Quoted 1 line
where?
Duh, near the very end it does have:
Message size exceeding 100 000 characters causes blocking.
But I believe that info is out of date.

If it is out of date, can you explain how/why we got stung by it?

Joe Perches

maintainer

2014-06-06 15:45 UTC #6 lore raw

Quoted 5 lines · Bruce W

Quoted 4 lines
Message size exceeding 100 000 characters causes blocking.
But I believe that info is out of date.
If it is out of date, can you explain how/why we got stung by it?

Dunno.

Too many recipients maybe?

cc'ing more than 20 or so people can run up against that 1024 total byte header limit.

Maybe send me a copy of the original rejected posting?

Allan, Bruce W

author

2014-06-06 15:51 UTC #7 lore raw

Quoted 15 lines

-----Original Message-----
From: Joe Perches [mailto:[email protected]]
Sent: Friday, June 06, 2014 8:45 AM
To: Allan, Bruce W
Cc: [email protected]; [email protected]; postmaster
Subject: Re: [PATCH] checkpatch: warn when patch exceeds a maximum message size
Quoted 5 lines · Bruce W
Quoted 4 lines
Message size exceeding 100 000 characters causes blocking.
But I believe that info is out of date.
If it is out of date, can you explain how/why we got stung by it?

Quoted 8 lines

Dunno.
Too many recipients maybe?
cc'ing more than 20 or so people can run up against that 1024 total byte header limit.
Maybe send me a copy of the original rejected posting?

Will-do, off-list.

Allan, Bruce W

author

2014-06-06 15:35 UTC #8 lore raw

In reply to Joe Perches

Quoted 36 lines

-----Original Message-----
From: Joe Perches [mailto:[email protected]]
Sent: Friday, June 06, 2014 8:27 AM
To: Allan, Bruce W
Cc: [email protected]; [email protected]; postmaster
Subject: Re: [PATCH] checkpatch: warn when patch exceeds a maximum message size
Quoted 10 lines · Bruce W
Quoted 10 lines
-----Original Message-----
From: Joe Perches [mailto:[email protected]]
Sent: Thursday, June 05, 2014 6:36 PM
To: Allan, Bruce W
Cc: [email protected]; [email protected]; postmaster
Subject: Re: [PATCH] checkpatch: warn when patch exceeds a maximum message size
Quoted 1 line · Bruce Allan
the maximum message size (100,000 characters) allowed by
Majordomo at
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
vger.kernel.org since that is where most Linux email lists are served.
I believe this is incorrect and the patch is unnecessary.
Actually, it is correct (it even says so at
http://vger.kernel.org/majordomo-info.html).
where?
All it says there that's relevant is:
------------------------------------------
All email sent to there must be TEXT/PLAIN, there can be no multipart messages, no VCARDs, nothing ``fancy''. In presence of such things, Majordomo will very likely do the wrong thing.

Quoted 7 lines

When you send there email, do make sure that all of the email headers, both visible and transport level, have same addresses in them. People experience problems when for example ``From:'', ``Sender:'' and possible ``Reply-To:'' headers present different addresses. The most common manifestation is complete silence from VGER!
------------------------------------------

http://vger.kernel.org/majordomo-info.html#taboo says:
The Majordomo is configured with a set of filter rules which when triggered will send the email to "/dev/null".
(List owner actually, but they are overworked elsewere, and use "d" button usually...) <snip>
* Message size exceeding 100 000 characters causes blocking.

Quoted 24 lines

Quoted 1 line
We just experienced 2 out of 4 patches in the initial RFC submittal of the
Intel(R)
Quoted 1 line
QuickAssist Technology (QAT) driver being blocked by Majordomo from
getting out
Quoted 1 line
on the linux-crypto mailing list.
Possible, but the largest patch I've seen recently on linux-kernel is about .5 Meg, 5 times your limit.
Quoted 2 lines
It took a while to figure out the problem and had to
break up those two blocked patches into smaller ones which went
through fine. I
Quoted 1 line
submitted this checkpatch patch in hopes to prevent others from having
to figure out
Quoted 1 line
the same.
Maybe specific to crypto?
Quoted 3 lines
Quoted 3 lines
It'd maybe more interesting if the email header sizes were calculated and if > 1024, then some bleating message were emitted.
This specific bit, email headers > 1K in total size, is an actual failure mode btw.