Skip to contentSkip to conversation

disable tags in post-receive hook

PATCH In review Ben Myers 4 participants 6 messages Apr 11, 2012 – Apr 13, 2012

lore
Share & tools
lore

Ben Myers

author
#0 lore raw

PATCH 1/1 disable tags in post-receive hook

Hey Folks,

Yesterday I pushed tags to the oss.sgi.com xfs git repository and forgot to disable the post-receive hook which generates email to the list at commit time. You probably saw the resulting mailbomb. I'm sorry about that. In order to make sure it doesn't happen again next time, I have applied the following patch to our post-receive hook:

post-receive-email.xfs.tags_disabled renamed from post-receive-email.xfs +6 -0
@@ -105,6 +105,9 @@
 			# un-annotated tag
 			refname_type="tag"
 			short_refname=${refname##refs/tags/}
+			# bpm: clear recipients to disable sending email for 
+			# tags.  I just mail bombed the list.  3/11/12
+			recipients=""
 			;;
 		refs/tags/*,tag)
 			# annotated tag
@@ -114,6 +117,9 @@
 			if [ -n "$announcerecipients" ]; then
 				recipients="$announcerecipients"
 			fi
+			# bpm: clear recipients to disable sending email for 
+			# tags.  I just mail bombed the list.  3/11/12
+			recipients=""
 			;;
 		refs/heads/*,commit)
 			# branch

I've tested it out and it seems to have the desired effect. So... just a heads up, you won't be receiving mail for new tags in the xfs git repo on oss.sgi.com.

Regards,
Ben

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

author
#2 lore raw

Hey,

Quoted 1 line · Christoph Hellwig

Looks like a useful change to me.

I don't think anyone will really miss the tag notification. If anyone really feels otherwise they have the opportunity to say so. ;)

Thanks,
Ben

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#3 lore raw
In reply to Ben Myers
Quoted 7 lines · Ben Myers

Hey Folks,

Yesterday I pushed tags to the oss.sgi.com xfs git repository and forgot to disable the post-receive hook which generates email to the list at commit time. You probably saw the resulting mailbomb. I'm sorry about that. In order to make sure it doesn't happen again next time, I have applied the following patch to our post-receive hook:

I curse you for the error Ben and praise you for this correction. Mailman unsubbed me sometime yesterday according to a subject line. There was no body, no reason give. But, I know the cause.

I was limiting concurrent SMTP connections to 1 to fight runaway bots. Was working great until this bombing run. The OSS list server runs Sendmail, which is dumb and opens a new connection for every message delivery. This behavior can potentially bring an MX to its knees due to smtpd process starvation.

Postfix on the other hand will open one connection and keep sending until all mail queued for the destination MX has been sent, or until there is more than a 2 second delay between deliveries.

I understand Brent Bice of SGI is building an 'evidence file' to justify a rebuild of OSS to use Postfix and the latest Mailman. Maybe you could lend your support or blessing.

The current OSS list server has other problems as well, including large delays between receiving posts and sending them back out. I'm assuming I'm not the only one to notice this.

Thanks.

Signature · 6 lines
Stan

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
#4 lore raw
Quoted 13 lines · Stan Hoeppner
Quoted 7 lines · Ben Myers

Hey Folks,

Yesterday I pushed tags to the oss.sgi.com xfs git repository and forgot to disable the post-receive hook which generates email to the list at commit time. You probably saw the resulting mailbomb. I'm sorry about that. In order to make sure it doesn't happen again next time, I have applied the following patch to our post-receive hook:

I curse you for the error Ben and praise you for this correction. Mailman unsubbed me sometime yesterday according to a subject line. There was no body, no reason give. But, I know the cause.

Quoted 5 lines

I was limiting concurrent SMTP connections to 1 to fight runaway bots. Was working great until this bombing run. The OSS list server runs Sendmail, which is dumb and opens a new connection for every message delivery. This behavior can potentially bring an MX to its knees due to smtpd process starvation.

Hmm yes and no.
There was some config issues with with queues on oss that hopefully I have significantly improved. Sendmail does do connection caching and will deliver as much mail as possible on the same connection as possible. The problem was that the queue runnners was set to 400 runners which was essentially was causing oss to grind itself into the ground and causing enough delays that most connection caches where probably timing out.

Also the queue sorting has been changed from the default "priority" sort which basically will be a time sort in this case to "host" sort which will try to optimize envelope address delivery. For a mail list server this should be a significant win since it should be able to better take advantage of the connection cache. (especially when "tag" bombs happen).

The queue run time has been changed from the default of 30m to 1m which should cut the mail list delays down significantly.

Changed the drive queue scheduler from cfq to deadline

I've been watching the headers since the change and the turn around times for mail leaving the originating host to landing in my mail server is about 1 - 2 min. Occasionally there is a delay on sgi's barracuda box but that is whole other box of worms.

Please send me any observations + or - so I know if the tuning is headed in the right direction.

-Russell

Quoted 8 lines

Postfix on the other hand will open one connection and keep sending until all mail queued for the destination MX has been sent, or until there is more than a 2 second delay between deliveries.

I understand Brent Bice of SGI is building an 'evidence file' to justify a rebuild of OSS to use Postfix and the latest Mailman. Maybe you could lend your support or blessing.

Heh another can of worms :-)

Quoted 7 lines

The current OSS list server has other problems as well, including large delays between receiving posts and sending them back out. I'm assuming I'm not the only one to notice this.

Thanks.

begin:vcard
fn:Russell Cattelan
n:Cattelan;Russell
org:Digital Elves Inc
adr:;;2824 Lakeview Ave;Roseville;MN;55113;usa
email;internet:[email protected]
tel;work:612 293 6009
tel;cell:612 805 3144
x-mozilla-html:FALSE
url:http://digitalelves.com
version:2.1
end:vcard

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs

#5 lore raw
Quoted 19 lines · Russell Cattelan
Quoted 13 lines · Stan Hoeppner
Quoted 7 lines · Ben Myers

Hey Folks,

Yesterday I pushed tags to the oss.sgi.com xfs git repository and forgot to disable the post-receive hook which generates email to the list at commit time. You probably saw the resulting mailbomb. I'm sorry about that. In order to make sure it doesn't happen again next time, I have applied the following patch to our post-receive hook:

I curse you for the error Ben and praise you for this correction. Mailman unsubbed me sometime yesterday according to a subject line. There was no body, no reason give. But, I know the cause.

Quoted 5 lines

I was limiting concurrent SMTP connections to 1 to fight runaway bots. Was working great until this bombing run. The OSS list server runs Sendmail, which is dumb and opens a new connection for every message delivery. This behavior can potentially bring an MX to its knees due to smtpd process starvation.

Quoted 8 lines

Hmm yes and no.
There was some config issues with with queues on oss that hopefully I have significantly improved. Sendmail does do connection caching and will deliver as much mail as possible on the same connection as possible. The problem was that the queue runnners was set to 400 runners which was essentially was causing oss to grind itself into the ground and causing enough delays that most connection caches where probably timing out.

I don't use Sendmail so my apologies if I mischaracterized its features/behavior. I was merely describing what I was seeing here. It seems that description may have been helpful in troubleshooting/tuning this, and in the end that's what counts.

Quoted 5 lines

Also the queue sorting has been changed from the default "priority" sort which basically will be a time sort in this case to "host" sort which will try to optimize envelope address delivery. For a mail list server this should be a significant win since it should be able to better take advantage of the connection cache. (especially when "tag" bombs happen).

Awesome.

Quoted 2 lines

The queue run time has been changed from the default of 30m to 1m which should cut the mail list delays down significantly.

Times two.

Quoted 1 line

Changed the drive queue scheduler from cfq to deadline

And again.

Quoted 4 lines

I've been watching the headers since the change and the turn around times for mail leaving the originating host to landing in my mail server is about 1 - 2 min. Occasionally there is a delay on sgi's barracuda box but that is whole other box of worms.

Anything under 5 minutes, regardless of where any delays are occurring, seems acceptable in my book, and much better than the previous setup.

Quoted 2 lines

Please send me any observations + or - so I know if the tuning is headed in the right direction.

I sure will Russel. A short while back when problems hit a threshold here, I searched the whois records for the appropriate contact and went from there, ended up working with Brent, who was eager to help if he could, but made it clear the list had another responsible party/owner. I guess that would be you. :)

Apologies if I've inadvertently stepped on anyone's toes, asked anyone to employ a can opener against his will. I just wanted to bring some long standing issues to attention and hopefully get them resolved. It appears that has happened.

Thanks for addressing these issues Russel. I'm sure others will be pleased by your efforts here as well. Whether the MTA is Sendmail or Postfix is irrelevant as long as it works properly.

Thanks again.

Signature · 6 lines
Stan

_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs