e100 driver hangs after period of moderate receive load
Hi,
After banging on an e100 card for about ten minutes with a ~60kpps stream, the interface stops receiving packets. Interrupts come in once every few seconds (from /proc/interrupts), but no packets are received anymore at all. Lots of slab corruption messages in the syslog that were generated during that packet stream (see other email I sent.) Stopping the packet stream still leaves the interface unusable. 'ifconfig eth1 down ; ifconfig eth1 up' seems to fix things.
Clues?
thanks,
Lennert
Quoted 9 lines · Lennert Buytenhek
Hi,
After banging on an e100 card for about ten minutes with a ~60kpps stream, the interface stops receiving packets. Interrupts come in once every few seconds (from
/proc/interrupts), but no packets are received anymore at all. Lots of slab corruption messages in the syslog that were generated during that packet stream (see other email I sent.) Stopping the packet stream still leaves the interface unusable. 'ifconfig eth1 down ; ifconfig eth1 up' seems to fix things.
Is NAPI enabled for this driver? The interrupt behavior seems normal for NAPI, but certainly the rest of the behavior does not...
Jeff
Quoted 10 lines · Jeff Garzik
Quoted 9 lines
After banging on an e100 card for about ten minutes with a ~60kpps stream, the interface stops receiving packets. Interrupts come in once every few seconds (from
/proc/interrupts), but no packets are received anymore at all.
Lots of slab corruption messages in the syslog that were generated during that packet stream (see other email I sent.) Stopping the packet stream still leaves the interface unusable. 'ifconfig eth1 down ; ifconfig eth1 up'
seems to fix things.
Quoted 2 lines
Is NAPI enabled for this driver? The interrupt behavior seems normal for NAPI, but certainly the rest of the behavior does not...
Yes, NAPI was indeed enabled.
I 'went back' to 2.6.1-rc1 and that seems fine now. Any patches you want me to try on top of 2.6.0-mm2?
--L
Quoted 11 lines · Lennert Buytenhek
Quoted 8 lines · Jeff Garzik
Quoted 7 lines
After banging on an e100 card for about ten minutes with a ~60kpps stream, the interface stops receiving packets. Interrupts come in once every few seconds (from
/proc/interrupts), but no packets are received anymore at all. Lots of slab corruption messages in the syslog that were generated during that packet stream (see other email I sent.) Stopping the packet stream still leaves the interface unusable. 'ifconfig eth1 down ; ifconfig eth1 up' seems to fix things.
Quoted 7 lines
Quoted 1 line
Is NAPI enabled for this driver? The interrupt behavior seems normal for NAPI, but certainly the rest of the behavior does not...
Yes, NAPI was indeed enabled.
I 'went back' to 2.6.1-rc1 and that seems fine now. Any patches you want me to try on top of 2.6.0-mm2?
Well, the two are vastly different, since -mm2 includes a complete rewrite of e100.
Does disabling NAPI in -mm2 change anything?
Jeff
Quoted 20 lines · Jeff Garzik
Quoted 19 lines · Lennert Buytenhek
Quoted 10 lines · Jeff Garzik
Quoted 9 lines
After banging on an e100 card for about ten minutes with a ~60kpps stream, the interface stops receiving packets. Interrupts come in once every few seconds (from
/proc/interrupts), but no packets are received anymore at all.
Lots of slab corruption messages in the syslog that were generated during that packet stream (see other email I sent.) Stopping the packet stream still leaves the interface unusable. 'ifconfig eth1 down ; ifconfig eth1 up'
seems to fix things.Quoted 2 lines
Is NAPI enabled for this driver? The interrupt behavior seems normal for NAPI, but certainly the rest of the behavior does not...
Yes, NAPI was indeed enabled.
Quoted 8 lines
Quoted 2 lines
I 'went back' to 2.6.1-rc1 and that seems fine now. Any patches you want me to try on top of 2.6.0-mm2?
Well, the two are vastly different, since -mm2 includes a complete rewrite of e100.
Does disabling NAPI in -mm2 change anything?
I have one of these beasts so I decided to test it after seeing this message. 2.6.0-mm2 NAPI enabled and I am not seeing any of the symptoms described here. My test consists of downloading an iso from RedHat's ftp site. I'm not sure that is generating the kind of traffic you are talking about, but I'm open to suggestions on what test to run trying to duplicate what is being reported.
Quoted 9 lines · Thomas Molina
[hangs on e100 under moderate receive load]
Quoted 8 lines
Quoted 4 lines
Yes, NAPI was indeed enabled.
I 'went back' to 2.6.1-rc1 and that seems fine now. Any patches you want me to try on top of 2.6.0-mm2?
Well, the two are vastly different, since -mm2 includes a complete rewrite of e100.
Quoted 8 lines
Quoted 1 line
Does disabling NAPI in -mm2 change anything?
I have one of these beasts so I decided to test it after seeing this message. 2.6.0-mm2 NAPI enabled and I am not seeing any of the symptoms described here. My test consists of downloading an iso from RedHat's ftp site. I'm not sure that is generating the kind of traffic you are talking about, but I'm open to suggestions on what test to run trying to duplicate what is being reported.
Make sure you have slab debugging enabled to see if you also get the slab corruption messages, and then hit the machine with anything above 50000 packets per second. pktgen from a different machine on the same subnet works nicely for that. I doubt that downloading a Red Hat iso would give you a load anywhere near that.
Oh, do you have an SMP box? This was on a 2-way (4-way HT) SMP box. Not sure if that matters here.
I'm just about to try 2.6.0-mm2 without NAPI.
--L
Quoted 4 lines · Jeff Garzik
Well, the two are vastly different, since -mm2 includes a complete rewrite of e100.
Does disabling NAPI in -mm2 change anything?
Disabling NAPI makes the slab corruption harder to trigger, but it still occurs.
--L