[2.4.21] 8139too 'too much work at interrupt'...
eth0 stopped receiving and sending packets. Please CC me for more information.
---
# dmesg [snip] 8139too Fast Ethernet driver 0.9.26 eth0: SMC1211TX EZCard 10/100 (RealTek RTL8139) at 0xfffffd0009442000, 00:e0:29:1c:7c:50, IRQ 23 eth0: Setting 100mbps full-duplex based on auto-negotiated partner ability 45e1. [snip, some time later] eth0: Too much work at interrupt, IntrStatus=0x0010. [eth0 not rx/txing packets]
# ethtool -i eth0 driver: 8139too version: 0.9.26 firmware-version: bus-info: 00:09.0
# ethtool eth0 Settings for eth0:
Supported ports: [ TP MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Supports auto-negotiation? Yes
Speed: 100Mb/s
Duplex: Full
Port: MII
PHYAD: 32
Transceiver: internal
Auto-negotiation: on
# lspci -s 00:09.0 -vxxx
00:09.0 Ethernet controller: Accton Technology Corporation SMC2-1211TX (rev
10)
Subsystem: Accton Technology Corporation EN-1207D Fast Ethernet
Adapter
Flags: bus master, medium devsel, latency 32, IRQ 23
I/O ports at 8400 [size=128]
Memory at 0000000009442000 (32-bit, non-prefetchable) [size=128]
00: 13 11 11 12 07 00 00 02 10 00 00 02 00 20 00 00
10: 01 84 00 00 00 20 44 09 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 13 11 11 12
30: 00 00 00 00 00 00 00 00 00 00 00 00 17 01 20 40
40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00# cat /proc/interrupts
CPU0 CPU1
1: 2 0 XT-PIC keyboard
2: 0 0 XT-PIC cascade
4: 4102 0 XT-PIC serial
8: 65836187 65835236 RTC +timer
14: 5 0 XT-PIC +ide0
19: 0 32045 DP264 aic7xxx
23: 0 2225944 DP264 eth0
27: 363080 0 DP264 elan
IPI: 3238154 3303111
ERR: 0# cat /proc/cpuinfo cpu : Alpha cpu model : EV67 cpu variation : 7 cpu revision : 0 cpu serial number : system type : Tsunami system variation : DP264 system revision : 0 system serial number : 007 cycle frequency [Hz] : 666666666 timer frequency [Hz] : 1024.00 page size [bytes] : 8192 phys. address bits : 44 max. addr. space # : 255 BogoMIPS : 1330.04 kernel unaligned acc : 0 (pc=0,va=0) user unaligned acc : 0 (pc=0,va=0) platform string : UP2000 666 MHz cpus detected : 2 cpus active : 2 cpu active mask : 0000000000000003
[restarted of eth0 interface here]
# ./rtl8139-diag -mmaaavvveef rtl8139-diag.c:v2.01 1/8/2001 Donald Becker ([email protected]) http://www.scyld.com/diag/index.html Index #1: Found a SMC1211TX EZCard 10/100 (RealTek RTL8139) adapter at 0x8400. RealTek chip registers at 0x8400 0x000: 1c29e000 0000507c 80000000 00000000 0008a0ba 0008a0b6 0008a0ba 0008a0ba 0x020: 94c20000 94c20600 94c20c00 94c21200 9e8d0000 0d0a0000 b308b2f8 0000807f 0x040: 60000600 0000f78e c4367d6d 00000000 000c10c6 00000000 00000080 00100000 0x060: 1100f00f 05e1782d 000145e1 00000000 00000024 000007c8 00000000 00000000. No interrupt sources are pending. The chip configuration is 0x10 0x0c, MII half-duplex mode. EEPROM size test returned 6, 0x204a7 / 0x29697. Parsing the EEPROM of a RealTek chip: PCI IDs -- Vendor 0x1113, Device 0x1211, Subsystem 0x1113. PCI timer settings -- minimum grant 32, maximum latency 64. General purpose pins -- direction 0xf3 value 0x10. Station Address 00:E0:29:1C:7C:50. Configuration register 0/1 -- 0x2c / 0x00. EEPROM active region checksum is 04c8. EEPROM contents: 8129 1113 1211 1113 1211 4020 f310 e000 1c29 507c 2c10 0000 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 The word-wide EEPROM checksum is 0x18da.
# ./rtl8139-diag-parse.pl /tmp/diag.txt ID registers: 00:E0:29:1C:7C:50 Reserved (06) = 0x00 Reserved (07) = 0x00 MAR0-7: 00:00:00:80:00:00:00:00 TSD0: SIZE=186 ERTXTH=256 NCC=0 OWN TOK TSD1: SIZE=82 ERTXTH=256 NCC=0 OWN TOK TSD2: SIZE=134 ERTXTH=256 NCC=0 OWN TOK TSD3: SIZE=186 ERTXTH=256 NCC=0 OWN TOK TSAD0 = 0x94C20000 TSAD1 = 0x94C20600 TSAD2 = 0x94C20C00 TSAD3 = 0x94C21200 RBSTART = 0x9E8D0000 ERBCR = 0x0D0A ERSR: ERGood EROVW CR: RE TE BUFE CAPR = 20720 CBR = 20736 IMR1: SERR FOVW PUN/LinkChg RXOVW TER TOK RER ROK ISR1: 0 TCR: CHIP=RTL-8139 TXRR=0 MAXDMA=1024 LOOP=normal RCR: MAXDMA=unlimited RBLEN=32K+16 RXFTH=none ERTH=none RCR (cont): WRAP AB AM APM TCTR = 3226309911 MPC = 0x00000000 9346CR: Config register write enable Config0: PL1 Config1: MEMMAP IOMAP Reserved (53) = 0x00 TimerInt = 0x00000000 MSR: TXFCE Config3: 0 Config4: 0 Reserved (5B) = 0x00 MISR: 0 RERID = 0x10 Reserved (5F) = 0x00 TSAD: TOK3 TOK2 TOK1 TOK0 OWN3 OWN2 OWN1 OWN0 BMCR: ANE FDX BMSR: 100baseT-HD 100baseT-HD 10baseT-FD 10baseT-HD ANC AutoNeg LinkStatus ExtCap ANAR: Pause TXFD TX 10FD 10 Sel0 ANLPAR: ACK Pause TXFD TX 10FD 10 Sel0 ANER: LP_NW_ABLE DIS = 0 FCSC = 0 NWAYTR: BIT5 FLAGABD Rx Err Count = 0 CSCR: BIT10 LD HEART_BEAT JBEN F_LINK_100 Reserved(76) = 0x0000 PHY1_PARM = 0x00000000 TW_PARM = 0x00000000
Signature · 8 lines
Daniel J Blueman NEU FÜR ALLE - GMX MediaCenter - für Fotos, Musik, Dateien... Fotoalbum, File Sharing, MMS, Multimedia-Gruß, GMX FotoService Jetzt kostenlos anmelden unter http://www.gmx.net +++ GMX - die erste Adresse für Mail, Message, More! +++
This happens to me also on 2.4.18 and 2.4.19 (yes, I know they are old).
Happens about once every 5 months, with the box running at about 1 month uptime per reboot (home server, there is no UPS)
Quoted 209 lines · Daniel Blueman
eth0 stopped receiving and sending packets. Please CC me for more information.
---
# dmesg [snip] 8139too Fast Ethernet driver 0.9.26 eth0: SMC1211TX EZCard 10/100 (RealTek RTL8139) at 0xfffffd0009442000, 00:e0:29:1c:7c:50, IRQ 23 eth0: Setting 100mbps full-duplex based on auto-negotiated partner ability 45e1. [snip, some time later] eth0: Too much work at interrupt, IntrStatus=0x0010. [eth0 not rx/txing packets]# ethtool -i eth0 driver: 8139too version: 0.9.26 firmware-version: bus-info: 00:09.0# ethtool eth0 Settings for eth0:Supported ports: [ TP MII ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/FullSupports auto-negotiation? Yes
Speed: 100Mb/s
Duplex: Full
Port: MII
PHYAD: 32
Transceiver: internal
Auto-negotiation: on# lspci -s 00:09.0 -vxxx 00:09.0 Ethernet controller: Accton Technology Corporation SMC2-1211TX (rev 10) Subsystem: Accton Technology Corporation EN-1207D Fast Ethernet Adapter Flags: bus master, medium devsel, latency 32, IRQ 23 I/O ports at 8400 [size=128] Memory at 0000000009442000 (32-bit, non-prefetchable) [size=128] 00: 13 11 11 12 07 00 00 02 10 00 00 02 00 20 00 00 10: 01 84 00 00 00 20 44 09 00 00 00 00 00 00 00 00 20: 00 00 00 00 00 00 00 00 00 00 00 00 13 11 11 12 30: 00 00 00 00 00 00 00 00 00 00 00 00 17 01 20 40 40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 60: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 70: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 90: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00# cat /proc/interrupts CPU0 CPU1 1: 2 0 XT-PIC keyboard 2: 0 0 XT-PIC cascade 4: 4102 0 XT-PIC serial 8: 65836187 65835236 RTC +timer 14: 5 0 XT-PIC +ide0 19: 0 32045 DP264 aic7xxx 23: 0 2225944 DP264 eth0 27: 363080 0 DP264 elan IPI: 3238154 3303111 ERR: 0# cat /proc/cpuinfo cpu : Alpha cpu model : EV67 cpu variation : 7 cpu revision : 0 cpu serial number : system type : Tsunami system variation : DP264 system revision : 0 system serial number : 007 cycle frequency [Hz] : 666666666 timer frequency [Hz] : 1024.00 page size [bytes] : 8192 phys. address bits : 44 max. addr. space # : 255 BogoMIPS : 1330.04 kernel unaligned acc : 0 (pc=0,va=0) user unaligned acc : 0 (pc=0,va=0) platform string : UP2000 666 MHz cpus detected : 2 cpus active : 2 cpu active mask : 0000000000000003[restarted of eth0 interface here]
# ./rtl8139-diag -mmaaavvveef rtl8139-diag.c:v2.01 1/8/2001 Donald Becker ([email protected]) http://www.scyld.com/diag/index.html Index #1: Found a SMC1211TX EZCard 10/100 (RealTek RTL8139) adapter at 0x8400. RealTek chip registers at 0x8400 0x000: 1c29e000 0000507c 80000000 00000000 0008a0ba 0008a0b6 0008a0ba 0008a0ba 0x020: 94c20000 94c20600 94c20c00 94c21200 9e8d0000 0d0a0000 b308b2f8 0000807f 0x040: 60000600 0000f78e c4367d6d 00000000 000c10c6 00000000 00000080 00100000 0x060: 1100f00f 05e1782d 000145e1 00000000 00000024 000007c8 00000000 00000000. No interrupt sources are pending. The chip configuration is 0x10 0x0c, MII half-duplex mode. EEPROM size test returned 6, 0x204a7 / 0x29697. Parsing the EEPROM of a RealTek chip: PCI IDs -- Vendor 0x1113, Device 0x1211, Subsystem 0x1113. PCI timer settings -- minimum grant 32, maximum latency 64. General purpose pins -- direction 0xf3 value 0x10. Station Address 00:E0:29:1C:7C:50. Configuration register 0/1 -- 0x2c / 0x00. EEPROM active region checksum is 04c8. EEPROM contents: 8129 1113 1211 1113 1211 4020 f310 e000 1c29 507c 2c10 0000 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 a5a5 The word-wide EEPROM checksum is 0x18da.# ./rtl8139-diag-parse.pl /tmp/diag.txt ID registers: 00:E0:29:1C:7C:50 Reserved (06) = 0x00 Reserved (07) = 0x00 MAR0-7: 00:00:00:80:00:00:00:00 TSD0: SIZE=186 ERTXTH=256 NCC=0 OWN TOK TSD1: SIZE=82 ERTXTH=256 NCC=0 OWN TOK TSD2: SIZE=134 ERTXTH=256 NCC=0 OWN TOK TSD3: SIZE=186 ERTXTH=256 NCC=0 OWN TOK TSAD0 = 0x94C20000 TSAD1 = 0x94C20600 TSAD2 = 0x94C20C00 TSAD3 = 0x94C21200 RBSTART = 0x9E8D0000 ERBCR = 0x0D0A ERSR: ERGood EROVW CR: RE TE BUFE CAPR = 20720 CBR = 20736 IMR1: SERR FOVW PUN/LinkChg RXOVW TER TOK RER ROK ISR1: 0 TCR: CHIP=RTL-8139 TXRR=0 MAXDMA=1024 LOOP=normal RCR: MAXDMA=unlimited RBLEN=32K+16 RXFTH=none ERTH=none RCR (cont): WRAP AB AM APM TCTR = 3226309911 MPC = 0x00000000 9346CR: Config register write enable Config0: PL1 Config1: MEMMAP IOMAP Reserved (53) = 0x00 TimerInt = 0x00000000 MSR: TXFCE Config3: 0 Config4: 0 Reserved (5B) = 0x00 MISR: 0 RERID = 0x10 Reserved (5F) = 0x00 TSAD: TOK3 TOK2 TOK1 TOK0 OWN3 OWN2 OWN1 OWN0 BMCR: ANE FDX BMSR: 100baseT-HD 100baseT-HD 10baseT-FD 10baseT-HD ANC AutoNeg LinkStatus ExtCap ANAR: Pause TXFD TX 10FD 10 Sel0 ANLPAR: ACK Pause TXFD TX 10FD 10 Sel0 ANER: LP_NW_ABLE DIS = 0 FCSC = 0 NWAYTR: BIT5 FLAGABD Rx Err Count = 0 CSCR: BIT10 LD HEART_BEAT JBEN F_LINK_100 Reserved(76) = 0x0000 PHY1_PARM = 0x00000000 TW_PARM = 0x00000000Signature · 14 lines
Daniel J Blueman NEU FÜR ALLE - GMX MediaCenter - für Fotos, Musik, Dateien... Fotoalbum, File Sharing, MMS, Multimedia-Gruß, GMX FotoService Jetzt kostenlos anmelden unter http://www.gmx.net +++ GMX - die erste Adresse für Mail, Message, More! +++ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Signature · 6 lines
winden/network 1. Dado un programa, siempre tiene al menos un fallo. 2. Dadas varias lineas de codigo, siempre se pueden acortar a menos lineas. 3. Por induccion, todos los programas se pueden reducir a una linea que no funciona.
Quoted 4 lines · Antonio Vargas
This happens to me also on 2.4.18 and 2.4.19 (yes, I know they are old).
Happens about once every 5 months, with the box running at about 1 month uptime per reboot (home server, there is no UPS)
It's fairly normal for this event to occur. It's due to the 8139 hardware.. sometimes (perhaps during a DoS or ping flood) you can receive far more tiny packets than the driver wishes to deal with in a single interrupt.
The real solution is to convert the driver to NAPI...
Jeff
Quoted 7 lines · Jeff Garzik
Quoted 4 lines · Antonio Vargas
This happens to me also on 2.4.18 and 2.4.19 (yes, I know they are old).
Happens about once every 5 months, with the box running at about 1 month uptime per reboot (home server, there is no UPS)
Quoted 8 lines
It's fairly normal for this event to occur. It's due to the 8139 hardware.. sometimes (perhaps during a DoS or ping flood) you can receive far more tiny packets than the driver wishes to deal with in a single interrupt.
The real solution is to convert the driver to NAPI...
Jeff
NAPI is the method where you block hardware interrupts and then handle the data by periodic polling? I wonder how could I get this error, given that my network is a 10Mbit one ;)
Signature · 6 lines
winden/network 1. Dado un programa, siempre tiene al menos un fallo. 2. Dadas varias lineas de codigo, siempre se pueden acortar a menos lineas. 3. Por induccion, todos los programas se pueden reducir a una linea que no funciona.
I have seen problems like this when a bad driver was spending loads of time in it's SA_INTERRUPT (ie meant to be 'fast') IRQ handler ...this buffered up *lots * of packets to be handled, and caused this message.
Perhaps we should profile?
Dan
Quoted 31 lines · Jeff Garzik
Quoted 2 lines
Quoted 1 line · Antonio Vargas
This happens to me also on 2.4.18 and 2.4.19 (yes, I know they are
old).
Quoted 5 lines
Quoted 3 lines
Happens about once every 5 months, with the box running at about 1 month uptime per reboot (home server, there is no UPS)
Quoted 8 lines
It's fairly normal for this event to occur. It's due to the 8139 hardware.. sometimes (perhaps during a DoS or ping flood) you can receive far more tiny packets than the driver wishes to deal with in a single interrupt.
The real solution is to convert the driver to NAPI...
Jeff
NAPI is the method where you block hardware interrupts and then handle the data by periodic polling? I wonder how could I get this error, given that my network is a 10Mbit one ;)
Signature · 7 lines
winden/network 1. Dado un programa, siempre tiene al menos un fallo. 2. Dadas varias lineas de codigo, siempre se pueden acortar a menos lineas. 3. Por induccion, todos los programas se pueden reducir a una linea que no funciona.
Signature · 8 lines
Daniel J Blueman NEU FÜR ALLE - GMX MediaCenter - für Fotos, Musik, Dateien... Fotoalbum, File Sharing, MMS, Multimedia-Gruß, GMX FotoService Jetzt kostenlos anmelden unter http://www.gmx.net +++ GMX - die erste Adresse für Mail, Message, More! +++
Quoted 5 lines · Daniel Blueman
I have seen problems like this when a bad driver was spending loads of time in it's
SA_INTERRUPT(ie meant to be 'fast') IRQ handler ...this buffered up *lots * of packets to be handled, and caused this message.Perhaps we should profile?
There is no need to profile, I described the problem precisely.
Jeff
Hi Jeff,
Yes, the 8139too driver is doing it's job correctly - I was just thinking about the source of the starvation, ie the cause of the effect. I.e. the brokeness is elsewhere, in other drivers etc.
Dan
Quoted 16 lines
Quoted 1 line · Daniel Blueman
I have seen problems like this when a bad driver was spending loads of
time
Quoted 1 line
in it's
SA_INTERRUPT(ie meant to be 'fast') IRQ handler ...thisbuffered up
Quoted 3 lines
*lots * of packets to be handled, and caused this message.
Perhaps we should profile?
There is no need to profile, I described the problem precisely.
Jeff
Signature · 8 lines
Daniel J Blueman NEU FÜR ALLE - GMX MediaCenter - für Fotos, Musik, Dateien... Fotoalbum, File Sharing, MMS, Multimedia-Gruß, GMX FotoService Jetzt kostenlos anmelden unter http://www.gmx.net +++ GMX - die erste Adresse für Mail, Message, More! +++
Quoted 1 line · Daniel Blueman
Yes, the 8139too driver is doing it's job correctly - I was just thinking
I didn't say this, because it is obviously not doing it job correctly.
It needs NAPI to do this. For now, the driver _should_ stay up, even with these 'too much work...' messages.
Quoted 2 lines
about the source of the starvation, ie the cause of the effect. I.e. the brokeness is elsewhere, in other drivers etc.
Quoting from my original reply:
It's due to the 8139 hardware
Tons of tiny packets can be sent before RTL8139 will throttle, which can easily be far more than your system can handle at that particular time, especially if it's experiencing high load elsewhere.
You need either NAPI (perfect solution) or "cap interrupt work limit at X packets" (stopgap solution) to prevent your 8139 hardware from being DoS'd off the network.
Jeff