Packet Corruption with Atheros Communications Inc. AR8121/AR8113/AR8114 Gigabit or Fast Ethernet (rev b0) Interface

Hi @all,

unfortunately I don't have the old postings any more, so I start a new thread.

Following situation:

I have a Server (NFS) and a Client with the Atheros network card. If I transfer big files, the md5sum of these files differ. If I try to scp the file I get "MAC corrupted on inpu".

Everything works fine, if running on any other OS (or with other NICs than the Atheros one).

So here is the data:

lspci:
02:00.0 Ethernet controller: Atheros Communications Inc. AR8121/AR8113/AR8114 Gigabit or Fast Ethernet (rev b0)
Subsystem: ASUSTeK Computer Inc. Device 831c
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 44
Region 0: Memory at fbec0000 (64-bit, non-prefetchable) [size=256K]
Region 2: I/O ports at dc00 [size=128]
Capabilities: [40] Power Management version 2
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot+,D3cold+)
Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [48] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee0f00c Data: 4181
Capabilities: [58] Express (v1) Endpoint, MSI 00
DevCap: MaxPayload 4096 bytes, PhantFunc 0, Latency L0s
<4us, L1 unlimited
ExtTag- AttnBtn+ AttnInd+ PwrInd+ RBE- FLReset-
DevCtl: Report errors: Correctable- Non-Fatal- Fatal-
Unsupported-
RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq+
AuxPwr+ TransPend-

                LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM L0s, 

Latency L0 unlimited, L1 unlimited
ClockPM- Surprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain-
CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+
DLActive- BWMgmt- ABWMgmt-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt-
UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES- TLP- FCP+ CmpltTO- CmpltAbrt-
UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr-
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
NonFatalErr-
AERCap: First Error Pointer: 14, GenCap+ CGenEn-
ChkCap+ ChkEn-
Capabilities: [180 v1] Device Serial Number ff-76-f8-79-00-26-18-ff Kernel driver in use: ATL1E

$ ethtool -k eth0
Offload parameters for eth0:
rx-checksumming: off
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: off

SCP initiated on client:

$ scp server:/export/no_backup/burn/openSUSE-12.2-DVD-x86_64.iso .
openSUSE-12.2-DVD-x86_64.iso 

                                                              1%   88MB 

9.8MB/s 07:26 ETA
Corrupted MAC on input.
Disconnecting: Packet corrupt
lost connection

SCP initiated on server:

$ scp /export/no_backup/burn/openSUSE-12.2-DVD-x86_64.iso client:/tmp
Enter passphrase for key '/home/chewie/tessun/.ssh/id_rsa':
openSUSE-12.2-DVD-x86_64.iso 

                                                              5%  229MB 

7.0MB/s 10:00 ETA
Received disconnect from 2a01:198:366:100::fba5: 2: Packet corrupt lost connection

ifconfig-output (Client):

          RX packets:773481 errors:0 dropped:7620 overruns:0 frame:0
          TX packets:589594 errors:0 dropped:0 overruns:0 carrier:1
          collisions:0 Sendewarteschlangenlänge:1000
          RX bytes:596932088 (569.2 Mb)  TX bytes:90583584 (86.3 Mb)
          Interrupt:44

ifconfig-Output (Server):

          RX packets:11579837 errors:0 dropped:0 overruns:0 frame:0
          TX packets:19734057 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:2717120721 (2591.2 Mb)  TX bytes:22047372376 
(21026.0 Mb)
          Interrupt:42 Base address:0xc000

As said: Other NICs than the Atheros work fine.

If you need additional Infos (tcpdumps, etc.) please let me know.

Regards,
Martin

Is it only happening for IPv6 traffic, or is it also buggy for IPv4 ?

If you disable tso , does the bug disappear ?

Am 21.11.2012 18:14, schrieb Eric Dumazet:

[snip bug description]

It also happens with IPv4 (usually not so fast as with IPv6):

SCP from client:

$ scp -4 server:/export/no_backup/burn/openSUSE-12.2-DVD-x86_64.iso .
openSUSE-12.2-DVD-x86_64.iso 

                 4%  184MB   9.3MB/s   07:37 ETA

Corrupted MAC on input.
Disconnecting: Packet corrupt
openSUSE-12.2-DVD-x86_64.iso

                 4%  193MB   9.2MB/s   07:40 ETA

lost connection

SCP initiated from the server:

$ scp -4 openSUSE-12.2-DVD-x86_64.iso uhura:/tmp
openSUSE-12.2-DVD-x86_64.iso 

                 3%  149MB   8.8MB/s   08:10 ETA

Received disconnect from 10.100.14.100: 2: Packet corrupt lost connection

With TSO disabled:

$ sudo ethtool -K eth0 tso off
$ sudo ethtool -k eth0
Offload parameters for eth0:
rx-checksumming: off
tx-checksumming: on
scatter-gather: on
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: off

SCP (IPv4) initiated from client:

$ scp -4 server:/export/no_backup/burn/openSUSE-12.2-DVD-x86_64.iso .
openSUSE-12.2-DVD-x86_64.iso 

                 2%   93MB  10.4MB/s   06:59 ETA

Corrupted MAC on input.
Disconnecting: Packet corrupt
lost connection

SCP (IPv4) initated from server:

$ scp -4 openSUSE-12.2-DVD-x86_64.iso uhura:/tmp
openSUSE-12.2-DVD-x86_64.iso 

                 4%  179MB   9.2MB/s   07:46 ETA

Received disconnect from 10.100.14.100: 2: Packet corrupt lost connection

Regards,
Martin