Skip to contentSkip to conversation

mv643xx_eth: Disable TSO by default

PATCH New Dr. Uwe Meyer-Gruhl 2 participants 3 messages May 29, 2015

lore
Share & tools
lore
#0 lore raw

cover mv643xx_eth: Disable TSO by default

Quoted 9 lines · Ian Campbell
Quoted 8 lines · Karl Beldan
Quoted 7 lines · Ezequiel Garcia
Quoted 6 lines

Several users ([1], [2]) have been reporting data corruption with TSO on Kirkwood platforms (i.e. using the mv643xx_eth driver).

Until we manage to find what's causing this, this simple patch will make the TSO path disabled by default. This patch should be queued

for stable,

Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Quoted 9 lines

fixing the TSO feature introduced in v3.16.

The corruption itself is very easy to reproduce: checkingmd5sum on a mounted NFS directory gives a different result each time. Same tests using the mvneta driver (Armada 370/38x/XP SoC) pass with no issues.

Frankly, I'm a bit puzzled about this, and so any ideas ordebugging hints are well received.

Hi,

Can you try this :

Quoted 8 lines
Quoted 4 lines

It fixes things for me, thanks!

Good thing, thanks for your feedbak Ian !

Karl --

That would be a good thing - although: Neither the patch to disable TSO altogether nor the one that fixes the underlying problem actually made it to the official kernel source tree, so it is still present in all kernels > 3.16 - I just stumbled over this in the current 4.0.4 version.

The fixes in the thread
http://marc.info/?l=linux-netdev&m=141517941900547&w=2 are not applicable any more to the current driver from the 4.0 kernel, as the whole respective logic seems to have been changed meanwhile, sadly without fixing the problem. Disabling TSO completely still works, though.

Can someone in the know please suggest a working fix to the kernel maintainers, preferably one that does not resort to disable TSO?

Uwe
--

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3740 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20150529/a14a7b4f/attachment.p7s>

#1 lore raw

cover mv643xx_eth: Disable TSO by default

Quoted 42 lines · Dr. Uwe Meyer-Gruhl
Quoted 10 lines · Ian Campbell
Quoted 9 lines · Karl Beldan
Quoted 8 lines · Ezequiel Garcia
Quoted 7 lines

Several users ([1], [2]) have been reporting data corruption with TSO on Kirkwood platforms (i.e. using the mv643xx_eth driver).

Until we manage to find what's causing this, this simple patch will make
the TSO path disabled by default. This patch should be queued

for stable,

Quoted 17 lines
Quoted 17 lines
Quoted 16 lines
Quoted 8 lines

fixing the TSO feature introduced in v3.16.

The corruption itself is very easy to reproduce: checkingmd5sum on a mounted
NFS directory gives a different result each time. Same tests using the mvneta
driver (Armada 370/38x/XP SoC) pass with no issues.

Quoted 4 lines

Frankly, I'm a bit puzzled about this, and so any ideas ordebugging hints
are well received.

Hi,

Can you try this :

Quoted 8 lines
Quoted 4 lines

It fixes things for me, thanks!

Good thing, thanks for your feedbak Ian !

Karl --

That would be a good thing - although: Neither the patch to disable TSO altogether nor the one that fixes the underlying problem actually made it to the official kernel source tree, so it is still present in all kernels > 3.16 - I just stumbled over this in the current 4.0.4 version.

Quoted 9 lines

The fixes in the thread
http://marc.info/?l=linux-netdev&m=141517941900547&w=2 are not applicable any more to the current driver from the 4.0 kernel, as the whole respective logic seems to have been changed meanwhile, sadly without fixing the problem. Disabling TSO completely still works, though.

Can someone in the know please suggest a working fix to the kernel maintainers, preferably one that does not resort to disable TSO?

Hello Uwe,

Thanks for reporting. There wasn't any patch left behind. The above fix was merged (and applied to the stable tree) as this commit:

commit 2c2a9cbd64387d6b70ac5db013e9bfe9412c7354

Author: Karl Beldan <[email protected]>
Date:   Wed Nov 5 15:32:59 2014 +0100

net: mv643xx_eth: reclaim TX skbs only when released by the HW

Can you explain in detail what sort of problem are you running into, which kernel version are you using, etc.?

Signature · 2 lines
Ezequiel Garcia, VanguardiaSur
www.vanguardiasur.com.ar
#2 lore raw

cover mv643xx_eth: Disable TSO by default

Am 29.05.2015 um 15:07 schrieb Ezequiel Garcia:

Quoted 53 lines · Dr. Uwe Meyer-Gruhl
Quoted 38 lines
Quoted 10 lines · Ian Campbell
Quoted 9 lines · Karl Beldan
Quoted 8 lines · Ezequiel Garcia
Quoted 7 lines

Several users ([1], [2]) have been reporting data corruption with TSO on Kirkwood platforms (i.e. using the mv643xx_eth driver).

Until we manage to find what's causing this, this simple patch will make
the TSO path disabled by default. This patch should be queued

for stable,

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 8 lines

fixing the TSO feature introduced in v3.16.

The corruption itself is very easy to reproduce: checkingmd5sum on a mounted
NFS directory gives a different result each time. Same tests using the mvneta
driver (Armada 370/38x/XP SoC) pass with no issues.

Quoted 4 lines

Frankly, I'm a bit puzzled about this, and so any ideas ordebugging hints
are well received.

Hi,

Quoted 8 lines
Quoted 5 lines
Quoted 1 line

Can you try this :

It fixes things for me, thanks!

Good thing, thanks for your feedbak Ian !

Karl --

That would be a good thing - although: Neither the patch to disable TSO altogether nor the one that fixes the underlying problem actually made it to the official kernel source tree, so it is still present in all kernels > 3.16 - I just stumbled over this in the current 4.0.4 version.

Quoted 9 lines

The fixes in the thread
http://marc.info/?l=linux-netdev&m=141517941900547&w=2 are not applicable any more to the current driver from the 4.0 kernel, as the whole respective logic seems to have been changed meanwhile, sadly without fixing the problem. Disabling TSO completely still works, though.

Can someone in the know please suggest a working fix to the kernel maintainers, preferably one that does not resort to disable TSO?

Hello Uwe,

Thanks for reporting. There wasn't any patch left behind. The above fix was merged (and applied to the stable tree) as this commit:

Quoted 8 lines

commit 2c2a9cbd64387d6b70ac5db013e9bfe9412c7354

Author: Karl Beldan <[email protected]>
Date:   Wed Nov 5 15:32:59 2014 +0100

net: mv643xx_eth: reclaim TX skbs only when released by the HW

Can you explain in detail what sort of problem are you running into, which kernel version are you using, etc.?

Sure, the same kind as had been reported initially.

Background: I have done a Debian port running on an Iomega iConnect (a Kirkwood variant).

One of my users reported corrupted data when he used the machine as a file and media server over Samba and DLNA. He ruled out any hardware or other faults and pointed to my kernel(s). He actually tried versions 3.18.1 and 4.0.4. However, my kernels are mostly plain vanilla from kernel.org without any relevant patches to the code itself. The user also found that a kernel 3.14 ran O.K. in the same context.

When I tried, I could just use md5sum on the /usr subtree repeatedly without problems locally, but over NFS with the iConnect as a server, I had differences on every try in different files each time.

After ruling out network issues as well, it was clear that the NIC driver must be the culprit and after a bit of searching which one it actually was, I quickly found the messages about the broken TSO feature in mv643xx_eth. Then, I tried the suggested fix of "ethtool -K eth0 tso off" and voila - the errors are gone. For now, I use a patch to change the default to TSO off, but an upstream patch to handle TSO correctly would be much better.

Uwe

Signature · 7 lines
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3740 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.infradead.org/pipermail/linux-arm-kernel/attachments/20150529/12a23844/attachment.p7s>