Skip to contentSkip to conversation

sky2 doesn't receive any packages after I ssh in via ipv6 and edit a file in vim (reproducable)

PATCH In review Thomas Glanzmann 5 participants 11 messages Aug 21, 2006 – Aug 26, 2006

Subsystems (3)
Networking Network drivers TCP
lore
Share & tools
lore

Thomas Glanzmann

author
#0 lore raw

Hello,
I have the newest sky2 driver (with the newest fix in version 1.5) running on my intel mac mini running under legacy bios modus. However if I ssh in via IPv6 and start vim in the shell to edit a file the sky2 network adapter stops to work. This is reproducable. A friend of mine thought that maybe the IPv6 code doesn't request the right checksum. But I have no clue if it is related to this. I can give access to the machine via ssh (including root) if someone is willing to track this down. I have a userland programming running which reboots the machine if the network connectivity is lost. At least this one is reliable. If noone wants a shell account maybe someone can guide me through debugging and tracking this down.

(mini) [/var/log] grep sky2 kern.log

Aug 20 20:52:10 mini kernel: sky2 v1.5 addr 0x90200000 irq 17 Yukon-EC (0xb6) rev 2
Aug 20 20:52:10 mini kernel: sky2 eth0: addr 00:16:cb:a3:91:3c
Aug 20 20:52:12 mini kernel: sky2 eth0: enabling interface
Aug 20 20:52:12 mini kernel: sky2 eth0: Link is up at 100 Mbps, full duplex, flow control both
Aug 20 21:01:34 mini kernel: sky2 eth0: hw error interrupt status 0x30
Aug 20 21:01:34 mini kernel: sky2 eth0: ram data read parity error
Aug 20 21:01:34 mini kernel: sky2 eth0: ram data write parity error
Aug 20 21:01:34 mini kernel: sky2 eth0: hw error interrupt status 0x8
Aug 20 21:01:34 mini kernel: sky2 eth0: MAC parity error
Aug 20 21:01:34 mini kernel: sky2 eth0: hw error interrupt status 0x20
Aug 20 21:01:34 mini kernel: sky2 eth0: ram data read parity error
Aug 20 21:01:34 mini kernel: sky2 eth0: hw error interrupt status 0x8
Aug 20 21:01:34 mini kernel: sky2 eth0: MAC parity error
Aug 21 11:23:49 mini kernel: sky2 v1.5 addr 0x90200000 irq 17 Yukon-EC (0xb6) rev 2
Aug 21 11:23:49 mini kernel: sky2 eth0: addr 00:16:cb:a3:91:3c
Aug 21 11:23:50 mini kernel: sky2 eth0: enabling interface
Aug 21 11:23:50 mini kernel: sky2 eth0: Link is up at 100 Mbps, full duplex, flow control both
Aug 21 18:38:44 mini kernel: sky2 eth0: hw error interrupt status 0x30
Aug 21 18:38:44 mini kernel: sky2 eth0: ram data read parity error
Aug 21 18:38:44 mini kernel: sky2 eth0: ram data write parity error
Aug 21 18:38:44 mini kernel: sky2 eth0: hw error interrupt status 0x8
Aug 21 18:38:44 mini kernel: sky2 eth0: MAC parity error
Aug 21 18:38:44 mini kernel: sky2 eth0: hw error interrupt status 0x20
Aug 21 18:38:44 mini kernel: sky2 eth0: ram data read parity error
Aug 21 18:38:44 mini kernel: sky2 eth0: hw error interrupt status 0x8
Aug 21 18:38:44 mini kernel: sky2 eth0: MAC parity error
Aug 21 18:48:31 mini kernel: sky2 v1.5 addr 0x90200000 irq 17 Yukon-EC (0xb6) rev 2
Aug 21 18:48:31 mini kernel: sky2 eth0: addr 00:16:cb:a3:91:3c
Aug 21 18:48:32 mini kernel: sky2 eth0: enabling interface
Aug 21 18:48:32 mini kernel: sky2 eth0: Link is up at 100 Mbps, full duplex, flow control both
Aug 21 20:53:56 mini kernel: sky2 eth0: tx timeout
Aug 21 20:53:56 mini kernel: sky2 eth0: transmit ring 53 .. 30 report=54 done=54
Aug 21 20:53:56 mini kernel: sky2 status report lost?
Aug 21 20:54:01 mini kernel: sky2 eth0: tx timeout
Aug 21 20:54:01 mini kernel: sky2 eth0: transmit ring 53 .. 30 report=54 done=54
Aug 21 20:54:01 mini kernel: sky2 status report lost?
Aug 21 20:54:06 mini kernel: sky2 eth0: tx timeout
Aug 21 20:54:06 mini kernel: sky2 eth0: transmit ring 53 .. 30 report=54 done=54
Aug 21 20:54:06 mini kernel: sky2 status report lost?
Aug 21 20:54:11 mini kernel: sky2 eth0: tx timeout
Aug 21 20:54:11 mini kernel: sky2 eth0: transmit ring 53 .. 30 report=54 done=54
Aug 21 20:54:11 mini kernel: sky2 status report lost?
Aug 21 20:54:16 mini kernel: sky2 eth0: tx timeout
Aug 21 20:54:16 mini kernel: sky2 eth0: transmit ring 53 .. 30 report=54 done=54
Aug 21 20:54:16 mini kernel: sky2 status report lost?
Aug 21 20:54:21 mini kernel: sky2 eth0: tx timeout
Aug 21 20:54:21 mini kernel: sky2 eth0: transmit ring 53 .. 30 report=54 done=54
Aug 21 20:54:21 mini kernel: sky2 status report lost?
Aug 21 20:54:26 mini kernel: sky2 eth0: tx timeout
Aug 21 20:54:26 mini kernel: sky2 eth0: transmit ring 53 .. 30 report=54 done=54
Aug 21 20:54:26 mini kernel: sky2 status report lost?
Aug 21 20:54:31 mini kernel: sky2 eth0: tx timeout
Aug 21 20:54:31 mini kernel: sky2 eth0: transmit ring 53 .. 30 report=54 done=54
Aug 21 20:54:31 mini kernel: sky2 status report lost?
Aug 21 20:54:36 mini kernel: sky2 eth0: tx timeout
Aug 21 20:54:36 mini kernel: sky2 eth0: transmit ring 53 .. 30 report=54 done=54
Aug 21 20:54:36 mini kernel: sky2 status report lost?
Aug 21 20:54:41 mini kernel: sky2 eth0: tx timeout
Aug 21 20:54:41 mini kernel: sky2 eth0: transmit ring 53 .. 30 report=54 done=54
Aug 21 20:54:41 mini kernel: sky2 status report lost?
Aug 21 20:54:46 mini kernel: sky2 eth0: tx timeout
Aug 21 20:54:46 mini kernel: sky2 eth0: transmit ring 53 .. 30 report=54 done=54
Aug 21 20:54:46 mini kernel: sky2 status report lost?
Aug 21 20:54:51 mini kernel: sky2 eth0: tx timeout
Aug 21 20:54:51 mini kernel: sky2 eth0: transmit ring 53 .. 30 report=54 done=54
Aug 21 20:54:51 mini kernel: sky2 status report lost?
Aug 21 20:56:17 mini kernel: sky2 v1.5 addr 0x90200000 irq 17 Yukon-EC (0xb6) rev 2
Aug 21 20:56:17 mini kernel: sky2 eth0: addr 00:16:cb:a3:91:3c
Aug 21 20:56:18 mini kernel: sky2 eth0: enabling interface
Aug 21 20:56:18 mini kernel: sky2 eth0: Link is up at 100 Mbps, full duplex, flow control both

This log messgaes are all related to the same problem. At the moment my userland watchdog reboots the machine after 500 seconds:

(mini) [/var/log] grep watchdog-tg daemon.log

Aug 20 20:42:32 mini watchdog-tg[1763]: No PONG received from 192.168.0.3 (failure 1 of 50)
...
Aug 20 20:50:48 mini watchdog-tg[1763]: No PONG received from 192.168.0.3 (failure 50 of 50)
Aug 20 20:50:48 mini watchdog-tg[1763]: Telling init to reboot and no longer feed watchdog
Aug 21 18:38:49 mini watchdog-tg[1761]: No PONG received from 192.168.0.3 (failure 1 of 50)
...
Aug 21 18:47:07 mini watchdog-tg[1761]: No PONG received from 192.168.0.3 (failure 50 of 50)
Aug 21 18:47:07 mini watchdog-tg[1761]: Telling init to reboot and no longer feed watchdog
Aug 21 20:46:31 mini watchdog-tg[1755]: No PONG received from 192.168.0.3 (failure 1 of 50)
...
Aug 21 20:54:49 mini watchdog-tg[1755]: No PONG received from 192.168.0.3 (failure 50 of 50)
Aug 21 20:54:49 mini watchdog-tg[1755]: Telling init to reboot and no longer feed watchdog

Thomas

#1 lore raw
Quoted 14 lines · On Mon, 21 Aug 2006 23:22:43 +0200 Thomas Glanzmann…

Hello,
I have the newest sky2 driver (with the newest fix in version 1.5) running on my intel mac mini running under legacy bios modus. However if I ssh in via IPv6 and start vim in the shell to edit a file the sky2 network adapter stops to work. This is reproducable. A friend of mine thought that maybe the IPv6 code doesn't request the right checksum. But I have no clue if it is related to this. I can give access to the machine via ssh (including root) if someone is willing to track this down. I have a userland programming running which reboots the machine if the network connectivity is lost. At least this one is reliable. If noone wants a shell account maybe someone can guide me through debugging and tracking this down.

(mini) [/var/log] grep sky2 kern.log

Quoted 9 lines
Aug 20 20:52:10 mini kernel: sky2 v1.5 addr 0x90200000 irq 17 Yukon-EC (0xb6) rev 2
Aug 20 20:52:10 mini kernel: sky2 eth0: addr 00:16:cb:a3:91:3c
Aug 20 20:52:12 mini kernel: sky2 eth0: enabling interface
Aug 20 20:52:12 mini kernel: sky2 eth0: Link is up at 100 Mbps, full duplex, flow control both
Aug 20 21:01:34 mini kernel: sky2 eth0: hw error interrupt status 0x30
Aug 20 21:01:34 mini kernel: sky2 eth0: ram data read parity error
Aug 20 21:01:34 mini kernel: sky2 eth0: ram data write parity error
Aug 20 21:01:34 mini kernel: sky2 eth0: hw error interrupt status 0x8
Aug 20 21:01:34 mini kernel: sky2 eth0: MAC parity error

Something is happening long before you start doing ipv6. The parity error's mean that the PCI transfers are going south. The driver might be better to just shut it self off after that.

Perhaps the PCI access isn't working properly. What is the config? Also are you running i386 or x86_64 on this.

Is this fix in your version?

	http://www.mail-archive.com/[email protected]/msg18603.html
Signature · 3 lines
Stephen Hemminger <[email protected]>

All non-trivial abstractions, to some degree, are leaky.
#3 lore raw

PATCH 1/1 IPV6 : segmentation offload not set correctly on TCP children

TCP over IPV6 would incorrectly inherit the GSO settings. This would cause kernel to send Tcp Segmentation Offload packets for IPV6 data to devices that can't handle it. It caused the sky2 driver to lock http://bugzilla.kernel.org/show_bug.cgi?id=7050
and the e1000 would generate bogus packets. I can't blame the hardware for gagging if the upper layers feed it garbage.

This was a new bug in 2.6.18 introduced with GSO support.

Signed-off-by Stephen Hemminger <[email protected]>
net/ipv6/tcp_ipv6.c +1 -1
@@ -944,7 +944,7 @@
 	 * comment in that function for the gory details. -acme
 	 */
 
-	sk->sk_gso_type = SKB_GSO_TCPV6;
+	newsk->sk_gso_type = SKB_GSO_TCPV6;
 	__ip6_dst_store(newsk, dst, NULL);
 
 	newtcp6sk = (struct tcp6_sock *)newsk;
maintainer
#4 lore raw

Re: [PATCH] IPV6 : segmentation offload not set correctly on TCP children

From: Stephen Hemminger <[email protected]>
Date: Fri, 25 Aug 2006 15:43:53 -0700

Quoted 10 lines

TCP over IPV6 would incorrectly inherit the GSO settings. This would cause kernel to send Tcp Segmentation Offload packets for IPV6 data to devices that can't handle it. It caused the sky2 driver to lock http://bugzilla.kernel.org/show_bug.cgi?id=7050
and the e1000 would generate bogus packets. I can't blame the hardware for gagging if the upper layers feed it garbage.

This was a new bug in 2.6.18 introduced with GSO support.

Signed-off-by Stephen Hemminger <[email protected]>

Good catch. Applied, thanks Stephen.

author
#5 lore raw

Re: [PATCH] IPV6 : segmentation offload not set correctly on TCP children

In reply to Stephen Hemminger

Hello Stephen,
thanks for the fix, it fixes the problem for me. I closed the bug. On which hardware did you reproduce the bug and how did you found it? Did you use git bisect?

Thomas

#6 lore raw

Re: [PATCH] IPV6 : segmentation offload not set correctly on TCP children

Quoted 6 lines · On Sat, 26 Aug 2006 01:06:26 +0200 Thomas Glanzmann…

Hello Stephen,
thanks for the fix, it fixes the problem for me. I closed the bug. On which hardware did you reproduce the bug and how did you found it? Did you use git bisect?

Thomas

Using sky2 on Intel motherboard and git bisect. But Jesse found same problem on e1000. Bisect was relatively fast since it was either base net code or driver. Starting with 2.6.17, everything worked, and latest was busted.

git bisect start net drivers/net/sky2.c

Test was ipv6 slogin, then run dmesg. If that worked then run top and cause lots of other traffic by just doing a firefox open-tabs to blast lots of connections. Wanted to induce tcp-ipv6 to get congested.

Signature · 3 lines
Stephen Hemminger <[email protected]>

All non-trivial abstractions, to some degree, are leaky.
#7 lore raw

Re: [PATCH] IPV6 : segmentation offload not set correctly on TCP children

In reply to Thomas Glanzmann
Quoted 6 lines · On Sat, 26 Aug 2006 01:06:26 +0200 Thomas Glanzmann…

Hello Stephen,
thanks for the fix, it fixes the problem for me. I closed the bug. On which hardware did you reproduce the bug and how did you found it? Did you use git bisect?

Thomas

Finding the line was luck. I spotted similar (but correct) code in DCCP over IPV6.

Signature · 3 lines
Stephen Hemminger <[email protected]>

All non-trivial abstractions, to some degree, are leaky.
#8 lore raw

Re: [PATCH] IPV6 : segmentation offload not set correctly on TCP children

In reply to Stephen Hemminger
Quoted 10 lines · Stephen Hemminger

TCP over IPV6 would incorrectly inherit the GSO settings. This would cause kernel to send Tcp Segmentation Offload packets for IPV6 data to devices that can't handle it. It caused the sky2 driver to lock http://bugzilla.kernel.org/show_bug.cgi?id=7050
and the e1000 would generate bogus packets. I can't blame the hardware for gagging if the upper layers feed it garbage.

This was a new bug in 2.6.18 introduced with GSO support.

Signed-off-by Stephen Hemminger <[email protected]>

Thanks for catching this Stephen!

Cheers,

Signature · 4 lines
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
author
#9 lore raw
In reply to Stephen Hemminger

Hello,

[resent because forget to CC netdev ]

Quoted 3 lines

Something is happening long before you start doing ipv6. The parity error's mean that the PCI transfers are going south. The driver might be better to just shut it self off after that.

I can ssh off the machine for days. But sshing in IPv6 and starting vim makes the network card hang.

Quoted 2 lines

Perhaps the PCI access isn't working properly. What is the config? Also are you running i386 or x86_64 on this.

I use i386, config attached.

Quoted 2 lines

Is this fix in your version?

	http://www.mail-archive.com/[email protected]/msg18603.html

No it isn't. But I am recompiling at the moment with the patch.

And btw. IPv4 is working perfectly. But of course not after the card stopped responding. I captured the network traffic and tried to replay it, but it didn't trigger the problem. But I still am able to reproduce it manually. If you want the capture, I can give it to you.

Thomas

#10 lore raw
Quoted 2 lines · Thomas Glanzmann

I can ssh off the machine for days. But sshing in IPv6 and starting vim makes the network card hang.

This is possibly related to an equally strange issue reported on the Gentoo bugzilla. As soon as iptables is started, sky2 hangs.

http://bugs.gentoo.org/136508

I'll be sending more details as soon as I have them.

Daniel