tg3 and Netgear GA302T x 2 locks machine

DISCUSSION Ben Greear 4 participants 13 messages Oct 4, 2002 – Oct 6, 2002 Network drivers

lore

Share & tools

lore

Ben Greear

author

2002-10-04 04:14 UTC #0 lore raw

Got my two new Netgear GA302t nics today. They seem to use the tg3 driver....

I put them into the 64/66 slots on my Tyan dual amd motherboard.. Running kernel 2.4.20-pre8

eth2 is plugged into eth3 over cross-over cable.

The tg3 found and used them, mostly:

tg3.c:v1.1 (Aug 30, 2002)
eth2: Tigon3 [partno(AC91002A1) rev 0105 PHY(5701)] (PCI:66MHz:32-bit) 10/100/1000BaseT Ethernet 00:40:f4:47:20:56
eth3: Tigon3 [partno(AC91002A1) rev 0105 PHY(5701)] (PCI:66MHz:32-bit) 10/100/1000BaseT Ethernet 00:40:f4:47:22:fd
tg3: eth3: Link is up at 1000 Mbps, full duplex.
tg3: eth3: Flow control is off for TX and off for RX.
...
tg3: eth3: Link is up at 1000 Mbps, full duplex.
tg3: eth3: Flow control is off for TX and off for RX.
tg3: eth3: Link is down.
tg3: eth3: Link is up at 1000 Mbps, full duplex.
tg3: eth3: Flow control is on for TX and on for RX.

I tried to start traffic with pktgen, and the machine hangs solid. The last thing I see is:

[root@localhost root]# tg3: eth2: transmit timed out, resetting

Any ideas? Is this NIC supposed to work with tg3?

Thanks,
Ben

Signature · 3 lines

Ben Greear <[email protected]>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

Ben Greear

author

2002-10-04 05:01 UTC #1 lore raw

Quoted 2 lines · Ben Greear

Got my two new Netgear GA302t nics today. They seem to use the tg3 driver....

Changed the PCI slot of eth2 to a 32-bit slot, and now it works better...

It passed packets for about 20 seconds, then spit out a bunch of these:

tg3: eth3: Error, poll already scheduled
tg3: eth2: Error, poll already scheduled
tg3: eth2: Error, poll already scheduled
tg3: eth2: Error, poll already scheduled
tg3: eth3: Error, poll already scheduled
tg3: eth2: Error, poll already scheduled
tg3: eth2: Error, poll already scheduled
tg3: eth3: Error, poll already scheduled
tg3: eth2: Error, poll already scheduled
tg3: eth2: Error, poll already scheduled

... Then the machine locked hard

Ben

Signature · 3 lines

Ben Greear <[email protected]>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

James Morris

2002-10-04 15:47 UTC #2 lore raw

Quoted 5 lines · Ben Greear

Changed the PCI slot of eth2 to a 32-bit slot, and now it works better...
It passed packets for about 20 seconds, then spit out a bunch of these:

Which kernel version? There was a lockup problem for the GA302T just after NAPI went in (which was fixed soon after).

- James

Signature · 2 lines

James Morris
<[email protected]>

David S. Miller

2002-10-04 21:26 UTC #4 lore raw

In reply to Ben Greear

From: Ben Greear <[email protected]>
Date: Thu, 03 Oct 2002 21:19:37 -0700

Got my two new Netgear GA302t nics today. They seem to use the tg3 driver....

I put them into the 64/66 slots on my Tyan dual amd motherboard.. Running kernel 2.4.20-pre8

You reported the other week problems with two Acenic's in this same machine right? The second Acenic wouldn't even probe properly. And the two Acenic's were identical.

I see a pattern developing... :-)

James Morris

2002-10-05 01:14 UTC #5 lore raw

Quoted 6 lines · David S. Miller

From: Ben Greear <[email protected]>
Date: Thu, 03 Oct 2002 21:19:37 -0700
Got my two new Netgear GA302t nics today. They seem to use the tg3 driver....

Quoted 7 lines

I put them into the 64/66 slots on my Tyan dual amd motherboard.. Running kernel 2.4.20-pre8
You reported the other week problems with two Acenic's in this same machine right? The second Acenic wouldn't even probe properly. And the two Acenic's were identical.

FWIW, my GA302T seems fine with the kernel he originally reported (2.4.20-pre8).

- James

Signature · 2 lines

James Morris
<[email protected]>

David S. Miller

2002-10-05 01:17 UTC #6 lore raw

From: James Morris <[email protected]>
Date: Sat, 5 Oct 2002 11:20:06 +1000 (EST)

Quoted 3 lines · David S. Miller

You reported the other week problems with two Acenic's in this same machine right? The second Acenic wouldn't even probe properly. And the two Acenic's were identical.

FWIW, my GA302T seems fine with the kernel he originally reported (2.4.20-pre8).

Yes but are you running parallel pktgen streams on two tg3's? :-)

Ben, by chance, does reverting the patch below cure your problems?

# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher. # This patch includes the following deltas:
# ChangeSet 1.670 -> 1.671
# drivers/net/tg3.c 1.35 -> 1.36
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 02/09/17 [email protected] 1.671
# [TIGON3]: Optimize NAPI irq masking a bit.
# --------------------------------------------
#

drivers/net/tg3.c +19 -6

diff -Nru a/drivers/net/tg3.c b/drivers/net/tg3.c
@@ -234,9 +234,23 @@
 		tw32(GRC_LOCAL_CTRL,
 		     tp->grc_local_ctrl | GRC_LCLCTRL_SETINT);
 	}
-#if 0
 	tr32(MAILBOX_INTERRUPT_0 + TG3_64BIT_REG_LOW);
-#endif
+}
+
+static inline void tg3_mask_ints(struct tg3 *tp)
+{
+	tw32(TG3PCI_MISC_HOST_CTRL,
+	     (tp->misc_host_ctrl | MISC_HOST_CTRL_MASK_PCI_INT));
+}
+
+static inline void tg3_unmask_ints(struct tg3 *tp)
+{
+	tw32(TG3PCI_MISC_HOST_CTRL,
+	     (tp->misc_host_ctrl & ~MISC_HOST_CTRL_MASK_PCI_INT));
+	if (tp->hw_status->status & SD_STATUS_UPDATED) {
+		tw32(GRC_LOCAL_CTRL,
+		     tp->grc_local_ctrl | GRC_LCLCTRL_SETINT);
+	}
 }
 
 static void tg3_switch_clocks(struct tg3 *tp)

@@ -2093,7 +2107,7 @@
 
 	if (done) {
 		netif_rx_complete(netdev);
-		tg3_enable_ints(tp);
+		tg3_unmask_ints(tp);
 	}
 
 	spin_unlock_irq(&tp->lock);

@@ -2120,11 +2134,10 @@
 		return;
 
 	if (netif_rx_schedule_prep(dev)) {
-		/* NOTE: This write is posted by the readback of
+		/* NOTE: These writes are posted by the readback of
 		 *       the mailbox register done by our caller.
 		 */
-		tw32(TG3PCI_MISC_HOST_CTRL,
-		     (tp->misc_host_ctrl | MISC_HOST_CTRL_MASK_PCI_INT));
+		tg3_mask_ints(tp);
 		__netif_rx_schedule(dev);
 	} else {
 		printk(KERN_ERR PFX "%s: Error, poll already scheduled\n",

Matthew N. Andrews

2002-10-05 02:06 UTC #11 lore raw

In reply to James Morris

Quoted 20 lines · James Morris

Quoted 6 lines · David S. Miller
From: Ben Greear <[email protected]>
Date: Thu, 03 Oct 2002 21:19:37 -0700
Got my two new Netgear GA302t nics today. They seem to use the tg3 driver....
Quoted 9 lines
I put them into the 64/66 slots on my Tyan dual amd motherboard.. Running kernel 2.4.20-pre8
You reported the other week problems with two Acenic's in this same machine right? The second Acenic wouldn't even probe properly. And the two Acenic's were identical.

Quoted 6 lines

FWIW, my GA302T seems fine with the kernel he originally reported (2.4.20-pre8).
- James

What version of the bios does your motherboard have? I had problems with a dual syskonnect board on this motherboard(tyan 2466) where it would not would not initialize properly under the older bios.

-Matt

Ben Greear

author

2002-10-05 05:42 UTC #12 lore raw

In reply to David S. Miller

Quoted 6 lines · David S. Miller

From: Ben Greear <[email protected]>
Date: Thu, 03 Oct 2002 21:19:37 -0700
Got my two new Netgear GA302t nics today. They seem to use the tg3 driver....

Quoted 9 lines

I put them into the 64/66 slots on my Tyan dual amd motherboard.. Running kernel 2.4.20-pre8
You reported the other week problems with two Acenic's in this same machine right? The second Acenic wouldn't even probe properly. And the two Acenic's were identical.
I see a pattern developing... :-)

Yep, one of my 64-bit PCI slots only seems to work with Intel NICs. Go figure.

At any rate, I have 4 32-bit slots, and they seem to work, at least long enough for the driver to puke messages and hard-lock my machine ;)

If you think this NIC should work with the tg3, then I can change them to a completely other machine, btw.

Ben

Signature · 3 lines

Ben Greear <[email protected]>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

Ben Greear

author

2002-10-05 05:45 UTC #3 lore raw

In reply to James Morris

Quoted 11 lines · James Morris

Quoted 5 lines · Ben Greear
Changed the PCI slot of eth2 to a 32-bit slot, and now it works better...
It passed packets for about 20 seconds, then spit out a bunch of these:
Which kernel version? There was a lockup problem for the GA302T just after NAPI went in (which was fixed soon after).

This was 2.4.20-pre8, ie quite recent. I didn't see any mention of tg3 in the pre9 patch that just came out.

I'm beginning to think SMP may not be worth it...will try in my trusty P-IV single-processor cheap-o box next.

Ben

Quoted 3 lines

- James

Signature · 3 lines

Ben Greear <[email protected]>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

Ben Greear

author

2002-10-05 05:58 UTC #7 lore raw

In reply to David S. Miller

Quoted 9 lines · David S. Miller

From: James Morris <[email protected]>
Date: Sat, 5 Oct 2002 11:20:06 +1000 (EST)
Quoted 3 lines · David S. Miller
You reported the other week problems with two Acenic's in this same machine right? The second Acenic wouldn't even probe properly. And the two Acenic's were identical.

Quoted 5 lines

FWIW, my GA302T seems fine with the kernel he originally reported (2.4.20-pre8).
Yes but are you running parallel pktgen streams on two tg3's? :-)

Heh, tis true, I'm abusing them horribly. ;)

So far, the tulip driver and DFE-570tx & Phobos NICs are the only thing I've found that can stand up to my tests... (e1000 pukes, old acenic pukes (after quite a while), tg3 pukes, at least currently...

I'll try this this weekend. Thanks for the suggestion.

Ben

Quoted 2 lines

Ben, by chance, does reverting the patch below cure your problems?

[patch snipped]

Signature · 3 lines

Ben Greear <[email protected]>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

Ben Greear

author

2002-10-05 20:02 UTC #8 lore raw

In reply to David S. Miller

Quoted 4 lines · David S. Miller

Yes but are you running parallel pktgen streams on two tg3's? :-)
Ben, by chance, does reverting the patch below cure your problems?

No, didn't fix it, maybe made it worse.

With pktgen: Locked up as soon as I started it (or less than .5 seconds afterwards).

With raw ethernet packets, sent from user-space, at around 40Mbps bi-directional, I see loads of these messages:

tg3: eth3: Error, poll already scheduled
tg3: eth2: Error, poll already scheduled
tg3: eth2: Error, poll already scheduled
tg3: eth3: Error, poll already scheduled
tg3: eth3: Error, poll already scheduled
tg3: eth3: Error, poll already scheduled
tg3: eth3: Error, poll already scheduled
tg3: eth3: Error, poll already scheduled

...

After about 1 minute, the whole machine locked.

Ben

Signature · 3 lines

Ben Greear <[email protected]>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

David S. Miller

2002-10-06 04:24 UTC #9 lore raw

From: Ben Greear <[email protected]>
Date: Sat, 05 Oct 2002 13:08:02 -0700

With raw ethernet packets, sent from user-space, at around 40Mbps bi-directional, I see loads of these messages:

tg3: eth3: Error, poll already scheduled

This, frankly, isn't possible.

When we get the first interrupt, we hold the spinlock and have IRQs disabled, in that environment we invoke netif_rx_schedule_prep(dev) and then disable device interrupts....

is the tg3 sharing it's IRQ with something else? That might be an important clue. In that case what you report might be possible.

Otherwise the message you see appears to be totally impossible.

Ben Greear

author

2002-10-06 04:52 UTC #10 lore raw

Quoted 8 lines · David S. Miller

From: Ben Greear <[email protected]>
Date: Sat, 05 Oct 2002 13:08:02 -0700
With raw ethernet packets, sent from user-space, at around 40Mbps bi-directional, I see loads of these messages:
tg3: eth3: Error, poll already scheduled

Quoted 8 lines

This, frankly, isn't possible.
When we get the first interrupt, we hold the spinlock and have IRQs disabled, in that environment we invoke netif_rx_schedule_prep(dev) and then disable device interrupts....
is the tg3 sharing it's IRQ with something else? That might be an important clue. In that case what you report might be possible.

Here is what the interrupts look like:

[root@localhost lanforge]# cat /proc/interrupts
CPU0 CPU1

  0:      14734      19562    IO-APIC-edge  timer
  1:          3          1    IO-APIC-edge  keyboard
  2:          0          0          XT-PIC  cascade
  4:       1163       1124    IO-APIC-edge  serial
  5:          0          0   IO-APIC-level  eth1
  8:          0          1    IO-APIC-edge  rtc
  9:        558          0   IO-APIC-level  eth2, eth4
 10:          0          0   IO-APIC-level  usb-ohci
 11:       1385       1454   IO-APIC-level  eth0, eth3, eth5
 12:         20         12    IO-APIC-edge  PS/2 Mouse
 14:       3958       4169    IO-APIC-edge  ide0
 15:          7         13    IO-APIC-edge  ide1
NMI:          0          0
LOC:      34138      34183
ERR:          0
MIS:          0

[root@localhost lanforge]#

eth0-1 is the 3com built-in nics
eth2-3 is the e1000 dual nic
eth4-5 is the tg3

Oct  5 21:42:10 localhost kernel: tg3.c:v1.1 (Aug 30, 2002)
Oct  5 21:42:11 localhost kernel: eth4: Tigon3 [partno(AC91002A1) rev 0105 PHY(5701)] (PCI:33MHz:32-bit) 10/100/1000BaseT Ethernet 00:40:f4:47:22:fd
Oct  5 21:42:11 localhost kernel: eth5: Tigon3 [partno(AC91002A1) rev 0105 PHY(5701)] (PCI:33MHz:32-bit) 10/100/1000BaseT Ethernet 00:40:f4:47:20:56

Upon starting a user-space TCP connection to myself (no traffic running on eth2-3, eth1 is not plugged in to a cable. eth0 is handling a small amount of traffic, no more than about 25 packets per second on average):

[root@localhost lanforge]# tg3: eth5: Error, poll already scheduled
tg3: eth5: Error, poll already scheduled
tg3: eth5: Error, poll already scheduled
tg3: eth5: Error, poll already scheduled
tg3: eth5: Error, poll already scheduled
tg3: eth5: Error, poll already scheduled
tg3: eth5: Error, poll already scheduled

Please let me know what other debugging info I can get you.

Quoted 1 line

Otherwise the message you see appears to be totally impossible.

I told the machine that...but it only blinked it's little leds in mirth! ;)

Thanks,
Ben

Signature · 3 lines

Ben Greear <[email protected]>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear