Skip to contentSkip to conversation

NAPI for eepro100

PATCH Zhang Fuxin 7 participants 17 messages Jun 12, 2002 – Jun 19, 2002

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

hi,all
Recently i've converted eepro100 driver to use napi,in order to improve
network performance of my poor 150M mips machine. It does eliminate the interrupt live lock seen before,maintaining a peak throughput under heavy load.
In case anybody are interested,i post the patches to the list. They are
3 incremental patchs:
eepro100-napi.patch is against 2.5.20 eepro100.c and provide basic
napi support
eepro100-proc.patch is proc file system support adapted from intel's
e100 driver. I am using it for debugging.
eepro100-mips.patch is mips specific patch to make it work(well) for
my mips
platform.
(i suppose people use: patch eepro100.c < x.patch to apply patch)

Tests are mainly done on my mips machine for 2.4 kernel,though i think
it should work for at least x86 on which minimal test is performed. Be careful.

A little pitty is that to achieve good performance under heavy load, the
/proc/sys/net/core/netdev_max_backlog value may need to be adjusted.

Feedbacks are always welcome:). But i am not on linux-kernel list,so
people
there please CC me.

eepro100-mips.c renamed from eepro100-napi-proc.c +26 -3
@@ -45,7 +45,7 @@
 
 /* Set the copy breakpoint for the copy-only-tiny-buffer Rx method.
    Lower values use more memory, but are faster. */
-#if defined(__alpha__) || defined(__sparc__) || defined(__mips__) || \
+#if defined(__alpha__) || defined(__sparc__) || /*defined(__mips__) ||*/ \
     defined(__arm__)
 static int rx_copybreak = 1518;
 #else
@@ -66,8 +66,8 @@
 
 /* A few values that may be tweaked. */
 /* The ring sizes should be a power of two for efficiency. */
-#define TX_RING_SIZE	64
-#define RX_RING_SIZE	64
+#define TX_RING_SIZE	32
+#define RX_RING_SIZE	32
 /* How much slots multicast filter setup may take.
    Do not descrease without changing set_rx_mode() implementaion. */
 #define TX_MULTICAST_SIZE   2
@@ -1298,7 +1298,14 @@
 		if (skb == NULL)
 			break;			/* OK.  Just initially short of Rx bufs. */
 		skb->dev = dev;			/* Mark as being used by this device. */
+#ifndef __mips__
 		rxf = (struct RxFD *)skb->tail;
+#else
+		/* use uncached address,use pci_dma_sync_xx seems 
+		 * problematic in my mips platform
+		 */
+		rxf = (struct RxFD *)(KSEG1ADDR(skb->tail));
+#endif
 		sp->rx_ringp[i] = rxf;
 		sp->rx_ring_dma[i] =
 			pci_map_single(sp->pdev, rxf,
@@ -1306,8 +1313,10 @@
 		skb_reserve(skb, sizeof(struct RxFD));
 		if (last_rxf) {
 			last_rxf->link = cpu_to_le32(sp->rx_ring_dma[i]);
+#ifndef __mips__
 			pci_dma_sync_single(sp->pdev, last_rxf_dma,
 					sizeof(struct RxFD), PCI_DMA_TODEVICE);
+#endif
 		}
 		last_rxf = rxf;
 		last_rxf_dma = sp->rx_ring_dma[i];
@@ -1316,14 +1325,18 @@
 		/* This field unused by i82557. */
 		rxf->rx_buf_addr = 0xffffffff;
 		rxf->count = cpu_to_le32(PKT_BUF_SZ << 16);
+#ifndef __mips__
 		pci_dma_sync_single(sp->pdev, sp->rx_ring_dma[i],
 				sizeof(struct RxFD), PCI_DMA_TODEVICE);
+#endif
 	}
 	sp->dirty_rx = (unsigned int)(i - RX_RING_SIZE);
 	/* Mark the last entry as end-of-list. */
 	last_rxf->status = cpu_to_le32(0xC0000002);	/* '2' is flag value only. */
+#ifndef __mips__
 	pci_dma_sync_single(sp->pdev, sp->rx_ring_dma[RX_RING_SIZE-1],
 			sizeof(struct RxFD), PCI_DMA_TODEVICE);
+#endif
 	sp->last_rxf = last_rxf;
 	sp->last_rxf_dma = last_rxf_dma;
 }
@@ -1733,15 +1746,21 @@
 #endif
 		return NULL;
 	}
+#ifndef __mips__
 	rxf = sp->rx_ringp[entry] = (struct RxFD *)skb->tail;
+#else
+	rxf = sp->rx_ringp[entry] = (struct RxFD *)(KSEG1ADDR(skb->tail));
+#endif
 	sp->rx_ring_dma[entry] =
 		pci_map_single(sp->pdev, rxf,
 					   PKT_BUF_SZ + sizeof(struct RxFD), PCI_DMA_FROMDEVICE);
 	skb->dev = dev;
 	skb_reserve(skb, sizeof(struct RxFD));
 	rxf->rx_buf_addr = 0xffffffff;
+#ifndef __mips__
 	pci_dma_sync_single(sp->pdev, sp->rx_ring_dma[entry],
 			sizeof(struct RxFD), PCI_DMA_TODEVICE);
+#endif
 	return rxf;
 }
 
@@ -1754,8 +1773,10 @@
 	rxf->count = cpu_to_le32(PKT_BUF_SZ << 16);
 	sp->last_rxf->link = cpu_to_le32(rxf_dma);
 	sp->last_rxf->status &= cpu_to_le32(~0xC0000000);
+#ifndef __mips__
 	pci_dma_sync_single(sp->pdev, sp->last_rxf_dma,
 			sizeof(struct RxFD), PCI_DMA_TODEVICE);
+#endif
 	sp->last_rxf = rxf;
 	sp->last_rxf_dma = rxf_dma;
 }
@@ -2274,8 +2295,10 @@
 		int status;
 		int pkt_len;
 
+#ifndef __mips__
 		pci_dma_sync_single(sp->pdev, sp->rx_ring_dma[entry],
 			sizeof(struct RxFD), PCI_DMA_FROMDEVICE);
+#endif
 		status = le32_to_cpu(sp->rx_ringp[entry]->status);
 		pkt_len = le32_to_cpu(sp->rx_ringp[entry]->count) & 0x3fff;
 
eepro100-napi.c renamed from eepro100.c.ori +224 -3

350 lines not on this page · raw message

eepro100-napi-proc.c renamed from eepro100-napi.c +372 -0

408 lines not on this page · raw message

#1 lore raw
Quoted 9 lines · Zhang Fuxin

hi,all
Recently i've converted eepro100 driver to use napi,in order to improve
network performance of my poor 150M mips machine. It does eliminate the interrupt live lock seen before,maintaining a peak throughput under heavy load.
In case anybody are interested,i post the patches to the list. They are
3 incremental patchs:
eepro100-napi.patch is against 2.5.20 eepro100.c and provide basic
napi support

Nifty, I'll take a look at this.

Quoted 5 lines

eepro100-proc.patch is proc file system support adapted from intel's
e100 driver. I am using it for debugging.
eepro100-mips.patch is mips specific patch to make it work(well) for
my mips
platform.

Just FWIW I'm not gonna apply these... for the 'proc' patch, that either needs to be moved to ethtool, or we should make a filesystem for net drivers that exports procfs-like inodes. for the 'mips' patch, it looks like the arch maintainer(s) need to fix the PCI DMA support...

Jeff

#2 lore raw
In reply to Zhang Fuxin
Quoted 9 lines · Zhang Fuxin

hi,all
Recently i've converted eepro100 driver to use napi,in order to improve
network performance of my poor 150M mips machine. It does eliminate the interrupt live lock seen before,maintaining a peak throughput under heavy load.
In case anybody are interested,i post the patches to the list. They are
3 incremental patchs:
eepro100-napi.patch is against 2.5.20 eepro100.c and provide basic
napi support

Nifty, I'll take a look at this.

Quoted 5 lines

eepro100-proc.patch is proc file system support adapted from intel's
e100 driver. I am using it for debugging.
eepro100-mips.patch is mips specific patch to make it work(well) for
my mips
platform.

Just FWIW I'm not gonna apply these... for the 'proc' patch, that either needs to be moved to ethtool, or we should make a filesystem for net drivers that exports procfs-like inodes. for the 'mips' patch, it looks like the arch maintainer(s) need to fix the PCI DMA support...

Jeff

#3 lore raw

From: Jeff Garzik <[email protected]>
Date: Wed, 12 Jun 2002 19:00:00 -0400

for the 'mips' patch, it looks
like the arch maintainer(s) need to fix the PCI DMA support...

No, it's worse than that.

See how non-consistent memory is used by the eepro100 driver for descriptor bits? The skb->tail bits?

That is very problematic.

#4 lore raw
In reply to Jeff Garzik

From: Jeff Garzik <[email protected]>
Date: Wed, 12 Jun 2002 19:00:00 -0400

for the 'mips' patch, it looks
like the arch maintainer(s) need to fix the PCI DMA support...

No, it's worse than that.

See how non-consistent memory is used by the eepro100 driver for descriptor bits? The skb->tail bits?

That is very problematic.

#5 lore raw
Quoted 6 lines · David S. Miller

From: Jeff Garzik <[email protected]>
Date: Wed, 12 Jun 2002 19:00:00 -0400

for the 'mips' patch, it looks
like the arch maintainer(s) need to fix the PCI DMA support...

Quoted 6 lines

No, it's worse than that.

See how non-consistent memory is used by the eepro100 driver for descriptor bits? The skb->tail bits?

That is very problematic.

Oh crap, you're right... eepro100 in general does funky stuff with the way packets are handled, mainly due to the need to issue commands to the NIC engine instead of the normal per-descriptor owner bit way of doing things.

Well, I accept patches to that clean eepro100 up... I'm not terribly motivated to clean it up myself, as we have e100 and an e100 maintainer we can beat on if such uglies arise :)

Jeff

#6 lore raw
In reply to David S. Miller
Quoted 6 lines · David S. Miller

From: Jeff Garzik <[email protected]>
Date: Wed, 12 Jun 2002 19:00:00 -0400

for the 'mips' patch, it looks
like the arch maintainer(s) need to fix the PCI DMA support...

Quoted 6 lines

No, it's worse than that.

See how non-consistent memory is used by the eepro100 driver for descriptor bits? The skb->tail bits?

That is very problematic.

What's the problem?
If it isn't allowed to do, then what is the meaning of PCI_DMA_BIDIRECTIONAL mappings?

Andrey

#7 lore raw
In reply to Zhang Fuxin
Quoted 6 lines · David S. Miller

From: Jeff Garzik <[email protected]>
Date: Wed, 12 Jun 2002 19:00:00 -0400

for the 'mips' patch, it looks
like the arch maintainer(s) need to fix the PCI DMA support...

Quoted 6 lines

No, it's worse than that.

See how non-consistent memory is used by the eepro100 driver for descriptor bits? The skb->tail bits?

That is very problematic.

Oh crap, you're right... eepro100 in general does funky stuff with the way packets are handled, mainly due to the need to issue commands to the NIC engine instead of the normal per-descriptor owner bit way of doing things.

Well, I accept patches to that clean eepro100 up... I'm not terribly motivated to clean it up myself, as we have e100 and an e100 maintainer we can beat on if such uglies arise :)

Jeff

#8 lore raw

From: Jeff Garzik <[email protected]>
Date: Wed, 12 Jun 2002 19:17:58 -0400

Oh crap, you're right... eepro100 in general does funky stuff with the
way packets are handled, mainly due to the need to issue commands to the NIC engine instead of the normal per-descriptor owner bit way of doing things.

The question is, do the descriptor bits have to live right before the RX packet data buffer or can other schemes be used?

#9 lore raw
In reply to Jeff Garzik

From: Jeff Garzik <[email protected]>
Date: Wed, 12 Jun 2002 19:17:58 -0400

Oh crap, you're right... eepro100 in general does funky stuff with the
way packets are handled, mainly due to the need to issue commands to the NIC engine instead of the normal per-descriptor owner bit way of doing things.

The question is, do the descriptor bits have to live right before the RX packet data buffer or can other schemes be used?

#10 lore raw
Quoted 5 lines · David S. Miller

From: Jeff Garzik <[email protected]>
Oh crap, you're right... eepro100 in general does funky stuff with the
way packets are handled, mainly due to the need to issue commands to the NIC engine instead of the normal per-descriptor owner bit way of doing things.

The eepro100 has a unique design in many different aspects.

Quoted 2 lines

The question is, do the descriptor bits have to live right before the RX packet data buffer or can other schemes be used?

With the current driver structure, yes, the descriptor words must be immediately before the packet data. You can use other Rx and Tx structures/modes to avoid this, but they use less efficient memory access. For instance, the current Tx structure allows transmitting a packet with a single PCI burst, rather than multiple transfers.

Signature · 4 lines
Donald Becker				[email protected]
Scyld Computing Corporation		http://www.scyld.com
410 Severn Ave. Suite 210		Second Generation Beowulf Clusters
Annapolis MD 21403			410-990-9993
#11 lore raw
In reply to David S. Miller
Quoted 5 lines · David S. Miller

From: Jeff Garzik <[email protected]>
Oh crap, you're right... eepro100 in general does funky stuff with the
way packets are handled, mainly due to the need to issue commands to the NIC engine instead of the normal per-descriptor owner bit way of doing things.

The eepro100 has a unique design in many different aspects.

Quoted 2 lines

The question is, do the descriptor bits have to live right before the RX packet data buffer or can other schemes be used?

With the current driver structure, yes, the descriptor words must be immediately before the packet data. You can use other Rx and Tx structures/modes to avoid this, but they use less efficient memory access. For instance, the current Tx structure allows transmitting a packet with a single PCI burst, rather than multiple transfers.

Signature · 4 lines
Donald Becker				[email protected]
Scyld Computing Corporation		http://www.scyld.com
410 Severn Ave. Suite 210		Second Generation Beowulf Clusters
Annapolis MD 21403			410-990-9993
#12 lore raw
In reply to David S. Miller
Quoted 9 lines · Donald Becker
Quoted 5 lines · David S. Miller

From: Jeff Garzik <[email protected]>
Oh crap, you're right... eepro100 in general does funky stuff with the
way packets are handled, mainly due to the need to issue commands to the NIC engine instead of the normal per-descriptor owner bit way of doing things.

The eepro100 has a unique design in many different aspects.

Quoted 8 lines
Quoted 2 lines

The question is, do the descriptor bits have to live right before the RX packet data buffer or can other schemes be used?

With the current driver structure, yes, the descriptor words must be immediately before the packet data. You can use other Rx and Tx structures/modes to avoid this, but they use less efficient memory access. For instance, the current Tx structure allows transmitting a packet with a single PCI burst, rather than multiple transfers.

Maybe a bit off topic, but we (at my work) are using plenty of eepro100 cards with both drivers ( e100 and eepro100 )(shipped with dell machines, and others).
We have lot of problem with these card: from link autonegociation to the really frequent cmd_timeout.
We expreienced some freezes, slowdowns, problems with copying from NFS to a Firwire disk ( systematic cmd_timeout at about 250 MB).

Do you have any advice ? should I test eepro100 NAPI driver ? I've try to play with ethtool(chang some eepro100 bits , like the "sleeping" one ...

I have quitely the same card at home wich doesn't make any problem ( I noticed some cmd_timeout when I changed my hub).
Is this hub related ? Is there a standart way autonegociation is working ( we use mostly cisco switches, are they compliant?).

We are actually trying to force 10FD or 100FD any new installed card because we think this is the best way to avoid performances problem ...

Thanks for any advice.
Sam

#13 lore raw
Quoted 7 lines · Samuel Maftoul

Maybe a bit off topic, but we (at my work) are using plenty of eepro100 cards with both drivers ( e100 and eepro100 )(shipped with dell machines, and others).
We have lot of problem with these card: from link autonegociation to the really frequent cmd_timeout.
We expreienced some freezes, slowdowns, problems with copying from NFS to a Firwire disk ( systematic cmd_timeout at about 250 MB).

Yes, this would be better in the cosl.networking, but a quick answer since it seems kernel related. I had problems with these until recent kernels. The e100 driver helped in some cases, but other issues were reported. I don't really have any problems now, on news servers which get ~250GB/day in and push ~700 out (yes, they run 70-80Mbit all day).

This may be related to the blessing of the new scheduler.

Quoted 3 lines

Do you have any advice ? should I test eepro100 NAPI driver ? I've try to play with ethtool(chang some eepro100 bits , like the "sleeping" one ...

We noticed long ago that different blades on the same switch would work right with either auto at both ends or 100TX forced at both ends. Don;t know why, just look at ifconfig reports for collisions and if you see them change the settings.

Quoted 4 lines

I have quitely the same card at home wich doesn't make any problem ( I noticed some cmd_timeout when I changed my hub).
Is this hub related ? Is there a standart way autonegociation is working ( we use mostly cisco switches, are they compliant?).

You have said both "hub" and "switch" in this paragraph, trying to run a hub full deplex will cause problems. I have no idea what a "switching hub" means, I see them around cheap, but not from Cisco.

Quoted 2 lines

We are actually trying to force 10FD or 100FD any new installed card because we think this is the best way to avoid performances problem ...

We have never found the slightest way to predect if auto or forced 100TX works with any given blade in any given switch. But it doesn't change, so get it right and it's fixed. See above, look for collisions, try auto first. No collisions, it's as good as it gets.

Signature · 3 lines
bill davidsen <[email protected]>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
#14 lore raw
In reply to Zhang Fuxin
Quoted 6 lines · David S. Miller

From: Jeff Garzik <[email protected]>
Date: Wed, 12 Jun 2002 19:00:00 -0400

for the 'mips' patch, it looks
like the arch maintainer(s) need to fix the PCI DMA support...

Quoted 6 lines

No, it's worse than that.

See how non-consistent memory is used by the eepro100 driver for descriptor bits? The skb->tail bits?

That is very problematic.

What's the problem?
If it isn't allowed to do, then what is the meaning of PCI_DMA_BIDIRECTIONAL mappings?

Andrey

#15 lore raw

From: Andrey Savochkin <[email protected]>
Date: Thu, 13 Jun 2002 12:57:53 +0400

Quoted 6 lines · David S. Miller

No, it's worse than that.

See how non-consistent memory is used by the eepro100 driver for descriptor bits? The skb->tail bits?

That is very problematic.

What's the problem?
If it isn't allowed to do, then what is the meaning of PCI_DMA_BIDIRECTIONAL mappings?

It's slow. Not wrong, just inefficient.

Descriptors were meant to be done using consistent mappings, not "pci_map_*()"'d memory. The latter is meant to be used for long linear DMA transfers to/from the device. It is not meant for things the cpu pokes small bits of data in and out of, that is what consistent DMA memory is for.

#16 lore raw
In reply to Andrey Savochkin

From: Andrey Savochkin <[email protected]>
Date: Thu, 13 Jun 2002 12:57:53 +0400

Quoted 6 lines · David S. Miller

No, it's worse than that.

See how non-consistent memory is used by the eepro100 driver for descriptor bits? The skb->tail bits?

That is very problematic.

What's the problem?
If it isn't allowed to do, then what is the meaning of PCI_DMA_BIDIRECTIONAL mappings?

It's slow. Not wrong, just inefficient.

Descriptors were meant to be done using consistent mappings, not "pci_map_*()"'d memory. The latter is meant to be used for long linear DMA transfers to/from the device. It is not meant for things the cpu pokes small bits of data in and out of, that is what consistent DMA memory is for.