NAPI for eepro100
Subsystems (3)
hi,all
Recently i've converted eepro100 driver to use napi,in order to improve
network performance of my poor 150M mips machine. It does eliminate the interrupt live lock seen before,maintaining a peak throughput under heavy load.
In case anybody are interested,i post the patches to the list. They are
3 incremental patchs:
eepro100-napi.patch is against 2.5.20 eepro100.c and provide basic
napi support
eepro100-proc.patch is proc file system support adapted from intel's
e100 driver. I am using it for debugging.
eepro100-mips.patch is mips specific patch to make it work(well) for
my mips
platform.
(i suppose people use: patch eepro100.c < x.patch to apply patch)
Tests are mainly done on my mips machine for 2.4 kernel,though i think
it should work for at least x86 on which minimal test is performed. Be careful.
A little pitty is that to achieve good performance under heavy load, the/proc/sys/net/core/netdev_max_backlog value may need to be adjusted.
Feedbacks are always welcome:). But i am not on linux-kernel list,so
people
there please CC me.
eepro100-mips.c renamed from eepro100-napi-proc.c +26 -3
@@ -45,7 +45,7 @@/* Set the copy breakpoint for the copy-only-tiny-buffer Rx method. Lower values use more memory, but are faster. */ -#if defined(__alpha__) || defined(__sparc__) || defined(__mips__) || \ +#if defined(__alpha__) || defined(__sparc__) ||/*defined(__mips__) ||*/ \ defined(__arm__)static int rx_copybreak =1518 ;#else
@@ -66,8 +66,8 @@/* A few values that may be tweaked. */ /* The ring sizes should be a power of two for efficiency. */ -#define TX_RING_SIZE 64 -#define RX_RING_SIZE 64 +#define TX_RING_SIZE 32 +#define RX_RING_SIZE 32/* How much slots multicast filter setup may take. Do not descrease without changing set_rx_mode() implementaion. */ #define TX_MULTICAST_SIZE 2
@@ -1298,7 +1298,14 @@if (skb ==NULL )break ;/* OK. Just initially short of Rx bufs. */ skb->dev = dev;/* Mark as being used by this device. */ +#ifndef __mips__ rxf = (struct RxFD *)skb->tail; +#else +/* use uncached address,use pci_dma_sync_xx seems +* problematic in my mips platform +*/ + rxf = (struct RxFD *)(KSEG1ADDR (skb->tail)); +#endif sp->rx_ringp[i] = rxf; sp->rx_ring_dma[i] = pci_map_single(sp->pdev, rxf,
@@ -1306,8 +1313,10 @@skb_reserve (skb,sizeof (struct RxFD ));if (last_rxf) { last_rxf->link =cpu_to_le32 (sp->rx_ring_dma[i]); +#ifndef __mips__pci_dma_sync_single (sp->pdev, last_rxf_dma,sizeof (struct RxFD ),PCI_DMA_TODEVICE ); +#endif } last_rxf = rxf; last_rxf_dma = sp->rx_ring_dma[i];
@@ -1316,14 +1325,18 @@/* This field unused by i82557. */ rxf->rx_buf_addr =0xffffffff ; rxf->count =cpu_to_le32 (PKT_BUF_SZ <<16 ); +#ifndef __mips__pci_dma_sync_single (sp->pdev, sp->rx_ring_dma[i],sizeof (struct RxFD ),PCI_DMA_TODEVICE ); +#endif } sp->dirty_rx = (unsigned int )(i -RX_RING_SIZE );/* Mark the last entry as end-of-list. */ last_rxf->status =cpu_to_le32 (0xC0000002 );/* '2' is flag value only. */ +#ifndef __mips__pci_dma_sync_single (sp->pdev, sp->rx_ring_dma[RX_RING_SIZE -1 ],sizeof (struct RxFD ),PCI_DMA_TODEVICE ); +#endif sp->last_rxf = last_rxf; sp->last_rxf_dma = last_rxf_dma; }
@@ -1733,15 +1746,21 @@#endif return NULL ; } +#ifndef __mips__ rxf = sp->rx_ringp[entry] = (struct RxFD *)skb->tail; +#else + rxf = sp->rx_ringp[entry] = (struct RxFD *)(KSEG1ADDR (skb->tail)); +#endif sp->rx_ring_dma[entry] =pci_map_single (sp->pdev, rxf,PKT_BUF_SZ +sizeof (struct RxFD ),PCI_DMA_FROMDEVICE ); skb->dev = dev;skb_reserve (skb,sizeof (struct RxFD )); rxf->rx_buf_addr =0xffffffff ; +#ifndef __mips__pci_dma_sync_single (sp->pdev, sp->rx_ring_dma[entry],sizeof (struct RxFD ),PCI_DMA_TODEVICE ); +#endif return rxf; }
@@ -1754,8 +1773,10 @@ rxf->count =cpu_to_le32 (PKT_BUF_SZ <<16 ); sp->last_rxf->link =cpu_to_le32 (rxf_dma); sp->last_rxf->status &=cpu_to_le32 (~0xC0000000 ); +#ifndef __mips__pci_dma_sync_single (sp->pdev, sp->last_rxf_dma,sizeof (struct RxFD ),PCI_DMA_TODEVICE ); +#endif sp->last_rxf = rxf; sp->last_rxf_dma = rxf_dma; }
@@ -2274,8 +2295,10 @@int status;int pkt_len; +#ifndef __mips__pci_dma_sync_single (sp->pdev, sp->rx_ring_dma[entry],sizeof (struct RxFD ),PCI_DMA_FROMDEVICE ); +#endif status =le32_to_cpu (sp->rx_ringp[entry]->status); pkt_len =le32_to_cpu (sp->rx_ringp[entry]->count) &0x3fff ;
eepro100-napi.c renamed from eepro100.c.ori +224 -3
350 lines not on this page · raw message
eepro100-napi-proc.c renamed from eepro100-napi.c +372 -0
408 lines not on this page · raw message
Quoted 9 lines · Zhang Fuxin
hi,all
Recently i've converted eepro100 driver to use napi,in order to improve
network performance of my poor 150M mips machine. It does eliminate the interrupt live lock seen before,maintaining a peak throughput under heavy load.
In case anybody are interested,i post the patches to the list. They are
3 incremental patchs:
eepro100-napi.patch is against 2.5.20eepro100.cand provide basic
napi support
Nifty, I'll take a look at this.
Quoted 5 lines
eepro100-proc.patch is proc file system support adapted from intel's
e100 driver. I am using it for debugging.
eepro100-mips.patch is mips specific patch to make it work(well) for
my mips
platform.
Just FWIW I'm not gonna apply these... for the 'proc' patch, that either needs to be moved to ethtool, or we should make a filesystem for net drivers that exports procfs-like inodes. for the 'mips' patch, it looks like the arch maintainer(s) need to fix the PCI DMA support...
Jeff
Quoted 9 lines · Zhang Fuxin
hi,all
Recently i've converted eepro100 driver to use napi,in order to improve
network performance of my poor 150M mips machine. It does eliminate the interrupt live lock seen before,maintaining a peak throughput under heavy load.
In case anybody are interested,i post the patches to the list. They are
3 incremental patchs:
eepro100-napi.patch is against 2.5.20eepro100.cand provide basic
napi support
Nifty, I'll take a look at this.
Quoted 5 lines
eepro100-proc.patch is proc file system support adapted from intel's
e100 driver. I am using it for debugging.
eepro100-mips.patch is mips specific patch to make it work(well) for
my mips
platform.
Just FWIW I'm not gonna apply these... for the 'proc' patch, that either needs to be moved to ethtool, or we should make a filesystem for net drivers that exports procfs-like inodes. for the 'mips' patch, it looks like the arch maintainer(s) need to fix the PCI DMA support...
Jeff
From: Jeff Garzik <[email protected]>
Date: Wed, 12 Jun 2002 19:00:00 -0400
for the 'mips' patch, it looks
like the arch maintainer(s) need to fix the PCI DMA support...
No, it's worse than that.
See how non-consistent memory is used by the eepro100 driver for descriptor bits? The skb->tail bits?
That is very problematic.
From: Jeff Garzik <[email protected]>
Date: Wed, 12 Jun 2002 19:00:00 -0400
for the 'mips' patch, it looks
like the arch maintainer(s) need to fix the PCI DMA support...
No, it's worse than that.
See how non-consistent memory is used by the eepro100 driver for descriptor bits? The skb->tail bits?
That is very problematic.
Quoted 6 lines · David S. Miller
From: Jeff Garzik <[email protected]>
Date: Wed, 12 Jun 2002 19:00:00 -0400for the 'mips' patch, it looks
like the arch maintainer(s) need to fix the PCI DMA support...
Quoted 6 lines
No, it's worse than that.
See how non-consistent memory is used by the eepro100 driver for descriptor bits? The
skb->tailbits?That is very problematic.
Oh crap, you're right... eepro100 in general does funky stuff with the way packets are handled, mainly due to the need to issue commands to the NIC engine instead of the normal per-descriptor owner bit way of doing things.
Well, I accept patches to that clean eepro100 up... I'm not terribly motivated to clean it up myself, as we have e100 and an e100 maintainer we can beat on if such uglies arise :)
Jeff
Quoted 6 lines · David S. Miller
From: Jeff Garzik <[email protected]>
Date: Wed, 12 Jun 2002 19:00:00 -0400for the 'mips' patch, it looks
like the arch maintainer(s) need to fix the PCI DMA support...
Quoted 6 lines
No, it's worse than that.
See how non-consistent memory is used by the eepro100 driver for descriptor bits? The
skb->tailbits?That is very problematic.
What's the problem?
If it isn't allowed to do, then what is the meaning of PCI_DMA_BIDIRECTIONAL mappings?
Andrey
Quoted 6 lines · David S. Miller
From: Jeff Garzik <[email protected]>
Date: Wed, 12 Jun 2002 19:00:00 -0400for the 'mips' patch, it looks
like the arch maintainer(s) need to fix the PCI DMA support...
Quoted 6 lines
No, it's worse than that.
See how non-consistent memory is used by the eepro100 driver for descriptor bits? The
skb->tailbits?That is very problematic.
Oh crap, you're right... eepro100 in general does funky stuff with the way packets are handled, mainly due to the need to issue commands to the NIC engine instead of the normal per-descriptor owner bit way of doing things.
Well, I accept patches to that clean eepro100 up... I'm not terribly motivated to clean it up myself, as we have e100 and an e100 maintainer we can beat on if such uglies arise :)
Jeff
From: Jeff Garzik <[email protected]>
Date: Wed, 12 Jun 2002 19:17:58 -0400
Oh crap, you're right... eepro100 in general does funky stuff with the
way packets are handled, mainly due to the need to issue commands to the NIC engine instead of the normal per-descriptor owner bit way of doing things.
The question is, do the descriptor bits have to live right before the RX packet data buffer or can other schemes be used?
From: Jeff Garzik <[email protected]>
Date: Wed, 12 Jun 2002 19:17:58 -0400
Oh crap, you're right... eepro100 in general does funky stuff with the
way packets are handled, mainly due to the need to issue commands to the NIC engine instead of the normal per-descriptor owner bit way of doing things.
The question is, do the descriptor bits have to live right before the RX packet data buffer or can other schemes be used?
Quoted 5 lines · David S. Miller
From: Jeff Garzik <[email protected]>
Oh crap, you're right... eepro100 in general does funky stuff with the
way packets are handled, mainly due to the need to issue commands to the NIC engine instead of the normal per-descriptor owner bit way of doing things.
The eepro100 has a unique design in many different aspects.
Quoted 2 lines
The question is, do the descriptor bits have to live right before the RX packet data buffer or can other schemes be used?
With the current driver structure, yes, the descriptor words must be immediately before the packet data. You can use other Rx and Tx structures/modes to avoid this, but they use less efficient memory access. For instance, the current Tx structure allows transmitting a packet with a single PCI burst, rather than multiple transfers.
Signature · 4 lines
Donald Becker [email protected] Scyld Computing Corporation http://www.scyld.com 410 Severn Ave. Suite 210 Second Generation Beowulf Clusters Annapolis MD 21403 410-990-9993
Quoted 5 lines · David S. Miller
From: Jeff Garzik <[email protected]>
Oh crap, you're right... eepro100 in general does funky stuff with the
way packets are handled, mainly due to the need to issue commands to the NIC engine instead of the normal per-descriptor owner bit way of doing things.
The eepro100 has a unique design in many different aspects.
Quoted 2 lines
The question is, do the descriptor bits have to live right before the RX packet data buffer or can other schemes be used?
With the current driver structure, yes, the descriptor words must be immediately before the packet data. You can use other Rx and Tx structures/modes to avoid this, but they use less efficient memory access. For instance, the current Tx structure allows transmitting a packet with a single PCI burst, rather than multiple transfers.
Signature · 4 lines
Donald Becker [email protected] Scyld Computing Corporation http://www.scyld.com 410 Severn Ave. Suite 210 Second Generation Beowulf Clusters Annapolis MD 21403 410-990-9993
Quoted 9 lines · Donald Becker
Quoted 5 lines · David S. Miller
From: Jeff Garzik <[email protected]>
Oh crap, you're right... eepro100 in general does funky stuff with the
way packets are handled, mainly due to the need to issue commands to the NIC engine instead of the normal per-descriptor owner bit way of doing things.The eepro100 has a unique design in many different aspects.
Quoted 8 lines
Quoted 2 lines
The question is, do the descriptor bits have to live right before the RX packet data buffer or can other schemes be used?
With the current driver structure, yes, the descriptor words must be immediately before the packet data. You can use other Rx and Tx structures/modes to avoid this, but they use less efficient memory access. For instance, the current Tx structure allows transmitting a packet with a single PCI burst, rather than multiple transfers.
Maybe a bit off topic, but we (at my work) are using plenty of eepro100 cards with both drivers ( e100 and eepro100 )(shipped with dell machines, and others).
We have lot of problem with these card: from link autonegociation to the really frequent cmd_timeout.
We expreienced some freezes, slowdowns, problems with copying from NFS to a Firwire disk ( systematic cmd_timeout at about 250 MB).
Do you have any advice ? should I test eepro100 NAPI driver ? I've try to play with ethtool(chang some eepro100 bits , like the "sleeping" one ...
I have quitely the same card at home wich doesn't make any problem ( I noticed some cmd_timeout when I changed my hub).
Is this hub related ? Is there a standart way autonegociation is working ( we use mostly cisco switches, are they compliant?).
We are actually trying to force 10FD or 100FD any new installed card because we think this is the best way to avoid performances problem ...
Thanks for any advice.
Sam
Quoted 7 lines · Samuel Maftoul
Maybe a bit off topic, but we (at my work) are using plenty of eepro100 cards with both drivers ( e100 and eepro100 )(shipped with dell machines, and others).
We have lot of problem with these card: from link autonegociation to the really frequentcmd_timeout.
We expreienced some freezes, slowdowns, problems with copying from NFS to a Firwire disk ( systematiccmd_timeoutat about 250 MB).
Yes, this would be better in the cosl.networking, but a quick answer since it seems kernel related. I had problems with these until recent kernels. The e100 driver helped in some cases, but other issues were reported. I don't really have any problems now, on news servers which get ~250GB/day in and push ~700 out (yes, they run 70-80Mbit all day).
This may be related to the blessing of the new scheduler.
Quoted 3 lines
Do you have any advice ? should I test eepro100 NAPI driver ? I've try to play with ethtool(chang some eepro100 bits , like the "sleeping" one ...
We noticed long ago that different blades on the same switch would work right with either auto at both ends or 100TX forced at both ends. Don;t know why, just look at ifconfig reports for collisions and if you see them change the settings.
Quoted 4 lines
I have quitely the same card at home wich doesn't make any problem ( I noticed some
cmd_timeoutwhen I changed my hub).
Is this hub related ? Is there a standart way autonegociation is working ( we use mostly cisco switches, are they compliant?).
You have said both "hub" and "switch" in this paragraph, trying to run a hub full deplex will cause problems. I have no idea what a "switching hub" means, I see them around cheap, but not from Cisco.
Quoted 2 lines
We are actually trying to force 10FD or 100FD any new installed card because we think this is the best way to avoid performances problem ...
We have never found the slightest way to predect if auto or forced 100TX works with any given blade in any given switch. But it doesn't change, so get it right and it's fixed. See above, look for collisions, try auto first. No collisions, it's as good as it gets.
Signature · 3 lines
bill davidsen <[email protected]> CTO, TMR Associates, Inc Doing interesting things with little computers since 1979.
Quoted 6 lines · David S. Miller
From: Jeff Garzik <[email protected]>
Date: Wed, 12 Jun 2002 19:00:00 -0400for the 'mips' patch, it looks
like the arch maintainer(s) need to fix the PCI DMA support...
Quoted 6 lines
No, it's worse than that.
See how non-consistent memory is used by the eepro100 driver for descriptor bits? The
skb->tailbits?That is very problematic.
What's the problem?
If it isn't allowed to do, then what is the meaning of PCI_DMA_BIDIRECTIONAL mappings?
Andrey
From: Andrey Savochkin <[email protected]>
Date: Thu, 13 Jun 2002 12:57:53 +0400
Quoted 6 lines · David S. Miller
No, it's worse than that.
See how non-consistent memory is used by the eepro100 driver for descriptor bits? The
skb->tailbits?That is very problematic.
What's the problem?
If it isn't allowed to do, then what is the meaning of PCI_DMA_BIDIRECTIONAL mappings?
It's slow. Not wrong, just inefficient.
Descriptors were meant to be done using consistent mappings, not "pci_map_*()"'d memory. The latter is meant to be used for long linear DMA transfers to/from the device. It is not meant for things the cpu pokes small bits of data in and out of, that is what consistent DMA memory is for.
From: Andrey Savochkin <[email protected]>
Date: Thu, 13 Jun 2002 12:57:53 +0400
Quoted 6 lines · David S. Miller
No, it's worse than that.
See how non-consistent memory is used by the eepro100 driver for descriptor bits? The
skb->tailbits?That is very problematic.
What's the problem?
If it isn't allowed to do, then what is the meaning of PCI_DMA_BIDIRECTIONAL mappings?
It's slow. Not wrong, just inefficient.
Descriptors were meant to be done using consistent mappings, not "pci_map_*()"'d memory. The latter is meant to be used for long linear DMA transfers to/from the device. It is not meant for things the cpu pokes small bits of data in and out of, that is what consistent DMA memory is for.