Advice needed WRT very slow nfs in new port...
I am trying to port 2.6.12 to a 32 bit SOC (ATI Xilleon). The same system is current running 2.4.29 with no problems.
I have an Intel Pro/100M Ethernet adapter on the PCI bus that I use to mount the root file system via nfs. There is also an ohci usb adapter on the bus.
My problem is that with my new 2.6.12 port the NFS is very slow. It takes about 10 minutes to mount the root filesystem, run init and finally run a shell on the serial port and present the shell prompt. Running commands like 'ls' works but can take 15 seconds in a small directory.
On the same hardware running 2.4.29 the system boots in 15 seconds and there is no delay for the 'ls'.
One theory I have is that there is a problem in either the interrupt or timer code somewhere. My evidence for this is that if I repeatedly plug and unplug a usb memory device things run more quickly. I think the interrupts from the USB may be kicking things into action.
Any pointers about where to look would be most appreciated.
Thanks
David Daney
Problems with Intel e100 driver on new MIPS port, was: Advice needed WRT very slow nfs in new port...
Quoted 11 lines · David Daney
I am trying to port 2.6.12 to a 32 bit SOC (ATI Xilleon). The same system is current running 2.4.29 with no problems.
I have an Intel Pro/100M Ethernet adapter on the PCI bus that I use to mount the root file system via nfs. There is also an ohci usb adapter on the bus.
My problem is that with my new 2.6.12 port the NFS is very slow. It takes about 10 minutes to mount the root filesystem, run init and finally run a shell on the serial port and present the shell prompt. Running commands like 'ls' works but can take 15 seconds in a small directory.
On the same hardware running 2.4.29 the system boots in 15 seconds and there is no delay for the 'ls'.
One theory I have is that there is a problem in either the interrupt or timer code somewhere. My evidence for this is that if I repeatedly plug and unplug a usb memory device things run more quickly. I think the interrupts from the USB may be kicking things into action.
Any pointers about where to look would be most appreciated.
Here is an update:
The CPU is a 32 bit MIPS 4Ke core (little endian).
My problem is only experienced with the drivers/net/e100.c driver. When I use an RTL8100 with the 8139too driver the network seems to work fine.
The Intel driver reports to be:
e100: Intel(R) PRO/100 Network Driver, 3.4.8-k2-NAPI
I am just bringing the 2.6.12 kernel up on the board so it is quite possible that I have screwed something up in the board specific portion of the kernel.
When using the Intel Pro/100M NIC the network seems to work fine in all respects except for the speed issue. If i run ping (either from the board or to the board from an external host) I get round trip times of almost exactly 1000mS.
With the same Intel NIC and the 2.4.29 kernel using the e100 driver I have no problems, so I don't think it is a hardware issue.
Does anyone have any idea what would cause 1000mS delay?
Thanks in advance for any insight,
David Daney
Re: Problems with Intel e100 driver on new MIPS port, was: Advice needed WRT very slow nfs in new port...
In message: <[email protected]>
David Daney <[email protected]> writes:
: Does anyone have any idea what would cause 1000mS delay?
That's remarkably close to 1s. This often indicates that the transmit of your next packet is causing the receive buffer to empty. This is usually due to blocked interrupts, or a failure to enable interrupts.
Warner
Re: Problems with Intel e100 driver on new MIPS port, was: Advice needed WRT very slow nfs in new port...
Quoted 8 lines · M. Warner Losh
In message: <[email protected]>
David Daney <[email protected]> writes:
: Does anyone have any idea what would cause 1000mS delay?That's remarkably close to 1s. This often indicates that the transmit of your next packet is causing the receive buffer to empty. This is usually due to blocked interrupts, or a failure to enable interrupts.
But I observe ever increasing counts for the device in /proc/interrupts.
So the interrupts are working somewhat.
David Daney
Re: Problems with Intel e100 driver on new MIPS port, was: Advice needed WRT very slow nfs in new port...
In message: <[email protected]>
David Daney <[email protected]> writes:
: M. Warner Losh wrote:
: > In message: <[email protected]>
: > David Daney <[email protected]> writes:
: > : Does anyone have any idea what would cause 1000mS delay? : >
: > That's remarkably close to 1s. This often indicates that the transmit : > of your next packet is causing the receive buffer to empty. This is : > usually due to blocked interrupts, or a failure to enable interrupts. : >
:
: But I observe ever increasing counts for the device in /proc/interrupts.
: So the interrupts are working somewhat.
Are you sure that you've routed the interrupts correctly? Maybe those interrupts are 'really' for a different device....
Warner
Re: Problems with Intel e100 driver on new MIPS port, was: Advice needed WRT very slow nfs in new port...
Quoted 11 lines · M. Warner Losh
In message: <[email protected]>
David Daney <[email protected]> writes:
: M. Warner Losh wrote:
: > In message: <[email protected]>
: > David Daney <[email protected]> writes:
: > : Does anyone have any idea what would cause 1000mS delay?
: > : > That's remarkably close to 1s. This often indicates that the transmit : > of your next packet is causing the receive buffer to empty. This is : > usually due to blocked interrupts, or a failure to enable interrupts.
: > : : But I observe ever increasing counts for the device in/proc/interrupts. : So the interrupts are working somewhat.
Quoted 4 lines
Are you sure that you've routed the interrupts correctly? Maybe those interrupts are 'really' for a different device....
Add some debugging to the interrupt routine of the e100 and see what happens.
Michael
Re: Problems with Intel e100 driver on new MIPS port, was: Advice needed WRT very slow nfs in new port...
Quoted 13 lines · Michael Stickel
Quoted 11 lines · M. Warner Losh
In message: <[email protected]>
David Daney <[email protected]> writes:
: M. Warner Losh wrote:
: > In message: <[email protected]>
: > David Daney <[email protected]> writes:
: > : Does anyone have any idea what would cause 1000mS delay?
: > : > That's remarkably close to 1s. This often indicates that the transmit : > of your next packet is causing the receive buffer to empty. This is : > usually due to blocked interrupts, or a failure to enable interrupts.
: > : : But I observe ever increasing counts for the device in/proc/interrupts. : So the interrupts are working somewhat.
Quoted 5 lines
Quoted 4 lines
Are you sure that you've routed the interrupts correctly? Maybe those interrupts are 'really' for a different device....
Add some debugging to the interrupt routine of the e100 and see what happens.
The interrupt routine is getting called each time a packet is received.
It looks like packets are not being transmitted until the interrupt for the the received packet is received.
If I ping the board at different intervals the round trip time is always almost exactly equal to the ping interval. So if I ping every 50mS the round trip time is 50mS, ping every 200mS gives a RTT of 200mS, etc.
Any more ideas?
I am thinking that perhaps the CPU write-back-queue is interfearing with writes to the NIC's registers. Perhaps I will try to disable it.
David Daney.