NFS root with cache on
Is anyone else having trouble with NFS root on the 2.4.0 kernel? It won't come up with the KSEG0 cache on unless I pepper the network driver with flush calls.
I'm using the new mips32 mm and cache code with my own patches to deal with the IDT 334's weird way bit (it's clear up at bit 12 even though the cache is only 2k).
Quinn
Quoted 4 lines · Quinn Jensen
Is anyone else having trouble with NFS root on
the 2.4.0 kernel? It won't come up with the
KSEG0 cache on unless I pepper the network driver
with flush calls.
That's expected for most old network drivers that don't yet use the new PCI DMA API documented in Documentation/DMA-mapping.txt.
What driver is this?
Ralf
Quoted 11 lines
Quoted 5 lines
Is anyone else having trouble with NFS root on
the 2.4.0 kernel? It won't come up with the
KSEG0 cache on unless I pepper the network driver
with flush calls.That's expected for most old network drivers that don't yet use the new PCI DMA API documented in
Documentation/DMA-mapping.txt.What driver is this?
Both the stock 2.4.0 tulip and eepro100 drivers. The problem doesn't happen when I go back to 2.3.99pre8.
In the tulip driver, both the rx and tx ring descriptors are adjusted with pci_alloc_consistent() and are only touched through KSEG1.
But I must be totally clueless because in both kernels tulip_rx() calls pci_unmap_single() which does nothing. But the skb data pointers all point to KSEG0. With cache on, how in the world will the kernel be able to see what just got DMA'd into the skb?
Another thing that has been haunting me is that
in 2.3.99pre8, kmalloc() has and #ifdef __mips__ that flushes the cache and bumps the address up to KSEG1. This is gone in 2.4.0, but from what I can tell, this case didn't happen for skb allocations (i.e. dev_alloc_skb) because they only set GFP_ATOMIC, and not GFP_DMA.
Quinn
Quoted 3 lines
Another thing that has been haunting me is that in 2.3.99pre8,
kmalloc()has and #ifdef__mips__that flushes the cache and bumps the address up to KSEG1.
That's really hideous! When did that happen?
It'll sure help cache coherency problems, but
the performance impact must have been monstrous.
Kevin K.
Quoted 7 lines · Quinn Jensen
Quoted 7 lines
Quoted 5 lines
Is anyone else having trouble with NFS root on
the 2.4.0 kernel? It won't come up with the
KSEG0 cache on unless I pepper the network driver
with flush calls.
Quoted 8 lines
Quoted 4 lines
That's expected for most old network drivers that don't yet use the new PCI DMA API documented in
Documentation/DMA-mapping.txt.What driver is this?
Both the stock 2.4.0 tulip and eepro100 drivers. The problem doesn't happen when I go back to 2.3.99pre8.
Quinn,
Did you set rx_copybreak to 1518? I sent patches long time ago to the driver authors for MIPS, but I am not sure they are not there.
Jun
Quoted 9 lines · [email protected] wrote:
Quoted 7 lines · Quinn Jensen
Quoted 7 lines
Quoted 4 lines
Is anyone else having trouble with NFS root on
the 2.4.0 kernel? It won't come up with the
KSEG0 cache on unless I pepper the network driver
with flush calls.That's expected for most old network drivers that don't yet use tye
he
Quoted 10 lines
Quoted 7 lines
Quoted 3 lines
new PCI DMA API documented in
Documentation/DMA-mapping.txt.What driver is this?
Both the stock 2.4.0 tulip and eepro100 drivers. The problem doesn't happen when I go back to 2.3.99pre8.
Did you set
rx_copybreakto 1518? I sent patches long time ago to the driver authors for MIPS, but I am not sure they are not there.
Jun,
I have tried that in this case but it didn't help, because the receive skb data pointers all point to the KSEG0 view of the data anyway. But the maddening thing is, when I force them to KSEG1 I still see the NFS timeout problem. So I'm starting to wonder if the problem is higher up in the stack. But I do know that the behavior does not occur on a similarly configured x86 build of the same kernel.
Quinn
Quoted 21 lines · Quinn Jensen
Quoted 9 lines · [email protected] wrote:
Quoted 7 lines · Quinn Jensen
Quoted 7 lines
Quoted 4 lines
Is anyone else having trouble with NFS root on
the 2.4.0 kernel? It won't come up with the
KSEG0 cache on unless I pepper the network driver
with flush calls.That's expected for most old network drivers that don't yet use tye
he
Quoted 8 lines
Quoted 7 lines
Quoted 3 lines
new PCI DMA API documented in
Documentation/DMA-mapping.txt.What driver is this?
Both the stock 2.4.0 tulip and eepro100 drivers. The problem doesn't happen when I go back to 2.3.99pre8.
Quoted 8 lines
Quoted 2 lines
Did you set
rx_copybreakto 1518? I sent patches long time ago to the driver authors for MIPS, but I am not sure they are not there.Jun,
I have tried that in this case but it didn't help, because the receive skb data pointers all point to the KSEG0 view of the data anyway.
I looked into similar problems a while back. If I remeber correctly, the data pointers do point to kseg0. It is up to the driver to do appropriate dma_cache_invalidate() (or some functions to that effect) at certain places.
What is the CPU? It seems logical to suspect about the dma cache routines.
Jun
Quoted 9 lines · [email protected] wrote:
Quoted 4 lines
I have tried that in this case but it didn't help, because the receive skb data pointers all point to the KSEG0 view of the data anyway.
I looked into similar problems a while back. If I remeber correctly, the data pointers do point to kseg0. It is up to the driver to do appropriate
dma_cache_invalidate()(or some functions to that effect) at certain places.
Yes, the tulip driver calls pci_unmap_single() on the receive buffer, but for mips (in asm-mips/pci.h) this call does nothing. And this is what is so confusing. Only if the receive buffer was forced to be in KSEG1 would this make sense.
Quoted 2 lines
What is the CPU? It seems logical to suspect about the dma cache routines.
Yes, I have scrubbed over my patch to the cache routines many times, especially since on the IDT 334 the cache-way selection for indexed cache ops is weird--they left the way-bit up at bit 12 as if it were an 8KB cache, when in reality it is only a 2KB cache.
Quinn
Quoted 2 lines · Jun Sun
Did you set
rx_copybreakto 1518? I sent patches long time ago to the driver authors for MIPS, but I am not sure they are not there.
Copybreak is just an optimization. So even with this unused or set to a wrong value the driver should work.
Ralf
netbooting indy - update, elf2ecoff?
Thank you very much for your help so far! Here's an update and another question:
Updates:
1. Had to attach a console (we have XZ graphics not XL)
2. An ecoff kernel image downloaded fine. The ELF ones download halfway and fail silently (even with the console attached)
Question: As the vmlinux-indy-2.2.1-990226.ecoff kernel begins the boot sequence, it hangs at "unable to open initial console." I had unpacked the filesystem on linux as the FAQ suggests, and my device files look right. So, on the NFS server, I symlinked /dev/console to /dev/ttyS0. Now the message is replaced with a cryptic: "S<<<<<"
Perhaps that kernel doesn't have console support? Is there any other explanation for the "S<<<<<" failure where I used to see "...initial console?"
I'd like to try a different kernel but I can't find an elf2ecoff binary. Does anyone have this converted or a known-good ecoff kernel?
Ken
Re: netbooting indy - update, elf2ecoff?
Quoted 5 lines · Kenneth C Barr
Question: As the vmlinux-indy-2.2.1-990226.ecoff kernel begins the boot sequence, it hangs at "unable to open initial console." I had unpacked the filesystem on linux as the FAQ suggests, and my device files look right. So, on the NFS server, I symlinked
/dev/consoleto/dev/ttyS0. Now the message is replaced with a cryptic: "S<<<<<"
However, /dev/console must _not_ be a symlink, but a character special device with major 5 minor 1 (cfr. Documentation/devices.txt).
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven ------------- Sony Software Development Center Europe (SDCE) [email protected] ------------------- Sint-Stevens-Woluwestraat 55 Voice +32-2-7248626 Fax +32-2-7262686 ---------------- B-1130 Brussels, Belgium
Re: netbooting indy - update, elf2ecoff?
Quoted 2 lines · Geert Uytterhoeven
However,
/dev/consolemust _not_ be a symlink, but a character special device with major 5 minor 1 (cfr.Documentation/devices.txt).
Unless you're using Linux/MIPS :)
The console implementation is horrible broken, so
this means that under most circumstances the 5,1 device doesn't work. So the only option is to make /dev/console a symlink to /dev/ttyS0.
Cya, Klaus
Signature · 4 lines
Full Name : Klaus Naumann | (http://www.mgnet.de/) (Germany) Nickname : Spock | Org.: Mad Guys Network Phone / FAX : ++49/177/7862964 | E-Mail: ([email protected]) PGP Key : www.mgnet.de/keys/key_spock.txt
Re: netbooting indy - update, elf2ecoff?
Quoted 9 lines · Klaus Naumann
Quoted 2 lines · Geert Uytterhoeven
However,
/dev/consolemust _not_ be a symlink, but a character special device with major 5 minor 1 (cfr.Documentation/devices.txt).Unless you're using Linux/MIPS :)
The console implementation is horrible broken, so
this means that under most circumstances the 5,1 device doesn't work. So the only option is to make/dev/consolea symlink to/dev/ttyS0.
This has nothing to do with "Linux/Mips" and is definitly not true on all subarchs. It might be true on SGI machines but i dont even there its necessary.
Flo
Signature · 2 lines
Florian Lohoff [email protected] +49-5201-669912 Why is it called "common sense" when nobody seems to have any?
Quoted 7 lines · Jun Sun
Quoted 3 lines
Did you set
rx_copybreakto 1518? I sent patches long time ago to the driver authors for MIPS, but I am not sure they are not there.Copybreak is just an optimization. So even with this unused or set to a wrong value the driver should work.
If you set it wrongly you can get network failures/stalls. The normal cases are not a problem however. You need to copybreak small frames to avoid filling the socket queue with empty spaces. Some platforms set it to 1518 to force a copy thereby aligning the IP header on a 4 byte boundary, which most PCI bus master drivers wont do if you are not copying