Question concerning cache coherency
I have an interesting issue that I would like to run past the MIPS/Linux newsgroup. I am
currently porting the MIPS/Linux code to a development board that has a IDT64475 MIPS
core (64-bit R4xxx core). I notice that this part does not have any method of maintaining
cache coherency (i.e., no hardware support for cache coherency). It is highly likely that we
will be plugging in a network card on a PCI bus that would be DMA'ing to a shared memory
space in SDRAM. I assume that the problem of cache coherency is fixed by mapping the shared
memory as uncached. I have not dug into the network drivers (or the kernel) enough to know whether
this is how the problem is addressed on typical MIPS architectures. I guess I have two questions
related to this issue; Do devices that DMA, typically access uncached memory and if so, is a second buffer
required to copy from kernel to user space? The second question is concerning the performance hit in
running out of uncached memory, Have people seen significant performance degradation when
using uncached memory. Any insight that anybody can provide would be greatly appreciated.
Thanks,
Jeff
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Jeff Harrell Work: (801) 619-6104
Broadband Access group/TI
[email protected]
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Quoted 8 lines · Jeff Harrell
I have an interesting issue that I would like to run past the MIPS/Linux newsgroup. I am currently porting the MIPS/Linux code to a development board that has a IDT64475 MIPS core (64-bit R4xxx core). I notice that this part does not have any method of maintaining cache coherency (i.e., no hardware support for cache coherency). It is highly likely that we will be plugging in a network card on a PCI bus that would be DMA'ing to a shared memory space in SDRAM. I assume that the problem of cache coherency is fixed by mapping the shared memory as uncached. I have not
Quoted 8 lines
dug into the network drivers (or the kernel) enough to know whether this is how the problem is addressed on typical MIPS architectures. I guess I have two questions related to this issue; Do devices that DMA, typically access uncached memory and if so, is a second buffer required to copy from kernel to user space? The second question is concerning the performance hit in running out of uncached memory, Have people seen significant performance degradation when using uncached memory. Any insight that anybody can provide would be greatly appreciated.
The performance hit by using uncached memory is tremenduous. Avoid it, if you can. Even if you cannot exploit the locality effects of caches you will still gain from cached access because of prefetch / burst access and write gathering.
The is one special case where you can not use caching, that is a cacheline worth of data might concurrently be manipulated both by both processor and a DMA device. The typical example are processors with 32-byte cache lines like the R4000 and a Ethernet chip like the Sonic which has ring entries of only 16 byte size. For such a configuration there is a case where
1) processor fetches cacheline
2) NIC write to that cacheline
3) processor writes cacheline back
-> the processor just corrupted the NIC written data.
The only way you can deal with that is by either stopping the NIC which you don't want to or by using uncached access.
Take a look at the bottom of <asm/io.h> which defines three functions which do the cache flushing for you. On machines that are cache coherent by hardware like SGI's Origins these functions will simply be no-ops.
Ralf