New API for non cache coherent ppc cpu's
To all NOT_COHERENT_CACHE users
About a month ago a new API made its way into the ppc, consistent_sync_page. For CONFIG_NOT_COHERENT_CACHE processors, requires
proper flushing of the page being used. Please review and provide feed back
-- armin
sample from patch
void consistent_sync_page(struct page *page, unsigned long offset, size_t size, int direction)
{
unsigned long start = (unsigned long)page->virtual;
unsigned long end = start + size;
switch (direction) {
case PCI_DMA_NONE:
BUG();
case PCI_DMA_FROMDEVICE: /* invalidate only */
invalidate_dcache_range(start, end);
break;
case PCI_DMA_TODEVICE: /* writeback only */
clean_dcache_range(start, end);
break;
case PCI_DMA_BIDIRECTIONAL: /* writeback and invalidate */
flush_dcache_range(start, end);
break;
}}
Hi,
Quoted 2 lines · Armin Kuster
About a month ago a new API made its way into the ppc,
consistent_sync_page.
Another one?
We have now 3 APIs for this:
- cache_(push|clear): that's the old m68k API (and also used by APUS)
- dma_cache_(inv|wback|wback_inv): used by mips(64), parisc, sh
- consistent_sync_page: ppc
Quoted 4 lines
For
CONFIG_NOT_COHERENT_CACHEprocessors,
requires
proper flushing of the page being used. Please review and provide feed back
Two comments:
- I'm not sure about the page argument, although I'd like to see it, the problem is the drivers usually don't have a page pointer.
- I think it was never defined, what should be done if offset/size isn't cache line aligned. That's especially a problem in the invalidate only case. I'd prefer to make this illegal, as it's mostly not a problem for drivers.
bye, Roman
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Quoted 14 lines · Roman Zippel
Hi,
Quoted 2 lines · Armin Kuster
About a month ago a new API made its way into the ppc,
consistent_sync_page.Another one?
We have now 3 APIs for this:
-cache_(push|clear): that's the old m68k API (and also used by APUS)
-dma_cache_(inv|wback|wback_inv): used by mips(64), parisc, sh
-consistent_sync_page: ppc
Quoted 8 lines
Quoted 4 lines
For
CONFIG_NOT_COHERENT_CACHEprocessors,
requires
proper flushing of the page being used. Please review and provide feed backTwo comments:
- I'm not sure about the page argument, although I'd like to see it, the problem is the drivers usually don't have a page pointer.
example as it is used today.
mapping = pci_map_page(pdev,
virt_to_page(cmd->request_buffer), ((unsigned long)cmd->request_buffer & ~PAGE_MASK), cmd->request_bufflen, dma_dir);
/*
* pci_{map,unmap}_single_page maps a kernel page to a dma_addr_t.
identical
* to pci_map_single, but takes a struct page instead of a virtual
address
*/
static inline dma_addr_t pci_map_page(struct pci_dev *hwdev, struct page *page,
unsigned long offset, size_t size, int direction)
{
if (direction == PCI_DMA_NONE)
BUG();
consistent_sync_page(page, offset, size, direction);
return (page - mem_map) * PAGE_SIZE + PCI_DRAM_OFFSET + offset;
}Quoted 5 lines
- I think it was never defined, what should be done if offset/size isn't cache line aligned. That's especially a problem in the invalidate only case. I'd prefer to make this illegal, as it's mostly not a problem for drivers.
wouldn't it also be a problem for consistent_sync? we don't check for offset/size are cache line aligned.
Quoted 1 line
bye, Roman
-- armin
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Hi,
Quoted 2 lines · Armin Kuster
mapping = pci_map_page(pdev,
virt_to_page(cmd->request_buffer),
This is the problem, virt_to_page doesn't work kmap'ed highmem pages.
Quoted 7 lines
Quoted 4 lines
- I think it was never defined, what should be done if offset/size isn't cache line aligned. That's especially a problem in the invalidate only case. I'd prefer to make this illegal, as it's mostly not a problem for drivers.
wouldn't it also be a problem for
consistent_sync? we don't check for offset/size are cache line aligned.
Yes, same problem.
bye, Roman
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Quoted 7 lines · Armin Kuster
To all
NOT_COHERENT_CACHEusersAbout a month ago a new API made its way into the ppc,
consistent_sync_page. ForCONFIG_NOT_COHERENT_CACHEprocessors, requires
proper flushing of the page being used. Please review and provide feed back
If we have to have a consistent_sync_page, it should be purely a local function in our implementation of the official DMA mapping API - see Documentation/DMA-mapping.txt. Drivers should be using functions such as pci_alloc_consistent, pci_map_single, pci_dma_sync_single, pci_unmap_single, etc. The implementation of those routines should do the correct cache flushing - if it doesn't then we need to fix it.
If you're talking about non-PCI devices, use the pci DMA API but just pass NULL for the dev (we need to make sure that will work ok on the non-cache-coherent cpus).
Paul.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Quoted 3 lines · Paul Mackerras
If we have to have a
consistent_sync_page, it should be purely a local function in our implementation of the official DMA mapping API - seeDocumentation/DMA-mapping.txt.
That's exactly what it is. We are not creating a new API, and it shouldn't have been described as such.
Quoted 4 lines
.... Drivers should be using functions such
aspci_alloc_consistent,pci_map_single,pci_dma_sync_single,pci_unmap_single, etc. The implementation of those routines should do the correct cache flushing - if it doesn't then we need to fix it.
They do. Recently a pci_sync_page was added for some reason (because Linux doesn't have a real VM implementation I guess :-). We just needed to add the underlying function to support it. I think it popped up in one of the SCSI drivers.
Quoted 3 lines
If you're talking about non-PCI devices, use the pci DMA API but just pass NULL for the dev (we need to make sure that will work ok on the non-cache-coherent cpus).
Ummm....I don't normally do this because using these functions requires the CONFIG_PCI option, which isn't supported or has bad effects when requested on processors that don't have PCI. For non-PCI devices, I call the consistent_* functions, which other architectures support and do as well. So, we kind of have a common interface here for non-PCI devices as well.
I stole all of this cache coherency stuff from ARM and Sparc (or MIPS, I don't remember) when I did it the first time.
Thanks.
-- Dan
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Hi,
Quoted 6 lines · Paul Mackerras
If we have to have a
consistent_sync_page, it should be purely a local function in our implementation of the official DMA mapping API - seeDocumentation/DMA-mapping.txt. Drivers should be using functions such aspci_alloc_consistent,pci_map_single,pci_dma_sync_single,pci_unmap_single, etc. The implementation of those routines should do the correct cache flushing - if it doesn't then we need to fix it.
This document only describes DMA _mappings_, it doesn't say anything about cache coherency.
Quoted 3 lines
If you're talking about non-PCI devices, use the pci DMA API but just pass NULL for the dev (we need to make sure that will work ok on the non-cache-coherent cpus).
That's the other problem, "non-PCI" sounds like ISA there, what about other buses?
bye, Roman
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Hi,
Quoted 7 lines · Dan Malek
For non-PCI devices,
I call the consistent_* functions, which other architectures support and do as well. So, we kind of have a common interface here for non-PCI devices as well.I stole all of this cache coherency stuff from ARM and Sparc (or MIPS, I don't remember) when I did it the first time.
The only other architecture which defines it is ARM?! The only API which exists at least as dummys everywhere is dma_cache_(inv|wback).
bye, Roman
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Quoted 2 lines · Roman Zippel
This document only describes DMA _mappings_, it doesn't say anything about cache coherency.
It talks about the conditions under which the cpu and the DMA device will see a consistent view of memory. The section on the streaming DMA mappings could be more explicit, but the idea is that between the pci_map_{single,sg} and the corresponding pci_unmap_*, the device owns the region and the cpu shouldn't touch it without first calling pci_dma_sync_*. (Hmmm, there is possibly a hole here - the API should maybe require that you call a sync function both before and after touching the memory.) At other times the cpu owns the region and should see a consistent view of that memory. That implies some cache flushing in pci_map_* and/or pci_unmap_* on non-cache-coherent systems.
Quoted 2 lines
That's the other problem, "non-PCI" sounds like ISA there, what about other buses?
That is implementation-dependent. My view is that we should make this API work for every kind of bus we have, if possible.
Note also that in 2.5 the struct pci_dev is going to transmogrify into a "struct device" that will be used for I/O devices on any kind of bus.
Paul.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Quoted 2 lines · Dan Malek
That's exactly what it is. We are not creating a new API, and it shouldn't have been described as such.
OK... Armin's message wasn't exactly clear... :)
Quoted 1 line
They do. Recently a
pci_sync_pagewas added for some reason (because
To go with pci_map_page and pci_unmap_page. These give you the ability to do DMA to highmem pages without using bounce buffers.
Quoted 10 lines
Quoted 3 lines
If you're talking about non-PCI devices, use the pci DMA API but just pass NULL for the dev (we need to make sure that will work ok on the non-cache-coherent cpus).
Ummm....I don't normally do this because using these functions requires the
CONFIG_PCIoption, which isn't supported or has bad effects when requested on processors that don't have PCI. For non-PCI devices, I call the consistent_* functions, which other architectures support and do as well. So, we kind of have a common interface here for non-PCI devices as well.
My preference would be to make the one set of functions work everywhere. OTOH these non-PCI devices you're talking about are presumably very specific to PPC embedded chips so maybe it doesn't matter so much.
Paul.
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/
Quoted 2 lines · Paul Mackerras
To go with
pci_map_pageandpci_unmap_page. These give you the ability to do DMA to highmem pages without using bounce buffers.
IIRC, there was a virt_to_bus in the code at one time, making it unusable for highmem pages, which is why I couldn't understand the appearance of the function.
Quoted 2 lines
My preference would be to make the one set of functions work everywhere.
Mine, too. I just wish this was a common concept across a variety of our I/O interfaces.
Quoted 3 lines
.... OTOH these non-PCI devices you're talking about are presumably very specific to PPC embedded chips so maybe it doesn't matter so much.
That is always a challenge with the embedded parts. It is often very convenient to make the (promper, IMHO) assumption this software isn't going to be used outside of these parts.
Thanks.
-- Dan
** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/