parisc: improve cach flushing in arch_sync_dma_for_cpu()
Helge Deller
maintainerPATCH 1/1 parisc: improve cach flushing in arch_sync_dma_for_cpu()
Handle the direction flag in arch_sync_dma_for_device() and arch_sync_dma_for_cpu().
When receiving data from the device (DMA_FROM_DEVICE and DMA_BIDIRECTIONAL) purge the data cache in arch_sync_dma_for_cpu().
Run-tested on C8000 workstation.
arch/parisc/kernel/pci-dma.c +27 -2
@@ -446,11 +446,36 @@ void arch_dma_free(struct device *dev, size_t size, void *vaddr,void arch_sync_dma_for_device (phys_addr_t paddr,size_t size,enum dma_data_direction dir) { -flush_kernel_dcache_range ((unsigned long )phys_to_virt (paddr), size); +unsigned long addr = (unsigned long )phys_to_virt (paddr); + +switch (dir) { +case DMA_TO_DEVICE : +flush_kernel_dcache_range (addr, size); +break ; +case DMA_FROM_DEVICE : +case DMA_BIDIRECTIONAL : +flush_kernel_dcache_range (addr, size); +purge_kernel_dcache_range_asm (addr, addr + size); +break ; +default : +BUG (); + } }void arch_sync_dma_for_cpu (phys_addr_t paddr,size_t size,enum dma_data_direction dir) { -flush_kernel_dcache_range ((unsigned long )phys_to_virt (paddr), size); +unsigned long addr = (unsigned long )phys_to_virt (paddr); + +switch (dir) { +case DMA_TO_DEVICE : +flush_kernel_dcache_range (addr, size); +return ; +case DMA_FROM_DEVICE : +case DMA_BIDIRECTIONAL : +purge_kernel_dcache_range_asm (addr, addr + size); +break ; +default : +BUG (); + } }
arch/parisc/kernel/pci-dma.c void arch_dma_free(struct device *dev, size_t size, void *vaddr, · line 458
Quoted 3 lines · Helge Deller
arch/parisc/kernel/pci-dma.c +3
-0+case DMA_BIDIRECTIONAL : +flush_kernel_dcache_range (addr, size); +purge_kernel_dcache_range_asm (addr, addr + size);
I don't think flush and purge are both needed.
Dave
Signature · 1 line
John David Anglin [email protected]
Helge Deller
maintainerQuoted 5 lines · John David Anglin
Quoted 3 lines · Helge Deller
+ case
DMA_BIDIRECTIONAL:
+flush_kernel_dcache_range(addr, size);
+purge_kernel_dcache_range_asm(addr, addr + size);I don't think flush and purge are both needed.
I'm not sure...
Just to fully understand it. Is this short summary correct: ?
- flush_kernel_dcache_range: flush cache back to memory, but keep data in cache.
Next read fetches the data which is still in the cache, thus the next
read doesn't checks if data in memory has been modified in the meantime (e.g. via DMA).
- purge_kernel_dcache_range_asm: ignore currently cached data & drop any cached data in that range. Even if cache has dirty memory which hasn't been written back yet, drop it and don't write back. Next read will fetch data from memory, thus return what DMA could have stored there.
Helge
Quoted 11 lines · Helge Deller
Quoted 5 lines · John David Anglin
Quoted 3 lines · Helge Deller
+ case
DMA_BIDIRECTIONAL:
+flush_kernel_dcache_range(addr, size);
+purge_kernel_dcache_range_asm(addr, addr + size);I don't think flush and purge are both needed.
I'm not sure...
Just to fully understand it. Is this short summary correct: ?
-flush_kernel_dcache_range: flush cache back to memory, but keep data in cache.
No. If present, fdc writes addressed cache line to memory if and only if line is dirty. It then invalidates line. It does not keep data in cache.
Next read loads from memory.
Quoted 4 lines
Next read fetches the data which is still in the cache, thus the next
read doesn't checks if data in memory has been modified in the meantime (e.g. via DMA).
-purge_kernel_dcache_range_asm: ignore currently cached data & drop any cached data in that range. Even if cache has dirty memory which hasn't been written back yet, drop it and don't write back.
if present, pdc invalidates line. At privilege level zero, an implementation may optionally write back a dirty line to memory. At non-zero privilege levels, fdc and pdc are effectively the same.
I don't know how to determine whether pdc does write back or not. It would be specified in processor ERS.
Quoted 1 line
Next read will fetch data from memory, thus return what DMA could have stored there.
Dave
Signature · 1 line
John David Anglin [email protected]
Helge Deller
maintainer* John David Anglin <[email protected]>:
Quoted 20 lines · Helge Deller
Quoted 11 lines · John David Anglin
Quoted 5 lines · Helge Deller
Quoted 3 lines
+ case
DMA_BIDIRECTIONAL:
+flush_kernel_dcache_range(addr, size);
+purge_kernel_dcache_range_asm(addr, addr + size);I don't think flush and purge are both needed.
I'm not sure...
Just to fully understand it. Is this short summary correct: ?
-flush_kernel_dcache_range: flush cache back to memory, but keep data in cache.No. If present, fdc writes addressed cache line to memory if and only if line is dirty. It then invalidates line. It does not keep data in cache.
Next read loads from memory.
Quoted 4 lines
Next read fetches the data which is still in the cache, thus the next
read doesn't checks if data in memory has been modified in the meantime (e.g. via DMA).
-purge_kernel_dcache_range_asm: ignore currently cached data & drop any cached data in that range. Even if cache has dirty memory which hasn't been written back yet, drop it and don't write back.
Quoted 5 lines
if present, pdc invalidates line. At privilege level zero, an implementation may optionally write back a dirty line to memory. At non-zero privilege levels, fdc and pdc are effectively the same.
I don't know how to determine whether pdc does write back or not. It would be specified in processor ERS.
Thanks for the explanation!
With that, I've attached an updated (untested) patch below.
Helge
[PATCH v2] parisc: improve cach flushing in arch_sync_dma_for_cpu()
Add comment in arch_sync_dma_for_device() and handle the direction flag in arch_sync_dma_for_cpu().
When receiving data from the device (DMA_FROM_DEVICE) unconditionally purge the data cache in arch_sync_dma_for_cpu().
arch/parisc/kernel/pci-dma.c +17 -1
@@ -446,11 +446,27 @@ void arch_dma_free(struct device *dev, size_t size, void *vaddr,void arch_sync_dma_for_device (phys_addr_t paddr,size_t size,enum dma_data_direction dir) { +/* +* fdc: The data cache line is written back to memory, if and only if +* it is dirty, and then invalidated from the data cache. +*/ flush_kernel_dcache_range ((unsigned long )phys_to_virt (paddr), size); }void arch_sync_dma_for_cpu (phys_addr_t paddr,size_t size,enum dma_data_direction dir) { -flush_kernel_dcache_range ((unsigned long )phys_to_virt (paddr), size); +unsigned long addr = (unsigned long )phys_to_virt (paddr); + +switch (dir) { +case DMA_TO_DEVICE : +case DMA_BIDIRECTIONAL : +flush_kernel_dcache_range (addr, size); +return ; +case DMA_FROM_DEVICE : +purge_kernel_dcache_range_asm (addr, addr + size); +return ; +default : +BUG (); + } }
Quoted 20 lines · Helge Deller
* John David Anglin <[email protected]>:
Quoted 19 lines · Helge Deller
Quoted 10 lines · John David Anglin
Quoted 5 lines · Helge Deller
Quoted 3 lines
+ case
DMA_BIDIRECTIONAL:
+flush_kernel_dcache_range(addr, size);
+purge_kernel_dcache_range_asm(addr, addr + size);I don't think flush and purge are both needed.
I'm not sure...
Just to fully understand it. Is this short summary correct: ?
-flush_kernel_dcache_range: flush cache back to memory, but keep data in cache.No. If present, fdc writes addressed cache line to memory if and only if line is dirty. It then invalidates line. It does not keep data in cache.
Next read loads from memory.
Quoted 4 lines
Next read fetches the data which is still in the cache, thus the next
read doesn't checks if data in memory has been modified in the meantime (e.g. via DMA).
-purge_kernel_dcache_range_asm: ignore currently cached data & drop any cached data in that range. Even if cache has dirty memory which hasn't been written back yet, drop it and don't write back.
Quoted 7 lines
Quoted 5 lines
if present, pdc invalidates line. At privilege level zero, an implementation may optionally write back a dirty line to memory. At non-zero privilege levels, fdc and pdc are effectively the same.
I don't know how to determine whether pdc does write back or not. It would be specified in processor ERS.
Thanks for the explanation!
With that, I've attached an updated (untested) patch below.
Seems to work okay on c8000. Don't know if it helps performance.
Dave
Quoted 42 lines
[PATCH v2] parisc: improve cach flushing in
arch_sync_dma_for_cpu()Add comment in
arch_sync_dma_for_device()and handle the direction flag inarch_sync_dma_for_cpu().When receiving data from the device (
DMA_FROM_DEVICE) unconditionally purge the data cache inarch_sync_dma_for_cpu().Signed-off-by Helge Deller <[email protected]>arch/parisc/kernel/pci-dma.c +17
-1@@ -446,11 +446,27 @@ void arch_dma_free(struct device *dev, size_t size, void *vaddr,void arch_sync_dma_for_device (phys_addr_t paddr,size_t size,enum dma_data_direction dir) { +/* +* fdc: The data cache line is written back to memory, if and only if +* it is dirty, and then invalidated from the data cache. +*/ flush_kernel_dcache_range ((unsigned long )phys_to_virt (paddr), size); }void arch_sync_dma_for_cpu (phys_addr_t paddr,size_t size,enum dma_data_direction dir) { -flush_kernel_dcache_range ((unsigned long )phys_to_virt (paddr), size); +unsigned long addr = (unsigned long )phys_to_virt (paddr); + +switch (dir) { +case DMA_TO_DEVICE : +case DMA_BIDIRECTIONAL : +flush_kernel_dcache_range (addr, size); +return ; +case DMA_FROM_DEVICE : +purge_kernel_dcache_range_asm (addr, addr + size); +return ; +default : +BUG (); + } }
Signature · 1 line
John David Anglin [email protected]
Helge Deller
maintainerQuoted 24 lines · John David Anglin
Quoted 23 lines · Helge Deller
* John David Anglin <[email protected]>:
Quoted 22 lines · Helge Deller
Quoted 10 lines · John David Anglin
Quoted 5 lines · Helge Deller
Quoted 3 lines
+ case
DMA_BIDIRECTIONAL:
+flush_kernel_dcache_range(addr, size);
+purge_kernel_dcache_range_asm(addr, addr + size);I don't think flush and purge are both needed.
I'm not sure...
Just to fully understand it. Is this short summary correct: ?
-flush_kernel_dcache_range: flush cache back to memory, but keep data in cache.No. If present, fdc writes addressed cache line to memory if and only if line is dirty. It then invalidates line. It does not keep data in cache.
Next read loads from memory.
Quoted 4 lines
Next read fetches the data which is still in the cache, thus the next
read doesn't checks if data in memory has been modified in the meantime (e.g. via DMA).
-purge_kernel_dcache_range_asm: ignore currently cached data & drop any cached data in that range. Even if cache has dirty memory which hasn't been written back yet, drop it and don't write back.if present, pdc invalidates line. At privilege level zero, an implementation may optionally write back a dirty line to memory. At non-zero privilege levels, fdc and pdc are effectively the same.
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
I don't know how to determine whether pdc does write back or not. It would be specified in processor ERS.
Thanks for the explanation!
With that, I've attached an updated (untested) patch below.Seems to work okay on c8000. Don't know if it helps performance.
Probably no performance change for c8000.
I noticed that pci-dma.c is only compiled on 32-bit kernels (for PCX-T machines) :-)
Helge