Skip to contentSkip to conversation

parisc: improve cach flushing in arch_sync_dma_for_cpu()

PATCH In review Helge Deller 2 participants 7 messages May 15, 2023 – May 20, 2023 PA-RISC architecture

lore
Share & tools
lore

Helge Deller

author maintainer
#0 lore raw

PATCH 1/1 parisc: improve cach flushing in arch_sync_dma_for_cpu()

Handle the direction flag in arch_sync_dma_for_device() and arch_sync_dma_for_cpu().

When receiving data from the device (DMA_FROM_DEVICE and DMA_BIDIRECTIONAL) purge the data cache in arch_sync_dma_for_cpu().

Run-tested on C8000 workstation.

arch/parisc/kernel/pci-dma.c +27 -2
index ba87f791323b..f8337c1820fc 100644
@@ -446,11 +446,36 @@ void arch_dma_free(struct device *dev, size_t size, void *vaddr,
 void arch_sync_dma_for_device(phys_addr_t paddr, size_t size,
 		enum dma_data_direction dir)
 {
-	flush_kernel_dcache_range((unsigned long)phys_to_virt(paddr), size);
+	unsigned long addr = (unsigned long) phys_to_virt(paddr);
+
+	switch (dir) {
+	case DMA_TO_DEVICE:
+		flush_kernel_dcache_range(addr, size);
+		break;
+	case DMA_FROM_DEVICE:
+	case DMA_BIDIRECTIONAL:
+		flush_kernel_dcache_range(addr, size);
+		purge_kernel_dcache_range_asm(addr, addr + size);
+		break;
+	default:
+		BUG();
+	}
 }
 
 void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size,
 		enum dma_data_direction dir)
 {
-	flush_kernel_dcache_range((unsigned long)phys_to_virt(paddr), size);
+	unsigned long addr = (unsigned long) phys_to_virt(paddr);
+
+	switch (dir) {
+	case DMA_TO_DEVICE:
+		flush_kernel_dcache_range(addr, size);
+		return;
+	case DMA_FROM_DEVICE:
+	case DMA_BIDIRECTIONAL:
+		purge_kernel_dcache_range_asm(addr, addr + size);
+		break;
+	default:
+		BUG();
+	}
 }
#1 lore raw

arch/parisc/kernel/pci-dma.c void arch_dma_free(struct device *dev, size_t size, void *vaddr, · line 458

Quoted 3 lines · Helge Deller
arch/parisc/kernel/pci-dma.c +3 -0
+	case DMA_BIDIRECTIONAL:
+		flush_kernel_dcache_range(addr, size);
+		purge_kernel_dcache_range_asm(addr, addr + size);

I don't think flush and purge are both needed.

Dave

Signature · 1 line
John David Anglin  [email protected]
author maintainer
#2 lore raw
Quoted 5 lines · John David Anglin
Quoted 3 lines · Helge Deller

+    case DMA_BIDIRECTIONAL:
+        flush_kernel_dcache_range(addr, size);
+        purge_kernel_dcache_range_asm(addr, addr + size);

I don't think flush and purge are both needed.

I'm not sure...

Just to fully understand it. Is this short summary correct: ?
- flush_kernel_dcache_range: flush cache back to memory, but keep data in cache.
Next read fetches the data which is still in the cache, thus the next
read doesn't checks if data in memory has been modified in the meantime (e.g. via DMA).
- purge_kernel_dcache_range_asm: ignore currently cached data & drop any cached data in that range. Even if cache has dirty memory which hasn't been written back yet, drop it and don't write back. Next read will fetch data from memory, thus return what DMA could have stored there.

Helge

#3 lore raw
Quoted 11 lines · Helge Deller
Quoted 5 lines · John David Anglin
Quoted 3 lines · Helge Deller

+    case DMA_BIDIRECTIONAL:
+        flush_kernel_dcache_range(addr, size);
+        purge_kernel_dcache_range_asm(addr, addr + size);

I don't think flush and purge are both needed.

I'm not sure...

Just to fully understand it. Is this short summary correct: ?
- flush_kernel_dcache_range: flush cache back to memory, but keep data in cache.

No.  If present, fdc writes addressed cache line to memory if and only if line is dirty.  It then invalidates line.  It does not keep data in cache.

Next read loads from memory.

Quoted 4 lines

    Next read fetches the data which is still in the cache, thus the next
    read doesn't checks if data in memory has been modified in the meantime (e.g. via DMA).
- purge_kernel_dcache_range_asm: ignore currently cached data & drop any cached data in that range. Even if cache has dirty memory which hasn't been written back yet, drop it and don't write back.

if present, pdc invalidates line.  At privilege level zero, an implementation may optionally write back a dirty line to memory.  At non-zero privilege levels, fdc and pdc are effectively the same.

I don't know how to determine whether pdc does write back or not. It would be specified in processor ERS.

Quoted 1 line

    Next read will fetch data from memory, thus return what DMA could have stored there.

Dave

Signature · 1 line
John David Anglin  [email protected]
author maintainer
#4 lore raw

* John David Anglin <[email protected]>:

Quoted 20 lines · Helge Deller
Quoted 11 lines · John David Anglin
Quoted 5 lines · Helge Deller
Quoted 3 lines

+    case DMA_BIDIRECTIONAL:
+        flush_kernel_dcache_range(addr, size);
+        purge_kernel_dcache_range_asm(addr, addr + size);

I don't think flush and purge are both needed.

I'm not sure...

Just to fully understand it. Is this short summary correct: ?
- flush_kernel_dcache_range: flush cache back to memory, but keep data in cache.

No.  If present, fdc writes addressed cache line to memory if and only if line is dirty.  It then invalidates line.  It does not keep data in cache.

Next read loads from memory.

Quoted 4 lines

    Next read fetches the data which is still in the cache, thus the next
    read doesn't checks if data in memory has been modified in the meantime (e.g. via DMA).
- purge_kernel_dcache_range_asm: ignore currently cached data & drop any cached data in that range. Even if cache has dirty memory which hasn't been written back yet, drop it and don't write back.

Quoted 5 lines

if present, pdc invalidates line.  At privilege level zero, an implementation may optionally write back a dirty line to memory.  At non-zero privilege levels, fdc and pdc are effectively the same.

I don't know how to determine whether pdc does write back or not. It would be specified in processor ERS.

Thanks for the explanation!
With that, I've attached an updated (untested) patch below.

Helge

[PATCH v2] parisc: improve cach flushing in arch_sync_dma_for_cpu()

Add comment in arch_sync_dma_for_device() and handle the direction flag in arch_sync_dma_for_cpu().

When receiving data from the device (DMA_FROM_DEVICE) unconditionally purge the data cache in arch_sync_dma_for_cpu().

arch/parisc/kernel/pci-dma.c +17 -1
index ba87f791323b..71ed5391f29d 100644
@@ -446,11 +446,27 @@ void arch_dma_free(struct device *dev, size_t size, void *vaddr,
 void arch_sync_dma_for_device(phys_addr_t paddr, size_t size,
 		enum dma_data_direction dir)
 {
+	/*
+	 * fdc: The data cache line is written back to memory, if and only if
+	 * it is dirty, and then invalidated from the data cache.
+	 */
 	flush_kernel_dcache_range((unsigned long)phys_to_virt(paddr), size);
 }
 
 void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size,
 		enum dma_data_direction dir)
 {
-	flush_kernel_dcache_range((unsigned long)phys_to_virt(paddr), size);
+	unsigned long addr = (unsigned long) phys_to_virt(paddr);
+
+	switch (dir) {
+	case DMA_TO_DEVICE:
+	case DMA_BIDIRECTIONAL:
+		flush_kernel_dcache_range(addr, size);
+		return;
+	case DMA_FROM_DEVICE:
+		purge_kernel_dcache_range_asm(addr, addr + size);
+		return;
+	default:
+		BUG();
+	}
 }
#5 lore raw
Quoted 20 lines · Helge Deller

* John David Anglin <[email protected]>:

Quoted 19 lines · Helge Deller
Quoted 10 lines · John David Anglin
Quoted 5 lines · Helge Deller
Quoted 3 lines

+    case DMA_BIDIRECTIONAL:
+        flush_kernel_dcache_range(addr, size);
+        purge_kernel_dcache_range_asm(addr, addr + size);

I don't think flush and purge are both needed.

I'm not sure...

Just to fully understand it. Is this short summary correct: ?
- flush_kernel_dcache_range: flush cache back to memory, but keep data in cache.

No.  If present, fdc writes addressed cache line to memory if and only if line is dirty.  It then invalidates line.  It does not keep data in cache.

Next read loads from memory.

Quoted 4 lines

    Next read fetches the data which is still in the cache, thus the next
    read doesn't checks if data in memory has been modified in the meantime (e.g. via DMA).
- purge_kernel_dcache_range_asm: ignore currently cached data & drop any cached data in that range. Even if cache has dirty memory which hasn't been written back yet, drop it and don't write back.

Quoted 7 lines
Quoted 5 lines

if present, pdc invalidates line.  At privilege level zero, an implementation may optionally write back a dirty line to memory.  At non-zero privilege levels, fdc and pdc are effectively the same.

I don't know how to determine whether pdc does write back or not. It would be specified in processor ERS.

Thanks for the explanation!
With that, I've attached an updated (untested) patch below.

Seems to work okay on c8000.  Don't know if it helps performance.

Dave

Quoted 42 lines

[PATCH v2] parisc: improve cach flushing in arch_sync_dma_for_cpu()

Add comment in arch_sync_dma_for_device() and handle the direction flag in arch_sync_dma_for_cpu().

When receiving data from the device (DMA_FROM_DEVICE) unconditionally purge the data cache in arch_sync_dma_for_cpu().

arch/parisc/kernel/pci-dma.c +17 -1
index ba87f791323b..71ed5391f29d 100644
@@ -446,11 +446,27 @@ void arch_dma_free(struct device *dev, size_t size, void *vaddr,
  void arch_sync_dma_for_device(phys_addr_t paddr, size_t size,
  		enum dma_data_direction dir)
  {
+	/*
+	 * fdc: The data cache line is written back to memory, if and only if
+	 * it is dirty, and then invalidated from the data cache.
+	 */
  	flush_kernel_dcache_range((unsigned long)phys_to_virt(paddr), size);
  }
 
  void arch_sync_dma_for_cpu(phys_addr_t paddr, size_t size,
  		enum dma_data_direction dir)
  {
-	flush_kernel_dcache_range((unsigned long)phys_to_virt(paddr), size);
+	unsigned long addr = (unsigned long) phys_to_virt(paddr);
+
+	switch (dir) {
+	case DMA_TO_DEVICE:
+	case DMA_BIDIRECTIONAL:
+		flush_kernel_dcache_range(addr, size);
+		return;
+	case DMA_FROM_DEVICE:
+		purge_kernel_dcache_range_asm(addr, addr + size);
+		return;
+	default:
+		BUG();
+	}
  }
Signature · 1 line
John David Anglin  [email protected]
author maintainer
#6 lore raw
Quoted 24 lines · John David Anglin
Quoted 23 lines · Helge Deller

* John David Anglin <[email protected]>:

Quoted 22 lines · Helge Deller
Quoted 10 lines · John David Anglin
Quoted 5 lines · Helge Deller
Quoted 3 lines

+    case DMA_BIDIRECTIONAL:
+        flush_kernel_dcache_range(addr, size);
+        purge_kernel_dcache_range_asm(addr, addr + size);

I don't think flush and purge are both needed.

I'm not sure...

Just to fully understand it. Is this short summary correct: ?
- flush_kernel_dcache_range: flush cache back to memory, but keep data in cache.

No.  If present, fdc writes addressed cache line to memory if and only if line is dirty.  It then invalidates line.  It does not keep data in cache.

Next read loads from memory.

Quoted 4 lines

     Next read fetches the data which is still in the cache, thus the next
     read doesn't checks if data in memory has been modified in the meantime (e.g. via DMA).
- purge_kernel_dcache_range_asm: ignore currently cached data & drop any cached data in that range. Even if cache has dirty memory which hasn't been written back yet, drop it and don't write back.

if present, pdc invalidates line.  At privilege level zero, an implementation may optionally write back a dirty line to memory.  At non-zero privilege levels, fdc and pdc are effectively the same.

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

I don't know how to determine whether pdc does write back or not. It would be specified in processor ERS.

Thanks for the explanation!
With that, I've attached an updated (untested) patch below.

Seems to work okay on c8000.  Don't know if it helps performance.

Probably no performance change for c8000.
I noticed that pci-dma.c is only compiled on 32-bit kernels (for PCX-T machines) :-)

Helge