ntb_transport: replace wmb() with dma_wmb() in TX paths
PATCH 1/1 ntb_transport: replace wmb() with dma_wmb() in TX paths
The TX paths currently use wmb() after memcpy_toio() to ensure that writes to the peer memory window are visible before subsequent notification.
The actual ordering requirement is to ensure that data written to the peer memory is observed before the associated notification or status update. This matches the semantics of dma_wmb(), which orders memory writes with respect to external observers.
The peer memory window is mapped with ioremap_wc(), allowing write-combining. On weakly ordered architectures such as arm64, dma_wmb() ensures that prior writes are committed from write-combining buffers and become visible to the peer before subsequent operations.
This aligns with common patterns in PCIe-based drivers, where it is used to order descriptor or payload writes before notification, while avoiding the stronger ordering semantics of wmb().
The ordering with respect to peer observation is completed by the subsequent notification mechanism.
No functional change is intended.
--- drivers/ntb/ntb_transport.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/ntb/ntb_transport.c +1 -1
@@ -1791,7 +1791,7 @@ static void ntb_memcpy_tx_on_stack(struct ntb_queue_entry *entry, void __iomem *#endif /* Ensure that the data is fully copied out before setting the flags */ -wmb (); +dma_wmb ();ntb_tx_copy_callback (entry,NULL ); }
Signature · 1 line
2.34.1
Dave Jiang
maintainerQuoted 18 lines · Xingbang Liu
The TX paths currently use
wmb()aftermemcpy_toio()to ensure that writes to the peer memory window are visible before subsequent notification.The actual ordering requirement is to ensure that data written to the peer memory is observed before the associated notification or status update. This matches the semantics of
dma_wmb(), which orders memory writes with respect to external observers.The peer memory window is mapped with
ioremap_wc(), allowing write-combining. On weakly ordered architectures such as arm64,dma_wmb()ensures that prior writes are committed from write-combining buffers and become visible to the peer before subsequent operations.This aligns with common patterns in PCIe-based drivers, where it is used to order descriptor or payload writes before notification, while avoiding the stronger ordering semantics of
wmb().
Quoted 6 lines
The ordering with respect to peer observation is completed by the subsequent notification mechanism.
No functional change is intended.
Signed-off-by Xingbang Liu <[email protected]>
NAK. dma_wmb() is used for shared memory between CPU and a device. For example:
memcpy(dma_buffer, data, size);dma_wmb(); /* Ensure CPU writes visible to device DMA */ writel(DOORBELL, device->reg + DB_OFFSET); /* trigger device DMA from dma_buffer */
In this case we are copying data to MMIO and therefore wmb() is needed to ensure all MMIO writes have happened before we proceed.
DJ
Quoted 17 lines
--- drivers/ntb/ntb_transport.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/ntb/ntb_transport.c +1
-1@@ -1791,7 +1791,7 @@ static void ntb_memcpy_tx_on_stack(struct ntb_queue_entry *entry, void __iomem *#endif /* Ensure that the data is fully copied out before setting the flags */ -wmb (); +dma_wmb ();ntb_tx_copy_callback (entry,NULL ); }
Hi DJ ,
Thanks for the review and the explanation.
I understand that dma_wmb() is intended for CPU → device DMA visibility, and since this path writes to MMIO (peer memory window), a full wmb() is required to guarantee that all writes are visible before the notification/doorbell.
I will withdraw this patch and keep the wmb() in place.
Thanks,
Xingbang Liu
Quoted 57 lines · Dave Jiang <[email protected]> 于2026年4月24日周五 23:32写道:
Quoted 18 lines · Xingbang Liu
The TX paths currently use
wmb()aftermemcpy_toio()to ensure that writes to the peer memory window are visible before subsequent notification.The actual ordering requirement is to ensure that data written to the peer memory is observed before the associated notification or status update. This matches the semantics of
dma_wmb(), which orders memory writes with respect to external observers.The peer memory window is mapped with
ioremap_wc(), allowing write-combining. On weakly ordered architectures such as arm64,dma_wmb()ensures that prior writes are committed from write-combining buffers and become visible to the peer before subsequent operations.This aligns with common patterns in PCIe-based drivers, where it is used to order descriptor or payload writes before notification, while avoiding the stronger ordering semantics of
wmb().Quoted 6 lines
The ordering with respect to peer observation is completed by the subsequent notification mechanism.
No functional change is intended.
Signed-off-by Xingbang Liu <[email protected]>NAK.
dma_wmb()is used for shared memory between CPU and a device. For example:
memcpy(dma_buffer, data, size);dma_wmb(); /* Ensure CPU writes visible to device DMA */writel(DOORBELL, device->reg + DB_OFFSET); /* trigger device DMA fromdma_buffer*/In this case we are copying data to MMIO and therefore
wmb()is needed to ensure all MMIO writes have happened before we proceed.DJ
Quoted 17 lines
--- drivers/ntb/ntb_transport.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/ntb/ntb_transport.c +1
-1@@ -1791,7 +1791,7 @@ static void ntb_memcpy_tx_on_stack(struct ntb_queue_entry *entry, void __iomem *#endif /* Ensure that the data is fully copied out before setting the flags */ -wmb (); +dma_wmb ();ntb_tx_copy_callback (entry,NULL ); }