Skip to contentSkip to conversation

locking/memory-barriers.txt: Improve documentation for writel() example

PATCH In review Parav Pandit 3 participants 5 messages Oct 27, 2022 – Nov 2, 2022

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v5 1/1 locking/memory-barriers.txt: Improve documentation for writel() example

The cited commit describes that when using writel(), explicit wmb() is not needed. wmb() is an expensive barrier. writel() uses the needed platform specific barrier instead of wmb().

writeX() section of "KERNEL I/O BARRIER EFFECTS" already describes ordering of I/O accessors with MMIO writes.

Hence add the comment for pseudo code of writel() and remove confusing text around writel() and wmb().

commit 5846581e3563 ("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example")

---
changelog:
v4->v5:
- Used suggested documentation update from Will
- Added comment to the writel() pseudo code example
- updated commit log for newer changes
v3->v4:
- further trimmed the documentation for redundant description v2->v3:
- removed redundant description for writeX()
- updated text for alignment and smaller change lines
- updated commit log with blank line before signed-off-by line v1->v2:
- Further improved description of writel() example
- changed commit subject from 'usage' to 'example' v0->v1:
- Corrected to mention I/O barrier instead of dma_wmb().
- removed numbered references in commit log
- corrected typo 'explcit' to 'explicit' in commit log

---
 Documentation/memory-barriers.txt | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)
Documentation/memory-barriers.txt +11 -11
index 06f80e3785c5..e698093bade1 100644
@@ -1910,7 +1910,8 @@ There are some more advanced barrier functions:
 
      These are for use with consistent memory to guarantee the ordering
      of writes or reads of shared memory accessible to both the CPU and a
-     DMA capable device.
+     DMA capable device. See Documentation/core-api/dma-api.rst file for more
+     information about consistent memory.
 
      For example, consider a device driver that shares memory with a device
      and uses a descriptor status value to indicate if the descriptor belongs
@@ -1931,22 +1932,21 @@ There are some more advanced barrier functions:
 		/* assign ownership */
 		desc->status = DEVICE_OWN;
 
-		/* notify device of new descriptors */
+		/* Make descriptor status visible to the device followed by
+		 * notify device of new descriptor
+		 */
 		writel(DESC_NOTIFY, doorbell);
 	}
 
-     The dma_rmb() allows us guarantee the device has released ownership
+     The dma_rmb() allows us to guarantee that the device has released ownership
      before we read the data from the descriptor, and the dma_wmb() allows
      us to guarantee the data is written to the descriptor before the device
      can see it now has ownership.  The dma_mb() implies both a dma_rmb() and
-     a dma_wmb().  Note that, when using writel(), a prior wmb() is not needed
-     to guarantee that the cache coherent memory writes have completed before
-     writing to the MMIO region.  The cheaper writel_relaxed() does not provide
-     this guarantee and must not be used here.
-
-     See the subsection "Kernel I/O barrier effects" for more information on
-     relaxed I/O accessors and the Documentation/core-api/dma-api.rst file for
-     more information on consistent memory.
+     a dma_wmb().
+
+     Note that the dma_*() barriers do not provide any ordering guarantees for
+     accesses to MMIO regions.  See the later "KERNEL I/O BARRIER EFFECTS"
+     subsection for more information about I/O accessors and MMIO ordering.
 
  (*) pmem_wmb();
 
Signature · 1 line
2.26.2
maintainer
#1 lore raw
Quoted 7 lines · Parav Pandit

The cited commit describes that when using writel(), explicit wmb() is not needed. wmb() is an expensive barrier. writel() uses the needed platform specific barrier instead of wmb().

writeX() section of "KERNEL I/O BARRIER EFFECTS" already describes ordering of I/O accessors with MMIO writes.

Quoted 6 lines

Hence add the comment for pseudo code of writel() and remove confusing text around writel() and wmb().

commit 5846581e3563 ("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example")

Hearing no immediate objections, I have pulled this in for further review. If all goes well, I intend to submit this during the upcoming v6.2 merge window.

Thanx, Paul

Quoted 72 lines

---
changelog:
v4->v5:
- Used suggested documentation update from Will
- Added comment to the writel() pseudo code example
- updated commit log for newer changes
v3->v4:
- further trimmed the documentation for redundant description v2->v3:
- removed redundant description for writeX()
- updated text for alignment and smaller change lines
- updated commit log with blank line before signed-off-by line v1->v2:
- Further improved description of writel() example
- changed commit subject from 'usage' to 'example' v0->v1:
- Corrected to mention I/O barrier instead of dma_wmb().
- removed numbered references in commit log
- corrected typo 'explcit' to 'explicit' in commit log

---
 Documentation/memory-barriers.txt | 22 +++++++++++-----------
 1 file changed, 11 insertions(+), 11 deletions(-)
Documentation/memory-barriers.txt +11 -11
index 06f80e3785c5..e698093bade1 100644
@@ -1910,7 +1910,8 @@ There are some more advanced barrier functions:
 
      These are for use with consistent memory to guarantee the ordering
      of writes or reads of shared memory accessible to both the CPU and a
-     DMA capable device.
+     DMA capable device. See Documentation/core-api/dma-api.rst file for more
+     information about consistent memory.
 
      For example, consider a device driver that shares memory with a device
      and uses a descriptor status value to indicate if the descriptor belongs
@@ -1931,22 +1932,21 @@ There are some more advanced barrier functions:
 		/* assign ownership */
 		desc->status = DEVICE_OWN;
 
-		/* notify device of new descriptors */
+		/* Make descriptor status visible to the device followed by
+		 * notify device of new descriptor
+		 */
 		writel(DESC_NOTIFY, doorbell);
 	}
 
-     The dma_rmb() allows us guarantee the device has released ownership
+     The dma_rmb() allows us to guarantee that the device has released ownership
      before we read the data from the descriptor, and the dma_wmb() allows
      us to guarantee the data is written to the descriptor before the device
      can see it now has ownership.  The dma_mb() implies both a dma_rmb() and
-     a dma_wmb().  Note that, when using writel(), a prior wmb() is not needed
-     to guarantee that the cache coherent memory writes have completed before
-     writing to the MMIO region.  The cheaper writel_relaxed() does not provide
-     this guarantee and must not be used here.
-
-     See the subsection "Kernel I/O barrier effects" for more information on
-     relaxed I/O accessors and the Documentation/core-api/dma-api.rst file for
-     more information on consistent memory.
+     a dma_wmb().
+
+     Note that the dma_*() barriers do not provide any ordering guarantees for
+     accesses to MMIO regions.  See the later "KERNEL I/O BARRIER EFFECTS"
+     subsection for more information about I/O accessors and MMIO ordering.
 
  (*) pmem_wmb();
 
Signature · 1 line
2.26.2
#2 lore raw

Hi Will, Paul,

Quoted 17 lines

From: Paul E. McKenney <[email protected]>
Sent: Friday, October 28, 2022 1:56 PM

Quoted 13 lines · Parav Pandit

The cited commit describes that when using writel(), explicit wmb() is not needed. wmb() is an expensive barrier. writel() uses the needed platform specific barrier instead of wmb().

writeX() section of "KERNEL I/O BARRIER EFFECTS" already describes ordering of I/O accessors with MMIO writes.

Hence add the comment for pseudo code of writel() and remove confusing text around writel() and wmb().

commit 5846581e3563 ("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example")

Quoted 7 lines
Quoted 1 line

Hearing no immediate objections, I have pulled this in for further review. If all goes well, I intend to submit this during the upcoming
v6.2 merge window.

Thanx, Paul

I have taken Will's email patch suggestion with small editorial modification in it. But I was afraid to add his Signed-off-by myself in v5 without consulting him.

Will,
Can you please reply to add your Signed-off-by as well or not?

maintainer Acked-by on v5 1/1
#3 lore raw
In reply to Parav Pandit
Quoted 12 lines · Parav Pandit

The cited commit describes that when using writel(), explicit wmb() is not needed. wmb() is an expensive barrier. writel() uses the needed platform specific barrier instead of wmb().

writeX() section of "KERNEL I/O BARRIER EFFECTS" already describes ordering of I/O accessors with MMIO writes.

Hence add the comment for pseudo code of writel() and remove confusing text around writel() and wmb().

commit 5846581e3563 ("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example")

Quoted 7 lines

---
changelog:
v4->v5:
- Used suggested documentation update from Will
- Added comment to the writel() pseudo code example
- updated commit log for newer changes

Sorry for the delay on this, I'm really behind on patches at the moment. This patch looks good to me, so thanks for doing it. You can either add my:

or, since we worked on this together:

Cheers,

Will

maintainer
#4 lore raw
Quoted 25 lines · Will Deacon
Quoted 12 lines · Parav Pandit

The cited commit describes that when using writel(), explicit wmb() is not needed. wmb() is an expensive barrier. writel() uses the needed platform specific barrier instead of wmb().

writeX() section of "KERNEL I/O BARRIER EFFECTS" already describes ordering of I/O accessors with MMIO writes.

Hence add the comment for pseudo code of writel() and remove confusing text around writel() and wmb().

commit 5846581e3563 ("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example")

Quoted 7 lines

---
changelog:
v4->v5:
- Used suggested documentation update from Will
- Added comment to the writel() pseudo code example
- updated commit log for newer changes

Sorry for the delay on this, I'm really behind on patches at the moment. This patch looks good to me, so thanks for doing it. You can either add my:

Quoted 6 lines

or, since we worked on this together:

Thank you! I will apply these tags on the next rebase.

Thanx, Paul