Skip to contentSkip to conversation

locking/memory-barriers.txt: Improve documentation for writel() example

PATCH Superseded Parav Pandit 3 participants 6 messages Oct 5, 2022 – Oct 6, 2022

Subsystems (3)

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 locking/memory-barriers.txt: Improve documentation for writel() example

The cited commit describes that when using writel(), explcit wmb() is not needed. wmb() is an expensive barrier. writel() uses the needed platform specific barrier instead of expensive wmb().

Hence update the example to be more accurate that matches the current implementation.

commit 5846581e3563 ("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example")

---
changelog:
v1->v2:
- Further improved description of writel() example
- changed commit subject from 'usage' to 'example' v0->v1:
- Corrected to mention I/O barrier instead of dma_wmb().
- removed numbered references in commit log
- corrected typo 'explcit' to 'explicit' in commit log

---
 Documentation/memory-barriers.txt | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
Documentation/memory-barriers.txt +6 -4
index 832b5d36e279..49e1433db407 100644
@@ -1927,10 +1927,12 @@ There are some more advanced barrier functions:
      before we read the data from the descriptor, and the dma_wmb() allows
      us to guarantee the data is written to the descriptor before the device
      can see it now has ownership.  The dma_mb() implies both a dma_rmb() and
-     a dma_wmb().  Note that, when using writel(), a prior wmb() is not needed
-     to guarantee that the cache coherent memory writes have completed before
-     writing to the MMIO region.  The cheaper writel_relaxed() does not provide
-     this guarantee and must not be used here.
+     a dma_wmb().  Note that, when using writel(), a prior barrier is not
+     needed to guarantee that the cache coherent memory writes have completed
+     before writing to the MMIO region.  The cheaper writel_relaxed() does not
+     provide this guarantee and must not be used here. Hence, writeX() is always
+     preferred which inserts needed platform specific barrier before writing to
+     the specified MMIO region.
 
      See the subsection "Kernel I/O barrier effects" for more information on
      relaxed I/O accessors and the Documentation/core-api/dma-api.rst file for
Signature · 1 line
2.26.2
#1 lore raw
Quoted 5 lines · Parav Pandit
Documentation/memory-barriers.txt +0 -1
@@ -1927,10 +1927,12 @@ There are some more advanced barrier functions:
      before we read the data from the descriptor, and the dma_wmb() allows
      us to guarantee the data is written to the descriptor before the device
      can see it now has ownership.  The dma_mb() implies both a dma_rmb() and
-     a dma_wmb().  Note that, when using writel(), a prior wmb() is not needed

Documentation/memory-barriers.txt There are some more advanced barrier functions: · line 1936

Quoted 10 lines
Documentation/memory-barriers.txt +6 -3
-     to guarantee that the cache coherent memory writes have completed before
-     writing to the MMIO region.  The cheaper writel_relaxed() does not provide
-     this guarantee and must not be used here.
+     a dma_wmb().  Note that, when using writel(), a prior barrier is not
+     needed to guarantee that the cache coherent memory writes have completed
+     before writing to the MMIO region.  The cheaper writel_relaxed() does not
+     provide this guarantee and must not be used here. Hence, writeX() is always
+     preferred which inserts needed platform specific barrier before writing to
+     the specified MMIO region.
 

Did you mean that writeX() is write() function family?

Signature · 1 line
An old man doll... just what I always wanted! - Clara
#2 lore raw
Quoted 24 lines

From: Bagas Sanjaya <[email protected]>
Sent: Thursday, October 6, 2022 7:31 AM

Review There are some more advanced barrier

Quoted 1 line · Parav Pandit
(quoted diff) +0 -0
@@ -1927,10 +1927,12 @@ There are some more advanced barrier

functions:

Review There are some more advanced barrier · line 1927

Quoted 1 line
(quoted diff) +0 -0
      before we read the data from the descriptor, and the dma_wmb()

allows

Review There are some more advanced barrier · line 1929

Quoted 2 lines
(quoted diff) +0 -0
      us to guarantee the data is written to the descriptor before the device
      can see it now has ownership.  The dma_mb() implies both a

dma_rmb() and

Review There are some more advanced barrier · line 1930

Quoted 1 line
(quoted diff) +0 -1
-     a dma_wmb().  Note that, when using writel(), a prior wmb() is not

needed

Review There are some more advanced barrier · line 1931

Quoted 1 line
(quoted diff) +0 -1
-     to guarantee that the cache coherent memory writes have completed

before

Review There are some more advanced barrier · line 1932

Quoted 1 line
(quoted diff) +0 -1
-     writing to the MMIO region.  The cheaper writel_relaxed() does not

provide

Documentation/memory-barriers.txt There are some more advanced barrier functions:

Quoted 3 lines
Documentation/memory-barriers.txt +2 -1
-     this guarantee and must not be used here.
+     a dma_wmb().  Note that, when using writel(), a prior barrier is not
+     needed to guarantee that the cache coherent memory writes have

completed

Review There are some more advanced barrier · line 1932

Quoted 1 line
(quoted diff) +1 -0
+     before writing to the MMIO region.  The cheaper writel_relaxed() does

not

Quoted 8 lines

Review There are some more advanced barrier · line 1933

Quoted 1 line
(quoted diff) +1 -0
+     provide this guarantee and must not be used here. Hence, writeX() is

always

Review There are some more advanced barrier · line 1934

Quoted 1 line
(quoted diff) +1 -0
+     preferred which inserts needed platform specific barrier before writing

to

Documentation/memory-barriers.txt There are some more advanced barrier functions: · line 1936

Quoted 2 lines
Documentation/memory-barriers.txt +1 -0
+     the specified MMIO region.
 

Did you mean that writeX() is write() function family?

Yes.

#3 lore raw
In reply to Parav Pandit

Hi,

Maybe I'm too nit-picky, but see below:

Quoted 8 lines · Parav Pandit

The cited commit describes that when using writel(), explcit wmb() is not needed. wmb() is an expensive barrier. writel() uses the needed platform specific barrier instead of expensive wmb().

Hence update the example to be more accurate that matches the current implementation.

commit 5846581e3563 ("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example")

You can cite the commit in the Changelog text.
Just say:

Commit 5846581e3563 ("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example") describes that when using writel(), ...

Also, a blank line is needed above S-o-b tags as a delimiter.

Quoted 20 lines

---
changelog:
v1->v2:
- Further improved description of writel() example
- changed commit subject from 'usage' to 'example' v0->v1:
- Corrected to mention I/O barrier instead of dma_wmb().
- removed numbered references in commit log
- corrected typo 'explcit' to 'explicit' in commit log

---
 Documentation/memory-barriers.txt | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
Documentation/memory-barriers.txt +0 -0
index 832b5d36e279..49e1433db407 100644

Documentation/memory-barriers.txt There are some more advanced barrier functions: · line 1930

Quoted 8 lines
Documentation/memory-barriers.txt +1 -4
@@ -1927,10 +1927,12 @@ There are some more advanced barrier functions:
      before we read the data from the descriptor, and the dma_wmb() allows
      us to guarantee the data is written to the descriptor before the device
      can see it now has ownership.  The dma_mb() implies both a dma_rmb() and
-     a dma_wmb().  Note that, when using writel(), a prior wmb() is not needed
-     to guarantee that the cache coherent memory writes have completed before
-     writing to the MMIO region.  The cheaper writel_relaxed() does not provide
-     this guarantee and must not be used here.
+     a dma_wmb().  Note that, when using writel(), a prior barrier is not

If you permit a slightly long line here, this hunk would be much easier to compare:

+ a dma_wmb(). Note that, when using writel(), a prior barrier is not needed
to guarantee that the cache coherent memory writes have completed before writing to the MMIO region. The cheaper writel_relaxed() does not provide

+     this guarantee and must not be used here. Hence, writeX() is always
+     preferred which inserts needed platform specific barrier before writing to
+     the specified MMIO region.

That said, I don't feel comfortable with the sentence you added. It looks to me it is redundant because such a guarantee of writeX() is already covered in the section of "KERNEL I/O BARRIER EFFECTS".
See the relevant explanation quoted below:

3. A writeX() by a CPU thread to the peripheral will first wait for the completion of all prior writes to memory either issued by, or propagated to, the same thread. This ensures that writes by the CPU to an outbound DMA buffer allocated by dma_alloc_coherent() will be visible to a DMA engine when the CPU writes to its MMIO control register to trigger the transfer.

Also please not that this document should not describe any implementation details of those accessors. This document is not meant as an implementation guide, but a guide for kernel developers who use them. This is clearly mentioned in "DISCLAIMER" at the top of this file.

Thanks, Akira

Quoted 8 lines
Documentation/memory-barriers.txt +5 -0
+     needed to guarantee that the cache coherent memory writes have completed
+     before writing to the MMIO region.  The cheaper writel_relaxed() does not
+     provide this guarantee and must not be used here. Hence, writeX() is always
+     preferred which inserts needed platform specific barrier before writing to
+     the specified MMIO region.
 
      See the subsection "Kernel I/O barrier effects" for more information on
      relaxed I/O accessors and the Documentation/core-api/dma-api.rst file for
#4 lore raw
Quoted 65 lines · Akira Yokosawa

Hi,

Maybe I'm too nit-picky, but see below:

Quoted 8 lines · Parav Pandit

The cited commit describes that when using writel(), explcit wmb() is not needed. wmb() is an expensive barrier. writel() uses the needed platform specific barrier instead of expensive wmb().

Hence update the example to be more accurate that matches the current implementation.

commit 5846581e3563 ("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example")

You can cite the commit in the Changelog text.
Just say:

Commit 5846581e3563 ("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example") describes that when using writel(), ...

Also, a blank line is needed above S-o-b tags as a delimiter.

Quoted 20 lines

---
changelog:
v1->v2:
- Further improved description of writel() example
- changed commit subject from 'usage' to 'example' v0->v1:
- Corrected to mention I/O barrier instead of dma_wmb().
- removed numbered references in commit log
- corrected typo 'explcit' to 'explicit' in commit log

---
 Documentation/memory-barriers.txt | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)
Documentation/memory-barriers.txt +0 -0
index 832b5d36e279..49e1433db407 100644

Documentation/memory-barriers.txt There are some more advanced barrier functions: · line 1930

Quoted 8 lines
Documentation/memory-barriers.txt +1 -4
@@ -1927,10 +1927,12 @@ There are some more advanced barrier functions:
      before we read the data from the descriptor, and the dma_wmb() allows
      us to guarantee the data is written to the descriptor before the device
      can see it now has ownership.  The dma_mb() implies both a dma_rmb() and
-     a dma_wmb().  Note that, when using writel(), a prior wmb() is not needed
-     to guarantee that the cache coherent memory writes have completed before
-     writing to the MMIO region.  The cheaper writel_relaxed() does not provide
-     this guarantee and must not be used here.
+     a dma_wmb().  Note that, when using writel(), a prior barrier is not

If you permit a slightly long line here, this hunk would be much easier to compare:

(quoted diff) +4 -0
+     a dma_wmb().  Note that, when using writel(), a prior barrier is not needed
      to guarantee that the cache coherent memory writes have completed before
      writing to the MMIO region.  The cheaper writel_relaxed() does not provide
+     this guarantee and must not be used here. Hence, writeX() is always
+     preferred which inserts needed platform specific barrier before writing to
+     the specified MMIO region.

That said, I don't feel comfortable with the sentence you added. It looks to me it is redundant because such a guarantee of writeX() is already covered in the section of "KERNEL I/O BARRIER EFFECTS".
See the relevant explanation quoted below:

Quoted 8 lines

3. A writeX() by a CPU thread to the peripheral will first wait for the completion of all prior writes to memory either issued by, or propagated to, the same thread. This ensures that writes by the CPU to an outbound DMA buffer allocated by dma_alloc_coherent() will be visible to a DMA engine when the CPU writes to its MMIO control register to trigger the transfer.

Also please not that this document should not describe any implementation

I meant note

Quoted 14 lines

details of those accessors. This document is not meant as an implementation guide, but a guide for kernel developers who use them. This is clearly mentioned in "DISCLAIMER" at the top of this file.

Thanks, Akira

Quoted 8 lines
Documentation/memory-barriers.txt +5 -0
+     needed to guarantee that the cache coherent memory writes have completed
+     before writing to the MMIO region.  The cheaper writel_relaxed() does not
+     provide this guarantee and must not be used here. Hence, writeX() is always
+     preferred which inserts needed platform specific barrier before writing to
+     the specified MMIO region.
 
      See the subsection "Kernel I/O barrier effects" for more information on
      relaxed I/O accessors and the Documentation/core-api/dma-api.rst file for
#5 lore raw
In reply to Akira Yokosawa
Quoted 13 lines

From: Akira Yokosawa <[email protected]>
Sent: Thursday, October 6, 2022 8:18 AM

Hi,

Maybe I'm too nit-picky, but see below:

Quoted 4 lines · Parav Pandit

The cited commit describes that when using writel(), explcit wmb() is not needed. wmb() is an expensive barrier. writel() uses the needed platform specific barrier instead of expensive wmb().

Quoted 9 lines
Quoted 5 lines

Hence update the example to be more accurate that matches the current implementation.

commit 5846581e3563 ("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example")

You can cite the commit in the Changelog text.
Just say:

I prefer to annotate the long commit as a separate line. This makes the description sentence easy to read.

Quoted 5 lines

Commit 5846581e3563 ("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example") describes that when using writel(), ...

Also, a blank line is needed above S-o-b tags as a delimiter.

Adding blank line in v3.

[..]

Quoted 3 lines

If you permit a slightly long line here, this hunk would be much easier to compare:

True, but document crosses the 80 lines. So I will keep the existing alignment of 80 characters per line.

Quoted 11 lines

+ a dma_wmb(). Note that, when using writel(), a prior barrier is
+ not needed
to guarantee that the cache coherent memory writes have completed
before
writing to the MMIO region. The cheaper writel_relaxed() does not
provide

(quoted diff) +2 -0
+     this guarantee and must not be used here. Hence, writeX() is always
+     preferred which inserts needed platform specific barrier before writing

to
+ the specified MMIO region.

Quoted 5 lines

That said, I don't feel comfortable with the sentence you added. It looks to me it is redundant because such a guarantee of writeX() is already covered in the section of "KERNEL I/O BARRIER EFFECTS".
See the relevant explanation quoted below:

I observed that too, but I thought it is worth short note to add along with this example. But I agree since the next para refers to it anyway, I will drop this 2nd addition in v3.

Quoted 9 lines

3. A writeX() by a CPU thread to the peripheral will first wait for the completion of all prior writes to memory either issued by, or propagated to, the same thread. This ensures that writes by the
CPU
to an outbound DMA buffer allocated by dma_alloc_coherent() will
be
visible to a DMA engine when the CPU writes to its MMIO control register to trigger the transfer.

Quoted 5 lines

Also please not that this document should not describe any implementation details of those accessors. This document is not meant as an implementation guide, but a guide for kernel developers who use them. This is clearly mentioned in "DISCLAIMER" at the top of this file.

In the disclaimer section I didn’t see any objection to "describe any implementation". :)
For example, disclaimer itself tells that a barrier may be no-op talking about a possible implementation on a platform.

But I agree to your above point that above #3 already covers with my addition. Hence, I will drop that part in v3.
Thanks for the review.

Quoted 15 lines

Thanks, Akira

Quoted 1 line

+ needed to guarantee that the cache coherent memory writes have

completed

Quoted 1 line

+ before writing to the MMIO region. The cheaper writel_relaxed() does

not

Quoted 1 line

+ provide this guarantee and must not be used here. Hence, writeX() is

always

Quoted 1 line

+ preferred which inserts needed platform specific barrier before writing

to

Quoted 5 lines

+ the specified MMIO region.

See the subsection "Kernel I/O barrier effects" for more information on relaxed I/O accessors and the Documentation/core-api/dma-api.rst
file for