locking/memory-barriers.txt: Improve documentation for writel() example
Subsystems (3)
PATCH v2 1/1 locking/memory-barriers.txt: Improve documentation for writel() example
The cited commit describes that when using writel(), explcit wmb() is not needed. wmb() is an expensive barrier. writel() uses the needed platform specific barrier instead of expensive wmb().
Hence update the example to be more accurate that matches the current implementation.
commit 5846581e3563 ("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example")
---
changelog:v1->v2:
- Further improved description of writel() example
- changed commit subject from 'usage' to 'example' v0->v1:
- Corrected to mention I/O barrier instead of dma_wmb().
- removed numbered references in commit log
- corrected typo 'explcit' to 'explicit' in commit log
--- Documentation/memory-barriers.txt | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)
Documentation/memory-barriers.txt +6 -4
@@ -1927,10 +1927,12 @@ There are some more advanced barrier functions: before we read the data from the descriptor, and the dma_wmb() allows us to guarantee the data is written to the descriptor before the device can see it now has ownership. The dma_mb() implies both a dma_rmb() and - a dma_wmb(). Note that, when using writel(), a prior wmb() is not needed - to guarantee that the cache coherent memory writes have completed before - writing to the MMIO region. The cheaper writel_relaxed() does not provide - this guarantee and must not be used here. + a dma_wmb(). Note that, when using writel(), a prior barrier is not + needed to guarantee that the cache coherent memory writes have completed + before writing to the MMIO region. The cheaper writel_relaxed() does not + provide this guarantee and must not be used here. Hence, writeX() is always + preferred which inserts needed platform specific barrier before writing to + the specified MMIO region. See the subsection "Kernel I/O barrier effects" for more information on relaxed I/O accessors and the Documentation/core-api/dma-api.rst file for
Signature · 1 line
2.26.2
Quoted 5 lines · Parav Pandit
Documentation/memory-barriers.txt +0
-1@@ -1927,10 +1927,12 @@ There are some more advanced barrier functions: before we read the data from the descriptor, and the dma_wmb() allows us to guarantee the data is written to the descriptor before the device can see it now has ownership. The dma_mb() implies both a dma_rmb() and - a dma_wmb(). Note that, when using writel(), a prior wmb() is not needed
Documentation/memory-barriers.txt There are some more advanced barrier functions: · line 1936
Quoted 10 lines
Documentation/memory-barriers.txt +6
-3- to guarantee that the cache coherent memory writes have completed before - writing to the MMIO region. The cheaper writel_relaxed() does not provide - this guarantee and must not be used here. + a dma_wmb(). Note that, when using writel(), a prior barrier is not + needed to guarantee that the cache coherent memory writes have completed + before writing to the MMIO region. The cheaper writel_relaxed() does not + provide this guarantee and must not be used here. Hence, writeX() is always + preferred which inserts needed platform specific barrier before writing to + the specified MMIO region.
Did you mean that writeX() is write() function family?
Signature · 1 line
An old man doll... just what I always wanted! - Clara
Quoted 24 lines
From: Bagas Sanjaya <[email protected]>
Sent: Thursday, October 6, 2022 7:31 AMReview There are some more advanced barrier
Quoted 1 line · Parav Pandit
(quoted diff) +0
-0@@ -1927,10 +1927,12 @@ There are some more advanced barrierfunctions:
Review There are some more advanced barrier · line 1927
Quoted 1 line
(quoted diff) +0
-0before we read the data from the descriptor, and the dma_wmb()allows
Review There are some more advanced barrier · line 1929
Quoted 2 lines
(quoted diff) +0
-0us to guarantee the data is written to the descriptor before the device can see it now has ownership. The dma_mb() implies both a
dma_rmb()andReview There are some more advanced barrier · line 1930
Quoted 1 line
(quoted diff) +0
-1- a dma_wmb(). Note that, when using writel(), a prior wmb() is notneeded
Review There are some more advanced barrier · line 1931
Quoted 1 line
(quoted diff) +0
-1- to guarantee that the cache coherent memory writes have completedbefore
Review There are some more advanced barrier · line 1932
Quoted 1 line
(quoted diff) +0
-1- writing to the MMIO region. The cheaper writel_relaxed() does notprovide
Documentation/memory-barriers.txt There are some more advanced barrier functions:
Quoted 3 lines
Documentation/memory-barriers.txt +2
-1- this guarantee and must not be used here. + a dma_wmb(). Note that, when using writel(), a prior barrier is not + needed to guarantee that the cache coherent memory writes havecompleted
Review There are some more advanced barrier · line 1932
Quoted 1 line
(quoted diff) +1
-0+ before writing to the MMIO region. The cheaper writel_relaxed() doesnot
Quoted 8 lines
Review There are some more advanced barrier · line 1933
Quoted 1 line
(quoted diff) +1
-0+ provide this guarantee and must not be used here. Hence, writeX() isalways
Review There are some more advanced barrier · line 1934
Quoted 1 line
(quoted diff) +1
-0+ preferred which inserts needed platform specific barrier before writingto
Documentation/memory-barriers.txt There are some more advanced barrier functions: · line 1936
Quoted 2 lines
Documentation/memory-barriers.txt +1
-0+ the specified MMIO region.Did you mean that
writeX()iswrite()function family?
Yes.
Akira Yokosawa
reviewerHi,
Maybe I'm too nit-picky, but see below:
Quoted 8 lines · Parav Pandit
The cited commit describes that when using
writel(), explcitwmb()is not needed.wmb()is an expensive barrier.writel()uses the needed platform specific barrier instead of expensivewmb().Hence update the example to be more accurate that matches the current implementation.
commit
5846581e3563("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example")
You can cite the commit in the Changelog text.
Just say:
Commit 5846581e3563 ("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example") describes that when using writel(), ...
Also, a blank line is needed above S-o-b tags as a delimiter.
Quoted 20 lines
Signed-off-by Parav Pandit <[email protected]>---
changelog:v1->v2:
- Further improved description ofwritel()example
- changed commit subject from 'usage' to 'example'v0->v1:
- Corrected to mention I/O barrier instead ofdma_wmb().
- removed numbered references in commit log
- corrected typo 'explcit' to 'explicit' in commit log--- Documentation/memory-barriers.txt | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)Documentation/memory-barriers.txt +0
-0
Documentation/memory-barriers.txt There are some more advanced barrier functions: · line 1930
Quoted 8 lines
Documentation/memory-barriers.txt +1
-4@@ -1927,10 +1927,12 @@ There are some more advanced barrier functions: before we read the data from the descriptor, and the dma_wmb() allows us to guarantee the data is written to the descriptor before the device can see it now has ownership. The dma_mb() implies both a dma_rmb() and - a dma_wmb(). Note that, when using writel(), a prior wmb() is not needed - to guarantee that the cache coherent memory writes have completed before - writing to the MMIO region. The cheaper writel_relaxed() does not provide - this guarantee and must not be used here. + a dma_wmb(). Note that, when using writel(), a prior barrier is not
If you permit a slightly long line here, this hunk would be much easier to compare:
+ a dma_wmb(). Note that, when using writel(), a prior barrier is not needed
to guarantee that the cache coherent memory writes have completed before writing to the MMIO region. The cheaper writel_relaxed() does not provide
+ this guarantee and must not be used here. Hence, writeX() is always + preferred which inserts needed platform specific barrier before writing to + the specified MMIO region.
That said, I don't feel comfortable with the sentence you added. It looks to me it is redundant because such a guarantee of writeX() is already covered in the section of "KERNEL I/O BARRIER EFFECTS".
See the relevant explanation quoted below:
3. A writeX() by a CPU thread to the peripheral will first wait for the completion of all prior writes to memory either issued by, or propagated to, the same thread. This ensures that writes by the CPU to an outbound DMA buffer allocated by dma_alloc_coherent() will be visible to a DMA engine when the CPU writes to its MMIO control register to trigger the transfer.
Also please not that this document should not describe any implementation details of those accessors. This document is not meant as an implementation guide, but a guide for kernel developers who use them. This is clearly mentioned in "DISCLAIMER" at the top of this file.
Thanks, Akira
Quoted 8 lines
Documentation/memory-barriers.txt +5
-0+ needed to guarantee that the cache coherent memory writes have completed + before writing to the MMIO region. The cheaper writel_relaxed() does not + provide this guarantee and must not be used here. Hence, writeX() is always + preferred which inserts needed platform specific barrier before writing to + the specified MMIO region. See the subsection "Kernel I/O barrier effects" for more information on relaxed I/O accessors and the Documentation/core-api/dma-api.rst file for
Akira Yokosawa
reviewerQuoted 65 lines · Akira Yokosawa
Hi,
Maybe I'm too nit-picky, but see below:
Quoted 8 lines · Parav Pandit
The cited commit describes that when using
writel(), explcitwmb()is not needed.wmb()is an expensive barrier.writel()uses the needed platform specific barrier instead of expensivewmb().Hence update the example to be more accurate that matches the current implementation.
commit
5846581e3563("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example")You can cite the commit in the Changelog text.
Just say:Commit
5846581e3563("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example") describes that when usingwritel(), ...Also, a blank line is needed above S-o-b tags as a delimiter.
Quoted 20 lines
Signed-off-by Parav Pandit <[email protected]>---
changelog:v1->v2:
- Further improved description ofwritel()example
- changed commit subject from 'usage' to 'example'v0->v1:
- Corrected to mention I/O barrier instead ofdma_wmb().
- removed numbered references in commit log
- corrected typo 'explcit' to 'explicit' in commit log--- Documentation/memory-barriers.txt | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-)Documentation/memory-barriers.txt +0
-0Documentation/memory-barriers.txt There are some more advanced barrier functions: · line 1930
Quoted 8 lines
Documentation/memory-barriers.txt +1
-4@@ -1927,10 +1927,12 @@ There are some more advanced barrier functions: before we read the data from the descriptor, and the dma_wmb() allows us to guarantee the data is written to the descriptor before the device can see it now has ownership. The dma_mb() implies both a dma_rmb() and - a dma_wmb(). Note that, when using writel(), a prior wmb() is not needed - to guarantee that the cache coherent memory writes have completed before - writing to the MMIO region. The cheaper writel_relaxed() does not provide - this guarantee and must not be used here. + a dma_wmb(). Note that, when using writel(), a prior barrier is notIf you permit a slightly long line here, this hunk would be much easier to compare:
(quoted diff) +4
-0+ a dma_wmb(). Note that, when using writel(), a prior barrier is not needed to guarantee that the cache coherent memory writes have completed before writing to the MMIO region. The cheaper writel_relaxed() does not provide + this guarantee and must not be used here. Hence, writeX() is always + preferred which inserts needed platform specific barrier before writing to + the specified MMIO region.That said, I don't feel comfortable with the sentence you added. It looks to me it is redundant because such a guarantee of
writeX()is already covered in the section of "KERNEL I/O BARRIER EFFECTS".
See the relevant explanation quoted below:
Quoted 8 lines
3. A
writeX()by a CPU thread to the peripheral will first wait for the completion of all prior writes to memory either issued by, or propagated to, the same thread. This ensures that writes by the CPU to an outbound DMA buffer allocated bydma_alloc_coherent()will be visible to a DMA engine when the CPU writes to its MMIO control register to trigger the transfer.Also please not that this document should not describe any implementation
I meant note
Quoted 14 lines
details of those accessors. This document is not meant as an implementation guide, but a guide for kernel developers who use them. This is clearly mentioned in "DISCLAIMER" at the top of this file.
Thanks, Akira
Quoted 8 lines
Documentation/memory-barriers.txt +5
-0+ needed to guarantee that the cache coherent memory writes have completed + before writing to the MMIO region. The cheaper writel_relaxed() does not + provide this guarantee and must not be used here. Hence, writeX() is always + preferred which inserts needed platform specific barrier before writing to + the specified MMIO region. See the subsection "Kernel I/O barrier effects" for more information on relaxed I/O accessors and the Documentation/core-api/dma-api.rst file for
Quoted 13 lines
From: Akira Yokosawa <[email protected]>
Sent: Thursday, October 6, 2022 8:18 AMHi,
Maybe I'm too nit-picky, but see below:
Quoted 4 lines · Parav Pandit
The cited commit describes that when using
writel(), explcitwmb()is not needed.wmb()is an expensive barrier.writel()uses the needed platform specific barrier instead of expensivewmb().
Quoted 9 lines
Quoted 5 lines
Hence update the example to be more accurate that matches the current implementation.
commit
5846581e3563("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example")You can cite the commit in the Changelog text.
Just say:
I prefer to annotate the long commit as a separate line. This makes the description sentence easy to read.
Quoted 5 lines
Commit
5846581e3563("locking/memory-barriers.txt: Fix broken DMA vs. MMIO ordering example") describes that when usingwritel(), ...Also, a blank line is needed above S-o-b tags as a delimiter.
Adding blank line in v3.
[..]
Quoted 3 lines
If you permit a slightly long line here, this hunk would be much easier to compare:
True, but document crosses the 80 lines. So I will keep the existing alignment of 80 characters per line.
Quoted 11 lines
+ a
dma_wmb(). Note that, when usingwritel(), a prior barrier is
+ not needed
to guarantee that the cache coherent memory writes have completed
before
writing to the MMIO region. The cheaperwritel_relaxed()does not
provide(quoted diff) +2
-0+ this guarantee and must not be used here. Hence, writeX() is always + preferred which inserts needed platform specific barrier before writingto
+ the specified MMIO region.
Quoted 5 lines
That said, I don't feel comfortable with the sentence you added. It looks to me it is redundant because such a guarantee of
writeX()is already covered in the section of "KERNEL I/O BARRIER EFFECTS".
See the relevant explanation quoted below:
I observed that too, but I thought it is worth short note to add along with this example. But I agree since the next para refers to it anyway, I will drop this 2nd addition in v3.
Quoted 9 lines
3. A
writeX()by a CPU thread to the peripheral will first wait for the completion of all prior writes to memory either issued by, or propagated to, the same thread. This ensures that writes by the
CPU
to an outbound DMA buffer allocated bydma_alloc_coherent()will
be
visible to a DMA engine when the CPU writes to its MMIO control register to trigger the transfer.
Quoted 5 lines
Also please not that this document should not describe any implementation details of those accessors. This document is not meant as an implementation guide, but a guide for kernel developers who use them. This is clearly mentioned in "DISCLAIMER" at the top of this file.
In the disclaimer section I didn’t see any objection to "describe any implementation". :)
For example, disclaimer itself tells that a barrier may be no-op talking about a possible implementation on a platform.
But I agree to your above point that above #3 already covers with my addition. Hence, I will drop that part in v3.
Thanks for the review.
Quoted 15 lines
Thanks, Akira
Quoted 1 line
+ needed to guarantee that the cache coherent memory writes have
completed
Quoted 1 line
+ before writing to the MMIO region. The cheaper
writel_relaxed()doesnot
Quoted 1 line
+ provide this guarantee and must not be used here. Hence,
writeX()isalways
Quoted 1 line
+ preferred which inserts needed platform specific barrier before writing
to
Quoted 5 lines
+ the specified MMIO region.
See the subsection "Kernel I/O barrier effects" for more information on relaxed I/O accessors and the
Documentation/core-api/dma-api.rst
file for