Documentation updates and fixes
Subsystems (3)
cover Documentation updates and fixes
Hello!
This patch series adds some documentation and fixes to the Linux-kernel memory model (LKMM):
1. Add more text to meomry-barrier.txt to de-emphasize
smp_read_barrier_depends().
2. Remove mention of the now-obsolete docker and gentoo images
from README.
3. Add documentation for the new ISA2+pooncelock+pooncelock+pombonce
litmus test.
4. Update the removal of rb-dep, smp_read_barrier_depends(), and
lockless_dereference() to reflect feedback.
Thanx, Paul
------------------------------------------------------------------------
Documentation/memory-barriers.txt | 26 +++++++++++++++-------- tools/memory-model/Documentation/cheatsheet.txt | 6 ++--- tools/memory-model/Documentation/explanation.txt | 4 +-- tools/memory-model/README | 15 ------------- tools/memory-model/linux-kernel.def | 2 - tools/memory-model/litmus-tests/README | 6 +++++ 6 files changed, 30 insertions(+), 29 deletions(-)
PATCH 1/4 memory-barriers: More text to de-emphasize smp_read_barrier_depends()
This commit makes further changes to memory-barrier.txt to further de-emphasize smp_read_barrier_depends(), but leaving some discussion for historical purposes.
--- Documentation/memory-barriers.txt | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-)
Documentation/memory-barriers.txt +18 -8
@@ -52,7 +52,7 @@ CONTENTS - Varieties of memory barrier. - What may not be assumed about memory barriers? - - Data dependency barriers. + - Data dependency barriers (historical). - Control dependencies. - SMP barrier pairing. - Examples of memory barrier sequences.
@@ -554,8 +554,15 @@ There are certain things that the Linux kernel memory barriers do not guarantee: Documentation/DMA-API.txt -DATA DEPENDENCY BARRIERS ------------------------- +DATA DEPENDENCY BARRIERS (HISTORICAL) +------------------------------------- + +As of v4.15 of the Linux kernel, an smp_read_barrier_depends() was +added to READ_ONCE(), which means that about the only people who +need to pay attention to this section are those working on DEC Alpha +architecture-specific code and those working on READ_ONCE() itself. +For those who need it, and for those who are interested in the history, +here is the story of data-dependency barriers. The usage requirements of data dependency barriers are a little subtle, and it's not always obvious that they're needed. To illustrate, consider the
@@ -2843,8 +2850,9 @@ as that committed on CPU 1. To intervene, we need to interpolate a data dependency barrier or a read -barrier between the loads. This will force the cache to commit its coherency -queue before processing any further requests: +barrier between the loads (which as of v4.15 is supplied unconditionally +by the READ_ONCE() macro). This will force the cache to commit its +coherency queue before processing any further requests: CPU 1 CPU 2 COMMENT =============== =============== =======================================
@@ -2873,8 +2881,8 @@ Other CPUs may also have split caches, but must coordinate between the various cachelets for normal memory accesses. The semantics of the Alpha removes the need for hardware coordination in the absence of memory barriers, which permitted Alpha to sport higher CPU clock rates back in the day. However, -please note that smp_read_barrier_depends() should not be used except in -Alpha arch-specific code and within the READ_ONCE() macro. +please note that (again, as of v4.15) smp_read_barrier_depends() should not +be used except in Alpha arch-specific code and within the READ_ONCE() macro. CACHE COHERENCY VS DMA
@@ -3039,7 +3047,9 @@ the data dependency barrier really becomes necessary as this synchronises both caches with the memory coherence system, thus making it seem like pointer changes vs new data occur in the right order. -The Alpha defines the Linux kernel's memory barrier model. +The Alpha defines the Linux kernel's memory model, although as of v4.15 +the Linux kernel's addition of smp_read_barrier_depends() to READ_ONCE() +greatly reduced Alpha's impact on the memory model. See the subsection on "Cache Coherency" above.
Signature · 1 line
2.5.2
PATCH 3/4 tools/memory-order: Add documentation of new litmus test
The litmus-tests/README file lacks any mention of the new litmus test ISA2+pooncelock+pooncelock+pombonce.litmus. This commit therefore adds a description of this test.
--- tools/memory-model/litmus-tests/README | 6 ++++++ 1 file changed, 6 insertions(+)
tools/memory-model/litmus-tests/README +6 -0
@@ -32,6 +32,12 @@ IRIW+poonceonces+OnceOnce.litmus order of a pair of writes, where each write is to a different variable by a different process? +ISA2+pooncelock+pooncelock+pombonce.litmus + Tests whether the ordering provided by a lock-protected S + litmus test is visible to an external process whose accesses are + separated by smp_mb(). This addition of an external process to + S is otherwise known as ISA2. + ISA2+poonceonces.litmus As below, but with store-release replaced with WRITE_ONCE() and load-acquire replaced with READ_ONCE().
Signature · 1 line
2.5.2
PATCH 4/4 tools/memory-model: Update: Remove rb-dep, smp_read_barrier_depends, and lockless_dereference
In reply to Paul E. McKenneyFrom: Alan Stern <[email protected]>
Commit bf28ae562744 ("tools/memory-model: Remove rb-dep, smp_read_barrier_depends, and lockless_dereference") was accidentally merged too early, while it was still in RFC form. This patch adds in the missing pieces.
Akira pointed out some typos in the original patch, and he noted that cheatsheet.txt should indicate that READ_ONCE() now implies an address dependency. Andrea suggested documenting the relationship betwwen unsuccessful RMW operations and address dependencies.
Andrea pointed out that the macro for rcu_dereference() in linux.def should now use the "once" annotation instead of "deref". He also suggested that the comments should mention commit 5a8897cc7631 ("locking/atomics/alpha: Add smp_read_barrier_depends() to _release()/_relaxed() atomics") as an important precursor, and he contributed commit cb13b424e986 ("locking/xchg/alpha: Add unconditional memory barrier to cmpxchg()"), another prerequisite.
--- tools/memory-model/Documentation/cheatsheet.txt | 6 +++--- tools/memory-model/Documentation/explanation.txt | 4 ++-- tools/memory-model/linux-kernel.def | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-)
tools/memory-model/Documentation/cheatsheet.txt +3 -3
@@ -1,11 +1,11 @@ Prior Operation Subsequent Operation --------------- --------------------------- C Self R W RWM Self R W DR DW RMW SV - __ ---- - - --- ---- - - -- -- --- -- + -- ---- - - --- ---- - - -- -- --- -- Store, e.g., WRITE_ONCE() Y Y -Load, e.g., READ_ONCE() Y Y Y -Unsuccessful RMW operation Y Y Y +Load, e.g., READ_ONCE() Y Y Y Y +Unsuccessful RMW operation Y Y Y Y rcu_dereference() Y Y Y Y Successful *_acquire() R Y Y Y Y Y Y Successful *_release() C Y Y Y W Y
tools/memory-model/Documentation/explanation.txt +2 -2
@@ -826,7 +826,7 @@ A-cumulative; they only affect the propagation of stores that are executed on C before the fence (i.e., those which precede the fence in program order). -read_lock(), rcu_read_unlock(), and synchronize_rcu() fences have +rcu_read_lock(), rcu_read_unlock(), and synchronize_rcu() fences have other properties which we discuss later.
@@ -1138,7 +1138,7 @@ final effect is that even though the two loads really are executed in program order, it appears that they aren't. This could not have happened if the local cache had processed the -incoming stores in FIFO order. In constrast, other architectures +incoming stores in FIFO order. By contrast, other architectures maintain at least the appearance of FIFO order. In practice, this difficulty is solved by inserting a special fence
tools/memory-model/linux-kernel.def +1 -1
@@ -13,7 +13,7 @@ WRITE_ONCE(X,V) { __store{once}(X,V); } smp_store_release(X,V) { __store{release}(*X,V); } smp_load_acquire(X) __load{acquire}(*X) rcu_assign_pointer(X,V) { __store{release}(X,V); } -rcu_dereference(X) __load{deref}(X) +rcu_dereference(X) __load{once}(X) // Fences smp_mb() { __fence{mb} ; }
Signature · 1 line
2.5.2
PATCH 2/4 tools/memory-model: Remove mention of docker/gentoo image
In reply to Paul E. McKenneyBecause the docker and gentoo images haven't been updated in quite some time, they are likely to provide more confusion than help. This commit therefore removes mention of them from the README file.
--- tools/memory-model/README | 15 --------------- 1 file changed, 15 deletions(-)
tools/memory-model/README +0 -15
@@ -27,21 +27,6 @@ separately: See "herdtools7/INSTALL.md" for installation instructions. -Alternatively, Abhishek Bhardwaj has kindly provided a Docker image -of these tools at "abhishek40/memory-model". Abhishek suggests the -following commands to install and use this image: - - - Users should install Docker for their distribution. - - docker run -itd abhishek40/memory-model - - docker attach <id-emitted-from-the-previous-command> - -Gentoo users might wish to make use of Patrick McLean's package: - - https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-util/herdtools7 - -These packages may not be up-to-date with respect to the GitHub -repository. - ================== BASIC USAGE: HERD7
Signature · 1 line
2.5.2
tip-bot for Paul E. McKenney
merge bot[tip:locking/core] locking/memory-barriers: De-emphasize smp_read_barrier_depends() some more
In reply to Paul E. McKenneylocking/memory-barriers: De-emphasize smp_read_barrier_depends() some more
Commit-ID: f28f0868feb1e79b460131bac37230e303a5f6a4 Gitweb: https://git.kernel.org/tip/f28f0868feb1e79b460131bac37230e303a5f6a4 Author: Paul E. McKenney <[email protected]> AuthorDate: Wed, 7 Mar 2018 09:27:37 -0800 Committer: Ingo Molnar <[email protected]> CommitDate: Sat, 10 Mar 2018 10:22:22 +0100
locking/memory-barriers: De-emphasize smp_read_barrier_depends() some more
This commit makes further changes to memory-barrier.txt to further de-emphasize smp_read_barrier_depends(), but leaving some discussion for historical purposes.
--- Documentation/memory-barriers.txt | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-)
Documentation/memory-barriers.txt +18 -8
@@ -52,7 +52,7 @@ CONTENTS - Varieties of memory barrier. - What may not be assumed about memory barriers? - - Data dependency barriers. + - Data dependency barriers (historical). - Control dependencies. - SMP barrier pairing. - Examples of memory barrier sequences.
@@ -554,8 +554,15 @@ There are certain things that the Linux kernel memory barriers do not guarantee: Documentation/DMA-API.txt -DATA DEPENDENCY BARRIERS ------------------------- +DATA DEPENDENCY BARRIERS (HISTORICAL) +------------------------------------- + +As of v4.15 of the Linux kernel, an smp_read_barrier_depends() was +added to READ_ONCE(), which means that about the only people who +need to pay attention to this section are those working on DEC Alpha +architecture-specific code and those working on READ_ONCE() itself. +For those who need it, and for those who are interested in the history, +here is the story of data-dependency barriers. The usage requirements of data dependency barriers are a little subtle, and it's not always obvious that they're needed. To illustrate, consider the
@@ -2843,8 +2850,9 @@ as that committed on CPU 1. To intervene, we need to interpolate a data dependency barrier or a read -barrier between the loads. This will force the cache to commit its coherency -queue before processing any further requests: +barrier between the loads (which as of v4.15 is supplied unconditionally +by the READ_ONCE() macro). This will force the cache to commit its +coherency queue before processing any further requests: CPU 1 CPU 2 COMMENT =============== =============== =======================================
@@ -2873,8 +2881,8 @@ Other CPUs may also have split caches, but must coordinate between the various cachelets for normal memory accesses. The semantics of the Alpha removes the need for hardware coordination in the absence of memory barriers, which permitted Alpha to sport higher CPU clock rates back in the day. However, -please note that smp_read_barrier_depends() should not be used except in -Alpha arch-specific code and within the READ_ONCE() macro. +please note that (again, as of v4.15) smp_read_barrier_depends() should not +be used except in Alpha arch-specific code and within the READ_ONCE() macro. CACHE COHERENCY VS DMA
@@ -3039,7 +3047,9 @@ the data dependency barrier really becomes necessary as this synchronises both caches with the memory coherence system, thus making it seem like pointer changes vs new data occur in the right order. -The Alpha defines the Linux kernel's memory barrier model. +The Alpha defines the Linux kernel's memory model, although as of v4.15 +the Linux kernel's addition of smp_read_barrier_depends() to READ_ONCE() +greatly reduced Alpha's impact on the memory model. See the subsection on "Cache Coherency" above.
tip-bot for Paul E. McKenney
merge bot[tip:locking/core] tools/memory-model: Remove mention of docker/gentoo image
In reply to Paul E. McKenneytools/memory-model: Remove mention of docker/gentoo image
Commit-ID: d095c12c53c7b941ad4ea96dc229a08296b37d2e Gitweb: https://git.kernel.org/tip/d095c12c53c7b941ad4ea96dc229a08296b37d2e Author: Paul E. McKenney <[email protected]> AuthorDate: Wed, 7 Mar 2018 09:27:38 -0800 Committer: Ingo Molnar <[email protected]> CommitDate: Sat, 10 Mar 2018 10:22:23 +0100
tools/memory-model: Remove mention of docker/gentoo image
Because the docker and gentoo images haven't been updated in quite some time, they are likely to provide more confusion than help. This commit therefore removes mention of them from the README file.
--- tools/memory-model/README | 15 --------------- 1 file changed, 15 deletions(-)
tools/memory-model/README +0 -15
@@ -27,21 +27,6 @@ separately: See "herdtools7/INSTALL.md" for installation instructions. -Alternatively, Abhishek Bhardwaj has kindly provided a Docker image -of these tools at "abhishek40/memory-model". Abhishek suggests the -following commands to install and use this image: - - - Users should install Docker for their distribution. - - docker run -itd abhishek40/memory-model - - docker attach <id-emitted-from-the-previous-command> - -Gentoo users might wish to make use of Patrick McLean's package: - - https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-util/herdtools7 - -These packages may not be up-to-date with respect to the GitHub -repository. - ================== BASIC USAGE: HERD7
tip-bot for Paul E. McKenney
merge bot[tip:locking/core] tools/memory-model: Add documentation of new litmus test
In reply to Paul E. McKenneytools/memory-model: Add documentation of new litmus test
Commit-ID: ff1fe5e079730f138c98b268ce2e8482a1d954b4 Gitweb: https://git.kernel.org/tip/ff1fe5e079730f138c98b268ce2e8482a1d954b4 Author: Paul E. McKenney <[email protected]> AuthorDate: Wed, 7 Mar 2018 09:27:39 -0800 Committer: Ingo Molnar <[email protected]> CommitDate: Sat, 10 Mar 2018 10:22:23 +0100
tools/memory-model: Add documentation of new litmus test
The litmus-tests/README file lacks any mention of the new litmus test ISA2+pooncelock+pooncelock+pombonce.litmus. This commit therefore adds a description of this test.
--- tools/memory-model/litmus-tests/README | 6 ++++++ 1 file changed, 6 insertions(+)
tools/memory-model/litmus-tests/README +6 -0
@@ -32,6 +32,12 @@ IRIW+poonceonces+OnceOnce.litmus order of a pair of writes, where each write is to a different variable by a different process? +ISA2+pooncelock+pooncelock+pombonce.litmus + Tests whether the ordering provided by a lock-protected S + litmus test is visible to an external process whose accesses are + separated by smp_mb(). This addition of an external process to + S is otherwise known as ISA2. + ISA2+poonceonces.litmus As below, but with store-release replaced with WRITE_ONCE() and load-acquire replaced with READ_ONCE().
tip-bot for Alan Stern
merge bot[tip:locking/core] tools/memory-model: Finish the removal of rb-dep, smp_read_barrier_depends(), and lockless_dereference()
In reply to Paul E. McKenneytools/memory-model: Finish the removal of rb-dep, smp_read_barrier_depends(), and lockless_dereference()
Commit-ID: bd5c0ba2cd78a4c116726ead84f8f37dc92d043e Gitweb: https://git.kernel.org/tip/bd5c0ba2cd78a4c116726ead84f8f37dc92d043e Author: Alan Stern <[email protected]> AuthorDate: Wed, 7 Mar 2018 09:27:40 -0800 Committer: Ingo Molnar <[email protected]> CommitDate: Sat, 10 Mar 2018 10:22:23 +0100
tools/memory-model: Finish the removal of rb-dep, smp_read_barrier_depends(), and lockless_dereference()
Commit:
bf28ae562744 ("tools/memory-model: Remove rb-dep, smp_read_barrier_depends, and lockless_dereference")
was merged too early, while it was still in RFC form. This patch adds in the missing pieces.
Akira pointed out some typos in the original patch, and he noted that cheatsheet.txt should indicate that READ_ONCE() now implies an address dependency. Andrea suggested documenting the relationship betwwen unsuccessful RMW operations and address dependencies.
Andrea pointed out that the macro for rcu_dereference() in linux.def should now use the "once" annotation instead of "deref". He also suggested that the comments should mention commit:
5a8897cc7631 ("locking/atomics/alpha: Add smp_read_barrier_depends() to _release()/_relaxed() atomics")
... as an important precursor, and he contributed commit:
cb13b424e986 ("locking/xchg/alpha: Add unconditional memory barrier to cmpxchg()")
which is another prerequisite.
--- tools/memory-model/Documentation/cheatsheet.txt | 6 +++--- tools/memory-model/Documentation/explanation.txt | 4 ++-- tools/memory-model/linux-kernel.def | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-)
tools/memory-model/Documentation/cheatsheet.txt +3 -3
@@ -1,11 +1,11 @@ Prior Operation Subsequent Operation --------------- --------------------------- C Self R W RWM Self R W DR DW RMW SV - __ ---- - - --- ---- - - -- -- --- -- + -- ---- - - --- ---- - - -- -- --- -- Store, e.g., WRITE_ONCE() Y Y -Load, e.g., READ_ONCE() Y Y Y -Unsuccessful RMW operation Y Y Y +Load, e.g., READ_ONCE() Y Y Y Y +Unsuccessful RMW operation Y Y Y Y rcu_dereference() Y Y Y Y Successful *_acquire() R Y Y Y Y Y Y Successful *_release() C Y Y Y W Y
tools/memory-model/Documentation/explanation.txt +2 -2
@@ -826,7 +826,7 @@ A-cumulative; they only affect the propagation of stores that are executed on C before the fence (i.e., those which precede the fence in program order). -read_lock(), rcu_read_unlock(), and synchronize_rcu() fences have +rcu_read_lock(), rcu_read_unlock(), and synchronize_rcu() fences have other properties which we discuss later.
@@ -1138,7 +1138,7 @@ final effect is that even though the two loads really are executed in program order, it appears that they aren't. This could not have happened if the local cache had processed the -incoming stores in FIFO order. In constrast, other architectures +incoming stores in FIFO order. By contrast, other architectures maintain at least the appearance of FIFO order. In practice, this difficulty is solved by inserting a special fence
tools/memory-model/linux-kernel.def +1 -1
@@ -13,7 +13,7 @@ WRITE_ONCE(X,V) { __store{once}(X,V); } smp_store_release(X,V) { __store{release}(*X,V); } smp_load_acquire(X) __load{acquire}(*X) rcu_assign_pointer(X,V) { __store{release}(X,V); } -rcu_dereference(X) __load{deref}(X) +rcu_dereference(X) __load{once}(X) // Fences smp_mb() { __fence{mb} ; }