KCSAN updates for v5.8
Subsystems (9)
Paul E. McKenney
maintainercover KCSAN updates for v5.8
Hello!
This series contains KCSAN updates. Unless otherwise noted, these are all courtesy of Marco Elver.
1. Add option to allow watcher interruptions.
2. Add option for verbose reporting.
3. Add current->state to implicitly atomic accesses.
4. Fix a typo in a comment, courtesy of Qiujun Huang.
5. Update Documentation/dev-tools/kcsan.rst.
6. Update API documentation in kcsan-checks.h.
7. Introduce report access_info and other_info.
8. Avoid blocking producers in prepare_report().
9. Add support for scoped accesses.
10. objtool, kcsan: Add explicit check functions to uaccess whitelist.
11. Introduce scoped ASSERT_EXCLUSIVE macros.
12. Move kcsan_{disable,enable}_current() to kcsan-checks.h.
13. Change data_race() to no longer require marking racing accesses.
14. Fix function matching in report.
15. Make reporting aware of KCSAN tests.
Thanx, Paul
------------------------------------------------------------------------
Documentation/dev-tools/kcsan.rst | 230 ++++++++++------ include/linux/compiler.h | 4 include/linux/kcsan-checks.h | 244 ++++++++++++++--- include/linux/kcsan.h | 19 - init/init_task.c | 1 kernel/kcsan/atomic.h | 21 - kernel/kcsan/core.c | 182 ++++++++---- kernel/kcsan/debugfs.c | 43 ++- kernel/kcsan/kcsan.h | 8 kernel/kcsan/report.c | 543 +++++++++++++++++++++++--------------- lib/Kconfig.kcsan | 24 + tools/objtool/check.c | 2 12 files changed, 887 insertions(+), 434 deletions(-)
[email protected]
maintainerPATCH 1/10 tools/memory-model: Add recent references
From: "Paul E. McKenney" <[email protected]>
This commit updates the list of LKMM-related publications in Documentation/references.txt.
--- tools/memory-model/Documentation/references.txt | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-)
tools/memory-model/Documentation/references.txt +19 -2
@@ -73,6 +73,18 @@ o Christopher Pulte, Shaked Flur, Will Deacon, Jon French, Linux-kernel memory model ========================= +o Jade Alglave, Will Deacon, Boqun Feng, David Howells, Daniel + Lustig, Luc Maranget, Paul E. McKenney, Andrea Parri, Nicholas + Piggin, Alan Stern, Akira Yokosawa, and Peter Zijlstra. + 2019. "Calibrating your fear of big bad optimizing compilers" + Linux Weekly News. https://lwn.net/Articles/799218/ + +o Jade Alglave, Will Deacon, Boqun Feng, David Howells, Daniel + Lustig, Luc Maranget, Paul E. McKenney, Andrea Parri, Nicholas + Piggin, Alan Stern, Akira Yokosawa, and Peter Zijlstra. + 2019. "Who's afraid of a big bad optimizing compiler?" + Linux Weekly News. https://lwn.net/Articles/793253/ + o Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and Alan Stern. 2018. "Frightening small children and disconcerting grown-ups: Concurrency in the Linux kernel". In Proceedings of
@@ -88,6 +100,11 @@ o Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and Alan Stern. 2017. "A formal kernel memory-ordering model (part 2)" Linux Weekly News. https://lwn.net/Articles/720550/ +o Jade Alglave, Luc Maranget, Paul E. McKenney, Andrea Parri, and + Alan Stern. 2017-2019. "A Formal Model of Linux-Kernel Memory + Ordering" (backup material for the LWN articles) + https://mirrors.edge.kernel.org/pub/linux/kernel/people/paulmck/LWNLinuxMM/ + Memory-model tooling ====================
@@ -110,5 +127,5 @@ Memory-model comparisons ======================== o Paul E. McKenney, Ulrich Weigand, Andrea Parri, and Boqun - Feng. 2016. "Linux-Kernel Memory Model". (6 June 2016). - http://open-std.org/JTC1/SC22/WG21/docs/papers/2016/p0124r2.html. + Feng. 2018. "Linux-Kernel Memory Model". (27 September 2018). + http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0124r6.html.
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH 2/10 tools/memory-model: Fix "conflict" definition
In reply to Paul E. McKenneyFrom: Marco Elver <[email protected]>
The definition of "conflict" should not include the type of access nor whether the accesses are concurrent or not, which this patch addresses. The definition of "data race" remains unchanged.
The definition of "conflict" as we know it and is cited by various papers on memory consistency models appeared in [1]: "Two accesses to the same variable conflict if at least one is a write; two operations conflict if they execute conflicting accesses."
The LKMM as well as the C11 memory model are adaptations of data-race-free, which are based on the work in [2]. Necessarily, we need both conflicting data operations (plain) and synchronization operations (marked). For example, C11's definition is based on [3], which defines a "data race" as: "Two memory operations conflict if they access the same memory location, and at least one of them is a store, atomic store, or atomic read-modify-write operation. In a sequentially consistent execution, two memory operations from different threads form a type 1 data race if they conflict, at least one of them is a data operation, and they are adjacent in <T (i.e., they may be executed concurrently)."
[1] D. Shasha, M. Snir, "Efficient and Correct Execution of Parallel Programs that Share Memory", 1988.
URL: http://snir.cs.illinois.edu/listed/J21.pdf
[2] S. Adve, "Designing Memory Consistency Models for Shared-Memory
Multiprocessors", 1993. URL: http://sadve.cs.illinois.edu/Publications/thesis.pdf
[3] H.-J. Boehm, S. Adve, "Foundations of the C++ Concurrency Memory
Model", 2008. URL: https://www.hpl.hp.com/techreports/2008/HPL-2008-56.pdf
--- tools/memory-model/Documentation/explanation.txt | 83 +++++++++++++----------- 1 file changed, 45 insertions(+), 38 deletions(-)
tools/memory-model/Documentation/explanation.txt +45 -38
@@ -1987,28 +1987,36 @@ outcome undefined. In technical terms, the compiler is allowed to assume that when the program executes, there will not be any data races. A "data race" -occurs when two conflicting memory accesses execute concurrently; -two memory accesses "conflict" if: +occurs when there are two memory accesses such that: - they access the same location, +1. they access the same location, - they occur on different CPUs (or in different threads on the - same CPU), +2. at least one of them is a store, - at least one of them is a plain access, +3. at least one of them is plain, - and at least one of them is a store. +4. they occur on different CPUs (or in different threads on the + same CPU), and -The LKMM tries to determine whether a program contains two conflicting -accesses which may execute concurrently; if it does then the LKMM says -there is a potential data race and makes no predictions about the -program's outcome. +5. they execute concurrently. -Determining whether two accesses conflict is easy; you can see that -all the concepts involved in the definition above are already part of -the memory model. The hard part is telling whether they may execute -concurrently. The LKMM takes a conservative attitude, assuming that -accesses may be concurrent unless it can prove they cannot. +In the literature, two accesses are said to "conflict" if they satisfy +1 and 2 above. We'll go a little farther and say that two accesses +are "race candidates" if they satisfy 1 - 4. Thus, whether or not two +race candidates actually do race in a given execution depends on +whether they are concurrent. + +The LKMM tries to determine whether a program contains race candidates +which may execute concurrently; if it does then the LKMM says there is +a potential data race and makes no predictions about the program's +outcome. + +Determining whether two accesses are race candidates is easy; you can +see that all the concepts involved in the definition above are already +part of the memory model. The hard part is telling whether they may +execute concurrently. The LKMM takes a conservative attitude, +assuming that accesses may be concurrent unless it can prove they +are not. If two memory accesses aren't concurrent then one must execute before the other. Therefore the LKMM decides two accesses aren't concurrent
@@ -2171,8 +2179,8 @@ again, now using plain accesses for buf: } This program does not contain a data race. Although the U and V -accesses conflict, the LKMM can prove they are not concurrent as -follows: +accesses are race candidates, the LKMM can prove they are not +concurrent as follows: The smp_wmb() fence in P0 is both a compiler barrier and a cumul-fence. It guarantees that no matter what hash of
@@ -2326,12 +2334,11 @@ could now perform the load of x before the load of ptr (there might be a control dependency but no address dependency at the machine level). Finally, it turns out there is a situation in which a plain write does -not need to be w-post-bounded: when it is separated from the -conflicting access by a fence. At first glance this may seem -impossible. After all, to be conflicting the second access has to be -on a different CPU from the first, and fences don't link events on -different CPUs. Well, normal fences don't -- but rcu-fence can! -Here's an example: +not need to be w-post-bounded: when it is separated from the other +race-candidate access by a fence. At first glance this may seem +impossible. After all, to be race candidates the two accesses must +be on different CPUs, and fences don't link events on different CPUs. +Well, normal fences don't -- but rcu-fence can! Here's an example: int x, y;
@@ -2367,7 +2374,7 @@ concurrent and there is no race, even though P1's plain store to y isn't w-post-bounded by any marked accesses. Putting all this material together yields the following picture. For -two conflicting stores W and W', where W ->co W', the LKMM says the +race-candidate stores W and W', where W ->co W', the LKMM says the stores don't race if W can be linked to W' by a w-post-bounded ; vis ; w-pre-bounded
@@ -2380,8 +2387,8 @@ sequence, and if W' is plain then they also have to be linked by a w-post-bounded ; vis ; r-pre-bounded -sequence. For a conflicting load R and store W, the LKMM says the two -accesses don't race if R can be linked to W by an +sequence. For race-candidate load R and store W, the LKMM says the +two accesses don't race if R can be linked to W by an r-post-bounded ; xb* ; w-pre-bounded
@@ -2413,20 +2420,20 @@ is, the rules governing the memory subsystem's choice of a store to satisfy a load request and its determination of where a store will fall in the coherence order): - If R and W conflict and it is possible to link R to W by one - of the xb* sequences listed above, then W ->rfe R is not - allowed (i.e., a load cannot read from a store that it + If R and W are race candidates and it is possible to link R to + W by one of the xb* sequences listed above, then W ->rfe R is + not allowed (i.e., a load cannot read from a store that it executes before, even if one or both is plain). - If W and R conflict and it is possible to link W to R by one - of the vis sequences listed above, then R ->fre W is not - allowed (i.e., if a store is visible to a load then the load - must read from that store or one coherence-after it). + If W and R are race candidates and it is possible to link W to + R by one of the vis sequences listed above, then R ->fre W is + not allowed (i.e., if a store is visible to a load then the + load must read from that store or one coherence-after it). - If W and W' conflict and it is possible to link W to W' by one - of the vis sequences listed above, then W' ->co W is not - allowed (i.e., if one store is visible to a second then the - second must come after the first in the coherence order). + If W and W' are race candidates and it is possible to link W + to W' by one of the vis sequences listed above, then W' ->co W + is not allowed (i.e., if one store is visible to a second then + the second must come after the first in the coherence order). This is the extent to which the LKMM deals with plain accesses. Perhaps it could say more (for example, plain accesses might
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH 3/10 Documentation: LKMM: Move MP+onceassign+derefonce to new litmus-tests/rcu/
In reply to Paul E. McKenneyFrom: "Joel Fernandes (Google)" <[email protected]>
Move MP+onceassign+derefonce to the new Documentation/litmus-tests/rcu/ directory.
More RCU-related litmus tests would be added here.
---
Documentation/litmus-tests/README | 9 +++++++++
.../litmus-tests/rcu}/MP+onceassign+derefonce.litmus | 0
tools/memory-model/litmus-tests/README | 3 ---
3 files changed, 9 insertions(+), 3 deletions(-)
create mode 100644 Documentation/litmus-tests/README
rename {tools/memory-model/litmus-tests => Documentation/litmus-tests/rcu}/MP+onceassign+derefonce.litmus (100%)Documentation/litmus-tests/README new file +9 -0
@@ -0,0 +1,9 @@ +============ +LITMUS TESTS +============ + +RCU (/rcu directory) +-------------------- +MP+onceassign+derefonce.litmus + Demonstrates that rcu_assign_pointer() and rcu_dereference() to + ensure that an RCU reader will not see pre-initialization garbage.
Documentation/litmus-tests/rcu/MP+onceassign+derefonce.litmus renamed from tools/memory-model/litmus-tests/MP+onceassign+derefonce.litmus +0 -0
tools/memory-model/litmus-tests/README +0 -3
@@ -63,9 +63,6 @@ LB+poonceonces.litmus As above, but with store-release replaced with WRITE_ONCE() and load-acquire replaced with READ_ONCE(). -MP+onceassign+derefonce.litmus - As below, but with rcu_assign_pointer() and an rcu_dereference(). - MP+polockmbonce+poacquiresilsil.litmus Protect the access with a lock and an smp_mb__after_spinlock() in one process, and use an acquire load followed by a pair of
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH 4/10 Documentation: LKMM: Add litmus test for RCU GP guarantee where updater frees object
In reply to Paul E. McKenneyFrom: "Joel Fernandes (Google)" <[email protected]>
This adds an example for the important RCU grace period guarantee, which shows an RCU reader can never span a grace period.
--- .../litmus-tests/rcu/RCU+sync+free.litmus | 42 ++++++++++++++++++++++ 1 file changed, 42 insertions(+) create mode 100644 Documentation/litmus-tests/rcu/RCU+sync+free.litmus
Documentation/litmus-tests/rcu/RCU+sync+free.litmus new file +42 -0
@@ -0,0 +1,42 @@ +C RCU+sync+free + +(* + * Result: Never + * + * This litmus test demonstrates that an RCU reader can never see a write that + * follows a grace period, if it did not see writes that precede that grace + * period. + * + * This is a typical pattern of RCU usage, where the write before the grace + * period assigns a pointer, and the writes following the grace period destroy + * the object that the pointer used to point to. + * + * This is one implication of the RCU grace-period guarantee, which says (among + * other things) that an RCU read-side critical section cannot span a grace period. + *) + +{ +int x = 1; +int *y = &x; +int z = 1; +} + +P0(int *x, int *z, int **y) +{ + int *r0; + int r1; + + rcu_read_lock(); + r0 = rcu_dereference(*y); + r1 = READ_ONCE(*r0); + rcu_read_unlock(); +} + +P1(int *x, int *z, int **y) +{ + rcu_assign_pointer(*y, z); + synchronize_rcu(); + WRITE_ONCE(*x, 0); +} + +exists (0:r0=x /\ 0:r1=0)
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH 5/10 Documentation: LKMM: Add litmus test for RCU GP guarantee where reader stores
In reply to Paul E. McKenneyFrom: "Joel Fernandes (Google)" <[email protected]>
This adds an example for the important RCU grace period guarantee, which shows an RCU reader can never span a grace period.
--- Documentation/litmus-tests/README | 5 +++ .../litmus-tests/rcu/RCU+sync+read.litmus | 37 ++++++++++++++++++++++ 2 files changed, 42 insertions(+) create mode 100644 Documentation/litmus-tests/rcu/RCU+sync+read.litmus
Documentation/litmus-tests/README +5 -0
@@ -7,3 +7,8 @@ RCU (/rcu directory) MP+onceassign+derefonce.litmus Demonstrates that rcu_assign_pointer() and rcu_dereference() to ensure that an RCU reader will not see pre-initialization garbage. + +RCU+sync+read.litmus +RCU+sync+free.litmus + Both the above litmus tests demonstrate the RCU grace period guarantee + that an RCU read-side critical section can never span a grace period.
Documentation/litmus-tests/rcu/RCU+sync+read.litmus new file +37 -0
@@ -0,0 +1,37 @@ +C RCU+sync+read + +(* + * Result: Never + * + * This litmus test demonstrates that after a grace period, an RCU updater always + * sees all stores done in prior RCU read-side critical sections. Such + * read-side critical sections would have ended before the grace period ended. + * + * This is one implication of the RCU grace-period guarantee, which says (among + * other things) that an RCU read-side critical section cannot span a grace period. + *) + +{ +int x = 0; +int y = 0; +} + +P0(int *x, int *y) +{ + rcu_read_lock(); + WRITE_ONCE(*x, 1); + WRITE_ONCE(*y, 1); + rcu_read_unlock(); +} + +P1(int *x, int *y) +{ + int r0; + int r1; + + r0 = READ_ONCE(*x); + synchronize_rcu(); + r1 = READ_ONCE(*y); +} + +exists (1:r0=1 /\ 1:r1=0)
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH 6/10 MAINTAINERS: Update maintainers for new Documentaion/litmus-tests/
In reply to Paul E. McKenneyFrom: "Joel Fernandes (Google)" <[email protected]>
Also add me as Reviewer for LKMM. Previously a patch to do this was Acked but somewhere along the line got lost. Add myself in this patch.
--- MAINTAINERS | 2 ++ 1 file changed, 2 insertions(+)
MAINTAINERS +2 -0
@@ -9806,6 +9806,7 @@ M: Luc Maranget <[email protected]> M: "Paul E. McKenney" <[email protected]> R: Akira Yokosawa <[email protected]> R: Daniel Lustig <[email protected]> +R: Joel Fernandes <[email protected]> L: [email protected] L: [email protected] S: Supported
@@ -9816,6 +9817,7 @@ F: Documentation/core-api/atomic_ops.rst F: Documentation/core-api/refcount-vs-atomic.rst F: Documentation/memory-barriers.txt F: tools/memory-model/ +F: Documentation/litmus-tests/ LIS3LV02D ACCELEROMETER DRIVER M: Eric Piel <[email protected]>
Signature · 1 line
2.9.5
Re: [PATCH lkmm tip/core/rcu 06/10] MAINTAINERS: Update maintainers for new Documentaion/litmus-tests/
Quoted 2 lines · On Wed, 2020-04-15 at 11:49 -0700, [email protected] wrote:
Also add me as Reviewer for LKMM. Previously a patch to do this was Acked but somewhere along the line got lost. Add myself in this patch.
[]
MAINTAINERS
Quoted 1 line
MAINTAINERS +0
-0
[]
MAINTAINERS F: Documentation/core-api/atomic_ops.rst · line 9820
Quoted 13 lines
MAINTAINERS +2
-0@@ -9806,6 +9806,7 @@ M: Luc Maranget <[email protected]> M: "Paul E. McKenney" <[email protected]> R: Akira Yokosawa <[email protected]> R: Daniel Lustig <[email protected]> +R: Joel Fernandes <[email protected]> L: [email protected] L: [email protected] S: Supported@@ -9816,6 +9817,7 @@ F: Documentation/core-api/atomic_ops.rst F: Documentation/core-api/refcount-vs-atomic.rst F: Documentation/memory-barriers.txt F: tools/memory-model/ +F: Documentation/litmus-tests/
trivia:
Alphabetic ordering of F: entries please. This should be between core-api and memory-barriers.
Quoted 2 lines
MAINTAINERS +0
-0LIS3LV02D ACCELEROMETER DRIVER M: Eric Piel <[email protected]>
Re: [PATCH lkmm tip/core/rcu 06/10] MAINTAINERS: Update maintainers for new Documentaion/litmus-tests/
Quoted 22 lines · Joe Perches
Quoted 2 lines · On Wed, 2020-04-15 at 11:49 -0700, [email protected] wrote:
Also add me as Reviewer for LKMM. Previously a patch to do this was Acked but somewhere along the line got lost. Add myself in this patch.
[]
MAINTAINERS
Quoted 1 line
MAINTAINERS +0
-0[]
MAINTAINERS F: Documentation/core-api/atomic_ops.rst · line 9820
Quoted 13 lines
MAINTAINERS +2
-0@@ -9806,6 +9806,7 @@ M: Luc Maranget <[email protected]> M: "Paul E. McKenney" <[email protected]> R: Akira Yokosawa <[email protected]> R: Daniel Lustig <[email protected]> +R: Joel Fernandes <[email protected]> L: [email protected] L: [email protected] S: Supported@@ -9816,6 +9817,7 @@ F: Documentation/core-api/atomic_ops.rst F: Documentation/core-api/refcount-vs-atomic.rst F: Documentation/memory-barriers.txt F: tools/memory-model/ +F: Documentation/litmus-tests/trivia:
Quoted 5 lines
Alphabetic ordering of F: entries please. This should be between core-api and memory-barriers.
Quoted 2 lines
MAINTAINERS +0
-0LIS3LV02D ACCELEROMETER DRIVER M: Eric Piel <[email protected]>
New one on me, but it does make a lot of sense, especially for cases with lots of scattered paths. How about the following?
Thanx, Paul
------------------------------------------------------------------------
commit e3b73adbd732e13e7e9f42c9adc95e7b9439426c
Author: Joel Fernandes (Google) <[email protected]> Date: Sun Mar 22 21:57:35 2020 -0400
MAINTAINERS: Update maintainers for new Documentation/litmus-tests
This commit adds Joel Fernandes as official LKMM reviewer.
MAINTAINERS +2 -0
@@ -9806,6 +9806,7 @@ M: Luc Maranget <[email protected]> M: "Paul E. McKenney" <[email protected]> R: Akira Yokosawa <[email protected]> R: Daniel Lustig <[email protected]> +R: Joel Fernandes <[email protected]> L: [email protected] L: [email protected] S: Supported
@@ -9814,6 +9815,7 @@ F: Documentation/atomic_bitops.txt F: Documentation/atomic_t.txt F: Documentation/core-api/atomic_ops.rst F: Documentation/core-api/refcount-vs-atomic.rst +F: Documentation/litmus-tests/ F: Documentation/memory-barriers.txt F: tools/memory-model/
Re: [PATCH lkmm tip/core/rcu 06/10] MAINTAINERS: Update maintainers for new Documentaion/litmus-tests/
Quoted 23 lines · Paul E. McKenney
Quoted 22 lines · Joe Perches
Quoted 2 lines · On Wed, 2020-04-15 at 11:49 -0700, [email protected] wrote:
Also add me as Reviewer for LKMM. Previously a patch to do this was Acked but somewhere along the line got lost. Add myself in this patch.
[]
MAINTAINERS
Quoted 1 line
MAINTAINERS +0
-0[]
MAINTAINERS F: Documentation/core-api/atomic_ops.rst · line 9820
Quoted 13 lines
MAINTAINERS +2
-0@@ -9806,6 +9806,7 @@ M: Luc Maranget <[email protected]> M: "Paul E. McKenney" <[email protected]> R: Akira Yokosawa <[email protected]> R: Daniel Lustig <[email protected]> +R: Joel Fernandes <[email protected]> L: [email protected] L: [email protected] S: Supported@@ -9816,6 +9817,7 @@ F: Documentation/core-api/atomic_ops.rst F: Documentation/core-api/refcount-vs-atomic.rst F: Documentation/memory-barriers.txt F: tools/memory-model/ +F: Documentation/litmus-tests/trivia:
Quoted 8 lines
Quoted 5 lines
Alphabetic ordering of F: entries please. This should be between core-api and memory-barriers.
Quoted 2 lines
MAINTAINERS +0
-0LIS3LV02D ACCELEROMETER DRIVER M: Eric Piel <[email protected]>New one on me, but it does make a lot of sense, especially for cases with lots of scattered paths. How about the following?
Thanks Paul.
If the recent commits that Linus did just before v5.7-rc1:
3b50142d8528 ("MAINTAINERS: sort field names for all entries") 4400b7d68f6e ("MAINTAINERS: sort entries by entry name")
don't create too many problems I suppose
$ scripts/parse-maintainers.pl --order --input=MAINTAINERS --output=MAINTAINERS
could be run just before every -rc1 to keep all this stuff organized.
We'll see.
Paul E. McKenney
maintainerRe: [PATCH lkmm tip/core/rcu 06/10] MAINTAINERS: Update maintainers for new Documentaion/litmus-tests/
Quoted 40 lines · Joe Perches
Quoted 23 lines · Paul E. McKenney
Quoted 22 lines · Joe Perches
Quoted 2 lines · On Wed, 2020-04-15 at 11:49 -0700, [email protected] wrote:
Also add me as Reviewer for LKMM. Previously a patch to do this was Acked but somewhere along the line got lost. Add myself in this patch.
[]
MAINTAINERS
Quoted 1 line
MAINTAINERS +0
-0[]
MAINTAINERS F: Documentation/core-api/atomic_ops.rst · line 9820
Quoted 13 lines
MAINTAINERS +2
-0@@ -9806,6 +9806,7 @@ M: Luc Maranget <[email protected]> M: "Paul E. McKenney" <[email protected]> R: Akira Yokosawa <[email protected]> R: Daniel Lustig <[email protected]> +R: Joel Fernandes <[email protected]> L: [email protected] L: [email protected] S: Supported@@ -9816,6 +9817,7 @@ F: Documentation/core-api/atomic_ops.rst F: Documentation/core-api/refcount-vs-atomic.rst F: Documentation/memory-barriers.txt F: tools/memory-model/ +F: Documentation/litmus-tests/trivia:
Quoted 8 lines
Quoted 5 lines
Alphabetic ordering of F: entries please. This should be between core-api and memory-barriers.
Quoted 2 lines
MAINTAINERS +0
-0LIS3LV02D ACCELEROMETER DRIVER M: Eric Piel <[email protected]>New one on me, but it does make a lot of sense, especially for cases with lots of scattered paths. How about the following?
Thanks Paul.
If the recent commits that Linus did just before v5.7-rc1:
3b50142d8528("MAINTAINERS: sort field names for all entries")4400b7d68f6e("MAINTAINERS: sort entries by entry name")
Quoted 7 lines
don't create too many problems I suppose
$ scripts/parse-maintainers.pl --order --input=MAINTAINERS --output=MAINTAINERScould be run just before every -rc1 to keep all this stuff organized.
We'll see.
Easier than me remembering, I suppose. ;-)
Thanx, Paul
[email protected]
maintainerPATCH 7/10 tools/memory-model: Add an exception for limitations on _unless() family
In reply to Paul E. McKenneyFrom: Boqun Feng <[email protected]>
According to Luc, atomic_add_unless() is directly provided by herd7, therefore it can be used in litmus tests. So change the limitation section in README to unlimit the use of atomic_add_unless().
--- tools/memory-model/README | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
tools/memory-model/README +7 -3
@@ -207,11 +207,15 @@ The Linux-kernel memory model (LKMM) has the following limitations: case as a store release. b. The "unless" RMW operations are not currently modeled: - atomic_long_add_unless(), atomic_add_unless(), - atomic_inc_unless_negative(), and - atomic_dec_unless_positive(). These can be emulated + atomic_long_add_unless(), atomic_inc_unless_negative(), + and atomic_dec_unless_positive(). These can be emulated in litmus tests, for example, by using atomic_cmpxchg(). + One exception of this limitation is atomic_add_unless(), + which is provided directly by herd7 (so no corresponding + definition in linux-kernel.def). atomic_add_unless() is + modeled by herd7 therefore it can be used in litmus tests. + c. The call_rcu() function is not modeled. It can be emulated in litmus tests by adding another process that invokes synchronize_rcu() and the body of the callback
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH 8/10 Documentation/litmus-tests: Introduce atomic directory
In reply to Paul E. McKenneyFrom: Boqun Feng <[email protected]>
Although we have atomic_t.txt and its friends to describe the semantics of atomic APIs and lib/atomic64_test.c for build testing and testing in UP mode, the tests for our atomic APIs in real SMP mode are still missing. Since now we have the LKMM tool in kernel and litmus tests can be used to generate kernel modules for testing purpose with "klitmus" (a tool from the LKMM toolset), it makes sense to put a few typical litmus tests into kernel so that
1) they are the examples to describe the conceptual mode of the
semantics of atomic APIs, and
2) they can be used to generate kernel test modules for anyone
who is interested to test the atomic APIs implementation (in most cases, is the one who implements the APIs for a new arch)
Therefore, introduce the atomic directory for this purpose. The directory is maintained by the LKMM group to make sure the litmus tests are always aligned with our memory model.
--- Documentation/litmus-tests/atomic/README | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 Documentation/litmus-tests/atomic/README
Documentation/litmus-tests/atomic/README new file +4 -0
@@ -0,0 +1,4 @@ +This directory contains litmus tests that are typical to describe the semantics +of our atomic APIs. For more information about how to "run" a litmus test or +how to generate a kernel test module based on a litmus test, please see +tools/memory-model/README.
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH 9/10 Documentation/litmus-tests/atomic: Add a test for atomic_set()
In reply to Paul E. McKenneyFrom: Boqun Feng <[email protected]>
We already use a litmus test in atomic_t.txt to describe the behavior of an atomic_set() with the an atomic RMW, so add it into atomic-tests directory to make it easily accessible for anyone who cares about the semantics of our atomic APIs.
Besides currently the litmus test "atomic-set" in atomic_t.txt has a few things to be improved:
1) The CPU/Processor numbers "P1,P2" are not only inconsistent with
the rest of the document, which uses "CPU0" and "CPU1", but also unacceptable by the herd tool, which requires processors start at "P0".
2) The initialization block uses a "atomic_set()", which is OK, but
it's better to use ATOMIC_INIT() to make clear this is an initialization.
3) The return value of atomic_add_unless() is discarded
inexplicitly, which is OK for C language, but it will be helpful to the herd tool if we use a void cast to make the discard explicit.
4) The name and the paragraph describing the test need to be more
accurate and aligned with our wording in LKMM.
Therefore fix these in both atomic_t.txt and the new added litmus test.
--- Documentation/atomic_t.txt | 14 ++++++------- ...Atomic-RMW-ops-are-atomic-WRT-atomic_set.litmus | 24 ++++++++++++++++++++++ Documentation/litmus-tests/atomic/README | 7 +++++++ 3 files changed, 38 insertions(+), 7 deletions(-) create mode 100644 Documentation/litmus-tests/atomic/Atomic-RMW-ops-are-atomic-WRT-atomic_set.litmus
Documentation/atomic_t.txt +7 -7
@@ -85,21 +85,21 @@ smp_store_release() respectively. Therefore, if you find yourself only using the Non-RMW operations of atomic_t, you do not in fact need atomic_t at all and are doing it wrong. -A subtle detail of atomic_set{}() is that it should be observable to the RMW -ops. That is: +A note for the implementation of atomic_set{}() is that it must not break the +atomicity of the RMW ops. That is: - C atomic-set + C Atomic-RMW-ops-are-atomic-WRT-atomic_set { - atomic_set(v, 1); + atomic_t v = ATOMIC_INIT(1); } - P1(atomic_t *v) + P0(atomic_t *v) { - atomic_add_unless(v, 1, 0); + (void)atomic_add_unless(v, 1, 0); } - P2(atomic_t *v) + P1(atomic_t *v) { atomic_set(v, 0); }
Documentation/litmus-tests/atomic/Atomic-RMW-ops-are-atomic-WRT-atomic_set.litmus new file +24 -0
@@ -0,0 +1,24 @@ +C Atomic-RMW-ops-are-atomic-WRT-atomic_set + +(* + * Result: Never + * + * Test that atomic_set() cannot break the atomicity of atomic RMWs. + *) + +{ + atomic_t v = ATOMIC_INIT(1); +} + +P0(atomic_t *v) +{ + (void)atomic_add_unless(v, 1, 0); +} + +P1(atomic_t *v) +{ + atomic_set(v, 0); +} + +exists +(v=2)
Documentation/litmus-tests/atomic/README +7 -0
@@ -2,3 +2,10 @@ This directory contains litmus tests that are typical to describe the semantics of our atomic APIs. For more information about how to "run" a litmus test or how to generate a kernel test module based on a litmus test, please see tools/memory-model/README. + +============ +LITMUS TESTS +============ + +Atomic-RMW-ops-are-atomic-WRT-atomic_set.litmus + Test that atomic_set() cannot break the atomicity of atomic RMWs.
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH 10/10 Documentation/litmus-tests/atomic: Add a test for smp_mb__after_atomic()
In reply to Paul E. McKenneyFrom: Boqun Feng <[email protected]>
We already use a litmus test in atomic_t.txt to describe atomic RMW + smp_mb__after_atomic() is stronger than acquire (both the read and the write parts are ordered). So make it a litmus test in atomic-tests directory, so that people can access the litmus easily.
Additionally, change the processor numbers "P1, P2" to "P0, P1" in atomic_t.txt for the consistency with the processor numbers in the litmus test, which herd can handle.
--- Documentation/atomic_t.txt | 10 +++---- ...b__after_atomic-is-stronger-than-acquire.litmus | 32 ++++++++++++++++++++++ Documentation/litmus-tests/atomic/README | 5 ++++ 3 files changed, 42 insertions(+), 5 deletions(-) create mode 100644 Documentation/litmus-tests/atomic/Atomic-RMW+mb__after_atomic-is-stronger-than-acquire.litmus
Documentation/atomic_t.txt +5 -5
@@ -233,19 +233,19 @@ as well. Similarly, something like: is an ACQUIRE pattern (though very much not typical), but again the barrier is strictly stronger than ACQUIRE. As illustrated: - C strong-acquire + C Atomic-RMW+mb__after_atomic-is-stronger-than-acquire { } - P1(int *x, atomic_t *y) + P0(int *x, atomic_t *y) { r0 = READ_ONCE(*x); smp_rmb(); r1 = atomic_read(y); } - P2(int *x, atomic_t *y) + P1(int *x, atomic_t *y) { atomic_inc(y); smp_mb__after_atomic();
@@ -253,14 +253,14 @@ strictly stronger than ACQUIRE. As illustrated: } exists - (r0=1 /\ r1=0) + (0:r0=1 /\ 0:r1=0) This should not happen; but a hypothetical atomic_inc_acquire() -- (void)atomic_fetch_inc_acquire() for instance -- would allow the outcome, because it would not order the W part of the RMW against the following WRITE_ONCE. Thus: - P1 P2 + P0 P1 t = LL.acq *y (0) t++;
Documentation/litmus-tests/atomic/Atomic-RMW+mb__after_atomic-is-stronger-than-acquire.litmus new file +32 -0
@@ -0,0 +1,32 @@ +C Atomic-RMW+mb__after_atomic-is-stronger-than-acquire + +(* + * Result: Never + * + * Test that an atomic RMW followed by a smp_mb__after_atomic() is + * stronger than a normal acquire: both the read and write parts of + * the RMW are ordered before the subsequential memory accesses. + *) + +{ +} + +P0(int *x, atomic_t *y) +{ + int r0; + int r1; + + r0 = READ_ONCE(*x); + smp_rmb(); + r1 = atomic_read(y); +} + +P1(int *x, atomic_t *y) +{ + atomic_inc(y); + smp_mb__after_atomic(); + WRITE_ONCE(*x, 1); +} + +exists +(0:r0=1 /\ 0:r1=0)
Documentation/litmus-tests/atomic/README +5 -0
@@ -7,5 +7,10 @@ tools/memory-model/README. LITMUS TESTS ============ +Atomic-RMW+mb__after_atomic-is-stronger-than-acquire + Test that an atomic RMW followed by a smp_mb__after_atomic() is + stronger than a normal acquire: both the read and write parts of + the RMW are ordered before the subsequential memory accesses. + Atomic-RMW-ops-are-atomic-WRT-atomic_set.litmus Test that atomic_set() cannot break the atomicity of atomic RMWs.
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH v4 1/15 kcsan: Add option to allow watcher interruptions
In reply to Paul E. McKenneyFrom: Marco Elver <[email protected]>
Add option to allow interrupts while a watchpoint is set up. This can be enabled either via CONFIG_KCSAN_INTERRUPT_WATCHER or via the boot parameter 'kcsan.interrupt_watcher=1'.
Note that, currently not all safe per-CPU access primitives and patterns are accounted for, which could result in false positives. For example, asm-generic/percpu.h uses plain operations, which by default are instrumented. On interrupts and subsequent accesses to the same variable, KCSAN would currently report a data race with this option.
Therefore, this option should currently remain disabled by default, but may be enabled for specific test scenarios.
To avoid new warnings, changes all uses of smp_processor_id() to use the raw version (as already done in kcsan_found_watchpoint()). The exact SMP processor id is for informational purposes in the report, and correctness is not affected.
--- kernel/kcsan/core.c | 34 ++++++++++------------------------ lib/Kconfig.kcsan | 11 +++++++++++ 2 files changed, 21 insertions(+), 24 deletions(-)
kernel/kcsan/core.c +10 -24
@@ -21,6 +21,7 @@ static bool kcsan_early_enable = IS_ENABLED(CONFIG_KCSAN_EARLY_ENABLE);static unsigned int kcsan_udelay_task =CONFIG_KCSAN_UDELAY_TASK ;static unsigned int kcsan_udelay_interrupt =CONFIG_KCSAN_UDELAY_INTERRUPT ;static long kcsan_skip_watch =CONFIG_KCSAN_SKIP_WATCH ; +static bool kcsan_interrupt_watcher =IS_ENABLED (CONFIG_KCSAN_INTERRUPT_WATCHER );#ifdef MODULE_PARAM_PREFIX #undef MODULE_PARAM_PREFIX
@@ -30,6 +31,7 @@ module_param_named(early_enable, kcsan_early_enable, bool, 0);module_param_named (udelay_task, kcsan_udelay_task, uint,0644 );module_param_named (udelay_interrupt, kcsan_udelay_interrupt, uint,0644 );module_param_named (skip_watch, kcsan_skip_watch, long,0644 ); +module_param_named (interrupt_watcher, kcsan_interrupt_watcher, bool,0444 );bool kcsan_enabled;
@@ -354,7 +356,7 @@ kcsan_setup_watchpoint(const volatile void *ptr, size_t size, int type)unsigned long access_mask;enum kcsan_value_change value_change =KCSAN_VALUE_CHANGE_MAYBE ;unsigned long ua_flags =user_access_save (); -unsigned long irq_flags; +unsigned long irq_flags =0 ;/* * Always reset kcsan_skip counter in slow-path to avoid underflow; see
@@ -370,26 +372,9 @@ kcsan_setup_watchpoint(const volatile void *ptr, size_t size, int type) goto out; } -/* -* Disable interrupts & preemptions to avoid another thread on the same -* CPU accessing memory locations for the set up watchpoint; this is to -* avoid reporting races to e.g. CPU-local data. -* -* An alternative would be adding the source CPU to the watchpoint -* encoding, and checking that watchpoint-CPU != this-CPU. There are -* several problems with this: -* 1. we should avoid stealing more bits from the watchpoint encoding -* as it would affect accuracy, as well as increase performance -* overhead in the fast-path; -* 2. if we are preempted, but there *is* a genuine data race, we -* would *not* report it -- since this is the common case (vs. -* CPU-local data accesses), it makes more sense (from a data race -* detection point of view) to simply disable preemptions to ensure -* as many tasks as possible run on other CPUs. -* -* Use raw versions, to avoid lockdep recursion via IRQ flags tracing. -*/ -raw_local_irq_save (irq_flags); +if (!kcsan_interrupt_watcher) +/* Use raw to avoid lockdep recursion via IRQ flags tracing. */ +raw_local_irq_save (irq_flags); watchpoint =insert_watchpoint ((unsigned long )ptr, size, is_write);if (watchpoint ==NULL ) {
@@ -507,7 +492,7 @@ kcsan_setup_watchpoint(const volatile void *ptr, size_t size, int type)if (is_assert && value_change ==KCSAN_VALUE_CHANGE_TRUE )kcsan_counter_inc (KCSAN_COUNTER_ASSERT_FAILURES ); -kcsan_report (ptr, size, type, value_change,smp_processor_id (), +kcsan_report (ptr, size, type, value_change,raw_smp_processor_id (),KCSAN_REPORT_RACE_SIGNAL ); }else if (value_change ==KCSAN_VALUE_CHANGE_TRUE ) {/* Inferring a race, since the value should not have changed. */
@@ -518,13 +503,14 @@ kcsan_setup_watchpoint(const volatile void *ptr, size_t size, int type)if (IS_ENABLED (CONFIG_KCSAN_REPORT_RACE_UNKNOWN_ORIGIN ) || is_assert)kcsan_report (ptr, size, type,KCSAN_VALUE_CHANGE_TRUE , -smp_processor_id (), +raw_smp_processor_id (),KCSAN_REPORT_RACE_UNKNOWN_ORIGIN ); }kcsan_counter_dec (KCSAN_COUNTER_USED_WATCHPOINTS ); out_unlock: -raw_local_irq_restore (irq_flags); +if (!kcsan_interrupt_watcher) +raw_local_irq_restore (irq_flags); out:user_access_restore (ua_flags); }
lib/Kconfig.kcsan +11 -0
@@ -88,6 +88,17 @@ config KCSAN_SKIP_WATCH_RANDOMIZE KCSAN_WATCH_SKIP. If false, the chosen value is always KCSAN_WATCH_SKIP. +config KCSAN_INTERRUPT_WATCHER + bool "Interruptible watchers" + help + If enabled, a task that set up a watchpoint may be interrupted while + delayed. This option will allow KCSAN to detect races between + interrupted tasks and other threads of execution on the same CPU. + + Currently disabled by default, because not all safe per-CPU access + primitives and patterns may be accounted for, and therefore could + result in false positives. + config KCSAN_REPORT_ONCE_IN_MS int "Duration in milliseconds, in which any given race is only reported once" default 3000
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH v4 2/15 kcsan: Add option for verbose reporting
In reply to Paul E. McKenneyFrom: Marco Elver <[email protected]>
Adds CONFIG_KCSAN_VERBOSE to optionally enable more verbose reports. Currently information about the reporting task's held locks and IRQ trace events are shown, if they are enabled.
--- kernel/kcsan/core.c | 4 +- kernel/kcsan/kcsan.h | 3 ++ kernel/kcsan/report.c | 103 +++++++++++++++++++++++++++++++++++++++++++++++++- lib/Kconfig.kcsan | 13 +++++++ 4 files changed, 120 insertions(+), 3 deletions(-)
kernel/kcsan/core.c +2 -2
@@ -18,8 +18,8 @@#include "kcsan.h" static bool kcsan_early_enable =IS_ENABLED (CONFIG_KCSAN_EARLY_ENABLE ); -static unsigned int kcsan_udelay_task =CONFIG_KCSAN_UDELAY_TASK ; -static unsigned int kcsan_udelay_interrupt =CONFIG_KCSAN_UDELAY_INTERRUPT ; +unsigned int kcsan_udelay_task =CONFIG_KCSAN_UDELAY_TASK ; +unsigned int kcsan_udelay_interrupt =CONFIG_KCSAN_UDELAY_INTERRUPT ;static long kcsan_skip_watch =CONFIG_KCSAN_SKIP_WATCH ;static bool kcsan_interrupt_watcher =IS_ENABLED (CONFIG_KCSAN_INTERRUPT_WATCHER );
kernel/kcsan/kcsan.h +3 -0
@@ -13,6 +13,9 @@/* The number of adjacent watchpoints to check. */ #define KCSAN_CHECK_ADJACENT 1 +extern unsigned int kcsan_udelay_task; +extern unsigned int kcsan_udelay_interrupt; +/* * Globally enable and disable KCSAN. */
kernel/kcsan/report.c +102 -1
144 lines not on this page · raw message
lib/Kconfig.kcsan +13 -0
@@ -20,6 +20,19 @@ menuconfig KCSAN if KCSAN +config KCSAN_VERBOSE + bool "Show verbose reports with more information about system state" + depends on PROVE_LOCKING + help + If enabled, reports show more information about the system state that + may help better analyze and debug races. This includes held locks and + IRQ trace events. + + While this option should generally be benign, we call into more + external functions on report generation; if a race report is + generated from any one of them, system stability may suffer due to + deadlocks or recursion. If in doubt, say N. + config KCSAN_DEBUG bool "Debugging of KCSAN internals"
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH v4 3/15 kcsan: Add current->state to implicitly atomic accesses
In reply to Paul E. McKenneyFrom: Marco Elver <[email protected]>
Add volatile current->state to list of implicitly atomic accesses. This is in preparation to eventually enable KCSAN on kernel/sched (which currently still has KCSAN_SANITIZE := n).
Since accesses that match the special check in atomic.h are rare, it makes more sense to move this check to the slow-path, avoiding the additional compare in the fast-path. With the microbenchmark, a speedup of ~6% is measured.
--- kernel/kcsan/atomic.h | 21 +++++++-------------- kernel/kcsan/core.c | 22 +++++++++++++++------- kernel/kcsan/debugfs.c | 27 ++++++++++++++++++--------- 3 files changed, 40 insertions(+), 30 deletions(-)
kernel/kcsan/atomic.h +7 -14
@@ -4,24 +4,17 @@#define _KERNEL_KCSAN_ATOMIC_H#include <linux/jiffies.h> +#include <linux/sched.h> /* -* Helper that returns true if access to @ptr should be considered an atomic -* access, even though it is not explicitly atomic. -* -* List all volatile globals that have been observed in races, to suppress -* data race reports between accesses to these variables. -* -* For now, we assume that volatile accesses of globals are as strong as atomic -* accesses (READ_ONCE, WRITE_ONCE cast to volatile). The situation is still not -* entirely clear, as on some architectures (Alpha) READ_ONCE/WRITE_ONCE do more -* than cast to volatile. Eventually, we hope to be able to remove this -* function. +* Special rules for certain memory where concurrent conflicting accesses are +* common, however, the current convention is to not mark them; returns true if +* access to @ptr should be considered atomic. Called from slow-path. */ -static __always_inline bool kcsan_is_atomic(const volatile void *ptr) +static bool kcsan_is_atomic_special (const volatile void *ptr) { -/* only jiffies for now */ -return ptr == &jiffies; +/* volatile globals that have been observed in data races. */ +return ptr == &jiffies || ptr == ¤t->state; }#endif /* _KERNEL_KCSAN_ATOMIC_H */
kernel/kcsan/core.c +15 -7
@@ -188,12 +188,13 @@ static __always_inline struct kcsan_ctx *get_ctx(void)return in_task () ? ¤t->kcsan_ctx :raw_cpu_ptr (&kcsan_cpu_ctx); } +/* Rules for generic atomic accesses. Called from fast-path. */ static __always_inline boolis_atomic (const volatile void *ptr,size_t size,int type) {struct kcsan_ctx *ctx; -if ((type &KCSAN_ACCESS_ATOMIC ) !=0 ) +if (type &KCSAN_ACCESS_ATOMIC )return true;/*
@@ -201,16 +202,16 @@ is_atomic(const volatile void *ptr, size_t size, int type)* as atomic. This allows using them also in atomic regions, such as * seqlocks, without implicitly changing their semantics. */ -if ((type &KCSAN_ACCESS_ASSERT ) !=0 ) +if (type &KCSAN_ACCESS_ASSERT )return false;if (IS_ENABLED (CONFIG_KCSAN_ASSUME_PLAIN_WRITES_ATOMIC ) && - (type &KCSAN_ACCESS_WRITE ) !=0 && size <=sizeof (long ) && + (type &KCSAN_ACCESS_WRITE ) && size <=sizeof (long ) &&IS_ALIGNED ((unsigned long )ptr, size))return true;/* Assume aligned writes up to word size are atomic. */ ctx =get_ctx (); -if (unlikely (ctx->atomic_next >0 )) { +if (ctx->atomic_next >0 ) {/* * Because we do not have separate contexts for nested * interrupts, in case atomic_next is set, we simply assume that
@@ -224,10 +225,8 @@ is_atomic(const volatile void *ptr, size_t size, int type) --ctx->atomic_next;/* in task, or outer interrupt */ return true; } -if (unlikely (ctx->atomic_nest_count >0 || ctx->in_flat_atomic)) -return true; -return kcsan_is_atomic (ptr); +return ctx->atomic_nest_count >0 || ctx->in_flat_atomic; }static __always_inline bool
@@ -367,6 +366,15 @@ kcsan_setup_watchpoint(const volatile void *ptr, size_t size, int type)if (!kcsan_is_enabled ()) goto out; +/* +* Special atomic rules: unlikely to be true, so we check them here in +* the slow-path, and not in the fast-path in is_atomic(). Call after +* kcsan_is_enabled(), as we may access memory that is not yet +* initialized during early boot. +*/ +if (!is_assert &&kcsan_is_atomic_special (ptr)) + goto out; +if (!check_encodable ((unsigned long )ptr, size)) {kcsan_counter_inc (KCSAN_COUNTER_UNENCODABLE_ACCESSES ); goto out;
kernel/kcsan/debugfs.c +18 -9
@@ -74,25 +74,34 @@ void kcsan_counter_dec(enum kcsan_counter_id id)*/ static noinline void microbenchmark(unsigned long iters) { +const struct kcsan_ctx ctx_save = current->kcsan_ctx; +const bool was_enabled =READ_ONCE (kcsan_enabled);cycles_t cycles; +/* We may have been called from an atomic region; reset context. */ +memset (¤t->kcsan_ctx,0 ,sizeof (current->kcsan_ctx)); +/* +* Disable to benchmark fast-path for all accesses, and (expected +* negligible) call into slow-path, but never set up watchpoints. +*/ +WRITE_ONCE (kcsan_enabled, false); +pr_info ("KCSAN: %s begin | iters: %lu\n" , __func__, iters); cycles =get_cycles ();while (iters--) { -/* -* We can run this benchmark from multiple tasks; this address -* calculation increases likelyhood of some accesses -* overlapping. Make the access type an atomic read, to never -* set up watchpoints and test the fast-path only. -*/ -unsigned long addr = - iters % (CONFIG_KCSAN_NUM_WATCHPOINTS *PAGE_SIZE ); -__kcsan_check_access ((void *)addr,sizeof (long ),KCSAN_ACCESS_ATOMIC ); +unsigned long addr = iters & ((PAGE_SIZE <<8 ) -1 ); +int type = !(iters &0x7f ) ?KCSAN_ACCESS_ATOMIC : + (!(iters &0xf ) ?KCSAN_ACCESS_WRITE :0 ); +__kcsan_check_access ((void *)addr,sizeof (long ), type); } cycles =get_cycles () - cycles;pr_info ("KCSAN: %s end | cycles: %llu\n" , __func__, cycles); + +WRITE_ONCE (kcsan_enabled, was_enabled); +/* restore context */ + current->kcsan_ctx = ctx_save; }/*
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH v4 4/15 kcsan: Fix a typo in a comment
In reply to Paul E. McKenneyFrom: Qiujun Huang <[email protected]>
s/slots slots/slots/
--- kernel/kcsan/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/kcsan/core.c +1 -1
@@ -45,7 +45,7 @@ static DEFINE_PER_CPU(struct kcsan_ctx, kcsan_cpu_ctx) = { };/* -* Helper macros to index into adjacent slots slots, starting from address slot +* Helper macros to index into adjacent slots, starting from address slot * itself, followed by the right and left slots. * * The purpose is 2-fold:
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH v4 5/15 kcsan: Update Documentation/dev-tools/kcsan.rst
In reply to Paul E. McKenneyFrom: Marco Elver <[email protected]>
Extend and improve based on recent changes, and summarize important bits that have been missing. Tested with "make htmldocs".
--- Documentation/dev-tools/kcsan.rst | 227 ++++++++++++++++++++++++-------------- 1 file changed, 144 insertions(+), 83 deletions(-)
Documentation/dev-tools/kcsan.rst +144 -83
@@ -1,27 +1,22 @@ The Kernel Concurrency Sanitizer (KCSAN) ======================================== -Overview --------- - -*Kernel Concurrency Sanitizer (KCSAN)* is a dynamic data race detector for -kernel space. KCSAN is a sampling watchpoint-based data race detector. Key -priorities in KCSAN's design are lack of false positives, scalability, and -simplicity. More details can be found in `Implementation Details`_. - -KCSAN uses compile-time instrumentation to instrument memory accesses. KCSAN is -supported in both GCC and Clang. With GCC it requires version 7.3.0 or later. -With Clang it requires version 7.0.0 or later. +The Kernel Concurrency Sanitizer (KCSAN) is a dynamic race detector, which +relies on compile-time instrumentation, and uses a watchpoint-based sampling +approach to detect races. KCSAN's primary purpose is to detect `data races`_. Usage ----- -To enable KCSAN configure kernel with:: +KCSAN is supported in both GCC and Clang. With GCC it requires version 7.3.0 or +later. With Clang it requires version 7.0.0 or later. + +To enable KCSAN configure the kernel with:: CONFIG_KCSAN = y KCSAN provides several other configuration options to customize behaviour (see -their respective help text for more info). +the respective help text in ``lib/Kconfig.kcsan`` for more info). Error reports ~~~~~~~~~~~~~
@@ -96,7 +91,8 @@ The other less common type of data race report looks like this:: This report is generated where it was not possible to determine the other racing thread, but a race was inferred due to the data value of the watched memory location having changed. These can occur either due to missing -instrumentation or e.g. DMA accesses. +instrumentation or e.g. DMA accesses. These reports will only be generated if +``CONFIG_KCSAN_REPORT_RACE_UNKNOWN_ORIGIN=y`` (selected by default). Selective analysis ~~~~~~~~~~~~~~~~~~
@@ -110,9 +106,26 @@ the below options are available: behaviour when encountering a data race is deemed safe. * Disabling data race detection for entire functions can be accomplished by - using the function attribute ``__no_kcsan`` (or ``__no_kcsan_or_inline`` for - ``__always_inline`` functions). To dynamically control for which functions - data races are reported, see the `debugfs`_ blacklist/whitelist feature. + using the function attribute ``__no_kcsan``:: + + __no_kcsan + void foo(void) { + ... + + To dynamically limit for which functions to generate reports, see the + `DebugFS interface`_ blacklist/whitelist feature. + + For ``__always_inline`` functions, replace ``__always_inline`` with + ``__no_kcsan_or_inline`` (which implies ``__always_inline``):: + + static __no_kcsan_or_inline void foo(void) { + ... + + Note: Older compiler versions (GCC < 9) also do not always honor the + ``__no_kcsan`` attribute on regular ``inline`` functions. If false positives + with these compilers cannot be tolerated, for small functions where + ``__always_inline`` would be appropriate, ``__no_kcsan_or_inline`` should be + preferred instead. * To disable data race detection for a particular compilation unit, add to the ``Makefile``::
@@ -124,13 +137,29 @@ the below options are available: KCSAN_SANITIZE := n -debugfs -~~~~~~~ +Furthermore, it is possible to tell KCSAN to show or hide entire classes of +data races, depending on preferences. These can be changed via the following +Kconfig options: + +* ``CONFIG_KCSAN_REPORT_VALUE_CHANGE_ONLY``: If enabled and a conflicting write + is observed via a watchpoint, but the data value of the memory location was + observed to remain unchanged, do not report the data race. + +* ``CONFIG_KCSAN_ASSUME_PLAIN_WRITES_ATOMIC``: Assume that plain aligned writes + up to word size are atomic by default. Assumes that such writes are not + subject to unsafe compiler optimizations resulting in data races. The option + causes KCSAN to not report data races due to conflicts where the only plain + accesses are aligned writes up to word size. + +DebugFS interface +~~~~~~~~~~~~~~~~~ + +The file ``/sys/kernel/debug/kcsan`` provides the following interface: -* The file ``/sys/kernel/debug/kcsan`` can be read to get stats. +* Reading ``/sys/kernel/debug/kcsan`` returns various runtime statistics. -* KCSAN can be turned on or off by writing ``on`` or ``off`` to - ``/sys/kernel/debug/kcsan``. +* Writing ``on`` or ``off`` to ``/sys/kernel/debug/kcsan`` allows turning KCSAN + on or off, respectively. * Writing ``!some_func_name`` to ``/sys/kernel/debug/kcsan`` adds ``some_func_name`` to the report filter list, which (by default) blacklists
@@ -142,91 +171,120 @@ debugfs can be used to silence frequently occurring data races; the whitelist feature can help with reproduction and testing of fixes. +Tuning performance +~~~~~~~~~~~~~~~~~~ + +Core parameters that affect KCSAN's overall performance and bug detection +ability are exposed as kernel command-line arguments whose defaults can also be +changed via the corresponding Kconfig options. + +* ``kcsan.skip_watch`` (``CONFIG_KCSAN_SKIP_WATCH``): Number of per-CPU memory + operations to skip, before another watchpoint is set up. Setting up + watchpoints more frequently will result in the likelihood of races to be + observed to increase. This parameter has the most significant impact on + overall system performance and race detection ability. + +* ``kcsan.udelay_task`` (``CONFIG_KCSAN_UDELAY_TASK``): For tasks, the + microsecond delay to stall execution after a watchpoint has been set up. + Larger values result in the window in which we may observe a race to + increase. + +* ``kcsan.udelay_interrupt`` (``CONFIG_KCSAN_UDELAY_INTERRUPT``): For + interrupts, the microsecond delay to stall execution after a watchpoint has + been set up. Interrupts have tighter latency requirements, and their delay + should generally be smaller than the one chosen for tasks. + +They may be tweaked at runtime via ``/sys/module/kcsan/parameters/``. + Data Races ---------- -Informally, two operations *conflict* if they access the same memory location, -and at least one of them is a write operation. In an execution, two memory -operations from different threads form a **data race** if they *conflict*, at -least one of them is a *plain access* (non-atomic), and they are *unordered* in -the "happens-before" order according to the `LKMM -<../../tools/memory-model/Documentation/explanation.txt>`_. +In an execution, two memory accesses form a *data race* if they *conflict*, +they happen concurrently in different threads, and at least one of them is a +*plain access*; they *conflict* if both access the same memory location, and at +least one is a write. For a more thorough discussion and definition, see `"Plain +Accesses and Data Races" in the LKMM`_. + +.. _"Plain Accesses and Data Races" in the LKMM: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/memory-model/Documentation/explanation.txt#n1922 -Relationship with the Linux Kernel Memory Model (LKMM) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Relationship with the Linux-Kernel Memory Consistency Model (LKMM) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The LKMM defines the propagation and ordering rules of various memory operations, which gives developers the ability to reason about concurrent code. Ultimately this allows to determine the possible executions of concurrent code, and if that code is free from data races. -KCSAN is aware of *atomic* accesses (``READ_ONCE``, ``WRITE_ONCE``, -``atomic_*``, etc.), but is oblivious of any ordering guarantees. In other -words, KCSAN assumes that as long as a plain access is not observed to race -with another conflicting access, memory operations are correctly ordered. +KCSAN is aware of *marked atomic operations* (``READ_ONCE``, ``WRITE_ONCE``, +``atomic_*``, etc.), but is oblivious of any ordering guarantees and simply +assumes that memory barriers are placed correctly. In other words, KCSAN +assumes that as long as a plain access is not observed to race with another +conflicting access, memory operations are correctly ordered. This means that KCSAN will not report *potential* data races due to missing -memory ordering. If, however, missing memory ordering (that is observable with -a particular compiler and architecture) leads to an observable data race (e.g. -entering a critical section erroneously), KCSAN would report the resulting -data race. - -Race conditions vs. data races -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Race conditions are logic bugs, where unexpected interleaving of racing -concurrent operations result in an erroneous state. - -Data races on the other hand are defined at the *memory model/language level*. -Many data races are also harmful race conditions, which a tool like KCSAN -reports! However, not all data races are race conditions and vice-versa. -KCSAN's intent is to report data races according to the LKMM. A data race -detector can only work at the memory model/language level. - -Deeper analysis, to find high-level race conditions only, requires conveying -the intended kernel logic to a tool. This requires (1) the developer writing a -specification or model of their code, and then (2) the tool verifying that the -implementation matches. This has been done for small bits of code using model -checkers and other formal methods, but does not scale to the level of what can -be covered with a dynamic analysis based data race detector such as KCSAN. - -For reasons outlined in this `article <https://lwn.net/Articles/793253/>`_, -data races can be much more subtle, but can cause no less harm than high-level -race conditions. +memory ordering. Developers should therefore carefully consider the required +memory ordering requirements that remain unchecked. If, however, missing +memory ordering (that is observable with a particular compiler and +architecture) leads to an observable data race (e.g. entering a critical +section erroneously), KCSAN would report the resulting data race. + +Race Detection Beyond Data Races +-------------------------------- + +For code with complex concurrency design, race-condition bugs may not always +manifest as data races. Race conditions occur if concurrently executing +operations result in unexpected system behaviour. On the other hand, data races +are defined at the C-language level. The following macros can be used to check +properties of concurrent code where bugs would not manifest as data races. + +.. kernel-doc:: include/linux/kcsan-checks.h + :functions: ASSERT_EXCLUSIVE_WRITER ASSERT_EXCLUSIVE_ACCESS + ASSERT_EXCLUSIVE_BITS Implementation Details ---------------------- -The general approach is inspired by `DataCollider +KCSAN relies on observing that two accesses happen concurrently. Crucially, we +want to (a) increase the chances of observing races (especially for races that +manifest rarely), and (b) be able to actually observe them. We can accomplish +(a) by injecting various delays, and (b) by using address watchpoints (or +breakpoints). + +If we deliberately stall a memory access, while we have a watchpoint for its +address set up, and then observe the watchpoint to fire, two accesses to the +same address just raced. Using hardware watchpoints, this is the approach taken +in `DataCollider <http://usenix.org/legacy/events/osdi10/tech/full_papers/Erickson.pdf>`_. Unlike DataCollider, KCSAN does not use hardware watchpoints, but instead -relies on compiler instrumentation. Watchpoints are implemented using an -efficient encoding that stores access type, size, and address in a long; the -benefits of using "soft watchpoints" are portability and greater flexibility in -limiting which accesses trigger a watchpoint. +relies on compiler instrumentation and "soft watchpoints". -More specifically, KCSAN requires instrumenting plain (unmarked, non-atomic) -memory operations; for each instrumented plain access: +In KCSAN, watchpoints are implemented using an efficient encoding that stores +access type, size, and address in a long; the benefits of using "soft +watchpoints" are portability and greater flexibility. KCSAN then relies on the +compiler instrumenting plain accesses. For each instrumented plain access: 1. Check if a matching watchpoint exists; if yes, and at least one access is a write, then we encountered a racing access. 2. Periodically, if no matching watchpoint exists, set up a watchpoint and - stall for a small delay. + stall for a small randomized delay. 3. Also check the data value before the delay, and re-check the data value after delay; if the values mismatch, we infer a race of unknown origin. -To detect data races between plain and atomic memory operations, KCSAN also -annotates atomic accesses, but only to check if a watchpoint exists -(``kcsan_check_atomic_*``); i.e. KCSAN never sets up a watchpoint on atomic -accesses. +To detect data races between plain and marked accesses, KCSAN also annotates +marked accesses, but only to check if a watchpoint exists; i.e. KCSAN never +sets up a watchpoint on marked accesses. By never setting up watchpoints for +marked operations, if all accesses to a variable that is accessed concurrently +are properly marked, KCSAN will never trigger a watchpoint and therefore never +report the accesses. Key Properties ~~~~~~~~~~~~~~ -1. **Memory Overhead:** The current implementation uses a small array of longs - to encode watchpoint information, which is negligible. +1. **Memory Overhead:** The overall memory overhead is only a few MiB + depending on configuration. The current implementation uses a small array of + longs to encode watchpoint information, which is negligible. 2. **Performance Overhead:** KCSAN's runtime aims to be minimal, using an efficient watchpoint encoding that does not require acquiring any shared
@@ -253,14 +311,17 @@ Key Properties Alternatives Considered ----------------------- -An alternative data race detection approach for the kernel can be found in +An alternative data race detection approach for the kernel can be found in the `Kernel Thread Sanitizer (KTSAN) <https://github.com/google/ktsan/wiki>`_. KTSAN is a happens-before data race detector, which explicitly establishes the happens-before order between memory operations, which can then be used to -determine data races as defined in `Data Races`_. To build a correct -happens-before relation, KTSAN must be aware of all ordering rules of the LKMM -and synchronization primitives. Unfortunately, any omission leads to false -positives, which is especially important in the context of the kernel which -includes numerous custom synchronization mechanisms. Furthermore, KTSAN's -implementation requires metadata for each memory location (shadow memory); -currently, for each page, KTSAN requires 4 pages of shadow memory. +determine data races as defined in `Data Races`_. + +To build a correct happens-before relation, KTSAN must be aware of all ordering +rules of the LKMM and synchronization primitives. Unfortunately, any omission +leads to large numbers of false positives, which is especially detrimental in +the context of the kernel which includes numerous custom synchronization +mechanisms. To track the happens-before relation, KTSAN's implementation +requires metadata for each memory location (shadow memory), which for each page +corresponds to 4 pages of shadow memory, and can translate into overhead of +tens of GiB on a large system.
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH v4 6/15 kcsan: Update API documentation in kcsan-checks.h
In reply to Paul E. McKenneyFrom: Marco Elver <[email protected]>
Update the API documentation for ASSERT_EXCLUSIVE_* macros and make them generate readable documentation for the code examples.
All @variable short summaries were missing ':', which was updated for the whole file.
Tested with "make htmldocs".
--- include/linux/kcsan-checks.h | 98 +++++++++++++++++++++++++++----------------- 1 file changed, 61 insertions(+), 37 deletions(-)
include/linux/kcsan-checks.h +61 -37
@@ -26,9 +26,9 @@/** * __kcsan_check_access - check generic access for races * -* @ptr address of access -* @size size of access -* @type access type modifier +* @ptr: address of access +* @size: size of access +* @type: access type modifier */ void __kcsan_check_access (const volatile void *ptr,size_t size,int type);
@@ -64,7 +64,7 @@ void kcsan_flat_atomic_end(void);* Force treating the next n memory accesses for the current context as atomic * operations. * -* @n number of following memory accesses to treat as atomic. +* @n: number of following memory accesses to treat as atomic. */ void kcsan_atomic_next (int n);
@@ -74,7 +74,7 @@ void kcsan_atomic_next(int n);* Set the access mask for all accesses for the current context if non-zero. * Only value changes to bits set in the mask will be reported. * -* @mask bitmask +* @mask: bitmask */ void kcsan_set_access_mask (unsigned long mask);
@@ -106,16 +106,16 @@ static inline void kcsan_check_access(const volatile void *ptr, size_t size,/** * __kcsan_check_read - check regular read access for races * -* @ptr address of access -* @size size of access +* @ptr: address of access +* @size: size of access */ #define __kcsan_check_read (ptr, size) __kcsan_check_access(ptr, size, 0)/** * __kcsan_check_write - check regular write access for races * -* @ptr address of access -* @size size of access +* @ptr: address of access +* @size: size of access */ #define __kcsan_check_write (ptr, size) \ __kcsan_check_access(ptr, size, KCSAN_ACCESS_WRITE)
@@ -123,16 +123,16 @@ static inline void kcsan_check_access(const volatile void *ptr, size_t size,/** * kcsan_check_read - check regular read access for races * -* @ptr address of access -* @size size of access +* @ptr: address of access +* @size: size of access */ #define kcsan_check_read (ptr, size) kcsan_check_access(ptr, size, 0)/** * kcsan_check_write - check regular write access for races * -* @ptr address of access -* @size size of access +* @ptr: address of access +* @size: size of access */ #define kcsan_check_write (ptr, size) \ kcsan_check_access(ptr, size, KCSAN_ACCESS_WRITE)
@@ -158,14 +158,26 @@ static inline void kcsan_check_access(const volatile void *ptr, size_t size,* allowed. This assertion can be used to specify properties of concurrent code, * where violation cannot be detected as a normal data race. * -* For example, if a per-CPU variable is only meant to be written by a single -* CPU, but may be read from other CPUs; in this case, reads and writes must be -* marked properly, however, if an off-CPU WRITE_ONCE() races with the owning -* CPU's WRITE_ONCE(), would not constitute a data race but could be a harmful -* race condition. Using this macro allows specifying this property in the code -* and catch such bugs. +* For example, if we only have a single writer, but multiple concurrent +* readers, to avoid data races, all these accesses must be marked; even +* concurrent marked writes racing with the single writer are bugs. +* Unfortunately, due to being marked, they are no longer data races. For cases +* like these, we can use the macro as follows: * -* @var variable to assert on +* .. code-block:: c +* +* void writer(void) { +* spin_lock(&update_foo_lock); +* ASSERT_EXCLUSIVE_WRITER(shared_foo); +* WRITE_ONCE(shared_foo, ...); +* spin_unlock(&update_foo_lock); +* } +* void reader(void) { +* // update_foo_lock does not need to be held! +* ... = READ_ONCE(shared_foo); +* } +* +* @var: variable to assert on */ #define ASSERT_EXCLUSIVE_WRITER (var) \ __kcsan_check_access(&(var), sizeof(var), KCSAN_ACCESS_ASSERT)
@@ -177,16 +189,22 @@ static inline void kcsan_check_access(const volatile void *ptr, size_t size,* writers). This assertion can be used to specify properties of concurrent * code, where violation cannot be detected as a normal data race. * -* For example, in a reference-counting algorithm where exclusive access is -* expected after the refcount reaches 0. We can check that this property -* actually holds as follows: +* For example, where exclusive access is expected after determining no other +* users of an object are left, but the object is not actually freed. We can +* check that this property actually holds as follows: +* +* .. code-block:: c * * if (refcount_dec_and_test(&obj->refcnt)) { * ASSERT_EXCLUSIVE_ACCESS(*obj); -* safely_dispose_of(obj); +* do_some_cleanup(obj); +* release_for_reuse(obj); * } * -* @var variable to assert on +* Note: For cases where the object is freed, `KASAN <kasan.html>`_ is a better +* fit to detect use-after-free bugs. +* +* @var: variable to assert on */ #define ASSERT_EXCLUSIVE_ACCESS (var) \ __kcsan_check_access(&(var), sizeof(var), KCSAN_ACCESS_WRITE | KCSAN_ACCESS_ASSERT)
@@ -200,7 +218,7 @@ static inline void kcsan_check_access(const volatile void *ptr, size_t size,* concurrent readers are permitted. This assertion captures more detailed * bit-level properties, compared to the other (word granularity) assertions. * Only the bits set in @mask are checked for concurrent modifications, while -* ignoring the remaining bits, i.e. concurrent writes (or reads) to ~@mask bits +* ignoring the remaining bits, i.e. concurrent writes (or reads) to ~mask bits * are ignored. * * Use this for variables, where some bits must not be modified concurrently,
@@ -210,17 +228,21 @@ static inline void kcsan_check_access(const volatile void *ptr, size_t size,* but other bits may still be modified concurrently. A reader may wish to * assert that this is true as follows: * +* .. code-block:: c +* * ASSERT_EXCLUSIVE_BITS(flags, READ_ONLY_MASK); * foo = (READ_ONCE(flags) & READ_ONLY_MASK) >> READ_ONLY_SHIFT; * -* Note: The access that immediately follows ASSERT_EXCLUSIVE_BITS() is -* assumed to access the masked bits only, and KCSAN optimistically assumes it -* is therefore safe, even in the presence of data races, and marking it with -* READ_ONCE() is optional from KCSAN's point-of-view. We caution, however, -* that it may still be advisable to do so, since we cannot reason about all -* compiler optimizations when it comes to bit manipulations (on the reader -* and writer side). If you are sure nothing can go wrong, we can write the -* above simply as: +* Note: The access that immediately follows ASSERT_EXCLUSIVE_BITS() is assumed +* to access the masked bits only, and KCSAN optimistically assumes it is +* therefore safe, even in the presence of data races, and marking it with +* READ_ONCE() is optional from KCSAN's point-of-view. We caution, however, that +* it may still be advisable to do so, since we cannot reason about all compiler +* optimizations when it comes to bit manipulations (on the reader and writer +* side). If you are sure nothing can go wrong, we can write the above simply +* as: +* +* .. code-block:: c * * ASSERT_EXCLUSIVE_BITS(flags, READ_ONLY_MASK); * foo = (flags & READ_ONLY_MASK) >> READ_ONLY_SHIFT;
@@ -230,15 +252,17 @@ static inline void kcsan_check_access(const volatile void *ptr, size_t size,* be modified concurrently. Writers, where other bits may change concurrently, * could use the assertion as follows: * +* .. code-block:: c +* * spin_lock(&foo_lock); * ASSERT_EXCLUSIVE_BITS(flags, FOO_MASK); -* old_flags = READ_ONCE(flags); +* old_flags = flags; * new_flags = (old_flags & ~FOO_MASK) | (new_foo << FOO_SHIFT); * if (cmpxchg(&flags, old_flags, new_flags) != old_flags) { ... } * spin_unlock(&foo_lock); * -* @var variable to assert on -* @mask only check for modifications to bits set in @mask +* @var: variable to assert on +* @mask: only check for modifications to bits set in @mask */ #define ASSERT_EXCLUSIVE_BITS (var, mask) \ do { \
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH v4 7/15 kcsan: Introduce report access_info and other_info
In reply to Paul E. McKenneyFrom: Marco Elver <[email protected]>
Improve readability by introducing access_info and other_info structs, and in preparation of the following commit in this series replaces the single instance of other_info with an array of size 1.
No functional change intended.
--- kernel/kcsan/core.c | 6 +-- kernel/kcsan/kcsan.h | 2 +- kernel/kcsan/report.c | 147 +++++++++++++++++++++++++------------------------- 3 files changed, 77 insertions(+), 78 deletions(-)
kernel/kcsan/core.c +2 -4
@@ -321,7 +321,7 @@ static noinline void kcsan_found_watchpoint(const volatile void *ptr, flags =user_access_save ();if (consumed) { -kcsan_report (ptr, size, type, true,raw_smp_processor_id (), +kcsan_report (ptr, size, type,KCSAN_VALUE_CHANGE_MAYBE ,KCSAN_REPORT_CONSUMED_WATCHPOINT ); }else {/*
@@ -500,8 +500,7 @@ kcsan_setup_watchpoint(const volatile void *ptr, size_t size, int type)if (is_assert && value_change ==KCSAN_VALUE_CHANGE_TRUE )kcsan_counter_inc (KCSAN_COUNTER_ASSERT_FAILURES ); -kcsan_report (ptr, size, type, value_change,raw_smp_processor_id (), -KCSAN_REPORT_RACE_SIGNAL ); +kcsan_report (ptr, size, type, value_change,KCSAN_REPORT_RACE_SIGNAL ); }else if (value_change ==KCSAN_VALUE_CHANGE_TRUE ) {/* Inferring a race, since the value should not have changed. */
@@ -511,7 +510,6 @@ kcsan_setup_watchpoint(const volatile void *ptr, size_t size, int type)if (IS_ENABLED (CONFIG_KCSAN_REPORT_RACE_UNKNOWN_ORIGIN ) || is_assert)kcsan_report (ptr, size, type,KCSAN_VALUE_CHANGE_TRUE , -raw_smp_processor_id (),KCSAN_REPORT_RACE_UNKNOWN_ORIGIN ); }
kernel/kcsan/kcsan.h +1 -1
@@ -135,7 +135,7 @@ enum kcsan_report_type {* Print a race report from thread that encountered the race. */ extern void kcsan_report (const volatile void *ptr,size_t size,int access_type, -enum kcsan_value_change value_change,int cpu_id, +enum kcsan_value_change value_change,enum kcsan_report_type type);#endif /* _KERNEL_KCSAN_KCSAN_H */
kernel/kcsan/report.c +74 -73
323 lines not on this page · raw message
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH v4 8/15 kcsan: Avoid blocking producers in prepare_report()
In reply to Paul E. McKenneyFrom: Marco Elver <[email protected]>
To avoid deadlock in case watchers can be interrupted, we need to ensure that producers of the struct other_info can never be blocked by an unrelated consumer. (Likely to occur with KCSAN_INTERRUPT_WATCHER.)
There are several cases that can lead to this scenario, for example:
1. A watchpoint A was set up by task T1, but interrupted by interrupt I1. Some other thread (task or interrupt) finds watchpoint A consumes it, and sets other_info. Then I1 also finds some unrelated watchpoint B, consumes it, but is blocked because other_info is in use. T1 cannot consume other_info because I1 never returns -> deadlock.
2. A watchpoint A was set up by task T1, but interrupted by interrupt I1, which also sets up a watchpoint B. Some other thread finds watchpoint A, and consumes it and sets up other_info with its information. Similarly some other thread finds watchpoint B and consumes it, but is then blocked because other_info is in use. When I1 continues it sees its watchpoint was consumed, and that it must wait for other_info, which currently contains information to be consumed by T1. However, T1 cannot unblock other_info because I1 never returns -> deadlock.
To avoid this, we need to ensure that producers of struct other_info always have a usable other_info entry. This is obviously not the case with only a single instance of struct other_info, as concurrent producers must wait for the entry to be released by some consumer (which may be locked up as illustrated above).
While it would be nice if producers could simply call kmalloc() and append their instance of struct other_info to a list, we are very limited in this code path: since KCSAN can instrument the allocators themselves, calling kmalloc() could lead to deadlock or corrupted allocator state.
Since producers of the struct other_info will always succeed at try_consume_watchpoint(), preceding the call into kcsan_report(), we know that the particular watchpoint slot cannot simply be reused or consumed by another potential other_info producer. If we move removal of a watchpoint after reporting (by the consumer of struct other_info), we can see a consumed watchpoint as a held lock on elements of other_info, if we create a one-to-one mapping of a watchpoint to an other_info element.
Therefore, the simplest solution is to create an array of struct other_info that is as large as the watchpoints array in core.c, and pass the watchpoint index to kcsan_report() for producers and consumers, and change watchpoints to be removed after reporting is done.
With a default config on a 64-bit system, the array other_infos consumes ~37KiB. For most systems today this is not a problem. On smaller memory constrained systems, the config value CONFIG_KCSAN_NUM_WATCHPOINTS can be reduced appropriately.
Overall, this change is a simplification of the prepare_report() code, and makes some of the checks (such as checking if at least one access is a write) redundant.
Tested:
$ tools/testing/selftests/rcutorture/bin/kvm.sh \ --cpus 12 --duration 10 --kconfig "CONFIG_DEBUG_INFO=y \ CONFIG_KCSAN=y CONFIG_KCSAN_ASSUME_PLAIN_WRITES_ATOMIC=n \ CONFIG_KCSAN_REPORT_VALUE_CHANGE_ONLY=n \ CONFIG_KCSAN_REPORT_ONCE_IN_MS=100000 CONFIG_KCSAN_VERBOSE=y \ CONFIG_KCSAN_INTERRUPT_WATCHER=y CONFIG_PROVE_LOCKING=y" \ --configs TREE03 => No longer hangs and runs to completion as expected.
--- kernel/kcsan/core.c | 31 +++++--- kernel/kcsan/kcsan.h | 3 +- kernel/kcsan/report.c | 212 ++++++++++++++++++++++++-------------------------- 3 files changed, 124 insertions(+), 122 deletions(-)
kernel/kcsan/core.c +21 -10
@@ -69,7 +69,6 @@ static DEFINE_PER_CPU(struct kcsan_ctx, kcsan_cpu_ctx) = {* slot=9: [10, 11, 9] * slot=63: [64, 65, 63] */ -#define NUM_SLOTS (1 + 2*KCSAN_CHECK_ADJACENT)#define SLOT_IDX (slot, i) (slot + ((i + KCSAN_CHECK_ADJACENT) % NUM_SLOTS))/*
@@ -171,12 +170,16 @@ try_consume_watchpoint(atomic_long_t *watchpoint, long encoded_watchpoint)return atomic_long_try_cmpxchg_relaxed (watchpoint, &encoded_watchpoint,CONSUMED_WATCHPOINT ); } -/* -* Return true if watchpoint was not touched, false if consumed. -*/ -static inline bool remove_watchpoint (atomic_long_t *watchpoint) +/* Return true if watchpoint was not touched, false if already consumed. */ +static inline bool consume_watchpoint (atomic_long_t *watchpoint) { -return atomic_long_xchg_relaxed (watchpoint,INVALID_WATCHPOINT ) !=CONSUMED_WATCHPOINT ; +return atomic_long_xchg_relaxed (watchpoint,CONSUMED_WATCHPOINT ) !=CONSUMED_WATCHPOINT ; +} + +/* Remove the watchpoint -- its slot may be reused after. */ +static inline void remove_watchpoint (atomic_long_t *watchpoint) +{ +atomic_long_set (watchpoint,INVALID_WATCHPOINT ); } static __always_inline struct kcsan_ctx *get_ctx(void)
@@ -322,7 +325,8 @@ static noinline void kcsan_found_watchpoint(const volatile void *ptr,if (consumed) {kcsan_report (ptr, size, type,KCSAN_VALUE_CHANGE_MAYBE , -KCSAN_REPORT_CONSUMED_WATCHPOINT ); +KCSAN_REPORT_CONSUMED_WATCHPOINT , + watchpoint - watchpoints); }else {/* * The other thread may not print any diagnostics, as it has
@@ -470,7 +474,7 @@ kcsan_setup_watchpoint(const volatile void *ptr, size_t size, int type) value_change =KCSAN_VALUE_CHANGE_TRUE ;/* Check if this access raced with another. */ -if (!remove_watchpoint (watchpoint)) { +if (!consume_watchpoint (watchpoint)) {/* * Depending on the access type, map a value_change of MAYBE to * TRUE (always report) or FALSE (never report).
@@ -500,7 +504,8 @@ kcsan_setup_watchpoint(const volatile void *ptr, size_t size, int type)if (is_assert && value_change ==KCSAN_VALUE_CHANGE_TRUE )kcsan_counter_inc (KCSAN_COUNTER_ASSERT_FAILURES ); -kcsan_report (ptr, size, type, value_change,KCSAN_REPORT_RACE_SIGNAL ); +kcsan_report (ptr, size, type, value_change,KCSAN_REPORT_RACE_SIGNAL , + watchpoint - watchpoints); }else if (value_change ==KCSAN_VALUE_CHANGE_TRUE ) {/* Inferring a race, since the value should not have changed. */
@@ -510,9 +515,15 @@ kcsan_setup_watchpoint(const volatile void *ptr, size_t size, int type)if (IS_ENABLED (CONFIG_KCSAN_REPORT_RACE_UNKNOWN_ORIGIN ) || is_assert)kcsan_report (ptr, size, type,KCSAN_VALUE_CHANGE_TRUE , -KCSAN_REPORT_RACE_UNKNOWN_ORIGIN ); +KCSAN_REPORT_RACE_UNKNOWN_ORIGIN , + watchpoint - watchpoints); } +/* +* Remove watchpoint; must be after reporting, since the slot may be +* reused after this point. +*/ +remove_watchpoint (watchpoint);kcsan_counter_dec (KCSAN_COUNTER_USED_WATCHPOINTS ); out_unlock:if (!kcsan_interrupt_watcher)
kernel/kcsan/kcsan.h +2 -1
@@ -12,6 +12,7 @@/* The number of adjacent watchpoints to check. */ #define KCSAN_CHECK_ADJACENT 1 +#define NUM_SLOTS (1 + 2*KCSAN_CHECK_ADJACENT)extern unsigned int kcsan_udelay_task;extern unsigned int kcsan_udelay_interrupt;
@@ -136,6 +137,6 @@ enum kcsan_report_type {*/ extern void kcsan_report (const volatile void *ptr,size_t size,int access_type,enum kcsan_value_change value_change, -enum kcsan_report_type type); +enum kcsan_report_type type,int watchpoint_idx);#endif /* _KERNEL_KCSAN_KCSAN_H */
kernel/kcsan/report.c +101 -111
299 lines not on this page · raw message
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH v4 9/15 kcsan: Add support for scoped accesses
In reply to Paul E. McKenneyFrom: Marco Elver <[email protected]>
This adds support for scoped accesses, where the memory range is checked for the duration of the scope. The feature is implemented by inserting the relevant access information into a list of scoped accesses for the current execution context, which are then checked (until removed) on every call (through instrumentation) into the KCSAN runtime.
An alternative, more complex, implementation could set up a watchpoint for the scoped access, and keep the watchpoint set up. This, however, would require first exposing a handle to the watchpoint, as well as dealing with cases such as accesses by the same thread while the watchpoint is still set up (and several more cases). It is also doubtful if this would provide any benefit, since the majority of delay where the watchpoint is set up is likely due to the injected delays by KCSAN. Therefore, the implementation in this patch is simpler and avoids hurting KCSAN's main use-case (normal data race detection); it also implicitly increases scoped-access race-detection-ability due to increased probability of setting up watchpoints by repeatedly calling __kcsan_check_access() throughout the scope of the access.
The implementation required adding an additional conditional branch to the fast-path. However, the microbenchmark showed a *speedup* of ~5% on the fast-path. This appears to be due to subtly improved codegen by GCC from moving get_ctx() and associated load of preempt_count earlier.
--- include/linux/kcsan-checks.h | 57 ++++++++++++++++++++++++++++++ include/linux/kcsan.h | 3 ++ init/init_task.c | 1 + kernel/kcsan/core.c | 83 +++++++++++++++++++++++++++++++++++++++----- kernel/kcsan/report.c | 33 ++++++++++++------ 5 files changed, 158 insertions(+), 19 deletions(-)
include/linux/kcsan-checks.h +57 -0
@@ -3,6 +3,8 @@#ifndef _LINUX_KCSAN_CHECKS_H#define _LINUX_KCSAN_CHECKS_H +/* Note: Only include what is already included by compiler.h. */ +#include <linux/compiler_attributes.h> #include <linux/types.h> /*
@@ -12,10 +14,12 @@* WRITE : write access; * ATOMIC: access is atomic; * ASSERT: access is not a regular access, but an assertion; +* SCOPED: access is a scoped access; */ #define KCSAN_ACCESS_WRITE 0x1#define KCSAN_ACCESS_ATOMIC 0x2#define KCSAN_ACCESS_ASSERT 0x4 +#define KCSAN_ACCESS_SCOPED 0x8/* * __kcsan_*: Always calls into the runtime when KCSAN is enabled. This may be used
@@ -78,6 +82,52 @@ void kcsan_atomic_next(int n);*/ void kcsan_set_access_mask (unsigned long mask); +/* Scoped access information. */ +struct kcsan_scoped_access { +struct list_head list; +const volatile void *ptr; +size_t size; +int type; +}; +/* +* Automatically call kcsan_end_scoped_access() when kcsan_scoped_access goes +* out of scope; relies on attribute "cleanup", which is supported by all +* compilers that support KCSAN. +*/ +#define __kcsan_cleanup_scoped \ + __maybe_unused __attribute__((__cleanup__(kcsan_end_scoped_access))) + +/** +* kcsan_begin_scoped_access - begin scoped access +* +* Begin scoped access and initialize @sa, which will cause KCSAN to +* continuously check the memory range in the current thread until +* kcsan_end_scoped_access() is called for @sa. +* +* Scoped accesses are implemented by appending @sa to an internal list for the +* current execution context, and then checked on every call into the KCSAN +* runtime. +* +* @ptr: address of access +* @size: size of access +* @type: access type modifier +* @sa: struct kcsan_scoped_access to use for the scope of the access +*/ +struct kcsan_scoped_access * +kcsan_begin_scoped_access (const volatile void *ptr,size_t size,int type, +struct kcsan_scoped_access *sa); + +/** +* kcsan_end_scoped_access - end scoped access +* +* End a scoped access, which will stop KCSAN checking the memory range. +* Requires that kcsan_begin_scoped_access() was previously called once for @sa. +* +* @sa: a previously initialized struct kcsan_scoped_access +*/ +void kcsan_end_scoped_access (struct kcsan_scoped_access *sa); + +#else /* CONFIG_KCSAN */ static inline void __kcsan_check_access(const volatile void *ptr, size_t size,
@@ -90,6 +140,13 @@ static inline void kcsan_flat_atomic_end(void) { }static inline void kcsan_atomic_next (int n) { }static inline void kcsan_set_access_mask (unsigned long mask) { } +struct kcsan_scoped_access { }; +#define __kcsan_cleanup_scoped __maybe_unused +static inline struct kcsan_scoped_access * +kcsan_begin_scoped_access (const volatile void *ptr,size_t size,int type, +struct kcsan_scoped_access *sa) {return sa; } +static inline void kcsan_end_scoped_access (struct kcsan_scoped_access *sa) { } +#endif /* CONFIG_KCSAN */ /*
include/linux/kcsan.h +3 -0
@@ -40,6 +40,9 @@ struct kcsan_ctx {* Access mask for all accesses if non-zero. */ unsigned long access_mask; + +/* List of scoped accesses. */ +struct list_head scoped_accesses; };/**
init/init_task.c +1 -0
@@ -168,6 +168,7 @@ struct task_struct init_task .atomic_nest_count = 0, .in_flat_atomic = false, .access_mask = 0, + .scoped_accesses = {LIST_POISON1, NULL}, }, #endif #ifdef CONFIG_TRACE_IRQFLAGS
kernel/kcsan/core.c +75 -8
134 lines not on this page · raw message
kernel/kcsan/report.c +22 -11
@@ -205,6 +205,20 @@ skip_report(enum kcsan_value_change value_change, unsigned long top_frame)static const char *get_access_type (int type) { +if (type &KCSAN_ACCESS_ASSERT ) { +if (type &KCSAN_ACCESS_SCOPED ) { +if (type &KCSAN_ACCESS_WRITE ) +return "assert no accesses (scoped)" ; +else +return "assert no writes (scoped)" ; + }else { +if (type &KCSAN_ACCESS_WRITE ) +return "assert no accesses" ; +else +return "assert no writes" ; + } + } +switch (type) {case 0 :return "read" ;
@@ -214,17 +228,14 @@ static const char *get_access_type(int type)return "write" ;case KCSAN_ACCESS_WRITE |KCSAN_ACCESS_ATOMIC :return "write (marked)" ; - -/* -* ASSERT variants: -*/ -case KCSAN_ACCESS_ASSERT : -case KCSAN_ACCESS_ASSERT |KCSAN_ACCESS_ATOMIC : -return "assert no writes" ; -case KCSAN_ACCESS_ASSERT |KCSAN_ACCESS_WRITE : -case KCSAN_ACCESS_ASSERT |KCSAN_ACCESS_WRITE |KCSAN_ACCESS_ATOMIC : -return "assert no accesses" ; - +case KCSAN_ACCESS_SCOPED : +return "read (scoped)" ; +case KCSAN_ACCESS_SCOPED |KCSAN_ACCESS_ATOMIC : +return "read (marked, scoped)" ; +case KCSAN_ACCESS_SCOPED |KCSAN_ACCESS_WRITE : +return "write (scoped)" ; +case KCSAN_ACCESS_SCOPED |KCSAN_ACCESS_WRITE |KCSAN_ACCESS_ATOMIC : +return "write (marked, scoped)" ;default :BUG (); }
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH v4 10/15 objtool, kcsan: Add explicit check functions to uaccess whitelist
In reply to Paul E. McKenneyFrom: Marco Elver <[email protected]>
Add explicitly invoked KCSAN check functions to objtool's uaccess whitelist. This is needed in order to permit calling into kcsan_check_scoped_accesses() from the fast-path, which in turn calls __kcsan_check_access(). __kcsan_check_access() is the generic variant of the already whitelisted specializations __tsan_{read,write}N.
--- tools/objtool/check.c | 2 ++ 1 file changed, 2 insertions(+)
tools/objtool/check.c +2 -0
@@ -468,8 +468,10 @@ static const char *uaccess_safe_builtin[] = {"__asan_report_store8_noabort" ,"__asan_report_store16_noabort" ,/* KCSAN */ +"__kcsan_check_access" ,"kcsan_found_watchpoint" ,"kcsan_setup_watchpoint" , +"kcsan_check_scoped_accesses" ,/* KCSAN/TSAN */ "__tsan_func_entry" , "__tsan_func_exit",
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH v4 11/15 kcsan: Introduce scoped ASSERT_EXCLUSIVE macros
In reply to Paul E. McKenneyFrom: Marco Elver <[email protected]>
Introduce ASSERT_EXCLUSIVE_*_SCOPED(), which provide an intuitive interface to use the scoped-access feature, without having to explicitly mark the start and end of the desired scope. Basing duration of the checks on scope avoids accidental misuse and resulting false positives, which may be hard to debug. See added comments for usage.
The macros are implemented using __attribute__((__cleanup__(func))), which is supported by all compilers that currently support KCSAN.
--- Documentation/dev-tools/kcsan.rst | 3 +- include/linux/kcsan-checks.h | 73 ++++++++++++++++++++++++++++++++++++++- kernel/kcsan/debugfs.c | 16 ++++++++- 3 files changed, 89 insertions(+), 3 deletions(-)
Documentation/dev-tools/kcsan.rst +2 -1
@@ -238,7 +238,8 @@ are defined at the C-language level. The following macros can be used to check properties of concurrent code where bugs would not manifest as data races. .. kernel-doc:: include/linux/kcsan-checks.h - :functions: ASSERT_EXCLUSIVE_WRITER ASSERT_EXCLUSIVE_ACCESS + :functions: ASSERT_EXCLUSIVE_WRITER ASSERT_EXCLUSIVE_WRITER_SCOPED + ASSERT_EXCLUSIVE_ACCESS ASSERT_EXCLUSIVE_ACCESS_SCOPED ASSERT_EXCLUSIVE_BITS Implementation Details
include/linux/kcsan-checks.h +72 -1
98 lines not on this page · raw message
kernel/kcsan/debugfs.c +15 -1
@@ -110,6 +110,7 @@ static noinline void microbenchmark(unsigned long iters)*/ static long test_dummy;static long test_flags; +static long test_scoped; static noinline void test_thread(unsigned long iters) {const long CHANGE_BITS =0xff00ff00ff00ff00L ;
@@ -120,7 +121,8 @@ static noinline void test_thread(unsigned long iters)memset (¤t->kcsan_ctx,0 ,sizeof (current->kcsan_ctx));pr_info ("KCSAN: %s begin | iters: %lu\n" , __func__, iters); -pr_info ("test_dummy@%px, test_flags@%px\n" , &test_dummy, &test_flags); +pr_info ("test_dummy@%px, test_flags@%px, test_scoped@%px,\n" , + &test_dummy, &test_flags, &test_scoped); cycles =get_cycles ();while (iters--) {
@@ -141,6 +143,18 @@ static noinline void test_thread(unsigned long iters) test_flags ^=CHANGE_BITS ;/* generate value-change */ __kcsan_check_write (&test_flags,sizeof (test_flags)); + +BUG_ON (current->kcsan_ctx.scoped_accesses.prev); + { +/* Should generate reports anywhere in this block. */ +ASSERT_EXCLUSIVE_WRITER_SCOPED (test_scoped); +ASSERT_EXCLUSIVE_ACCESS_SCOPED (test_scoped); +BUG_ON (!current->kcsan_ctx.scoped_accesses.prev); +/* Unrelated accesses. */ +__kcsan_check_access (&cycles,sizeof (cycles),0 ); +__kcsan_check_access (&cycles,sizeof (cycles),KCSAN_ACCESS_ATOMIC ); + } +BUG_ON (current->kcsan_ctx.scoped_accesses.prev); } cycles =get_cycles () - cycles;
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH v4 12/15 kcsan: Move kcsan_{disable,enable}_current() to kcsan-checks.h
In reply to Paul E. McKenneyFrom: Marco Elver <[email protected]>
Both affect access checks, and should therefore be in kcsan-checks.h. This is in preparation to use these in compiler.h.
--- include/linux/kcsan-checks.h | 16 ++++++++++++++++ include/linux/kcsan.h | 16 ---------------- 2 files changed, 16 insertions(+), 16 deletions(-)
include/linux/kcsan-checks.h +16 -0
@@ -37,6 +37,20 @@void __kcsan_check_access (const volatile void *ptr,size_t size,int type);/** +* kcsan_disable_current - disable KCSAN for the current context +* +* Supports nesting. +*/ +void kcsan_disable_current (void ); + +/** +* kcsan_enable_current - re-enable KCSAN for the current context +* +* Supports nesting. +*/ +void kcsan_enable_current (void ); + +/** * kcsan_nestable_atomic_begin - begin nestable atomic region * * Accesses within the atomic region may appear to race with other accesses but
@@ -133,6 +147,8 @@ void kcsan_end_scoped_access(struct kcsan_scoped_access *sa);static inline void __kcsan_check_access (const volatile void *ptr,size_t size,int type) { } +static inline void kcsan_disable_current (void ) { } +static inline void kcsan_enable_current (void ) { }static inline void kcsan_nestable_atomic_begin (void ) { }static inline void kcsan_nestable_atomic_end (void ) { }static inline void kcsan_flat_atomic_begin (void ) { }
include/linux/kcsan.h +0 -16
@@ -50,25 +50,9 @@ struct kcsan_ctx {*/ void kcsan_init (void ); -/** -* kcsan_disable_current - disable KCSAN for the current context -* -* Supports nesting. -*/ -void kcsan_disable_current (void ); - -/** -* kcsan_enable_current - re-enable KCSAN for the current context -* -* Supports nesting. -*/ -void kcsan_enable_current (void ); -#else /* CONFIG_KCSAN */ static inline void kcsan_init (void ) { } -static inline void kcsan_disable_current (void ) { } -static inline void kcsan_enable_current (void ) { }#endif /* CONFIG_KCSAN */
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH v4 13/15 kcsan: Change data_race() to no longer require marking racing accesses
In reply to Paul E. McKenneyFrom: Marco Elver <[email protected]>
Thus far, accesses marked with data_race() would still require the racing access to be marked in some way (be it with READ_ONCE(), WRITE_ONCE(), or data_race() itself), as otherwise KCSAN would still report a data race. This requirement, however, seems to be unintuitive, and some valid use-cases demand *not* marking other accesses, as it might hide more serious bugs (e.g. diagnostic reads).
Therefore, this commit changes data_race() to no longer require marking racing accesses (although it's still recommended if possible).
The alternative would have been introducing another variant of data_race(), however, since usage of data_race() already needs to be carefully reasoned about, distinguishing between these cases likely adds more complexity in the wrong place.
--- include/linux/compiler.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
include/linux/compiler.h +2 -2
@@ -326,9 +326,9 @@ unsigned long read_word_at_a_time(const void *addr)#define data_race (expr) \ ({ \ typeof(({ expr; })) __val; \ - kcsan_nestable_atomic_begin(); \ + kcsan_disable_current(); \ __val = ({ expr; }); \ - kcsan_nestable_atomic_end(); \ + kcsan_enable_current(); \ __val; \ })#else
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH v4 14/15 kcsan: Fix function matching in report
In reply to Paul E. McKenneyFrom: Marco Elver <[email protected]>
Pass string length as returned by scnprintf() to strnstr(), since strnstr() searches exactly len bytes in haystack, even if it contains a NUL-terminator before haystack+len.
--- kernel/kcsan/report.c | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-)
kernel/kcsan/report.c +9 -9
@@ -192,11 +192,11 @@ skip_report(enum kcsan_value_change value_change, unsigned long top_frame)* maintainers. */ char buf[64 ]; +int len =scnprintf (buf,sizeof (buf),"%ps" , (void *)top_frame); -snprintf (buf,sizeof (buf),"%ps" , (void *)top_frame); -if (!strnstr (buf,"rcu_" ,sizeof (buf)) && - !strnstr (buf,"_rcu" ,sizeof (buf)) && - !strnstr (buf,"_srcu" ,sizeof (buf))) +if (!strnstr (buf,"rcu_" , len) && + !strnstr (buf,"_rcu" , len) && + !strnstr (buf,"_srcu" , len))return true; }
@@ -262,15 +262,15 @@ static const char *get_thread_desc(int task_id)static int get_stack_skipnr (const unsigned long stack_entries[],int num_entries) {char buf[64 ]; +int len;int skip =0 ;for (; skip < num_entries; ++skip) { -snprintf (buf,sizeof (buf),"%ps" , (void *)stack_entries[skip]); -if (!strnstr (buf,"csan_" ,sizeof (buf)) && - !strnstr (buf,"tsan_" ,sizeof (buf)) && - !strnstr (buf,"_once_size" ,sizeof (buf))) { + len =scnprintf (buf,sizeof (buf),"%ps" , (void *)stack_entries[skip]); +if (!strnstr (buf,"csan_" , len) && + !strnstr (buf,"tsan_" , len) && + !strnstr (buf,"_once_size" , len))break ; - } }return skip; }
Signature · 1 line
2.9.5
[email protected]
maintainerPATCH v4 15/15 kcsan: Make reporting aware of KCSAN tests
In reply to Paul E. McKenneyFrom: Marco Elver <[email protected]>
Reporting hides KCSAN runtime functions in the stack trace, with filtering done based on function names. Currently this included all functions (or modules) that would match "kcsan_". Make the filter aware of KCSAN tests, which contain "kcsan_test", and are no longer skipped in the report.
This is in preparation for adding a KCSAN test module.
--- kernel/kcsan/report.c | 30 +++++++++++++++++++++++------- 1 file changed, 23 insertions(+), 7 deletions(-)
kernel/kcsan/report.c +23 -7
@@ -262,16 +262,32 @@ static const char *get_thread_desc(int task_id)static int get_stack_skipnr (const unsigned long stack_entries[],int num_entries) {char buf[64 ]; -int len; -int skip =0 ; +char *cur; +int len, skip; -for (; skip < num_entries; ++skip) { +for (skip =0 ; skip < num_entries; ++skip) { len =scnprintf (buf,sizeof (buf),"%ps" , (void *)stack_entries[skip]); -if (!strnstr (buf,"csan_" , len) && - !strnstr (buf,"tsan_" , len) && - !strnstr (buf,"_once_size" , len)) -break ; + +/* Never show tsan_* or {read,write}_once_size. */ +if (strnstr (buf,"tsan_" , len) || +strnstr (buf,"_once_size" , len)) +continue ; + + cur =strnstr (buf,"kcsan_" , len); +if (cur) { + cur +=sizeof ("kcsan_" ) -1 ; +if (strncmp (cur,"test" ,sizeof ("test" ) -1 )) +continue ;/* KCSAN runtime function. */ +/* KCSAN related test. */ + } + +/* +* No match for runtime functions -- @skip entries to skip to +* get to first frame of interest. +*/ +break ; } +return skip; }
Signature · 1 line
2.9.5