asm-generic/mmiowb: Mark accesses to fix KCSAN warnings
PATCH 1/1 asm-generic/mmiowb: Mark accesses to fix KCSAN warnings
Prior to this patch, data races are detectable by KCSAN of the following forms:
[1] Asynchronous calls to mmiowb_set_pending() from an interrupt context or otherwise outside of a critical section
[2] Interrupted critical sections, where the interrupt will itself acquire a lock
In case [1], calling context does not need an mmiowb() call to be issued, otherwise it would do so itself. Such calls to mmiowb_set_pending() are either idempotent or no-ops.
In case [2], irrespective of when the interrupt occurs, the interrupt will acquire and release its locks prior to its return, nesting_count will continue balanced. In the worst case, the interrupted critical section during a mmiowb_spin_unlock() call observes an mmiowb to be pending and afterward is interrupted, leading to an extraneous call to mmiowb(). This data race is clearly innocuous.
Resolve KCSAN warnings of type [1] by means of READ_ONCE, WRITE_ONCE. As increments and decrements to nesting_count are balanced by interrupt contexts, resolve type [2] warnings by simply revoking instrumentation, with data_race() rather than READ_ONCE() and WRITE_ONCE(), the memory consistency semantics of plain-accesses will still lead to correct behaviour.
---
Previously discussed here:
https://lore.kernel.org/linuxppc-dev/[email protected]/
But pushed back due to affecting other architectures. Reissuing, to linuxppc-dev, as it does not enact a functional change.
--- include/asm-generic/mmiowb.h | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
include/asm-generic/mmiowb.h +9 -6
@@ -37,25 +37,28 @@ static inline void mmiowb_set_pending(void)struct mmiowb_state *ms =__mmiowb_state ();if (likely (ms->nesting_count)) - ms->mmiowb_pending = ms->nesting_count; + WRITE_ONCE(ms->mmiowb_pending, ms->nesting_count); }static inline void mmiowb_spin_lock (void ) {struct mmiowb_state *ms =__mmiowb_state (); - ms->nesting_count++; + +/* Increment need not be atomic. Nestedness is balanced over interrupts. */ +data_race (ms->nesting_count++); }static inline void mmiowb_spin_unlock (void ) {struct mmiowb_state *ms =__mmiowb_state (); +u16 pending =READ_ONCE (ms->mmiowb_pending); -if (unlikely (ms->mmiowb_pending)) { - ms->mmiowb_pending =0 ; +WRITE_ONCE (ms->mmiowb_pending,0 ); +if (unlikely (pending))mmiowb (); - } - ms->nesting_count--; +/* Decrement need not be atomic. Nestedness is balanced over interrupts. */ +data_race (ms->nesting_count--); }#else #define mmiowb_set_pending () do { } while (0)
Signature · 1 line
2.44.0
Quoted 59 lines · Rohan McLure
Prior to this patch, data races are detectable by KCSAN of the following forms:
[1] Asynchronous calls to
mmiowb_set_pending()from an interrupt context or otherwise outside of a critical section
[2] Interrupted critical sections, where the interrupt will itself acquire a lockIn case [1], calling context does not need an
mmiowb()call to be issued, otherwise it would do so itself. Such calls tommiowb_set_pending()are either idempotent or no-ops.In case [2], irrespective of when the interrupt occurs, the interrupt will acquire and release its locks prior to its return,
nesting_countwill continue balanced. In the worst case, the interrupted critical section during ammiowb_spin_unlock()call observes an mmiowb to be pending and afterward is interrupted, leading to an extraneous call tommiowb(). This data race is clearly innocuous.Resolve KCSAN warnings of type [1] by means of
READ_ONCE,WRITE_ONCE. As increments and decrements tonesting_countare balanced by interrupt contexts, resolve type [2] warnings by simply revoking instrumentation, withdata_race()rather thanREAD_ONCE()andWRITE_ONCE(), the memory consistency semantics of plain-accesses will still lead to correct behaviour.Signed-off-by Rohan McLure <[email protected]>Reported-by Michael Ellerman <[email protected]>Reported-by Gautam Menghani <[email protected]>Tested-by Gautam Menghani <[email protected]>Acked-by Arnd Bergmann <[email protected]>---
Previously discussed here:
https://lore.kernel.org/linuxppc-dev/[email protected]/
But pushed back due to affecting other architectures. Reissuing, to linuxppc-dev, as it does not enact a functional change.--- include/asm-generic/mmiowb.h | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)include/asm-generic/mmiowb.h +4
-2@@ -37,25 +37,28 @@ static inline void mmiowb_set_pending(void)struct mmiowb_state *ms =__mmiowb_state ();if (likely (ms->nesting_count)) - ms->mmiowb_pending = ms->nesting_count; +WRITE_ONCE (ms->mmiowb_pending, ms->nesting_count); }static inline void mmiowb_spin_lock (void ) {struct mmiowb_state *ms =__mmiowb_state (); - ms->nesting_count++; + +/* Increment need not be atomic. Nestedness is balanced over interrupts. */ +data_race (ms->nesting_count++);
include/asm-generic/mmiowb.h static inline void mmiowb_set_pending(void) · line 56
Quoted 10 lines
include/asm-generic/mmiowb.h +2
-2}static inline void mmiowb_spin_unlock (void ) {struct mmiowb_state *ms =__mmiowb_state (); +u16 pending =READ_ONCE (ms->mmiowb_pending); -if (unlikely (ms->mmiowb_pending)) { - ms->mmiowb_pending =0 ; +WRITE_ONCE (ms->mmiowb_pending,0 );
Why are you changing this store to be unconditional?
Will