Compiler-Based Capability- and Locking-Analysis
Subsystems (21)
Marco Elver
maintainercover Compiler-Based Capability- and Locking-Analysis
[ Note: Bart and I had concurrently been working on bringing Clang's
-Wthread-safety to the kernel:
https://lore.kernel.org/all/[email protected]/
Having both RFCs out should hopefully provide a good picture on these design points and trade-offs - the approaches differ significantly. ]
Capability analysis is a C language extension, which enables statically checking that user-definable "capabilities" are acquired and released where required. An obvious application is lock-safety checking for the kernel's various synchronization primitives (each of which represents a "capability"), and checking that locking rules are not violated.
Clang originally called the feature "Thread Safety Analysis" [1], with some terminology still using the thread-safety-analysis-only names. This was later changed and the feature became more flexible, gaining the ability to define custom "capabilities". Its foundations can be found in "capability systems", used to specify the permissibility of operations to depend on some capability being held (or not held).
[1] https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
[2] https://www.cs.cornell.edu/talc/papers/capabilities.pdf
Because the feature is not just able to express capabilities related to synchronization primitives, the naming chosen for the kernel departs from Clang's initial "Thread Safety" nomenclature and refers to the feature as "Capability Analysis" to avoid confusion. The implementation still makes references to the older terminology in some places, such as -Wthread-safety being the warning enabled option that also still appears in diagnostic messages.
Enabling capability analysis can be seen as enabling a dialect of Linux C with a Capability System.
Additional details can be found in the added kernel-doc documentation.
=== Development Approach ===
Prior art exists in the form of Sparse's context tracking. Locking annotations on functions exist, so the concept of analyzing locking rules is not foreign to the kernel's codebase.
However, Clang's analysis is more complete vs. Sparse's, with the typical trade-offs in static analysis: improved completeness is sacrificed for more possible false positives or additional annotations required by the programmer. Numerous options exist to disable or opt out certain code from analysis.
This series aims to retain compatibility with Sparse, which can provide tree-wide analysis of a subset of the capability analysis introduced. For the most part, the new (and old) keywords used for annotations are shared between Sparse and Clang.
One big question is how to enable this feature, given we end up with a new dialect of C - 2 approaches have been considered:
A. Tree-wide all-or-nothing approach. This approach requires
tree-wide changes, adding annotations or selective opt-outs. Making additional primitives capability-enabled increases churn, esp. where maintainers are unaware of the feature's existence and how to use it.
Because we can't change the programming language (even if from one C dialect to another) of the kernel overnight, a different approach might cause less friction.
B. A selective, incremental, and much less intrusive approach.
Maintainers of subsystems opt in their modules or directories into "capability analysis" (via Makefile):
CAPABILITY_ANALYSIS_foo.o := y # foo.o only
CAPABILITY_ANALYSIS := y # all TUs
Most (eventually all) synchronization primitives and more capabilities (including ones that could track "irq disabled", "preemption" disabled, etc.) could be supported.
The approach taken by this series if B. This ensures that only subsystems where maintainers are willing to deal with any warnings one way or another. Introducing the feature can be done incrementally, without large tree-wide changes and adding numerous opt-outs and annotations to the majority of code.
=== Initial Uses ===
With this initial series, the following synchronization primitives are
supported: raw_spinlock_t, spinlock_t, rwlock_t, mutex, seqlock_t, bit_spinlock, RCU, SRCU (srcu_struct), rw_semaphore, local_lock_t.
As an initial proof-of-concept, this series also enables capability analysis for the following subsystems: kfence, kcov, stackdepot, rhashtable. (Those subsystems were chosen because I am familiar with their locking rules; rhashtable was chosen semi-randomly as a test because it combines a bunch of things: RCU, mutex, bit_spinlock.)
The initial benefits are static detection of violations of locking rules. As more capabilities are added, we would see more static checking beyond what regular C can provide, all while remaining easy (read quick) to use via the Clang compiler.
=== Appendix ===
The following pending Clang patch is recommended, but not a strong dependency:
https://github.com/llvm/llvm-project/pull/123063
This series is also available at this Git tree:
https://git.kernel.org/pub/scm/linux/kernel/git/melver/linux.git/log/?h=cap-analysis
Marco Elver (24):
compiler_types: Move lock checking attributes to
compiler-capability-analysis.h
compiler-capability-analysis: Rename __cond_lock() to __cond_acquire() compiler-capability-analysis: Add infrastructure for Clang's
capability analysis
compiler-capability-analysis: Add test stub
Documentation: Add documentation for Compiler-Based Capability
Analysis
checkpatch: Warn about capability_unsafe() without comment
cleanup: Basic compatibility with capability analysis
lockdep: Annotate lockdep assertions for capability analysis locking/rwlock, spinlock: Support Clang's capability analysis compiler-capability-analysis: Change __cond_acquires to take return
value
locking/mutex: Support Clang's capability analysis locking/seqlock: Support Clang's capability analysis
bit_spinlock: Include missing <asm/processor.h>
bit_spinlock: Support Clang's capability analysis
rcu: Support Clang's capability analysis
srcu: Support Clang's capability analysis
kref: Add capability-analysis annotations
locking/rwsem: Support Clang's capability analysis locking/local_lock: Support Clang's capability analysis
debugfs: Make debugfs_cancellation a capability struct
kfence: Enable capability analysis
kcov: Enable capability analysis
stackdepot: Enable capability analysis
rhashtable: Enable capability analysis
.../dev-tools/capability-analysis.rst | 149 ++++++ Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/sparse.rst | 4 + Makefile | 1 + .../net/wireless/intel/iwlwifi/iwl-trans.h | 2 +- .../wireless/intel/iwlwifi/pcie/internal.h | 2 +- fs/dlm/lock.c | 2 +- include/linux/bit_spinlock.h | 24 +- include/linux/cleanup.h | 18 +- include/linux/compiler-capability-analysis.h | 407 +++++++++++++++ include/linux/compiler_types.h | 18 +- include/linux/debugfs.h | 12 +- include/linux/kref.h | 2 + include/linux/list_bl.h | 2 + include/linux/local_lock.h | 18 +- include/linux/local_lock_internal.h | 41 +- include/linux/lockdep.h | 12 +- include/linux/mm.h | 6 +- include/linux/mutex.h | 29 +- include/linux/mutex_types.h | 4 +- include/linux/rcupdate.h | 73 ++- include/linux/refcount.h | 6 +- include/linux/rhashtable.h | 14 +- include/linux/rwlock.h | 27 +- include/linux/rwlock_api_smp.h | 29 +- include/linux/rwlock_rt.h | 37 +- include/linux/rwlock_types.h | 10 +- include/linux/rwsem.h | 56 +- include/linux/sched/signal.h | 2 +- include/linux/seqlock.h | 24 + include/linux/seqlock_types.h | 5 +- include/linux/spinlock.h | 61 ++- include/linux/spinlock_api_smp.h | 14 +- include/linux/spinlock_api_up.h | 71 +-- include/linux/spinlock_rt.h | 27 +- include/linux/spinlock_types.h | 10 +- include/linux/spinlock_types_raw.h | 5 +- include/linux/srcu.h | 61 ++- kernel/Makefile | 2 + kernel/kcov.c | 40 +- kernel/time/posix-timers.c | 2 +- lib/Kconfig.debug | 43 ++ lib/Makefile | 6 + lib/rhashtable.c | 12 +- lib/stackdepot.c | 24 +- lib/test_capability-analysis.c | 481 ++++++++++++++++++ mm/kfence/Makefile | 2 + mm/kfence/core.c | 24 +- mm/kfence/kfence.h | 18 +- mm/kfence/kfence_test.c | 4 + mm/kfence/report.c | 8 +- net/ipv4/tcp_sigpool.c | 2 +- scripts/Makefile.capability-analysis | 5 + scripts/Makefile.lib | 10 + scripts/checkpatch.pl | 8 + tools/include/linux/compiler_types.h | 4 +- 56 files changed, 1682 insertions(+), 299 deletions(-) create mode 100644 Documentation/dev-tools/capability-analysis.rst create mode 100644 include/linux/compiler-capability-analysis.h create mode 100644 lib/test_capability-analysis.c create mode 100644 scripts/Makefile.capability-analysis
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRFC PATCH 1/24 compiler_types: Move lock checking attributes to compiler-capability-analysis.h
The conditional definition of lock checking macros and attributes is about to become more complex. Factor them out into their own header for better readability, and to make it obvious which features are supported by which mode (currently only Sparse). This is the first step towards generalizing towards "capability analysis".
No functional change intended.
--- include/linux/compiler-capability-analysis.h | 32 ++++++++++++++++++++ include/linux/compiler_types.h | 18 ++--------- 2 files changed, 34 insertions(+), 16 deletions(-) create mode 100644 include/linux/compiler-capability-analysis.h
include/linux/compiler-capability-analysis.h new file +32 -0
@@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Macros and attributes for compiler-based static capability analysis. +*/ + +#ifndef _LINUX_COMPILER_CAPABILITY_ANALYSIS_H +#define _LINUX_COMPILER_CAPABILITY_ANALYSIS_H + +#ifdef __CHECKER__ + +/* Sparse context/lock checking support. */ +# define __must_hold (x) __attribute__((context(x,1,1))) +# define __acquires (x) __attribute__((context(x,0,1))) +# define __cond_acquires (x) __attribute__((context(x,0,-1))) +# define __releases (x) __attribute__((context(x,1,0))) +# define __acquire (x) __context__(x,1) +# define __release (x) __context__(x,-1) +# define __cond_lock (x, c) ((c) ? ({ __acquire(x); 1; }) : 0) + +#else /* !__CHECKER__ */ + +# define __must_hold (x) +# define __acquires (x) +# define __cond_acquires (x) +# define __releases (x) +# define __acquire (x) (void)0 +# define __release (x) (void)0 +# define __cond_lock (x, c) (c) + +#endif /* __CHECKER__ */ + +#endif /* _LINUX_COMPILER_CAPABILITY_ANALYSIS_H */
include/linux/compiler_types.h +2 -16
@@ -24,6 +24,8 @@# define BTF_TYPE_TAG (value)/* nothing */ #endif +#include <linux/compiler-capability-analysis.h> +/* sparse defines __CHECKER__; see Documentation/dev-tools/sparse.rst */ #ifdef __CHECKER__/* address spaces */
@@ -34,14 +36,6 @@# define __rcu __attribute__((noderef, address_space(__rcu)))static inline void __chk_user_ptr (const volatile void __user *ptr) { }static inline void __chk_io_ptr (const volatile void __iomem *ptr) { } -/* context/locking */ -# define __must_hold (x) __attribute__((context(x,1,1))) -# define __acquires (x) __attribute__((context(x,0,1))) -# define __cond_acquires (x) __attribute__((context(x,0,-1))) -# define __releases (x) __attribute__((context(x,1,0))) -# define __acquire (x) __context__(x,1) -# define __release (x) __context__(x,-1) -# define __cond_lock (x,c) ((c) ? ({ __acquire(x); 1; }) : 0)/* other */ # define __force __attribute__((force))# define __nocast __attribute__((nocast))
@@ -62,14 +56,6 @@ static inline void __chk_io_ptr(const volatile void __iomem *ptr) { }# define __chk_user_ptr (x) (void)0# define __chk_io_ptr (x) (void)0 -/* context/locking */ -# define __must_hold (x) -# define __acquires (x) -# define __cond_acquires (x) -# define __releases (x) -# define __acquire (x) (void)0 -# define __release (x) (void)0 -# define __cond_lock (x,c) (c)/* other */ # define __force# define __nocast
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Re: [PATCH RFC 01/24] compiler_types: Move lock checking attributes to compiler-capability-analysis.h
include/linux/compiler-capability-analysis.h line 18
Quoted 8 lines · Marco Elver
include/linux/compiler-capability-analysis.h new file +8
-0+/* Sparse context/lock checking support. */ +# define __must_hold (x) __attribute__((context(x,1,1))) +# define __acquires (x) __attribute__((context(x,0,1))) +# define __cond_acquires (x) __attribute__((context(x,0,-1))) +# define __releases (x) __attribute__((context(x,1,0))) +# define __acquire (x) __context__(x,1) +# define __release (x) __context__(x,-1) +# define __cond_lock (x, c) ((c) ? ({ __acquire(x); 1; }) : 0)
If support for Clang thread-safety attributes is added, an important question is what to do with the sparse context attribute. I think that more developers are working on improving and maintaining Clang than sparse. How about reducing the workload of kernel maintainers by only supporting the Clang thread-safety approach and by dropping support for the sparse context attribute?
Thanks,
Bart.
Marco Elver
maintainerRe: [PATCH RFC 01/24] compiler_types: Move lock checking attributes to compiler-capability-analysis.h
Quoted 17 lines · Bart Van Assche
include/linux/compiler-capability-analysis.h line 18
Quoted 8 lines · Marco Elver
include/linux/compiler-capability-analysis.h new file +8
-0+/* Sparse context/lock checking support. */ +# define __must_hold (x) __attribute__((context(x,1,1))) +# define __acquires (x) __attribute__((context(x,0,1))) +# define __cond_acquires (x) __attribute__((context(x,0,-1))) +# define __releases (x) __attribute__((context(x,1,0))) +# define __acquire (x) __context__(x,1) +# define __release (x) __context__(x,-1) +# define __cond_lock (x, c) ((c) ? ({ __acquire(x); 1; }) : 0)If support for Clang thread-safety attributes is added, an important question is what to do with the sparse context attribute. I think that more developers are working on improving and maintaining Clang than sparse. How about reducing the workload of kernel maintainers by only supporting the Clang thread-safety approach and by dropping support for the sparse context attribute?
My 2c: I think Sparse's context tracking is a subset, and generally less complete, favoring false negatives over false positives (also does not support guarded_by).
So in theory they can co-exist.
In practice, I agree, there will be issues with maintaining both, because there will always be some odd corner-case which doesn't quite work with one or the other (specifically Sparse is happy to auto-infer acquired and released capabilities/contexts of functions and doesn't warn you if you still hold a lock when returning from a function).
I'd be in favor of deprecating Sparse's context tracking support, should there be consensus on that.
Thanks,
-- Marco
Peter Zijlstra
maintainerRe: [PATCH RFC 01/24] compiler_types: Move lock checking attributes to compiler-capability-analysis.h
Quoted 29 lines · Marco Elver
Quoted 17 lines · Bart Van Assche
include/linux/compiler-capability-analysis.h line 18
Quoted 8 lines · Marco Elver
include/linux/compiler-capability-analysis.h new file +8
-0+/* Sparse context/lock checking support. */ +# define __must_hold (x) __attribute__((context(x,1,1))) +# define __acquires (x) __attribute__((context(x,0,1))) +# define __cond_acquires (x) __attribute__((context(x,0,-1))) +# define __releases (x) __attribute__((context(x,1,0))) +# define __acquire (x) __context__(x,1) +# define __release (x) __context__(x,-1) +# define __cond_lock (x, c) ((c) ? ({ __acquire(x); 1; }) : 0)If support for Clang thread-safety attributes is added, an important question is what to do with the sparse context attribute. I think that more developers are working on improving and maintaining Clang than sparse. How about reducing the workload of kernel maintainers by only supporting the Clang thread-safety approach and by dropping support for the sparse context attribute?
My 2c: I think Sparse's context tracking is a subset, and generally less complete, favoring false negatives over false positives (also does not support
guarded_by).
So in theory they can co-exist.
In practice, I agree, there will be issues with maintaining both, because there will always be some odd corner-case which doesn't quite work with one or the other (specifically Sparse is happy to auto-infer acquired and released capabilities/contexts of functions and doesn't warn you if you still hold a lock when returning from a function).
Quoted 2 lines
I'd be in favor of deprecating Sparse's context tracking support, should there be consensus on that.
I don't think I've ever seen a useful sparse locking report, so yeah, no tears shed on removing it.
Marco Elver
maintainerRFC PATCH 2/24 compiler-capability-analysis: Rename __cond_lock() to __cond_acquire()
In reply to Marco ElverJust like the pairing of attribute __acquires() with a matching function-like macro __acquire(), the attribute __cond_acquires() should have a matching function-like macro __cond_acquire().
To be consistent, rename __cond_lock() to __cond_acquire().
--- drivers/net/wireless/intel/iwlwifi/iwl-trans.h | 2 +- drivers/net/wireless/intel/iwlwifi/pcie/internal.h | 2 +- include/linux/compiler-capability-analysis.h | 4 ++-- include/linux/mm.h | 6 +++--- include/linux/rwlock.h | 4 ++-- include/linux/rwlock_rt.h | 4 ++-- include/linux/sched/signal.h | 2 +- include/linux/spinlock.h | 12 ++++++------ include/linux/spinlock_rt.h | 6 +++--- kernel/time/posix-timers.c | 2 +- tools/include/linux/compiler_types.h | 4 ++-- 11 files changed, 24 insertions(+), 24 deletions(-)
drivers/net/wireless/intel/iwlwifi/iwl-trans.h +1 -1
@@ -1136,7 +1136,7 @@ void iwl_trans_set_bits_mask(struct iwl_trans *trans, u32 reg,bool _iwl_trans_grab_nic_access (struct iwl_trans *trans);#define iwl_trans_grab_nic_access (trans) \ - __cond_lock(nic_access, \ + __cond_acquire(nic_access, \ likely(_iwl_trans_grab_nic_access(trans)))void __releases (nic_access )
drivers/net/wireless/intel/iwlwifi/pcie/internal.h +1 -1
@@ -560,7 +560,7 @@ void iwl_trans_pcie_free_pnvm_dram_regions(struct iwl_dram_regions *dram_regionsbool __iwl_trans_pcie_grab_nic_access (struct iwl_trans *trans);#define _iwl_trans_pcie_grab_nic_access (trans) \ - __cond_lock(nic_access_nobh, \ + __cond_acquire(nic_access_nobh, \ likely(__iwl_trans_pcie_grab_nic_access(trans)))void iwl_trans_pcie_check_product_reset_status (struct pci_dev *pdev);
include/linux/compiler-capability-analysis.h +2 -2
@@ -15,7 +15,7 @@# define __releases (x) __attribute__((context(x,1,0)))# define __acquire (x) __context__(x,1)# define __release (x) __context__(x,-1) -# define __cond_lock (x, c) ((c) ? ({ __acquire(x); 1; }) : 0) +# define __cond_acquire (x, c) ((c) ? ({ __acquire(x); 1; }) : 0)#else /* !__CHECKER__ */
@@ -25,7 +25,7 @@# define __releases (x)# define __acquire (x) (void)0# define __release (x) (void)0 -# define __cond_lock (x, c) (c) +# define __cond_acquire (x, c) (c)#endif /* __CHECKER__ */
include/linux/mm.h +3 -3
@@ -2738,7 +2738,7 @@ static inline pte_t *get_locked_pte(struct mm_struct *mm, unsigned long addr, spinlock_t **ptl) {pte_t *ptep; -__cond_lock (*ptl, ptep =__get_locked_pte (mm, addr, ptl)); +__cond_acquire (*ptl, ptep =__get_locked_pte (mm, addr, ptl));return ptep; }
@@ -3029,7 +3029,7 @@ static inline pte_t *__pte_offset_map(pmd_t *pmd, unsigned long addr, {pte_t *pte; -__cond_lock (RCU , pte =___pte_offset_map (pmd, addr, pmdvalp)); +__cond_acquire (RCU , pte =___pte_offset_map (pmd, addr, pmdvalp));return pte; }static inline pte_t *pte_offset_map (pmd_t *pmd,unsigned long addr)
@@ -3044,7 +3044,7 @@ static inline pte_t *pte_offset_map_lock(struct mm_struct *mm, pmd_t *pmd, {pte_t *pte; -__cond_lock (RCU ,__cond_lock (*ptlp, +__cond_acquire (RCU ,__cond_acquire (*ptlp, pte =__pte_offset_map_lock (mm, pmd, addr, ptlp)));return pte; }
include/linux/rwlock.h +2 -2
@@ -49,8 +49,8 @@ do { \* regardless of whether CONFIG_SMP or CONFIG_PREEMPT are set. The various * methods are defined as nops in the case they are not required. */ -#define read_trylock (lock) __cond_lock(lock, _raw_read_trylock(lock)) -#define write_trylock (lock) __cond_lock(lock, _raw_write_trylock(lock)) +#define read_trylock (lock) __cond_acquire(lock, _raw_read_trylock(lock)) +#define write_trylock (lock) __cond_acquire(lock, _raw_write_trylock(lock))#define write_lock (lock) _raw_write_lock(lock)#define read_lock (lock) _raw_read_lock(lock)
include/linux/rwlock_rt.h +2 -2
@@ -55,7 +55,7 @@ static __always_inline void read_lock_irq(rwlock_t *rwlock) flags =0 ; \ } while (0) -#define read_trylock (lock) __cond_lock(lock, rt_read_trylock(lock)) +#define read_trylock (lock) __cond_acquire(lock, rt_read_trylock(lock)) static __always_inline void read_unlock(rwlock_t *rwlock) {
@@ -111,7 +111,7 @@ static __always_inline void write_lock_irq(rwlock_t *rwlock) flags =0 ; \ } while (0) -#define write_trylock (lock) __cond_lock(lock, rt_write_trylock(lock)) +#define write_trylock (lock) __cond_acquire(lock, rt_write_trylock(lock))#define write_trylock_irqsave (lock, flags) \ ({ \
include/linux/sched/signal.h +1 -1
@@ -741,7 +741,7 @@ static inline struct sighand_struct *lock_task_sighand(struct task_struct *task,struct sighand_struct *ret; ret =__lock_task_sighand (task, flags); - (void )__cond_lock (&task->sighand->siglock, ret); + (void )__cond_acquire (&task->sighand->siglock, ret);return ret; }
include/linux/spinlock.h +6 -6
@@ -212,7 +212,7 @@ static inline void do_raw_spin_unlock(raw_spinlock_t *lock) __releases(lock)* various methods are defined as nops in the case they are not * required. */ -#define raw_spin_trylock (lock) __cond_lock(lock, _raw_spin_trylock(lock)) +#define raw_spin_trylock (lock) __cond_acquire(lock, _raw_spin_trylock(lock))#define raw_spin_lock (lock) _raw_spin_lock(lock)
@@ -284,7 +284,7 @@ static inline void do_raw_spin_unlock(raw_spinlock_t *lock) __releases(lock)#define raw_spin_unlock_bh (lock) _raw_spin_unlock_bh(lock)#define raw_spin_trylock_bh (lock) \ - __cond_lock(lock, _raw_spin_trylock_bh(lock)) + __cond_acquire(lock, _raw_spin_trylock_bh(lock))#define raw_spin_trylock_irq (lock) \ ({ \
@@ -499,21 +499,21 @@ static inline int rwlock_needbreak(rwlock_t *lock)*/ extern int _atomic_dec_and_lock (atomic_t *atomic,spinlock_t *lock);#define atomic_dec_and_lock (atomic, lock) \ - __cond_lock(lock, _atomic_dec_and_lock(atomic, lock)) + __cond_acquire(lock, _atomic_dec_and_lock(atomic, lock))extern int _atomic_dec_and_lock_irqsave (atomic_t *atomic,spinlock_t *lock,unsigned long *flags);#define atomic_dec_and_lock_irqsave (atomic, lock, flags) \ - __cond_lock(lock, _atomic_dec_and_lock_irqsave(atomic, lock, &(flags))) + __cond_acquire(lock, _atomic_dec_and_lock_irqsave(atomic, lock, &(flags)))extern int _atomic_dec_and_raw_lock (atomic_t *atomic,raw_spinlock_t *lock);#define atomic_dec_and_raw_lock (atomic, lock) \ - __cond_lock(lock, _atomic_dec_and_raw_lock(atomic, lock)) + __cond_acquire(lock, _atomic_dec_and_raw_lock(atomic, lock))extern int _atomic_dec_and_raw_lock_irqsave (atomic_t *atomic,raw_spinlock_t *lock,unsigned long *flags);#define atomic_dec_and_raw_lock_irqsave (atomic, lock, flags) \ - __cond_lock(lock, _atomic_dec_and_raw_lock_irqsave(atomic, lock, &(flags))) + __cond_acquire(lock, _atomic_dec_and_raw_lock_irqsave(atomic, lock, &(flags))) int __alloc_bucket_spinlocks(spinlock_t **locks, unsigned int *lock_mask, size_t max_size, unsigned int cpu_mult,
include/linux/spinlock_rt.h +3 -3
@@ -123,13 +123,13 @@ static __always_inline void spin_unlock_irqrestore(spinlock_t *lock, }#define spin_trylock (lock) \ - __cond_lock(lock, rt_spin_trylock(lock)) + __cond_acquire(lock, rt_spin_trylock(lock))#define spin_trylock_bh (lock) \ - __cond_lock(lock, rt_spin_trylock_bh(lock)) + __cond_acquire(lock, rt_spin_trylock_bh(lock))#define spin_trylock_irq (lock) \ - __cond_lock(lock, rt_spin_trylock(lock)) + __cond_acquire(lock, rt_spin_trylock(lock))#define spin_trylock_irqsave (lock, flags) \ ({ \
kernel/time/posix-timers.c +1 -1
@@ -63,7 +63,7 @@ static struct k_itimer *__lock_timer(timer_t timer_id, unsigned long *flags);#define lock_timer (tid, flags) \ ({ struct k_itimer *__timr; \ - __cond_lock(&__timr->it_lock, __timr = __lock_timer(tid, flags)); \ + __cond_acquire(&__timr->it_lock, __timr = __lock_timer(tid, flags)); \ __timr; \ })
tools/include/linux/compiler_types.h +2 -2
@@ -20,7 +20,7 @@# define __releases (x) __attribute__((context(x,1,0)))# define __acquire (x) __context__(x,1)# define __release (x) __context__(x,-1) -# define __cond_lock (x,c) ((c) ? ({ __acquire(x); 1; }) : 0) +# define __cond_acquire (x,c) ((c) ? ({ __acquire(x); 1; }) : 0)#else /* __CHECKER__ */ /* context/locking */ # define __must_hold (x)
@@ -28,7 +28,7 @@# define __releases (x)# define __acquire (x) (void)0# define __release (x) (void)0 -# define __cond_lock (x,c) (c) +# define __cond_acquire (x,c) (c)#endif /* __CHECKER__ */ /* Compiler specific macros. */
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Peter Zijlstra
maintainerRe: [PATCH RFC 02/24] compiler-capability-analysis: Rename __cond_lock() to __cond_acquire()
Quoted 5 lines · Marco Elver
Just like the pairing of attribute
__acquires()with a matching function-like macro__acquire(), the attribute__cond_acquires()should have a matching function-like macro__cond_acquire().To be consistent, rename
__cond_lock()to__cond_acquire().
So I hate this __cond_lock() thing we have with a passion. I think it is one of the very worst annotations possible since it makes a trainwreck of the trylock code.
It is a major reason why mutex is not annotated with this nonsense.
Also, I think very dim of sparse in general -- I don't think I've ever managed to get a useful warning from between all the noise it generates.
Marco Elver
maintainerRe: [PATCH RFC 02/24] compiler-capability-analysis: Rename __cond_lock() to __cond_acquire()
Quoted 7 lines · Peter Zijlstra
Quoted 5 lines · Marco Elver
Just like the pairing of attribute
__acquires()with a matching function-like macro__acquire(), the attribute__cond_acquires()should have a matching function-like macro__cond_acquire().To be consistent, rename
__cond_lock()to__cond_acquire().
Quoted 8 lines
So I hate this
__cond_lock()thing we have with a passion. I think it is one of the very worst annotations possible since it makes a trainwreck of the trylock code.It is a major reason why mutex is not annotated with this nonsense.
Also, I think very dim of sparse in general -- I don't think I've ever managed to get a useful warning from between all the noise it generates.
Happy to reduce the use of __cond_lock(). :-)
Though one problem I found is it's still needed for those complex statement-expression *_trylock that spinlock.h/rwlock.h has, where we e.g. have (with my changes):
#define raw_spin_trylock_irqsave(lock, flags) \
__cond_acquire(lock, ({ \
local_irq_save(flags); \
_raw_spin_trylock(lock) ? \
1 : ({ local_irq_restore(flags); 0; }); \
}))Because there's an inner condition using _raw_spin_trylock() and the result of _raw_spin_trylock() is no longer directly used in a branch that also does the unlock, Clang becomes unhappy and complains. I.e. annotating _raw_spin_trylock with __cond_acquires(1, lock) doesn't work for this case because it's in a complex statement-expression. The only way to make it work was to wrap it into a function that has attribute __cond_acquires(1, lock) which is what I made __cond_lock/acquire do.
For some of the trivial uses, like e.g.
#define raw_spin_trylock(lock) __cond_acquire(lock, _raw_spin_trylock(lock))
it's easy enough to remove the outer __cond_lock/acquire if e.g. the _raw_spin_trylock has the attribute __cond_acquires. I kept these around for Sparse compatibility, but if we want to get rid of Sparse compatibility, some of those can be simplified.
Peter Zijlstra
maintainerRe: [PATCH RFC 02/24] compiler-capability-analysis: Rename __cond_lock() to __cond_acquire()
Quoted 29 lines · Marco Elver
Quoted 7 lines · Peter Zijlstra
Quoted 5 lines · Marco Elver
Just like the pairing of attribute
__acquires()with a matching function-like macro__acquire(), the attribute__cond_acquires()should have a matching function-like macro__cond_acquire().To be consistent, rename
__cond_lock()to__cond_acquire().Quoted 8 lines
So I hate this
__cond_lock()thing we have with a passion. I think it is one of the very worst annotations possible since it makes a trainwreck of the trylock code.It is a major reason why mutex is not annotated with this nonsense.
Also, I think very dim of sparse in general -- I don't think I've ever managed to get a useful warning from between all the noise it generates.
Happy to reduce the use of
__cond_lock(). :-)
Though one problem I found is it's still needed for those complex statement-expression*_trylockthatspinlock.h/rwlock.hhas, where we e.g. have (with my changes):#define raw_spin_trylock_irqsave(lock, flags) \ __cond_acquire(lock, ({ \ local_irq_save(flags); \ _raw_spin_trylock(lock) ? \ 1 : ({ local_irq_restore(flags); 0; }); \ }))
Quoted 7 lines
Because there's an inner condition using
_raw_spin_trylock()and the result of_raw_spin_trylock()is no longer directly used in a branch that also does the unlock, Clang becomes unhappy and complains. I.e. annotating_raw_spin_trylockwith__cond_acquires(1, lock)doesn't work for this case because it's in a complex statement-expression. The only way to make it work was to wrap it into a function that has attribute__cond_acquires(1, lock)which is what I made __cond_lock/acquire do.
Does something like:
static inline bool _raw_spin_trylock_irqsave(raw_spinlock_t *lock, unsigned long *flags)
__cond_acquire(1, lock)
{
local_irq_save(*flags);
if (_raw_spin_trylock(lock))
return true;
local_irq_restore(*flags);
return false;
}#define raw_spin_trylock_irqsave(lock, flags) \ _raw_spin_trylock_irqsave((lock), &(flags))
work?
Marco Elver
maintainerRe: [PATCH RFC 02/24] compiler-capability-analysis: Rename __cond_lock() to __cond_acquire()
Quoted 52 lines · Peter Zijlstra
Quoted 29 lines · Marco Elver
Quoted 7 lines · Peter Zijlstra
Quoted 5 lines · Marco Elver
Just like the pairing of attribute
__acquires()with a matching function-like macro__acquire(), the attribute__cond_acquires()should have a matching function-like macro__cond_acquire().To be consistent, rename
__cond_lock()to__cond_acquire().Quoted 8 lines
So I hate this
__cond_lock()thing we have with a passion. I think it is one of the very worst annotations possible since it makes a trainwreck of the trylock code.It is a major reason why mutex is not annotated with this nonsense.
Also, I think very dim of sparse in general -- I don't think I've ever managed to get a useful warning from between all the noise it generates.
Happy to reduce the use of
__cond_lock(). :-)
Though one problem I found is it's still needed for those complex statement-expression*_trylockthatspinlock.h/rwlock.hhas, where we e.g. have (with my changes):#define raw_spin_trylock_irqsave(lock, flags) \ __cond_acquire(lock, ({ \ local_irq_save(flags); \ _raw_spin_trylock(lock) ? \ 1 : ({ local_irq_restore(flags); 0; }); \ }))Quoted 7 lines
Because there's an inner condition using
_raw_spin_trylock()and the result of_raw_spin_trylock()is no longer directly used in a branch that also does the unlock, Clang becomes unhappy and complains. I.e. annotating_raw_spin_trylockwith__cond_acquires(1, lock)doesn't work for this case because it's in a complex statement-expression. The only way to make it work was to wrap it into a function that has attribute__cond_acquires(1, lock)which is what I made __cond_lock/acquire do.Does something like:
static inline bool
_raw_spin_trylock_irqsave(raw_spinlock_t *lock, unsigned long *flags)__cond_acquire(1, lock) { local_irq_save(*flags); if (_raw_spin_trylock(lock)) return true; local_irq_restore(*flags); return false; }
Quoted 4 lines
#define raw_spin_trylock_irqsave(lock, flags) \ _raw_spin_trylock_irqsave((lock), &(flags))work?
Yup it does (tested). Ok, so getting rid of __cond_lock should be doable. :-)
Marco Elver
maintainerRFC PATCH 3/24 compiler-capability-analysis: Add infrastructure for Clang's capability analysis
In reply to Marco ElverCapability analysis is a C language extension, which enables statically checking that user-definable "capabilities" are acquired and released where required. An obvious application is lock-safety checking for the kernel's various synchronization primitives (each of which represents a "capability"), and checking that locking rules are not violated.
Clang originally called the feature "Thread Safety Analysis" [1], with some terminology still using the thread-safety-analysis-only names. This was later changed and the feature became more flexible, gaining the ability to define custom "capabilities". Its foundations can be found in "capability systems", used to specify the permissibility of operations to depend on some capability being held (or not held).
[1] https://clang.llvm.org/docs/ThreadSafetyAnalysis.html
[2] https://www.cs.cornell.edu/talc/papers/capabilities.pdf
Because the feature is not just able to express capabilities related to synchronization primitives, the naming chosen for the kernel departs from Clang's initial "Thread Safety" nomenclature and refers to the feature as "Capability Analysis" to avoid confusion. The implementation still makes references to the older terminology in some places, such as -Wthread-safety being the warning enabled option that also still appears in diagnostic messages.
See more details in the kernel-doc documentation added in this and the subsequent changes.
[ RFC Note: A Clang version that supports -Wthread-safety-addressof is
recommended, but not required:
https://github.com/llvm/llvm-project/pull/123063
Should this patch series reach non-RFC stage, it is planned to be committed to Clang before. ]
--- Makefile | 1 + include/linux/compiler-capability-analysis.h | 385 ++++++++++++++++++- lib/Kconfig.debug | 29 ++ scripts/Makefile.capability-analysis | 5 + scripts/Makefile.lib | 10 + 5 files changed, 423 insertions(+), 7 deletions(-) create mode 100644 scripts/Makefile.capability-analysis
Makefile +1 -0
@@ -1082,6 +1082,7 @@ include-$(CONFIG_KCOV) += scripts/Makefile.kcov include-$(CONFIG_RANDSTRUCT) += scripts/Makefile.randstruct include-$(CONFIG_AUTOFDO_CLANG) += scripts/Makefile.autofdo include-$(CONFIG_PROPELLER_CLANG) += scripts/Makefile.propeller +include-$(CONFIG_WARN_CAPABILITY_ANALYSIS) += scripts/Makefile.capability-analysis include-$(CONFIG_GCC_PLUGINS) += scripts/Makefile.gcc-plugins include $(addprefix $(srctree)/, $(include-y))
include/linux/compiler-capability-analysis.h +378 -7
404 lines not on this page · raw message
lib/Kconfig.debug +29 -0
@@ -603,6 +603,35 @@ config DEBUG_FORCE_WEAK_PER_CPU To ensure that generic code follows the above rules, this option forces all percpu variables to be defined as weak. +config WARN_CAPABILITY_ANALYSIS + bool "Compiler capability-analysis warnings" + depends on CC_IS_CLANG && $(cc-option,-Wthread-safety -fexperimental-late-parse-attributes) + # Branch profiling re-defines "if", which messes with the compiler's + # ability to analyze __cond_acquire(..), resulting in false positives. + depends on !TRACE_BRANCH_PROFILING + default y + help + Capability analysis is a C language extension, which enables + statically checking that user-definable "capabilities" are acquired + and released where required. + + Clang's name of the feature ("Thread Safety Analysis") refers to + the original name of the feature; it was later expanded to be a + generic "Capability Analysis" framework. + + Produces warnings by default. Select CONFIG_WERROR if you wish to + turn these warnings into errors. + +config WARN_CAPABILITY_ANALYSIS_ALL + bool "Enable capability analysis for all source files" + depends on WARN_CAPABILITY_ANALYSIS + depends on EXPERT && !COMPILE_TEST + help + Enable tree-wide capability analysis. This is likely to produce a + large number of false positives - enable at your own risk. + + If unsure, say N. + endmenu # "Compiler options" menu "Generic Kernel Debugging Instruments"
scripts/Makefile.capability-analysis new file +5 -0
@@ -0,0 +1,5 @@ +# SPDX-License-Identifier: GPL-2.0 + +export CFLAGS_CAPABILITY_ANALYSIS := -DWARN_CAPABILITY_ANALYSIS \ + -fexperimental-late-parse-attributes -Wthread-safety \ + $(call cc-option,-Wthread-safety-addressof)
scripts/Makefile.lib +10 -0
@@ -191,6 +191,16 @@ _c_flags += $(if $(patsubst n%,, \ -D__KCSAN_INSTRUMENT_BARRIERS__) endif +# +# Enable capability analysis flags only where explicitly opted in. +# (depends on variables CAPABILITY_ANALYSIS_obj.o, CAPABILITY_ANALYSIS) +# +ifeq ($(CONFIG_WARN_CAPABILITY_ANALYSIS),y) +_c_flags += $(if $(patsubst n%,, \ + $(CAPABILITY_ANALYSIS_$(target-stem).o)$(CAPABILITY_ANALYSIS)$(if $(is-kernel-object),$(CONFIG_WARN_CAPABILITY_ANALYSIS_ALL))), \ + $(CFLAGS_CAPABILITY_ANALYSIS)) +endif + # # Enable AutoFDO build flags except some files or directories we don't want to # enable (depends on variables AUTOFDO_PROFILE_obj.o and AUTOFDO_PROFILE).
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRFC PATCH 4/24 compiler-capability-analysis: Add test stub
In reply to Marco ElverAdd a simple test stub where we will add common supported patterns that should not generate false positive of each new supported capability.
--- lib/Kconfig.debug | 14 ++++++++++++++ lib/Makefile | 3 +++ lib/test_capability-analysis.c | 18 ++++++++++++++++++ 3 files changed, 35 insertions(+) create mode 100644 lib/test_capability-analysis.c
lib/Kconfig.debug +14 -0
@@ -2764,6 +2764,20 @@ config LINEAR_RANGES_TEST If unsure, say N. +config CAPABILITY_ANALYSIS_TEST + bool "Compiler capability-analysis warnings test" + depends on EXPERT + help + This builds the test for compiler-based capability analysis. The test + does not add executable code to the kernel, but is meant to test that + common patterns supported by the analysis do not result in false + positive warnings. + + When adding support for new capabilities, it is strongly recommended + to add supported patterns to this test. + + If unsure, say N. + config CMDLINE_KUNIT_TEST tristate "KUnit test for cmdline API" if !KUNIT_ALL_TESTS depends on KUNIT
lib/Makefile +3 -0
@@ -394,6 +394,9 @@ obj-$(CONFIG_CRC_KUNIT_TEST) += crc_kunit.o obj-$(CONFIG_SIPHASH_KUNIT_TEST) += siphash_kunit.o obj-$(CONFIG_USERCOPY_KUNIT_TEST) += usercopy_kunit.o +CAPABILITY_ANALYSIS_test_capability-analysis.o := y +obj-$(CONFIG_CAPABILITY_ANALYSIS_TEST) += test_capability-analysis.o + obj-$(CONFIG_GENERIC_LIB_DEVMEM_IS_ALLOWED) += devmem_is_allowed.o obj-$(CONFIG_FIRMWARE_TABLE) += fw_table.o
lib/test_capability-analysis.c new file +18 -0
@@ -0,0 +1,18 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Compile-only tests for common patterns that should not generate false +* positive errors when compiled with Clang's capability analysis. +*/ + +#include <linux/build_bug.h> + +/* +* Test that helper macros work as expected. +*/ +static void __used test_common_helpers (void ) +{ +BUILD_BUG_ON (capability_unsafe (3 ) !=3 );/* plain expression */ + BUILD_BUG_ON(capability_unsafe((void)2; 3;) != 3); /* does not swallow semi-colon */ +BUILD_BUG_ON (capability_unsafe ((void )2 ,3 ) !=3 );/* does not swallow commas */ + capability_unsafe(do { } while (0)); /* works with void statements */ +}
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRFC PATCH 5/24 Documentation: Add documentation for Compiler-Based Capability Analysis
In reply to Marco ElverAdds documentation in Documentation/dev-tools/capability-analysis.rst, and adds it to the index and cross-references from Sparse's document.
--- .../dev-tools/capability-analysis.rst | 147 ++++++++++++++++++ Documentation/dev-tools/index.rst | 1 + Documentation/dev-tools/sparse.rst | 4 + 3 files changed, 152 insertions(+) create mode 100644 Documentation/dev-tools/capability-analysis.rst
Documentation/dev-tools/capability-analysis.rst new file +147 -0
@@ -0,0 +1,147 @@ +.. SPDX-License-Identifier: GPL-2.0 +.. Copyright (C) 2025, Google LLC. + +.. _capability-analysis: + +Compiler-Based Capability Analysis +================================== + +Capability analysis is a C language extension, which enables statically +checking that user-definable "capabilities" are acquired and released where +required. An obvious application is lock-safety checking for the kernel's +various synchronization primitives (each of which represents a "capability"), +and checking that locking rules are not violated. + +The Clang compiler currently supports the full set of capability analysis +features. To enable for Clang, configure the kernel with:: + + CONFIG_WARN_CAPABILITY_ANALYSIS=y + +The analysis is *opt-in by default*, and requires declaring which modules and +subsystems should be analyzed in the respective `Makefile`:: + + CAPABILITY_ANALYSIS_mymodule.o := y + +Or for all translation units in the directory:: + + CAPABILITY_ANALYSIS := y + +It is possible to enable the analysis tree-wide, however, which will result in +numerous false positive warnings currently and is *not* generally recommended:: + + CONFIG_WARN_CAPABILITY_ANALYSIS_ALL=y + +Independent of the above Clang support, a subset of the analysis is supported +by :ref:`Sparse <sparse>`, with weaker guarantees (fewer false positives with +tree-wide analysis, more more false negatives). Compared to Sparse, Clang's +analysis is more complete. + +Programming Model +----------------- + +The below describes the programming model around using capability-enabled +types. + +.. note:: + Enabling capability analysis can be seen as enabling a dialect of Linux C with + a Capability System. Some valid patterns involving complex control-flow are + constrained (such as conditional acquisition and later conditional release + in the same function, or returning pointers to capabilities from functions. + +Capability analysis is a way to specify permissibility of operations to depend +on capabilities being held (or not held). Typically we are interested in +protecting data and code by requiring some capability to be held, for example a +specific lock. The analysis ensures that the caller cannot perform the +operation without holding the appropriate capability. + +Capabilities are associated with named structs, along with functions that +operate on capability-enabled struct instances to acquire and release the +associated capability. + +Capabilities can be held either exclusively or shared. This mechanism allows +assign more precise privileges when holding a capability, typically to +distinguish where a thread may only read (shared) or also write (exclusive) to +guarded data. + +The set of capabilities that are actually held by a given thread at a given +point in program execution is a run-time concept. The static analysis works by +calculating an approximation of that set, called the capability environment. +The capability environment is calculated for every program point, and describes +the set of capabilities that are statically known to be held, or not held, at +that particular point. This environment is a conservative approximation of the +full set of capabilities that will actually held by a thread at run-time. + +More details are also documented `here +<https://clang.llvm.org/docs/ThreadSafetyAnalysis.html>`_. + +.. note:: + Unlike Sparse's context tracking analysis, Clang's analysis explicitly does + not infer capabilities acquired or released by inline functions. It requires + explicit annotations to (a) assert that it's not a bug if a capability is + released or acquired, and (b) to retain consistency between inline and + non-inline function declarations. + +Supported Kernel Primitives +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. Currently the following synchronization primitives are supported: + +For capabilities with an initialization function (e.g., `spin_lock_init()`), +calling this function on the capability instance before initializing any +guarded members or globals prevents the compiler from issuing warnings about +unguarded initialization. + +Lockdep assertions, such as `lockdep_assert_held()`, inform the compiler's +capability analysis that the associated synchronization primitive is held after +the assertion. This avoids false positives in complex control-flow scenarios +and encourages the use of Lockdep where static analysis is limited. For +example, this is useful when a function doesn't *always* require a lock, making +`__must_hold()` inappropriate. + +Keywords +~~~~~~~~ + +.. kernel-doc:: include/linux/compiler-capability-analysis.h + :identifiers: struct_with_capability + token_capability token_capability_instance + __var_guarded_by __ref_guarded_by + __must_hold + __must_not_hold + __acquires + __cond_acquires + __releases + __must_hold_shared + __acquires_shared + __cond_acquires_shared + __releases_shared + __acquire + __release + __cond_acquire + __acquire_shared + __release_shared + __cond_acquire_shared + capability_unsafe + __no_capability_analysis + disable_capability_analysis enable_capability_analysis + +Background +---------- + +Clang originally called the feature `Thread Safety Analysis +<https://clang.llvm.org/docs/ThreadSafetyAnalysis.html>`_, with some +terminology still using the thread-safety-analysis-only names. This was later +changed and the feature become more flexible, gaining the ability to define +custom "capabilities". + +Indeed, its foundations can be found in `capability systems +<https://www.cs.cornell.edu/talc/papers/capabilities.pdf>`_, used to specify +the permissibility of operations to depend on some capability being held (or +not held). + +Because the feature is not just able to express capabilities related to +synchronization primitives, the naming chosen for the kernel departs from +Clang's initial "Thread Safety" nomenclature and refers to the feature as +"Capability Analysis" to avoid confusion. The implementation still makes +references to the older terminology in some places, such as `-Wthread-safety` +being the warning enabled option that also still appears in diagnostic +messages.
Documentation/dev-tools/index.rst +1 -0
@@ -18,6 +18,7 @@ Documentation/process/debugging/index.rst :maxdepth: 2 testing-overview + capability-analysis checkpatch clang-format coccinelle
Documentation/dev-tools/sparse.rst +4 -0
@@ -2,6 +2,8 @@ .. Copyright 2004 Pavel Machek <[email protected]> .. Copyright 2006 Bob Copeland <[email protected]> +.. _sparse: + Sparse ======
@@ -72,6 +74,8 @@ releasing the lock inside the function in a balanced way, no annotation is needed. The three annotations above are for cases where sparse would otherwise report a context imbalance. +Also see :ref:`Compiler-Based Capability Analysis <capability-analysis>`. + Getting sparse --------------
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRFC PATCH 6/24 checkpatch: Warn about capability_unsafe() without comment
In reply to Marco ElverWarn about applications of capability_unsafe() without a comment, to encourage documenting the reasoning behind why it was deemed safe.
--- scripts/checkpatch.pl | 8 ++++++++ 1 file changed, 8 insertions(+)
scripts/checkpatch.pl +8 -0
@@ -6693,6 +6693,14 @@ sub process { } } +# check for capability_unsafe without a comment. + if ($line =~ /\bcapability_unsafe\b/) { + if (!ctx_has_comment($first_line, $linenr)) { + WARN("CAPABILITY_UNSAFE", + "capability_unsafe without comment\n" . $herecurr); + } + } + # check of hardware specific defines if ($line =~ m@^.\s*\#\s*if.*\b(__i386__|__powerpc64__|__sun__|__s390x__)\b@ && $realfile !~ m@include/asm-@) { CHK("ARCH_DEFINES",
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRFC PATCH 7/24 cleanup: Basic compatibility with capability analysis
In reply to Marco ElverDue to the scoped cleanup helpers used for lock guards wrapping acquire/release around their own constructors/destructors that store pointers to the passed locks in a separate struct, we currently cannot accurately annotate *destructors* which lock was released. While it's possible to annotate the constructor to say which lock was acquired, that alone would result in false positives claiming the lock was not released on function return.
Instead, to avoid false positives, we can claim that the constructor "asserts" that the taken lock is held. This will ensure we can still benefit from the analysis where scoped guards are used to protect access to guarded variables, while avoiding false positives. The only downside are false negatives where we might accidentally lock the same lock again:
raw_spin_lock(&my_lock); ... guard(raw_spinlock)(&my_lock); // no warning
Arguably, lockdep will immediately catch issues like this.
While Clang's analysis supports scoped guards in C++ [1], there's no way to apply this to C right now. Better support for Linux's scoped guard design could be added in future if deemed critical.
[1] https://clang.llvm.org/docs/ThreadSafetyAnalysis.html#scoped-capability
--- include/linux/cleanup.h | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-)
include/linux/cleanup.h +10 -4
@@ -223,7 +223,7 @@ const volatile void * __must_check_fn(const volatile void *val)* @exit is an expression using '_T' -- similar to FREE above. * @init is an expression in @init_args resulting in @type * -* EXTEND_CLASS(name, ext, init, init_args...): +* EXTEND_CLASS(name, ext, ctor_attrs, init, init_args...): * extends class @name to @name@ext with the new constructor * * CLASS(name, var)(args...):
@@ -243,15 +243,18 @@ const volatile void * __must_check_fn(const volatile void *val) #define DEFINE_CLASS(_name, _type, _exit, _init, _init_args...) \ typedef _type class_##_name##_t; \ static inline void class_##_name##_destructor(_type *p) \ + __no_capability_analysis \ { _type _T = *p; _exit; } \ static inline _type class_##_name##_constructor(_init_args) \ + __no_capability_analysis \ { _type t = _init; return t; } -#define EXTEND_CLASS(_name, ext, _init, _init_args...) \ +#define EXTEND_CLASS(_name, ext, ctor_attrs, _init, _init_args...) \ typedef class_##_name##_t class_##_name##ext##_t; \ static inline void class_##_name##ext##_destructor(class_##_name##_t *p)\ { class_##_name##_destructor(p); } \ static inline class_##_name##_t class_##_name##ext##_constructor(_init_args) \ + __no_capability_analysis ctor_attrs \ { class_##_name##_t t = _init; return t; }#define CLASS (_name, var) \
@@ -299,7 +302,7 @@ static __maybe_unused const bool class_##_name##_is_conditional = _is_cond#define DEFINE_GUARD_COND (_name, _ext, _condlock) \ __DEFINE_CLASS_IS_CONDITIONAL(_name##_ext, true); \ - EXTEND_CLASS(_name, _ext, \ + EXTEND_CLASS(_name, _ext,, \ ({ void *_t = _T; if (_T && !(_condlock)) _t = NULL; _t; }), \ class_##_name##_t _T) \ static inline void * class_##_name##_ext##_lock_ptr(class_##_name##_t *_T) \
@@ -371,6 +374,7 @@ typedef struct { \ } class_##_name##_t; \ \ static inline void class_##_name##_destructor(class_##_name##_t *_T) \ + __no_capability_analysis \ { \if (_T->lock) { _unlock; } \ } \
@@ -383,6 +387,7 @@ static inline void *class_##_name##_lock_ptr(class_##_name##_t *_T) \#define __DEFINE_LOCK_GUARD_1 (_name, _type, _lock) \ static inline class_##_name##_t class_##_name##_constructor(_type *l) \ + __no_capability_analysis __asserts_cap(l) \ { \ class_##_name##_t _t = { .lock = l }, *_T = &_t; \ _lock; \
@@ -391,6 +396,7 @@ static inline class_##_name##_t class_##_name##_constructor(_type *l) \#define __DEFINE_LOCK_GUARD_0 (_name, _lock) \ static inline class_##_name##_t class_##_name##_constructor(void) \ + __no_capability_analysis \ { \ class_##_name##_t _t = { .lock = (void*)1 }, \ *_T __maybe_unused = &_t; \
@@ -410,7 +416,7 @@ __DEFINE_LOCK_GUARD_0(_name, _lock)#define DEFINE_LOCK_GUARD_1_COND (_name, _ext, _condlock) \ __DEFINE_CLASS_IS_CONDITIONAL(_name##_ext, true); \ - EXTEND_CLASS(_name, _ext, \ + EXTEND_CLASS(_name, _ext, __asserts_cap(l), \ ({ class_##_name##_t _t = { .lock = l }, *_T = &_t;\ if (_T->lock && !(_condlock)) _T->lock = NULL; \ _t; }), \
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Re: [PATCH RFC 07/24] cleanup: Basic compatibility with capability analysis
include/linux/cleanup.h const volatile void * __must_check_fn(const volatile void *val) · line 250
Quoted 9 lines · Marco Elver
include/linux/cleanup.h +2
-0@@ -243,15 +243,18 @@ const volatile void * __must_check_fn(const volatile void *val) #define DEFINE_CLASS(_name, _type, _exit, _init, _init_args...) \ typedef _type class_##_name##_t; \ static inline void class_##_name##_destructor(_type *p) \ + __no_capability_analysis \ { _type _T = *p; _exit; } \ static inline _type class_##_name##_constructor(_init_args) \ + __no_capability_analysis \ { _type t = _init; return t; }
guard() uses the constructor and destructor functions defined by DEFINE_GUARD(). The DEFINE_GUARD() implementation uses DEFINE_CLASS(). Here is an example that I found in <linux/mutex.h>:
DEFINE_GUARD(mutex, struct mutex *, mutex_lock(_T), mutex_unlock(_T))
For this example, how is the compiler told that mutex _T is held around the code protected by guard()?
Thanks,
Bart.
Marco Elver
maintainerRe: [PATCH RFC 07/24] cleanup: Basic compatibility with capability analysis
Quoted 16 lines · Bart Van Assche
include/linux/cleanup.h const volatile void * __must_check_fn(const volatile void *val) · line 250
Quoted 9 lines · Marco Elver
include/linux/cleanup.h +2
-0@@ -243,15 +243,18 @@ const volatile void * __must_check_fn(const volatile void *val) #define DEFINE_CLASS(_name, _type, _exit, _init, _init_args...) \ typedef _type class_##_name##_t; \ static inline void class_##_name##_destructor(_type *p) \ + __no_capability_analysis \ { _type _T = *p; _exit; } \ static inline _type class_##_name##_constructor(_init_args) \ + __no_capability_analysis \ { _type t = _init; return t; }
guard()uses the constructor and destructor functions defined byDEFINE_GUARD(). TheDEFINE_GUARD()implementation usesDEFINE_CLASS(). Here is an example that I found in <linux/mutex.h>:
Quoted 4 lines
DEFINE_GUARD(mutex, struct mutex *, mutex_lock(_T), mutex_unlock(_T))For this example, how is the compiler told that mutex _T is held around the code protected by
guard()?
DEFINE_GUARD is the generic variant usable for more than just locking primitives. DEFINE_LOCK_GUARD_X is a specialization of DEFINE_GUARD intended for locking primitives, all of which should be capability-enabled.
So I added automatic support for DEFINE_LOCK_GUARD_1 (keeping in mind the limitations as described in the commit message). All later patches that introduce support for a locking primitive that had been using DEFINE_GUARD are switched over to DEFINE_LOCK_GUARD. There's no additional runtime cost (_T is just a struct containing _T->lock). For example, the change for mutex [1] switches it to use DEFINE_LOCK_GUARD_1.
[1] https://lore.kernel.org/all/[email protected]/
(For every primitive added I have added tests in test_capability-analysis.c, including testing that the scoped guard() helpers work and do not produce false positives.)
The RCU patch [15/24] also makes it work for LOCK_GUARD_0, by simply adding an optional helper macro to declare the attributes for lock and unlock. There's no need for additional variants of DEFINE_LOCK_GUARD_X.
Should the need arise to add add annotations for DEFINE_GUARD, we can introduce DECLARE_GUARD_ATTRS(), similar to DECLARE_LOCK_GUARD_0_ATTRS() introduced in [15/24]. But it's omitted because DEFINE_GUARD() can be replaced by DEFINE_LOCK_GUARD for locking primitives.
In general I wanted to keep the current interface for defining guards untouched, and keeping it simpler.
Marco Elver
maintainerRFC PATCH 8/24 lockdep: Annotate lockdep assertions for capability analysis
In reply to Marco ElverClang's capability analysis can be made aware of functions that assert that capabilities/locks are held.
Presence of these annotations causes the analysis to assume the capability is held after calls to the annotated function, and avoid false positives with complex control-flow; for example, where not all control-flow paths in a function require a held lock, and therefore marking the function with __must_hold(..) is inappropriate.
--- include/linux/lockdep.h | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
include/linux/lockdep.h +6 -6
@@ -282,16 +282,16 @@ extern void lock_unpin_lock(struct lockdep_map *lock, struct pin_cookie); do { WARN_ON_ONCE(debug_locks && !(cond)); } while (0)#define lockdep_assert_held (l) \ - lockdep_assert(lockdep_is_held(l) != LOCK_STATE_NOT_HELD) + do { lockdep_assert(lockdep_is_held(l) != LOCK_STATE_NOT_HELD); __assert_cap(l); } while (0)#define lockdep_assert_not_held (l) \ lockdep_assert(lockdep_is_held(l) != LOCK_STATE_HELD)#define lockdep_assert_held_write (l) \ - lockdep_assert(lockdep_is_held_type(l, 0)) + do { lockdep_assert(lockdep_is_held_type(l, 0)); __assert_cap(l); } while (0)#define lockdep_assert_held_read (l) \ - lockdep_assert(lockdep_is_held_type(l, 1)) + do { lockdep_assert(lockdep_is_held_type(l, 1)); __assert_shared_cap(l); } while (0)#define lockdep_assert_held_once (l) \ lockdep_assert_once(lockdep_is_held(l) != LOCK_STATE_NOT_HELD)
@@ -389,10 +389,10 @@ extern int lockdep_is_held(const void *);#define lockdep_assert (c) do { } while (0)#define lockdep_assert_once (c) do { } while (0) -#define lockdep_assert_held (l) do { (void)(l); } while (0) +#define lockdep_assert_held (l) __assert_cap(l)#define lockdep_assert_not_held (l) do { (void)(l); } while (0) -#define lockdep_assert_held_write (l) do { (void)(l); } while (0) -#define lockdep_assert_held_read (l) do { (void)(l); } while (0) +#define lockdep_assert_held_write (l) __assert_cap(l) +#define lockdep_assert_held_read (l) __assert_shared_cap(l)#define lockdep_assert_held_once (l) do { (void)(l); } while (0)#define lockdep_assert_none_held_once () do { } while (0)
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Re: [PATCH RFC 08/24] lockdep: Annotate lockdep assertions for capability analysis
Quoted 11 lines · Marco Elver
include/linux/lockdep.h +1
-1@@ -282,16 +282,16 @@ extern void lock_unpin_lock(struct lockdep_map *lock, struct pin_cookie); do { WARN_ON_ONCE(debug_locks && !(cond)); } while (0)#define lockdep_assert_held (l) \ - lockdep_assert(lockdep_is_held(l) != LOCK_STATE_NOT_HELD) + do { lockdep_assert(lockdep_is_held(l) != LOCK_STATE_NOT_HELD); __assert_cap(l); } while (0)
include/linux/lockdep.h extern void lock_unpin_lock(struct lockdep_map *lock, struct pin_cookie); · line 294
Quoted 10 lines
include/linux/lockdep.h +2
-2#define lockdep_assert_not_held (l) \ lockdep_assert(lockdep_is_held(l) != LOCK_STATE_HELD)#define lockdep_assert_held_write (l) \ - lockdep_assert(lockdep_is_held_type(l, 0)) + do { lockdep_assert(lockdep_is_held_type(l, 0)); __assert_cap(l); } while (0)#define lockdep_assert_held_read (l) \ - lockdep_assert(lockdep_is_held_type(l, 1)) + do { lockdep_assert(lockdep_is_held_type(l, 1)); __assert_shared_cap(l); } while (0)
These changes look wrong to me. The current behavior of lockdep_assert_held(lock) is that it issues a kernel warning at runtime if lock is not held when a lockdep_assert_held() statement is executed. __assert_cap(lock) tells the compiler to *ignore* the absence of __must_hold(lock). I think this is wrong. The compiler should complain if a __must_hold(lock) annotation is missing. While sparse does not support interprocedural analysis for lock contexts, the Clang thread-safety checker supports this. If function declarations are annotated with __must_hold(lock), Clang will complain if the caller does not hold lock.
In other words, the above changes disable a useful compile-time check. I think that useful compile-time checks should not be disabled.
Bart.
Marco Elver
maintainerRe: [PATCH RFC 08/24] lockdep: Annotate lockdep assertions for capability analysis
Quoted 35 lines · Bart Van Assche
Quoted 11 lines · Marco Elver
include/linux/lockdep.h +1
-1@@ -282,16 +282,16 @@ extern void lock_unpin_lock(struct lockdep_map *lock, struct pin_cookie); do { WARN_ON_ONCE(debug_locks && !(cond)); } while (0)#define lockdep_assert_held (l) \ - lockdep_assert(lockdep_is_held(l) != LOCK_STATE_NOT_HELD) + do { lockdep_assert(lockdep_is_held(l) != LOCK_STATE_NOT_HELD); __assert_cap(l); } while (0)include/linux/lockdep.h extern void lock_unpin_lock(struct lockdep_map *lock, struct pin_cookie); · line 294
Quoted 10 lines
include/linux/lockdep.h +2
-2#define lockdep_assert_not_held (l) \ lockdep_assert(lockdep_is_held(l) != LOCK_STATE_HELD)#define lockdep_assert_held_write (l) \ - lockdep_assert(lockdep_is_held_type(l, 0)) + do { lockdep_assert(lockdep_is_held_type(l, 0)); __assert_cap(l); } while (0)#define lockdep_assert_held_read (l) \ - lockdep_assert(lockdep_is_held_type(l, 1)) + do { lockdep_assert(lockdep_is_held_type(l, 1)); __assert_shared_cap(l); } while (0)These changes look wrong to me. The current behavior of
lockdep_assert_held(lock)is that it issues a kernel warning at runtime iflockis not held when alockdep_assert_held()statement is executed.__assert_cap(lock)tells the compiler to *ignore* the absence of__must_hold(lock). I think this is wrong. The compiler should complain if a__must_hold(lock)annotation is missing. While sparse does not support interprocedural analysis for lock contexts, the Clang thread-safety checker supports this. If function declarations are annotated with__must_hold(lock), Clang will complain if the caller does not holdlock.
Quoted 2 lines
In other words, the above changes disable a useful compile-time check. I think that useful compile-time checks should not be disabled.
The assert_capability attribute was designed precisely for assertions that check at runtime that the lock is held, and delegate to runtime verification where the static analysis is just not powerful enough. In the commit description:
Presence of these annotations causes the analysis to assume the capability is held after calls to the annotated function, and avoid false positives with complex control-flow; for example, where not all control-flow paths in a function require a held lock, and therefore marking the function with __must_hold(..) is inappropriate.
If you try to write code where you access a guarded_by variable, but the lock is held not in all paths we can write it like this:
struct bar {
spinlock_t lock;
bool a; // true if lock held
int counter __var_guarded_by(&lock);
};
void foo(struct bar *d)
{
...
if (d->a) {
lockdep_assert_held(&d->lock);
d->counter++;
} else {
// lock not held! }
...
}
Without lockdep_assert_held() you get false positives, and there's no other good way to express this if you do not want to always call foo() with the lock held.
It essentially forces addition of lockdep checks where the static analysis can't quite prove what you've done is right. This is desirable over adding no-analysis attributes and not checking anything at all.
Re: [PATCH RFC 08/24] lockdep: Annotate lockdep assertions for capability analysis
Quoted 20 lines · Marco Elver
If you try to write code where you access a
guarded_byvariable, but the lock is held not in all paths we can write it like this:struct bar { spinlock_t lock; bool a; // true if lock held int counter __var_guarded_by(&lock); }; void foo(struct bar *d) { ... if (d->a) { lockdep_assert_held(&d->lock); d->counter++; } else { // lock not held! }...
}
Quoted 8 lines
Without
lockdep_assert_held()you get false positives, and there's no other good way to express this if you do not want to always callfoo()with the lock held.It essentially forces addition of lockdep checks where the static analysis can't quite prove what you've done is right. This is desirable over adding no-analysis attributes and not checking anything at all.
In the above I see that two different options have been mentioned for code that includes conditional lockdep_assert_held() calls:
- Either include __assert_cap() in the lockdep_assert_held() definition.
- Or annotate the entire function with __no_thread_safety_analysis.
I think there is a third possibility: add an explicit __assert_cap() call under the lockdep_assert_held() call. With this approach the thread-safety analysis remains enabled for the annotated function and
the compiler will complain if neither __must_hold() nor __assert_cap()
has been used.
I prefer the third option since conditional lockdep_assert_held() calls are relatively rare in the kernel. If I counted correctly, there are about 40 times more unconditional lockdep_assert_held() calls than conditional lockdep_assert_held() calls.
Bart.
Marco Elver
maintainerRe: [PATCH RFC 08/24] lockdep: Annotate lockdep assertions for capability analysis
Quoted 37 lines · Bart Van Assche
Quoted 20 lines · Marco Elver
If you try to write code where you access a
guarded_byvariable, but the lock is held not in all paths we can write it like this:struct bar { spinlock_t lock; bool a; // true if lock held int counter __var_guarded_by(&lock); }; void foo(struct bar *d) { ... if (d->a) { lockdep_assert_held(&d->lock); d->counter++; } else { // lock not held! }...
}Quoted 8 lines
Without
lockdep_assert_held()you get false positives, and there's no other good way to express this if you do not want to always callfoo()with the lock held.It essentially forces addition of lockdep checks where the static analysis can't quite prove what you've done is right. This is desirable over adding no-analysis attributes and not checking anything at all.
In the above I see that two different options have been mentioned for code that includes conditional
lockdep_assert_held()calls:
- Either include__assert_cap()in thelockdep_assert_held()definition.
- Or annotate the entire function with__no_thread_safety_analysis.
Quoted 5 lines
I think there is a third possibility: add an explicit
__assert_cap()call under thelockdep_assert_held()call. With this approach the thread-safety analysis remains enabled for the annotated function and the compiler will complain if neither__must_hold()nor__assert_cap()has been used.
That's just adding more clutter. Being able to leverage existing lockdep_assert to avoid false positives (at potential cost of few false negatives) is a decent trade-off. Sure, having maximum checking guarantees would be nice, but there's a balance we have to strike vs. ergonomics, usability, and pointless clutter.
Can we initially try to avoid clutter as much as possible? Then, if you feel coverage is not good enough, make the analysis stricter by e.g. removing the implicit assert from lockdep_assert in later patches and see how it goes.
I'm basing my judgement here on experience having worked on other analysis in the kernel, and the biggest request from maintainers has always been to "avoid useless clutter and false positives at all cost", often at the cost of increased potential for false negatives but avoiding false positives and reducing annotations (I can dig out discussions we had for KMSAN if you do not believe me...).
Marco Elver
maintainerRFC PATCH 9/24 locking/rwlock, spinlock: Support Clang's capability analysis
In reply to Marco ElverAdd support for Clang's capability analysis for raw_spinlock_t, spinlock_t, and rwlock. This wholesale conversion is required because all three of them are interdependent.
To avoid warnings in constructors, the initialization functions mark a capability as acquired when initialized before guarded variables.
The test verifies that common patterns do not generate false positives.
--- .../dev-tools/capability-analysis.rst | 3 +- include/linux/rwlock.h | 25 ++-- include/linux/rwlock_api_smp.h | 29 +++- include/linux/rwlock_rt.h | 35 +++-- include/linux/rwlock_types.h | 10 +- include/linux/spinlock.h | 45 +++--- include/linux/spinlock_api_smp.h | 14 +- include/linux/spinlock_api_up.h | 71 +++++----- include/linux/spinlock_rt.h | 21 +-- include/linux/spinlock_types.h | 10 +- include/linux/spinlock_types_raw.h | 5 +- lib/test_capability-analysis.c | 128 ++++++++++++++++++ 12 files changed, 299 insertions(+), 97 deletions(-)
Documentation/dev-tools/capability-analysis.rst +2 -1
@@ -84,7 +84,8 @@ More details are also documented `here Supported Kernel Primitives ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. Currently the following synchronization primitives are supported: +Currently the following synchronization primitives are supported: +`raw_spinlock_t`, `spinlock_t`, `rwlock_t`. For capabilities with an initialization function (e.g., `spin_lock_init()`), calling this function on the capability instance before initializing any
include/linux/rwlock.h +13 -12
@@ -22,23 +22,24 @@ do { \static struct lock_class_key __key; \ \ __rwlock_init((lock), #lock, &__key); \ + __assert_cap(lock); \ } while (0)#else # define rwlock_init (lock) \ - do { *(lock) = __RW_LOCK_UNLOCKED(lock); } while (0) + do { *(lock) = __RW_LOCK_UNLOCKED(lock); __assert_cap(lock); } while (0)#endif #ifdef CONFIG_DEBUG_SPINLOCK - extern void do_raw_read_lock(rwlock_t *lock) __acquires(lock); + extern void do_raw_read_lock(rwlock_t *lock) __acquires_shared(lock);extern int do_raw_read_trylock (rwlock_t *lock); - extern void do_raw_read_unlock(rwlock_t *lock) __releases(lock); + extern void do_raw_read_unlock(rwlock_t *lock) __releases_shared(lock); extern void do_raw_write_lock(rwlock_t *lock) __acquires(lock);extern int do_raw_write_trylock (rwlock_t *lock); extern void do_raw_write_unlock(rwlock_t *lock) __releases(lock);#else -# define do_raw_read_lock (rwlock) do {__acquire(lock); arch_read_lock(&(rwlock)->raw_lock); } while (0) +# define do_raw_read_lock (rwlock) do {__acquire_shared(lock); arch_read_lock(&(rwlock)->raw_lock); } while (0)# define do_raw_read_trylock (rwlock) arch_read_trylock(&(rwlock)->raw_lock) -# define do_raw_read_unlock (rwlock) do {arch_read_unlock(&(rwlock)->raw_lock); __release(lock); } while (0) +# define do_raw_read_unlock (rwlock) do {arch_read_unlock(&(rwlock)->raw_lock); __release_shared(lock); } while (0)# define do_raw_write_lock (rwlock) do {__acquire(lock); arch_write_lock(&(rwlock)->raw_lock); } while (0)# define do_raw_write_trylock (rwlock) arch_write_trylock(&(rwlock)->raw_lock)# define do_raw_write_unlock (rwlock) do {arch_write_unlock(&(rwlock)->raw_lock); __release(lock); } while (0)
@@ -49,7 +50,7 @@ do { \* regardless of whether CONFIG_SMP or CONFIG_PREEMPT are set. The various * methods are defined as nops in the case they are not required. */ -#define read_trylock (lock) __cond_acquire(lock, _raw_read_trylock(lock)) +#define read_trylock (lock) __cond_acquire_shared(lock, _raw_read_trylock(lock))#define write_trylock (lock) __cond_acquire(lock, _raw_write_trylock(lock))#define write_lock (lock) _raw_write_lock(lock)
@@ -112,12 +113,12 @@ do { \ } while (0)#define write_unlock_bh (lock) _raw_write_unlock_bh(lock) -#define write_trylock_irqsave (lock, flags) \ -({ \ - local_irq_save(flags); \ - write_trylock(lock) ? \ - 1 : ({ local_irq_restore(flags); 0; }); \ -}) +#define write_trylock_irqsave (lock, flags) \ + __cond_acquire(lock, ({ \ + local_irq_save(flags); \ + _raw_write_trylock(lock) ? \ + 1 : ({ local_irq_restore(flags); 0; }); \ + }))#ifdef arch_rwlock_is_contended#define rwlock_is_contended (lock) \
include/linux/rwlock_api_smp.h +23 -6
@@ -15,12 +15,12 @@* Released under the General Public License (GPL). */ -void __lockfunc _raw_read_lock(rwlock_t *lock) __acquires(lock); +void __lockfunc _raw_read_lock(rwlock_t *lock) __acquires_shared(lock); void __lockfunc _raw_write_lock(rwlock_t *lock) __acquires(lock); void __lockfunc _raw_write_lock_nested(rwlock_t *lock, int subclass) __acquires(lock); -void __lockfunc _raw_read_lock_bh(rwlock_t *lock) __acquires(lock); +void __lockfunc _raw_read_lock_bh(rwlock_t *lock) __acquires_shared(lock); void __lockfunc _raw_write_lock_bh(rwlock_t *lock) __acquires(lock); -void __lockfunc _raw_read_lock_irq(rwlock_t *lock) __acquires(lock); +void __lockfunc _raw_read_lock_irq(rwlock_t *lock) __acquires_shared(lock); void __lockfunc _raw_write_lock_irq(rwlock_t *lock) __acquires(lock); unsigned long __lockfunc _raw_read_lock_irqsave(rwlock_t *lock) __acquires(lock);
@@ -28,11 +28,11 @@ unsigned long __lockfunc _raw_write_lock_irqsave(rwlock_t *lock)__acquires (lock); int __lockfunc _raw_read_trylock(rwlock_t *lock); int __lockfunc _raw_write_trylock(rwlock_t *lock); -void __lockfunc _raw_read_unlock(rwlock_t *lock) __releases(lock); +void __lockfunc _raw_read_unlock(rwlock_t *lock) __releases_shared(lock); void __lockfunc _raw_write_unlock(rwlock_t *lock) __releases(lock); -void __lockfunc _raw_read_unlock_bh(rwlock_t *lock) __releases(lock); +void __lockfunc _raw_read_unlock_bh(rwlock_t *lock) __releases_shared(lock); void __lockfunc _raw_write_unlock_bh(rwlock_t *lock) __releases(lock); -void __lockfunc _raw_read_unlock_irq(rwlock_t *lock) __releases(lock); +void __lockfunc _raw_read_unlock_irq(rwlock_t *lock) __releases_shared(lock); void __lockfunc _raw_write_unlock_irq(rwlock_t *lock) __releases(lock); void __lockfunc_raw_read_unlock_irqrestore (rwlock_t *lock,unsigned long flags)
@@ -145,6 +145,7 @@ static inline int __raw_write_trylock(rwlock_t *lock)#if !defined(CONFIG_GENERIC_LOCKBREAK ) || defined(CONFIG_DEBUG_LOCK_ALLOC )static inline void __raw_read_lock (rwlock_t *lock) +__acquires_shared (lock) __no_capability_analysis {preempt_disable ();rwlock_acquire_read (&lock->dep_map,0 ,0 , _RET_IP_);
@@ -152,6 +153,7 @@ static inline void __raw_read_lock(rwlock_t *lock) }static inline unsigned long __raw_read_lock_irqsave (rwlock_t *lock) +__acquires_shared (lock) __no_capability_analysis {unsigned long flags;
@@ -163,6 +165,7 @@ static inline unsigned long __raw_read_lock_irqsave(rwlock_t *lock) }static inline void __raw_read_lock_irq (rwlock_t *lock) +__acquires_shared (lock) __no_capability_analysis {local_irq_disable ();preempt_disable ();
@@ -171,6 +174,7 @@ static inline void __raw_read_lock_irq(rwlock_t *lock) }static inline void __raw_read_lock_bh (rwlock_t *lock) +__acquires_shared (lock) __no_capability_analysis {__local_bh_disable_ip (_RET_IP_,SOFTIRQ_LOCK_OFFSET );rwlock_acquire_read (&lock->dep_map,0 ,0 , _RET_IP_);
@@ -178,6 +182,7 @@ static inline void __raw_read_lock_bh(rwlock_t *lock) }static inline unsigned long __raw_write_lock_irqsave (rwlock_t *lock) +__acquires (lock) __no_capability_analysis {unsigned long flags;
@@ -189,6 +194,7 @@ static inline unsigned long __raw_write_lock_irqsave(rwlock_t *lock) }static inline void __raw_write_lock_irq (rwlock_t *lock) +__acquires (lock) __no_capability_analysis {local_irq_disable ();preempt_disable ();
@@ -197,6 +203,7 @@ static inline void __raw_write_lock_irq(rwlock_t *lock) }static inline void __raw_write_lock_bh (rwlock_t *lock) +__acquires (lock) __no_capability_analysis {__local_bh_disable_ip (_RET_IP_,SOFTIRQ_LOCK_OFFSET );rwlock_acquire (&lock->dep_map,0 ,0 , _RET_IP_);
@@ -204,6 +211,7 @@ static inline void __raw_write_lock_bh(rwlock_t *lock) }static inline void __raw_write_lock (rwlock_t *lock) +__acquires (lock) __no_capability_analysis {preempt_disable ();rwlock_acquire (&lock->dep_map,0 ,0 , _RET_IP_);
@@ -211,6 +219,7 @@ static inline void __raw_write_lock(rwlock_t *lock) }static inline void __raw_write_lock_nested (rwlock_t *lock,int subclass) +__acquires (lock) __no_capability_analysis {preempt_disable ();rwlock_acquire (&lock->dep_map, subclass,0 , _RET_IP_);
@@ -220,6 +229,7 @@ static inline void __raw_write_lock_nested(rwlock_t *lock, int subclass)#endif /* !CONFIG_GENERIC_LOCKBREAK || CONFIG_DEBUG_LOCK_ALLOC */ static inline void __raw_write_unlock (rwlock_t *lock) +__releases (lock) {rwlock_release (&lock->dep_map, _RET_IP_);do_raw_write_unlock (lock);
@@ -227,6 +237,7 @@ static inline void __raw_write_unlock(rwlock_t *lock) }static inline void __raw_read_unlock (rwlock_t *lock) +__releases_shared (lock) {rwlock_release (&lock->dep_map, _RET_IP_);do_raw_read_unlock (lock);
@@ -235,6 +246,7 @@ static inline void __raw_read_unlock(rwlock_t *lock)static inline void __raw_read_unlock_irqrestore (rwlock_t *lock,unsigned long flags) +__releases_shared (lock) {rwlock_release (&lock->dep_map, _RET_IP_);do_raw_read_unlock (lock);
@@ -243,6 +255,7 @@ __raw_read_unlock_irqrestore(rwlock_t *lock, unsigned long flags) }static inline void __raw_read_unlock_irq (rwlock_t *lock) +__releases_shared (lock) {rwlock_release (&lock->dep_map, _RET_IP_);do_raw_read_unlock (lock);
@@ -251,6 +264,7 @@ static inline void __raw_read_unlock_irq(rwlock_t *lock) }static inline void __raw_read_unlock_bh (rwlock_t *lock) +__releases_shared (lock) {rwlock_release (&lock->dep_map, _RET_IP_);do_raw_read_unlock (lock);
@@ -259,6 +273,7 @@ static inline void __raw_read_unlock_bh(rwlock_t *lock)static inline void __raw_write_unlock_irqrestore (rwlock_t *lock,unsigned long flags) +__releases (lock) {rwlock_release (&lock->dep_map, _RET_IP_);do_raw_write_unlock (lock);
@@ -267,6 +282,7 @@ static inline void __raw_write_unlock_irqrestore(rwlock_t *lock, }static inline void __raw_write_unlock_irq (rwlock_t *lock) +__releases (lock) {rwlock_release (&lock->dep_map, _RET_IP_);do_raw_write_unlock (lock);
@@ -275,6 +291,7 @@ static inline void __raw_write_unlock_irq(rwlock_t *lock) }static inline void __raw_write_unlock_bh (rwlock_t *lock) +__releases (lock) {rwlock_release (&lock->dep_map, _RET_IP_);do_raw_write_unlock (lock);
include/linux/rwlock_rt.h +24 -11
@@ -22,28 +22,32 @@ do { \ \init_rwbase_rt (&(rwl)->rwbase); \ __rt_rwlock_init(rwl, #rwl, &__key); \ +__assert_cap (rwl); \ }while (0 ) -extern void rt_read_lock(rwlock_t *rwlock) __acquires(rwlock); +extern void rt_read_lock(rwlock_t *rwlock) __acquires_shared(rwlock);extern int rt_read_trylock (rwlock_t *rwlock); -extern void rt_read_unlock(rwlock_t *rwlock) __releases(rwlock); +extern void rt_read_unlock(rwlock_t *rwlock) __releases_shared(rwlock); extern void rt_write_lock(rwlock_t *rwlock) __acquires(rwlock); extern void rt_write_lock_nested(rwlock_t *rwlock, int subclass) __acquires(rwlock);extern int rt_write_trylock (rwlock_t *rwlock); extern void rt_write_unlock(rwlock_t *rwlock) __releases(rwlock); static __always_inline void read_lock(rwlock_t *rwlock) +__acquires_shared (rwlock) {rt_read_lock (rwlock); } static __always_inline void read_lock_bh(rwlock_t *rwlock) +__acquires_shared (rwlock) {local_bh_disable ();rt_read_lock (rwlock); } static __always_inline void read_lock_irq(rwlock_t *rwlock) +__acquires_shared (rwlock) {rt_read_lock (rwlock); }
@@ -55,37 +59,43 @@ static __always_inline void read_lock_irq(rwlock_t *rwlock) flags =0 ; \ } while (0) -#define read_trylock (lock) __cond_acquire(lock, rt_read_trylock(lock)) +#define read_trylock (lock) __cond_acquire_shared(lock, rt_read_trylock(lock)) static __always_inline void read_unlock(rwlock_t *rwlock) +__releases_shared (rwlock) {rt_read_unlock (rwlock); } static __always_inline void read_unlock_bh(rwlock_t *rwlock) +__releases_shared (rwlock) {rt_read_unlock (rwlock);local_bh_enable (); } static __always_inline void read_unlock_irq(rwlock_t *rwlock) +__releases_shared (rwlock) {rt_read_unlock (rwlock); } static __always_inline void read_unlock_irqrestore(rwlock_t *rwlock,unsigned long flags) +__releases_shared (rwlock) {rt_read_unlock (rwlock); } static __always_inline void write_lock(rwlock_t *rwlock) +__acquires (rwlock) {rt_write_lock (rwlock); }#ifdef CONFIG_DEBUG_LOCK_ALLOC static __always_inline void write_lock_nested(rwlock_t *rwlock, int subclass) +__acquires (rwlock) {rt_write_lock_nested (rwlock, subclass); }
@@ -94,12 +104,14 @@ static __always_inline void write_lock_nested(rwlock_t *rwlock, int subclass)#endif static __always_inline void write_lock_bh(rwlock_t *rwlock) +__acquires (rwlock) {local_bh_disable ();rt_write_lock (rwlock); } static __always_inline void write_lock_irq(rwlock_t *rwlock) +__acquires (rwlock) {rt_write_lock (rwlock); }
@@ -114,33 +126,34 @@ static __always_inline void write_lock_irq(rwlock_t *rwlock)#define write_trylock (lock) __cond_acquire(lock, rt_write_trylock(lock))#define write_trylock_irqsave (lock, flags) \ -({ \ - int __locked; \ - \ - typecheck(unsigned long, flags); \ - flags = 0; \ - __locked = write_trylock(lock); \ - __locked; \ -}) + __cond_acquire(lock, ({ \ + typecheck(unsigned long, flags); \ + flags = 0; \ + rt_write_trylock(lock); \ + })) static __always_inline void write_unlock(rwlock_t *rwlock) +__releases (rwlock) {rt_write_unlock (rwlock); } static __always_inline void write_unlock_bh(rwlock_t *rwlock) +__releases (rwlock) {rt_write_unlock (rwlock);local_bh_enable (); } static __always_inline void write_unlock_irq(rwlock_t *rwlock) +__releases (rwlock) {rt_write_unlock (rwlock); } static __always_inline void write_unlock_irqrestore(rwlock_t *rwlock,unsigned long flags) +__releases (rwlock) {rt_write_unlock (rwlock); }
include/linux/rwlock_types.h +6 -4
@@ -22,7 +22,7 @@* portions Copyright 2005, Red Hat, Inc., Ingo Molnar * Released under the General Public License (GPL). */ -typedef struct { +struct_with_capability (rwlock) {arch_rwlock_t raw_lock;#ifdef CONFIG_DEBUG_SPINLOCK unsigned int magic, owner_cpu;
@@ -31,7 +31,8 @@ typedef struct {#ifdef CONFIG_DEBUG_LOCK_ALLOC struct lockdep_map dep_map;#endif -} rwlock_t; +}; +typedef struct rwlock rwlock_t ;#define RWLOCK_MAGIC 0xdeaf1eed
@@ -54,13 +55,14 @@ typedef struct {#include <linux/rwbase_rt.h> -typedef struct { +struct_with_capability (rwlock) {struct rwbase_rt rwbase;atomic_t readers;#ifdef CONFIG_DEBUG_LOCK_ALLOC struct lockdep_map dep_map;#endif -}rwlock_t ; +}; +typedef struct rwlock rwlock_t ;#define __RWLOCK_RT_INITIALIZER (name) \ { \
include/linux/spinlock.h +29 -16
139 lines not on this page · raw message
include/linux/spinlock_api_smp.h +12 -2
@@ -34,8 +34,8 @@ unsigned long __lockfunc _raw_spin_lock_irqsave(raw_spinlock_t *lock) unsigned long __lockfunc _raw_spin_lock_irqsave_nested(raw_spinlock_t *lock, int subclass) __acquires(lock); -int __lockfunc _raw_spin_trylock(raw_spinlock_t *lock); -int __lockfunc _raw_spin_trylock_bh(raw_spinlock_t *lock); +int __lockfunc _raw_spin_trylock(raw_spinlock_t *lock) __cond_acquires(lock); +int __lockfunc _raw_spin_trylock_bh(raw_spinlock_t *lock) __cond_acquires(lock); void __lockfunc _raw_spin_unlock(raw_spinlock_t *lock) __releases(lock); void __lockfunc _raw_spin_unlock_bh(raw_spinlock_t *lock) __releases(lock); void __lockfunc _raw_spin_unlock_irq(raw_spinlock_t *lock) __releases(lock);
@@ -84,6 +84,7 @@ _raw_spin_unlock_irqrestore(raw_spinlock_t *lock, unsigned long flags)#endif static inline int __raw_spin_trylock (raw_spinlock_t *lock) +__cond_acquires (lock) {preempt_disable ();if (do_raw_spin_trylock (lock)) {
@@ -102,6 +103,7 @@ static inline int __raw_spin_trylock(raw_spinlock_t *lock)#if !defined(CONFIG_GENERIC_LOCKBREAK ) || defined(CONFIG_DEBUG_LOCK_ALLOC )static inline unsigned long __raw_spin_lock_irqsave (raw_spinlock_t *lock) +__acquires (lock) __no_capability_analysis {unsigned long flags;
@@ -113,6 +115,7 @@ static inline unsigned long __raw_spin_lock_irqsave(raw_spinlock_t *lock) }static inline void __raw_spin_lock_irq (raw_spinlock_t *lock) +__acquires (lock) __no_capability_analysis {local_irq_disable ();preempt_disable ();
@@ -121,6 +124,7 @@ static inline void __raw_spin_lock_irq(raw_spinlock_t *lock) }static inline void __raw_spin_lock_bh (raw_spinlock_t *lock) +__acquires (lock) __no_capability_analysis {__local_bh_disable_ip (_RET_IP_,SOFTIRQ_LOCK_OFFSET );spin_acquire (&lock->dep_map,0 ,0 , _RET_IP_);
@@ -128,6 +132,7 @@ static inline void __raw_spin_lock_bh(raw_spinlock_t *lock) }static inline void __raw_spin_lock (raw_spinlock_t *lock) +__acquires (lock) __no_capability_analysis {preempt_disable ();spin_acquire (&lock->dep_map,0 ,0 , _RET_IP_);
@@ -137,6 +142,7 @@ static inline void __raw_spin_lock(raw_spinlock_t *lock)#endif /* !CONFIG_GENERIC_LOCKBREAK || CONFIG_DEBUG_LOCK_ALLOC */ static inline void __raw_spin_unlock (raw_spinlock_t *lock) +__releases (lock) {spin_release (&lock->dep_map, _RET_IP_);do_raw_spin_unlock (lock);
@@ -145,6 +151,7 @@ static inline void __raw_spin_unlock(raw_spinlock_t *lock)static inline void __raw_spin_unlock_irqrestore (raw_spinlock_t *lock,unsigned long flags) +__releases (lock) {spin_release (&lock->dep_map, _RET_IP_);do_raw_spin_unlock (lock);
@@ -153,6 +160,7 @@ static inline void __raw_spin_unlock_irqrestore(raw_spinlock_t *lock, }static inline void __raw_spin_unlock_irq (raw_spinlock_t *lock) +__releases (lock) {spin_release (&lock->dep_map, _RET_IP_);do_raw_spin_unlock (lock);
@@ -161,6 +169,7 @@ static inline void __raw_spin_unlock_irq(raw_spinlock_t *lock) }static inline void __raw_spin_unlock_bh (raw_spinlock_t *lock) +__releases (lock) {spin_release (&lock->dep_map, _RET_IP_);do_raw_spin_unlock (lock);
@@ -168,6 +177,7 @@ static inline void __raw_spin_unlock_bh(raw_spinlock_t *lock) }static inline int __raw_spin_trylock_bh (raw_spinlock_t *lock) +__cond_acquires (lock) {__local_bh_disable_ip (_RET_IP_,SOFTIRQ_LOCK_OFFSET );if (do_raw_spin_trylock (lock)) {
include/linux/spinlock_api_up.h +40 -31
108 lines not on this page · raw message
include/linux/spinlock_rt.h +13 -8
@@ -20,6 +20,7 @@ static inline void __rt_spin_lock_init(spinlock_t *lock, const char *name,do { \rt_mutex_base_init (&(slock)->lock); \__rt_spin_lock_init (slock, name, key, percpu); \ +__assert_cap (slock); \ } while (0)#define _spin_lock_init (slock, percpu) \
@@ -40,6 +41,7 @@ extern int rt_spin_trylock_bh(spinlock_t *lock);extern int rt_spin_trylock (spinlock_t *lock); static __always_inline void spin_lock(spinlock_t *lock) +__acquires (lock) {rt_spin_lock (lock); }
@@ -82,6 +84,7 @@ static __always_inline void spin_lock(spinlock_t *lock) __spin_lock_irqsave_nested(lock, flags, subclass) static __always_inline void spin_lock_bh(spinlock_t *lock) +__acquires (lock) {/* Investigate: Drop bh when blocking ? */ local_bh_disable ();
@@ -89,6 +92,7 @@ static __always_inline void spin_lock_bh(spinlock_t *lock) } static __always_inline void spin_lock_irq(spinlock_t *lock) +__acquires (lock) {rt_spin_lock (lock); }
@@ -101,23 +105,27 @@ static __always_inline void spin_lock_irq(spinlock_t *lock) }while (0 ) static __always_inline void spin_unlock(spinlock_t *lock) +__releases (lock) {rt_spin_unlock (lock); } static __always_inline void spin_unlock_bh(spinlock_t *lock) +__releases (lock) {rt_spin_unlock (lock);local_bh_enable (); } static __always_inline void spin_unlock_irq(spinlock_t *lock) +__releases (lock) {rt_spin_unlock (lock); } static __always_inline void spin_unlock_irqrestore(spinlock_t *lock,unsigned long flags) +__releases (lock) {rt_spin_unlock (lock); }
@@ -132,14 +140,11 @@ static __always_inline void spin_unlock_irqrestore(spinlock_t *lock, __cond_acquire(lock, rt_spin_trylock(lock))#define spin_trylock_irqsave (lock, flags) \ -({ \ - int __locked; \ - \ - typecheck(unsigned long, flags); \ - flags = 0; \ - __locked = spin_trylock(lock); \ - __locked; \ -}) + __cond_acquire(lock, ({ \ + typecheck(unsigned long, flags); \ + flags = 0; \ + rt_spin_trylock(lock); \ + }))#define spin_is_contended (lock) (((void)(lock), 0))
include/linux/spinlock_types.h +6 -4
@@ -14,7 +14,7 @@#ifndef CONFIG_PREEMPT_RT /* Non PREEMPT_RT kernels map spinlock to raw_spinlock */ -typedef struct spinlock { +struct_with_capability (spinlock) {union { struct raw_spinlock rlock;
@@ -26,7 +26,8 @@ typedef struct spinlock { };#endif }; -} spinlock_t; +}; +typedef struct spinlock spinlock_t ;#define ___SPIN_LOCK_INITIALIZER (lockname) \ { \
@@ -47,12 +48,13 @@ typedef struct spinlock {/* PREEMPT_RT kernels map spinlock to rt_mutex */ #include <linux/rtmutex.h> -typedef struct spinlock { +struct_with_capability (spinlock) {struct rt_mutex_base lock;#ifdef CONFIG_DEBUG_LOCK_ALLOC struct lockdep_map dep_map;#endif -}spinlock_t ; +}; +typedef struct spinlock spinlock_t ;#define __SPIN_LOCK_UNLOCKED (name) \ { \
include/linux/spinlock_types_raw.h +3 -2
@@ -11,7 +11,7 @@#include <linux/lockdep_types.h> -typedef struct raw_spinlock { +struct_with_capability (raw_spinlock) {arch_spinlock_t raw_lock;#ifdef CONFIG_DEBUG_SPINLOCK unsigned int magic, owner_cpu;
@@ -20,7 +20,8 @@ typedef struct raw_spinlock {#ifdef CONFIG_DEBUG_LOCK_ALLOC struct lockdep_map dep_map;#endif -} raw_spinlock_t; +}; +typedef struct raw_spinlock raw_spinlock_t ;#define SPINLOCK_MAGIC 0xdead4ead
lib/test_capability-analysis.c +128 -0
137 lines not on this page · raw message
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRFC PATCH 10/24 compiler-capability-analysis: Change __cond_acquires to take return value
In reply to Marco ElverWhile Sparse is oblivious to the return value of conditional acquire functions, Clang's capability analysis needs to know the return value which indicates successful acquisition.
Add the additional argument, and convert existing uses.
No functional change intended.
--- fs/dlm/lock.c | 2 +- include/linux/compiler-capability-analysis.h | 14 +++++++++----- include/linux/refcount.h | 6 +++--- include/linux/spinlock.h | 6 +++--- include/linux/spinlock_api_smp.h | 8 ++++---- net/ipv4/tcp_sigpool.c | 2 +- 6 files changed, 21 insertions(+), 17 deletions(-)
fs/dlm/lock.c +1 -1
@@ -343,7 +343,7 @@ void dlm_hold_rsb(struct dlm_rsb *r)/* TODO move this to lib/refcount.c */ static __must_check bool dlm_refcount_dec_and_write_lock_bh (refcount_t *r,rwlock_t *lock) -__cond_acquires (lock) +__cond_acquires (1 , lock) {if (refcount_dec_not_one (r))return false;
include/linux/compiler-capability-analysis.h +9 -5
@@ -231,7 +231,7 @@# define __must_hold (x) __attribute__((context(x,1,1)))# define __must_not_hold (x)# define __acquires (x) __attribute__((context(x,0,1))) -# define __cond_acquires (x) __attribute__((context(x,0,-1))) +# define __cond_acquires (ret, x) __attribute__((context(x,0,-1)))# define __releases (x) __attribute__((context(x,1,0)))# define __acquire (x) __context__(x,1)# define __release (x) __context__(x,-1)
@@ -277,12 +277,14 @@/** * __cond_acquires() - function attribute, function conditionally * acquires a capability exclusively +* @ret: value returned by function if capability acquired * @x: capability instance pointer * * Function attribute declaring that the function conditionally acquires the -* given capability instance @x exclusively, but does not release it. +* given capability instance @x exclusively, but does not release it. The +* function return value @ret denotes when the capability is acquired. */ -# define __cond_acquires (x) __try_acquires_cap(1, x) +# define __cond_acquires (ret, x) __try_acquires_cap(ret, x)/** * __releases() - function attribute, function releases a capability exclusively
@@ -349,12 +351,14 @@/** * __cond_acquires_shared() - function attribute, function conditionally * acquires a capability shared +* @ret: value returned by function if capability acquired * @x: capability instance pointer * * Function attribute declaring that the function conditionally acquires the -* given capability instance @x with shared access, but does not release it. +* given capability instance @x with shared access, but does not release it. The +* function return value @ret denotes when the capability is acquired. */ -# define __cond_acquires_shared (x) __try_acquires_shared_cap(1, x) +# define __cond_acquires_shared (ret, x) __try_acquires_shared_cap(ret, x)/** * __releases_shared() - function attribute, function releases a
include/linux/refcount.h +3 -3
@@ -353,9 +353,9 @@ static inline void refcount_dec(refcount_t *r)extern __must_check bool refcount_dec_if_one (refcount_t *r);extern __must_check bool refcount_dec_not_one (refcount_t *r); -extern __must_check bool refcount_dec_and_mutex_lock(refcount_t *r, struct mutex *lock) __cond_acquires(lock); -extern __must_check bool refcount_dec_and_lock(refcount_t *r, spinlock_t *lock) __cond_acquires(lock); +extern __must_check bool refcount_dec_and_mutex_lock(refcount_t *r, struct mutex *lock) __cond_acquires(1, lock); +extern __must_check bool refcount_dec_and_lock(refcount_t *r, spinlock_t *lock) __cond_acquires(1, lock); extern __must_check bool refcount_dec_and_lock_irqsave(refcount_t *r, spinlock_t *lock, - unsigned long *flags) __cond_acquires(lock); + unsigned long *flags) __cond_acquires(1, lock);#endif /* _LINUX_REFCOUNT_H */
include/linux/spinlock.h +3 -3
@@ -362,7 +362,7 @@ static __always_inline void spin_lock_bh(spinlock_t *lock) } static __always_inline int spin_trylock(spinlock_t *lock) -__cond_acquires (lock) __no_capability_analysis +__cond_acquires (1 , lock) __no_capability_analysis {return raw_spin_trylock (&lock->rlock); }
@@ -420,13 +420,13 @@ static __always_inline void spin_unlock_irqrestore(spinlock_t *lock, unsigned lo } static __always_inline int spin_trylock_bh(spinlock_t *lock) -__cond_acquires (lock) __no_capability_analysis +__cond_acquires (1 , lock) __no_capability_analysis {return raw_spin_trylock_bh (&lock->rlock); } static __always_inline int spin_trylock_irq(spinlock_t *lock) -__cond_acquires (lock) __no_capability_analysis +__cond_acquires (1 , lock) __no_capability_analysis {return raw_spin_trylock_irq (&lock->rlock); }
include/linux/spinlock_api_smp.h +4 -4
@@ -34,8 +34,8 @@ unsigned long __lockfunc _raw_spin_lock_irqsave(raw_spinlock_t *lock) unsigned long __lockfunc _raw_spin_lock_irqsave_nested(raw_spinlock_t *lock, int subclass) __acquires(lock); -int __lockfunc _raw_spin_trylock(raw_spinlock_t *lock) __cond_acquires(lock); -int __lockfunc _raw_spin_trylock_bh(raw_spinlock_t *lock) __cond_acquires(lock); +int __lockfunc _raw_spin_trylock(raw_spinlock_t *lock) __cond_acquires(1, lock); +int __lockfunc _raw_spin_trylock_bh(raw_spinlock_t *lock) __cond_acquires(1, lock); void __lockfunc _raw_spin_unlock(raw_spinlock_t *lock) __releases(lock); void __lockfunc _raw_spin_unlock_bh(raw_spinlock_t *lock) __releases(lock); void __lockfunc _raw_spin_unlock_irq(raw_spinlock_t *lock) __releases(lock);
@@ -84,7 +84,7 @@ _raw_spin_unlock_irqrestore(raw_spinlock_t *lock, unsigned long flags)#endif static inline int __raw_spin_trylock (raw_spinlock_t *lock) -__cond_acquires (lock) +__cond_acquires (1 , lock) {preempt_disable ();if (do_raw_spin_trylock (lock)) {
@@ -177,7 +177,7 @@ static inline void __raw_spin_unlock_bh(raw_spinlock_t *lock) }static inline int __raw_spin_trylock_bh (raw_spinlock_t *lock) -__cond_acquires (lock) +__cond_acquires (1 , lock) {__local_bh_disable_ip (_RET_IP_,SOFTIRQ_LOCK_OFFSET );if (do_raw_spin_trylock (lock)) {
net/ipv4/tcp_sigpool.c +1 -1
@@ -257,7 +257,7 @@ void tcp_sigpool_get(unsigned int id) }EXPORT_SYMBOL_GPL (tcp_sigpool_get); -int tcp_sigpool_start (unsigned int id,struct tcp_sigpool *c)__cond_acquires (RCU_BH ) +int tcp_sigpool_start (unsigned int id,struct tcp_sigpool *c)__cond_acquires (0 ,RCU_BH ) {struct crypto_ahash *hash;
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRFC PATCH 11/24 locking/mutex: Support Clang's capability analysis
In reply to Marco ElverAdd support for Clang's capability analysis for mutex.
--- .../dev-tools/capability-analysis.rst | 2 +- include/linux/mutex.h | 29 +++++---- include/linux/mutex_types.h | 4 +- lib/test_capability-analysis.c | 64 +++++++++++++++++++ 4 files changed, 82 insertions(+), 17 deletions(-)
Documentation/dev-tools/capability-analysis.rst +1 -1
@@ -85,7 +85,7 @@ Supported Kernel Primitives ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Currently the following synchronization primitives are supported: -`raw_spinlock_t`, `spinlock_t`, `rwlock_t`. +`raw_spinlock_t`, `spinlock_t`, `rwlock_t`, `mutex`. For capabilities with an initialization function (e.g., `spin_lock_init()`), calling this function on the capability instance before initializing any
include/linux/mutex.h +15 -14
@@ -62,6 +62,7 @@ do { \static struct lock_class_key __key; \ \ __mutex_init((mutex), #mutex, &__key); \ +__assert_cap (mutex); \ }while (0 )/**
@@ -154,14 +155,14 @@ static inline int __devm_mutex_init(struct device *dev, struct mutex *lock)* Also see Documentation/locking/mutex-design.rst. */ #ifdef CONFIG_DEBUG_LOCK_ALLOC -extern void mutex_lock_nested (struct mutex *lock,unsigned int subclass); +extern void mutex_lock_nested(struct mutex *lock, unsigned int subclass) __acquires(lock);extern void _mutex_lock_nest_lock (struct mutex *lock,struct lockdep_map *nest_lock); extern int __must_check mutex_lock_interruptible_nested(struct mutex *lock, -unsigned int subclass); + unsigned int subclass) __cond_acquires(0, lock); extern int __must_check mutex_lock_killable_nested(struct mutex *lock, -unsigned int subclass); -extern void mutex_lock_io_nested (struct mutex *lock,unsigned int subclass); + unsigned int subclass) __cond_acquires(0, lock); +extern void mutex_lock_io_nested(struct mutex *lock, unsigned int subclass) __acquires(lock);#define mutex_lock (lock) mutex_lock_nested(lock, 0)#define mutex_lock_interruptible (lock) mutex_lock_interruptible_nested(lock, 0)
@@ -175,10 +176,10 @@ do { \ } while (0)#else -extern void mutex_lock (struct mutex *lock); -extern int __must_check mutex_lock_interruptible (struct mutex *lock); -extern int __must_check mutex_lock_killable (struct mutex *lock); -extern void mutex_lock_io (struct mutex *lock); +extern void mutex_lock(struct mutex *lock) __acquires(lock); +extern int __must_check mutex_lock_interruptible(struct mutex *lock) __cond_acquires(0, lock); +extern int __must_check mutex_lock_killable(struct mutex *lock) __cond_acquires(0, lock); +extern void mutex_lock_io(struct mutex *lock) __acquires(lock);# define mutex_lock_nested (lock, subclass) mutex_lock(lock)# define mutex_lock_interruptible_nested (lock, subclass) mutex_lock_interruptible(lock)
@@ -193,13 +194,13 @@ extern void mutex_lock_io(struct mutex *lock);* * Returns 1 if the mutex has been acquired successfully, and 0 on contention. */ -extern int mutex_trylock (struct mutex *lock); -extern void mutex_unlock (struct mutex *lock); +extern int mutex_trylock(struct mutex *lock) __cond_acquires(1, lock); +extern void mutex_unlock(struct mutex *lock) __releases(lock); -extern int atomic_dec_and_mutex_lock (atomic_t *cnt,struct mutex *lock); +extern int atomic_dec_and_mutex_lock(atomic_t *cnt, struct mutex *lock) __cond_acquires(1, lock); -DEFINE_GUARD(mutex, struct mutex *, mutex_lock(_T), mutex_unlock(_T)) -DEFINE_GUARD_COND(mutex, _try, mutex_trylock(_T)) -DEFINE_GUARD_COND(mutex, _intr, mutex_lock_interruptible(_T) == 0) +DEFINE_LOCK_GUARD_1(mutex, struct mutex, mutex_lock(_T->lock), mutex_unlock(_T->lock)) +DEFINE_LOCK_GUARD_1_COND(mutex, _try, mutex_trylock(_T->lock)) +DEFINE_LOCK_GUARD_1_COND(mutex, _intr, mutex_lock_interruptible(_T->lock) == 0)#endif /* __LINUX_MUTEX_H */
include/linux/mutex_types.h +2 -2
@@ -38,7 +38,7 @@* - detects multi-task circular deadlocks and prints out all affected * locks and tasks (and only those tasks) */ -struct mutex { +struct_with_capability (mutex) {atomic_long_t owner;raw_spinlock_t wait_lock;#ifdef CONFIG_MUTEX_SPIN_ON_OWNER
@@ -59,7 +59,7 @@ struct mutex {*/ #include <linux/rtmutex.h> -struct mutex { +struct_with_capability (mutex) {struct rt_mutex_base rtmutex;#ifdef CONFIG_DEBUG_LOCK_ALLOC struct lockdep_map dep_map;
lib/test_capability-analysis.c +64 -0
73 lines not on this page · raw message
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Peter Zijlstra
maintainerRe: [PATCH RFC 11/24] locking/mutex: Support Clang's capability analysis
Review
Quoted 11 lines · Marco Elver
extern int
__must_checkmutex_lock_interruptible_nested(struct mutex *lock,(quoted diff) +6
-0+ unsigned int subclass) __cond_acquires(0, lock); extern int __must_check mutex_lock_killable_nested(struct mutex *lock, + unsigned int subclass) __cond_acquires(0, lock); +extern int __must_check mutex_lock_interruptible(struct mutex *lock) __cond_acquires(0, lock); +extern int __must_check mutex_lock_killable(struct mutex *lock) __cond_acquires(0, lock); +extern int mutex_trylock(struct mutex *lock) __cond_acquires(1, lock); +extern int atomic_dec_and_mutex_lock(atomic_t *cnt, struct mutex *lock) __cond_acquires(1, lock);
So this form is *MUCH* saner than what we currently have.
Can we please fix up all the existing __cond_lock() code too?
Re: [PATCH RFC 11/24] locking/mutex: Support Clang's capability analysis
Quoted 1 line · Peter Zijlstra
Can we please fix up all the existing
__cond_lock()code too?
It would be great to get rid of __cond_lock().
In the description of commit 4a557a5d1a61 ("sparse: introduce
conditional lock acquire function attribute") I found the following
URL: https://lore.kernel.org/all/CAHk-=wjZfO9hGqJ2_hGQG3U_XzSh9_XaXze=HgPdvJbgrvASfA@mail.gmail.com/
That URL points at an e-mail from Linus Torvalds with a patch for sparse that implements support for __cond_acquires(). It seems to me that the sparse patch has never been applied to the sparse code base (the git URL for sparse is available at https://sparse.docs.kernel.org/en/latest/).
Additionally, the most recent commit to the sparse code base is from
more than a year ago (see also https://git.kernel.org/pub/scm/devel/sparse/sparse.git/).
In other words, switching from __cond_lock() to __cond_acquires() probably will make sparse report more "context imbalance" warnings.
If this is a concern to anyone, please speak up.
Thanks,
Bart.
Marco Elver
maintainerRFC PATCH 12/24 locking/seqlock: Support Clang's capability analysis
In reply to Marco ElverAdd support for Clang's capability analysis for seqlock_t.
--- .../dev-tools/capability-analysis.rst | 2 +- include/linux/seqlock.h | 24 +++++++++++ include/linux/seqlock_types.h | 5 ++- lib/test_capability-analysis.c | 43 +++++++++++++++++++ 4 files changed, 71 insertions(+), 3 deletions(-)
Documentation/dev-tools/capability-analysis.rst +1 -1
@@ -85,7 +85,7 @@ Supported Kernel Primitives ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Currently the following synchronization primitives are supported: -`raw_spinlock_t`, `spinlock_t`, `rwlock_t`, `mutex`. +`raw_spinlock_t`, `spinlock_t`, `rwlock_t`, `mutex`, `seqlock_t`. For capabilities with an initialization function (e.g., `spin_lock_init()`), calling this function on the capability instance before initializing any
include/linux/seqlock.h +24 -0
@@ -816,6 +816,7 @@ static __always_inline void write_seqcount_latch_end(seqcount_latch_t *s)do { \spin_lock_init (&(sl)->lock); \seqcount_spinlock_init (&(sl)->seqcount, &(sl)->lock); \ +__assert_cap (sl); \ }while (0 )/**
@@ -832,6 +833,7 @@ static __always_inline void write_seqcount_latch_end(seqcount_latch_t *s)* Return: count, to be passed to read_seqretry() */ static inline unsigned read_seqbegin (const seqlock_t *sl) +__acquires_shared (sl) __no_capability_analysis {return read_seqcount_begin (&sl->seqcount); }
@@ -848,6 +850,7 @@ static inline unsigned read_seqbegin(const seqlock_t *sl)* Return: true if a read section retry is required, else false */ static inline unsigned read_seqretry (const seqlock_t *sl,unsigned start) +__releases_shared (sl) __no_capability_analysis {return read_seqcount_retry (&sl->seqcount, start); }
@@ -872,6 +875,7 @@ static inline unsigned read_seqretry(const seqlock_t *sl, unsigned start)* _irqsave or _bh variants of this function instead. */ static inline void write_seqlock (seqlock_t *sl) +__acquires (sl) __no_capability_analysis {spin_lock (&sl->lock);do_write_seqcount_begin (&sl->seqcount.seqcount);
@@ -885,6 +889,7 @@ static inline void write_seqlock(seqlock_t *sl)* critical section of given seqlock_t. */ static inline void write_sequnlock (seqlock_t *sl) +__releases (sl) __no_capability_analysis {do_write_seqcount_end (&sl->seqcount.seqcount);spin_unlock (&sl->lock);
@@ -898,6 +903,7 @@ static inline void write_sequnlock(seqlock_t *sl)* other write side sections, can be invoked from softirq contexts. */ static inline void write_seqlock_bh (seqlock_t *sl) +__acquires (sl) __no_capability_analysis {spin_lock_bh (&sl->lock);do_write_seqcount_begin (&sl->seqcount.seqcount);
@@ -912,6 +918,7 @@ static inline void write_seqlock_bh(seqlock_t *sl)* write_seqlock_bh(). */ static inline void write_sequnlock_bh (seqlock_t *sl) +__releases (sl) __no_capability_analysis {do_write_seqcount_end (&sl->seqcount.seqcount);spin_unlock_bh (&sl->lock);
@@ -925,6 +932,7 @@ static inline void write_sequnlock_bh(seqlock_t *sl)* other write sections, can be invoked from hardirq contexts. */ static inline void write_seqlock_irq (seqlock_t *sl) +__acquires (sl) __no_capability_analysis {spin_lock_irq (&sl->lock);do_write_seqcount_begin (&sl->seqcount.seqcount);
@@ -938,12 +946,14 @@ static inline void write_seqlock_irq(seqlock_t *sl)* seqlock_t write side section opened with write_seqlock_irq(). */ static inline void write_sequnlock_irq (seqlock_t *sl) +__releases (sl) __no_capability_analysis {do_write_seqcount_end (&sl->seqcount.seqcount);spin_unlock_irq (&sl->lock); }static inline unsigned long __write_seqlock_irqsave (seqlock_t *sl) +__acquires (sl) __no_capability_analysis {unsigned long flags;
@@ -976,6 +986,7 @@ static inline unsigned long __write_seqlock_irqsave(seqlock_t *sl)*/ static inline void write_sequnlock_irqrestore (seqlock_t *sl,unsigned long flags) +__releases (sl) __no_capability_analysis {do_write_seqcount_end (&sl->seqcount.seqcount);spin_unlock_irqrestore (&sl->lock, flags);
@@ -998,6 +1009,7 @@ write_sequnlock_irqrestore(seqlock_t *sl, unsigned long flags)* The opened read section must be closed with read_sequnlock_excl(). */ static inline void read_seqlock_excl (seqlock_t *sl) +__acquires_shared (sl) __no_capability_analysis {spin_lock (&sl->lock); }
@@ -1007,6 +1019,7 @@ static inline void read_seqlock_excl(seqlock_t *sl)* @sl: Pointer to seqlock_t */ static inline void read_sequnlock_excl (seqlock_t *sl) +__releases_shared (sl) __no_capability_analysis {spin_unlock (&sl->lock); }
@@ -1021,6 +1034,7 @@ static inline void read_sequnlock_excl(seqlock_t *sl)* from softirq contexts. */ static inline void read_seqlock_excl_bh (seqlock_t *sl) +__acquires_shared (sl) __no_capability_analysis {spin_lock_bh (&sl->lock); }
@@ -1031,6 +1045,7 @@ static inline void read_seqlock_excl_bh(seqlock_t *sl)* @sl: Pointer to seqlock_t */ static inline void read_sequnlock_excl_bh (seqlock_t *sl) +__releases_shared (sl) __no_capability_analysis {spin_unlock_bh (&sl->lock); }
@@ -1045,6 +1060,7 @@ static inline void read_sequnlock_excl_bh(seqlock_t *sl)* hardirq context. */ static inline void read_seqlock_excl_irq (seqlock_t *sl) +__acquires_shared (sl) __no_capability_analysis {spin_lock_irq (&sl->lock); }
@@ -1055,11 +1071,13 @@ static inline void read_seqlock_excl_irq(seqlock_t *sl)* @sl: Pointer to seqlock_t */ static inline void read_sequnlock_excl_irq (seqlock_t *sl) +__releases_shared (sl) __no_capability_analysis {spin_unlock_irq (&sl->lock); }static inline unsigned long __read_seqlock_excl_irqsave (seqlock_t *sl) +__acquires_shared (sl) __no_capability_analysis {unsigned long flags;
@@ -1089,6 +1107,7 @@ static inline unsigned long __read_seqlock_excl_irqsave(seqlock_t *sl)*/ static inline void read_sequnlock_excl_irqrestore (seqlock_t *sl,unsigned long flags) +__releases_shared (sl) __no_capability_analysis {spin_unlock_irqrestore (&sl->lock, flags); }
@@ -1125,6 +1144,7 @@ read_sequnlock_excl_irqrestore(seqlock_t *sl, unsigned long flags)* parameter of the next read_seqbegin_or_lock() iteration. */ static inline void read_seqbegin_or_lock (seqlock_t *lock,int *seq) +__acquires_shared (lock) __no_capability_analysis {if (!(*seq &1 ))/* Even */ *seq =read_seqbegin (lock);
@@ -1140,6 +1160,7 @@ static inline void read_seqbegin_or_lock(seqlock_t *lock, int *seq)* Return: true if a read section retry is required, false otherwise */ static inline int need_seqretry (seqlock_t *lock,int seq) +__releases_shared (lock) __no_capability_analysis {return !(seq &1 ) &&read_seqretry (lock, seq); }
@@ -1153,6 +1174,7 @@ static inline int need_seqretry(seqlock_t *lock, int seq)* with read_seqbegin_or_lock() and validated by need_seqretry(). */ static inline void done_seqretry (seqlock_t *lock,int seq) + __no_capability_analysis {if (seq &1 )read_sequnlock_excl (lock);
@@ -1180,6 +1202,7 @@ static inline void done_seqretry(seqlock_t *lock, int seq)*/ static inline unsigned long read_seqbegin_or_lock_irqsave (seqlock_t *lock,int *seq) +__acquires_shared (lock) __no_capability_analysis {unsigned long flags =0 ;
@@ -1205,6 +1228,7 @@ read_seqbegin_or_lock_irqsave(seqlock_t *lock, int *seq)*/ static inline void done_seqretry_irqrestore (seqlock_t *lock,int seq,unsigned long flags) + __no_capability_analysis {if (seq &1 )read_sequnlock_excl_irqrestore (lock, flags);
include/linux/seqlock_types.h +3 -2
@@ -81,13 +81,14 @@ SEQCOUNT_LOCKNAME(mutex, struct mutex, true, mutex)* - Comments on top of seqcount_t * - Documentation/locking/seqlock.rst */ -typedef struct { +struct_with_capability (seqlock) {/* * Make sure that readers don't starve writers on PREEMPT_RT: use * seqcount_spinlock_t instead of seqcount_t. Check __SEQ_LOCK(). */ seqcount_spinlock_t seqcount;spinlock_t lock; -}seqlock_t ; +}; +typedef struct seqlock seqlock_t ;#endif /* __LINUX_SEQLOCK_TYPES_H */
lib/test_capability-analysis.c +43 -0
52 lines not on this page · raw message
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRFC PATCH 13/24 bit_spinlock: Include missing <asm/processor.h>
In reply to Marco ElverIncluding <linux/bit_spinlock.h> into an empty TU will result in the compiler complaining:
./include/linux/bit_spinlock.h:34:4: error: call to undeclared function 'cpu_relax'; <...>
34 | cpu_relax();
| ^1 error generated.
Include <asm/processor.h> to allow including bit_spinlock.h where <asm/processor.h> is not otherwise included.
--- include/linux/bit_spinlock.h | 2 ++ 1 file changed, 2 insertions(+)
include/linux/bit_spinlock.h +2 -0
@@ -7,6 +7,8 @@#include <linux/atomic.h> #include <linux/bug.h> +#include <asm/processor.h> /* for cpu_relax() */ +/* * bit-based spin_lock() *
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRFC PATCH 14/24 bit_spinlock: Support Clang's capability analysis
In reply to Marco ElverThe annotations for bit_spinlock.h have simply been using "bitlock" as the token. For Sparse, that was likely sufficient in most cases. But Clang's capability analysis is more precise, and we need to ensure we can distinguish different bitlocks.
To do so, add a token capability, and a macro __bitlock(bitnum, addr) that is used to construct unique per-bitlock tokens.
Add the appropriate test.
<linux/list_bl.h> is implicitly included through other includes, and requires 2 annotations to indicate that acquisition (without release) and release (without prior acquisition) of its bitlock is intended.
--- .../dev-tools/capability-analysis.rst | 3 ++- include/linux/bit_spinlock.h | 22 +++++++++++++--- include/linux/list_bl.h | 2 ++ lib/test_capability-analysis.c | 26 +++++++++++++++++++ 4 files changed, 48 insertions(+), 5 deletions(-)
Documentation/dev-tools/capability-analysis.rst +2 -1
@@ -85,7 +85,8 @@ Supported Kernel Primitives ~~~~~~~~~~~~~~~~~~~~~~~~~~~ Currently the following synchronization primitives are supported: -`raw_spinlock_t`, `spinlock_t`, `rwlock_t`, `mutex`, `seqlock_t`. +`raw_spinlock_t`, `spinlock_t`, `rwlock_t`, `mutex`, `seqlock_t`, +`bit_spinlock`. For capabilities with an initialization function (e.g., `spin_lock_init()`), calling this function on the capability instance before initializing any
include/linux/bit_spinlock.h +18 -4
@@ -9,6 +9,16 @@#include <asm/processor.h> /* for cpu_relax() */ +/* +* For static capability analysis, we need a unique token for each possible bit +* that can be used as a bit_spinlock. The easiest way to do that is to create a +* fake capability that we can cast to with the __bitlock(bitnum, addr) macro +* below, which will give us unique instances for each (bit, addr) pair that the +* static analysis can use. +*/ +struct_with_capability (__capability_bitlock) { }; +#define __bitlock (bitnum, addr) (struct __capability_bitlock *)(bitnum + (addr)) +/* * bit-based spin_lock() *
@@ -16,6 +26,7 @@* are significantly faster. */ static inline void bit_spin_lock (int bitnum,unsigned long *addr) +__acquires (__bitlock (bitnum, addr)) {/* * Assuming the lock is uncontended, this never enters
@@ -34,13 +45,14 @@ static inline void bit_spin_lock(int bitnum, unsigned long *addr) preempt_disable(); }#endif -__acquire (bitlock); +__acquire (__bitlock (bitnum ,addr )); }/* * Return true if it was acquired */ static inline int bit_spin_trylock (int bitnum,unsigned long *addr) +__cond_acquires (1 ,__bitlock (bitnum, addr)) {preempt_disable ();#if defined(CONFIG_SMP ) || defined(CONFIG_DEBUG_SPINLOCK )
@@ -49,7 +61,7 @@ static inline int bit_spin_trylock(int bitnum, unsigned long *addr) return 0; }#endif -__acquire (bitlock); +__acquire (__bitlock (bitnum ,addr ));return 1 ; }
@@ -57,6 +69,7 @@ static inline int bit_spin_trylock(int bitnum, unsigned long *addr)* bit-based spin_unlock() */ static inline void bit_spin_unlock (int bitnum,unsigned long *addr) +__releases (__bitlock (bitnum, addr)) {#ifdef CONFIG_DEBUG_SPINLOCK BUG_ON (!test_bit (bitnum, addr));
@@ -65,7 +78,7 @@ static inline void bit_spin_unlock(int bitnum, unsigned long *addr)clear_bit_unlock (bitnum, addr);#endif preempt_enable (); -__release (bitlock); +__release (__bitlock (bitnum, addr)); }/*
@@ -74,6 +87,7 @@ static inline void bit_spin_unlock(int bitnum, unsigned long *addr)* protecting the rest of the flags in the word. */ static inline void __bit_spin_unlock (int bitnum,unsigned long *addr) +__releases (__bitlock (bitnum, addr)) {#ifdef CONFIG_DEBUG_SPINLOCK BUG_ON (!test_bit (bitnum, addr));
@@ -82,7 +96,7 @@ static inline void __bit_spin_unlock(int bitnum, unsigned long *addr)__clear_bit_unlock (bitnum, addr);#endif preempt_enable (); -__release (bitlock); +__release (__bitlock (bitnum, addr)); }/*
include/linux/list_bl.h +2 -0
@@ -144,11 +144,13 @@ static inline void hlist_bl_del_init(struct hlist_bl_node *n) }static inline void hlist_bl_lock (struct hlist_bl_head *b) +__acquires (__bitlock (0 , b)) {bit_spin_lock (0 , (unsigned long *)b); }static inline void hlist_bl_unlock (struct hlist_bl_head *b) +__releases (__bitlock (0 , b)) {__bit_spin_unlock (0 , (unsigned long *)b); }
lib/test_capability-analysis.c +26 -0
@@ -4,6 +4,7 @@* positive errors when compiled with Clang's capability analysis. */ +#include <linux/bit_spinlock.h> #include <linux/build_bug.h> #include <linux/mutex.h> #include <linux/seqlock.h>
@@ -251,3 +252,28 @@ static void __used test_seqlock_writer(struct test_seqlock_data *d) d->counter++;write_sequnlock_irqrestore (&d->sl, flags); } + +struct test_bit_spinlock_data { +unsigned long bits; + int counter __var_guarded_by(__bitlock(3, &bits)); +}; + +static void __used test_bit_spin_lock (struct test_bit_spinlock_data *d) +{ +/* +* Note, the analysis seems to have false negatives, because it won't +* precisely recognize the bit of the fake __bitlock() token. +*/ +bit_spin_lock (3 , &d->bits); + d->counter++; +bit_spin_unlock (3 , &d->bits); + +bit_spin_lock (3 , &d->bits); + d->counter++; +__bit_spin_unlock (3 , &d->bits); + +if (bit_spin_trylock (3 , &d->bits)) { + d->counter++; +bit_spin_unlock (3 , &d->bits); + } +}
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRFC PATCH 15/24 rcu: Support Clang's capability analysis
In reply to Marco ElverImprove the existing annotations to properly support Clang's capability analysis.
The old annotations distinguished between RCU, RCU_BH, and RCU_SCHED. However, it does not make sense to acquire rcu_read_lock_bh() after rcu_read_lock() - annotate the _bh() and _sched() variants to also acquire 'RCU', so that Clang (and also Sparse) can warn about it.
The above change also simplified introducing annotations, where it would not matter if RCU, RCU_BH, or RCU_SCHED is acquired: through the introduction of __rcu_guarded, we can use Clang's capability analysis to warn if a pointer is dereferenced without any of the RCU locks held, or updated without the appropriate helpers.
The primitives rcu_assign_pointer() and friends are wrapped with capability_unsafe(), which enforces using them to update RCU-protected pointers marked with __rcu_guarded.
--- .../dev-tools/capability-analysis.rst | 2 +- include/linux/cleanup.h | 4 + include/linux/rcupdate.h | 73 +++++++++++++------ lib/test_capability-analysis.c | 68 +++++++++++++++++ 4 files changed, 123 insertions(+), 24 deletions(-)
Documentation/dev-tools/capability-analysis.rst +1 -1
@@ -86,7 +86,7 @@ Supported Kernel Primitives Currently the following synchronization primitives are supported: `raw_spinlock_t`, `spinlock_t`, `rwlock_t`, `mutex`, `seqlock_t`, -`bit_spinlock`. +`bit_spinlock`, RCU. For capabilities with an initialization function (e.g., `spin_lock_init()`), calling this function on the capability instance before initializing any
include/linux/cleanup.h +4 -0
@@ -404,6 +404,10 @@ static inline class_##_name##_t class_##_name##_constructor(void) \return _t; \ } +#define DECLARE_LOCK_GUARD_0_ATTRS (_name, _lock, _unlock) \ +static inline class_##_name##_t class_##_name##_constructor(void) _lock;\ +static inline void class_##_name##_destructor(class_##_name##_t *_T) _unlock +#define DEFINE_LOCK_GUARD_1 (_name, _type, _lock, _unlock, ...) \ __DEFINE_CLASS_IS_CONDITIONAL(_name, false); \ __DEFINE_UNLOCK_GUARD(_name, _type, _unlock, __VA_ARGS__) \
include/linux/rcupdate.h +50 -23
216 lines not on this page · raw message
lib/test_capability-analysis.c +68 -0
77 lines not on this page · raw message
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Paul E. McKenney
maintainerRe: [PATCH RFC 15/24] rcu: Support Clang's capability analysis
Quoted 7 lines · Marco Elver
Improve the existing annotations to properly support Clang's capability analysis.
The old annotations distinguished between RCU,
RCU_BH, andRCU_SCHED. However, it does not make sense to acquirercu_read_lock_bh()afterrcu_read_lock()- annotate the_bh()and_sched()variants to also acquire 'RCU', so that Clang (and also Sparse) can warn about it.
You lost me on this one. What breaks if rcu_read_lock_bh() is invoked while rcu_read_lock() is in effect?
Thanx, Paul
Quoted 370 lines
The above change also simplified introducing annotations, where it would not matter if RCU,
RCU_BH, orRCU_SCHEDis acquired: through the introduction of__rcu_guarded, we can use Clang's capability analysis to warn if a pointer is dereferenced without any of the RCU locks held, or updated without the appropriate helpers.The primitives
rcu_assign_pointer()and friends are wrapped withcapability_unsafe(), which enforces using them to update RCU-protected pointers marked with__rcu_guarded.Signed-off-by Marco Elver <[email protected]>--- .../dev-tools/capability-analysis.rst | 2 +- include/linux/cleanup.h | 4 + include/linux/rcupdate.h | 73 +++++++++++++------ lib/test_capability-analysis.c | 68 +++++++++++++++++ 4 files changed, 123 insertions(+), 24 deletions(-)Documentation/dev-tools/capability-analysis.rst +1
-1@@ -86,7 +86,7 @@ Supported Kernel Primitives Currently the following synchronization primitives are supported: `raw_spinlock_t`, `spinlock_t`, `rwlock_t`, `mutex`, `seqlock_t`, -`bit_spinlock`. +`bit_spinlock`, RCU. For capabilities with an initialization function (e.g., `spin_lock_init()`), calling this function on the capability instance before initializing anyinclude/linux/cleanup.h +4
-0@@ -404,6 +404,10 @@ static inline class_##_name##_t class_##_name##_constructor(void) \return _t; \ } +#define DECLARE_LOCK_GUARD_0_ATTRS (_name, _lock, _unlock) \ +static inline class_##_name##_t class_##_name##_constructor(void) _lock;\ +static inline void class_##_name##_destructor(class_##_name##_t *_T) _unlock +#define DEFINE_LOCK_GUARD_1 (_name, _type, _lock, _unlock, ...) \ __DEFINE_CLASS_IS_CONDITIONAL(_name, false); \ __DEFINE_UNLOCK_GUARD(_name, _type, _unlock, __VA_ARGS__) \include/linux/rcupdate.h +50
-23@@ -31,6 +31,16 @@#include <asm/processor.h> #include <linux/context_tracking_irq.h> +token_capability (RCU ); +token_capability_instance (RCU ,RCU_SCHED ); +token_capability_instance (RCU ,RCU_BH ); + +/* +* A convenience macro that can be used for RCU-protected globals or struct +* members; adds type qualifier __rcu, and also enforces __var_guarded_by(RCU). +*/ +#define __rcu_guarded __rcu __var_guarded_by(RCU) +#define ULONG_CMP_GE (a, b) (ULONG_MAX / 2 >= (a) - (b))#define ULONG_CMP_LT (a, b) (ULONG_MAX / 2 < (a) - (b))@@ -431,7 +441,8 @@ static inline void rcu_preempt_sleep_check(void) { }// See RCU_LOCKDEP_WARN() for an explanation of the double call to // debug_lockdep_rcu_enabled(). -static inline bool lockdep_assert_rcu_helper (bool c) +static inline bool lockdep_assert_rcu_helper (bool c,const struct __capability_RCU *cap) +__asserts_shared_cap (RCU )__asserts_shared_cap (cap) {return debug_lockdep_rcu_enabled () && (c || !rcu_is_watching() || !rcu_lockdep_current_cpu_online()) &&@@ -444,7 +455,7 @@ static inline bool lockdep_assert_rcu_helper(bool c)* Splats if lockdep is enabled and there is no rcu_read_lock() in effect. */ #define lockdep_assert_in_rcu_read_lock () \ - WARN_ON_ONCE(lockdep_assert_rcu_helper(!lock_is_held(&rcu_lock_map))) + WARN_ON_ONCE(lockdep_assert_rcu_helper(!lock_is_held(&rcu_lock_map), RCU))/** * lockdep_assert_in_rcu_read_lock_bh - WARN if not protected by rcu_read_lock_bh() @@ -454,7 +465,7 @@ static inline bool lockdep_assert_rcu_helper(bool c)* actual rcu_read_lock_bh() is required. */ #define lockdep_assert_in_rcu_read_lock_bh () \ - WARN_ON_ONCE(lockdep_assert_rcu_helper(!lock_is_held(&rcu_bh_lock_map))) + WARN_ON_ONCE(lockdep_assert_rcu_helper(!lock_is_held(&rcu_bh_lock_map), RCU_BH))/** * lockdep_assert_in_rcu_read_lock_sched - WARN if not protected by rcu_read_lock_sched() @@ -464,7 +475,7 @@ static inline bool lockdep_assert_rcu_helper(bool c)* instead an actual rcu_read_lock_sched() is required. */ #define lockdep_assert_in_rcu_read_lock_sched () \ - WARN_ON_ONCE(lockdep_assert_rcu_helper(!lock_is_held(&rcu_sched_lock_map))) + WARN_ON_ONCE(lockdep_assert_rcu_helper(!lock_is_held(&rcu_sched_lock_map), RCU_SCHED))/** * lockdep_assert_in_rcu_reader - WARN if not within some type of RCU reader @@ -482,17 +493,17 @@ static inline bool lockdep_assert_rcu_helper(bool c)WARN_ON_ONCE (lockdep_assert_rcu_helper (!lock_is_held (&rcu_lock_map) && \ !lock_is_held (&rcu_bh_lock_map) && \ !lock_is_held (&rcu_sched_lock_map) && \ - preemptible())) + preemptible(), RCU))#else /* #ifdef CONFIG_PROVE_RCU */ #define RCU_LOCKDEP_WARN (c, s) do { } while (0 && (c))#define rcu_sleep_check () do { } while (0) -#define lockdep_assert_in_rcu_read_lock () do { } while (0) -#define lockdep_assert_in_rcu_read_lock_bh () do { } while (0) -#define lockdep_assert_in_rcu_read_lock_sched () do { } while (0) -#define lockdep_assert_in_rcu_reader () do { } while (0) +#define lockdep_assert_in_rcu_read_lock () __assert_shared_cap(RCU) +#define lockdep_assert_in_rcu_read_lock_bh () __assert_shared_cap(RCU_BH) +#define lockdep_assert_in_rcu_read_lock_sched () __assert_shared_cap(RCU_SCHED) +#define lockdep_assert_in_rcu_reader () __assert_shared_cap(RCU)#endif /* #else #ifdef CONFIG_PROVE_RCU */ @@ -512,11 +523,11 @@ static inline bool lockdep_assert_rcu_helper(bool c)#endif /* #else #ifdef __CHECKER__ */ #define __unrcu_pointer (p, local) \ -({ \ +capability_unsafe( \ typeof(*p) *local = (typeof(*p) *__force)(p); \ rcu_check_sparse(p, __rcu); \ ((typeof(*p) __force __kernel *)(local)); \ -}) +)/** * unrcu_pointer - mark a pointer as not being RCU protected * @p: pointer needing to lose its __rcu property @@ -592,7 +603,7 @@ static inline bool lockdep_assert_rcu_helper(bool c)* other macros that it invokes. */ #define rcu_assign_pointer (p, v) \ -do { \ +capability_unsafe( \ uintptr_t _r_a_p__v = (uintptr_t)(v); \ rcu_check_sparse(p, __rcu); \ \@@ -600,7 +611,7 @@ do { \WRITE_ONCE ((p), (typeof (p))(_r_a_p__v)); \ else \ smp_store_release(&p, RCU_INITIALIZER((typeof(p))_r_a_p__v)); \ -}while (0 ) +)/** * rcu_replace_pointer() - replace an RCU pointer, returning its old value @@ -843,9 +854,10 @@ do { \* only when acquiring spinlocks that are subject to priority inheritance. */ static __always_inline void rcu_read_lock(void) +__acquires_shared (RCU ) {__rcu_read_lock (); -__acquire (RCU ); +__acquire_shared (RCU );rcu_lock_acquire (&rcu_lock_map);RCU_LOCKDEP_WARN (!rcu_is_watching (),"rcu_read_lock() used illegally while idle" );@@ -874,11 +886,12 @@ static __always_inline void rcu_read_lock(void)* See rcu_read_lock() for more information. */ static inline void rcu_read_unlock (void ) +__releases_shared (RCU ) {RCU_LOCKDEP_WARN (!rcu_is_watching (),"rcu_read_unlock() used illegally while idle" );rcu_lock_release (&rcu_lock_map);/* Keep acq info for rls diags. */ -__release (RCU ); +__release_shared (RCU );__rcu_read_unlock (); }@@ -897,9 +910,11 @@ static inline void rcu_read_unlock(void)* was invoked from some other task. */ static inline void rcu_read_lock_bh (void ) +__acquires_shared (RCU )__acquires_shared (RCU_BH ) {local_bh_disable (); -__acquire (RCU_BH ); +__acquire_shared (RCU ); +__acquire_shared (RCU_BH );rcu_lock_acquire (&rcu_bh_lock_map);RCU_LOCKDEP_WARN (!rcu_is_watching (),"rcu_read_lock_bh() used illegally while idle" );@@ -911,11 +926,13 @@ static inline void rcu_read_lock_bh(void)* See rcu_read_lock_bh() for more information. */ static inline void rcu_read_unlock_bh (void ) +__releases_shared (RCU )__releases_shared (RCU_BH ) {RCU_LOCKDEP_WARN (!rcu_is_watching (),"rcu_read_unlock_bh() used illegally while idle" );rcu_lock_release (&rcu_bh_lock_map); -__release (RCU_BH ); +__release_shared (RCU_BH ); +__release_shared (RCU );local_bh_enable (); }@@ -935,9 +952,11 @@ static inline void rcu_read_unlock_bh(void)* rcu_read_lock_sched() was invoked from an NMI handler. */ static inline void rcu_read_lock_sched (void ) +__acquires_shared (RCU )__acquires_shared (RCU_SCHED ) {preempt_disable (); -__acquire (RCU_SCHED ); +__acquire_shared (RCU ); +__acquire_shared (RCU_SCHED );rcu_lock_acquire (&rcu_sched_lock_map);RCU_LOCKDEP_WARN (!rcu_is_watching (),"rcu_read_lock_sched() used illegally while idle" );@@ -945,9 +964,11 @@ static inline void rcu_read_lock_sched(void)/* Used by lockdep and tracing: cannot be traced, cannot call lockdep. */ static inline notrace void rcu_read_lock_sched_notrace(void) +__acquires_shared (RCU )__acquires_shared (RCU_SCHED ) {preempt_disable_notrace (); -__acquire (RCU_SCHED ); +__acquire_shared (RCU ); +__acquire_shared (RCU_SCHED ); }/** @@ -956,18 +977,22 @@ static inline notrace void rcu_read_lock_sched_notrace(void)* See rcu_read_lock_sched() for more information. */ static inline void rcu_read_unlock_sched (void ) +__releases_shared (RCU )__releases_shared (RCU_SCHED ) {RCU_LOCKDEP_WARN (!rcu_is_watching (),"rcu_read_unlock_sched() used illegally while idle" );rcu_lock_release (&rcu_sched_lock_map); -__release (RCU_SCHED ); +__release_shared (RCU_SCHED ); +__release_shared (RCU );preempt_enable (); }/* Used by lockdep and tracing: cannot be traced, cannot call lockdep. */ static inline notrace void rcu_read_unlock_sched_notrace(void) +__releases_shared (RCU )__releases_shared (RCU_SCHED ) { -__release (RCU_SCHED ); +__release_shared (RCU_SCHED ); +__release_shared (RCU );preempt_enable_notrace (); }@@ -1010,10 +1035,10 @@ static inline notrace void rcu_read_unlock_sched_notrace(void)* ordering guarantees for either the CPU or the compiler. */ #define RCU_INIT_POINTER (p, v) \ - do { \ + capability_unsafe( \ rcu_check_sparse(p, __rcu); \ WRITE_ONCE(p, RCU_INITIALIZER(v)); \ - } while (0) + )/** * RCU_POINTER_INITIALIZER() - statically initialize an RCU protected pointer @@ -1172,4 +1197,6 @@ DEFINE_LOCK_GUARD_0(rcu, } while (0), rcu_read_unlock()) +DECLARE_LOCK_GUARD_0_ATTRS (rcu,__acquires_shared (RCU ),__releases_shared (RCU )); +#endif /* __LINUX_RCUPDATE_H */ lib/test_capability-analysis.c +68
-0@@ -7,6 +7,7 @@#include <linux/bit_spinlock.h> #include <linux/build_bug.h> #include <linux/mutex.h> +#include <linux/rcupdate.h> #include <linux/seqlock.h> #include <linux/spinlock.h> @@ -277,3 +278,70 @@ static void __used test_bit_spin_lock(struct test_bit_spinlock_data *d)bit_spin_unlock (3 , &d->bits); } } + +/* +* Test that we can mark a variable guarded by RCU, and we can dereference and +* write to the pointer with RCU's primitives. +*/ +struct test_rcu_data { +long __rcu_guarded *data; +}; + +static void __used test_rcu_guarded_reader (struct test_rcu_data *d) +{ +rcu_read_lock (); + (void )rcu_dereference (d->data); +rcu_read_unlock (); + +rcu_read_lock_bh (); + (void )rcu_dereference (d->data); +rcu_read_unlock_bh (); + +rcu_read_lock_sched (); + (void )rcu_dereference (d->data); +rcu_read_unlock_sched (); +} + +static void __used test_rcu_guard (struct test_rcu_data *d) +{ +guard (rcu)(); + (void )rcu_dereference (d->data); +} + +static void __used test_rcu_guarded_updater (struct test_rcu_data *d) +{ +rcu_assign_pointer (d->data,NULL ); +RCU_INIT_POINTER (d->data,NULL ); + (void )unrcu_pointer (d->data); +} + +static void wants_rcu_held (void )__must_hold_shared (RCU ) { } +static void wants_rcu_held_bh (void )__must_hold_shared (RCU_BH ) { } +static void wants_rcu_held_sched (void )__must_hold_shared (RCU_SCHED ) { } + +static void __used test_rcu_lock_variants (void ) +{ +rcu_read_lock (); +wants_rcu_held (); +rcu_read_unlock (); + +rcu_read_lock_bh (); +wants_rcu_held_bh (); +rcu_read_unlock_bh (); + +rcu_read_lock_sched (); +wants_rcu_held_sched (); +rcu_read_unlock_sched (); +} + +static void __used test_rcu_assert_variants (void ) +{ +lockdep_assert_in_rcu_read_lock (); +wants_rcu_held (); + +lockdep_assert_in_rcu_read_lock_bh (); +wants_rcu_held_bh (); + +lockdep_assert_in_rcu_read_lock_sched (); +wants_rcu_held_sched (); +}Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRe: [PATCH RFC 15/24] rcu: Support Clang's capability analysis
Quoted 5 lines · Paul E. McKenney
Quoted 3 lines · Marco Elver
Improve the existing annotations to properly support Clang's capability analysis.
Quoted 7 lines
Quoted 4 lines
The old annotations distinguished between RCU,
RCU_BH, andRCU_SCHED. However, it does not make sense to acquirercu_read_lock_bh()afterrcu_read_lock()- annotate the_bh()and_sched()variants to also acquire 'RCU', so that Clang (and also Sparse) can warn about it.You lost me on this one. What breaks if
rcu_read_lock_bh()is invoked whilercu_read_lock()is in effect?
I thought something like this does not make sense:
rcu_read_lock_bh(); .. rcu_read_lock(); .. rcu_read_unlock(); .. rcu_read_unlock_bh();
However, the inverse may well be something we might find somewhere in the kernel?
Another problem was that if we want to indicate that "RCU" read lock is held, then we should just be able to write "__must_hold_shared(RCU)", and it shouldn't matter if rcu_read_lock() or rcu_read_lock_bh() was used. Previously each of them acquired their own capability "RCU" and "RCU_BH" respectively. But rather, we're dealing with one acquiring a superset of the other, and expressing that is also what I attempted to solve.
Let me rethink this...
Thanks,
-- Marco
Paul E. McKenney
maintainerRe: [PATCH RFC 15/24] rcu: Support Clang's capability analysis
Quoted 16 lines · Marco Elver
Quoted 5 lines · Paul E. McKenney
Quoted 3 lines · Marco Elver
Improve the existing annotations to properly support Clang's capability analysis.
Quoted 7 lines
Quoted 4 lines
The old annotations distinguished between RCU,
RCU_BH, andRCU_SCHED. However, it does not make sense to acquirercu_read_lock_bh()afterrcu_read_lock()- annotate the_bh()and_sched()variants to also acquire 'RCU', so that Clang (and also Sparse) can warn about it.You lost me on this one. What breaks if
rcu_read_lock_bh()is invoked whilercu_read_lock()is in effect?I thought something like this does not make sense:
Quoted 7 lines
rcu_read_lock_bh(); .. rcu_read_lock(); .. rcu_read_unlock(); .. rcu_read_unlock_bh();
If you have the choice, it is often better to do the rcu_read_lock() first and the rcu_read_lock_bh() second.
Quoted 2 lines
However, the inverse may well be something we might find somewhere in the kernel?
Suppose that one function walks an RCU-protected list, calling some function from some other subsystem on each element. Suppose that each element has another RCU protected list.
It would be good if the two subsystems could just choose their desired flavor of RCU reader, without having to know about each other.
Quoted 8 lines
Another problem was that if we want to indicate that "RCU" read lock is held, then we should just be able to write "
__must_hold_shared(RCU)", and it shouldn't matter ifrcu_read_lock()orrcu_read_lock_bh()was used. Previously each of them acquired their own capability "RCU" and "RCU_BH" respectively. But rather, we're dealing with one acquiring a superset of the other, and expressing that is also what I attempted to solve.
Let me rethink this...
Would it work to have just one sort of RCU reader, relying on a separate BH-disable capability for the additional semantics of rcu_read_lock_bh()?
Thanx, Paul
Marco Elver
maintainerRe: [PATCH RFC 15/24] rcu: Support Clang's capability analysis
On Thu, 20 Feb 2025 at 23:36, Paul E. McKenney <[email protected]> wrote:
[...]
Quoted 6 lines
Suppose that one function walks an RCU-protected list, calling some function from some other subsystem on each element. Suppose that each element has another RCU protected list.
It would be good if the two subsystems could just choose their desired flavor of RCU reader, without having to know about each other.
That's what I figured might be the case - thanks for clarifying.
Quoted 11 lines
Quoted 8 lines
Another problem was that if we want to indicate that "RCU" read lock is held, then we should just be able to write "
__must_hold_shared(RCU)", and it shouldn't matter ifrcu_read_lock()orrcu_read_lock_bh()was used. Previously each of them acquired their own capability "RCU" and "RCU_BH" respectively. But rather, we're dealing with one acquiring a superset of the other, and expressing that is also what I attempted to solve.
Let me rethink this...Would it work to have just one sort of RCU reader, relying on a separate BH-disable capability for the additional semantics of
rcu_read_lock_bh()?
That's what I've tried with this patch (rcu_read_lock_bh() also acquires "RCU", on top of "RCU_BH"). I need to add a re-entrancy test, and make sure it doesn't complain about that. At a later stage we might also want to add more general "BH" and "IRQ" capabilities to denote they're disabled when held, but that'd overcomplicate the first version of this series.
Thanks,
-- Marco
Paul E. McKenney
maintainerRe: [PATCH RFC 15/24] rcu: Support Clang's capability analysis
Quoted 23 lines · Marco Elver
On Thu, 20 Feb 2025 at 23:36, Paul E. McKenney <[email protected]> wrote:
[...]Quoted 6 lines
Suppose that one function walks an RCU-protected list, calling some function from some other subsystem on each element. Suppose that each element has another RCU protected list.
It would be good if the two subsystems could just choose their desired flavor of RCU reader, without having to know about each other.
That's what I figured might be the case - thanks for clarifying.
Quoted 11 lines
Quoted 8 lines
Another problem was that if we want to indicate that "RCU" read lock is held, then we should just be able to write "
__must_hold_shared(RCU)", and it shouldn't matter ifrcu_read_lock()orrcu_read_lock_bh()was used. Previously each of them acquired their own capability "RCU" and "RCU_BH" respectively. But rather, we're dealing with one acquiring a superset of the other, and expressing that is also what I attempted to solve.
Let me rethink this...Would it work to have just one sort of RCU reader, relying on a separate BH-disable capability for the additional semantics of
rcu_read_lock_bh()?
Quoted 6 lines
That's what I've tried with this patch (
rcu_read_lock_bh()also acquires "RCU", on top of "RCU_BH"). I need to add a re-entrancy test, and make sure it doesn't complain about that. At a later stage we might also want to add more general "BH" and "IRQ" capabilities to denote they're disabled when held, but that'd overcomplicate the first version of this series.
Fair enough! Then would it work to just do "RCU" now, and ad the "BH" and "IRQ" when those capabilities are added?
Thanx, Paul
Marco Elver
maintainerRe: [PATCH RFC 15/24] rcu: Support Clang's capability analysis
On Thu, Feb 20, 2025 at 05:26PM -0800, Paul E. McKenney wrote:
[...]
Quoted 9 lines
Quoted 6 lines
That's what I've tried with this patch (
rcu_read_lock_bh()also acquires "RCU", on top of "RCU_BH"). I need to add a re-entrancy test, and make sure it doesn't complain about that. At a later stage we might also want to add more general "BH" and "IRQ" capabilities to denote they're disabled when held, but that'd overcomplicate the first version of this series.Fair enough! Then would it work to just do "RCU" now, and ad the "BH" and "IRQ" when those capabilities are added?
I tried if this kind of re-entrant locking works - a test like this:
| --- a/lib/test_capability-analysis.c
| +++ b/lib/test_capability-analysis.c
| @@ -370,6 +370,15 @@ static void __used test_rcu_guarded_reader(struct test_rcu_data *d)
| rcu_read_unlock_sched();
| }
|
| +static void __used test_rcu_reentrancy(struct test_rcu_data *d)
| +{
| + rcu_read_lock();
| + rcu_read_lock_bh();
| + (void)rcu_dereference(d->data);
| + rcu_read_unlock_bh();
| + rcu_read_unlock();
| +}| $ make lib/test_capability-analysis.o | DESCEND objtool | CC arch/x86/kernel/asm-offsets.s | INSTALL libsubcmd_headers | CALL scripts/checksyscalls.sh | CC lib/test_capability-analysis.o | lib/test_capability-analysis.c:376:2: error: acquiring __capability_RCU 'RCU' that is already held [-Werror,-Wthread-safety-analysis] | 376 | rcu_read_lock_bh(); | | ^ | lib/test_capability-analysis.c:375:2: note: __capability_RCU acquired here | 375 | rcu_read_lock(); | | ^ | lib/test_capability-analysis.c:379:2: error: releasing __capability_RCU 'RCU' that was not held [-Werror,-Wthread-safety-analysis] | 379 | rcu_read_unlock(); | | ^ | lib/test_capability-analysis.c:378:2: note: __capability_RCU released here | 378 | rcu_read_unlock_bh(); | | ^ | 2 errors generated. | make[3]: *** [scripts/Makefile.build:207: lib/test_capability-analysis.o] Error 1 | make[2]: *** [scripts/Makefile.build:465: lib] Error 2
... unfortunately even for shared locks, the compiler does not like re-entrancy yet. It's not yet supported, and to fix that I'd have to go and implement that in Clang first before coming back to this.
I see 2 options for now:
a. Accepting the limitation that doing a rcu_read_lock() (and
variants) while the RCU read lock is already held in the same function will result in a false positive warning (like above). Cases like that will need to disable the analysis for that piece of code.
b. Make the compiler not warn about unbalanced rcu_read_lock/unlock(),
but instead just help enforce a rcu_read_lock() was issued somewhere in the function before an RCU-guarded access.
Option (b) is obviously weaker than (a), but avoids the false positives while accepting more false negatives.
For all the code that I have already tested this on I observed no false positives, so I'd go with (a), but I'm also fine with the weaker checking for now until the compiler gains re-entrancy support.
Preferences?
Thanks,
-- Marco
Paul E. McKenney
maintainerRe: [PATCH RFC 15/24] rcu: Support Clang's capability analysis
Quoted 43 lines · Marco Elver
On Thu, Feb 20, 2025 at 05:26PM -0800, Paul E. McKenney wrote:
[...]Quoted 9 lines
Quoted 6 lines
That's what I've tried with this patch (
rcu_read_lock_bh()also acquires "RCU", on top of "RCU_BH"). I need to add a re-entrancy test, and make sure it doesn't complain about that. At a later stage we might also want to add more general "BH" and "IRQ" capabilities to denote they're disabled when held, but that'd overcomplicate the first version of this series.Fair enough! Then would it work to just do "RCU" now, and ad the "BH" and "IRQ" when those capabilities are added?
I tried if this kind of re-entrant locking works - a test like this:
| --- a/lib/test_capability-analysis.c | +++ b/lib/test_capability-analysis.c | @@ -370,6 +370,15 @@ static void __used test_rcu_guarded_reader(struct test_rcu_data *d) | rcu_read_unlock_sched(); | } | | +static void __used test_rcu_reentrancy(struct test_rcu_data *d) | +{ | + rcu_read_lock(); | + rcu_read_lock_bh(); | + (void)rcu_dereference(d->data); | + rcu_read_unlock_bh(); | + rcu_read_unlock(); | +}| $ make lib/test_capability-analysis.o | DESCEND objtool | CC arch/x86/kernel/asm-offsets.s | INSTALL libsubcmd_headers | CALL scripts/checksyscalls.sh | CC lib/test_capability-analysis.o | lib/test_capability-analysis.c:376:2: error: acquiring __capability_RCU 'RCU' that is already held [-Werror,-Wthread-safety-analysis] | 376 | rcu_read_lock_bh(); | | ^ | lib/test_capability-analysis.c:375:2: note: __capability_RCU acquired here | 375 | rcu_read_lock(); | | ^ | lib/test_capability-analysis.c:379:2: error: releasing __capability_RCU 'RCU' that was not held [-Werror,-Wthread-safety-analysis]
Quoted 8 lines
| 379 | rcu_read_unlock(); | | ^ | lib/test_capability-analysis.c:378:2: note: __capability_RCU released here | 378 | rcu_read_unlock_bh(); | | ^ | 2 errors generated. | make[3]: *** [scripts/Makefile.build:207: lib/test_capability-analysis.o] Error 1 | make[2]: *** [scripts/Makefile.build:465: lib] Error 2
I was hoping! Ah well... ;-)
Quoted 3 lines
... unfortunately even for shared locks, the compiler does not like re-entrancy yet. It's not yet supported, and to fix that I'd have to go and implement that in Clang first before coming back to this.
This would be needed for some types of reader-writer locks, and also for reference counting, so here is hoping that such support is forthcoming sooner rather than later.
Quoted 11 lines
I see 2 options for now:
a. Accepting the limitation that doing a
rcu_read_lock()(and
variants) while the RCU read lock is already held in the same function will result in a false positive warning (like above). Cases like that will need to disable the analysis for that piece of code.b. Make the compiler not warn about unbalanced rcu_read_lock/unlock(),
but instead just help enforce arcu_read_lock()was issued somewhere in the function before an RCU-guarded access.
Quoted 8 lines
Option (b) is obviously weaker than (a), but avoids the false positives while accepting more false negatives.
For all the code that I have already tested this on I observed no false positives, so I'd go with (a), but I'm also fine with the weaker checking for now until the compiler gains re-entrancy support.
Preferences?
Whichever one provides the best checking without false positives. Which sounds to me like (a) unless and until false positives crop up, in which case (b). Which looks to be where you were going anyway. ;-)
Thanx, Paul
Peter Zijlstra
maintainerRe: [PATCH RFC 15/24] rcu: Support Clang's capability analysis
Quoted 7 lines · Paul E. McKenney
Quoted 3 lines
... unfortunately even for shared locks, the compiler does not like re-entrancy yet. It's not yet supported, and to fix that I'd have to go and implement that in Clang first before coming back to this.
This would be needed for some types of reader-writer locks, and also for reference counting, so here is hoping that such support is forthcoming sooner rather than later.
Right, so I read the clang documentation for this feature the other day, and my take away was that this was all really primitive and lots of work will need to go into making this more capable before we can cover much of the more interesting things we do in the kernel.
Notably the whole guarded_by member annotations, which are very cool in concept, are very primitive in practise and will need much extensions.
To that effect, and because this is basically a static analysis pass with no codegen implications, I would suggest that we keep the whole feature limited to the very latest clang version for now and don't bother supporting older versions at all.
Marco Elver
maintainerRe: [PATCH RFC 15/24] rcu: Support Clang's capability analysis
Quoted 11 lines · Peter Zijlstra
Quoted 7 lines · Paul E. McKenney
Quoted 3 lines
... unfortunately even for shared locks, the compiler does not like re-entrancy yet. It's not yet supported, and to fix that I'd have to go and implement that in Clang first before coming back to this.
This would be needed for some types of reader-writer locks, and also for reference counting, so here is hoping that such support is forthcoming sooner rather than later.
Quoted 7 lines
Right, so I read the clang documentation for this feature the other day, and my take away was that this was all really primitive and lots of work will need to go into making this more capable before we can cover much of the more interesting things we do in the kernel.
Notably the whole
guarded_bymember annotations, which are very cool in concept, are very primitive in practise and will need much extensions.
I have one extension in flight:
https://github.com/llvm/llvm-project/pull/127396 - it'll improve coverage for pointer passing of guarded_by members.
Anything else you see as urgent? Re-entrant locks support a deal breaker?
But yes, a lot of complex locking patterns will not easily be expressible right away.
Quoted 4 lines
To that effect, and because this is basically a static analysis pass with no codegen implications, I would suggest that we keep the whole feature limited to the very latest clang version for now and don't bother supporting older versions at all.
Along those lines, in an upcoming v2, I'm planning to bump it up to Clang 20+ because that version introduced a reasonable way to ignore warnings in not-yet-annotated headers:
https://git.kernel.org/pub/scm/linux/kernel/git/melver/linux.git/commit/?h=cap-analysis/dev&id=2432a39eae8197f5058c578430bd1906c18480c3
Peter Zijlstra
maintainerRe: [PATCH RFC 15/24] rcu: Support Clang's capability analysis
Quoted 1 line · Marco Elver
Anything else you see as urgent? Re-entrant locks support a deal breaker?
Most actual locks are not recursive -- RCU being the big exception here.
As to this being deal breakers, I don't think so. We should just start with the bits we can do and chip away at stuff. Raise the LLVM version requirement every time new stuff gets added.
Marco Elver
maintainerRFC PATCH 16/24 srcu: Support Clang's capability analysis
In reply to Marco ElverAdd support for Clang's capability analysis for SRCU.
--- .../dev-tools/capability-analysis.rst | 2 +- include/linux/srcu.h | 61 +++++++++++++------ lib/test_capability-analysis.c | 24 ++++++++ 3 files changed, 66 insertions(+), 21 deletions(-)
Documentation/dev-tools/capability-analysis.rst +1 -1
@@ -86,7 +86,7 @@ Supported Kernel Primitives Currently the following synchronization primitives are supported: `raw_spinlock_t`, `spinlock_t`, `rwlock_t`, `mutex`, `seqlock_t`, -`bit_spinlock`, RCU. +`bit_spinlock`, RCU, SRCU (`srcu_struct`). For capabilities with an initialization function (e.g., `spin_lock_init()`), calling this function on the capability instance before initializing any
include/linux/srcu.h +41 -20
161 lines not on this page · raw message
lib/test_capability-analysis.c +24 -0
@@ -10,6 +10,7 @@#include <linux/rcupdate.h> #include <linux/seqlock.h> #include <linux/spinlock.h> +#include <linux/srcu.h> /* * Test that helper macros work as expected.
@@ -345,3 +346,26 @@ static void __used test_rcu_assert_variants(void)lockdep_assert_in_rcu_read_lock_sched ();wants_rcu_held_sched (); } + +struct test_srcu_data { +struct srcu_struct srcu; +long __rcu_guarded *data; +}; + +static void __used test_srcu (struct test_srcu_data *d) +{ +init_srcu_struct (&d->srcu); + +int idx =srcu_read_lock (&d->srcu); +long *data =srcu_dereference (d->data, &d->srcu); + (void )data; +srcu_read_unlock (&d->srcu, idx); + +rcu_assign_pointer (d->data,NULL ); +} + +static void __used test_srcu_guard (struct test_srcu_data *d) +{ +guard (srcu)(&d->srcu); + (void )srcu_dereference (d->data, &d->srcu); +}
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRFC PATCH 17/24 kref: Add capability-analysis annotations
In reply to Marco ElverMark functions that conditionally acquire the passed lock.
--- include/linux/kref.h | 2 ++ 1 file changed, 2 insertions(+)
include/linux/kref.h +2 -0
@@ -81,6 +81,7 @@ static inline int kref_put(struct kref *kref, void (*release)(struct kref *kref)static inline int kref_put_mutex (struct kref *kref,void (*release)(struct kref *kref),struct mutex *mutex) +__cond_acquires (1 , mutex) {if (refcount_dec_and_mutex_lock (&kref->refcount, mutex)) {release (kref);
@@ -102,6 +103,7 @@ static inline int kref_put_mutex(struct kref *kref,static inline int kref_put_lock (struct kref *kref,void (*release)(struct kref *kref),spinlock_t *lock) +__cond_acquires (1 , lock) {if (refcount_dec_and_lock (&kref->refcount, lock)) {release (kref);
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRFC PATCH 18/24 locking/rwsem: Support Clang's capability analysis
In reply to Marco ElverAdd support for Clang's capability analysis for rw_semaphore.
--- .../dev-tools/capability-analysis.rst | 2 +- include/linux/rwsem.h | 56 +++++++++------- lib/test_capability-analysis.c | 64 +++++++++++++++++++ 3 files changed, 97 insertions(+), 25 deletions(-)
Documentation/dev-tools/capability-analysis.rst +1 -1
@@ -86,7 +86,7 @@ Supported Kernel Primitives Currently the following synchronization primitives are supported: `raw_spinlock_t`, `spinlock_t`, `rwlock_t`, `mutex`, `seqlock_t`, -`bit_spinlock`, RCU, SRCU (`srcu_struct`). +`bit_spinlock`, RCU, SRCU (`srcu_struct`), `rw_semaphore`. For capabilities with an initialization function (e.g., `spin_lock_init()`), calling this function on the capability instance before initializing any
include/linux/rwsem.h +32 -24
158 lines not on this page · raw message
lib/test_capability-analysis.c +64 -0
76 lines not on this page · raw message
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRFC PATCH 19/24 locking/local_lock: Support Clang's capability analysis
In reply to Marco ElverAdd support for Clang's capability analysis for local_lock_t.
--- .../dev-tools/capability-analysis.rst | 2 +- include/linux/local_lock.h | 18 ++++---- include/linux/local_lock_internal.h | 41 ++++++++++++++--- lib/test_capability-analysis.c | 46 +++++++++++++++++++ 4 files changed, 90 insertions(+), 17 deletions(-)
Documentation/dev-tools/capability-analysis.rst +1 -1
@@ -86,7 +86,7 @@ Supported Kernel Primitives Currently the following synchronization primitives are supported: `raw_spinlock_t`, `spinlock_t`, `rwlock_t`, `mutex`, `seqlock_t`, -`bit_spinlock`, RCU, SRCU (`srcu_struct`), `rw_semaphore`. +`bit_spinlock`, RCU, SRCU (`srcu_struct`), `rw_semaphore`, `local_lock_t`. For capabilities with an initialization function (e.g., `spin_lock_init()`), calling this function on the capability instance before initializing any
include/linux/local_lock.h +9 -9
@@ -51,12 +51,12 @@#define local_unlock_irqrestore (lock, flags) \ __local_unlock_irqrestore(lock, flags) -DEFINE_GUARD(local_lock, local_lock_t __percpu*, -local_lock (_T), - local_unlock(_T)) -DEFINE_GUARD(local_lock_irq, local_lock_t __percpu*, -local_lock_irq (_T), - local_unlock_irq(_T)) +DEFINE_LOCK_GUARD_1 (local_lock, local_lock_t__percpu , +local_lock (_T->lock), + local_unlock(_T->lock)) +DEFINE_LOCK_GUARD_1 (local_lock_irq, local_lock_t__percpu , +local_lock_irq (_T->lock), + local_unlock_irq(_T->lock)) DEFINE_LOCK_GUARD_1(local_lock_irqsave, local_lock_t __percpu, local_lock_irqsave(_T->lock, _T->flags), local_unlock_irqrestore(_T->lock, _T->flags),
@@ -68,8 +68,8 @@ DEFINE_LOCK_GUARD_1(local_lock_irqsave, local_lock_t __percpu,#define local_unlock_nested_bh (_lock) \ __local_unlock_nested_bh(_lock) -DEFINE_GUARD(local_lock_nested_bh, local_lock_t __percpu*, -local_lock_nested_bh (_T), - local_unlock_nested_bh(_T)) +DEFINE_LOCK_GUARD_1 (local_lock_nested_bh, local_lock_t__percpu , +local_lock_nested_bh (_T->lock), + local_unlock_nested_bh(_T->lock))#endif
include/linux/local_lock_internal.h +34 -7
137 lines not on this page · raw message
lib/test_capability-analysis.c +46 -0
56 lines not on this page · raw message
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRFC PATCH 20/24 debugfs: Make debugfs_cancellation a capability struct
In reply to Marco ElverWhen compiling include/linux/debugfs.h with CAPABILITY_ANALYSIS enabled, we can see this error:
./include/linux/debugfs.h:239:17: error: use of undeclared identifier 'cancellation'
239 | void __acquires(cancellation)
Move the __acquires(..) attribute after the declaration, so that the compiler can see the cancellation function argument, as well as making struct debugfs_cancellation a real capability to benefit from Clang's capability analysis.
--- include/linux/debugfs.h | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-)
include/linux/debugfs.h +5 -7
@@ -240,18 +240,16 @@ ssize_t debugfs_read_file_str(struct file *file, char __user *user_buf,* @cancel: callback to call * @cancel_data: extra data for the callback to call */ -struct debugfs_cancellation { +struct_with_capability (debugfs_cancellation) {struct list_head list;void (*cancel)(struct dentry *,void *);void *cancel_data; }; -void __acquires(cancellation) -debugfs_enter_cancellation(struct file *file, - struct debugfs_cancellation *cancellation); -void __releases(cancellation) -debugfs_leave_cancellation(struct file *file, - struct debugfs_cancellation *cancellation); +void debugfs_enter_cancellation(struct file *file, + struct debugfs_cancellation *cancellation) __acquires(cancellation); +void debugfs_leave_cancellation(struct file *file, + struct debugfs_cancellation *cancellation) __releases(cancellation);#else
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRFC PATCH 21/24 kfence: Enable capability analysis
In reply to Marco ElverEnable capability analysis for the KFENCE subsystem.
Notable, kfence_handle_page_fault() required minor restructure, which also fixed a subtle race; arguably that function is more readable now.
--- mm/kfence/Makefile | 2 ++ mm/kfence/core.c | 24 +++++++++++++++++------- mm/kfence/kfence.h | 18 ++++++++++++------ mm/kfence/kfence_test.c | 4 ++++ mm/kfence/report.c | 8 ++++++-- 5 files changed, 41 insertions(+), 15 deletions(-)
mm/kfence/Makefile +2 -0
@@ -1,5 +1,7 @@ # SPDX-License-Identifier: GPL-2.0 +CAPABILITY_ANALYSIS := y + obj-y := core.o report.o CFLAGS_kfence_test.o := -fno-omit-frame-pointer -fno-optimize-sibling-calls
mm/kfence/core.c +17 -7
@@ -7,6 +7,8 @@#define pr_fmt (fmt) "kfence: " fmt +disable_capability_analysis (); +#include <linux/atomic.h> #include <linux/bug.h> #include <linux/debugfs.h>
@@ -34,6 +36,8 @@#include <asm/kfence.h> +enable_capability_analysis (); +#include "kfence.h" /* Disables KFENCE on the first warning assuming an irrecoverable error. */
@@ -132,8 +136,8 @@ struct kfence_metadata *kfence_metadata __read_mostly;static struct kfence_metadata *kfence_metadata_init__read_mostly ;/* Freelist with available objects. */ -static struct list_head kfence_freelist =LIST_HEAD_INIT (kfence_freelist); -static DEFINE_RAW_SPINLOCK (kfence_freelist_lock);/* Lock protecting freelist. */ +DEFINE_RAW_SPINLOCK (kfence_freelist_lock);/* Lock protecting freelist. */ +static struct list_head kfence_freelist __var_guarded_by(&kfence_freelist_lock) = LIST_HEAD_INIT(kfence_freelist);/* * The static key to set up a KFENCE allocation; or if static keys are not used
@@ -253,6 +257,7 @@ static bool kfence_unprotect(unsigned long addr) }static inline unsigned long metadata_to_pageaddr (const struct kfence_metadata *meta) + __must_hold(&meta->lock) {unsigned long offset = (meta - kfence_metadata +1 ) *PAGE_SIZE *2 ;unsigned long pageaddr = (unsigned long )&__kfence_pool[offset];
@@ -288,6 +293,7 @@ static inline bool kfence_obj_allocated(const struct kfence_metadata *meta) static noinline voidmetadata_update_state (struct kfence_metadata *meta,enum kfence_object_state next,unsigned long *stack_entries,size_t num_stack_entries) + __must_hold(&meta->lock) {struct kfence_track *track = next ==KFENCE_OBJECT_ALLOCATED ? &meta->alloc_track : &meta->free_track;
@@ -485,7 +491,7 @@ static void *kfence_guarded_alloc(struct kmem_cache *cache, size_t size, gfp_t galloc_covered_add (alloc_stack_hash,1 );/* Set required slab fields. */ - slab =virt_to_slab ((void *)meta->addr); + slab =virt_to_slab (addr); slab->slab_cache = cache; slab->objects =1 ;
@@ -514,6 +520,7 @@ static void *kfence_guarded_alloc(struct kmem_cache *cache, size_t size, gfp_t gstatic void kfence_guarded_free (void *addr,struct kfence_metadata *meta,bool zombie) {struct kcsan_scoped_access assert_page_exclusive; +u32 alloc_stack_hash;unsigned long flags;bool init;
@@ -546,9 +553,10 @@ static void kfence_guarded_free(void *addr, struct kfence_metadata *meta, bool z/* Mark the object as freed. */ metadata_update_state (meta,KFENCE_OBJECT_FREED ,NULL ,0 ); init =slab_want_init_on_free (meta->cache); + alloc_stack_hash = meta->alloc_stack_hash;raw_spin_unlock_irqrestore (&meta->lock, flags); -alloc_covered_add (meta->alloc_stack_hash,-1 ); +alloc_covered_add (alloc_stack_hash,-1 );/* Check canary bytes for memory corruption. */ check_canary (meta);
@@ -593,6 +601,7 @@ static void rcu_guarded_free(struct rcu_head *h)* which partial initialization succeeded. */ static unsigned long kfence_init_pool (void ) + __no_capability_analysis {unsigned long addr;struct page *pages;
@@ -1192,6 +1201,7 @@ bool kfence_handle_page_fault(unsigned long addr, bool is_write, struct pt_regs {const int page_index = (addr - (unsigned long )__kfence_pool) /PAGE_SIZE ;struct kfence_metadata *to_report =NULL ; +unsigned long unprotected_page =0 ;enum kfence_error_type error_type;unsigned long flags;
@@ -1225,9 +1235,8 @@ bool kfence_handle_page_fault(unsigned long addr, bool is_write, struct pt_regsif (!to_report) goto out; -raw_spin_lock_irqsave (&to_report->lock, flags); - to_report->unprotected_page = addr; error_type =KFENCE_ERROR_OOB ; + unprotected_page = addr;/* * If the object was freed before we took the look we can still
@@ -1239,7 +1248,6 @@ bool kfence_handle_page_fault(unsigned long addr, bool is_write, struct pt_regsif (!to_report) goto out; -raw_spin_lock_irqsave (&to_report->lock, flags); error_type =KFENCE_ERROR_UAF ;/* * We may race with __kfence_alloc(), and it is possible that a
@@ -1251,6 +1259,8 @@ bool kfence_handle_page_fault(unsigned long addr, bool is_write, struct pt_regs out:if (to_report) { +raw_spin_lock_irqsave (&to_report->lock, flags); + to_report->unprotected_page = unprotected_page;kfence_report_error (addr, is_write, regs, to_report, error_type);raw_spin_unlock_irqrestore (&to_report->lock, flags); }else {
mm/kfence/kfence.h +12 -6
@@ -9,6 +9,8 @@#ifndef MM_KFENCE_KFENCE_H #define MM_KFENCE_KFENCE_H +disable_capability_analysis (); +#include <linux/mm.h> #include <linux/slab.h> #include <linux/spinlock.h>
@@ -16,6 +18,8 @@#include "../slab.h" /* for struct kmem_cache */ +enable_capability_analysis (); +/* * Get the canary byte pattern for @addr. Use a pattern that varies based on the * lower 3 bits of the address, to detect memory corruptions with higher
@@ -34,6 +38,8 @@/* Maximum stack depth for reports. */ #define KFENCE_STACK_DEPTH 64 +extern raw_spinlock_t kfence_freelist_lock; +/* KFENCE object states. */ enum kfence_object_state {KFENCE_OBJECT_UNUSED ,/* Object is unused. */
@@ -53,7 +59,7 @@ struct kfence_track {/* KFENCE metadata per guarded allocation. */ struct kfence_metadata { -struct list_head list;/* Freelist node; access under kfence_freelist_lock. */ + struct list_head list __var_guarded_by(&kfence_freelist_lock); /* Freelist node. */struct rcu_head rcu_head;/* For delayed freeing. */ /*
@@ -91,13 +97,13 @@ struct kfence_metadata {* In case of an invalid access, the page that was unprotected; we * optimistically only store one address. */ -unsigned long unprotected_page; + unsigned long unprotected_page __var_guarded_by(&lock);/* Allocation and free stack information. */ -struct kfence_track alloc_track; -struct kfence_track free_track; + struct kfence_track alloc_track __var_guarded_by(&lock); + struct kfence_track free_track __var_guarded_by(&lock);/* For updating alloc_covered on frees. */ -u32 alloc_stack_hash; + u32 alloc_stack_hash __var_guarded_by(&lock);#ifdef CONFIG_MEMCG struct slabobj_ext obj_exts;#endif
@@ -141,6 +147,6 @@ enum kfence_error_type {void kfence_report_error (unsigned long address,bool is_write,struct pt_regs *regs,const struct kfence_metadata *meta,enum kfence_error_type type); -void kfence_print_object (struct seq_file *seq,const struct kfence_metadata *meta); +void kfence_print_object(struct seq_file *seq, const struct kfence_metadata *meta) __must_hold(&meta->lock);#endif /* MM_KFENCE_KFENCE_H */
mm/kfence/kfence_test.c +4 -0
@@ -11,6 +11,8 @@* Marco Elver <[email protected]> */ +disable_capability_analysis (); +#include <kunit/test.h> #include <linux/jiffies.h> #include <linux/kernel.h>
@@ -26,6 +28,8 @@#include <asm/kfence.h> +enable_capability_analysis (); +#include "kfence.h" /* May be overridden by <asm/kfence.h>. */
mm/kfence/report.c +6 -2
@@ -5,6 +5,8 @@* Copyright (C) 2020, Google LLC. */ +disable_capability_analysis (); +#include <linux/stdarg.h> #include <linux/kernel.h>
@@ -22,6 +24,8 @@#include <asm/kfence.h> +enable_capability_analysis (); +#include "kfence.h" /* May be overridden by <asm/kfence.h>. */
@@ -106,6 +110,7 @@ static int get_stack_skipnr(const unsigned long stack_entries[], int num_entriesstatic void kfence_print_stack (struct seq_file *seq,const struct kfence_metadata *meta,bool show_alloc) + __must_hold(&meta->lock) {const struct kfence_track *track = show_alloc ? &meta->alloc_track : &meta->free_track;u64 ts_sec = track->ts_nsec;
@@ -207,8 +212,6 @@ void kfence_report_error(unsigned long address, bool is_write, struct pt_regs *rif (WARN_ON (type !=KFENCE_ERROR_INVALID && !meta))return ; -if (meta) -lockdep_assert_held (&meta->lock);/* * Because we may generate reports in printk-unfriendly parts of the * kernel, such as scheduler code, the use of printk() could deadlock.
@@ -263,6 +266,7 @@ void kfence_report_error(unsigned long address, bool is_write, struct pt_regs *rstack_trace_print (stack_entries + skipnr, num_stack_entries - skipnr,0 );if (meta) { +lockdep_assert_held (&meta->lock);pr_err ("\n" );kfence_print_object (NULL , meta); }
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRFC PATCH 22/24 kcov: Enable capability analysis
In reply to Marco ElverEnable capability analysis for the KCOV subsystem.
--- kernel/Makefile | 2 ++ kernel/kcov.c | 40 +++++++++++++++++++++++++++++----------- 2 files changed, 31 insertions(+), 11 deletions(-)
kernel/Makefile +2 -0
@@ -39,6 +39,8 @@ KASAN_SANITIZE_kcov.o := n KCSAN_SANITIZE_kcov.o := n UBSAN_SANITIZE_kcov.o := n KMSAN_SANITIZE_kcov.o := n + +CAPABILITY_ANALYSIS_kcov.o := y CFLAGS_kcov.o := $(call cc-option, -fno-conserve-stack) -fno-stack-protector obj-y += sched/
kernel/kcov.c +29 -11
@@ -1,6 +1,8 @@// SPDX-License-Identifier: GPL-2.0 #define pr_fmt (fmt) "kcov: " fmt +disable_capability_analysis (); +#define DISABLE_BRANCH_PROFILING #include <linux/atomic.h> #include <linux/compiler.h>
@@ -27,6 +29,8 @@#include <linux/log2.h> #include <asm/setup.h> +enable_capability_analysis (); +#define kcov_debug (fmt, ...) pr_debug("%s: " fmt, __func__, ##__VA_ARGS__)/* Number of 64-bit words written per one comparison: */
@@ -55,13 +59,13 @@ struct kcov {refcount_t refcount;/* The lock protects mode, size, area and t. */ spinlock_t lock; -enum kcov_mode mode; + enum kcov_mode mode __var_guarded_by(&lock);/* Size of arena (in long's). */ -unsigned int size; + unsigned int size __var_guarded_by(&lock);/* Coverage buffer shared with user space. */ -void *area; + void *area __var_guarded_by(&lock);/* Task for which we collect coverage, or NULL. */ -struct task_struct *t; + struct task_struct *t __var_guarded_by(&lock);/* Collecting coverage from remote (background) threads. */ bool remote;/* Size of remote area (in long's). */
@@ -391,6 +395,7 @@ void kcov_task_init(struct task_struct *t) }static void kcov_reset (struct kcov *kcov) + __must_hold(&kcov->lock) { kcov->t =NULL ; kcov->mode =KCOV_MODE_INIT ;
@@ -400,6 +405,7 @@ static void kcov_reset(struct kcov *kcov) }static void kcov_remote_reset (struct kcov *kcov) + __must_hold(&kcov->lock) {int bkt;struct kcov_remote *remote;
@@ -419,6 +425,7 @@ static void kcov_remote_reset(struct kcov *kcov) }static void kcov_disable (struct task_struct *t,struct kcov *kcov) + __must_hold(&kcov->lock) {kcov_task_reset (t);if (kcov->remote)
@@ -435,8 +442,11 @@ static void kcov_get(struct kcov *kcov)static void kcov_put (struct kcov *kcov) {if (refcount_dec_and_test (&kcov->refcount)) { -kcov_remote_reset (kcov); -vfree (kcov->area); +/* Capability-safety: no references left, object being destroyed. */ +capability_unsafe ( + kcov_remote_reset(kcov); +vfree (kcov->area); + );kfree (kcov); } }
@@ -491,6 +501,7 @@ static int kcov_mmap(struct file *filep, struct vm_area_struct *vma)unsigned long size, off;struct page *page;unsigned long flags; +unsigned long *area;spin_lock_irqsave (&kcov->lock, flags); size = kcov->size *sizeof (unsigned long );
@@ -499,10 +510,11 @@ static int kcov_mmap(struct file *filep, struct vm_area_struct *vma) res = -EINVAL ; goto exit; } + area = kcov->area;spin_unlock_irqrestore (&kcov->lock, flags);vm_flags_set (vma,VM_DONTEXPAND );for (off =0 ; off < size; off +=PAGE_SIZE ) { - page =vmalloc_to_page (kcov->area + off); + page =vmalloc_to_page (area + off); res =vm_insert_page (vma, vma->vm_start + off, page);if (res) {pr_warn_once ("kcov: vm_insert_page() failed\n" );
@@ -522,10 +534,10 @@ static int kcov_open(struct inode *inode, struct file *filep) kcov =kzalloc (sizeof (*kcov),GFP_KERNEL );if (!kcov)return -ENOMEM ; +spin_lock_init (&kcov->lock); kcov->mode =KCOV_MODE_DISABLED ; kcov->sequence =1 ;refcount_set (&kcov->refcount,1 ); -spin_lock_init (&kcov->lock); filep->private_data = kcov;return nonseekable_open (inode, filep); }
@@ -556,6 +568,7 @@ static int kcov_get_mode(unsigned long arg)* vmalloc fault handling path is instrumented. */ static void kcov_fault_in_area (struct kcov *kcov) + __must_hold(&kcov->lock) {unsigned long stride =PAGE_SIZE /sizeof (unsigned long );unsigned long *area = kcov->area;
@@ -584,6 +597,7 @@ static inline bool kcov_check_handle(u64 handle, bool common_valid,static int kcov_ioctl_locked (struct kcov *kcov,unsigned int cmd,unsigned long arg) + __must_hold(&kcov->lock) {struct task_struct *t;unsigned long flags, unused;
@@ -814,6 +828,7 @@ static inline bool kcov_mode_enabled(unsigned int mode) }static void kcov_remote_softirq_start (struct task_struct *t) + __must_hold(&kcov_percpu_data.lock) {struct kcov_percpu_data *data =this_cpu_ptr (&kcov_percpu_data);unsigned int mode;
@@ -831,6 +846,7 @@ static void kcov_remote_softirq_start(struct task_struct *t) }static void kcov_remote_softirq_stop (struct task_struct *t) + __must_hold(&kcov_percpu_data.lock) {struct kcov_percpu_data *data =this_cpu_ptr (&kcov_percpu_data);
@@ -896,10 +912,12 @@ void kcov_remote_start(u64 handle)/* Put in kcov_remote_stop(). */ kcov_get (kcov);/* -* Read kcov fields before unlock to prevent races with -* KCOV_DISABLE / kcov_remote_reset(). +* Read kcov fields before unlocking kcov_remote_lock to prevent races +* with KCOV_DISABLE and kcov_remote_reset(); cannot acquire kcov->lock +* here, because it might lead to deadlock given kcov_remote_lock is +* acquired _after_ kcov->lock elsewhere. */ - mode = kcov->mode; + mode =capability_unsafe (kcov->mode); sequence = kcov->sequence;if (in_task ()) { size = kcov->remote_size;
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRFC PATCH 23/24 stackdepot: Enable capability analysis
In reply to Marco ElverEnable capability analysis for stackdepot.
--- lib/Makefile | 1 + lib/stackdepot.c | 24 ++++++++++++++++++------ 2 files changed, 19 insertions(+), 6 deletions(-)
lib/Makefile +1 -0
@@ -270,6 +270,7 @@ obj-$(CONFIG_POLYNOMIAL) += polynomial.o # Prevent the compiler from calling builtins like memcmp() or bcmp() from this # file. CFLAGS_stackdepot.o += -fno-builtin +CAPABILITY_ANALYSIS_stackdepot.o := y obj-$(CONFIG_STACKDEPOT) += stackdepot.o KASAN_SANITIZE_stackdepot.o := n # In particular, instrumenting stackdepot.c with KMSAN will result in infinite
lib/stackdepot.c +18 -6
@@ -14,6 +14,8 @@#define pr_fmt (fmt) "stackdepot: " fmt +disable_capability_analysis (); +#include <linux/debugfs.h> #include <linux/gfp.h> #include <linux/jhash.h>
@@ -36,6 +38,8 @@#include <linux/memblock.h> #include <linux/kasan-enabled.h> +enable_capability_analysis (); +#define DEPOT_POOLS_CAP 8192/* The pool_index is offset by 1 so the first record does not have a 0 handle. */ #define DEPOT_MAX_POOLS \
@@ -61,18 +65,18 @@ static unsigned int stack_bucket_number_order;/* Hash mask for indexing the table. */ static unsigned int stack_hash_mask; +/* The lock must be held when performing pool or freelist modifications. */ +static DEFINE_RAW_SPINLOCK (pool_lock);/* Array of memory regions that store stack records. */ -static void *stack_pools[DEPOT_MAX_POOLS ]; +static void *stack_pools[DEPOT_MAX_POOLS] __var_guarded_by(&pool_lock);/* Newly allocated pool that is not yet added to stack_pools. */ static void *new_pool;/* Number of pools in stack_pools. */ static int pools_num;/* Offset to the unused space in the currently used pool. */ -static size_t pool_offset =DEPOT_POOL_SIZE ; +static size_t pool_offset __var_guarded_by(&pool_lock) = DEPOT_POOL_SIZE;/* Freelist of stack records within stack_pools. */ -static LIST_HEAD (free_stacks); -/* The lock must be held when performing pool or freelist modifications. */ -static DEFINE_RAW_SPINLOCK (pool_lock); +static __var_guarded_by(&pool_lock) LIST_HEAD(free_stacks);/* Statistics counters for debugfs. */ enum depot_counter_id {
@@ -242,6 +246,7 @@ EXPORT_SYMBOL_GPL(stack_depot_init);* Initializes new stack pool, and updates the list of pools. */ static bool depot_init_pool (void **prealloc) + __must_hold(&pool_lock) {lockdep_assert_held (&pool_lock);
@@ -289,6 +294,7 @@ static bool depot_init_pool(void **prealloc)/* Keeps the preallocated memory to be used for a new stack depot pool. */ static void depot_keep_new_pool (void **prealloc) + __must_hold(&pool_lock) {lockdep_assert_held (&pool_lock);
@@ -308,6 +314,7 @@ static void depot_keep_new_pool(void **prealloc)* the current pre-allocation. */ static struct stack_record *depot_pop_free_pool (void **prealloc,size_t size) + __must_hold(&pool_lock) {struct stack_record *stack;void *current_pool;
@@ -342,6 +349,7 @@ static struct stack_record *depot_pop_free_pool(void **prealloc, size_t size)/* Try to find next free usable entry from the freelist. */ static struct stack_record *depot_pop_free (void ) + __must_hold(&pool_lock) {struct stack_record *stack;
@@ -379,6 +387,7 @@ static inline size_t depot_stack_record_size(struct stack_record *s, unsigned in/* Allocates a new stack in a stack depot pool. */ static struct stack_record *depot_alloc_stack (unsigned long *entries,unsigned int nr_entries,u32 hash,depot_flags_t flags,void **prealloc) + __must_hold(&pool_lock) {struct stack_record *stack =NULL ;size_t record_size;
@@ -437,6 +446,7 @@ depot_alloc_stack(unsigned long *entries, unsigned int nr_entries, u32 hash, dep }static struct stack_record *depot_fetch_stack (depot_stack_handle_t handle) + __must_not_hold(&pool_lock) {const int pools_num_cached =READ_ONCE (pools_num);union handle_parts parts = { .handle = handle };
@@ -453,7 +463,8 @@ static struct stack_record *depot_fetch_stack(depot_stack_handle_t handle)return NULL ; } - pool = stack_pools[pool_index]; +/* @pool_index either valid, or user passed in corrupted value. */ + pool =capability_unsafe (stack_pools[pool_index]);if (WARN_ON (!pool))return NULL ;
@@ -466,6 +477,7 @@ static struct stack_record *depot_fetch_stack(depot_stack_handle_t handle)/* Links stack into the freelist. */ static void depot_free_stack (struct stack_record *stack) + __must_not_hold(&pool_lock) {unsigned long flags;
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerRFC PATCH 24/24 rhashtable: Enable capability analysis
In reply to Marco ElverEnable capability analysis for rhashtable, which was used as an initial test as it contains a combination of RCU, mutex, and bit_spinlock usage.
Users of rhashtable now also benefit from annotations on the API, which will now warn if the RCU read lock is not held where required.
--- include/linux/rhashtable.h | 14 +++++++++++--- lib/Makefile | 2 ++ lib/rhashtable.c | 12 +++++++++--- 3 files changed, 22 insertions(+), 6 deletions(-)
include/linux/rhashtable.h +11 -3
@@ -245,16 +245,17 @@ void *rhashtable_insert_slow(struct rhashtable *ht, const void *key,void rhashtable_walk_enter (struct rhashtable *ht,struct rhashtable_iter *iter);void rhashtable_walk_exit (struct rhashtable_iter *iter); -int rhashtable_walk_start_check(struct rhashtable_iter *iter) __acquires(RCU); +int rhashtable_walk_start_check(struct rhashtable_iter *iter) __acquires_shared(RCU);static inline void rhashtable_walk_start (struct rhashtable_iter *iter) +__acquires_shared (RCU ) { (void )rhashtable_walk_start_check (iter); }void *rhashtable_walk_next (struct rhashtable_iter *iter);void *rhashtable_walk_peek (struct rhashtable_iter *iter); -void rhashtable_walk_stop(struct rhashtable_iter *iter) __releases(RCU); +void rhashtable_walk_stop(struct rhashtable_iter *iter) __releases_shared(RCU); void rhashtable_free_and_destroy(struct rhashtable *ht, void (*free_fn)(void *ptr, void *arg),
@@ -325,6 +326,7 @@ static inline struct rhash_lock_head __rcu **rht_bucket_insert(static inline unsigned long rht_lock (struct bucket_table *tbl,struct rhash_lock_head __rcu **bkt) +__acquires (__bitlock (0 , bkt)) {unsigned long flags;
@@ -337,6 +339,7 @@ static inline unsigned long rht_lock(struct bucket_table *tbl,static inline unsigned long rht_lock_nested (struct bucket_table *tbl,struct rhash_lock_head __rcu **bucket,unsigned int subclass) +__acquires (__bitlock (0 , bucket)) {unsigned long flags;
@@ -349,6 +352,7 @@ static inline unsigned long rht_lock_nested(struct bucket_table *tbl,static inline void rht_unlock (struct bucket_table *tbl,struct rhash_lock_head __rcu **bkt,unsigned long flags) +__releases (__bitlock (0 , bkt)) {lock_map_release (&tbl->dep_map);bit_spin_unlock (0 , (unsigned long *)bkt);
@@ -402,13 +406,14 @@ static inline void rht_assign_unlock(struct bucket_table *tbl,struct rhash_lock_head __rcu **bkt, struct rhash_head *obj, unsigned long flags) + __releases(__bitlock(0, bkt)) { if (rht_is_a_nulls(obj)) obj =NULL ;lock_map_release (&tbl->dep_map);rcu_assign_pointer (*bkt, (void *)obj);preempt_enable (); -__release (bitlock); +__release (__bitlock (0 , bkt));local_irq_restore (flags); }
@@ -589,6 +594,7 @@ static inline int rhashtable_compare(struct rhashtable_compare_arg *arg,static inline struct rhash_head *__rhashtable_lookup (struct rhashtable *ht,const void *key,const struct rhashtable_params params) +__must_hold_shared (RCU ) {struct rhashtable_compare_arg arg = { .ht = ht,
@@ -642,6 +648,7 @@ static inline struct rhash_head *__rhashtable_lookup(static inline void *rhashtable_lookup (struct rhashtable *ht,const void *key,const struct rhashtable_params params) +__must_hold_shared (RCU ) {struct rhash_head *he =__rhashtable_lookup (ht, key, params);
@@ -692,6 +699,7 @@ static inline void *rhashtable_lookup_fast(static inline struct rhlist_head *rhltable_lookup (struct rhltable *hlt,const void *key,const struct rhashtable_params params) +__must_hold_shared (RCU ) {struct rhash_head *he =__rhashtable_lookup (&hlt->ht, key, params);
lib/Makefile +2 -0
@@ -45,6 +45,8 @@ lib-$(CONFIG_MIN_HEAP) += min_heap.o lib-y += kobject.o klist.o obj-y += lockref.o +CAPABILITY_ANALYSIS_rhashtable.o := y + obj-y += bcd.o sort.o parser.o debug_locks.o random32.o \ bust_spinlocks.o kasprintf.o bitmap.o scatterlist.o \ list_sort.o uuid.o iov_iter.o clz_ctz.o \
lib/rhashtable.c +9 -3
@@ -11,6 +11,10 @@* pointer as suggested by Josh Triplett */ +#include <linux/rhashtable.h> + +disable_capability_analysis (); +#include <linux/atomic.h> #include <linux/kernel.h> #include <linux/init.h>
@@ -22,10 +26,11 @@#include <linux/mm.h> #include <linux/jhash.h> #include <linux/random.h> -#include <linux/rhashtable.h> #include <linux/err.h> #include <linux/export.h> +enable_capability_analysis (); +#define HASH_DEFAULT_SIZE 64UL#define HASH_MIN_SIZE 4U
@@ -358,6 +363,7 @@ static int rhashtable_rehash_table(struct rhashtable *ht)static int rhashtable_rehash_alloc (struct rhashtable *ht,struct bucket_table *old_tbl,unsigned int size) + __must_hold(&ht->mutex) {struct bucket_table *new_tbl;int err;
@@ -392,6 +398,7 @@ static int rhashtable_rehash_alloc(struct rhashtable *ht,* bucket locks or concurrent RCU protected lookups and traversals. */ static int rhashtable_shrink (struct rhashtable *ht) + __must_hold(&ht->mutex) {struct bucket_table *old_tbl =rht_dereference (ht->tbl, ht);unsigned int nelems =atomic_read (&ht->nelems);
@@ -724,7 +731,7 @@ EXPORT_SYMBOL_GPL(rhashtable_walk_exit);* resize events and always continue. */ int rhashtable_walk_start_check (struct rhashtable_iter *iter) -__acquires (RCU ) +__acquires_shared (RCU ) {struct rhashtable *ht = iter->ht;bool rhlist = ht->rhlist;
@@ -940,7 +947,6 @@ EXPORT_SYMBOL_GPL(rhashtable_walk_peek);* hash table. */ void rhashtable_walk_stop (struct rhashtable_iter *iter) -__releases (RCU ) {struct rhashtable *ht;struct bucket_table *tbl = iter->walker.tbl;
Signature · 1 line
2.48.1.502.g6dc24dfdaf-goog
Marco Elver
maintainerOn Thu, 6 Feb 2025 at 19:17, Marco Elver <[email protected]> wrote:
[...]
Quoted 5 lines
Capability analysis is a C language extension, which enables statically checking that user-definable "capabilities" are acquired and released where required. An obvious application is lock-safety checking for the kernel's various synchronization primitives (each of which represents a "capability"), and checking that locking rules are not violated.
[...]
Quoted 1 line
This series is also available at this Git tree:
https://web.git.kernel.org/pub/scm/linux/kernel/git/melver/linux.git/log/?h=cap-analysis/dev
I'm planning to send a v2 soon (Clang just gained -Wthread-safety-pointer which I wanted to have committed before). Preview at the above tree.
Thanks,
-- Marco