locking/lockdep: make chain-hlocks average depth configurable
Subsystems (2)
PATCH v2 1/1 locking/lockdep: make chain-hlocks average depth configurable
The static chain_hlocks[] pool is sized as:
MAX_LOCKDEP_CHAIN_HLOCKS = MAX_LOCKDEP_CHAINS * AVG_LOCKDEP_CHAIN_DEPTH
MAX_LOCKDEP_CHAINS is already tunable via CONFIG_LOCKDEP_CHAINS_BITS, but AVG_LOCKDEP_CHAIN_DEPTH is hardcoded to 5 and has no Kconfig knob. Workloads that build unusually deep individual lock chains -- rather than simply a large number of distinct chains -- can exhaust chain_hlocks[] and trip:
BUG: MAX_LOCKDEP_CHAIN_HLOCKS too low!
well before MAX_LOCKDEP_CHAINS itself is anywhere near full, which silently disables lock debugging for the rest of the boot (debug_locks_off_graph_unlock()). Bumping LOCKDEP_CHAINS_BITS alone does not help in that case since the bottleneck is chain depth, not chain count.
Observed on a PREEMPT_DYNAMIC + RCU lockdep + KASAN debug build, reproducing at every boot from add_chain_cache() failing to allocate out of the static pool, hit from sched wakeup, hrtimer, and btrfs flush-workqueue paths (deep IRQ/softirq nesting stacked on top of deep filesystem/scheduler call chains).
Add CONFIG_LOCKDEP_CHAIN_DEPTH so this can be tuned like LOCKDEP_BITS/LOCKDEP_CHAINS_BITS, defaulting to 5 to preserve current behavior for everyone who isn't hitting this.
struct lock_chain::base is a 24-bit index into chain_hlocks[], and add_chain_cache() enforces that with:
BUILD_BUG_ON((1UL << 24) <= ARRAY_SIZE(chain_hlocks));
so MAX_LOCKDEP_CHAINS * LOCKDEP_CHAIN_DEPTH must stay under 2^24 for every reachable combination, not just the default. LOCKDEP_CHAINS_BITS goes up to 21, so the new knob's range is capped at 7 -- one more than that would let 2^21 * 8 == 2^24 reach the limit and fail the build at the top of the CHAINS_BITS range. The help text explains the cap instead of pointing people at a combination that can break the build.
--- kernel/locking/lockdep_internals.h | 2 +- lib/Kconfig.debug | 25 +++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-)
kernel/locking/lockdep_internals.h +1 -1
@@ -121,7 +121,7 @@ enum {#define MAX_LOCKDEP_CHAINS (1UL << MAX_LOCKDEP_CHAINS_BITS) -#define AVG_LOCKDEP_CHAIN_DEPTH 5 +#define AVG_LOCKDEP_CHAIN_DEPTH CONFIG_LOCKDEP_CHAIN_DEPTH#define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS * AVG_LOCKDEP_CHAIN_DEPTH)extern struct lock_chain lock_chains[];
lib/Kconfig.debug +25 -0
@@ -1614,6 +1614,31 @@ config LOCKDEP_CHAINS_BITS help Try increasing this value if you hit "BUG: MAX_LOCKDEP_CHAINS too low!" message. +config LOCKDEP_CHAIN_DEPTH + int "Average depth for MAX_LOCKDEP_CHAIN_HLOCKS" + depends on LOCKDEP + range 3 7 + default 5 + help + Average per-chain depth used to size the static chain_hlocks[] + pool: MAX_LOCKDEP_CHAIN_HLOCKS = MAX_LOCKDEP_CHAINS * + LOCKDEP_CHAIN_DEPTH. + + Workloads that build unusually deep lock chains (heavy irq/softirq + nesting stacked on top of deep filesystem or scheduler call chains) + can exhaust this pool and trip "BUG: MAX_LOCKDEP_CHAIN_HLOCKS too + low!" well before MAX_LOCKDEP_CHAINS itself is exhausted, silently + disabling lock debugging. Increase this value if you hit that + message and LOCKDEP_CHAINS_BITS increases alone don't help. + + The upper bound of 7 is not arbitrary: struct lock_chain::base is + a 24-bit index into chain_hlocks[], and add_chain_cache() has + BUILD_BUG_ON((1UL << 24) <= ARRAY_SIZE(chain_hlocks)). At the + maximum LOCKDEP_CHAINS_BITS of 21, a depth of 8 or higher makes + MAX_LOCKDEP_CHAIN_HLOCKS reach 2^24 and fails the build, so the + range here is capped to stay safe for every valid + LOCKDEP_CHAINS_BITS setting rather than just the default. + config LOCKDEP_STACK_TRACE_BITS int "Size for MAX_STACK_TRACE_ENTRIES (as Nth power of 2)" depends on LOCKDEP && !LOCKDEP_SMALL
Signature · 1 line
2.55.0
ping.
Quoted 100 lines · Muhammad Bilal
The static chain_hlocks[] pool is sized as:
MAX_LOCKDEP_CHAIN_HLOCKS=MAX_LOCKDEP_CHAINS*AVG_LOCKDEP_CHAIN_DEPTH
MAX_LOCKDEP_CHAINSis already tunable viaCONFIG_LOCKDEP_CHAINS_BITS, butAVG_LOCKDEP_CHAIN_DEPTHis hardcoded to 5 and has noKconfigknob. Workloads that build unusually deep individual lock chains -- rather than simply a large number of distinct chains -- can exhaust chain_hlocks[] and trip:BUG: MAX_LOCKDEP_CHAIN_HLOCKS too low!well before
MAX_LOCKDEP_CHAINSitself is anywhere near full, which silently disables lock debugging for the rest of the boot (debug_locks_off_graph_unlock()). BumpingLOCKDEP_CHAINS_BITSalone does not help in that case since the bottleneck is chain depth, not chain count.Observed on a
PREEMPT_DYNAMIC+ RCU lockdep + KASAN debug build, reproducing at every boot fromadd_chain_cache()failing to allocate out of the static pool, hit from sched wakeup, hrtimer, and btrfs flush-workqueue paths (deep IRQ/softirq nesting stacked on top of deep filesystem/scheduler call chains).Add
CONFIG_LOCKDEP_CHAIN_DEPTHso this can be tuned like LOCKDEP_BITS/LOCKDEP_CHAINS_BITS, defaulting to 5 to preserve current behavior for everyone who isn't hitting this.struct
lock_chain::baseis a 24-bit index into chain_hlocks[], andadd_chain_cache()enforces that with:BUILD_BUG_ON((1UL << 24) <= ARRAY_SIZE(chain_hlocks));so
MAX_LOCKDEP_CHAINS*LOCKDEP_CHAIN_DEPTHmust stay under 2^24 for every reachable combination, not just the default.LOCKDEP_CHAINS_BITSgoes up to 21, so the new knob's range is capped at 7 -- one more than that would let 2^21 * 8 == 2^24 reach the limit and fail the build at the top of theCHAINS_BITSrange. The help text explains the cap instead of pointing people at a combination that can break the build.Reported-by Zhan Xusheng <[email protected]>Signed-off-by Muhammad Bilal <[email protected]>--- kernel/locking/lockdep_internals.h | 2 +- lib/Kconfig.debug | 25 +++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-)kernel/locking/lockdep_internals.h +1
-1@@ -121,7 +121,7 @@ enum {#define MAX_LOCKDEP_CHAINS (1UL << MAX_LOCKDEP_CHAINS_BITS) -#define AVG_LOCKDEP_CHAIN_DEPTH 5 +#define AVG_LOCKDEP_CHAIN_DEPTH CONFIG_LOCKDEP_CHAIN_DEPTH#define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS * AVG_LOCKDEP_CHAIN_DEPTH)extern struct lock_chain lock_chains[];lib/Kconfig.debug +25
-1@@ -1614,6 +1614,31 @@ config LOCKDEP_CHAINS_BITS help Try increasing this value if you hit "BUG: MAX_LOCKDEP_CHAINS too low!" message. +config LOCKDEP_CHAIN_DEPTH + int "Average depth for MAX_LOCKDEP_CHAIN_HLOCKS" + depends on LOCKDEP + range 3 7 + default 5 + help + Average per-chain depth used to size the static chain_hlocks[] + pool: MAX_LOCKDEP_CHAIN_HLOCKS = MAX_LOCKDEP_CHAINS * + LOCKDEP_CHAIN_DEPTH. + + Workloads that build unusually deep lock chains (heavy irq/softirq + nesting stacked on top of deep filesystem or scheduler call chains) + can exhaust this pool and trip "BUG: MAX_LOCKDEP_CHAIN_HLOCKS too + low!" well before MAX_LOCKDEP_CHAINS itself is exhausted, silently + disabling lock debugging. Increase this value if you hit that + message and LOCKDEP_CHAINS_BITS increases alone don't help. + + The upper bound of 7 is not arbitrary: struct lock_chain::base is + a 24-bit index into chain_hlocks[], and add_chain_cache() has + BUILD_BUG_ON((1UL << 24) <= ARRAY_SIZE(chain_hlocks)). At the + maximum LOCKDEP_CHAINS_BITS of 21, a depth of 8 or higher makes + MAX_LOCKDEP_CHAIN_HLOCKS reach 2^24 and fails the build, so the + range here is capped to stay safe for every valid + LOCKDEP_CHAINS_BITS setting rather than just the default. + config LOCKDEP_STACK_TRACE_BITS int "Size for MAX_STACK_TRACE_ENTRIES (as Nth power of 2)" depends on LOCKDEP && !LOCKDEP_SMALL --2.55.0