Skip to contentSkip to conversation

locking/lockdep: make chain-hlocks average depth configurable

PATCH In review Muhammad Bilal 1 participant 2 messages Aug 4 – Aug 18

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 locking/lockdep: make chain-hlocks average depth configurable

The static chain_hlocks[] pool is sized as:

MAX_LOCKDEP_CHAIN_HLOCKS = MAX_LOCKDEP_CHAINS * AVG_LOCKDEP_CHAIN_DEPTH

MAX_LOCKDEP_CHAINS is already tunable via CONFIG_LOCKDEP_CHAINS_BITS, but AVG_LOCKDEP_CHAIN_DEPTH is hardcoded to 5 and has no Kconfig knob. Workloads that build unusually deep individual lock chains -- rather than simply a large number of distinct chains -- can exhaust chain_hlocks[] and trip:

  BUG: MAX_LOCKDEP_CHAIN_HLOCKS too low!

well before MAX_LOCKDEP_CHAINS itself is anywhere near full, which silently disables lock debugging for the rest of the boot (debug_locks_off_graph_unlock()). Bumping LOCKDEP_CHAINS_BITS alone does not help in that case since the bottleneck is chain depth, not chain count.

Observed on a PREEMPT_DYNAMIC + RCU lockdep + KASAN debug build, reproducing at every boot from add_chain_cache() failing to allocate out of the static pool, hit from sched wakeup, hrtimer, and btrfs flush-workqueue paths (deep IRQ/softirq nesting stacked on top of deep filesystem/scheduler call chains).

Add CONFIG_LOCKDEP_CHAIN_DEPTH so this can be tuned like LOCKDEP_BITS/LOCKDEP_CHAINS_BITS, defaulting to 5 to preserve current behavior for everyone who isn't hitting this.

struct lock_chain::base is a 24-bit index into chain_hlocks[], and add_chain_cache() enforces that with:

  BUILD_BUG_ON((1UL << 24) <= ARRAY_SIZE(chain_hlocks));

so MAX_LOCKDEP_CHAINS * LOCKDEP_CHAIN_DEPTH must stay under 2^24 for every reachable combination, not just the default. LOCKDEP_CHAINS_BITS goes up to 21, so the new knob's range is capped at 7 -- one more than that would let 2^21 * 8 == 2^24 reach the limit and fail the build at the top of the CHAINS_BITS range. The help text explains the cap instead of pointing people at a combination that can break the build.

---
 kernel/locking/lockdep_internals.h |  2 +-
 lib/Kconfig.debug                  | 25 +++++++++++++++++++++++++
 2 files changed, 26 insertions(+), 1 deletion(-)
kernel/locking/lockdep_internals.h +1 -1
index 0e5e6ffe91a3..ed566681f3c8 100644
@@ -121,7 +121,7 @@ enum {
 
 #define MAX_LOCKDEP_CHAINS	(1UL << MAX_LOCKDEP_CHAINS_BITS)
 
-#define AVG_LOCKDEP_CHAIN_DEPTH		5
+#define AVG_LOCKDEP_CHAIN_DEPTH		CONFIG_LOCKDEP_CHAIN_DEPTH
 #define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS * AVG_LOCKDEP_CHAIN_DEPTH)
 
 extern struct lock_chain lock_chains[];
lib/Kconfig.debug +25 -0
index 1244dcac2294..723df2bf8c5d 100644
@@ -1614,6 +1614,31 @@ config LOCKDEP_CHAINS_BITS
 	help
 	  Try increasing this value if you hit "BUG: MAX_LOCKDEP_CHAINS too low!" message.
 
+config LOCKDEP_CHAIN_DEPTH
+	int "Average depth for MAX_LOCKDEP_CHAIN_HLOCKS"
+	depends on LOCKDEP
+	range 3 7
+	default 5
+	help
+	  Average per-chain depth used to size the static chain_hlocks[]
+	  pool: MAX_LOCKDEP_CHAIN_HLOCKS = MAX_LOCKDEP_CHAINS *
+	  LOCKDEP_CHAIN_DEPTH.
+
+	  Workloads that build unusually deep lock chains (heavy irq/softirq
+	  nesting stacked on top of deep filesystem or scheduler call chains)
+	  can exhaust this pool and trip "BUG: MAX_LOCKDEP_CHAIN_HLOCKS too
+	  low!" well before MAX_LOCKDEP_CHAINS itself is exhausted, silently
+	  disabling lock debugging. Increase this value if you hit that
+	  message and LOCKDEP_CHAINS_BITS increases alone don't help.
+
+	  The upper bound of 7 is not arbitrary: struct lock_chain::base is
+	  a 24-bit index into chain_hlocks[], and add_chain_cache() has
+	  BUILD_BUG_ON((1UL << 24) <= ARRAY_SIZE(chain_hlocks)). At the
+	  maximum LOCKDEP_CHAINS_BITS of 21, a depth of 8 or higher makes
+	  MAX_LOCKDEP_CHAIN_HLOCKS reach 2^24 and fails the build, so the
+	  range here is capped to stay safe for every valid
+	  LOCKDEP_CHAINS_BITS setting rather than just the default.
+
 config LOCKDEP_STACK_TRACE_BITS
 	int "Size for MAX_STACK_TRACE_ENTRIES (as Nth power of 2)"
 	depends on LOCKDEP && !LOCKDEP_SMALL
Signature · 1 line
2.55.0
#1 lore raw

ping.

Quoted 100 lines · Muhammad Bilal

The static chain_hlocks[] pool is sized as:

MAX_LOCKDEP_CHAIN_HLOCKS = MAX_LOCKDEP_CHAINS * AVG_LOCKDEP_CHAIN_DEPTH

MAX_LOCKDEP_CHAINS is already tunable via CONFIG_LOCKDEP_CHAINS_BITS, but AVG_LOCKDEP_CHAIN_DEPTH is hardcoded to 5 and has no Kconfig knob. Workloads that build unusually deep individual lock chains -- rather than simply a large number of distinct chains -- can exhaust chain_hlocks[] and trip:

  BUG: MAX_LOCKDEP_CHAIN_HLOCKS too low!

well before MAX_LOCKDEP_CHAINS itself is anywhere near full, which silently disables lock debugging for the rest of the boot (debug_locks_off_graph_unlock()). Bumping LOCKDEP_CHAINS_BITS alone does not help in that case since the bottleneck is chain depth, not chain count.

Observed on a PREEMPT_DYNAMIC + RCU lockdep + KASAN debug build, reproducing at every boot from add_chain_cache() failing to allocate out of the static pool, hit from sched wakeup, hrtimer, and btrfs flush-workqueue paths (deep IRQ/softirq nesting stacked on top of deep filesystem/scheduler call chains).

Add CONFIG_LOCKDEP_CHAIN_DEPTH so this can be tuned like LOCKDEP_BITS/LOCKDEP_CHAINS_BITS, defaulting to 5 to preserve current behavior for everyone who isn't hitting this.

struct lock_chain::base is a 24-bit index into chain_hlocks[], and add_chain_cache() enforces that with:

  BUILD_BUG_ON((1UL << 24) <= ARRAY_SIZE(chain_hlocks));

so MAX_LOCKDEP_CHAINS * LOCKDEP_CHAIN_DEPTH must stay under 2^24 for every reachable combination, not just the default. LOCKDEP_CHAINS_BITS goes up to 21, so the new knob's range is capped at 7 -- one more than that would let 2^21 * 8 == 2^24 reach the limit and fail the build at the top of the CHAINS_BITS range. The help text explains the cap instead of pointing people at a combination that can break the build.

---
 kernel/locking/lockdep_internals.h |  2 +-
 lib/Kconfig.debug                  | 25 +++++++++++++++++++++++++
 2 files changed, 26 insertions(+), 1 deletion(-)
kernel/locking/lockdep_internals.h +1 -1
index 0e5e6ffe91a3..ed566681f3c8 100644
@@ -121,7 +121,7 @@ enum {
 
 #define MAX_LOCKDEP_CHAINS     (1UL << MAX_LOCKDEP_CHAINS_BITS)
 
-#define AVG_LOCKDEP_CHAIN_DEPTH                5
+#define AVG_LOCKDEP_CHAIN_DEPTH                CONFIG_LOCKDEP_CHAIN_DEPTH
 #define MAX_LOCKDEP_CHAIN_HLOCKS (MAX_LOCKDEP_CHAINS * AVG_LOCKDEP_CHAIN_DEPTH)
 
 extern struct lock_chain lock_chains[];
lib/Kconfig.debug +25 -1
index 1244dcac2294..723df2bf8c5d 100644
@@ -1614,6 +1614,31 @@ config LOCKDEP_CHAINS_BITS
        help
          Try increasing this value if you hit "BUG: MAX_LOCKDEP_CHAINS too low!" message.
 
+config LOCKDEP_CHAIN_DEPTH
+       int "Average depth for MAX_LOCKDEP_CHAIN_HLOCKS"
+       depends on LOCKDEP
+       range 3 7
+       default 5
+       help
+         Average per-chain depth used to size the static chain_hlocks[]
+         pool: MAX_LOCKDEP_CHAIN_HLOCKS = MAX_LOCKDEP_CHAINS *
+         LOCKDEP_CHAIN_DEPTH.
+
+         Workloads that build unusually deep lock chains (heavy irq/softirq
+         nesting stacked on top of deep filesystem or scheduler call chains)
+         can exhaust this pool and trip "BUG: MAX_LOCKDEP_CHAIN_HLOCKS too
+         low!" well before MAX_LOCKDEP_CHAINS itself is exhausted, silently
+         disabling lock debugging. Increase this value if you hit that
+         message and LOCKDEP_CHAINS_BITS increases alone don't help.
+
+         The upper bound of 7 is not arbitrary: struct lock_chain::base is
+         a 24-bit index into chain_hlocks[], and add_chain_cache() has
+         BUILD_BUG_ON((1UL << 24) <= ARRAY_SIZE(chain_hlocks)). At the
+         maximum LOCKDEP_CHAINS_BITS of 21, a depth of 8 or higher makes
+         MAX_LOCKDEP_CHAIN_HLOCKS reach 2^24 and fails the build, so the
+         range here is capped to stay safe for every valid
+         LOCKDEP_CHAINS_BITS setting rather than just the default.
+
 config LOCKDEP_STACK_TRACE_BITS
        int "Size for MAX_STACK_TRACE_ENTRIES (as Nth power of 2)"
        depends on LOCKDEP && !LOCKDEP_SMALL
--

2.55.0