debugobjects: Various backports
Subsystems (3)
Thomas Gleixner
maintainercover debugobjects: Various backports
This series contains the necessary backports to apply the recent important debugobject fixes to the linux-6.18.y series.
Thanks,
tglx
Thomas Gleixner
maintainerPATCH 1/4 debugobjects: Allow to refill the pool before SYSTEM_SCHEDULING
From: Sebastian Andrzej Siewior <[email protected]>
commit 06e0ae988f6e3499785c407429953ade19c1096b upstream.
The pool of free objects is refilled on several occasions such as object initialisation. On PREEMPT_RT refilling is limited to preemptible sections due to sleeping locks used by the memory allocator. The system boots with disabled interrupts so the pool can not be refilled.
If too many objects are initialized and the pool gets empty then debugobjects disables itself.
Refiling can also happen early in the boot with disabled interrupts as long as the scheduler is not operational. If the scheduler can not preempt a task then a sleeping lock can not be contended.
Allow to additionally refill the pool if the scheduler is not operational.
--- lib/debugobjects.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
---
lib/debugobjects.c +1 -1
@@ -731,7 +731,7 @@ static void debug_objects_fill_pool(void* raw_spinlock_t are basically the same type and this lock-type * inversion works just fine. */ -if (!IS_ENABLED (CONFIG_PREEMPT_RT ) ||preemptible ()) { +if (!IS_ENABLED (CONFIG_PREEMPT_RT ) ||preemptible () || system_state <SYSTEM_SCHEDULING ) {/* * Annotate away the spinlock_t inside raw_spinlock_t warning * by temporarily raising the wait-type to WAIT_SLEEP, matching
Thomas Gleixner
maintainerPATCH 2/4 debugobjects: Use LD_WAIT_CONFIG instead of LD_WAIT_SLEEP
In reply to Thomas GleixnerFrom: Sebastian Andrzej Siewior <[email protected]>
commit 37de2dbc318ee10577c1c2704de5a803e75e55a2 upstream.
fill_pool_map is used to suppress nesting violations caused by acquiring a spinlock_t (from within the memory allocator) while holding a raw_spinlock_t. The used annotation is wrong.
LD_WAIT_SLEEP is for always sleeping lock types such as mutex_t. LD_WAIT_CONFIG is for lock type which are sleeping while spinning on PREEMPT_RT such as spinlock_t.
Use LD_WAIT_CONFIG as override.
--- lib/debugobjects.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
---
lib/debugobjects.c +2 -2
@@ -734,10 +734,10 @@ static void debug_objects_fill_pool(void)if (!IS_ENABLED (CONFIG_PREEMPT_RT ) ||preemptible () || system_state <SYSTEM_SCHEDULING ) {/* * Annotate away the spinlock_t inside raw_spinlock_t warning -* by temporarily raising the wait-type to WAIT_SLEEP, matching +* by temporarily raising the wait-type to LD_WAIT_CONFIG, matching * the preemptible() condition above. */ - static DEFINE_WAIT_OVERRIDE_MAP(fill_pool_map, LD_WAIT_SLEEP); + static DEFINE_WAIT_OVERRIDE_MAP(fill_pool_map, LD_WAIT_CONFIG);lock_map_acquire_try (&fill_pool_map);fill_pool ();lock_map_release (&fill_pool_map);
Thomas Gleixner
maintainerPATCH 3/4 debugobjects: Do not fill_pool() if pi_blocked_on
In reply to Thomas GleixnerFrom: Helen Koike <[email protected]>
commit 5f41161059fd0f1bbf18c90f3180e38cc45a14eb upstream.
On RT enabled kernels, fill_pool() ends up calling rtlock_lock(), which asserts if current::pi_blocked_on is set, because a task can obviously only block on one lock as otherwise the priority inheritenace chain gets corrupted.
Prevent this by expanding the conditional to take current::pi_blocked_on into account.
--- lib/debugobjects.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-)
---
lib/debugobjects.c +14 -4
@@ -711,6 +711,15 @@ static struct debug_obj *lookup_object_or_alloc(void *addr, struct debug_bucketreturn NULL ; } +static inline bool debug_objects_is_pi_blocked_on (void ) +{ +#ifdef CONFIG_RT_MUTEXES +return current->pi_blocked_on !=NULL ; +#else +return false; +#endif +} +static void debug_objects_fill_pool (void ) {if (!static_branch_likely (&obj_cache_enabled))
@@ -727,11 +736,12 @@ static void debug_objects_fill_pool(void)/* * On RT enabled kernels the pool refill must happen in preemptible -* context -- for !RT kernels we rely on the fact that spinlock_t and -* raw_spinlock_t are basically the same type and this lock-type -* inversion works just fine. +* context and not enqueued on an rt_mutex -- for !RT kernels we rely +* on the fact that spinlock_t and raw_spinlock_t are basically the +* same type and this lock-type inversion works just fine. */ -if (!IS_ENABLED (CONFIG_PREEMPT_RT ) ||preemptible () || system_state <SYSTEM_SCHEDULING ) { +if (!IS_ENABLED (CONFIG_PREEMPT_RT ) || system_state <SYSTEM_SCHEDULING || + (preemptible () && !debug_objects_is_pi_blocked_on ())) {/* * Annotate away the spinlock_t inside raw_spinlock_t warning * by temporarily raising the wait-type to LD_WAIT_CONFIG, matching
Thomas Gleixner
maintainerPATCH 4/4 debugobjects: Dont call fill_pool() in early boot hardirq context
In reply to Thomas GleixnerFrom: Waiman Long <[email protected]>
commit 0d046ae106255cba5eb83b23f78ee93f3620247d upstream.
When booting a debug PREEMPT_RT kernel on an ARM64 system, a "inconsistent {HARDIRQ-ON-W} -> {IN-HARDIRQ-W} usage" lockdep warning message was reported to the console.
During early boot, interrupts are enabled before the scheduler is enabled. In this window (before SYSTEM_SCHEDULING is set) interrupts can fire and in the hard interrupt context handler attempt to fill the pool
This can lead to a deadlock when the interrupt occurred when the interrupt hits a region which holds a lock that is required to be taken in the allocation path.
Add a new can_fill_pool() helper and reorder the exception rule and forbid this scenario by excluding allocations from hard interrupt context.
--- lib/debugobjects.c | 46 +++++++++++++++++++++++++++++++++++++--------- 1 file changed, 37 insertions(+), 9 deletions(-)
---
lib/debugobjects.c +37 -9
@@ -720,6 +720,41 @@ static inline bool debug_objects_is_pi_blocked_on(void)#endif } +static inline bool can_fill_pool (void ) +{ +/* +* On !RT enabled kernels there are no restrictions and spinlock_t and +* raw_spinlock_t are the same types. +*/ +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +return true; + +/* +* On RT enabled kernels, the task must not be blocked on a lock as +* that could corrupt the PI state when blocking on a lock in the +* allocation path. +*/ +if (debug_objects_is_pi_blocked_on ()) +return false; + +/* +* On RT enabled kernels the pool refill should happen in preemptible +* context. +*/ +if (preemptible ()) +return true; + +/* +* Though during system boot before scheduling is set up, preemption is +* disabled and the pool can get exhausted. Before scheduling is active +* a task cannot be blocked on a sleeping lock, but it might hold a lock +* and if interrupted then hard interrupt context might run into a lock +* inversion. So exclude hard interrupt context from allocations before +* scheduling is active. +*/ +return system_state <SYSTEM_SCHEDULING && !in_hardirq (); +} +static void debug_objects_fill_pool (void ) {if (!static_branch_likely (&obj_cache_enabled))
@@ -734,18 +769,11 @@ static void debug_objects_fill_pool(void)if (likely (!pool_should_refill (&pool_global)))return ; -/* -* On RT enabled kernels the pool refill must happen in preemptible -* context and not enqueued on an rt_mutex -- for !RT kernels we rely -* on the fact that spinlock_t and raw_spinlock_t are basically the -* same type and this lock-type inversion works just fine. -*/ -if (!IS_ENABLED (CONFIG_PREEMPT_RT ) || system_state <SYSTEM_SCHEDULING || - (preemptible () && !debug_objects_is_pi_blocked_on ())) { +if (can_fill_pool ()) {/* * Annotate away the spinlock_t inside raw_spinlock_t warning * by temporarily raising the wait-type to LD_WAIT_CONFIG, matching -* the preemptible() condition above. +* the preemptible() condition in can_fill_pool(). */ static DEFINE_WAIT_OVERRIDE_MAP(fill_pool_map, LD_WAIT_CONFIG);lock_map_acquire_try (&fill_pool_map);
Sasha Levin
maintainerQuoted 1 line · Thomas Gleixner
[patch v6.18.y 0/4] debugobjects: Various backports
All four queued for 6.18, thanks!
Signature · 2 lines
Thanks, Sasha