debugobjects: Don't call fill_pool() in early boot hardirq context
Subsystems (3)
v6 cover debugobjects: Don't call fill_pool() in early boot hardirq context
v6:
- Break the v5 patch into 2 separate patches with the first patch just adding a can_fill_pool() helper with no functional change as suggested by Ingo.
Patch 1 adds a can_fill_pool() helper to break down the conditional check in debug_objects_fill_pool() to determine if fill_pool() should be called into easier to read components.
Patch 2 fix the lockdep warning caused by calling fill_pool() in early boot hardirq context.
These 2 patches together produce the same code change as in the v5 patch [1]. So the Reviewed-by tag from Sebastian is kept in the patch 2.
[1] https://lore.kernel.org/lkml/[email protected]/
Waiman Long (2):
debugobjects: Add a can_fill_pool() helper for
debug_objects_fill_pool()
debugobjects: Don't call fill_pool() in early boot hardirq context
lib/debugobjects.c | 46 +++++++++++++++++++++++++++++++++++++--------- 1 file changed, 37 insertions(+), 9 deletions(-)
Signature · 1 line
2.54.0
PATCH v6 1/2 debugobjects: Add a can_fill_pool() helper for debug_objects_fill_pool()
The conditional check in debug_objects_fill_pool() for determining if fill_pool() can be called is now quite complicated. Add a new can_fill_pool() helper to break down the conditions individually to make them easier to understand. No functional change is expected.
--- lib/debugobjects.c | 35 ++++++++++++++++++++++++++--------- 1 file changed, 26 insertions(+), 9 deletions(-)
lib/debugobjects.c +26 -9
@@ -720,6 +720,30 @@ static inline bool debug_objects_is_pi_blocked_on(void)#endif } +static inline bool can_fill_pool (void ) +{ +/* +* On !RT enabled kernels there are no restrictions and spinlock_t and +* raw_spinlock_t are the same types. +*/ +if (!IS_ENABLED (CONFIG_PREEMPT_RT )) +return true; + +/* +* On RT enabled kernels the pool refill must happen in preemptible +* context and the task must not be blocked on a lock as that could +* corrupt the PI state when blocking on a lock in the allocation path. +*/ +if (preemptible () && !debug_objects_is_pi_blocked_on ()) +return true; + +/* +* Allow refilling to happen early in the boot with disabled interrupts +* as long as the scheduler is not operational. +*/ +return system_state <SYSTEM_SCHEDULING ; +} +static void debug_objects_fill_pool (void ) {if (!static_branch_likely (&obj_cache_enabled))
@@ -734,18 +758,11 @@ static void debug_objects_fill_pool(void)if (likely (!pool_should_refill (&pool_global)))return ; -/* -* On RT enabled kernels the pool refill must happen in preemptible -* context and not enqueued on an rt_mutex -- for !RT kernels we rely -* on the fact that spinlock_t and raw_spinlock_t are basically the -* same type and this lock-type inversion works just fine. -*/ -if (!IS_ENABLED (CONFIG_PREEMPT_RT ) || system_state <SYSTEM_SCHEDULING || - (preemptible () && !debug_objects_is_pi_blocked_on ())) { +if (can_fill_pool ()) {/* * Annotate away the spinlock_t inside raw_spinlock_t warning * by temporarily raising the wait-type to LD_WAIT_CONFIG, matching -* the preemptible() condition above. +* the preemptible() condition in can_fill_pool(). */ static DEFINE_WAIT_OVERRIDE_MAP(fill_pool_map, LD_WAIT_CONFIG);lock_map_acquire_try (&fill_pool_map);
Signature · 1 line
2.54.0
PATCH v6 2/2 debugobjects: Don't call fill_pool() in early boot hardirq context
In reply to Waiman LongWhen booting a debug PREEMPT_RT kernel on an arm64 system with grace processor, a "inconsistent {HARDIRQ-ON-W} -> {IN-HARDIRQ-W} usage" lockdep warning message was reported to the console.
During early boot, interrupts are getting enabled before the scheduler is enabled. In this window (before SYSTEM_SCHEDULING is set) interrupts can fire and attempt to fill the pool from within the hardirq. This can lead to a deadlock the interrupt occurred while in the memory allocator.
Reorder the exception rules and forbid this scenario by excluding allocations from hardirq.
--- lib/debugobjects.c | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-)
lib/debugobjects.c +18 -7
@@ -730,18 +730,29 @@ static inline bool can_fill_pool(void)return true;/* -* On RT enabled kernels the pool refill must happen in preemptible -* context and the task must not be blocked on a lock as that could -* corrupt the PI state when blocking on a lock in the allocation path. +* On RT enabled kernels, the task must not be blocked on a lock as +* that could corrupt the PI state when blocking on a lock in the +* allocation path. */ -if (preemptible () && !debug_objects_is_pi_blocked_on ()) +if (debug_objects_is_pi_blocked_on ()) +return false; + +/* +* On RT enabled kernels the pool refill should happen in preemptible +* context. +*/ +if (preemptible ())return true;/* -* Allow refilling to happen early in the boot with disabled interrupts -* as long as the scheduler is not operational. +* Though during system boot before scheduling is set up, preemption is +* disabled and the pool can get exhausted. Before scheduling is active +* a task cannot be blocked on a sleeping lock, but it might hold a lock +* and if interrupted then hard interrupt context might run into a lock +* inversion. So exclude hard interrupt context from allocations before +* scheduling is active. */ -return system_state <SYSTEM_SCHEDULING ; +return system_state <SYSTEM_SCHEDULING && !in_hardirq (); }static void debug_objects_fill_pool (void )
Signature · 1 line
2.54.0
Sebastian Andrzej Siewior
maintainerQuoted 4 lines · Waiman Long
v6:
- Break the v5 patch into 2 separate patches with the first patch just adding acan_fill_pool()helper with no functional change as suggested by Ingo.
V5 has been merged as of
https://git.kernel.org/tip/0d046ae106255cba5eb83b23f78ee93f3620247d
Sebastian