Skip to contentSkip to conversation

lockdep: skip debugobjects for lock_classes[] init

PATCH In review Qian Cai 3 participants 4 messages Mar 3, 2019 – Mar 9, 2019 Locking primitives

lore
Share & tools
lore

Qian Cai

author
#0 lore raw

PATCH 1/1 lockdep: skip debugobjects for lock_classes[] init

The linux-next commit a0b0fd53e1e6 ("locking/lockdep: Free lock classes that are no longer in use") let init_data_structures_once() call init_rcu_head(). As the result, smp_setup_processor_id() calls debug_object_init() before debug_objects_early_init() triggering errors below on arm64 and ppc64le. Since debug_objects_early_init() depends on smp_setup_processor_id(), so it is not straightforward to move the former earlier in start_kernel(). Hence, just skip all debugobjects creations in init_data_structures_once() as a good trade-off by losing track of some RCU list heads debugging.

BUG: spinlock bad magic on CPU#0, swapper/0
 lock: obj_hash+0xc3048/0x100000, .magic: 00000000, .owner: <none>/-1,
.owner_cpu: 0
CPU: 0 PID: 0 Comm: swapper Not tainted 5.0.0-rc8-next-20190301+ #1
Call trace:
 dump_backtrace+0x0/0x450
 show_stack+0x20/0x2c
 dump_stack+0xe0/0x16c
 spin_bug+0x200/0x224
 do_raw_spin_lock+0x1bc/0x23c
 _raw_spin_lock_irqsave+0xac/0xd0
 __debug_object_init+0xf0/0x19dc
 debug_object_init+0x54/0x6c
 init_rcu_head+0x24/0x30
 init_data_structures_once+0xb0/0x41c
 register_lock_class+0x70c/0x1454
 __lock_acquire+0xc4/0x235c
 lock_acquire+0x508/0x668
 _raw_spin_lock+0x88/0xac
 vprintk_emit+0x33c/0x53c
 vprintk_default+0x54/0x7c
 vprintk_func+0x320/0x40c
 printk+0x64/0x8c
 smp_setup_processor_id+0x6c/0x84
 start_kernel+0x38/0x8f8

ODEBUG: Out of memory. ODEBUG disabled

---
 kernel/locking/lockdep.c | 1 -
 1 file changed, 1 deletion(-)
kernel/locking/lockdep.c +0 -1
index 21cb81fe6359..1309a6fe88ee 100644
@@ -988,7 +988,6 @@ static void init_data_structures_once(void)
 
 	initialization_happened = true;
 
-	init_rcu_head(&delayed_free.rcu_head);
 	INIT_LIST_HEAD(&delayed_free.pf[0].zapped);
 	INIT_LIST_HEAD(&delayed_free.pf[1].zapped);
 
Signature · 1 line
2.17.2 (Apple Git-113)
#1 lore raw
Quoted 47 lines · Qian Cai

The linux-next commit a0b0fd53e1e6 ("locking/lockdep: Free lock classes that are no longer in use") let init_data_structures_once() call init_rcu_head(). As the result, smp_setup_processor_id() calls debug_object_init() before debug_objects_early_init() triggering errors below on arm64 and ppc64le. Since debug_objects_early_init() depends on smp_setup_processor_id(), so it is not straightforward to move the former earlier in start_kernel(). Hence, just skip all debugobjects creations in init_data_structures_once() as a good trade-off by losing track of some RCU list heads debugging.

BUG: spinlock bad magic on CPU#0, swapper/0
  lock: obj_hash+0xc3048/0x100000, .magic: 00000000, .owner: <none>/-1,
.owner_cpu: 0
CPU: 0 PID: 0 Comm: swapper Not tainted 5.0.0-rc8-next-20190301+ #1
Call trace:
  dump_backtrace+0x0/0x450
  show_stack+0x20/0x2c
  dump_stack+0xe0/0x16c
  spin_bug+0x200/0x224
  do_raw_spin_lock+0x1bc/0x23c
  _raw_spin_lock_irqsave+0xac/0xd0
  __debug_object_init+0xf0/0x19dc
  debug_object_init+0x54/0x6c
  init_rcu_head+0x24/0x30
  init_data_structures_once+0xb0/0x41c
  register_lock_class+0x70c/0x1454
  __lock_acquire+0xc4/0x235c
  lock_acquire+0x508/0x668
  _raw_spin_lock+0x88/0xac
  vprintk_emit+0x33c/0x53c
  vprintk_default+0x54/0x7c
  vprintk_func+0x320/0x40c
  printk+0x64/0x8c
  smp_setup_processor_id+0x6c/0x84
  start_kernel+0x38/0x8f8

ODEBUG: Out of memory. ODEBUG disabled

---
  kernel/locking/lockdep.c | 1 -
  1 file changed, 1 deletion(-)
kernel/locking/lockdep.c +0 -0
index 21cb81fe6359..1309a6fe88ee 100644

kernel/locking/lockdep.c static void init_data_structures_once(void) · line 992

Quoted 6 lines
kernel/locking/lockdep.c +0 -1
@@ -988,7 +988,6 @@ static void init_data_structures_once(void)
 
  	initialization_happened = true;
 
-	init_rcu_head(&delayed_free.rcu_head);
  	INIT_LIST_HEAD(&delayed_free.pf[0].zapped);
  	INIT_LIST_HEAD(&delayed_free.pf[1].zapped);

Hi Qian,

Thanks for having reported this issue. Had you considered to follow an approach similar to the patch below?

Thanks,

Bart.

[PATCH] locking/lockdep: Only call init_rcu_head() after RCU has been
initialized

init_data_structures_once() is called for the first time before RCU has been initialized. Make sure that init_rcu_head() is called before the RCU head is used and after RCU has been initialized.

---
 kernel/locking/lockdep.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)
kernel/locking/lockdep.c +11 -4
index 21cb81fe6359..c8c83e1e2664 100644
@@ -980,15 +980,22 @@ static inline void check_data_structures(void) { }
  */
 static void init_data_structures_once(void)
 {
-	static bool initialization_happened;
+	static bool ds_initialized, rcu_head_initialized;
 	int i;
 
-	if (likely(initialization_happened))
+	if (likely(rcu_head_initialized))
 		return;
 
-	initialization_happened = true;
+	if (system_state >= SYSTEM_SCHEDULING) {
+		init_rcu_head(&delayed_free.rcu_head);
+		rcu_head_initialized = true;
+	}
+
+	if (ds_initialized)
+		return;
+
+	ds_initialized = true;
 
-	init_rcu_head(&delayed_free.rcu_head);
 	INIT_LIST_HEAD(&delayed_free.pf[0].zapped);
 	INIT_LIST_HEAD(&delayed_free.pf[1].zapped);
 
Signature · 1 line
2.20.1
author Tested-by
#2 lore raw
Quoted 8 lines · Bart Van Assche

[PATCH] locking/lockdep: Only call init_rcu_head() after RCU has been initialized

init_data_structures_once() is called for the first time before RCU has been initialized. Make sure that init_rcu_head() is called before the RCU head is used and after RCU has been initialized.

I like your approach more.

#3 lore raw

[tip:locking/urgent] locking/lockdep: Only call init_rcu_head() after RCU has been initialized

In reply to Bart Van Assche
locking/lockdep: Only call init_rcu_head() after RCU has been initialized
Commit-ID:  0126574fca2ce0f0d5beb9dade6efb823ff7407b
Gitweb:     https://git.kernel.org/tip/0126574fca2ce0f0d5beb9dade6efb823ff7407b
Author:     Bart Van Assche <[email protected]>
AuthorDate: Sun, 3 Mar 2019 10:19:01 -0800
Committer:  Ingo Molnar <[email protected]>
CommitDate: Sat, 9 Mar 2019 14:15:51 +0100

locking/lockdep: Only call init_rcu_head() after RCU has been initialized

init_data_structures_once() is called for the first time before RCU has been initialized. Make sure that init_rcu_head() is called before the RCU head is used and after RCU has been initialized.

Cc Andrew Morton <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Dave Hansen <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Will Deacon <[email protected]>
---
 kernel/locking/lockdep.c | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)
kernel/locking/lockdep.c +11 -4
index 35a144dfddf5..34cdcbedda49 100644
@@ -982,15 +982,22 @@ static inline void check_data_structures(void) { }
  */
 static void init_data_structures_once(void)
 {
-	static bool initialization_happened;
+	static bool ds_initialized, rcu_head_initialized;
 	int i;
 
-	if (likely(initialization_happened))
+	if (likely(rcu_head_initialized))
 		return;
 
-	initialization_happened = true;
+	if (system_state >= SYSTEM_SCHEDULING) {
+		init_rcu_head(&delayed_free.rcu_head);
+		rcu_head_initialized = true;
+	}
+
+	if (ds_initialized)
+		return;
+
+	ds_initialized = true;
 
-	init_rcu_head(&delayed_free.rcu_head);
 	INIT_LIST_HEAD(&delayed_free.pf[0].zapped);
 	INIT_LIST_HEAD(&delayed_free.pf[1].zapped);