Skip to contentSkip to conversation

notifier: Make atomic_notifiers use raw_spinlock

PATCH In review Sebastian Andrzej Siewior 2 participants 4 messages Aug 6, 2021

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 notifier: Make atomic_notifiers use raw_spinlock

From: Valentin Schneider <[email protected]>
Date: Sun, 22 Nov 2020 20:19:04 +0000

Booting a recent PREEMPT_RT kernel (v5.10-rc3-rt7-rebase) on my arm64 Juno leads to the idle task blocking on an RT sleeping spinlock down some notifier path:

| BUG: scheduling while atomic: swapper/5/0/0x00000002 …

|  atomic_notifier_call_chain_robust (kernel/notifier.c:71 kernel/notifier.c:118 kernel/notifier.c:186)
|  cpu_pm_enter (kernel/cpu_pm.c:39 kernel/cpu_pm.c:93)
|  psci_enter_idle_state (drivers/cpuidle/cpuidle-psci.c:52 drivers/cpuidle/cpuidle-psci.c:129)
|  cpuidle_enter_state (drivers/cpuidle/cpuidle.c:238)
|  cpuidle_enter (drivers/cpuidle/cpuidle.c:353)
|  do_idle (kernel/sched/idle.c:132 kernel/sched/idle.c:213 kernel/sched/idle.c:273)
|  cpu_startup_entry (kernel/sched/idle.c:368 (discriminator 1))
|  secondary_start_kernel (arch/arm64/kernel/smp.c:273)

Two points worth noting:

1) That this is conceptually the same issue as pointed out in:
313c8c16ee62 ("PM / CPU: replace raw_notifier with atomic_notifier")
2) Only the _robust() variant of atomic_notifier callchains suffer from this

AFAICT only the cpu_pm_notifier_chain really needs to be changed, but singling it out would mean introducing a new (truly) non-blocking API. At the same time, callers that are fine with any blocking within the call chain should use blocking notifiers, so patching up all atomic_notifier's doesn't seem *too* crazy to me.

---

What do we do with this?
Do we merge this as-is, add another "robust atomic notifier" using only raw_spinlock_t for registration and notification (for only cpu_pm_notifier_chain) instead of switching to raw_spinlock_t for all atomic notifier in -tree?

 include/linux/notifier.h |  6 +++---
 kernel/notifier.c        | 12 ++++++------
 2 files changed, 9 insertions(+), 9 deletions(-)
include/linux/notifier.h +3 -3
index 2fb373a5c1ede..723bc2df63882 100644
@@ -58,7 +58,7 @@ struct notifier_block {
 };
 
 struct atomic_notifier_head {
-	spinlock_t lock;
+	raw_spinlock_t lock;
 	struct notifier_block __rcu *head;
 };
 
@@ -78,7 +78,7 @@ struct srcu_notifier_head {
 };
 
 #define ATOMIC_INIT_NOTIFIER_HEAD(name) do {	\
-		spin_lock_init(&(name)->lock);	\
+		raw_spin_lock_init(&(name)->lock);	\
 		(name)->head = NULL;		\
 	} while (0)
 #define BLOCKING_INIT_NOTIFIER_HEAD(name) do {	\
@@ -95,7 +95,7 @@ extern void srcu_init_notifier_head(struct srcu_notifier_head *nh);
 		cleanup_srcu_struct(&(name)->srcu);
 
 #define ATOMIC_NOTIFIER_INIT(name) {				\
-		.lock = __SPIN_LOCK_UNLOCKED(name.lock),	\
+		.lock = __RAW_SPIN_LOCK_UNLOCKED(name.lock),	\
 		.head = NULL }
 #define BLOCKING_NOTIFIER_INIT(name) {				\
 		.rwsem = __RWSEM_INITIALIZER((name).rwsem),	\
kernel/notifier.c +6 -6
index 1b019cbca594a..c20782f076432 100644
@@ -142,9 +142,9 @@ int atomic_notifier_chain_register(struct atomic_notifier_head *nh,
 	unsigned long flags;
 	int ret;
 
-	spin_lock_irqsave(&nh->lock, flags);
+	raw_spin_lock_irqsave(&nh->lock, flags);
 	ret = notifier_chain_register(&nh->head, n);
-	spin_unlock_irqrestore(&nh->lock, flags);
+	raw_spin_unlock_irqrestore(&nh->lock, flags);
 	return ret;
 }
 EXPORT_SYMBOL_GPL(atomic_notifier_chain_register);
@@ -164,9 +164,9 @@ int atomic_notifier_chain_unregister(struct atomic_notifier_head *nh,
 	unsigned long flags;
 	int ret;
 
-	spin_lock_irqsave(&nh->lock, flags);
+	raw_spin_lock_irqsave(&nh->lock, flags);
 	ret = notifier_chain_unregister(&nh->head, n);
-	spin_unlock_irqrestore(&nh->lock, flags);
+	raw_spin_unlock_irqrestore(&nh->lock, flags);
 	synchronize_rcu();
 	return ret;
 }
@@ -182,9 +182,9 @@ int atomic_notifier_call_chain_robust(struct atomic_notifier_head *nh,
 	 * Musn't use RCU; because then the notifier list can
 	 * change between the up and down traversal.
 	 */
-	spin_lock_irqsave(&nh->lock, flags);
+	raw_spin_lock_irqsave(&nh->lock, flags);
 	ret = notifier_call_chain_robust(&nh->head, val_up, val_down, v);
-	spin_unlock_irqrestore(&nh->lock, flags);
+	raw_spin_unlock_irqrestore(&nh->lock, flags);
 
 	return ret;
 }
Signature · 1 line
2.32.0
#1 lore raw
Quoted 5 lines · Sebastian Andrzej Siewior

What do we do with this?
Do we merge this as-is, add another "robust atomic notifier" using only raw_spinlock_t for registration and notification (for only cpu_pm_notifier_chain) instead of switching to raw_spinlock_t for all atomic notifier in -tree?

Right, so the problem I see with this is that notifier_chain_{,un}register() are O(n). Hardly something we should be putting under raw_spin_lock :/

#2 lore raw
Quoted 10 lines · Peter Zijlstra
Quoted 5 lines · Sebastian Andrzej Siewior

What do we do with this?
Do we merge this as-is, add another "robust atomic notifier" using only raw_spinlock_t for registration and notification (for only cpu_pm_notifier_chain) instead of switching to raw_spinlock_t for all atomic notifier in -tree?

Right, so the problem I see with this is that notifier_chain_{,un}register() are O(n). Hardly something we should be putting under raw_spin_lock :/

Yup, pretty much. So we make one robust notifier for cpu_pm_notifier_chain?

Sebastian

#3 lore raw
Quoted 8 lines · Sebastian Andrzej Siewior
Quoted 7 lines · Peter Zijlstra
Quoted 5 lines · Sebastian Andrzej Siewior

What do we do with this?
Do we merge this as-is, add another "robust atomic notifier" using only raw_spinlock_t for registration and notification (for only cpu_pm_notifier_chain) instead of switching to raw_spinlock_t for all atomic notifier in -tree?

Quoted 6 lines
Quoted 3 lines

Right, so the problem I see with this is that notifier_chain_{,un}register() are O(n). Hardly something we should be putting under raw_spin_lock :/

Yup, pretty much. So we make one robust notifier for cpu_pm_notifier_chain?

Yeah, I suppose so :-( Ideally that whole pm notifier thing goes, but that's *far* more work and I really don't want to be responsible for the brain damange resulting from looking at all that 'special' idle code.