Skip to contentSkip to conversation

sched/psi: Use try_cmpxchg() helpers in event and rtpoll wakeup paths

PATCH New David Carlier 1 participant 1 message Apr 18

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 sched/psi: Use try_cmpxchg() helpers in event and rtpoll wakeup paths

In update_triggers(), the event-generation gate uses cmpxchg(&t->event, 0, 1) == 0 to detect a 0 -> 1 transition. On x86 this compiles to CMPXCHG followed by a redundant compare of the returned value against the expected. try_cmpxchg() avoids the extra compare by reading ZF directly from CMPXCHG. Convert the call site to the try_cmpxchg() form, using a per-iteration comparand local so the failure-path writeback does not leak across loop iterations.

In psi_rtpoll_worker(), the wait_event_interruptible() condition uses atomic_cmpxchg(&group->rtpoll_wakeup, 1, 0) as a test-and-clear of the wakeup flag. An inline conversion is unsafe: try_cmpxchg() writes the observed value back into the expected-value lvalue on failure, and wait_event_interruptible() re-evaluates the condition many times per call, which would cause the comparand to drift and wakeups to be lost.

Factor the test-and-clear into psi_rtpoll_consume_wakeup(), which materializes a fresh comparand per invocation, and call it from the wait_event_interruptible() condition. Use the fully ordered atomic_try_cmpxchg() to preserve the original memory ordering.

No functional change.

Assisted-by Claude:claude-opus-4-7 [tool claude-code]
---
 kernel/sched/psi.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
kernel/sched/psi.c +9 -2
index d9c9d9480a45..868833bca4d4 100644
@@ -482,6 +482,7 @@ static void update_triggers(struct psi_group *group, u64 now,
 	 */
 	list_for_each_entry(t, triggers, node) {
 		u64 growth;
+		int zero = 0;
 		bool new_stall;
 
 		new_stall = aggregator_total[t->state] != total[t->state];
@@ -510,7 +511,7 @@ static void update_triggers(struct psi_group *group, u64 now,
 			continue;
 
 		/* Generate an event */
-		if (cmpxchg(&t->event, 0, 1) == 0) {
+		if (try_cmpxchg(&t->event, &zero, 1)) {
 			if (t->of)
 				kernfs_notify(t->of->kn);
 			else
@@ -735,6 +736,12 @@ static void psi_rtpoll_work(struct psi_group *group)
 	mutex_unlock(&group->rtpoll_trigger_lock);
 }
 
+static bool psi_rtpoll_consume_wakeup(struct psi_group *group)
+{
+	int wakeup = 1;
+	return atomic_try_cmpxchg(&group->rtpoll_wakeup, &wakeup, 0);
+}
+
 static int psi_rtpoll_worker(void *data)
 {
 	struct psi_group *group = (struct psi_group *)data;
@@ -743,7 +750,7 @@ static int psi_rtpoll_worker(void *data)
 
 	while (true) {
 		wait_event_interruptible(group->rtpoll_wait,
-				atomic_cmpxchg(&group->rtpoll_wakeup, 1, 0) ||
+				psi_rtpoll_consume_wakeup(group) ||
 				kthread_should_stop());
 		if (kthread_should_stop())
 			break;
Signature · 1 line
2.53.0