sched/psi: Use try_cmpxchg() helpers in event and rtpoll wakeup paths
Subsystems (2)
PATCH 1/1 sched/psi: Use try_cmpxchg() helpers in event and rtpoll wakeup paths
In update_triggers(), the event-generation gate uses cmpxchg(&t->event, 0, 1) == 0 to detect a 0 -> 1 transition. On x86 this compiles to CMPXCHG followed by a redundant compare of the returned value against the expected. try_cmpxchg() avoids the extra compare by reading ZF directly from CMPXCHG. Convert the call site to the try_cmpxchg() form, using a per-iteration comparand local so the failure-path writeback does not leak across loop iterations.
In psi_rtpoll_worker(), the wait_event_interruptible() condition uses atomic_cmpxchg(&group->rtpoll_wakeup, 1, 0) as a test-and-clear of the wakeup flag. An inline conversion is unsafe: try_cmpxchg() writes the observed value back into the expected-value lvalue on failure, and wait_event_interruptible() re-evaluates the condition many times per call, which would cause the comparand to drift and wakeups to be lost.
Factor the test-and-clear into psi_rtpoll_consume_wakeup(), which materializes a fresh comparand per invocation, and call it from the wait_event_interruptible() condition. Use the fully ordered atomic_try_cmpxchg() to preserve the original memory ordering.
No functional change.
--- kernel/sched/psi.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
kernel/sched/psi.c +9 -2
@@ -482,6 +482,7 @@ static void update_triggers(struct psi_group *group, u64 now,*/ list_for_each_entry (t, triggers, node) {u64 growth; +int zero =0 ;bool new_stall; new_stall = aggregator_total[t->state] != total[t->state];
@@ -510,7 +511,7 @@ static void update_triggers(struct psi_group *group, u64 now,continue ;/* Generate an event */ -if (cmpxchg (&t->event,0 ,1 ) ==0 ) { +if (try_cmpxchg (&t->event, &zero,1 )) {if (t->of)kernfs_notify (t->of->kn);else
@@ -735,6 +736,12 @@ static void psi_rtpoll_work(struct psi_group *group)mutex_unlock (&group->rtpoll_trigger_lock); } +static bool psi_rtpoll_consume_wakeup (struct psi_group *group) +{ +int wakeup =1 ; +return atomic_try_cmpxchg (&group->rtpoll_wakeup, &wakeup,0 ); +} +static int psi_rtpoll_worker (void *data) {struct psi_group *group = (struct psi_group *)data;
@@ -743,7 +750,7 @@ static int psi_rtpoll_worker(void *data)while (true) {wait_event_interruptible (group->rtpoll_wait, -atomic_cmpxchg (&group->rtpoll_wakeup,1 ,0 ) || +psi_rtpoll_consume_wakeup (group) ||kthread_should_stop ());if (kthread_should_stop ())break ;
Signature · 1 line
2.53.0