Skip to contentSkip to conversation

sched: Fix race between yield_to() and try_to_wake_up()

PATCH Applied Tianchen Ding 3 participants 4 messages Dec 31, 2024 – Jan 15, 2025 Scheduler

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 sched: Fix race between yield_to() and try_to_wake_up()

We met a SCHED_WARN in set_next_buddy():
__warn_printk
set_next_buddy
yield_to_task_fair
yield_to
kvm_vcpu_yield_to [kvm]
...

After a short dig, we found the rq_lock held by yield_to() may not be exactly the rq that the target task belongs to. There is a race window against try_to_wake_up().

CPU0 target_task

blocking on CPU1
lock rq0 & rq1
double check task_rq == p_rq, ok
woken to CPU2 (lock task_pi & rq2) task_rq = rq2
yield_to_task_fair (w/o lock rq2)

In this race window, yield_to() is operating the task w/o the currect lock. Fix this by taking task pi_lock first.

Fixes d95f41220065 ("sched: Add yield_to(task, preempt) functionality")
---
 kernel/sched/syscalls.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/syscalls.c +1 -1
index ff0e5ab4e37c..943406c4ee86 100644
@@ -1433,7 +1433,7 @@ int __sched yield_to(struct task_struct *p, bool preempt)
 	struct rq *rq, *p_rq;
 	int yielded = 0;
 
-	scoped_guard (irqsave) {
+	scoped_guard (raw_spinlock_irqsave, &p->pi_lock) {
 		rq = this_rq();
 
 again:
Signature · 1 line
2.39.3
#1 lore raw
Quoted 14 lines · Tianchen Ding

We met a SCHED_WARN in set_next_buddy():

   __warn_printk
   set_next_buddy
   yield_to_task_fair
   yield_to
   kvm_vcpu_yield_to [kvm]
   ...

After a short dig, we found the rq_lock held by yield_to() may not be exactly the rq that the target task belongs to. There is a race window against try_to_wake_up().

CPU0 target_task

Quoted 8 lines

blocking on CPU1
lock rq0 & rq1
double check task_rq == p_rq, ok
woken to CPU2 (lock task_pi & rq2) task_rq = rq2
yield_to_task_fair (w/o lock rq2)

In this race window, yield_to() is operating the task w/o the currect

Sorry... a typo for "correct"

maintainer
#2 lore raw
In reply to Tianchen Ding
Quoted 35 lines · Tianchen Ding

We met a SCHED_WARN in set_next_buddy():
__warn_printk
set_next_buddy
yield_to_task_fair
yield_to
kvm_vcpu_yield_to [kvm]
...

After a short dig, we found the rq_lock held by yield_to() may not be exactly the rq that the target task belongs to. There is a race window against try_to_wake_up().

CPU0 target_task

blocking on CPU1
lock rq0 & rq1
double check task_rq == p_rq, ok
woken to CPU2 (lock task_pi & rq2) task_rq = rq2
yield_to_task_fair (w/o lock rq2)

In this race window, yield_to() is operating the task w/o the currect lock. Fix this by taking task pi_lock first.

Fixes d95f41220065 ("sched: Add yield_to(task, preempt) functionality")
---
 kernel/sched/syscalls.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/syscalls.c +0 -0
index ff0e5ab4e37c..943406c4ee86 100644

kernel/sched/syscalls.c int __sched yield_to(struct task_struct *p, bool preempt) · line 1437

Quoted 6 lines
kernel/sched/syscalls.c +1 -1
@@ -1433,7 +1433,7 @@ int __sched yield_to(struct task_struct *p, bool preempt)
 	struct rq *rq, *p_rq;
 	int yielded = 0;
 
-	scoped_guard (irqsave) {
+	scoped_guard (raw_spinlock_irqsave, &p->pi_lock) {
 		rq = this_rq();

Thanks!

merge bot Applied
#3 lore raw
In reply to Tianchen Ding
The following commit has been merged into the sched/core branch of tip:

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     5d808c78d97251af1d3a3e4f253e7d6c39fd871e
Gitweb:        https://git.kernel.org/tip/5d808c78d97251af1d3a3e4f253e7d6c39fd871e
Author:        Tianchen Ding <[email protected]>
AuthorDate:    Tue, 31 Dec 2024 13:50:20 +08:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Mon, 13 Jan 2025 14:10:22 +01:00

sched: Fix race between yield_to() and try_to_wake_up()

We met a SCHED_WARN in set_next_buddy():
__warn_printk
set_next_buddy
yield_to_task_fair
yield_to
kvm_vcpu_yield_to [kvm]
...

After a short dig, we found the rq_lock held by yield_to() may not be exactly the rq that the target task belongs to. There is a race window against try_to_wake_up().

CPU0 target_task

blocking on CPU1
lock rq0 & rq1
double check task_rq == p_rq, ok
woken to CPU2 (lock task_pi & rq2) task_rq = rq2
yield_to_task_fair (w/o lock rq2)

In this race window, yield_to() is operating the task w/o the correct lock. Fix this by taking task pi_lock first.

Fixes d95f41220065 ("sched: Add yield_to(task, preempt) functionality")
---
 kernel/sched/syscalls.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/syscalls.c +1 -1
index ff0e5ab..943406c 100644
@@ -1433,7 +1433,7 @@ int __sched yield_to(struct task_struct *p, bool preempt)
 	struct rq *rq, *p_rq;
 	int yielded = 0;
 
-	scoped_guard (irqsave) {
+	scoped_guard (raw_spinlock_irqsave, &p->pi_lock) {
 		rq = this_rq();
 
 again: