sched: fix warning in sched_setaffinity
PATCH 1/1 sched: fix warning in sched_setaffinity
Commit 8f9ea86fdf99b added some logic to sched_setaffinity that included a WARN when a per-task affinity assignment races with a cpuset update.
Specifically, we can have a race where a cpuset update results in the task affinity no longer being a subset of the cpuset. That's fine; we have a fallback to instead use the cpuset mask. However, we have a WARN set up that will trigger if the cpuset mask has no overlap at all with the requested task affinity. This shouldn't be a warning condition; its trivial to create this condition.
Reproduced the warning by the following setup:
- $PID inside a cpuset cgroup
- another thread repeatedly switching the cpuset cpus from 1-2 to just 1
- another thread repeatedly setting the $PID affinity (via taskset) to 2
--- kernel/sched/syscalls.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/syscalls.c +1 -1
@@ -1321,7 +1321,7 @@ int __sched_setaffinity(struct task_struct *p, struct affinity_context *ctx)bool empty = !cpumask_and (new_mask, new_mask, ctx->user_mask); -if (WARN_ON_ONCE (empty)) +if (empty)cpumask_copy (new_mask, cpus_allowed); }__set_cpus_allowed_ptr (p, ctx);
Signature · 1 line
2.46.0.469.g59c65b2a67-goog
Quoted 27 lines · Josh Don
Commit
8f9ea86fdf99badded some logic tosched_setaffinitythat included a WARN when a per-task affinity assignment races with a cpuset update.Specifically, we can have a race where a cpuset update results in the task affinity no longer being a subset of the cpuset. That's fine; we have a fallback to instead use the cpuset mask. However, we have a WARN set up that will trigger if the cpuset mask has no overlap at all with the requested task affinity. This shouldn't be a warning condition; its trivial to create this condition.
Reproduced the warning by the following setup:
- $PID inside a cpuset cgroup
- another thread repeatedly switching the cpuset cpus from 1-2 to just 1
- another thread repeatedly setting the $PID affinity (via taskset) to 2Fixes 8f9ea86fdf99b ("sched: Always preserve the user requested cpumask")Signed-off-by Josh Don <[email protected]>--- kernel/sched/syscalls.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)kernel/sched/syscalls.c +0
-0
kernel/sched/syscalls.c int __sched_setaffinity(struct task_struct *p, struct affinity_context *ctx) · line 1327
Quoted 8 lines
kernel/sched/syscalls.c +1
-1@@ -1321,7 +1321,7 @@ int __sched_setaffinity(struct task_struct *p, struct affinity_context *ctx)bool empty = !cpumask_and (new_mask, new_mask, ctx->user_mask); -if (WARN_ON_ONCE (empty)) +if (empty)cpumask_copy (new_mask, cpus_allowed); }__set_cpus_allowed_ptr (p, ctx);
Taking out the WARN_ON_ONCE() should be fine.
Hi Josh Don,
Quoted 10 lines · Josh Don
Commit
8f9ea86fdf99badded some logic tosched_setaffinitythat included a WARN when a per-task affinity assignment races with a cpuset update.Specifically, we can have a race where a cpuset update results in the task affinity no longer being a subset of the cpuset. That's fine; we have a fallback to instead use the cpuset mask. However, we have a WARN set up that will trigger if the cpuset mask has no overlap at all with the requested task affinity. This shouldn't be a warning condition; its trivial to create this condition.
Quoted 6 lines
Reproduced the warning by the following setup:
- $PID inside a cpuset cgroup
- another thread repeatedly switching the cpuset cpus from 1-2 to just 1
- another thread repeatedly setting the $PID affinity (via taskset) to 2
I was testing the patch using the following two scripts run concurrently:
Script 1:
while true; do
echo 1 > /sys/fs/cgroup/test_group/cpuset.cpus;
echo 1-2 > /sys/fs/cgroup/test_group/cpuset.cpus;done
Script 2:
while true; do
sudo taskset -p 0x2 $$;
done
However, I am unable to trigger the warning in dmesg on the unpatched kernel. I was expecting to see the warning as described, but it doesn't seem to appear.
Additionally, I also tried the following script to increase the chances of triggering the race condition:
while true; do
echo 1 > /sys/fs/cgroup/test_group/cpuset.cpus;
sudo taskset -p 0x2 $$;
sleep 0.1;
echo 1-2 > /sys/fs/cgroup/test_group/cpuset.cpus;done
Despite this, the warning still does not appear in dmesg.
Am I missing something in my testing approach, or is there a different setup required to reproduce the issue?
Thanks and Regards
Madadi Vineeth Reddy
Quoted 2 lines
Fixes 8f9ea86fdf99b ("sched: Always preserve the user requested cpumask")Signed-off-by Josh Don <[email protected]>
Hi Madadi,
Quoted 45 lines · Madadi Vineeth Reddy
Hi Josh Don,
Quoted 10 lines · Josh Don
Commit
8f9ea86fdf99badded some logic tosched_setaffinitythat included a WARN when a per-task affinity assignment races with a cpuset update.Specifically, we can have a race where a cpuset update results in the task affinity no longer being a subset of the cpuset. That's fine; we have a fallback to instead use the cpuset mask. However, we have a WARN set up that will trigger if the cpuset mask has no overlap at all with the requested task affinity. This shouldn't be a warning condition; its trivial to create this condition.
Quoted 6 lines
Reproduced the warning by the following setup:
- $PID inside a cpuset cgroup
- another thread repeatedly switching the cpuset cpus from 1-2 to just 1
- another thread repeatedly setting the $PID affinity (via taskset) to 2I was testing the patch using the following two scripts run concurrently:
Script 1:
while true; doecho 1 > /sys/fs/cgroup/test_group/cpuset.cpus; echo 1-2 > /sys/fs/cgroup/test_group/cpuset.cpus;done
Script 2:
while true; dosudo taskset -p 0x2 $$;done
However, I am unable to trigger the warning in dmesg on the unpatched kernel. I was expecting to see the warning as described, but it doesn't seem to appear.
Additionally, I also tried the following script to increase the chances of triggering the race condition:
while true; do
echo 1 > /sys/fs/cgroup/test_group/cpuset.cpus; sudo taskset -p 0x2 $$; sleep 0.1; echo 1-2 > /sys/fs/cgroup/test_group/cpuset.cpus;
Quoted 6 lines
done
Despite this, the warning still does not appear in dmesg.
Am I missing something in my testing approach, or is there a different setup required to reproduce the issue?
taskset -p 0x2 $$ will affine to cpu 1 :)
I'd recommend using the '-c' arg to specify the mask as a cpulist, as it is easier to validate.
taskset -c -p 2 $$
Quoted 7 lines
Thanks and Regards
Madadi Vineeth ReddyQuoted 2 lines
Fixes 8f9ea86fdf99b ("sched: Always preserve the user requested cpumask")Signed-off-by Josh Don <[email protected]>
Best,
Josh
Madadi Vineeth Reddy
Tested-byHi Josh Don,
Quoted 56 lines · Josh Don
Hi Madadi,
Quoted 45 lines · Madadi Vineeth Reddy
Hi Josh Don,
Quoted 10 lines · Josh Don
Commit
8f9ea86fdf99badded some logic tosched_setaffinitythat included a WARN when a per-task affinity assignment races with a cpuset update.Specifically, we can have a race where a cpuset update results in the task affinity no longer being a subset of the cpuset. That's fine; we have a fallback to instead use the cpuset mask. However, we have a WARN set up that will trigger if the cpuset mask has no overlap at all with the requested task affinity. This shouldn't be a warning condition; its trivial to create this condition.
Quoted 6 lines
Reproduced the warning by the following setup:
- $PID inside a cpuset cgroup
- another thread repeatedly switching the cpuset cpus from 1-2 to just 1
- another thread repeatedly setting the $PID affinity (via taskset) to 2I was testing the patch using the following two scripts run concurrently:
Script 1:
while true; doecho 1 > /sys/fs/cgroup/test_group/cpuset.cpus; echo 1-2 > /sys/fs/cgroup/test_group/cpuset.cpus;done
Script 2:
while true; dosudo taskset -p 0x2 $$;done
However, I am unable to trigger the warning in dmesg on the unpatched kernel. I was expecting to see the warning as described, but it doesn't seem to appear.
Additionally, I also tried the following script to increase the chances of triggering the race condition:
while true; do
echo 1 > /sys/fs/cgroup/test_group/cpuset.cpus; sudo taskset -p 0x2 $$; sleep 0.1; echo 1-2 > /sys/fs/cgroup/test_group/cpuset.cpus;Quoted 6 lines
done
Despite this, the warning still does not appear in dmesg.
Am I missing something in my testing approach, or is there a different setup required to reproduce the issue?
Quoted 7 lines
taskset -p
0x2$$ will affine to cpu 1 :)I'd recommend using the '-c' arg to specify the mask as a cpulist, as it is easier to validate.
taskset -c -p 2 $$
Thanks for the clarification.
Thanks and Regards
Madadi Vineeth Reddy
Quoted 10 lines
Quoted 7 lines
Thanks and Regards
Madadi Vineeth ReddyQuoted 2 lines
Fixes 8f9ea86fdf99b ("sched: Always preserve the user requested cpumask")Signed-off-by Josh Don <[email protected]>Best,
Josh
Quoted 11 lines · Josh Don
Commit
8f9ea86fdf99badded some logic tosched_setaffinitythat included a WARN when a per-task affinity assignment races with a cpuset update.Specifically, we can have a race where a cpuset update results in the task affinity no longer being a subset of the cpuset. That's fine; we have a fallback to instead use the cpuset mask. However, we have a WARN set up that will trigger if the cpuset mask has no overlap at all with the requested task affinity. This shouldn't be a warning condition; its trivial to create this condition.
Quoted 8 lines
Reproduced the warning by the following setup:
- $PID inside a cpuset cgroup
- another thread repeatedly switching the cpuset cpus from 1-2 to just 1
- another thread repeatedly setting the $PID affinity (via taskset) to 2Fixes 8f9ea86fdf99b ("sched: Always preserve the user requested cpumask")Signed-off-by Josh Don <[email protected]>
Gentle ping to bump this in case it got lost.
I've also collected the following:
Best,
Josh
Vincent Guittot
maintainerQuoted 15 lines · Josh Don
Quoted 13 lines · Josh Don
Commit
8f9ea86fdf99badded some logic tosched_setaffinitythat included a WARN when a per-task affinity assignment races with a cpuset update.Specifically, we can have a race where a cpuset update results in the task affinity no longer being a subset of the cpuset. That's fine; we have a fallback to instead use the cpuset mask. However, we have a WARN set up that will trigger if the cpuset mask has no overlap at all with the requested task affinity. This shouldn't be a warning condition; its trivial to create this condition.
Reproduced the warning by the following setup:
Quoted 8 lines
Quoted 6 lines
- $PID inside a cpuset cgroup
- another thread repeatedly switching the cpuset cpus from 1-2 to just 1
- another thread repeatedly setting the $PID affinity (via taskset) to 2Fixes 8f9ea86fdf99b ("sched: Always preserve the user requested cpumask")Signed-off-by Josh Don <[email protected]>Gentle ping to bump this in case it got lost.
I have also been able to reproduce the race and the WARN with the steps described in the commit description
Quoted 8 lines
I've also collected the following:
Acked-by Waiman Long <[email protected]>Tested-by Madadi Vineeth Reddy <[email protected]>Best,
Josh
Quoted 1 line
Gentle ping to bump this in case it got lost.
Nudge :)
Quoted 4 lines
I've also collected the following:
Acked-by Waiman Long <[email protected]>Tested-by Madadi Vineeth Reddy <[email protected]>
Also adding
Best,
Josh