sched/core: Mask out offline CPUs when user_cpus_ptr is used
PATCH v2 1/1 sched/core: Mask out offline CPUs when user_cpus_ptr is used
Chen Ridong reported that cpuset could report a kernel warning for a task due to set_cpus_allowed_ptr() returning failure in the corner case that:
1) the task used sched_setaffinity(2) to set its CPU affinity mask to be the same as the cpuset.cpus of its cpuset,
2) all the CPUs assigned to that cpuset were taken offline, and
3) cpuset v1 is in use and the task had to be migrated to the top cpuset.
Due to the fact that CPU affinity of the tasks in the top cpuset are not updated when a CPU hotplug online/offline event happens, offline CPUs are included in CPU affinity of those tasks. It is possible that further masking with user_cpus_ptr set by sched_setaffinity(2) in __set_cpus_allowed_ptr() will leave only offline CPUs in the new mask causing the subsequent call to __set_cpus_allowed_ptr_locked() to return failure with an empty CPU affinity.
Fix this failure by skipping user_cpus_ptr masking if there is no online CPU left.
--- kernel/sched/core.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
kernel/sched/core.c +4 -3
@@ -3170,12 +3170,13 @@ int __set_cpus_allowed_ptr(struct task_struct *p, struct affinity_context *ctx) rq =task_rq_lock (p, &rf);/* -* Masking should be skipped if SCA_USER or any of the SCA_MIGRATE_* -* flags are set. +* Masking should be skipped if SCA_USER, any of the SCA_MIGRATE_* +* flags are set or no online CPU left. */ if (p->user_cpus_ptr && !(ctx->flags & (SCA_USER |SCA_MIGRATE_ENABLE |SCA_MIGRATE_DISABLE )) && -cpumask_and (rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr)) +cpumask_and (rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr) && +cpumask_intersects (rq->scratch_mask, cpu_active_mask)) ctx->new_mask = rq->scratch_mask;return __set_cpus_allowed_ptr_locked (p, ctx, rq, &rf);
Signature · 1 line
2.50.0
Quoted 19 lines · Waiman Long
Chen Ridong reported that cpuset could report a kernel warning for a task due to
set_cpus_allowed_ptr()returning failure in the corner case that:1) the task used
sched_setaffinity(2)to set its CPU affinity mask to be the same as the cpuset.cpus of its cpuset,
2) all the CPUs assigned to that cpuset were taken offline, and
3) cpuset v1 is in use and the task had to be migrated to the top cpuset.Due to the fact that CPU affinity of the tasks in the top cpuset are not updated when a CPU hotplug online/offline event happens, offline CPUs are included in CPU affinity of those tasks. It is possible that further masking with
user_cpus_ptrset bysched_setaffinity(2)in__set_cpus_allowed_ptr()will leave only offline CPUs in the new mask causing the subsequent call to__set_cpus_allowed_ptr_locked()to return failure with an empty CPU affinity.Fix this failure by skipping
user_cpus_ptrmasking if there is no online CPU left.
Quoted 7 lines
Reported-by Chen Ridong <[email protected]>Fixes da019032819a ("sched: Enforce user requested affinity")Signed-off-by Waiman Long <[email protected]>--- kernel/sched/core.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
Sorry, I forgot to change the patch title. Will send out a v3.
Cheers,
Longman
Quoted 21 lines
kernel/sched/core.c +4
-3@@ -3170,12 +3170,13 @@ int __set_cpus_allowed_ptr(struct task_struct *p, struct affinity_context *ctx) rq =task_rq_lock (p, &rf);/* -* Masking should be skipped if SCA_USER or any of the SCA_MIGRATE_* -* flags are set. +* Masking should be skipped if SCA_USER, any of the SCA_MIGRATE_* +* flags are set or no online CPU left. */ if (p->user_cpus_ptr && !(ctx->flags & (SCA_USER |SCA_MIGRATE_ENABLE |SCA_MIGRATE_DISABLE )) && -cpumask_and (rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr)) +cpumask_and (rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr) && +cpumask_intersects (rq->scratch_mask, cpu_active_mask)) ctx->new_mask = rq->scratch_mask;return __set_cpus_allowed_ptr_locked (p, ctx, rq, &rf);
PATCH v3 1/1 sched/core: Skip user_cpus_ptr masking if no online CPU left
In reply to Waiman LongChen Ridong reported that cpuset could report a kernel warning for a task due to set_cpus_allowed_ptr() returning failure in the corner case that:
1) the task used sched_setaffinity(2) to set its CPU affinity mask to be the same as the cpuset.cpus of its cpuset,
2) all the CPUs assigned to that cpuset were taken offline, and
3) cpuset v1 is in use and the task had to be migrated to the top cpuset.
Due to the fact that CPU affinity of the tasks in the top cpuset are not updated when a CPU hotplug online/offline event happens, offline CPUs are included in CPU affinity of those tasks. It is possible that further masking with user_cpus_ptr set by sched_setaffinity(2) in __set_cpus_allowed_ptr() will leave only offline CPUs in the new mask causing the subsequent call to __set_cpus_allowed_ptr_locked() to return failure with an empty CPU affinity.
Fix this failure by skipping user_cpus_ptr masking if there is no online CPU left.
--- kernel/sched/core.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
kernel/sched/core.c +4 -3
@@ -3170,12 +3170,13 @@ int __set_cpus_allowed_ptr(struct task_struct *p, struct affinity_context *ctx) rq =task_rq_lock (p, &rf);/* -* Masking should be skipped if SCA_USER or any of the SCA_MIGRATE_* -* flags are set. +* Masking should be skipped if SCA_USER, any of the SCA_MIGRATE_* +* flags are set or no online CPU left. */ if (p->user_cpus_ptr && !(ctx->flags & (SCA_USER |SCA_MIGRATE_ENABLE |SCA_MIGRATE_DISABLE )) && -cpumask_and (rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr)) +cpumask_and (rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr) && +cpumask_intersects (rq->scratch_mask, cpu_active_mask)) ctx->new_mask = rq->scratch_mask;return __set_cpus_allowed_ptr_locked (p, ctx, rq, &rf);
Signature · 1 line
2.50.0
Michal Koutný
maintainerRe: [PATCH v3] sched/core: Skip user_cpus_ptr masking if no online CPU left
Quoted 7 lines · Waiman Long
Chen Ridong reported that cpuset could report a kernel warning for a task due to
set_cpus_allowed_ptr()returning failure in the corner case that:1) the task used
sched_setaffinity(2)to set its CPU affinity mask to be the same as the cpuset.cpus of its cpuset,
2) all the CPUs assigned to that cpuset were taken offline, and
3) cpuset v1 is in use and the task had to be migrated to the top cpuset.
Does this make sense for cpuset v2 (or no cpuset at all for that matter)? I'm asking whether this mask modification could only be extracted into cpuset-v1.c (like cgroup_tranfer_tasks() or a new function)
Thanks,
Michal
Re: [PATCH v3] sched/core: Skip user_cpus_ptr masking if no online CPU left
Quoted 11 lines · Michal Koutný
Quoted 7 lines · Waiman Long
Chen Ridong reported that cpuset could report a kernel warning for a task due to
set_cpus_allowed_ptr()returning failure in the corner case that:1) the task used
sched_setaffinity(2)to set its CPU affinity mask to be the same as the cpuset.cpus of its cpuset,
2) all the CPUs assigned to that cpuset were taken offline, and
3) cpuset v1 is in use and the task had to be migrated to the top cpuset.Does this make sense for cpuset v2 (or no cpuset at all for that matter)? I'm asking whether this mask modification could only be extracted into
cpuset-v1.c(likecgroup_tranfer_tasks()or a new function)
This corner case as specified in Chen Ridong's patch only happens with a cpuset v1 environment, but it is still the case that the default cpu affinity of the root cgroup (with or without CONFIG_CGROUPS) will include offline CPUs, if present. So it still make senses to skip the sched_setaffinity() setting if there is no online CPU left, though it will be much harder to have such a condition without using cpuset v1.
Cheers,
Longman
Michal Koutný
maintainerRe: [PATCH v3] sched/core: Skip user_cpus_ptr masking if no online CPU left
Hi.
I had a look after a while (thanks for reminders Ridong).
Quoted 4 lines · Waiman Long
This corner case as specified in Chen Ridong's patch only happens with a cpuset v1 environment, but it is still the case that the default cpu affinity of the root cgroup (with or without
CONFIG_CGROUPS) will include offline CPUs, if present.
IIUC, the generic sched_setaffinity(2) is ready for that, simply returning an EINVAL.
Quoted 3 lines
So it still make senses to skip the
sched_setaffinity()setting if there is no online CPU left, though it will be much harder to have such a condition without using cpuset v1.
That sounds like there'd be no issue without cpuset v1 and the source of the warning has quite a telling comment:
* fail. TODO: have a better way to handle failure here */ WARN_ON_ONCE(set_cpus_allowed_ptr(task, cpus_attach));
The trouble is that this is from cpuset_attach() (cgroup_subsys.attach) where no errors are expected. So I'd say the place for the check should be earlier in cpuset_can_attach() [1]. I'm not sure if that's universally immune against cpu offlining but it'd be sufficient for the reported sequential offlining.
HTH,
Michal
[1] Although the error propagates, it ends up without recovery in remove_tasks_in_empty_cpuset() "only" as an error message. But that's likely all what can be done in this workfn context -- it's better than silently skipping the migration as consequence of this patch.
Re: [PATCH v3] sched/core: Skip user_cpus_ptr masking if no online CPU left
Quoted 11 lines · Michal Koutný
Hi.
I had a look after a while (thanks for reminders Ridong).
Quoted 4 lines · Waiman Long
This corner case as specified in Chen Ridong's patch only happens with a cpuset v1 environment, but it is still the case that the default cpu affinity of the root cgroup (with or without
CONFIG_CGROUPS) will include offline CPUs, if present.IIUC, the generic
sched_setaffinity(2)is ready for that, simply returning an EINVAL.
The modified code will not be executed when called from sched_setaffiity() as the SCA_USER flag will be set.
In the described scenario, sched_setaffinity() was called without failure as the request was valid at the time.
Quoted 11 lines
Quoted 4 lines
So it still make senses to skip the
sched_setaffinity()setting if there is no online CPU left, though it will be much harder to have such a condition without using cpuset v1.That sounds like there'd be no issue without cpuset v1 and the source of the warning has quite a telling comment:
* fail. TODO: have a better way to handle failure here */ WARN_ON_ONCE(set_cpus_allowed_ptr(task, cpus_attach));
Quoted 5 lines
The trouble is that this is from
cpuset_attach()(cgroup_subsys.attach) where no errors are expected. So I'd say the place for the check should be earlier incpuset_can_attach()[1]. I'm not sure if that's universally immune against cpu offlining but it'd be sufficient for the reported sequential offlining.
Cpuset1 has no concept of effective cpumask that excludes offline CPUs unless "cpuset_v2_mode" mount option is used. So when the cpuset has no CPU left, it will force migrate the tasks to its parent and the __set_cpus_allowed_ptr() function will be invoked. The parent will likely have those offline CPUs in their cpus_allowed list and __set_cpus_allowed_ptr_locked() will be called with only the offline CPUs causing the warning. Migrating to the top_cpuset is probably not needed to illustrate the problem.
Cheers,
Longman
Quoted 7 lines
HTH,
Michal[1] Although the error propagates, it ends up without recovery in
remove_tasks_in_empty_cpuset()"only" as an error message. But that's likely all what can be done in this workfn context -- it's better than silently skipping the migration as consequence of this patch.
Re: [PATCH v3] sched/core: Skip user_cpus_ptr masking if no online CPU left
In reply to Waiman LongQuoted 38 lines · Waiman Long
Chen Ridong reported that cpuset could report a kernel warning for a task due to
set_cpus_allowed_ptr()returning failure in the corner case that:1) the task used
sched_setaffinity(2)to set its CPU affinity mask to be the same as the cpuset.cpus of its cpuset,
2) all the CPUs assigned to that cpuset were taken offline, and
3) cpuset v1 is in use and the task had to be migrated to the top cpuset.Due to the fact that CPU affinity of the tasks in the top cpuset are not updated when a CPU hotplug online/offline event happens, offline CPUs are included in CPU affinity of those tasks. It is possible that further masking with
user_cpus_ptrset bysched_setaffinity(2)in__set_cpus_allowed_ptr()will leave only offline CPUs in the new mask causing the subsequent call to__set_cpus_allowed_ptr_locked()to return failure with an empty CPU affinity.Fix this failure by skipping
user_cpus_ptrmasking if there is no online CPU left.Reported-by Chen Ridong <[email protected]>Fixes da019032819a ("sched: Enforce user requested affinity")Signed-off-by Waiman Long <[email protected]>--- kernel/sched/core.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)kernel/sched/core.c +1
-2@@ -3170,12 +3170,13 @@ int __set_cpus_allowed_ptr(struct task_struct *p, struct affinity_context *ctx) rq =task_rq_lock (p, &rf);/* -* Masking should be skipped if SCA_USER or any of the SCA_MIGRATE_* -* flags are set. +* Masking should be skipped if SCA_USER, any of the SCA_MIGRATE_*
kernel/sched/core.c int __set_cpus_allowed_ptr(struct task_struct *p, struct affinity_context *ctx) · line 3182
Quoted 10 lines
kernel/sched/core.c +3
-1+* flags are set or no online CPU left. */ if (p->user_cpus_ptr && !(ctx->flags & (SCA_USER |SCA_MIGRATE_ENABLE |SCA_MIGRATE_DISABLE )) && -cpumask_and (rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr)) +cpumask_and (rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr) && +cpumask_intersects (rq->scratch_mask, cpu_active_mask)) ctx->new_mask = rq->scratch_mask;return __set_cpus_allowed_ptr_locked (p, ctx, rq, &rf);
Re: [PATCH v3] sched/core: Skip user_cpus_ptr masking if no online CPU left
Quoted 50 lines · Chen Ridong
Quoted 47 lines · Waiman Long
Chen Ridong reported that cpuset could report a kernel warning for a task due to
set_cpus_allowed_ptr()returning failure in the corner case that:1) the task used
sched_setaffinity(2)to set its CPU affinity mask to be the same as the cpuset.cpus of its cpuset,
2) all the CPUs assigned to that cpuset were taken offline, and
3) cpuset v1 is in use and the task had to be migrated to the top cpuset.Due to the fact that CPU affinity of the tasks in the top cpuset are not updated when a CPU hotplug online/offline event happens, offline CPUs are included in CPU affinity of those tasks. It is possible that further masking with
user_cpus_ptrset bysched_setaffinity(2)in__set_cpus_allowed_ptr()will leave only offline CPUs in the new mask causing the subsequent call to__set_cpus_allowed_ptr_locked()to return failure with an empty CPU affinity.Fix this failure by skipping
user_cpus_ptrmasking if there is no online CPU left.Reported-by Chen Ridong <[email protected]>Fixes da019032819a ("sched: Enforce user requested affinity")Signed-off-by Waiman Long <[email protected]>--- kernel/sched/core.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)kernel/sched/core.c +4
-3@@ -3170,12 +3170,13 @@ int __set_cpus_allowed_ptr(struct task_struct *p, struct affinity_context *ctx) rq =task_rq_lock (p, &rf);/* -* Masking should be skipped if SCA_USER or any of the SCA_MIGRATE_* -* flags are set. +* Masking should be skipped if SCA_USER, any of the SCA_MIGRATE_* +* flags are set or no online CPU left. */ if (p->user_cpus_ptr && !(ctx->flags & (SCA_USER |SCA_MIGRATE_ENABLE |SCA_MIGRATE_DISABLE )) && -cpumask_and (rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr)) +cpumask_and (rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr) && +cpumask_intersects (rq->scratch_mask, cpu_active_mask)) ctx->new_mask = rq->scratch_mask;
Quoted 4 lines
kernel/sched/core.c int __set_cpus_allowed_ptr(struct task_struct *p, struct affinity_context *ctx) · line 3182
Quoted 1 line
kernel/sched/core.c +0
-0return __set_cpus_allowed_ptr_locked (p, ctx, rq, &rf);Tested-by Chen Ridong <[email protected]>
Friendly ping.
Best regards,
Ridong
Re: [PATCH v3] sched/core: Skip user_cpus_ptr masking if no online CPU left
Quoted 58 lines · Chen Ridong
Quoted 50 lines · Chen Ridong
Quoted 47 lines · Waiman Long
Chen Ridong reported that cpuset could report a kernel warning for a task due to
set_cpus_allowed_ptr()returning failure in the corner case that:1) the task used
sched_setaffinity(2)to set its CPU affinity mask to be the same as the cpuset.cpus of its cpuset,
2) all the CPUs assigned to that cpuset were taken offline, and
3) cpuset v1 is in use and the task had to be migrated to the top cpuset.Due to the fact that CPU affinity of the tasks in the top cpuset are not updated when a CPU hotplug online/offline event happens, offline CPUs are included in CPU affinity of those tasks. It is possible that further masking with
user_cpus_ptrset bysched_setaffinity(2)in__set_cpus_allowed_ptr()will leave only offline CPUs in the new mask causing the subsequent call to__set_cpus_allowed_ptr_locked()to return failure with an empty CPU affinity.Fix this failure by skipping
user_cpus_ptrmasking if there is no online CPU left.Reported-by Chen Ridong <[email protected]>Fixes da019032819a ("sched: Enforce user requested affinity")Signed-off-by Waiman Long <[email protected]>--- kernel/sched/core.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)kernel/sched/core.c +4
-3@@ -3170,12 +3170,13 @@ int __set_cpus_allowed_ptr(struct task_struct *p, struct affinity_context *ctx) rq =task_rq_lock (p, &rf);/* -* Masking should be skipped if SCA_USER or any of the SCA_MIGRATE_* -* flags are set. +* Masking should be skipped if SCA_USER, any of the SCA_MIGRATE_* +* flags are set or no online CPU left. */ if (p->user_cpus_ptr && !(ctx->flags & (SCA_USER |SCA_MIGRATE_ENABLE |SCA_MIGRATE_DISABLE )) && -cpumask_and (rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr)) +cpumask_and (rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr) && +cpumask_intersects (rq->scratch_mask, cpu_active_mask)) ctx->new_mask = rq->scratch_mask;Quoted 4 lines
kernel/sched/core.c int __set_cpus_allowed_ptr(struct task_struct *p, struct affinity_context *ctx) · line 3182
Quoted 1 line
kernel/sched/core.c +0
-0return __set_cpus_allowed_ptr_locked (p, ctx, rq, &rf);Tested-by Chen Ridong <[email protected]>
Quoted 5 lines
Friendly ping.
Best regards,
Ridong
Could someone please review this patch?
Signature · 2 lines
Best regards, Ridong
PATCH RESEND v3 1/1 sched/core: Skip user_cpus_ptr masking if no online CPU left
In reply to Waiman LongChen Ridong reported that cpuset could report a kernel warning for a task due to set_cpus_allowed_ptr() returning failure in the corner case that:
1) the task used sched_setaffinity(2) to set its CPU affinity mask to be the same as the cpuset.cpus of its cpuset,
2) all the CPUs assigned to that cpuset were taken offline, and
3) cpuset v1 is in use and the task had to be migrated to top_cpuset. Task migration is not needed for cpuset v2.
Due to the fact that CPU affinity of the tasks in the top cpuset are not updated when a CPU hotplug online/offline event happens, offline CPUs are included in CPU affinity of those tasks. It is possible that further masking with user_cpus_ptr set by sched_setaffinity(2) in __set_cpus_allowed_ptr() will leave only offline CPUs in the new mask causing the subsequent call to __set_cpus_allowed_ptr_locked() to return failure with an empty CPU affinity.
Fix this failure by skipping user_cpus_ptr masking if there is no online CPU left.
--- kernel/sched/core.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
kernel/sched/core.c +4 -3
@@ -3146,12 +3146,13 @@ int __set_cpus_allowed_ptr(struct task_struct *p, struct affinity_context *ctx) rq =task_rq_lock (p, &rf);/* -* Masking should be skipped if SCA_USER or any of the SCA_MIGRATE_* -* flags are set. +* Masking should be skipped if SCA_USER, any of the SCA_MIGRATE_* +* flags are set or no online CPU left. */ if (p->user_cpus_ptr && !(ctx->flags & (SCA_USER |SCA_MIGRATE_ENABLE |SCA_MIGRATE_DISABLE )) && -cpumask_and (rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr)) +cpumask_and (rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr) && +cpumask_intersects (rq->scratch_mask, cpu_active_mask)) ctx->new_mask = rq->scratch_mask;return __set_cpus_allowed_ptr_locked (p, ctx, rq, &rf);
Signature · 1 line
2.51.0
Re: [RESEND PATCH v3] sched/core: Skip user_cpus_ptr masking if no online CPU left
Hi Waiman.
On Wed, Oct 29, 2025 at 05:27:24PM -0400, Waiman Long <[email protected]> wrote:
...
Quoted 7 lines
Reported-by Chen Ridong <[email protected]>Fixes da019032819a ("sched: Enforce user requested affinity")Signed-off-by Waiman Long <[email protected]>--- kernel/sched/core.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
(I had looked at v2 back then and reached the conclusion such a fix had made sense.)
Re: [RESEND PATCH v3] sched/core: Skip user_cpus_ptr masking if no online CPU left
In reply to Waiman LongQuoted 39 lines · Waiman Long
Chen Ridong reported that cpuset could report a kernel warning for a task due to
set_cpus_allowed_ptr()returning failure in the corner case that:1) the task used
sched_setaffinity(2)to set its CPU affinity mask to be the same as the cpuset.cpus of its cpuset,
2) all the CPUs assigned to that cpuset were taken offline, and
3) cpuset v1 is in use and the task had to be migrated totop_cpuset. Task migration is not needed for cpuset v2.Due to the fact that CPU affinity of the tasks in the top cpuset are not updated when a CPU hotplug online/offline event happens, offline CPUs are included in CPU affinity of those tasks. It is possible that further masking with
user_cpus_ptrset bysched_setaffinity(2)in__set_cpus_allowed_ptr()will leave only offline CPUs in the new mask causing the subsequent call to__set_cpus_allowed_ptr_locked()to return failure with an empty CPU affinity.Fix this failure by skipping
user_cpus_ptrmasking if there is no online CPU left.Reported-by Chen Ridong <[email protected]>Fixes da019032819a ("sched: Enforce user requested affinity")Signed-off-by Waiman Long <[email protected]>--- kernel/sched/core.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)kernel/sched/core.c +1
-2@@ -3146,12 +3146,13 @@ int __set_cpus_allowed_ptr(struct task_struct *p, struct affinity_context *ctx) rq =task_rq_lock (p, &rf);/* -* Masking should be skipped if SCA_USER or any of the SCA_MIGRATE_* -* flags are set. +* Masking should be skipped if SCA_USER, any of the SCA_MIGRATE_*
kernel/sched/core.c int __set_cpus_allowed_ptr(struct task_struct *p, struct affinity_context *ctx) · line 3158
Quoted 10 lines
kernel/sched/core.c +3
-1+* flags are set or no online CPU left. */ if (p->user_cpus_ptr && !(ctx->flags & (SCA_USER |SCA_MIGRATE_ENABLE |SCA_MIGRATE_DISABLE )) && -cpumask_and (rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr)) +cpumask_and (rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr) && +cpumask_intersects (rq->scratch_mask, cpu_active_mask)) ctx->new_mask = rq->scratch_mask;return __set_cpus_allowed_ptr_locked (p, ctx, rq, &rf);
This patch have been tested with our test project.
Signature · 2 lines
Best regards, Ridong
Re: [RESEND PATCH v3] sched/core: Skip user_cpus_ptr masking if no online CPU left
In reply to Waiman LongQuoted 39 lines · Waiman Long
Chen Ridong reported that cpuset could report a kernel warning for a task due to
set_cpus_allowed_ptr()returning failure in the corner case that:1) the task used
sched_setaffinity(2)to set its CPU affinity mask to be the same as the cpuset.cpus of its cpuset,
2) all the CPUs assigned to that cpuset were taken offline, and
3) cpuset v1 is in use and the task had to be migrated totop_cpuset. Task migration is not needed for cpuset v2.Due to the fact that CPU affinity of the tasks in the top cpuset are not updated when a CPU hotplug online/offline event happens, offline CPUs are included in CPU affinity of those tasks. It is possible that further masking with
user_cpus_ptrset bysched_setaffinity(2)in__set_cpus_allowed_ptr()will leave only offline CPUs in the new mask causing the subsequent call to__set_cpus_allowed_ptr_locked()to return failure with an empty CPU affinity.Fix this failure by skipping
user_cpus_ptrmasking if there is no online CPU left.Reported-by Chen Ridong <[email protected]>Fixes da019032819a ("sched: Enforce user requested affinity")Signed-off-by Waiman Long <[email protected]>--- kernel/sched/core.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)kernel/sched/core.c +1
-2@@ -3146,12 +3146,13 @@ int __set_cpus_allowed_ptr(struct task_struct *p, struct affinity_context *ctx) rq =task_rq_lock (p, &rf);/* -* Masking should be skipped if SCA_USER or any of the SCA_MIGRATE_* -* flags are set. +* Masking should be skipped if SCA_USER, any of the SCA_MIGRATE_*
kernel/sched/core.c int __set_cpus_allowed_ptr(struct task_struct *p, struct affinity_context *ctx) · line 3158
Quoted 10 lines
kernel/sched/core.c +3
-1+* flags are set or no online CPU left. */ if (p->user_cpus_ptr && !(ctx->flags & (SCA_USER |SCA_MIGRATE_ENABLE |SCA_MIGRATE_DISABLE )) && -cpumask_and (rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr)) +cpumask_and (rq->scratch_mask, ctx->new_mask, p->user_cpus_ptr) && +cpumask_intersects (rq->scratch_mask, cpu_active_mask)) ctx->new_mask = rq->scratch_mask;return __set_cpus_allowed_ptr_locked (p, ctx, rq, &rf);
Hi, Peter,
This is an issue that should be addressed. Should we proceed with applying this patch, or is more review needed?
Signature · 2 lines
Best regards, Ridong