cgroup/cpuset: Miscellaneous fixes and cleanups
Subsystems (4)
Waiman Long
maintainerv2 cover cgroup/cpuset: Miscellaneous fixes and cleanups
v2:
- Update patch 1 as suggested by Ridong Chen and add new test cases.
- Minor update to patch 2 code and comment log.
Patch 1 updates compute_effective_cpumask() and adds new compute_effective_nodemask() helper to make sure that effective_cpus and effective_mems will inherit parent's versions for v2 if cpuset.cpus/cpuset.mems is empty.
Patch 2 makes cpuset_update_tasks_nodemask() to perform memory rebind and migration only for thread group leader like cpuset_attach().
Waiman Long (2):
cgroup/cpuset: Avoid unnecessary cpus & mems update in
cpuset_hotplug_update_tasks()
cgroup/cpuset: Rebind/migrate mm only for threadgroup leader in
cpuset_update_tasks_nodemask()
Documentation/admin-guide/cgroup-v2.rst | 7 +++ kernel/cgroup/cpuset.c | 49 ++++++++++++------- .../selftests/cgroup/test_cpuset_prs.sh | 11 ++++- 3 files changed, 46 insertions(+), 21 deletions(-)
Signature · 1 line
2.54.0
Waiman Long
maintainerPATCH v2 1/2 cgroup/cpuset: Avoid unnecessary cpus & mems update in cpuset_hotplug_update_tasks()
As reported by sashiko [1], cpuset_hotplug_update_tasks() may perform unnecessary task iteration and updating of tasks' CPU and node masks when mems_allowed and/or cpus_allowed are not set in cpuset v2. It is due to the fact that the temporary new_cpus and new_mems masks do not inherit parent's effective_cpus/mems when they are empty which is the expected behavior for cpuset v2 since commit 4ec22e9c5a90 ("cpuset:
Enable cpuset controller in default hierarchy").
Fix that and avoid unnecessay work by enhancing compute_effective_cpumask() to add the empty cpumask check and inheriting the parent's versions if empty when in v2. A new compute_effective_nodemask() helper is also added to perform similar function for new effective_mems.
Add new test_cpuset_prs.sh test cases to confirm that effective_cpus will inherit the parent's version if cpuset.cpus is empty.
[1] https://sashiko.dev/#/patchset/20260621032816.1806773-1-longman%40redhat.com
--- kernel/cgroup/cpuset.c | 45 +++++++++++-------- .../selftests/cgroup/test_cpuset_prs.sh | 11 ++++- 2 files changed, 35 insertions(+), 21 deletions(-)
kernel/cgroup/cpuset.c +26 -19
@@ -1094,12 +1094,35 @@ void cpuset_update_tasks_cpumask(struct cpuset *cs, struct cpumask *new_cpus)* @cs: the cpuset the need to recompute the new effective_cpus mask * @parent: the parent cpuset * +* For v2, the parent's effective_cpus is inherited if cpumask empty. * The result is valid only if the given cpuset isn't a partition root. */ static void compute_effective_cpumask (struct cpumask *new_cpus,struct cpuset *cs,struct cpuset *parent) { -cpumask_and (new_cpus, cs->cpus_allowed, parent->effective_cpus); +bool has_cpus; + + has_cpus =cpumask_and (new_cpus, cs->cpus_allowed, parent->effective_cpus); +if (!has_cpus &&is_in_v2_mode ()) +cpumask_copy (new_cpus, parent->effective_cpus); +} + +/** +* compute_effective_nodemask - Compute the effective nodemask of the cpuset +* @new_cpus: the temp variable for the new effective_mems mask +* @cs: the cpuset the need to recompute the new effective_mems mask +* @parent: the parent cpuset +* +* For v2, the parent's effective_mems is inherited if nodemask empty. +*/ +static void compute_effective_nodemask (nodemask_t *new_mems, +struct cpuset *cs,struct cpuset *parent) +{ +bool has_mems; + + has_mems =nodes_and (*new_mems, cs->mems_allowed, parent->effective_mems); +if (!has_mems &&is_in_v2_mode ()) +nodes_copy (*new_mems, parent->effective_mems); }/*
@@ -2148,15 +2171,6 @@ static void update_cpumasks_hier(struct cpuset *cs, struct tmpmasks *tmp, goto update_parent_effective; } -/* -* If it becomes empty, inherit the effective mask of the -* parent, which is guaranteed to have some CPUs unless -* it is a partition root that has explicitly distributed -* out all its CPUs. -*/ -if (is_in_v2_mode () && !remote &&cpumask_empty (tmp->new_cpus)) -cpumask_copy (tmp->new_cpus, parent->effective_cpus); -/* * Skip the whole subtree if * 1) the cpumask remains the same,
@@ -2704,14 +2718,7 @@ static void update_nodemasks_hier(struct cpuset *cs, nodemask_t *new_mems)cpuset_for_each_descendant_pre (cp, pos_css, cs) {struct cpuset *parent =parent_cs (cp); -bool has_mems =nodes_and (*new_mems, cp->mems_allowed, parent->effective_mems); - -/* -* If it becomes empty, inherit the effective mask of the -* parent, which is guaranteed to have some MEMs. -*/ -if (is_in_v2_mode () && !has_mems) - *new_mems = parent->effective_mems; +compute_effective_nodemask (new_mems, cp, parent);/* Skip the whole subtree if the nodemask remains the same. */ if (nodes_equal (*new_mems, cp->effective_mems)) {
@@ -3923,7 +3930,7 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp) parent =parent_cs (cs);compute_effective_cpumask (&new_cpus, cs, parent); -nodes_and (new_mems, cs->mems_allowed, parent->effective_mems); +compute_effective_nodemask (&new_mems, cs, parent);if (!tmp || !cs->partition_root_state) goto update_tasks;
tools/testing/selftests/cgroup/test_cpuset_prs.sh +9 -2
@@ -495,13 +495,20 @@ REMOTE_TEST_MATRIX=( # Narrowing cpuset.cpus to previously sibling-excluded CPUs should # not return CPUs that were never actually owned. " C1-4:P1 . C1-2:P1 C1-3:P2 . . \ - . . . C3 . . p1:4|c11:1-2|c12:3 \ + . . . C3 . . p1:4|c11:1-2|c12:3 \ p1:P1|c11:P1|c12:P2 3" # Expanding cpuset.cpus to include a previously sibling-excluded CPU # after the sibling has become a member should correctly request it. " C1-4:P1 . C1-2:P1 C1-3:P2 . . \ - . . P0 C2-3 . . p1:1,4|c11:1|c12:2-3 \ + . . P0 C2-3 . . p1:1,4|c11:1|c12:2-3 \ p1:P1|c11:P0|c12:P2 2-3" + # Cpusets with empty cpuset.cpus should inherit parent's effective_cpus + " C1-4:P1 C5-6 C1-2 . C5 . \ + . P1 P1 . . . p1:3-4|p2:5-6|c11:1-2|c12:3-4|c21:5|c22:5-6 \ + p1:P1|p2:P1|c11:P1" + " C1-4:P1 C5-6 C1-2 . C5 . \ + . P1 P1 . O5=0 . p1:3-4|p2:6|c11:1-2|c12:3-4|c21:6|c22:6 \ + p1:P1|p2:P1|c11:P1" ) #
Signature · 1 line
2.54.0
Re: [PATCH v2 1/2] cgroup/cpuset: Avoid unnecessary cpus & mems update in cpuset_hotplug_update_tasks()
Quoted 126 lines · Waiman Long
As reported by sashiko [1],
cpuset_hotplug_update_tasks()may perform unnecessary task iteration and updating of tasks' CPU and node masks whenmems_allowedand/orcpus_allowedare not set in cpuset v2. It is due to the fact that the temporarynew_cpusandnew_memsmasks do not inherit parent's effective_cpus/mems when they are empty which is the expected behavior for cpuset v2 since commit4ec22e9c5a90("cpuset:
Enable cpuset controller in default hierarchy").Fix that and avoid unnecessay work by enhancing
compute_effective_cpumask()to add the empty cpumask check and inheriting the parent's versions if empty when in v2. A newcompute_effective_nodemask()helper is also added to perform similar function for neweffective_mems.Add new
test_cpuset_prs.shtest cases to confirm thateffective_cpuswill inherit the parent's version if cpuset.cpus is empty.[1] https://sashiko.dev/#/patchset/20260621032816.1806773-1-longman%40redhat.com
Suggested-by Ridong Chen <[email protected]>Fixes 4ec22e9c5a90 ("cpuset: Enable cpuset controller in default hierarchy")Signed-off-by Waiman Long <[email protected]>--- kernel/cgroup/cpuset.c | 45 +++++++++++-------- .../selftests/cgroup/test_cpuset_prs.sh | 11 ++++- 2 files changed, 35 insertions(+), 21 deletions(-)kernel/cgroup/cpuset.c +26
-19@@ -1094,12 +1094,35 @@ void cpuset_update_tasks_cpumask(struct cpuset *cs, struct cpumask *new_cpus)* @cs: the cpuset the need to recompute the new effective_cpus mask * @parent: the parent cpuset * +* For v2, the parent's effective_cpus is inherited if cpumask empty. * The result is valid only if the given cpuset isn't a partition root. */ static void compute_effective_cpumask (struct cpumask *new_cpus,struct cpuset *cs,struct cpuset *parent) { -cpumask_and (new_cpus, cs->cpus_allowed, parent->effective_cpus); +bool has_cpus; + + has_cpus =cpumask_and (new_cpus, cs->cpus_allowed, parent->effective_cpus); +if (!has_cpus &&is_in_v2_mode ()) +cpumask_copy (new_cpus, parent->effective_cpus); +} + +/** +* compute_effective_nodemask - Compute the effective nodemask of the cpuset +* @new_cpus: the temp variable for the new effective_mems mask +* @cs: the cpuset the need to recompute the new effective_mems mask +* @parent: the parent cpuset +* +* For v2, the parent's effective_mems is inherited if nodemask empty. +*/ +static void compute_effective_nodemask (nodemask_t *new_mems, +struct cpuset *cs,struct cpuset *parent) +{ +bool has_mems; + + has_mems =nodes_and (*new_mems, cs->mems_allowed, parent->effective_mems); +if (!has_mems &&is_in_v2_mode ()) +nodes_copy (*new_mems, parent->effective_mems); }/* @@ -2148,15 +2171,6 @@ static void update_cpumasks_hier(struct cpuset *cs, struct tmpmasks *tmp, goto update_parent_effective; } -/* -* If it becomes empty, inherit the effective mask of the -* parent, which is guaranteed to have some CPUs unless -* it is a partition root that has explicitly distributed -* out all its CPUs. -*/ -if (is_in_v2_mode () && !remote &&cpumask_empty (tmp->new_cpus)) -cpumask_copy (tmp->new_cpus, parent->effective_cpus); -/* * Skip the whole subtree if * 1) the cpumask remains the same, @@ -2704,14 +2718,7 @@ static void update_nodemasks_hier(struct cpuset *cs, nodemask_t *new_mems)cpuset_for_each_descendant_pre (cp, pos_css, cs) {struct cpuset *parent =parent_cs (cp); -bool has_mems =nodes_and (*new_mems, cp->mems_allowed, parent->effective_mems); - -/* -* If it becomes empty, inherit the effective mask of the -* parent, which is guaranteed to have some MEMs. -*/ -if (is_in_v2_mode () && !has_mems) - *new_mems = parent->effective_mems; +compute_effective_nodemask (new_mems, cp, parent);/* Skip the whole subtree if the nodemask remains the same. */ if (nodes_equal (*new_mems, cp->effective_mems)) {@@ -3923,7 +3930,7 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp) parent =parent_cs (cs);compute_effective_cpumask (&new_cpus, cs, parent); -nodes_and (new_mems, cs->mems_allowed, parent->effective_mems); +compute_effective_nodemask (&new_mems, cs, parent);if (!tmp || !cs->partition_root_state) goto update_tasks;tools/testing/selftests/cgroup/test_cpuset_prs.sh +2
-2@@ -495,13 +495,20 @@ REMOTE_TEST_MATRIX=( # Narrowing cpuset.cpus to previously sibling-excluded CPUs should # not return CPUs that were never actually owned. " C1-4:P1 . C1-2:P1 C1-3:P2 . . \ - . . . C3 . . p1:4|c11:1-2|c12:3 \ + . . . C3 . . p1:4|c11:1-2|c12:3 \ p1:P1|c11:P1|c12:P2 3" # Expanding cpuset.cpus to include a previously sibling-excluded CPU # after the sibling has become a member should correctly request it. " C1-4:P1 . C1-2:P1 C1-3:P2 . . \ - . . P0 C2-3 . . p1:1,4|c11:1|c12:2-3 \ + . . P0 C2-3 . . p1:1,4|c11:1|c12:2-3 \ p1:P1|c11:P0|c12:P2 2-3"
tools/testing/selftests/cgroup/test_cpuset_prs.sh REMOTE_TEST_MATRIX=( · line 514
Quoted 10 lines
tools/testing/selftests/cgroup/test_cpuset_prs.sh +7
-0+ # Cpusets with empty cpuset.cpus should inherit parent's effective_cpus + " C1-4:P1 C5-6 C1-2 . C5 . \ + . P1 P1 . . . p1:3-4|p2:5-6|c11:1-2|c12:3-4|c21:5|c22:5-6 \ + p1:P1|p2:P1|c11:P1" + " C1-4:P1 C5-6 C1-2 . C5 . \ + . P1 P1 . O5=0 . p1:3-4|p2:6|c11:1-2|c12:3-4|c21:6|c22:6 \ + p1:P1|p2:P1|c11:P1" ) #
LGTM.
Signature · 2 lines
Best regards Ridong
Re: [PATCH v2 1/2] cgroup/cpuset: Avoid unnecessary cpus & mems update in cpuset_hotplug_update_tasks()
In reply to Waiman LongQuoted 8 lines · Waiman Long
As reported by sashiko [1],
cpuset_hotplug_update_tasks()may perform unnecessary task iteration and updating of tasks' CPU and node masks whenmems_allowedand/orcpus_allowedare not set in cpuset v2. It is due to the fact that the temporarynew_cpusandnew_memsmasks do not inherit parent's effective_cpus/mems when they are empty which is the expected behavior for cpuset v2 since commit4ec22e9c5a90("cpuset:
Enable cpuset controller in default hierarchy").
Quoted 5 lines
Fix that and avoid unnecessay work by enhancing
compute_effective_cpumask()to add the empty cpumask check and inheriting the parent's versions if empty when in v2. A newcompute_effective_nodemask()helper is also added to perform similar function for neweffective_mems.
perform a similar function
or
perform similar functions
kernel/cgroup/cpuset.c void cpuset_update_tasks_cpumask(struct cpuset *cs, struct · line 1098
Quoted 11 lines
[...]
kernel/cgroup/cpuset.c +1
-0@@ -1094,12 +1094,35 @@ void cpuset_update_tasks_cpumask(struct cpuset *cs, struct cpumask *new_cpus) * @cs: the cpuset the need to recompute the new effective_cpus mask * @parent: the parent cpuset * + * For v2, the parent's effective_cpus is inherited if cpumask empty.
+ * For v2, the parent's effective_cpus is inherited if cpumask is empty.
Thanks
Manuel
Waiman Long
maintainerPATCH v2 2/2 cgroup/cpuset: Rebind/migrate mm only for threadgroup leader in cpuset_update_tasks_nodemask()
In reply to Waiman LongAs reported by sashiko [1], cpuset_update_tasks_nodemask() will do mpol_rebind_mm() and possibly cpuset_migrate_mm() for all threads of a multithreaded process. Since commit 3df9ca0a2b8b ("cpuset: migrate memory only for threadgroup leaders"), cpuset_attach() had been updated to rebind and migrate memory only for threadgroup leaders to mark the group leader as the owner of the mm_struct.
To be consistent and avoid unnecessary performance overhead for heavily multithreaded processes, follow the cpuset_attach() example and perform memory rebind and migration only for threadgroup leaders.
Also add a paragraph in cgroup-v2.rst under cpuset.mems that the threadgroup leader is the memory owner of that threadgroup. Therefore the non-leading threads shouldn't be in other cgroups whose "cpuset.mems" doesn't fully overlap that of the group leader.
[1] https://sashiko.dev/#/patchset/20260621032816.1806773-1-longman%40redhat.com
--- Documentation/admin-guide/cgroup-v2.rst | 7 +++++++ kernel/cgroup/cpuset.c | 4 ++++ 2 files changed, 11 insertions(+)
Documentation/admin-guide/cgroup-v2.rst +7 -0
@@ -2527,6 +2527,13 @@ Cpuset Interface Files a need to change "cpuset.mems" with active tasks, it shouldn't be done frequently. + For a multithreaded process, the threadgroup leader is + considered the owner of the group's memory. Memory policy + rebinding and migration will only happen with respect to the + threadgroup leader. To avoid unexpected result, non-leading + threads shouldn't be put into another cgroup whose "cpuset.mems" + doesn't fully overlap that of the threadgroup leader. + cpuset.mems.effective A read-only multiple values file which exists on all cpuset-enabled cgroups.
kernel/cgroup/cpuset.c +4 -0
@@ -2673,6 +2673,10 @@ void cpuset_update_tasks_nodemask(struct cpuset *cs)cpuset_change_task_nodemask (task, &newmems); +/* Rebind and migrate mm only for thread group leader */ +if (!thread_group_leader (task)) +continue ; + mm =get_task_mm (task);if (!mm)continue ;
Signature · 1 line
2.54.0
A message that is not in the archive (sent off-list or not yet received): [email protected]
Waiman Long
maintainerRe: [PATCH v2 2/2] cgroup/cpuset: Rebind/migrate mm only for threadgroup leader in cpuset_update_tasks_nodemask()
Quoted 65 lines · On 6/23/26 7:18 PM, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
New issues:
- [High]cpuset_update_tasks_nodemask()silently fails to migrate pages or rebind memory policies for multithreaded processes if the thread group leader has exited.Pre-existing issues:
- [High]cpuset_attach()fails to migrate or rebind memory if the thread group leader has exited. --commit
3c37b1f067b61f1adb460635452e2f060088cb23
Author: Waiman Long <[email protected]>cgroup/cpuset: Rebind/migrate mm only for threadgroup leader in
cpuset_update_tasks_nodemask()This patch limits memory policy rebinding and migration in
cpuset_update_tasks_nodemask()to only the threadgroup leader of a multithreaded process. It aims to reduce performance overhead by avoiding duplicate work for every thread.[ ... ]
Quoted 5 lines
kernel/cgroup/cpuset.c +0
-0kernel/cgroup/cpuset.c void cpuset_update_tasks_nodemask(struct cpuset *cs) · line 2670
Quoted 10 lines
kernel/cgroup/cpuset.c +4
-0@@ -2661,6 +2661,10 @@ void cpuset_update_tasks_nodemask(struct cpuset *cs)cpuset_change_task_nodemask (task, &newmems); +/* Rebind and migrate mm only for thread group leader */ +if (!thread_group_leader (task)) +continue ; + mm =get_task_mm (task);if (!mm)continue ;[Severity: High]
Does this change silently fail to migrate pages or rebind memory policies if the thread group leader has exited?If a user creates a multithreaded process, forces its main thread to exit (e.g., via
pthread_exit()), and continues executing work in the surviving worker threads, the main thread becomes a zombie.When cpuset.mems is updated, this new check will explicitly skip worker threads because !thread_group_leader(task) is true. It will also skip the zombie leader because
get_task_mm(task)returns NULL for exited tasks.Could this allow a process to permanently bypass cpuset memory limits?
[Severity: High]
This isn't a bug introduced by this patch, but doescpuset_attach()also fail to migrate or rebind memory if the thread group leader has exited?Looking at
cpuset_attach()inkernel/cgroup/cpuset.c:cgroup_taskset_for_each_leader(leader, css, tset) { struct mm_struct *mm = get_task_mm(leader);If the main thread exits and the process is migrated to a new cgroup via cgroup.procs, the migration core explicitly skips tasks with
PF_EXITING, omitting the zombie leader from the migration taskset.
Quoted 7 lines
Because
cgroup_taskset_for_each_leaderexecutes only for tasks where task ==task->group_leader, this condition is false for all surviving worker threads, bypassing the memory migration entirely.Can this be triggered by an unprivileged user to escape cpuset memory policy enforcement?
If a threadgroup without a leader is moved to another cpuset whose cpuset.mems are missing nodes in the original cpuset, it is possible that some of the process memory will stay in memory nodes that are not allowed by the new cpuset. This is an exceptional case that should rarely happen. It can be argued the amount of effort we should spend on handling this rare case. I will leave it unresolved for now unless there is a consensus that this should not be allowed.
Cheers,
Longman
Re: [PATCH v2 2/2] cgroup/cpuset: Rebind/migrate mm only for threadgroup leader in cpuset_update_tasks_nodemask()
In reply to Waiman LongHi
Quoted 5 lines · Waiman Long
[...]
Also add a paragraph incgroup-v2.rstunder cpuset.mems that the threadgroup leader is the memory owner of that threadgroup. Therefore the non-leading threads shouldn't be in other cgroups whose "cpuset.mems" doesn't fully overlap that of the group leader.
This sentence is long and complex, split into two if possible. I couldn't figure out how to do so.
Documentation/admin-guide/cgroup-v2.rst Cpuset Interface Files · line 2533
Quoted 11 lines
[...]
Documentation/admin-guide/cgroup-v2.rst +0
-0@@ -2527,6 +2527,13 @@ Cpuset Interface Filesa need to change "cpuset.mems" with active tasks, it shouldn't be done frequently.
Documentation/admin-guide/cgroup-v2.rst +4
-0+ For a multithreaded process, the threadgroup leader is + considered the owner of the group's memory. Memory policy + rebinding and migration will only happen with respect to the + threadgroup leader. To avoid unexpected result, non-leading
/result/results/
or
To avoid an unexpected result,
Documentation/admin-guide/cgroup-v2.rst Cpuset Interface Files · line 2535
Quoted 2 lines
Documentation/admin-guide/cgroup-v2.rst +2
-0+ threads shouldn't be put into another cgroup whose "cpuset.mems" + doesn't fully overlap that of the threadgroup leader.
maybe
/threadgroup/threadgroups/
Thanks
Manuel
Tejun Heo
maintainerQuoted 5 lines
Waiman Long (2):
cgroup/cpuset: Avoid unnecessary cpus & mems update in
cpuset_hotplug_update_tasks()
cgroup/cpuset: Rebind/migrate mm only for threadgroup leader in
cpuset_update_tasks_nodemask()
Applied 1-2 to cgroup/for-7.3. I folded in a few minor fixups: a changelog typo, the compute_effective_nodemask() kerneldoc parameter name (new_cpus to new_mems), and the comment and doc grammar nits Manuel noted. Also added Ridong's Reviewed-by to patch 1.
Thanks.
--
tejun
Waiman Long
maintainerQuoted 9 lines · Tejun Heo
Quoted 5 lines
Waiman Long (2):
cgroup/cpuset: Avoid unnecessary cpus & mems update in
cpuset_hotplug_update_tasks()
cgroup/cpuset: Rebind/migrate mm only for threadgroup leader in
cpuset_update_tasks_nodemask()Applied 1-2 to cgroup/for-7.3. I folded in a few minor fixups: a changelog typo, the
compute_effective_nodemask()kerneldoc parameter name (new_cpustonew_mems), and the comment and doc grammar nits Manuel noted. Also added Ridong's Reviewed-by to patch 1.
Thanks for the fixups.
Cheers,
Longman