cgroup/cpuset: More fix patches
Subsystems (3)
Waiman Long
maintainerv4 cover cgroup/cpuset: More fix patches
[v4]
- Drop v3 patch 1 as it is fixed in a different way in patch 5.
- Add a new patch 2 fixing a minor bug reported by sashiko.
- Add patch 4/5 to fix a remote partition hotplug problem reported by Guopeng.
- Add patch 6 to fix a bug in lockdep_is_cpuset_held().
[v3]
- Drop patch 1, 4 & 5 of v2 as they are merged.
- Update the new patch 1 as suggested by Chen Ridong
- Add a new fix patch 3.
[v2]
- Reorder the series by putting the original patch 4 before patch 2.
- Update the Fixes tags in patch 1 and 5 as suggested by sashiko.
- Update the new patch 3 to remove obsolete WARN_ON_ONCE() and update comment as reported by sashiko.
- Minor fix to the test case in patch 5 as reported by sashiko.
This patch series contains an additional set of cpuset fix patches.
Waiman Long (6):
cgroup/cpuset: Fix incorrect partition invalidation logic in
cpuset_hotplug_update_tasks()
cgroup/cpuset: excpus parameter of remote_cpus_update() must have at
least 1 active CPU
cgroup/cpuset: Check remote partition underneath before making an
invalid partition valid
cgroup/cpuset: Revert "remove global remote_children list" cgroup/cpuset: Handle remote partitions before local partitions in
cpuset_handle_hotplug()
cgroup/cpuset: Fix lockdep_is_cpuset_held()
kernel/cgroup/cpuset-internal.h | 10 +- kernel/cgroup/cpuset-v1.c | 16 +- kernel/cgroup/cpuset.c | 178 +++++++++++++----- .../selftests/cgroup/test_cpuset_prs.sh | 6 +- 4 files changed, 149 insertions(+), 61 deletions(-)
Signature · 1 line
2.56.0
Waiman Long
maintainerPATCH v4 1/6 cgroup/cpuset: Fix incorrect partition invalidation logic in cpuset_hotplug_update_tasks()
When most of the active CPUs of a system are allocated to sub-partitions below the top_cpuset partition, it is possible that a CPU offline operation may remove the last active CPU in the top_cpuset partition. When that really happens, cpuset_handle_hotplug() will clear the subpartitions_cpus to make sure active CPUs are present in the top partition and let the other sub-partitions get invalidated and then competes for their exclusive CPUs again.
In cpuset_hotplug_update_tasks(), the current code use "cpumask_empty(subpartitions_cpus)" to determine if that partition should be invalidated. That is fine if there is only one sub-partition. As reported by sashiko that when there are more than one sub-partitions and the first sub-partition will get invalidated and then it will try to acquire the exclusive CPUs and become valid again. When this happens, the subpartitions_cpus will no longer be empty. The remaining sub-partitions will assume their exclusive CPUs are safe and proceed as normal. As a result, active CPUs owned by those sub-partitions remain in both top_cpuset.effective_cpus and the sub-partitions' effective_cpus while missing from subpartitions_cpus, breaking partition CPU exclusivity and triggering WARN_ON_ONCE() in generate_sched_domains() and remote_partition_disable().
Fix that by checking if the sub-partitions' effective_xcpus are subset of subpartitions_cpus to determine if they should be invalidated. Also remove the obsolete WARN_ON_ONCE() call in remote_partition_disable().
--- kernel/cgroup/cpuset.c | 19 +++++-------------- 1 file changed, 5 insertions(+), 14 deletions(-)
kernel/cgroup/cpuset.c +5 -14
@@ -1768,15 +1768,6 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,static void remote_partition_disable (struct cpuset *cs,struct tmpmasks *tmp) {WARN_ON_ONCE (!is_remote_partition (cs)); -/* -* When a CPU is offlined, top_cpuset may end up with no available CPUs, -* which should clear subpartitions_cpus. We should not emit a warning for this -* scenario: the hierarchy is updated from top to bottom, so subpartitions_cpus -* may already be cleared when disabling the partition. -*/ -WARN_ON_ONCE (!cpumask_subset (cs->effective_xcpus, subpartitions_cpus) && - !cpumask_empty (subpartitions_cpus)); -spin_lock_irq (&callback_lock); cs->remote_partition = false;partition_xcpus_del (cs->partition_root_state,NULL , cs->effective_xcpus);
@@ -3983,9 +3974,9 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)if (!tmp || !cs->partition_root_state) goto update_tasks; -if (is_remote_partition (cs) && (cpumask_empty (subpartitions_cpus) || - (cpumask_empty (&new_cpus) && -partition_is_populated (cs,NULL )))) { +if (is_remote_partition (cs) && + (!cpumask_subset (cs->effective_xcpus, subpartitions_cpus) || + (cpumask_empty (&new_cpus) &&partition_is_populated (cs,NULL )))) {WRITE_ONCE (cs->prs_err,PERR_HOTPLUG );remote_partition_disable (cs, tmp);compute_effective_cpumask (&new_cpus, cs, parent);
@@ -3997,12 +3988,12 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)* 1) empty effective cpus but not valid empty partition. * 2) parent is invalid or doesn't grant any cpus to child * partitions. -* 3) subpartitions_cpus is empty. +* 3) effective_xcpus isn't a subset of subpartitions_cpus. */ if (is_local_partition (cs) && (!is_partition_valid (parent) ||tasks_nocpu_error (parent, cs, &new_cpus) || -cpumask_empty (subpartitions_cpus))) + !cpumask_subset (cs->effective_xcpus, subpartitions_cpus))) partcmd = partcmd_invalidate;/* * On the other hand, an invalid partition root may be transitioned
Signature · 1 line
2.56.0
Re: [PATCH-next v4 1/6] cgroup/cpuset: Fix incorrect partition invalidation logic in cpuset_hotplug_update_tasks()
Quoted 73 lines · Waiman Long
When most of the active CPUs of a system are allocated to sub-partitions below the
top_cpusetpartition, it is possible that a CPU offline operation may remove the last active CPU in thetop_cpusetpartition. When that really happens,cpuset_handle_hotplug()will clear thesubpartitions_cpusto make sure active CPUs are present in the top partition and let the other sub-partitions get invalidated and then competes for their exclusive CPUs again.In
cpuset_hotplug_update_tasks(), the current code use "cpumask_empty(subpartitions_cpus)" to determine if that partition should be invalidated. That is fine if there is only one sub-partition. As reported by sashiko that when there are more than one sub-partitions and the first sub-partition will get invalidated and then it will try to acquire the exclusive CPUs and become valid again. When this happens, thesubpartitions_cpuswill no longer be empty. The remaining sub-partitions will assume their exclusive CPUs are safe and proceed as normal. As a result, active CPUs owned by those sub-partitions remain in both top_cpuset.effective_cpus and the sub-partitions'effective_cpuswhile missing fromsubpartitions_cpus, breaking partition CPU exclusivity and triggeringWARN_ON_ONCE()ingenerate_sched_domains()andremote_partition_disable().Fix that by checking if the sub-partitions'
effective_xcpusare subset ofsubpartitions_cpusto determine if they should be invalidated. Also remove the obsoleteWARN_ON_ONCE()call inremote_partition_disable().Fixes 0c7f293efc87 ("cgroup/cpuset: Add cpuset.cpus.exclusive.effective for v2")Reported-by Sashiko <[email protected]>Reviewed-by Ridong Chen <[email protected]>Signed-off-by Waiman Long <[email protected]>--- kernel/cgroup/cpuset.c | 19 +++++-------------- 1 file changed, 5 insertions(+), 14 deletions(-)kernel/cgroup/cpuset.c +3
-13@@ -1768,15 +1768,6 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,static void remote_partition_disable (struct cpuset *cs,struct tmpmasks *tmp) {WARN_ON_ONCE (!is_remote_partition (cs)); -/* -* When a CPU is offlined, top_cpuset may end up with no available CPUs, -* which should clear subpartitions_cpus. We should not emit a warning for this -* scenario: the hierarchy is updated from top to bottom, so subpartitions_cpus -* may already be cleared when disabling the partition. -*/ -WARN_ON_ONCE (!cpumask_subset (cs->effective_xcpus, subpartitions_cpus) && - !cpumask_empty (subpartitions_cpus)); -spin_lock_irq (&callback_lock); cs->remote_partition = false;partition_xcpus_del (cs->partition_root_state,NULL , cs->effective_xcpus);@@ -3983,9 +3974,9 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)if (!tmp || !cs->partition_root_state) goto update_tasks; -if (is_remote_partition (cs) && (cpumask_empty (subpartitions_cpus) || - (cpumask_empty (&new_cpus) && -partition_is_populated (cs,NULL )))) { +if (is_remote_partition (cs) && + (!cpumask_subset (cs->effective_xcpus, subpartitions_cpus) || + (cpumask_empty (&new_cpus) &&partition_is_populated (cs,NULL )))) {WRITE_ONCE (cs->prs_err,PERR_HOTPLUG );remote_partition_disable (cs, tmp);compute_effective_cpumask (&new_cpus, cs, parent);@@ -3997,12 +3988,12 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)* 1) empty effective cpus but not valid empty partition. * 2) parent is invalid or doesn't grant any cpus to child * partitions. -* 3) subpartitions_cpus is empty.
kernel/cgroup/cpuset.c static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp) · line 3999
Quoted 10 lines
kernel/cgroup/cpuset.c +2
-1+* 3) effective_xcpus isn't a subset of subpartitions_cpus. */ if (is_local_partition (cs) && (!is_partition_valid (parent) ||tasks_nocpu_error (parent, cs, &new_cpus) || -cpumask_empty (subpartitions_cpus))) + !cpumask_subset (cs->effective_xcpus, subpartitions_cpus))) partcmd = partcmd_invalidate;/* * On the other hand, an invalid partition root may be transitioned
Thanks,
Guopeng
Waiman Long
maintainerPATCH v4 2/6 cgroup/cpuset: excpus parameter of remote_cpus_update() must have at least 1 active CPU
In reply to Waiman LongIt is reported that remote_cpus_update() fails to invalidate a populated remote partition when an ancestor cgroup update reduces its effective_xcpus mask to only offline CPUs.
When update_cpumasks_hier() updates a descendant remote partition, it passes the new effective_xcpus mask (excpus) to remote_cpus_update(). Because excpus still contains offline CPUs, cpumask_empty(excpus) is false and remote_cpus_update() only checks tasks_nocpu_error() when adding CPUs.
With the partition left valid, update_cpumasks_hier() computes an empty effective_cpus mask and jumps to get_css. The populated partition stays valid with no effective CPUs instead of inheriting its parent's CPUs, and its tasks remain affine to the online CPUs returned to top_cpuset.
Fix it by invalidating the remote partition if there is no active CPU available in the given excpus parameter.
--- kernel/cgroup/cpuset.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/cgroup/cpuset.c +2 -1
@@ -1810,7 +1810,8 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *xcpus,WARN_ON_ONCE (!cpumask_subset (cs->effective_xcpus, subpartitions_cpus)); -if (cpumask_empty (excpus)) { +/* New effective_xcpus must have at least one active CPU */ +if (!cpumask_intersects (excpus, cpu_active_mask)) {WRITE_ONCE (cs->prs_err,PERR_CPUSEMPTY ); goto invalidate; }
Signature · 1 line
2.56.0
Ridong Chen
reviewerRe: [PATCH-next v4 2/6] cgroup/cpuset: excpus parameter of remote_cpus_update() must have at least 1 active CPU
Quoted 29 lines · Waiman Long
It is reported that
remote_cpus_update()fails to invalidate a populated remote partition when an ancestor cgroup update reduces itseffective_xcpusmask to only offline CPUs.When
update_cpumasks_hier()updates a descendant remote partition, it passes the neweffective_xcpusmask (excpus) toremote_cpus_update(). Because excpus still contains offline CPUs,cpumask_empty(excpus)is false andremote_cpus_update()only checkstasks_nocpu_error()when adding CPUs.With the partition left valid,
update_cpumasks_hier()computes an emptyeffective_cpusmask and jumps toget_css. The populated partition stays valid with no effective CPUs instead of inheriting its parent's CPUs, and its tasks remain affine to the online CPUs returned totop_cpuset.Fix it by invalidating the remote partition if there is no active CPU available in the given excpus parameter.
Reported-by Sashiko <[email protected]>Signed-off-by Waiman Long <[email protected]>--- kernel/cgroup/cpuset.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)kernel/cgroup/cpuset.c +0
-0
kernel/cgroup/cpuset.c static void remote_cpus_update(struct cpuset *cs, struct cpumask *xcpus, · line 1817
Quoted 9 lines
kernel/cgroup/cpuset.c +2
-1@@ -1810,7 +1810,8 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *xcpus,WARN_ON_ONCE (!cpumask_subset (cs->effective_xcpus, subpartitions_cpus)); -if (cpumask_empty (excpus)) { +/* New effective_xcpus must have at least one active CPU */ +if (!cpumask_intersects (excpus, cpu_active_mask)) {WRITE_ONCE (cs->prs_err,PERR_CPUSEMPTY ); goto invalidate; }
This also invalidates a remote partition with no task whose excpus only has offline CPUs. According to cgroup-v2.rst, such a partition is valid: cpuset.cpus.exclusive.effective may contain offline CPUs, and cpuset.cpus.effective can be empty if there is no task in the partition. The hotplug path and tasks_nocpu_error() follow the same rule. How about only invalidating it when it is populated?
if (cpumask_empty(excpus) ||
(!cpumask_intersects(excpus, cpu_active_mask) &&
partition_is_populated(cs, NULL))) {Also, it probably needs a Fixes tag, maybe:
Signature · 2 lines
Best regards Ridong
Waiman Long
maintainerRe: [PATCH-next v4 2/6] cgroup/cpuset: excpus parameter of remote_cpus_update() must have at least 1 active CPU
Quoted 48 lines · Ridong Chen
Quoted 29 lines · Waiman Long
It is reported that
remote_cpus_update()fails to invalidate a populated remote partition when an ancestor cgroup update reduces itseffective_xcpusmask to only offline CPUs.When
update_cpumasks_hier()updates a descendant remote partition, it passes the neweffective_xcpusmask (excpus) toremote_cpus_update(). Because excpus still contains offline CPUs,cpumask_empty(excpus)is false andremote_cpus_update()only checkstasks_nocpu_error()when adding CPUs.With the partition left valid,
update_cpumasks_hier()computes an emptyeffective_cpusmask and jumps toget_css. The populated partition stays valid with no effective CPUs instead of inheriting its parent's CPUs, and its tasks remain affine to the online CPUs returned totop_cpuset.Fix it by invalidating the remote partition if there is no active CPU available in the given excpus parameter.
Reported-by Sashiko <[email protected]>Signed-off-by Waiman Long <[email protected]>---
kernel/cgroup/cpuset.c| 3 ++-1 file changed, 2 insertions(+), 1 deletion(-)kernel/cgroup/cpuset.c +0
-0kernel/cgroup/cpuset.c static void remote_cpus_update(struct cpuset · line 1817
Quoted 8 lines
kernel/cgroup/cpuset.c +2
-0@@ -1810,7 +1810,8 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *xcpus, WARN_ON_ONCE(!cpumask_subset(cs->effective_xcpus, subpartitions_cpus)); - if (cpumask_empty(excpus)) { + /* New effective_xcpus must have at least one active CPU */ + if (!cpumask_intersects(excpus, cpu_active_mask)) { WRITE_ONCE(cs->prs_err, PERR_CPUSEMPTY); goto invalidate; }This also invalidates a remote partition with no task whose excpus only has offline CPUs. According to
cgroup-v2.rst, such a partition is valid: cpuset.cpus.exclusive.effective may contain offline CPUs, and cpuset.cpus.effective can be empty if there is no task in the partition. The hotplug path andtasks_nocpu_error()follow the same rule. How about only invalidating it when it is populated?
Quoted 3 lines
if (
cpumask_empty(excpus)|| (!cpumask_intersects(excpus,cpu_active_mask) &&partition_is_populated(cs, NULL))) {
A partition can only be created if there are active CPUs in the effective_xcpus. However, such a partition root can distribute its exclusive CPUs down to child local partitions with the results that effective_cpus can be empty if there is no task there. IOW, effective_xcpus must have active CPUs while effective_cpus does not need to have active CPUs if there are child partitions underneath. I am going to clarify that in the doc. I will check the hotplug path and task_nocpu_error() to make sure that it follows this rule. Does that rule make sense to you?
Quoted 4 lines
Also, it probably needs a Fixes tag, maybe:
Fixes f62a5d39368e ("cgroup/cpuset: Remove remote_partition_check() & make update_cpumasks_hier() handle remote partition")
Yes, I miss the fix tag. Will update that in the next version.
Cheers,
Longman
Ridong Chen
reviewerRe: [PATCH-next v4 2/6] cgroup/cpuset: excpus parameter of remote_cpus_update() must have at least 1 active CPU
Quoted 49 lines · Waiman Long
Quoted 48 lines · Ridong Chen
Quoted 29 lines · Waiman Long
It is reported that
remote_cpus_update()fails to invalidate a populated remote partition when an ancestor cgroup update reduces itseffective_xcpusmask to only offline CPUs.When
update_cpumasks_hier()updates a descendant remote partition, it passes the neweffective_xcpusmask (excpus) toremote_cpus_update(). Because excpus still contains offline CPUs,cpumask_empty(excpus)is false andremote_cpus_update()only checkstasks_nocpu_error()when adding CPUs.With the partition left valid,
update_cpumasks_hier()computes an emptyeffective_cpusmask and jumps toget_css. The populated partition stays valid with no effective CPUs instead of inheriting its parent's CPUs, and its tasks remain affine to the online CPUs returned totop_cpuset.Fix it by invalidating the remote partition if there is no active CPU available in the given excpus parameter.
Reported-by Sashiko <[email protected]>Signed-off-by Waiman Long <[email protected]>---
kernel/cgroup/cpuset.c| 3 ++-1 file changed, 2 insertions(+), 1 deletion(-)kernel/cgroup/cpuset.c +0
-0kernel/cgroup/cpuset.c static void remote_cpus_update(struct cpuset *cs, · line 1817
Quoted 8 lines
kernel/cgroup/cpuset.c +2
-0@@ -1810,7 +1810,8 @@ static void remote_cpus_update(struct cpuset *cs,struct cpumask *xcpus, WARN_ON_ONCE(!cpumask_subset(cs->effective_xcpus, subpartitions_cpus)); - if (cpumask_empty(excpus)) { + /* New effective_xcpus must have at least one active CPU */ + if (!cpumask_intersects(excpus, cpu_active_mask)) { WRITE_ONCE(cs->prs_err, PERR_CPUSEMPTY); goto invalidate; }This also invalidates a remote partition with no task whose excpus only has offline CPUs. According to
cgroup-v2.rst, such a partition is valid: cpuset.cpus.exclusive.effective may contain offline CPUs, and cpuset.cpus.effective can be empty if there is no task in the partition. The hotplug path andtasks_nocpu_error()follow the same rule. How about only invalidating it when it is populated?
Quoted 6 lines
Quoted 3 lines
if (
cpumask_empty(excpus)|| (!cpumask_intersects(excpus,cpu_active_mask) &&partition_is_populated(cs, NULL))) {A partition can only be created if there are active CPUs in the
effective_xcpus. However, such a partition root can distribute its exclusive CPUs down to child local partitions with the results thateffective_cpuscan be empty if there is no task there. IOW,effective_xcpusmust have active CPUs whileeffective_cpusdoes not need to have active CPUs if there are child partitions underneath. I am going to clarify that in the doc. I will check the hotplug path andtask_nocpu_error()to make sure that it follows this rule. Does that rule make sense to you?
Yes, the rule makes sense to me, and it is clearer.
Note that the hotplug path currently keeps a partition with no task valid after all its CPUs are offlined, e.g. these test cases in test_cpuset_prs.sh:
" C2-3:P1 C3:P1 . . O3=0 . . . 0 A1:2|A2: A1:P1|A2:P1" "$SETUP_A123_PARTITIONS . O3=0 . . . 0 A1:1|A2:2|A3: A1:P1|A2:P1|A3:P1"
So the hotplug path and these test cases need to be updated together with this patch to keep the behavior consistent?
Quoted 10 lines
Quoted 5 lines
Also, it probably needs a Fixes tag, maybe:
Fixes f62a5d39368e ("cgroup/cpuset: Remove remote_partition_check() & make update_cpumasks_hier() handle remote partition")Yes, I miss the fix tag. Will update that in the next version.
Cheers,
Longman
Signature · 2 lines
Best regards Ridong
Waiman Long
maintainerPATCH v4 3/6 cgroup/cpuset: Check remote partition underneath before making an invalid partition valid
In reply to Waiman LongWhen re-evaluating an invalid local partition via partcmd_update, update_parent_effective_cpumask() does not check whether a descendant remote partition is using its exclusive CPUs, promoting the cpuset to a valid partition root over the remote partition. This condition should only be possible if parent is the top_cpuset and exclusive_cpus isn't empty.
This leaves an active remote partition underneath a valid local partition root with overlapping exclusive CPUs, breaking scheduler domain partitioning and triggering warnings in compute_partition_effective_cpumask() and generate_sched_domains().
Fix that by checking the effective_xcpus against subpartitions_cpus when an underneath remote partition is possible to make sure that all the exclusive CPUs are available before allowing the invalid partition to be switched back to valid.
Also make sure that effective_xcpus is non-empty before swtiching an invalid partition back to valid.
Add a test case in test_cpuset_prs.sh to verify that an invalid partition with a remote partition underneath cannot be switched back to valid with a hotplug event.
In the special case when the last active CPU in the top_cpuset partition is offlined, the hotplug code will empty subpartitions_cpus and let the existing local partition to compete for the exclusive CPUs again. If there is an invalid local partition above a remote partition and that invalid local partition is able to switch to a valid partition, the remote partition below will now share some the of exclusive CPUs violating the CPU exclusivity rule. This special case will be handled in a later patch.
--- kernel/cgroup/cpuset.c | 42 ++++++++++++------- .../selftests/cgroup/test_cpuset_prs.sh | 4 ++ 2 files changed, 30 insertions(+), 16 deletions(-)
kernel/cgroup/cpuset.c +26 -16
@@ -2029,11 +2029,7 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,* tasks_nocpu_error() with this very mask, and a failing * write is routed to partcmd_invalidate instead. */ -if (is_partition_invalid (cs)) { - adding = false; - deleting =cpumask_and (tmp->delmask, - newmask, parent->effective_xcpus); - }else { +if (is_partition_valid (cs)) {cpumask_andnot (tmp->addmask, cs->effective_xcpus, newmask); adding =cpumask_and (tmp->addmask, tmp->addmask, parent->effective_xcpus);
@@ -2041,6 +2037,17 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,cpumask_andnot (tmp->delmask, newmask, cs->effective_xcpus); deleting =cpumask_and (tmp->delmask, tmp->delmask, parent->effective_xcpus); + }else { + deleting =cpumask_and (tmp->delmask, + newmask, parent->effective_xcpus); + +/* Check if a remote partition may be underneath */ +if ((parent == &top_cpuset) && + !cpumask_empty (cs->exclusive_cpus) && +cpumask_intersects (tmp->delmask, subpartitions_cpus)) { + part_error =PERR_REMOTE ; + deleting = false; + } }/*
@@ -2107,34 +2114,37 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd, }else if (is_partition_invalid (cs)) {struct cgroup_subsys_state *css;struct cpuset *child; -bool exclusive = true;/* * Convert invalid partition to valid has to pass the -* cpu exclusivity test when cpuset.cpus.exclusive is -* empty as well as the housekeeping check. +* following tests: +* 1) housekeeping CPU conflict check +* 2) cpu exclusivity test when cpuset.cpus.exclusive is +* empty. +* 3) if parent is top_cpuset, none of the effective_xcpus +* (tmp->delmask) are in subpartitions_cpus, if not a +* remote partition may be underneath. */ if (prstate_housekeeping_conflict (cs->partition_root_state, parent_prs, tmp->delmask,NULL )) { part_error =PERR_HKEEPING ; - goto write_error; - } -if (cpumask_empty (cs->exclusive_cpus)) { + }else if (cpumask_empty (cs->exclusive_cpus)) {rcu_read_lock ();cpuset_for_each_child (child, css, parent) {if (child == cs)continue ;if (!cpusets_are_exclusive (cs, child)) { - exclusive = false; + part_error =PERR_NOTEXCL ;break ; } }rcu_read_unlock (); + }else if ((parent == &top_cpuset) && +cpumask_intersects (tmp->delmask, subpartitions_cpus)) { + part_error =PERR_REMOTE ; } -if (exclusive) +if (!part_error) deleting = !cpumask_empty (tmp->delmask); -else - part_error =PERR_NOTEXCL ; } }
@@ -2155,7 +2165,7 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,break ;case PRS_INVALID_ROOT :case PRS_INVALID_ISOLATED : -if (!part_error) +if (!part_error && deleting) new_prs = -old_prs;break ; }
tools/testing/selftests/cgroup/test_cpuset_prs.sh +4 -0
@@ -563,6 +563,10 @@ TEST_MATRIX=( " C1-6:P1 C3-6:P2 C5-6:P2 . . P1 . . 0 A1:1-2|A2:3-4|A3:5-6 \ A1:P1|A2:P1|A3:P2 5-6" + # A hotplug event will not incorrectly activate an invalid partition with + # a remote partition underneath. + " CX1-4 C3-4:P1 . . P1 O1=0 . . 0 A1:2|A2:3-4 A1:P-1|A2:P1" + # # old-A1 old-A2 old-A3 old-B1 new-A1 new-A2 new-A3 new-B1 fail ECPUs Pstate ISOLCPUS # ------ ------ ------ ------ ------ ------ ------ ------ ---- ----- ------ --------
Signature · 1 line
2.56.0
Re: [PATCH-next v4 3/6] cgroup/cpuset: Check remote partition underneath before making an invalid partition valid
Quoted 124 lines · Waiman Long
When re-evaluating an invalid local partition via
partcmd_update,update_parent_effective_cpumask()does not check whether a descendant remote partition is using its exclusive CPUs, promoting the cpuset to a valid partition root over the remote partition. This condition should only be possible if parent is thetop_cpusetandexclusive_cpusisn't empty.This leaves an active remote partition underneath a valid local partition root with overlapping exclusive CPUs, breaking scheduler domain partitioning and triggering warnings in
compute_partition_effective_cpumask()andgenerate_sched_domains().Fix that by checking the
effective_xcpusagainstsubpartitions_cpuswhen an underneath remote partition is possible to make sure that all the exclusive CPUs are available before allowing the invalid partition to be switched back to valid.Also make sure that
effective_xcpusis non-empty before swtiching an invalid partition back to valid.Add a test case in
test_cpuset_prs.shto verify that an invalid partition with a remote partition underneath cannot be switched back to valid with a hotplug event.In the special case when the last active CPU in the
top_cpusetpartition is offlined, the hotplug code will emptysubpartitions_cpusand let the existing local partition to compete for the exclusive CPUs again. If there is an invalid local partition above a remote partition and that invalid local partition is able to switch to a valid partition, the remote partition below will now share some the of exclusive CPUs violating the CPU exclusivity rule. This special case will be handled in a later patch.Fixes 181c8e091aae ("cgroup/cpuset: Introduce remote partition")Reported-by Sashiko <[email protected]>Signed-off-by Waiman Long <[email protected]>--- kernel/cgroup/cpuset.c | 42 ++++++++++++------- .../selftests/cgroup/test_cpuset_prs.sh | 4 ++ 2 files changed, 30 insertions(+), 16 deletions(-)kernel/cgroup/cpuset.c +25
-15@@ -2029,11 +2029,7 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,* tasks_nocpu_error() with this very mask, and a failing * write is routed to partcmd_invalidate instead. */ -if (is_partition_invalid (cs)) { - adding = false; - deleting =cpumask_and (tmp->delmask, - newmask, parent->effective_xcpus); - }else { +if (is_partition_valid (cs)) {cpumask_andnot (tmp->addmask, cs->effective_xcpus, newmask); adding =cpumask_and (tmp->addmask, tmp->addmask, parent->effective_xcpus);@@ -2041,6 +2037,17 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,cpumask_andnot (tmp->delmask, newmask, cs->effective_xcpus); deleting =cpumask_and (tmp->delmask, tmp->delmask, parent->effective_xcpus); + }else { + deleting =cpumask_and (tmp->delmask, + newmask, parent->effective_xcpus); + +/* Check if a remote partition may be underneath */ +if ((parent == &top_cpuset) && + !cpumask_empty (cs->exclusive_cpus) && +cpumask_intersects (tmp->delmask, subpartitions_cpus)) { + part_error =PERR_REMOTE ; + deleting = false; + } }/* @@ -2107,34 +2114,37 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd, }else if (is_partition_invalid (cs)) {struct cgroup_subsys_state *css;struct cpuset *child; -bool exclusive = true;/* * Convert invalid partition to valid has to pass the -* cpu exclusivity test when cpuset.cpus.exclusive is -* empty as well as the housekeeping check. +* following tests: +* 1) housekeeping CPU conflict check +* 2) cpu exclusivity test when cpuset.cpus.exclusive is +* empty. +* 3) if parent is top_cpuset, none of the effective_xcpus +* (tmp->delmask) are in subpartitions_cpus, if not a +* remote partition may be underneath. */ if (prstate_housekeeping_conflict (cs->partition_root_state, parent_prs, tmp->delmask,NULL )) { part_error =PERR_HKEEPING ; - goto write_error; - } -if (cpumask_empty (cs->exclusive_cpus)) { + }else if (cpumask_empty (cs->exclusive_cpus)) {rcu_read_lock ();cpuset_for_each_child (child, css, parent) {if (child == cs)continue ;if (!cpusets_are_exclusive (cs, child)) { - exclusive = false; + part_error =PERR_NOTEXCL ;break ; } }rcu_read_unlock (); + }else if ((parent == &top_cpuset) && +cpumask_intersects (tmp->delmask, subpartitions_cpus)) { + part_error =PERR_REMOTE ; } -if (exclusive) +if (!part_error) deleting = !cpumask_empty (tmp->delmask); -else - part_error =PERR_NOTEXCL ; } }
kernel/cgroup/cpuset.c static int update_parent_effective_cpumask(struct cpuset *cs, int cmd, · line 2171
Quoted 8 lines
kernel/cgroup/cpuset.c +1
-1@@ -2155,7 +2165,7 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,break ;case PRS_INVALID_ROOT :case PRS_INVALID_ISOLATED : -if (!part_error) +if (!part_error && deleting) new_prs = -old_prs;break ; }
Nit: deleting is always true when part_error is 0, so this check doesn't change anything. It's fine to keep it as a safeguard, but then the changelog shouldn't describe it as a fix.
Typos in the changelog: "swtiching", "share some the of".
Otherwise,
Quoted 15 lines
tools/testing/selftests/cgroup/test_cpuset_prs.sh +4
-0@@ -563,6 +563,10 @@ TEST_MATRIX=( " C1-6:P1 C3-6:P2 C5-6:P2 . . P1 . . 0 A1:1-2|A2:3-4|A3:5-6 \ A1:P1|A2:P1|A3:P2 5-6" + # A hotplug event will not incorrectly activate an invalid partition with + # a remote partition underneath. + " CX1-4 C3-4:P1 . . P1 O1=0 . . 0 A1:2|A2:3-4 A1:P-1|A2:P1" + # # old-A1 old-A2 old-A3 old-B1 new-A1 new-A2 new-A3 new-B1 fail ECPUs Pstate ISOLCPUS # ------ ------ ------ ------ ------ ------ ------ ------ ---- ----- ------ --------
Signature · 2 lines
Best regards Ridong
Waiman Long
maintainerRe: [PATCH-next v4 3/6] cgroup/cpuset: Check remote partition underneath before making an invalid partition valid
Quoted 132 lines · Ridong Chen
Quoted 129 lines · Waiman Long
When re-evaluating an invalid local partition via
partcmd_update,update_parent_effective_cpumask()does not check whether a descendant remote partition is using its exclusive CPUs, promoting the cpuset to a valid partition root over the remote partition. This condition should only be possible if parent is thetop_cpusetandexclusive_cpusisn't empty.This leaves an active remote partition underneath a valid local partition root with overlapping exclusive CPUs, breaking scheduler domain partitioning and triggering warnings in
compute_partition_effective_cpumask()andgenerate_sched_domains().Fix that by checking the
effective_xcpusagainstsubpartitions_cpuswhen an underneath remote partition is possible to make sure that all the exclusive CPUs are available before allowing the invalid partition to be switched back to valid.Also make sure that
effective_xcpusis non-empty before swtiching an invalid partition back to valid.Add a test case in
test_cpuset_prs.shto verify that an invalid partition with a remote partition underneath cannot be switched back to valid with a hotplug event.In the special case when the last active CPU in the
top_cpusetpartition is offlined, the hotplug code will emptysubpartitions_cpusand let the existing local partition to compete for the exclusive CPUs again. If there is an invalid local partition above a remote partition and that invalid local partition is able to switch to a valid partition, the remote partition below will now share some the of exclusive CPUs violating the CPU exclusivity rule. This special case will be handled in a later patch.Fixes 181c8e091aae ("cgroup/cpuset: Introduce remote partition")Reported-by Sashiko <[email protected]>Signed-off-by Waiman Long <[email protected]>---
kernel/cgroup/cpuset.c | 42 ++++++++++++------- .../selftests/cgroup/test_cpuset_prs.sh | 4 ++2 files changed, 30 insertions(+), 16 deletions(-)kernel/cgroup/cpuset.c +1
-5@@ -2029,11 +2029,7 @@ static intupdate_parent_effective_cpumask(struct cpuset *cs, int cmd, * tasks_nocpu_error() with this very mask, and a failing * write is routed to partcmd_invalidate instead. */ - if (is_partition_invalid(cs)) { - adding = false; - deleting = cpumask_and(tmp->delmask, - newmask, parent->effective_xcpus); - } else { + if (is_partition_valid(cs)) { cpumask_andnot(tmp->addmask, cs->effective_xcpus, newmask); adding = cpumask_and(tmp->addmask, tmp->addmask,
parent->effective_xcpus);kernel/cgroup/cpuset.c +24
-10@@ -2041,6 +2037,17 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd, cpumask_andnot(tmp->delmask, newmask, cs->effective_xcpus); deleting = cpumask_and(tmp->delmask, tmp->delmask, parent->effective_xcpus); + } else { + deleting = cpumask_and(tmp->delmask, + newmask, parent->effective_xcpus); + + /* Check if a remote partition may be underneath */ + if ((parent == &top_cpuset) && + !cpumask_empty(cs->exclusive_cpus) && + cpumask_intersects(tmp->delmask, subpartitions_cpus)) { + part_error = PERR_REMOTE; + deleting = false; + } } /*@@ -2107,34 +2114,37 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd, } else if (is_partition_invalid(cs)) { struct cgroup_subsys_state *css; struct cpuset *child; - bool exclusive = true; /* * Convert invalid partition to valid has to pass the - * cpu exclusivity test when cpuset.cpus.exclusive is - * empty as well as the housekeeping check. + * following tests: + * 1) housekeeping CPU conflict check + * 2) cpu exclusivity test when cpuset.cpus.exclusive is + * empty. + * 3) if parent is top_cpuset, none of the effective_xcpus + * (tmp->delmask) are in subpartitions_cpus, if not a + * remote partition may be underneath. */ if (prstate_housekeeping_conflict(cs->partition_root_state, parent_prs, tmp->delmask, NULL)) { part_error = PERR_HKEEPING; - goto write_error; - } - if (cpumask_empty(cs->exclusive_cpus)) { + } else if (cpumask_empty(cs->exclusive_cpus)) { rcu_read_lock(); cpuset_for_each_child(child, css, parent) { if (child == cs) continue; if (!cpusets_are_exclusive(cs, child)) { - exclusive = false; + part_error = PERR_NOTEXCL; break; } } rcu_read_unlock(); + } else if ((parent == &top_cpuset) && + cpumask_intersects(tmp->delmask, subpartitions_cpus)) { + part_error = PERR_REMOTE; } - if (exclusive) + if (!part_error) deleting = !cpumask_empty(tmp->delmask); - else - part_error = PERR_NOTEXCL; } }@@ -2155,7 +2165,7 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd, break;
casePRS_INVALID_ROOT:
casePRS_INVALID_ISOLATED:
- if (!part_error)
Quoted 8 lines
Quoted 4 lines
+ if (!part_error && deleting)
new_prs= -old_prs;
break;
}Nit: deleting is always true when
part_erroris 0, so this check doesn't change anything. It's fine to keep it as a safeguard, but then the changelog shouldn't describe it as a fix.
tmp->delmask can be empty and deleting will be set to false in this case. Of course, we should not activate an invalid partition root with empty effective_xcpus.
Quoted 3 lines
Typos in the changelog: "swtiching", "share some the of".
Will fix that in the next version.
Thanks,
Longman
Quoted 22 lines
Otherwise,
Reviewed-by Ridong Chen <[email protected]>Quoted 16 lines
tools/testing/selftests/cgroup/test_cpuset_prs.sh +0
-0@@ -563,6 +563,10 @@ TEST_MATRIX=( " C1-6:P1 C3-6:P2 C5-6:P2 . . P1 . . 0 A1:1-2|A2:3-4|A3:5-6 \A1:P1|A2:P1|A3:P2 5-6"
+ # A hotplug event will not incorrectly activate an invalid partition with + # a remote partition underneath.
+ " CX1-4 C3-4:P1 . . P1 O1=0 . . 0
A1:2|A2:3-4 A1:P-1|A2:P1"
+
#
# old-A1 old-A2 old-A3 old-B1 new-A1 new-A2 new-A3 new-B1 fail ECPUs Pstate ISOLCPUS # ------ ------ ------ ------ ------ ------ ------ ------ ----
----- ------ --------
A message that is not in the archive (sent off-list or not yet received): [email protected]
Waiman Long
maintainerRe: [PATCH-next v4 3/6] cgroup/cpuset: Check remote partition underneath before making an invalid partition valid
Quoted 86 lines · On 11/15/26 8:48 PM, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Medium] cgroup/cpuset: newmask check inupdate_parent_effective_cpumask()misses active remote partition
--commit
957838c49c66e004ba9111067a9ab3098b2792cc
Author: Waiman Long <[email protected]>cgroup/cpuset: Check remote partition underneath before making an invalid partition valid
This commit checks whether an invalid local partition's
effective_xcpusoverlapsubpartitions_cpusbefore promoting it back to a valid partition root, and requireseffective_xcpusto be non-empty on transition.Quoted 23 lines
kernel/cgroup/cpuset.c +3
-5@@ -2035,11 +2035,7 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,* tasks_nocpu_error() with this very mask, and a failing * write is routed to partcmd_invalidate instead. */ -if (is_partition_invalid (cs)) { - adding = false; - deleting =cpumask_and (tmp->delmask, - newmask, parent->effective_xcpus); - }else { +if (is_partition_valid (cs)) {cpumask_andnot (tmp->addmask, cs->effective_xcpus, newmask); adding =cpumask_and (tmp->addmask, tmp->addmask, parent->effective_xcpus);@@ -2047,6 +2043,17 @@ static int update_parent_effective_cpumask(struct cpuset *cs, int cmd,cpumask_andnot (tmp->delmask, newmask, cs->effective_xcpus); deleting =cpumask_and (tmp->delmask, tmp->delmask, parent->effective_xcpus); + }else { + deleting = cpumask_and(tmp->delmask,kernel/cgroup/cpuset.c static int update_parent_effective_cpumask(struct cpuset *cs, int cmd, · line 2057
Quoted 10 lines
kernel/cgroup/cpuset.c +9
-0+ newmask, parent->effective_xcpus); + +/* Check if a remote partition may be underneath */ +if ((parent == &top_cpuset) && + !cpumask_empty (cs->exclusive_cpus) && +cpumask_intersects (tmp->delmask, subpartitions_cpus)) { + part_error =PERR_REMOTE ; + deleting = false; + } }[Severity: Medium]
Can this check miss an active descendant remote partition when newmask is disjoint fromcs->effective_xcpus?In
update_exclusive_cpumask(),partition_cpus_change()callsupdate_parent_effective_cpumask()beforecs->effective_xcpusis updated and beforeupdate_cpumasks_hier()updates or invalidates the subtree:kernel/cgroup/cpuset.c:update_exclusive_cpumask() { ... partition_cpus_change(cs, trialcs, &tmp); spin_lock_irq(&callback_lock); cpumask_copy(cs->exclusive_cpus, trialcs->exclusive_cpus); cpumask_copy(cs->effective_xcpus, trialcs->effective_xcpus); ... if (is_partition_valid(cs) || force) update_cpumasks_hier(cs, &tmp, force); ... }At the time of this check, a remote partition underneath cs is still using CPUs from cs's old
effective_xcpus, not from newmask (tmp->delmask). For example, if child A1 hasexclusive_cpus3-4 and is an invalid partition because grandchild A2 is a remote partition using 3-4, writing 1-2 to A1/cpuset.cpus.exclusive producestmp->delmask= 1-2, which does not intersectsubpartitions_cpus(3-4).
update_parent_effective_cpumask()then promotes A1 toPRS_ROOTwhile A2 is still an active remote partition underneath it. The subsequent pre-order walk inupdate_cpumasks_hier()visits A1 before A2 and triggers WARN_ON_ONCE(is_remote_partition(child)) incompute_partition_effective_cpumask()(and if A2'suser_xcpus()also overlaps 1-2,WARN_ON_ONCE(cpumask_intersects(tmp->addmask, subpartitions_cpus))inremote_cpus_update()).
Quoted 3 lines
Checking
cpumask_intersects(cs->effective_xcpus, subpartitions_cpus)as well when newmask is set would catch a descendant remote partition still holding CPUs from cs's oldeffective_xcpus.
Good point. I will add this additional check in the next version.
Cheers,
Longman
Waiman Long
maintainerPATCH v4 4/6 cgroup/cpuset: Revert "remove global remote_children list"
In reply to Waiman LongCommit 648d43da64f0 ("cpuset: remove global remote_children list") is being reverted as there is now a need to iterate the list of remote partition roots available.
--- kernel/cgroup/cpuset-internal.h | 10 +++------- kernel/cgroup/cpuset.c | 14 +++++++++++--- 2 files changed, 14 insertions(+), 10 deletions(-)
kernel/cgroup/cpuset-internal.h +3 -7
@@ -154,13 +154,6 @@ struct cpuset {/* partition root state */ int partition_root_state; -/* -* Whether cpuset is a remote partition. -* It used to be a list anchoring all remote partitions — we can switch back -* to a list if we need to iterate over the remote partitions. -*/ -bool remote_partition; -/* * Set to true when a remote partition is undergoing update. */
@@ -186,6 +179,9 @@ struct cpuset {/* Handle for cpuset.cpus.partition */ struct cgroup_file partition_file; +/* Remote partition silbling list anchored at remote_children */ +struct list_head remote_sibling; +#ifdef CONFIG_CPUSETS_V1 struct fmeter fmeter;/* memory_pressure filter */
kernel/cgroup/cpuset.c +11 -3
@@ -148,6 +148,11 @@ static cpumask_var_t subpartitions_cpus; /* RWCS */*/ static cpumask_var_t isolated_cpus;/* CSCB */ +/* +* List of remote partition root children +*/ +static struct list_head remote_children;/* RWCS */ +/* * Set if housekeeping cpumasks are to be updated. */
@@ -286,6 +291,7 @@ struct cpuset top_cpuset = { .flags = BIT(CS_CPU_EXCLUSIVE) | BIT(CS_MEM_EXCLUSIVE) | BIT(CS_SCHED_LOAD_BALANCE), .partition_root_state = PRS_ROOT, + .remote_sibling = LIST_HEAD_INIT(top_cpuset.remote_sibling), .dl_bw_cpu = -1, };
@@ -1213,7 +1219,7 @@ static void reset_partition_data(struct cpuset *cs)static inline bool is_remote_partition (struct cpuset *cs) { -return cs->remote_partition; +return !list_empty (&cs->remote_sibling); }static inline bool is_local_partition (struct cpuset *cs)
@@ -1742,7 +1748,7 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,spin_lock_irq (&callback_lock);partition_xcpus_add (new_prs,NULL , tmp->new_cpus); - cs->remote_partition = true; +list_add (&cs->remote_sibling, &remote_children);cpumask_copy (cs->effective_xcpus, tmp->new_cpus);spin_unlock_irq (&callback_lock);cpuset_force_rebuild ();
@@ -1769,7 +1775,7 @@ static void remote_partition_disable(struct cpuset *cs, struct tmpmasks *tmp) {WARN_ON_ONCE (!is_remote_partition (cs));spin_lock_irq (&callback_lock); - cs->remote_partition = false; +list_del_init (&cs->remote_sibling);partition_xcpus_del (cs->partition_root_state,NULL , cs->effective_xcpus);if (cs->prs_err) cs->partition_root_state = -cs->partition_root_state;
@@ -3677,6 +3683,7 @@ cpuset_css_alloc(struct cgroup_subsys_state *parent_css)return ERR_PTR (-ENOMEM );__set_bit (CS_SCHED_LOAD_BALANCE , &cs->flags); +INIT_LIST_HEAD (&cs->remote_sibling);cpuset1_init (cs);/* Set CS_MEMORY_MIGRATE for default hierarchy */
@@ -3920,6 +3927,7 @@ int __init cpuset_init(void)cpumask_setall (top_cpuset.exclusive_cpus);nodes_setall (top_cpuset.effective_mems);init_llist_node (&top_cpuset.attach_node); +INIT_LIST_HEAD (&remote_children);cpuset1_init (&top_cpuset);
Signature · 1 line
2.56.0
Waiman Long
maintainerPATCH v4 5/6 cgroup/cpuset: Handle remote partitions before local partitions in cpuset_handle_hotplug()
In reply to Waiman LongWhen there is a valid remote partition underneath an invalid local partition with subpartitions_cpus cleared because of missing active CPUs in the top_cpuset partition, the current iteration logic in cpuset_handle_hotplug() will handle the invalid local partition first and potentially make the local partition valid. The remote partition will be handled later with the potential that both the local and remote partitions will be valid with overlapping exclusive CPUs violating the cpu exclusivity rule.
Fix this problem by adding a new remote_partition_scan() helper to iterate the existing remote partitions first before iterating down the hierarchy to handle the local partitions. That also means that when subpartitions_cpus is cleared, remote partitions have precedence to acquire the exclusive CPUs first before local partitions. It also gives remote partition the ability to recover from the clearing of subpartitions_cpus as all the remote partitions will be disabled before this commit.
A remote partition can only be created if there is at least one active CPU in its effective_xcpus. So remote_partition_scan() will also invalidate the remote partition if there is no active CPU left in its effective_xcpus. This requires a minor change in a test_cpuset_prs.sh test case where a remote partition with no active CPU left will now be invalidated.
With the addition of the remote_partition_scan() call in cpuset_handle_hotplug(), there is no longer a need to do a remote partition check in cpuset_hotplug_update_tasks() and the corresponding code can be removed.
While at it, the subpartition_cpus clearing logic is enhanced to clear it when no domain housekeeping CPUs are left in the top_cpuset partition. When the remote partition is disabled, the effective_cpus is modified in a such a way that cpus_updated will be set true later in cpuset_hotplug_update_tasks() so that the CPU affinity of the tasks within the partition will be updated. This will also fix a bug reported by sashiko that CPU affinity will not be updated when remote_partition_disable() is called in such circumstance.
When efffective_cpus is read during this interim period, the content will be invalid before and after this commit.
--- kernel/cgroup/cpuset.c | 97 ++++++++++++++++--- .../selftests/cgroup/test_cpuset_prs.sh | 2 +- 2 files changed, 84 insertions(+), 15 deletions(-)
kernel/cgroup/cpuset.c +83 -14
@@ -153,6 +153,12 @@ static cpumask_var_t isolated_cpus; /* CSCB */*/ static struct list_head remote_children;/* RWCS */ +/* +* Temporary cpumask for local cpumask computation +* The cpumask may be changed when a cpuset helper is being called. +*/ +static cpumask_var_t tmp_cpus;/* RWCS */ +/* * Set if housekeeping cpumasks are to be updated. */
@@ -1873,6 +1879,67 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *xcpus,remote_partition_disable (cs, tmp); } +/** +* remote_partition_scan: scan remote partitions to check exclusive CPUs +* @tmpmask: ptr to tmpmasks structure +* +* This function is called in cpuset_handle_hotplug() after the update on +* top_cpuset.effective_cpus when there is cpu online/offline operation. +* As subpartitions_cpus may have been cleared, we need to check if the remote +* partition should acquire the exclusive CPUs again without impacting +* effective_cpus or it needs to be disabled when one or more of the following +* conditions is true. +* 1) All the domain housekeeping CPUs in the top_cpuset will be taken away +* 2) A populated partition with no active CPU or no active CPU in +* effective_xcpus +*/ +static void remote_partition_scan (struct tmpmasks *tmp) +{ +struct cpuset *cs, *next; + +rcu_read_lock (); +list_for_each_entry_safe (cs, next, &remote_children, remote_sibling) { +bool noxcpus = !cpumask_subset (cs->effective_xcpus, subpartitions_cpus); +bool nocpus = !cpumask_intersects (cs->effective_xcpus, cpu_active_mask) || + (!cpumask_intersects (cs->effective_cpus, cpu_active_mask) && +partition_is_populated (cs,NULL )); +int cpu; + +cpumask_and (tmp_cpus, top_cpuset.effective_cpus, +housekeeping_cpumask (HK_TYPE_DOMAIN_BOOT )); +if (noxcpus && !nocpus && !cpumask_subset (tmp_cpus, cs->effective_xcpus)) { +/* +* Reacquire exclusive CPUs & update top_cpuset.effective_cpus +*/ +spin_lock_irq (&callback_lock); +cpumask_andnot (subpartitions_cpus, subpartitions_cpus, + cs->effective_xcpus); +cpumask_andnot (top_cpuset.effective_cpus, top_cpuset.effective_xcpus, + cs->effective_xcpus); +spin_unlock_irq (&callback_lock); +continue ; + } + +if (noxcpus || nocpus ||cpumask_empty (tmp_cpus)) { +/* +* Disable remote partition & modifiy effective_cpus +* so that cpus_updated will be set true in +* cpuset_hotplug_update_tasks(). +*/ +WRITE_ONCE (cs->prs_err,PERR_HOTPLUG ); +remote_partition_disable (cs, tmp); + cpu =cpumask_first (cs->effective_cpus); +spin_lock_irq (&callback_lock); +if (cpu < nr_cpu_ids) +cpumask_clear_cpu (cpu, cs->effective_cpus); +else +cpumask_set_cpu (0 , cs->effective_cpus); +spin_unlock_irq (&callback_lock); + } + } +rcu_read_unlock (); +} +/** * update_parent_effective_cpumask - update effective_cpus mask of parent cpuset * @cs: The cpuset that requests change in partition root state
@@ -3919,6 +3986,7 @@ int __init cpuset_init(void)BUG_ON (!zalloc_cpumask_var (&subpartitions_cpus,GFP_KERNEL ));BUG_ON (!zalloc_cpumask_var (&isolated_cpus,GFP_KERNEL ));BUG_ON (!zalloc_cpumask_var (&isolated_hk_cpus,GFP_KERNEL )); +BUG_ON (!alloc_cpumask_var (&tmp_cpus,GFP_KERNEL ));cpumask_setall (top_cpuset.cpus_allowed);nodes_setall (top_cpuset.mems_allowed);
@@ -3993,14 +4061,6 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)if (!tmp || !cs->partition_root_state) goto update_tasks; -if (is_remote_partition (cs) && - (!cpumask_subset (cs->effective_xcpus, subpartitions_cpus) || - (cpumask_empty (&new_cpus) &&partition_is_populated (cs,NULL )))) { -WRITE_ONCE (cs->prs_err,PERR_HOTPLUG ); -remote_partition_disable (cs, tmp); -compute_effective_cpumask (&new_cpus, cs, parent); - } -/* * Force the partition to become invalid if either one of * the following conditions hold:
@@ -4104,22 +4164,31 @@ static void cpuset_handle_hotplug(void)if (!on_dfl)cpumask_copy (top_cpuset.cpus_allowed, &new_cpus);/* -* Make sure that CPUs allocated to child partitions -* do not show up in effective_cpus. If no CPU is left, -* we clear the subpartitions_cpus & let the child partitions -* fight for the CPUs again. +* Make sure that CPUs allocated to child partitions do not +* show up in effective_cpus. If no housekeeping CPU is left, +* we clear the subpartitions_cpus, scan the remote partitions +* to determine if they can remain valid & let the descendant +* local partitions fight for the CPUs again. */ if (!cpumask_empty (subpartitions_cpus)) { +cpumask_andnot (&new_cpus, cpu_active_mask, +housekeeping_cpumask (HK_TYPE_DOMAIN_BOOT ));if (cpumask_subset (&new_cpus, subpartitions_cpus)) {cpumask_clear (subpartitions_cpus); +cpumask_copy (&new_cpus, cpu_active_mask); }else { -cpumask_andnot (&new_cpus, &new_cpus, +cpumask_andnot (&new_cpus, cpu_active_mask, subpartitions_cpus); } }cpumask_copy (top_cpuset.effective_cpus, &new_cpus);spin_unlock_irq (&callback_lock); -/* we don't mess with cpumasks of tasks in top_cpuset */ +/* +* We don't mess with cpumasks of tasks in top_cpuset, but we +* have to examine impact on remote partition roots if present. +*/ +if (cpuset_v2 () && ptmp) +remote_partition_scan (ptmp); }/* synchronize mems_allowed to N_MEMORY */
tools/testing/selftests/cgroup/test_cpuset_prs.sh +1 -1
@@ -378,7 +378,7 @@ TEST_MATRIX=( # Remote partition offline tests " C0-3 C1-3 C2-3 . X2-3 X2-3 X2-3:P2:O2=0 . 0 A1:0-1|A2:1|A3:3 A1:P0|A3:P2 2-3" " C0-3 C1-3 C2-3 . X2-3 X2-3 X2-3:P2:O2=0 O2=1 0 A1:0-1|A2:1|A3:2-3 A1:P0|A3:P2 2-3" - " C0-3 C1-3 C3 . X2-3 X2-3 P2:O3=0 . 0 A1:0-2|A2:1-2|A3: A1:P0|A3:P2 3" + " C0-3 C1-3 C3 . X2-3 X2-3 P2:O3=0 . 0 A1:0-2|A2:1-2|A3:1-2 A1:P0|A3:P-2 3|" " C0-3 C1-3 C3 . X2-3 X2-3 T:P2:O3=0 . 0 A1:0-2|A2:1-2|A3:1-2 A1:P0|A3:P-2 3|" # An invalidated remote partition cannot self-recover from hotplug
Signature · 1 line
2.56.0
Ridong Chen
reviewerRe: [PATCH-next v4 5/6] cgroup/cpuset: Handle remote partitions before local partitions in cpuset_handle_hotplug()
Quoted 9 lines · Waiman Long
When there is a valid remote partition underneath an invalid local partition with
subpartitions_cpuscleared because of missing active CPUs in thetop_cpusetpartition, the current iteration logic incpuset_handle_hotplug()will handle the invalid local partition first and potentially make the local partition valid. The remote partition will be handled later with the potential that both the local and remote partitions will be valid with overlapping exclusive CPUs violating the cpu exclusivity rule.
Hi Longman,
Patch 5 and Guopeng's patch [1] handle the same case: after hotplug, top_cpuset has no usable CPU left (no active CPU, or no active boot domain housekeeping CPU). But they take different approaches:
- Current code: clear subpartitions_cpus and invalidate all the partitions.
- [1]: only invalidate the partitions that can give an active housekeeping CPU back to top_cpuset.
- Patch 5: invalidate all the local partitions but keep the remote ones valid if possible.
I think we should agree on the expected behavior first. My preference is to keep it simple and invalidate all the partitions as the current code does. The problem is that in the same hotplug pass, after valid partition A is invalidated and its CPUs are given back to top_cpuset, partition B, which was invalid before, may become valid again. B puts its CPUs back into subpartitions_cpus, so a valid partition visited after B is not invalidated (patch 1), or B overlaps with a remote partition underneath it (patch 5). If invalid partitions are not allowed to become valid again in that pass, both issues go away. Do we really need to keep some partitions valid in this case?
[1] https://lore.kernel.org/cgroups/[email protected]/
Signature · 2 lines
Best regards Ridong
kernel test robot
build botRe: [PATCH-next v4 5/6] cgroup/cpuset: Handle remote partitions before local partitions in cpuset_handle_hotplug()
In reply to Waiman Longkernel test robot noticed the following build warnings:
Hi Waiman,
kernel test robot noticed the following build warnings:
[auto build test WARNING on tj-cgroup/for-next]
[also build test WARNING on next-20261113]
[cannot apply to linus/master v7.4-rc2]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Waiman-Long/cgroup-cpuset-Fix-incorrect-partition-invalidation-logic-in-cpuset_hotplug_update_tasks/20261115-202304 base: https://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git for-next patch link: https://lore.kernel.org/r/20261116012330.2271010-6-longman%40redhat.com patch subject: [PATCH-next v4 5/6] cgroup/cpuset: Handle remote partitions before local partitions in cpuset_handle_hotplug() config: riscv-defconfig (https://download.01.org/0day-ci/archive/20261116/[email protected]/config) compiler: clang version 24.0.0git (https://github.com/llvm/llvm-project ca14e3d05c4e63d6e8103945c86b958ed2f55d5f) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20261116/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
>> Warning: kernel/cgroup/cpuset.c:1895 function parameter 'tmp' not described in 'remote_partition_scan' >> Warning: kernel/cgroup/cpuset.c:1895 Excess function parameter 'tmpmask' description in 'remote_partition_scan' (did you mean one of: 'tmp') >> Warning: kernel/cgroup/cpuset.c:1895 function parameter 'tmp' not described in 'remote_partition_scan' >> Warning: kernel/cgroup/cpuset.c:1895 Excess function parameter 'tmpmask' description in 'remote_partition_scan' (did you mean one of: 'tmp')
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
Re: [PATCH-next v4 5/6] cgroup/cpuset: Handle remote partitions before local partitions in cpuset_handle_hotplug()
In reply to Waiman LongQuoted 42 lines · Waiman Long
When there is a valid remote partition underneath an invalid local partition with
subpartitions_cpuscleared because of missing active CPUs in thetop_cpusetpartition, the current iteration logic incpuset_handle_hotplug()will handle the invalid local partition first and potentially make the local partition valid. The remote partition will be handled later with the potential that both the local and remote partitions will be valid with overlapping exclusive CPUs violating the cpu exclusivity rule.Fix this problem by adding a new
remote_partition_scan()helper to iterate the existing remote partitions first before iterating down the hierarchy to handle the local partitions. That also means that whensubpartitions_cpusis cleared, remote partitions have precedence to acquire the exclusive CPUs first before local partitions. It also gives remote partition the ability to recover from the clearing ofsubpartitions_cpusas all the remote partitions will be disabled before this commit.A remote partition can only be created if there is at least one active CPU in its
effective_xcpus. Soremote_partition_scan()will also invalidate the remote partition if there is no active CPU left in itseffective_xcpus. This requires a minor change in atest_cpuset_prs.shtest case where a remote partition with no active CPU left will now be invalidated.With the addition of the
remote_partition_scan()call incpuset_handle_hotplug(), there is no longer a need to do a remote partition check incpuset_hotplug_update_tasks()and the corresponding code can be removed.While at it, the
subpartition_cpusclearing logic is enhanced to clear it when no domain housekeeping CPUs are left in thetop_cpusetpartition. When the remote partition is disabled, theeffective_cpusis modified in a such a way thatcpus_updatedwill be set true later incpuset_hotplug_update_tasks()so that the CPU affinity of the tasks within the partition will be updated. This will also fix a bug reported by sashiko that CPU affinity will not be updated whenremote_partition_disable()is called in such circumstance.When
efffective_cpusis read during this interim period, the content will be invalid before and after this commit.
Quoted 6 lines
Fixes 181c8e091aae ("cgroup/cpuset: Introduce remote partition")Reported-by Guopeng Zhang <[email protected]>Reported-by Sashiko <[email protected]>Signed-off-by Waiman Long <[email protected]>
Hi,Longman
Tested all six patches on next-20261113 (32-CPU VM, cgroup v2, no boot isolation, all CPUs initially online). I can still reproduce an invalid local partition becoming valid while its remote descendant remains valid:
cd /sys/fs/cgroup
echo +cpuset > cgroup.subtree_control
echo 0 > /sys/devices/system/cpu/cpu31/online
mkdir A
echo 0-30 > A/cpuset.cpus
echo 0-30 > A/cpuset.cpus.exclusive
echo root > A/cpuset.cpus.partition # invalid local root
echo +cpuset > A/cgroup.subtree_control
mkdir A/B
echo 20-25 > A/B/cpuset.cpus
echo 20-25 > A/B/cpuset.cpus.exclusive
echo root > A/B/cpuset.cpus.partition # valid remote partition
echo 1 > /sys/devices/system/cpu/cpu31/online
After CPU31 comes online, A is a valid root (exclusive.effective 0-30), A/B is still a valid remote (exclusive.effective 20-25), and the top cpuset's effective CPUs are 31. The kernel reports:
WARNING: cpuset.c:1654 compute_partition_effective_cpumask()
WARN_ON_ONCE(is_remote_partition(child))Two more results:
1. Local partitions break too, no remote involved. test_cpuset_prs.sh fails at TEST_MATRIX[19] (C0-3:P1 C2-3:P1 + O2=0): with cpu2 offlined, both partitions become "root invalid" instead of remaining valid. A1's effective_cpus then falls back to 0-1,3 instead of 0-1.
2. Booted with isolcpus=31, the same kernel passes the whole suite.
Thanks,
Guopeng> ---
Quoted 163 lines
kernel/cgroup/cpuset.c | 97 ++++++++++++++++--- .../selftests/cgroup/test_cpuset_prs.sh | 2 +- 2 files changed, 84 insertions(+), 15 deletions(-)kernel/cgroup/cpuset.c +83
-14@@ -153,6 +153,12 @@ static cpumask_var_t isolated_cpus; /* CSCB */*/ static struct list_head remote_children;/* RWCS */ +/* +* Temporary cpumask for local cpumask computation +* The cpumask may be changed when a cpuset helper is being called. +*/ +static cpumask_var_t tmp_cpus;/* RWCS */ +/* * Set if housekeeping cpumasks are to be updated. */ @@ -1873,6 +1879,67 @@ static void remote_cpus_update(struct cpuset *cs, struct cpumask *xcpus,remote_partition_disable (cs, tmp); } +/** +* remote_partition_scan: scan remote partitions to check exclusive CPUs +* @tmpmask: ptr to tmpmasks structure +* +* This function is called in cpuset_handle_hotplug() after the update on +* top_cpuset.effective_cpus when there is cpu online/offline operation. +* As subpartitions_cpus may have been cleared, we need to check if the remote +* partition should acquire the exclusive CPUs again without impacting +* effective_cpus or it needs to be disabled when one or more of the following +* conditions is true. +* 1) All the domain housekeeping CPUs in the top_cpuset will be taken away +* 2) A populated partition with no active CPU or no active CPU in +* effective_xcpus +*/ +static void remote_partition_scan (struct tmpmasks *tmp) +{ +struct cpuset *cs, *next; + +rcu_read_lock (); +list_for_each_entry_safe (cs, next, &remote_children, remote_sibling) { +bool noxcpus = !cpumask_subset (cs->effective_xcpus, subpartitions_cpus); +bool nocpus = !cpumask_intersects (cs->effective_xcpus, cpu_active_mask) || + (!cpumask_intersects (cs->effective_cpus, cpu_active_mask) && +partition_is_populated (cs,NULL )); +int cpu; + +cpumask_and (tmp_cpus, top_cpuset.effective_cpus, +housekeeping_cpumask (HK_TYPE_DOMAIN_BOOT )); +if (noxcpus && !nocpus && !cpumask_subset (tmp_cpus, cs->effective_xcpus)) { +/* +* Reacquire exclusive CPUs & update top_cpuset.effective_cpus +*/ +spin_lock_irq (&callback_lock); +cpumask_andnot (subpartitions_cpus, subpartitions_cpus, + cs->effective_xcpus); +cpumask_andnot (top_cpuset.effective_cpus, top_cpuset.effective_xcpus, + cs->effective_xcpus); +spin_unlock_irq (&callback_lock); +continue ; + } + +if (noxcpus || nocpus ||cpumask_empty (tmp_cpus)) { +/* +* Disable remote partition & modifiy effective_cpus +* so that cpus_updated will be set true in +* cpuset_hotplug_update_tasks(). +*/ +WRITE_ONCE (cs->prs_err,PERR_HOTPLUG ); +remote_partition_disable (cs, tmp); + cpu =cpumask_first (cs->effective_cpus); +spin_lock_irq (&callback_lock); +if (cpu < nr_cpu_ids) +cpumask_clear_cpu (cpu, cs->effective_cpus); +else +cpumask_set_cpu (0 , cs->effective_cpus); +spin_unlock_irq (&callback_lock); + } + } +rcu_read_unlock (); +} +/** * update_parent_effective_cpumask - update effective_cpus mask of parent cpuset * @cs: The cpuset that requests change in partition root state @@ -3919,6 +3986,7 @@ int __init cpuset_init(void)BUG_ON (!zalloc_cpumask_var (&subpartitions_cpus,GFP_KERNEL ));BUG_ON (!zalloc_cpumask_var (&isolated_cpus,GFP_KERNEL ));BUG_ON (!zalloc_cpumask_var (&isolated_hk_cpus,GFP_KERNEL )); +BUG_ON (!alloc_cpumask_var (&tmp_cpus,GFP_KERNEL ));cpumask_setall (top_cpuset.cpus_allowed);nodes_setall (top_cpuset.mems_allowed);@@ -3993,14 +4061,6 @@ static void cpuset_hotplug_update_tasks(struct cpuset *cs, struct tmpmasks *tmp)if (!tmp || !cs->partition_root_state) goto update_tasks; -if (is_remote_partition (cs) && - (!cpumask_subset (cs->effective_xcpus, subpartitions_cpus) || - (cpumask_empty (&new_cpus) &&partition_is_populated (cs,NULL )))) { -WRITE_ONCE (cs->prs_err,PERR_HOTPLUG ); -remote_partition_disable (cs, tmp); -compute_effective_cpumask (&new_cpus, cs, parent); - } -/* * Force the partition to become invalid if either one of * the following conditions hold: @@ -4104,22 +4164,31 @@ static void cpuset_handle_hotplug(void)if (!on_dfl)cpumask_copy (top_cpuset.cpus_allowed, &new_cpus);/* -* Make sure that CPUs allocated to child partitions -* do not show up in effective_cpus. If no CPU is left, -* we clear the subpartitions_cpus & let the child partitions -* fight for the CPUs again. +* Make sure that CPUs allocated to child partitions do not +* show up in effective_cpus. If no housekeeping CPU is left, +* we clear the subpartitions_cpus, scan the remote partitions +* to determine if they can remain valid & let the descendant +* local partitions fight for the CPUs again. */ if (!cpumask_empty (subpartitions_cpus)) { +cpumask_andnot (&new_cpus, cpu_active_mask, +housekeeping_cpumask (HK_TYPE_DOMAIN_BOOT ));if (cpumask_subset (&new_cpus, subpartitions_cpus)) {cpumask_clear (subpartitions_cpus); +cpumask_copy (&new_cpus, cpu_active_mask); }else { -cpumask_andnot (&new_cpus, &new_cpus, +cpumask_andnot (&new_cpus, cpu_active_mask, subpartitions_cpus); } }cpumask_copy (top_cpuset.effective_cpus, &new_cpus);spin_unlock_irq (&callback_lock); -/* we don't mess with cpumasks of tasks in top_cpuset */ +/* +* We don't mess with cpumasks of tasks in top_cpuset, but we +* have to examine impact on remote partition roots if present. +*/ +if (cpuset_v2 () && ptmp) +remote_partition_scan (ptmp); }/* synchronize mems_allowed to N_MEMORY */ tools/testing/selftests/cgroup/test_cpuset_prs.sh +1
-1@@ -378,7 +378,7 @@ TEST_MATRIX=( # Remote partition offline tests " C0-3 C1-3 C2-3 . X2-3 X2-3 X2-3:P2:O2=0 . 0 A1:0-1|A2:1|A3:3 A1:P0|A3:P2 2-3" " C0-3 C1-3 C2-3 . X2-3 X2-3 X2-3:P2:O2=0 O2=1 0 A1:0-1|A2:1|A3:2-3 A1:P0|A3:P2 2-3" - " C0-3 C1-3 C3 . X2-3 X2-3 P2:O3=0 . 0 A1:0-2|A2:1-2|A3: A1:P0|A3:P2 3" + " C0-3 C1-3 C3 . X2-3 X2-3 P2:O3=0 . 0 A1:0-2|A2:1-2|A3:1-2 A1:P0|A3:P-2 3|" " C0-3 C1-3 C3 . X2-3 X2-3 T:P2:O3=0 . 0 A1:0-2|A2:1-2|A3:1-2 A1:P0|A3:P-2 3|" # An invalidated remote partition cannot self-recover from hotplug
Waiman Long
maintainerPATCH v4 6/6 cgroup/cpuset: Fix lockdep_is_cpuset_held()
In reply to Waiman LongCommit a84097e625f2 ("cgroup/cpuset: Call housekeeping_update() without holding cpus_read_lock") changes lockdep_is_cpuset_held() to return true if either cpuset_mutex or cpuset_top_mutex is held. lockdep_is_cpuset_held() is used by housekeeping_dereference_check() as one of the conditions that rcu_read_lock() is not needed to access HK_TYPE_DOMAIN cpumask that can be changed at run time. In reality, it should return true only if cpuset_top_mutex is held as housekeeping_update() is called with only cpuset_top_mutex held.
If only cpuset_mutex is held, it is possible that there can be a parallel housekeeping_update() call leading to a potential use-after-free situation when HK_TYPE_DOMAIN housekeeping cpumask is being accessed. It also means that rcu_read_lock will be required if only cpuset_mutex is being held like when calling from hotplug, scheduler or cgroup.
--- kernel/cgroup/cpuset-v1.c | 16 ++++++++++------ kernel/cgroup/cpuset.c | 9 +++++++-- 2 files changed, 17 insertions(+), 8 deletions(-)
kernel/cgroup/cpuset-v1.c +10 -6
@@ -686,6 +686,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,int csn;/* how many cpuset ptrs in csa so far */ int i, j;/* indices for partition finding loops */ cpumask_var_t *doms;/* resulting partition; i.e. sched domains */ +cpumask_var_t hk_cpus;/* HK_TYPE_DOMAIN housekeeping CPUs */ struct sched_domain_attr *dattr;/* attributes for custom domains */ int ndoms =0 ;/* number of sched domains in result */ int nslot;/* next empty doms[] struct cpumask slot */
@@ -698,6 +699,11 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains, dattr =NULL ; csa =NULL ; +alloc_cpumask_var (&hk_cpus,GFP_KERNEL ); +scoped_guard (rcu) { +cpumask_copy (hk_cpus,housekeeping_cpumask (HK_TYPE_DOMAIN )); + } +/* Special case for the 99% of systems with one, full, sched domain */ if (is_sched_load_balance (&top_cpuset)) { ndoms =1 ;
@@ -710,9 +716,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains, *dattr =SD_ATTR_INIT ;update_domain_attr_tree (dattr, &top_cpuset); } -cpumask_and (doms[0 ], top_cpuset.effective_cpus, -housekeeping_cpumask (HK_TYPE_DOMAIN )); - +cpumask_and (doms[0 ], top_cpuset.effective_cpus, hk_cpus); goto done; }
@@ -736,8 +740,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,*/ if (!cpumask_empty (cp->cpus_allowed) && !(is_sched_load_balance (cp) && -cpumask_intersects (cp->cpus_allowed, -housekeeping_cpumask (HK_TYPE_DOMAIN )))) +cpumask_intersects (cp->cpus_allowed, hk_cpus)))continue ; if (is_sched_load_balance(cp) &&
@@ -794,7 +797,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains, *(dattr + nslot) =SD_ATTR_INIT ; }cpumask_or (dp, dp, csa[j]->effective_cpus); -cpumask_and (dp, dp,housekeeping_cpumask (HK_TYPE_DOMAIN )); +cpumask_and (dp, dp, hk_cpus);if (dattr)update_domain_attr_tree (dattr + nslot, csa[j]); }
@@ -806,6 +809,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains, done:kfree (csa); +free_cpumask_var (hk_cpus);/* * Fallback to the default domain if kmalloc() failed.
kernel/cgroup/cpuset.c +7 -2
@@ -343,10 +343,13 @@ void cpuset_full_unlock(void) }#ifdef CONFIG_LOCKDEP +/* +* Holding cpuset_top_mutex will guarantee that a parallel +* housekeeping_update() call modifying housekeeping cpumasks is not possible. +*/ bool lockdep_is_cpuset_held (void ) { -return lockdep_is_held (&cpuset_mutex) || -lockdep_is_held (&cpuset_top_mutex); +return lockdep_is_held (&cpuset_top_mutex); }#endif
@@ -1416,8 +1419,10 @@ static bool prstate_housekeeping_conflict(int new_prs, int parent_prs, } } +rcu_read_lock ();cpumask_and (full_hk_cpus,housekeeping_cpumask (HK_TYPE_KERNEL_NOISE ),housekeeping_cpumask (HK_TYPE_DOMAIN )); +rcu_read_unlock ();cpumask_andnot (full_hk_cpus, full_hk_cpus, isolated_cpus);cpumask_and (full_hk_cpus, full_hk_cpus, cpu_active_mask); res = !cpumask_weight_andnot (full_hk_cpus, new_cpus);
Signature · 1 line
2.56.0
Ridong Chen
reviewerRe: [PATCH-next v4 6/6] cgroup/cpuset: Fix lockdep_is_cpuset_held()
Quoted 35 lines · Waiman Long
Commit
a84097e625f2("cgroup/cpuset: Callhousekeeping_update()without holdingcpus_read_lock") changeslockdep_is_cpuset_held()to return true if eithercpuset_mutexorcpuset_top_mutexis held.lockdep_is_cpuset_held()is used byhousekeeping_dereference_check()as one of the conditions thatrcu_read_lock()is not needed to accessHK_TYPE_DOMAINcpumask that can be changed at run time. In reality, it should return true only ifcpuset_top_mutexis held ashousekeeping_update()is called with onlycpuset_top_mutexheld.If only
cpuset_mutexis held, it is possible that there can be a parallelhousekeeping_update()call leading to a potential use-after-free situation whenHK_TYPE_DOMAINhousekeeping cpumask is being accessed. It also means thatrcu_read_lockwill be required if onlycpuset_mutexis being held like when calling from hotplug, scheduler or cgroup.Fixes a84097e625f2 ("cgroup/cpuset: Call housekeeping_update() without holding cpus_read_lock")Signed-off-by Waiman Long <[email protected]>--- kernel/cgroup/cpuset-v1.c | 16 ++++++++++------ kernel/cgroup/cpuset.c | 9 +++++++-- 2 files changed, 17 insertions(+), 8 deletions(-)kernel/cgroup/cpuset-v1.c +1
-0@@ -686,6 +686,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,int csn;/* how many cpuset ptrs in csa so far */ int i, j;/* indices for partition finding loops */ cpumask_var_t *doms;/* resulting partition; i.e. sched domains */ +cpumask_var_t hk_cpus;/* HK_TYPE_DOMAIN housekeeping CPUs */ struct sched_domain_attr *dattr;/* attributes for custom domains */ int ndoms =0 ;/* number of sched domains in result */ int nslot;/* next empty doms[] struct cpumask slot */
kernel/cgroup/cpuset-v1.c int cpuset1_generate_sched_domains(cpumask_var_t **domains, · line 706
Quoted 8 lines
kernel/cgroup/cpuset-v1.c +5
-0@@ -698,6 +699,11 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains, dattr =NULL ; csa =NULL ; +alloc_cpumask_var (&hk_cpus,GFP_KERNEL ); +scoped_guard (rcu) { +cpumask_copy (hk_cpus,housekeeping_cpumask (HK_TYPE_DOMAIN )); + } +
The return value of alloc_cpumask_var() isn't checked. With CONFIG_CPUMASK_OFFSTACK=y, hk_cpus is NULL on failure. Maybe goto done like the other allocation failures here?
Also, rebuild_sched_domains() can be called from outside cpuset, e.g. arch_topology and the energy model, with only cpus_read_lock and cpuset_mutex held. partition_sched_domains_locked() reads HK_TYPE_DOMAIN in asym_cpu_capacity_scan() and in its fallback path. Will these trigger a lockdep warning after this change? If so, do they need an RCU read lock as well?
Quoted 72 lines
kernel/cgroup/cpuset-v1.c +4
-6/* Special case for the 99% of systems with one, full, sched domain */ if (is_sched_load_balance (&top_cpuset)) { ndoms =1 ;@@ -710,9 +716,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains, *dattr =SD_ATTR_INIT ;update_domain_attr_tree (dattr, &top_cpuset); } -cpumask_and (doms[0 ], top_cpuset.effective_cpus, -housekeeping_cpumask (HK_TYPE_DOMAIN )); - +cpumask_and (doms[0 ], top_cpuset.effective_cpus, hk_cpus); goto done; }@@ -736,8 +740,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,*/ if (!cpumask_empty (cp->cpus_allowed) && !(is_sched_load_balance (cp) && -cpumask_intersects (cp->cpus_allowed, -housekeeping_cpumask (HK_TYPE_DOMAIN )))) +cpumask_intersects (cp->cpus_allowed, hk_cpus)))continue ; if (is_sched_load_balance(cp) &&@@ -794,7 +797,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains, *(dattr + nslot) =SD_ATTR_INIT ; }cpumask_or (dp, dp, csa[j]->effective_cpus); -cpumask_and (dp, dp,housekeeping_cpumask (HK_TYPE_DOMAIN )); +cpumask_and (dp, dp, hk_cpus);if (dattr)update_domain_attr_tree (dattr + nslot, csa[j]); }@@ -806,6 +809,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains, done:kfree (csa); +free_cpumask_var (hk_cpus);/* * Fallback to the default domain if kmalloc() failed. kernel/cgroup/cpuset.c +7
-2@@ -343,10 +343,13 @@ void cpuset_full_unlock(void) }#ifdef CONFIG_LOCKDEP +/* +* Holding cpuset_top_mutex will guarantee that a parallel +* housekeeping_update() call modifying housekeeping cpumasks is not possible. +*/ bool lockdep_is_cpuset_held (void ) { -return lockdep_is_held (&cpuset_mutex) || -lockdep_is_held (&cpuset_top_mutex); +return lockdep_is_held (&cpuset_top_mutex); }#endif @@ -1416,8 +1419,10 @@ static bool prstate_housekeeping_conflict(int new_prs, int parent_prs, } } +rcu_read_lock ();cpumask_and (full_hk_cpus,housekeeping_cpumask (HK_TYPE_KERNEL_NOISE ),housekeeping_cpumask (HK_TYPE_DOMAIN )); +rcu_read_unlock ();cpumask_andnot (full_hk_cpus, full_hk_cpus, isolated_cpus);cpumask_and (full_hk_cpus, full_hk_cpus, cpu_active_mask); res = !cpumask_weight_andnot (full_hk_cpus, new_cpus);
Signature · 2 lines
Best regards Ridong
Waiman Long
maintainerRe: [PATCH-next v4 6/6] cgroup/cpuset: Fix lockdep_is_cpuset_held()
Quoted 52 lines · Ridong Chen
Quoted 36 lines · Waiman Long
Commit
a84097e625f2("cgroup/cpuset: Callhousekeeping_update()without holdingcpus_read_lock") changeslockdep_is_cpuset_held()to return true if eithercpuset_mutexorcpuset_top_mutexis held.lockdep_is_cpuset_held()is used byhousekeeping_dereference_check()as one of the conditions thatrcu_read_lock()is not needed to accessHK_TYPE_DOMAINcpumask that can be changed at run time. In reality, it should return true only ifcpuset_top_mutexis held ashousekeeping_update()is called with onlycpuset_top_mutexheld.If only
cpuset_mutexis held, it is possible that there can be a parallelhousekeeping_update()call leading to a potential use-after-free situation whenHK_TYPE_DOMAINhousekeeping cpumask is being accessed. It also means thatrcu_read_lockwill be required if onlycpuset_mutexis being held like when calling from hotplug, scheduler or cgroup.Fixes a84097e625f2 ("cgroup/cpuset: Call housekeeping_update() without holding cpus_read_lock")Signed-off-by Waiman Long <[email protected]>---
kernel/cgroup/cpuset-v1.c| 16 ++++++++++------
kernel/cgroup/cpuset.c | 9 +++++++--2 files changed, 17 insertions(+), 8 deletions(-)kernel/cgroup/cpuset-v1.c +0
-0@@ -686,6 +686,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,int csn; /* how many cpuset ptrs in csa so far */ int i, j; /* indices for partition finding loops */
cpumask_var_t*doms; /* resulting partition; i.e. sched domains */ +cpumask_var_thk_cpus; /*HK_TYPE_DOMAINhousekeeping CPUs */ structsched_domain_attr*dattr; /* attributes for custom domains */ int ndoms = 0; /* number of sched domains in result */ int nslot; /* next empty doms[] struct cpumask slot */kernel/cgroup/cpuset-v1.c int cpuset1_generate_sched_domains(cpumask_var_t · line 706
Quoted 8 lines
kernel/cgroup/cpuset-v1.c +4
-0@@ -698,6 +699,11 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains, dattr = NULL; csa = NULL; + alloc_cpumask_var(&hk_cpus, GFP_KERNEL); + scoped_guard(rcu) { + cpumask_copy(hk_cpus, housekeeping_cpumask(HK_TYPE_DOMAIN)); + } +The return value of
alloc_cpumask_var()isn't checked. WithCONFIG_CPUMASK_OFFSTACK=y,hk_cpusis NULL on failure. Maybe goto done like the other allocation failures here?
Quoted 8 lines
Also,
rebuild_sched_domains()can be called from outside cpuset, e.g.arch_topologyand the energy model, with onlycpus_read_lockandcpuset_mutexheld.partition_sched_domains_locked()readsHK_TYPE_DOMAINinasym_cpu_capacity_scan()and in its fallback path. Will these trigger a lockdep warning after this change? If so, do they need an RCU read lock as well?
Yes, I realize that some additional scheduler functions will need to be updated as well for this to work. I am planning to have an alternative solution to work around this problem. I am going to drop this patch and send out another patch soon.
Cheers,
Longman
Quoted 71 lines
Quoted 70 lines
kernel/cgroup/cpuset-v1.c +0
-0/* Special case for the 99% of systems with one, full, sched domain */ if (is_sched_load_balance(&top_cpuset)) {ndoms = 1;
kernel/cgroup/cpuset-v1.c +1
-3@@ -710,9 +716,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains, *dattr = SD_ATTR_INIT; update_domain_attr_tree(dattr, &top_cpuset); } - cpumask_and(doms[0], top_cpuset.effective_cpus, - housekeeping_cpumask(HK_TYPE_DOMAIN)); - + cpumask_and(doms[0], top_cpuset.effective_cpus, hk_cpus); goto done; }@@ -736,8 +740,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains, */
if (!cpumask_empty(cp->cpus_allowed) && !(is_sched_load_balance(cp) &&
- cpumask_intersects(cp->cpus_allowed,
-housekeeping_cpumask(HK_TYPE_DOMAIN))))
+cpumask_intersects(cp->cpus_allowed, hk_cpus))) continue;
if (is_sched_load_balance(cp)&&kernel/cgroup/cpuset-v1.c +1
-1@@ -794,7 +797,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains, *(dattr + nslot) = SD_ATTR_INIT; } cpumask_or(dp, dp, csa[j]->effective_cpus); - cpumask_and(dp, dp, housekeeping_cpumask(HK_TYPE_DOMAIN)); + cpumask_and(dp, dp, hk_cpus); if (dattr)
update_domain_attr_tree(dattr + nslot, csa[j]); }kernel/cgroup/cpuset-v1.c +0
-0@@ -806,6 +809,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,done:
kfree(csa);
+free_cpumask_var(hk_cpus);
/*
* Fallback to the default domain ifkmalloc()failed.kernel/cgroup/cpuset.c +3
-0@@ -343,10 +343,13 @@ void cpuset_full_unlock(void) } #ifdef CONFIG_LOCKDEP +/* +* Holding cpuset_top_mutex will guarantee that a parallel +* housekeeping_update() call modifying housekeeping cpumasks is not possible.
+ */
boollockdep_is_cpuset_held(void)
{
- returnlockdep_is_held(&cpuset_mutex)||
-lockdep_is_held(&cpuset_top_mutex);
+ returnlockdep_is_held(&cpuset_top_mutex);
}
#endif
@@ -1416,8 +1419,10 @@ static bool prstate_housekeeping_conflict(intnew_prs, intparent_prs, }
}
+rcu_read_lock();
cpumask_and(full_hk_cpus, housekeeping_cpumask(HK_TYPE_KERNEL_NOISE), housekeeping_cpumask(HK_TYPE_DOMAIN));
+rcu_read_unlock();
cpumask_andnot(full_hk_cpus, full_hk_cpus, isolated_cpus);cpumask_and(full_hk_cpus, full_hk_cpus, cpu_active_mask);
res = !cpumask_weight_andnot(full_hk_cpus,new_cpus);
Re: [PATCH-next v4 6/6] cgroup/cpuset: Fix lockdep_is_cpuset_held()
In reply to Waiman LongQuoted 35 lines · Waiman Long
Commit
a84097e625f2("cgroup/cpuset: Callhousekeeping_update()without holdingcpus_read_lock") changeslockdep_is_cpuset_held()to return true if eithercpuset_mutexorcpuset_top_mutexis held.lockdep_is_cpuset_held()is used byhousekeeping_dereference_check()as one of the conditions thatrcu_read_lock()is not needed to accessHK_TYPE_DOMAINcpumask that can be changed at run time. In reality, it should return true only ifcpuset_top_mutexis held ashousekeeping_update()is called with onlycpuset_top_mutexheld.If only
cpuset_mutexis held, it is possible that there can be a parallelhousekeeping_update()call leading to a potential use-after-free situation whenHK_TYPE_DOMAINhousekeeping cpumask is being accessed. It also means thatrcu_read_lockwill be required if onlycpuset_mutexis being held like when calling from hotplug, scheduler or cgroup.Fixes a84097e625f2 ("cgroup/cpuset: Call housekeeping_update() without holding cpus_read_lock")Signed-off-by Waiman Long <[email protected]>--- kernel/cgroup/cpuset-v1.c | 16 ++++++++++------ kernel/cgroup/cpuset.c | 9 +++++++-- 2 files changed, 17 insertions(+), 8 deletions(-)kernel/cgroup/cpuset-v1.c +1
-0@@ -686,6 +686,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,int csn;/* how many cpuset ptrs in csa so far */ int i, j;/* indices for partition finding loops */ cpumask_var_t *doms;/* resulting partition; i.e. sched domains */ +cpumask_var_t hk_cpus;/* HK_TYPE_DOMAIN housekeeping CPUs */ struct sched_domain_attr *dattr;/* attributes for custom domains */ int ndoms =0 ;/* number of sched domains in result */ int nslot;/* next empty doms[] struct cpumask slot */
kernel/cgroup/cpuset-v1.c int cpuset1_generate_sched_domains(cpumask_var_t **domains, · line 706
Quoted 8 lines
kernel/cgroup/cpuset-v1.c +5
-0@@ -698,6 +699,11 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains, dattr =NULL ; csa =NULL ; +alloc_cpumask_var (&hk_cpus,GFP_KERNEL ); +scoped_guard (rcu) { +cpumask_copy (hk_cpus,housekeeping_cpumask (HK_TYPE_DOMAIN )); + } +
Can alloc_cpumask_var() fail? Should we add the following check?
if (!alloc_cpumask_var(&hk_cpus, GFP_KERNEL)) goto done;
Thanks,
Guopeng
Quoted 72 lines
kernel/cgroup/cpuset-v1.c +4
-6/* Special case for the 99% of systems with one, full, sched domain */ if (is_sched_load_balance (&top_cpuset)) { ndoms =1 ;@@ -710,9 +716,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains, *dattr =SD_ATTR_INIT ;update_domain_attr_tree (dattr, &top_cpuset); } -cpumask_and (doms[0 ], top_cpuset.effective_cpus, -housekeeping_cpumask (HK_TYPE_DOMAIN )); - +cpumask_and (doms[0 ], top_cpuset.effective_cpus, hk_cpus); goto done; }@@ -736,8 +740,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains,*/ if (!cpumask_empty (cp->cpus_allowed) && !(is_sched_load_balance (cp) && -cpumask_intersects (cp->cpus_allowed, -housekeeping_cpumask (HK_TYPE_DOMAIN )))) +cpumask_intersects (cp->cpus_allowed, hk_cpus)))continue ; if (is_sched_load_balance(cp) &&@@ -794,7 +797,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains, *(dattr + nslot) =SD_ATTR_INIT ; }cpumask_or (dp, dp, csa[j]->effective_cpus); -cpumask_and (dp, dp,housekeeping_cpumask (HK_TYPE_DOMAIN )); +cpumask_and (dp, dp, hk_cpus);if (dattr)update_domain_attr_tree (dattr + nslot, csa[j]); }@@ -806,6 +809,7 @@ int cpuset1_generate_sched_domains(cpumask_var_t **domains, done:kfree (csa); +free_cpumask_var (hk_cpus);/* * Fallback to the default domain if kmalloc() failed. kernel/cgroup/cpuset.c +7
-2@@ -343,10 +343,13 @@ void cpuset_full_unlock(void) }#ifdef CONFIG_LOCKDEP +/* +* Holding cpuset_top_mutex will guarantee that a parallel +* housekeeping_update() call modifying housekeeping cpumasks is not possible. +*/ bool lockdep_is_cpuset_held (void ) { -return lockdep_is_held (&cpuset_mutex) || -lockdep_is_held (&cpuset_top_mutex); +return lockdep_is_held (&cpuset_top_mutex); }#endif @@ -1416,8 +1419,10 @@ static bool prstate_housekeeping_conflict(int new_prs, int parent_prs, } } +rcu_read_lock ();cpumask_and (full_hk_cpus,housekeeping_cpumask (HK_TYPE_KERNEL_NOISE ),housekeeping_cpumask (HK_TYPE_DOMAIN )); +rcu_read_unlock ();cpumask_andnot (full_hk_cpus, full_hk_cpus, isolated_cpus);cpumask_and (full_hk_cpus, full_hk_cpus, cpu_active_mask); res = !cpumask_weight_andnot (full_hk_cpus, new_cpus);