cgroup/cpuset: prevent overlapping local and remote partition creation
Subsystems (4)
PATCH 1/1 cgroup/cpuset: prevent overlapping local and remote partition creation
Fix two partition validation bugs in kernel/cgroup/cpuset.c:
1. In remote_partition_enable(), check whether the requested effective xcpus intersect parent->subpartitions_cpus so a remote partition cannot claim CPUs already delegated to a local child partition.
2. In validate_partition(), verify that enabling a local partition does not overlap CPUs already allocated to an active remote partition.
---
kernel/cgroup/cpuset.c +10 -3
@@ -713,7 +713,10 @@ static inline bool cpus_excl_conflict(struct cpuset *trial, struct cpuset *siblireturn true;/* Exclusive_cpus cannot intersect */ -return cpumask_intersects (trial->exclusive_cpus, sibling->exclusive_cpus); +return cpumask_intersects (trial->exclusive_cpus, +cpumask_empty (sibling->exclusive_cpus) + ? sibling->effective_xcpus + : sibling->exclusive_cpus); }static inline bool mems_excl_conflict (struct cpuset *cs1,struct cpuset *cs2)
@@ -1591,8 +1594,8 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,* above it or remote partition root underneath it is not allowed. */ compute_excpus (cs, tmp->new_cpus); -WARN_ON_ONCE (cpumask_intersects (tmp->new_cpus, subpartitions_cpus));if (!cpumask_intersects (tmp->new_cpus, cpu_active_mask) || +cpumask_intersects (tmp->new_cpus, subpartitions_cpus) ||cpumask_subset (top_cpuset.effective_cpus, tmp->new_cpus))return PERR_INVCPUS ; if (((new_prs == PRS_ISOLATED) &&
@@ -2411,6 +2414,10 @@ static enum prs_errcode validate_partition(struct cpuset *cs, struct cpuset *triif (cpumask_empty (trialcs->effective_xcpus))return PERR_INVCPUS ; +if ((parent == &top_cpuset) && +cpumask_intersects (trialcs->effective_xcpus, subpartitions_cpus)) +return PERR_REMOTE ; +if (prstate_housekeeping_conflict (trialcs->partition_root_state, trialcs->effective_xcpus))return PERR_HKEEPING ;
@@ -2970,7 +2977,7 @@ static int update_prstate(struct cpuset *cs, int new_prs)* local or remote partition. */ if ((parent == &top_cpuset) && -cpumask_intersects (cs->exclusive_cpus, subpartitions_cpus)) { +cpumask_intersects (user_xcpus (cs), subpartitions_cpus)) { err =PERR_REMOTE ; goto out; }
Ridong Chen
reviewerQuoted 11 lines · Hui Peng
Fix two partition validation bugs in
kernel/cgroup/cpuset.c:1. In
remote_partition_enable(), check whether the requested effective xcpus intersectparent->subpartitions_cpusso a remote partition cannot claim CPUs already delegated to a local child partition.
2. Invalidate_partition(), verify that enabling a local partition does not overlap CPUs already allocated to an active remote partition.Fixes aa7d3a56a20f ("cpuset: fix warning when disabling remote partition")Assisted-by LLMSigned-off-by Hui Peng <[email protected]>
Could you please first describe what the issue is and how it can be triggered? Or is there any reproducer?
Quoted 47 lines
---
kernel/cgroup/cpuset.c +10
-3@@ -713,7 +713,10 @@ static inline bool cpus_excl_conflict(struct cpuset *trial, struct cpuset *siblireturn true;/* Exclusive_cpus cannot intersect */ -return cpumask_intersects (trial->exclusive_cpus, sibling->exclusive_cpus); +return cpumask_intersects (trial->exclusive_cpus, +cpumask_empty (sibling->exclusive_cpus) + ? sibling->effective_xcpus + : sibling->exclusive_cpus); }static inline bool mems_excl_conflict (struct cpuset *cs1,struct cpuset *cs2)@@ -1591,8 +1594,8 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,* above it or remote partition root underneath it is not allowed. */ compute_excpus (cs, tmp->new_cpus); -WARN_ON_ONCE (cpumask_intersects (tmp->new_cpus, subpartitions_cpus));if (!cpumask_intersects (tmp->new_cpus, cpu_active_mask) || +cpumask_intersects (tmp->new_cpus, subpartitions_cpus) ||cpumask_subset (top_cpuset.effective_cpus, tmp->new_cpus))return PERR_INVCPUS ; if (((new_prs == PRS_ISOLATED) &&@@ -2411,6 +2414,10 @@ static enum prs_errcode validate_partition(struct cpuset *cs, struct cpuset *triif (cpumask_empty (trialcs->effective_xcpus))return PERR_INVCPUS ; +if ((parent == &top_cpuset) && +cpumask_intersects (trialcs->effective_xcpus, subpartitions_cpus)) +return PERR_REMOTE ; +if (prstate_housekeeping_conflict (trialcs->partition_root_state, trialcs->effective_xcpus))return PERR_HKEEPING ;@@ -2970,7 +2977,7 @@ static int update_prstate(struct cpuset *cs, int new_prs)* local or remote partition. */ if ((parent == &top_cpuset) && -cpumask_intersects (cs->exclusive_cpus, subpartitions_cpus)) { +cpumask_intersects (user_xcpus (cs), subpartitions_cpus)) { err =PERR_REMOTE ; goto out; }
Signature · 2 lines
Best regards Ridong
Quoted 2 lines · Ridong Chen
Could you please first describe what the issue is and how it can be triggered? Or is there any reproducer?
Hi Ridong,
Thanks for taking a look. Below is a detailed description of both scenarios, how each is triggered in the current code, the dmesg WARNINGs produced on 7.3.0-rc3, and the minimized shell reproducers.
------------------------------------------------------------------------
Scenario 1: Enabling a remote partition underneath an ancestor local
partition root (remote_partition_enable)
------------------------------------------------------------------------
How it happens:
1. Suppose top-level cgroup "A" is a valid local partition root owning CPU 1 (cpuset.cpus = 1, cpuset.cpus.exclusive = 1, cpuset.cpus.partition = root). Enabling "A" calls partition_xcpus_add(..., &top_cpuset, {1}), which adds CPU 1 to subpartitions_cpus.
2. Child "A/B" is a normal non-partition member (cpuset.cpus.partition = member) with cpuset.cpus = 1 and cpuset.cpus.exclusive = 1.
3. Grandchild "A/B/D" has cpuset.cpus = 1, cpuset.cpus.exclusive = 1. When "root" is written to "A/B/D/cpuset.cpus.partition", update_prstate() checks is_partition_valid(parent) on A/B (which is false because A/B is PRS_MEMBER) and therefore calls remote_partition_enable(D, PRS_ROOT, &tmpmask).
4. In remote_partition_enable(), compute_excpus(D, tmp->new_cpus) walks up D -> B -> A and computes tmp->new_cpus = {1}. Because ancestor "A" is already a valid local partition root, CPU 1 is already present in subpartitions_cpus.
5. remote_partition_enable() hits:
WARN_ON_ONCE(cpumask_intersects(tmp->new_cpus, subpartitions_cpus)); at kernel/cgroup/cpuset.c:1594 and continues without returning an error, enabling "A/B/D" as a valid remote partition on CPU 1 while ancestor "A" is simultaneously a valid local partition on CPU 1, which then also triggers a second WARNING in rebuild_sched_domains_locked() (kernel/cgroup/cpuset.c:906).
Minimized reproducer (Scenario 1):
#!/bin/sh
mkdir -p /tmp/cg1
mount -t cgroup2 none /tmp/cg1
echo "+cpuset" > /tmp/cg1/cgroup.subtree_control
mkdir /tmp/cg1/A
echo 1 > /tmp/cg1/A/cpuset.cpus
echo 1 > /tmp/cg1/A/cpuset.cpus.exclusive echo root > /tmp/cg1/A/cpuset.cpus.partition echo "+cpuset" > /tmp/cg1/A/cgroup.subtree_control
mkdir /tmp/cg1/A/B
echo 1 > /tmp/cg1/A/B/cpuset.cpus
echo 1 > /tmp/cg1/A/B/cpuset.cpus.exclusive echo "+cpuset" > /tmp/cg1/A/B/cgroup.subtree_control
mkdir /tmp/cg1/A/B/D
echo 1 > /tmp/cg1/A/B/D/cpuset.cpus
echo 1 > /tmp/cg1/A/B/D/cpuset.cpus.exclusive echo root > /tmp/cg1/A/B/D/cpuset.cpus.partition
dmesg output on 7.3.0-rc3:
WARNING: kernel/cgroup/cpuset.c:1594 at update_prstate+0xbef/0xd70, CPU#2 Call Trace: cpuset_partition_write+0x112/0x140 WARNING: kernel/cgroup/cpuset.c:906 at rebuild_sched_domains_locked+0x4f2/0x710, CPU#2 Call Trace: cpuset_partition_write+0x112/0x140
------------------------------------------------------------------------
Scenario 2: Invalid top-level local partition transitioning to valid
over an active remote child partition (validate_partition)
------------------------------------------------------------------------
How it happens:
1. Suppose top-level cgroup "A" is configured as a partition root (echo root > A/cpuset.cpus.partition) and given all online CPUs (e.g., echo 0-3 > A/cpuset.cpus.exclusive on a 4-CPU system), which transitions "A" to PRS_INVALID_ROOT ("root invalid (Parent unable to distribute cpu downstream)").
2. Because "A" is invalid (!is_partition_valid(A)), creating child "A/R" with cpuset.cpus = 1, cpuset.cpus.exclusive = 1, and cpuset.cpus.partition = root succeeds via remote_partition_enable(), making "A/R" a valid remote partition on CPU 1 (adding CPU 1 to subpartitions_cpus and removing CPU 1 from top_cpuset.effective_cpus).
3. Next, shrinking "A/cpuset.cpus.exclusive" from "0-3" to "1" calls update_exclusive_cpumask() -> partition_cpus_change(A, trialcs, &tmp) -> validate_partition(A, trialcs).
4. Unlike update_prstate() (which checks (parent == &top_cpuset) && cpumask_intersects(..., subpartitions_cpus) and returns PERR_REMOTE), validate_partition() omits the subpartitions_cpus check and returns PERR_NONE (0).
5. partition_cpus_change() then calls update_parent_effective_cpumask(A, partcmd_update, trialcs->effective_xcpus, &tmp) to activate "A" on CPU 1, which is already removed from top_cpuset.effective_cpus by "A/R", triggering WARN_ON_ONCE(!cpumask_subset(tmp->new_cpus, parent->effective_cpus)) at kernel/cgroup/cpuset.c:1943 and WARN_ON_ONCE(old_prs < 0) in partition_xcpus_del() at kernel/cgroup/cpuset.c:1342.
Minimized reproducer (Scenario 2, on a 4-CPU VM):
#!/bin/sh
mkdir -p /tmp/cg2
mount -t cgroup2 none /tmp/cg2
echo "+cpuset" > /tmp/cg2/cgroup.subtree_control
mkdir /tmp/cg2/A
echo root > /tmp/cg2/A/cpuset.cpus.partition echo 0-3 > /tmp/cg2/A/cpuset.cpus.exclusive echo "+cpuset" > /tmp/cg2/A/cgroup.subtree_control
mkdir /tmp/cg2/A/R
echo 1 > /tmp/cg2/A/R/cpuset.cpus
echo 1 > /tmp/cg2/A/R/cpuset.cpus.exclusive
echo root > /tmp/cg2/A/R/cpuset.cpus.partition
# Shrink A's exclusive_cpus to 1
echo 1 > /tmp/cg2/A/cpuset.cpus.exclusive
dmesg output on 7.3.0-rc3:
WARNING: kernel/cgroup/cpuset.c:1943 at update_parent_effective_cpumask+0x189b/0x1fd0 Call Trace: cpuset_write_resmask+0xcf2/0x1690 WARNING: kernel/cgroup/cpuset.c:1342 at partition_xcpus_del+0x15b/0x1b0 Call Trace: update_parent_effective_cpumask+0x118c/0x1fd0 cpuset_write_resmask+0xcf2/0x1690
Also, in v2 of the patch, I will refine the check in validate_partition() to exclude cs's own existing effective_xcpus when cs is already a valid local partition (!is_partition_valid(cs)), and split the two scenarios into separate patches with these reproducers in the commit messages if you prefer.
Best regards,
Hui Peng
PATCH v2 1/2 cgroup/cpuset: return PERR_NOCPUS in remote_partition_enable() on subpartitions_cpus conflict
In reply to Ridong ChenWhen a remote partition is created underneath an existing local partition via a non-partition (PRS_MEMBER) intermediate cgroup, update_prstate() sees parent->partition_root_state == PRS_MEMBER and calls remote_partition_enable().
Commit 86888c7bd117 ("cgroup/cpuset: Add warnings to catch inconsistency in exclusive CPUs") replaced the cpumask_intersects(tmp->new_cpus, subpartitions_cpus) error check in remote_partition_enable() with WARN_ON_ONCE(). As a result, remote_partition_enable() emits a warning and proceeds to enable the remote partition on CPUs that are already owned by the ancestor local partition in subpartitions_cpus.
This can be reproduced on Linux 7.3.0-rc3 with:
mkdir -p /tmp/cg1
mount -t cgroup2 none /tmp/cg1
echo "+cpuset" > /tmp/cg1/cgroup.subtree_control
mkdir /tmp/cg1/A
echo 1 > /tmp/cg1/A/cpuset.cpus
echo 1 > /tmp/cg1/A/cpuset.cpus.exclusive echo root > /tmp/cg1/A/cpuset.cpus.partition echo "+cpuset" > /tmp/cg1/A/cgroup.subtree_control
mkdir /tmp/cg1/A/B
echo 1 > /tmp/cg1/A/B/cpuset.cpus
echo 1 > /tmp/cg1/A/B/cpuset.cpus.exclusive echo "+cpuset" > /tmp/cg1/A/B/cgroup.subtree_control
mkdir /tmp/cg1/A/B/D
echo 1 > /tmp/cg1/A/B/D/cpuset.cpus
echo 1 > /tmp/cg1/A/B/D/cpuset.cpus.exclusive echo root > /tmp/cg1/A/B/D/cpuset.cpus.partition
which triggers:
WARNING: kernel/cgroup/cpuset.c:1594 at remote_partition_enable+0x1c1/0x300
and leaves both /tmp/cg1/A and /tmp/cg1/A/B/D as active root partitions claiming exclusive CPU 1.
Fix this by returning PERR_NOCPUS when tmp->new_cpus intersects subpartitions_cpus in remote_partition_enable(), matching the error code used by remote_cpus_update() for the same subpartitions_cpus conflict, and add a regression test case to tools/testing/selftests/cgroup/test_cpuset_prs.sh.
Tested in QEMU on Linux 7.3.0-rc3 using the reproducer above and tools/testing/selftests/cgroup/test_cpuset_prs.sh.
---
Changes in v2:
- Split into a 2-patch series (1/2 for remote_partition_enable(), 2/2 for validate_partition() and cpus_excl_conflict()).
- Return PERR_NOCPUS instead of PERR_INVCPUS when tmp->new_cpus intersects subpartitions_cpus to match remote_cpus_update(), as suggested by Guopeng Zhang.
- Add regression test to tools/testing/selftests/cgroup/test_cpuset_prs.sh and include the step-by-step reproducer in the commit description as requested by Ridong Chen.
kernel/cgroup/cpuset.c | 3 ++- tools/testing/selftests/cgroup/test_cpuset_prs.sh | 2 ++ 2 files changed, 4 insertions(+), 1 deletion(-)
kernel/cgroup/cpuset.c +2 -1
@@ -1591,10 +1591,11 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,* above it or remote partition root underneath it is not allowed. */ compute_excpus (cs, tmp->new_cpus); -WARN_ON_ONCE (cpumask_intersects (tmp->new_cpus, subpartitions_cpus));if (!cpumask_intersects (tmp->new_cpus, cpu_active_mask) ||cpumask_subset (top_cpuset.effective_cpus, tmp->new_cpus))return PERR_INVCPUS ; +if (cpumask_intersects (tmp->new_cpus, subpartitions_cpus)) +return PERR_NOCPUS ;if (((new_prs ==PRS_ISOLATED ) && !isolated_cpus_can_update (tmp->new_cpus,NULL )) ||prstate_housekeeping_conflict (new_prs, tmp->new_cpus))
tools/testing/selftests/cgroup/test_cpuset_prs.sh +2 -0
@@ -298,6 +298,8 @@ TEST_MATRIX=( " C0-4:X2-4 C1-4:X2-4:P2 C2-4:X4:P1 \ . . . X1 . 0 A1:0-1|A2:2-4|A3:2-4 \ A1:P0|A2:P2|A3:P-1 2-4" + " CX1-3:P1 CX1-3 CX1-3 . . . P1 . 0 A1:1-3|A2:1-3|A3:1-3 \ + A1:P1|A2:P0|A3:P-1" # Remote partition offline tests " C0-3 C1-3 C2-3 . X2-3 X2-3 X2-3:P2:O2=0 . 0 A1:0-1|A2:1|A3:3 A1:P0|A3:P2 2-3"
Signature · 1 line
2.49.0
PATCH v2 2/2 cgroup/cpuset: prevent local partition activation over remote partition and sibling xcpus conflict
Commit 6da580ec656a ("cgroup/cpuset: Don't allow creation of local partition over a remote one") added a check in update_prstate() to reject creating a top-level local partition whose cs->exclusive_cpus intersects subpartitions_cpus.
However, two paths still allow overlapping exclusive CPU ownership between a top-level local partition and a remote partition or sibling:
1. If a top-level cgroup A1 has cpuset.cpus.exclusive set, is first set to "root" while cpuset.cpus is empty (becoming "root invalid"), has a child A1/A2 enabled as a remote partition on those exclusive CPUs, and then receives CPUs via A1/cpuset.cpus, update_cpumasks_hier() calls validate_partition() without going through update_prstate(). Because validate_partition() does not check trialcs->exclusive_cpus against subpartitions_cpus, A1 transitions from "root invalid" to a valid local partition over the existing remote partition A1/A2 and triggers:
WARNING: kernel/cgroup/cpuset.c:1866 at update_parent_effective_cpumask+0x54a/0x8a0
2. In cpus_excl_conflict(), when a valid local partition A1 uses implicit exclusive CPUs (cpuset.cpus set without cpuset.cpus.exclusive, so sibling->exclusive_cpus is empty while sibling->effective_xcpus is populated), a sibling cgroup B1 can still set cpuset.cpus.exclusive on the same CPUs because cpus_excl_conflict() only checks sibling->exclusive_cpus.
Fix these issues by:
- Checking trialcs->exclusive_cpus against subpartitions_cpus in validate_partition() when !is_partition_valid(cs).
- Falling back to sibling->effective_xcpus in cpus_excl_conflict() when sibling->exclusive_cpus is empty.
- Adding regression test cases for both paths to tools/testing/selftests/cgroup/test_cpuset_prs.sh.
Tested in QEMU on Linux 7.3.0-rc3 using tools/testing/selftests/cgroup/test_cpuset_prs.sh.
---
Changes in v2:
- Split from the remote_partition_enable() fix into patch 2/2.
- Check trialcs->exclusive_cpus (rather than effective_xcpus) in validate_partition() when !is_partition_valid(cs) so sibling local partitions with overlapping cpuset.cpus continue to share remaining available CPUs as expected by test_cpuset_prs.sh.
- Add regression tests to tools/testing/selftests/cgroup/test_cpuset_prs.sh.
kernel/cgroup/cpuset.c | 9 ++++++++- tools/testing/selftests/cgroup/test_cpuset_prs.sh | 3 +++ 2 files changed, 11 insertions(+), 1 deletion(-)
kernel/cgroup/cpuset.c +8 -1
@@ -713,7 +713,10 @@ static inline bool cpus_excl_conflict(struct cpuset *trial, struct cpuset *siblireturn true;/* Exclusive_cpus cannot intersect */ -return cpumask_intersects (trial->exclusive_cpus, sibling->exclusive_cpus); +return cpumask_intersects (trial->exclusive_cpus, +cpumask_empty (sibling->exclusive_cpus) + ? sibling->effective_xcpus + : sibling->exclusive_cpus); }static inline bool mems_excl_conflict (struct cpuset *cs1,struct cpuset *cs2)
@@ -2415,6 +2418,10 @@ static enum prs_errcode validate_partition(struct cpuset *cs, struct cpuset *triif (cpumask_empty (trialcs->effective_xcpus))return PERR_INVCPUS ; +if ((parent == &top_cpuset) && !is_partition_valid (cs) && +cpumask_intersects (trialcs->exclusive_cpus, subpartitions_cpus)) +return PERR_REMOTE ; +if (prstate_housekeeping_conflict (trialcs->partition_root_state, trialcs->effective_xcpus))return PERR_HKEEPING ;
tools/testing/selftests/cgroup/test_cpuset_prs.sh +3 -0
@@ -321,6 +321,8 @@ TEST_MATRIX=( " C1-3:P2 X4:P2 . . . X3:P2 . . 0 A1:1-2|XA1:1-3|A2:3:XA2:3 A1:P2|A2:P2 1-3" " C0-3:P2 . . C4-6 C0-4 . . . 0 A1:0-4|B1:5-6 A1:P2|B1:P0" " C0-3:P2 . . C4-6 C0-4:C0-3 . . . 0 A1:0-3|B1:4-6 A1:P2|B1:P0 0-3" + " X1-3:P1 CX2-3:P2 . . C1-3 . . . 0 A1:1|A2:2-3|XA2:2-3 \ + A1:P-1|A2:P2 2-3" # Local partition invalidation tests " C0-3:X1-3:P2 C1-3:X2-3:P2 C2-3:X3:P2 \
@@ -443,6 +445,7 @@ TEST_MATRIX=( # Changes to cpuset.cpus.exclusive that violate exclusivity rule is rejected " C0-3 . . C4-5 X0-3 . . X3-5 1 A1:0-3|B1:4-5" + " C0-3:P1 . . C4-5 . . . X3-5 1 A1:0-3|B1:4-5 A1:P1|B1:P0" # cpuset.cpus.exclusive cannot be set to a superset of sibling's cpuset.cpus " C0-3 . . C4-5 X3-5 . . . 1 A1:0-3|B1:4-5"
Signature · 1 line
2.49.0
Re: [PATCH v2 2/2] cgroup/cpuset: prevent local partition activation over remote partition and sibling xcpus conflict
Quoted 34 lines · Hui Peng
Commit
6da580ec656a("cgroup/cpuset: Don't allow creation of local partition over a remote one") added a check inupdate_prstate()to reject creating a top-level local partition whosecs->exclusive_cpusintersectssubpartitions_cpus.However, two paths still allow overlapping exclusive CPU ownership between a top-level local partition and a remote partition or sibling:
1. If a top-level cgroup A1 has cpuset.cpus.exclusive set, is first set to "root" while cpuset.cpus is empty (becoming "root invalid"), has a child A1/A2 enabled as a remote partition on those exclusive CPUs, and then receives CPUs via A1/cpuset.cpus,
update_cpumasks_hier()callsvalidate_partition()without going throughupdate_prstate(). Becausevalidate_partition()does not checktrialcs->exclusive_cpusagainstsubpartitions_cpus, A1 transitions from "root invalid" to a valid local partition over the existing remote partition A1/A2 and triggers:WARNING: kernel/cgroup/cpuset.c:1866 at update_parent_effective_cpumask+0x54a/0x8a02. In
cpus_excl_conflict(), when a valid local partition A1 uses implicit exclusive CPUs (cpuset.cpus set without cpuset.cpus.exclusive, sosibling->exclusive_cpusis empty whilesibling->effective_xcpusis populated), a sibling cgroup B1 can still set cpuset.cpus.exclusive on the same CPUs becausecpus_excl_conflict()only checkssibling->exclusive_cpus.Fix these issues by:
- Checkingtrialcs->exclusive_cpusagainstsubpartitions_cpusinvalidate_partition()when !is_partition_valid(cs).
- Falling back tosibling->effective_xcpusincpus_excl_conflict()whensibling->exclusive_cpusis empty.
- Adding regression test cases for both paths totools/testing/selftests/cgroup/test_cpuset_prs.sh.
Quoted 4 lines
Tested in QEMU on Linux 7.3.0-rc3 using
tools/testing/selftests/cgroup/test_cpuset_prs.sh.Fixes 6da580ec656a ("cgroup/cpuset: Don't allow creation of local partition over a remote one")
I think this patch is still missing a Fixes: tag for the cpus_excl_conflict() change.
The validate_partition() change can be covered by 6da580ec656a.
However, the sibling CPU ownership issue fixed in cpus_excl_conflict() was introduced by the following commit:
Thanks,
Guopeng
Quoted 63 lines
Signed-off-by Hui Peng <[email protected]>---
Changes in v2:
- Split from theremote_partition_enable()fix into patch 2/2.
- Checktrialcs->exclusive_cpus(rather thaneffective_xcpus) invalidate_partition()when !is_partition_valid(cs) so sibling local partitions with overlapping cpuset.cpus continue to share remaining available CPUs as expected bytest_cpuset_prs.sh.
- Add regression tests totools/testing/selftests/cgroup/test_cpuset_prs.sh.kernel/cgroup/cpuset.c | 9 ++++++++- tools/testing/selftests/cgroup/test_cpuset_prs.sh | 3 +++ 2 files changed, 11 insertions(+), 1 deletion(-)kernel/cgroup/cpuset.c +8
-1@@ -713,7 +713,10 @@ static inline bool cpus_excl_conflict(struct cpuset *trial, struct cpuset *siblireturn true;/* Exclusive_cpus cannot intersect */ -return cpumask_intersects (trial->exclusive_cpus, sibling->exclusive_cpus); +return cpumask_intersects (trial->exclusive_cpus, +cpumask_empty (sibling->exclusive_cpus) + ? sibling->effective_xcpus + : sibling->exclusive_cpus); }static inline bool mems_excl_conflict (struct cpuset *cs1,struct cpuset *cs2)@@ -2415,6 +2418,10 @@ static enum prs_errcode validate_partition(struct cpuset *cs, struct cpuset *triif (cpumask_empty (trialcs->effective_xcpus))return PERR_INVCPUS ; +if ((parent == &top_cpuset) && !is_partition_valid (cs) && +cpumask_intersects (trialcs->exclusive_cpus, subpartitions_cpus)) +return PERR_REMOTE ; +if (prstate_housekeeping_conflict (trialcs->partition_root_state, trialcs->effective_xcpus))return PERR_HKEEPING ;tools/testing/selftests/cgroup/test_cpuset_prs.sh +3
-0@@ -321,6 +321,8 @@ TEST_MATRIX=( " C1-3:P2 X4:P2 . . . X3:P2 . . 0 A1:1-2|XA1:1-3|A2:3:XA2:3 A1:P2|A2:P2 1-3" " C0-3:P2 . . C4-6 C0-4 . . . 0 A1:0-4|B1:5-6 A1:P2|B1:P0" " C0-3:P2 . . C4-6 C0-4:C0-3 . . . 0 A1:0-3|B1:4-6 A1:P2|B1:P0 0-3" + " X1-3:P1 CX2-3:P2 . . C1-3 . . . 0 A1:1|A2:2-3|XA2:2-3 \ + A1:P-1|A2:P2 2-3" # Local partition invalidation tests " C0-3:X1-3:P2 C1-3:X2-3:P2 C2-3:X3:P2 \@@ -443,6 +445,7 @@ TEST_MATRIX=( # Changes to cpuset.cpus.exclusive that violate exclusivity rule is rejected " C0-3 . . C4-5 X0-3 . . X3-5 1 A1:0-3|B1:4-5" + " C0-3:P1 . . C4-5 . . . X3-5 1 A1:0-3|B1:4-5 A1:P1|B1:P0" # cpuset.cpus.exclusive cannot be set to a superset of sibling's cpuset.cpus " C0-3 . . C4-5 X3-5 . . . 1 A1:0-3|B1:4-5"
Re: [PATCH v2 2/2] cgroup/cpuset: prevent local partition activation over remote partition and sibling xcpus conflict
Quoted 8 lines · Guopeng Zhang
I think this patch is still missing a Fixes: tag for the
cpus_excl_conflict()change.The
validate_partition()change can be covered by6da580ec656a.However, the sibling CPU ownership issue fixed in
cpus_excl_conflict()was introduced by the following commit:Fixes 2a3602030d80 ("cgroup/cpuset: Don't invalidate sibling partitions on cpuset.cpus conflict")
Thanks for pointing this out, Guopeng!
In v3, I have split the validate_partition() change (with
[PATCH v3 3/3]) so that each patch fixes a single logical issue with its corresponding Fixes: tag.
Best regards,
Hui
PATCH v3 1/3 cgroup/cpuset: return PERR_NOCPUS in remote_partition_enable()
In reply to Guopeng ZhangCommit 6da580ec656a ("cgroup/cpuset: Don't allow creation of local partition over a remote one") updated update_prstate() to prevent creating a top-level local partition whose exclusive CPUs conflict with an existing remote partition.
However, if a top-level local partition A1 is created on CPUs 1-3, has a non-partition member child A1/A2 with cpuset.cpus.exclusive = 1-3, and a grandchild A1/A2/A3 is then enabled as a partition root on 1-3, update_prstate() sees !is_partition_valid(parent) (since A1/A2 is a member) and calls remote_partition_enable(). Because remote_partition_enable() does not check whether tmp->new_cpus intersects subpartitions_cpus, A1/A2/A3 is enabled as a remote partition on CPUs already owned by A1, triggering:
WARNING: kernel/cgroup/cpuset.c:1594 at update_prstate+0xbef/0xd70
Return PERR_NOCPUS in remote_partition_enable() when tmp->new_cpus intersects subpartitions_cpus, matching the error code used by remote_cpus_update() for subpartitions_cpus conflicts, and add a corresponding regression test case to tools/testing/selftests/cgroup/test_cpuset_prs.sh.
Tested in QEMU on Linux 7.3.0-rc3 using tools/testing/selftests/cgroup/test_cpuset_prs.sh.
---
Changes in v3:
- Split the series into 3 patches (one per logical change and Fixes: tag).
kernel/cgroup/cpuset.c | 4 +++- tools/testing/selftests/cgroup/test_cpuset_prs.sh | 2 ++ 2 files changed, 5 insertions(+), 1 deletion(-)
kernel/cgroup/cpuset.c +4 -1
@@ -1678,8 +1678,10 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,* Callers must have validated that effective_xcpus is not empty. */ spin_lock_irq (&callback_lock); -if (cpumask_subset (top_cpuset.effective_cpus, tmp->new_cpus)) { +if (cpumask_intersects (tmp->new_cpus, subpartitions_cpus) || +cpumask_subset (top_cpuset.effective_cpus, tmp->new_cpus)) {spin_unlock_irq (&callback_lock); +if (cpumask_intersects (tmp->new_cpus, subpartitions_cpus)) +return PERR_NOCPUS ;return PERR_HKEEPING ; }
tools/testing/selftests/cgroup/test_cpuset_prs.sh +2 -0
@@ -298,6 +298,8 @@ TEST_MATRIX=( " C0-4:X2-4 C1-4:X2-4:P2 C2-4:X4:P1 \ . . . X1 . 0 A1:0-1|A2:2-4|A3:2-4 \ A1:P0|A2:P2|A3:P-1 2-4" + " CX1-3:P1 CX1-3 CX1-3 . . . P1 . 0 A1:1-3|A2:1-3|A3:1-3 \ + A1:P1|A2:P0|A3:P-1" # Remote partition offline tests " C0-3 C1-3 C2-3 . X2-3 X2-3 X2-3:P2:O2=0 . 0 A1:0-1|A2:1|A3:3 A1:P0|A3:P2 2-3"
Signature · 1 line
2.49.0
PATCH v3 2/3 cgroup/cpuset: prevent activating local partition over remote one
Commit 6da580ec656a ("cgroup/cpuset: Don't allow creation of local partition over a remote one") added a check in update_prstate() to reject creating a top-level local partition whose cs->exclusive_cpus intersects subpartitions_cpus.
However, if a top-level cgroup A1 is first set to "root" with all CPUs (CX0-7:P1, becoming "root invalid" because no CPUs would remain in top_cpuset), has a child A1/A2 enabled as a remote partition on a subset of those exclusive CPUs (CX2-3:P2), and A1 is then updated to a valid CPU subset (CX1-3), update_exclusive_cpumask() / update_cpumask() calls validate_partition() without going through update_prstate(). Because validate_partition() does not check trialcs->exclusive_cpus against subpartitions_cpus, A1 transitions from "root invalid" to a valid local partition over the existing remote partition A1/A2 and triggers:
WARNING: kernel/cgroup/cpuset.c:1943 at update_parent_effective_cpumask+0x189b/0x1fd0
Check trialcs->exclusive_cpus against subpartitions_cpus in validate_partition() when !is_partition_valid(cs), and add a regression test case to tools/testing/selftests/cgroup/test_cpuset_prs.sh.
Tested in QEMU on Linux 7.3.0-rc3 using tools/testing/selftests/cgroup/test_cpuset_prs.sh.
---
Changes in v3:
- Split the validate_partition() fix (Fixes: 6da580ec656a) and the cpus_excl_conflict() fix (Fixes: 2a3602030d80) into separate patches.
kernel/cgroup/cpuset.c | 4 ++++ tools/testing/selftests/cgroup/test_cpuset_prs.sh | 2 ++ 2 files changed, 6 insertions(+)
kernel/cgroup/cpuset.c +4 -0
@@ -2415,6 +2415,10 @@ static enum prs_errcode validate_partition(struct cpuset *cs, struct cpuset *triif (cpumask_empty (trialcs->effective_xcpus))return PERR_INVCPUS ; +if ((parent == &top_cpuset) && !is_partition_valid (cs) && +cpumask_intersects (trialcs->exclusive_cpus, subpartitions_cpus)) +return PERR_REMOTE ; +if (prstate_housekeeping_conflict (trialcs->partition_root_state, trialcs->effective_xcpus))return PERR_HKEEPING ;
tools/testing/selftests/cgroup/test_cpuset_prs.sh +2 -0
@@ -321,6 +321,8 @@ TEST_MATRIX=( " C1-3:P2 X4:P2 . . . X3:P2 . . 0 A1:1-2|XA1:1-3|A2:3:XA2:3 A1:P2|A2:P2 1-3" " C0-3:P2 . . C4-6 C0-4 . . . 0 A1:0-4|B1:5-6 A1:P2|B1:P0" " C0-3:P2 . . C4-6 C0-4:C0-3 . . . 0 A1:0-3|B1:4-6 A1:P2|B1:P0 0-3" + " CX0-7:P1 CX2-3:P2 . . CX1-3 . . . 0 A1:1|A2:2-3|XA2:2-3 \ + A1:P-1|A2:P2 2-3" # Local partition invalidation tests " C0-3:X1-3:P2 C1-3:X2-3:P2 C2-3:X3:P2 \
Signature · 1 line
2.49.0
Waiman Long
maintainerRe: [PATCH v3 2/3] cgroup/cpuset: prevent activating local partition over remote one
Quoted 42 lines · Hui Peng
Commit
6da580ec656a("cgroup/cpuset: Don't allow creation of local partition over a remote one") added a check inupdate_prstate()to reject creating a top-level local partition whosecs->exclusive_cpusintersectssubpartitions_cpus.However, if a top-level cgroup A1 is first set to "root" with all CPUs (CX0-7:P1, becoming "root invalid" because no CPUs would remain in
top_cpuset), has a child A1/A2 enabled as a remote partition on a subset of those exclusive CPUs (CX2-3:P2), and A1 is then updated to a valid CPU subset (CX1-3),update_exclusive_cpumask()/update_cpumask()callsvalidate_partition()without going throughupdate_prstate(). Becausevalidate_partition()does not checktrialcs->exclusive_cpusagainstsubpartitions_cpus, A1 transitions from "root invalid" to a valid local partition over the existing remote partition A1/A2 and triggers:WARNING: kernel/cgroup/cpuset.c:1943 at update_parent_effective_cpumask+0x189b/0x1fd0Check
trialcs->exclusive_cpusagainstsubpartitions_cpusinvalidate_partition()when !is_partition_valid(cs), and add a regression test case totools/testing/selftests/cgroup/test_cpuset_prs.sh.Tested in QEMU on Linux 7.3.0-rc3 using
tools/testing/selftests/cgroup/test_cpuset_prs.sh.Fixes 6da580ec656a ("cgroup/cpuset: Don't allow creation of local partition over a remote one")Assisted-by LLMSigned-off-by Hui Peng <[email protected]>---
Changes in v3:
- Split thevalidate_partition()fix (Fixes:6da580ec656a) and thecpus_excl_conflict()fix (Fixes:2a3602030d80) into separate patches.kernel/cgroup/cpuset.c | 4 ++++ tools/testing/selftests/cgroup/test_cpuset_prs.sh | 2 ++ 2 files changed, 6 insertions(+)kernel/cgroup/cpuset.c +0
-0
kernel/cgroup/cpuset.c static enum prs_errcode validate_partition(struct cpuset *cs, struct cpuset *tri · line 2421
Quoted 7 lines
kernel/cgroup/cpuset.c +4
-0@@ -2415,6 +2415,10 @@ static enum prs_errcode validate_partition(struct cpuset *cs, struct cpuset *triif (cpumask_empty (trialcs->effective_xcpus))return PERR_INVCPUS ; +if ((parent == &top_cpuset) && !is_partition_valid (cs) && +cpumask_intersects (trialcs->exclusive_cpus, subpartitions_cpus)) +return PERR_REMOTE ; +
This additional check looks good.
tools/testing/selftests/cgroup/test_cpuset_prs.sh TEST_MATRIX=( · line 325
Quoted 13 lines
kernel/cgroup/cpuset.c +0
-0if (prstate_housekeeping_conflict (trialcs->partition_root_state, trialcs->effective_xcpus))return PERR_HKEEPING ;tools/testing/selftests/cgroup/test_cpuset_prs.sh +2
-0@@ -321,6 +321,8 @@ TEST_MATRIX=( " C1-3:P2 X4:P2 . . . X3:P2 . . 0 A1:1-2|XA1:1-3|A2:3:XA2:3 A1:P2|A2:P2 1-3" " C0-3:P2 . . C4-6 C0-4 . . . 0 A1:0-4|B1:5-6 A1:P2|B1:P0" " C0-3:P2 . . C4-6 C0-4:C0-3 . . . 0 A1:0-3|B1:4-6 A1:P2|B1:P0 0-3" + " CX0-7:P1 CX2-3:P2 . . CX1-3 . . . 0 A1:1|A2:2-3|XA2:2-3 \ + A1:P-1|A2:P2 2-3"
This test will not work because you are assuming the test system only have 8 CPUs. That may not be the case and 2 local partitions may be created instead.
Cheers,
Longman
Quoted 3 lines
tools/testing/selftests/cgroup/test_cpuset_prs.sh +0
-0# Local partition invalidation tests " C0-3:X1-3:P2 C1-3:X2-3:P2 C2-3:X3:P2 \
PATCH v3 3/3 cgroup/cpuset: check sibling effective_xcpus in cpus_excl_conflict()
In reply to Hui PengIn cpus_excl_conflict(), when a valid local partition A1 uses implicit exclusive CPUs (cpuset.cpus set without cpuset.cpus.exclusive, so sibling->exclusive_cpus is empty while sibling->effective_xcpus is populated), a sibling cgroup B1 can still set cpuset.cpus.exclusive on the same CPUs because cpus_excl_conflict() only checks sibling->exclusive_cpus.
Fall back to sibling->effective_xcpus in cpus_excl_conflict() when sibling->exclusive_cpus is empty, and add a regression test case to tools/testing/selftests/cgroup/test_cpuset_prs.sh.
Tested in QEMU on Linux 7.3.0-rc3 using tools/testing/selftests/cgroup/test_cpuset_prs.sh.
---
Changes in v3:
- Split out from the validate_partition() fix into patch 3/3 and add the Fixes tag for commit 2a3602030d80, as pointed out by Guopeng Zhang.
kernel/cgroup/cpuset.c | 5 ++++- tools/testing/selftests/cgroup/test_cpuset_prs.sh | 1 + 2 files changed, 5 insertions(+), 1 deletion(-)
kernel/cgroup/cpuset.c +4 -1
@@ -713,7 +713,10 @@ static inline bool cpus_excl_conflict(struct cpuset *trial, struct cpuset *siblireturn true;/* Exclusive_cpus cannot intersect */ -return cpumask_intersects (trial->exclusive_cpus, sibling->exclusive_cpus); +return cpumask_intersects (trial->exclusive_cpus, +cpumask_empty (sibling->exclusive_cpus) + ? sibling->effective_xcpus + : sibling->exclusive_cpus); }static inline bool mems_excl_conflict (struct cpuset *cs1,struct cpuset *cs2)
tools/testing/selftests/cgroup/test_cpuset_prs.sh +1 -0
@@ -445,6 +445,7 @@ TEST_MATRIX=( # Changes to cpuset.cpus.exclusive that violate exclusivity rule is rejected " C0-3 . . C4-5 X0-3 . . X3-5 1 A1:0-3|B1:4-5" + " C0-3:P1 . . C4-5 . . . X3-5 1 A1:0-3|B1:4-5 A1:P1|B1:P0" # cpuset.cpus.exclusive cannot be set to a superset of sibling's cpuset.cpus " C0-3 . . C4-5 X3-5 . . . 1 A1:0-3|B1:4-5"
Signature · 1 line
2.49.0
Ridong Chen
reviewerRe: [PATCH v3 1/3] cgroup/cpuset: return PERR_NOCPUS in remote_partition_enable()
In reply to Hui PengQuoted 24 lines · Hui Peng
Commit
6da580ec656a("cgroup/cpuset: Don't allow creation of local partition over a remote one") updatedupdate_prstate()to prevent creating a top-level local partition whose exclusive CPUs conflict with an existing remote partition.However, if a top-level local partition A1 is created on CPUs 1-3, has a non-partition member child A1/A2 with cpuset.cpus.exclusive = 1-3, and a grandchild A1/A2/A3 is then enabled as a partition root on 1-3,
update_prstate()sees !is_partition_valid(parent) (since A1/A2 is a member) and callsremote_partition_enable(). Becauseremote_partition_enable()does not check whethertmp->new_cpusintersectssubpartitions_cpus, A1/A2/A3 is enabled as a remote partition on CPUs already owned by A1, triggering:WARNING: kernel/cgroup/cpuset.c:1594 at update_prstate+0xbef/0xd70Return
PERR_NOCPUSinremote_partition_enable()whentmp->new_cpusintersectssubpartitions_cpus, matching the error code used byremote_cpus_update()forsubpartitions_cpusconflicts, and add a corresponding regression test case totools/testing/selftests/cgroup/test_cpuset_prs.sh.Tested in QEMU on Linux 7.3.0-rc3 using
tools/testing/selftests/cgroup/test_cpuset_prs.sh.
Quoted 9 lines
Fixes 6da580ec656a ("cgroup/cpuset: Don't allow creation of local partition over a remote one")Suggested-by Guopeng Zhang <[email protected]>Assisted-by LLMSigned-off-by Hui Peng <[email protected]>---
Changes in v3:
- Split the series into 3 patches (one per logical change and Fixes: tag).
What is going on here?
Please do not reply with a new series. And I think the changes should be listed in the cover letter. Otherwise, it will get very confusing.
Quoted 11 lines
kernel/cgroup/cpuset.c | 4 +++- tools/testing/selftests/cgroup/test_cpuset_prs.sh | 2 ++ 2 files changed, 5 insertions(+), 1 deletion(-)kernel/cgroup/cpuset.c +0
-0@@ -1678,8 +1678,10 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,* Callers must have validated that effective_xcpus is not empty.
kernel/cgroup/cpuset.c static int remote_partition_enable(struct cpuset *cs, int new_prs, · line 1687
Quoted 10 lines
kernel/cgroup/cpuset.c +4
-1*/ spin_lock_irq (&callback_lock); -if (cpumask_subset (top_cpuset.effective_cpus, tmp->new_cpus)) { +if (cpumask_intersects (tmp->new_cpus, subpartitions_cpus) || +cpumask_subset (top_cpuset.effective_cpus, tmp->new_cpus)) {spin_unlock_irq (&callback_lock); +if (cpumask_intersects (tmp->new_cpus, subpartitions_cpus)) +return PERR_NOCPUS ;return PERR_HKEEPING ; }
I cannot apply this patch. It seems quite different from v2.
Quoted 13 lines
tools/testing/selftests/cgroup/test_cpuset_prs.sh +2
-0@@ -298,6 +298,8 @@ TEST_MATRIX=( " C0-4:X2-4 C1-4:X2-4:P2 C2-4:X4:P1 \ . . . X1 . 0 A1:0-1|A2:2-4|A3:2-4 \ A1:P0|A2:P2|A3:P-1 2-4" + " CX1-3:P1 CX1-3 CX1-3 . . . P1 . 0 A1:1-3|A2:1-3|A3:1-3 \ + A1:P1|A2:P0|A3:P-1" # Remote partition offline tests " C0-3 C1-3 C2-3 . X2-3 X2-3 X2-3:P2:O2=0 . 0 A1:0-1|A2:1|A3:3 A1:P0|A3:P2 2-3"
Signature · 2 lines
Best regards Ridong
Re: [PATCH v3 1/3] cgroup/cpuset: return PERR_NOCPUS in remote_partition_enable()
Quoted 8 lines · Ridong Chen
What is going on here?
Please do not reply with a new series. And I think the changes should be listed in the cover letter. Otherwise, it will get very confusing.
...
I cannot apply this patch. It seems quite different from v2.
Hi Ridong,
Sorry for the confusion and the noise. Two mistakes happened when I prepared v3 after splitting patch 2/2 per Guopeng's feedback:
1. I mistakenly passed --in-reply-to pointing to the v2 thread and omitted the 0/3 cover letter.
2. Patch 1/3 was accidentally overwritten with a broken diff instead of the remote_partition_enable() hunk from v2 1/2:
@@ -1591,10 +1591,11 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,
* above it or remote partition root underneath it is not allowed. */
compute_excpus(cs, tmp->new_cpus);
- WARN_ON_ONCE(cpumask_intersects(tmp->new_cpus, subpartitions_cpus));
if (!cpumask_intersects(tmp->new_cpus, cpu_active_mask) ||
cpumask_subset(top_cpuset.effective_cpus, tmp->new_cpus))
return PERR_INVCPUS;
+ if (cpumask_intersects(tmp->new_cpus, subpartitions_cpus))
+ return PERR_NOCPUS;
Would it be OK if I post a clean, standalone [PATCH v4 0/3..3/3] series (as a new thread with a 0/3 cover letter listing the changelog and with the v2 1/2 diff restored in patch 1/3)?
Best regards,
Hui
Waiman Long
maintainerRe: [PATCH v3 1/3] cgroup/cpuset: return PERR_NOCPUS in remote_partition_enable()
Quoted 30 lines · Hui Peng
Quoted 8 lines · Ridong Chen
What is going on here?
Please do not reply with a new series. And I think the changes should be listed in the cover letter. Otherwise, it will get very confusing.
...
I cannot apply this patch. It seems quite different from v2.
Hi Ridong,
Sorry for the confusion and the noise. Two mistakes happened when I prepared v3 after splitting patch 2/2 per Guopeng's feedback:
1. I mistakenly passed --in-reply-to pointing to the v2 thread and omitted the 0/3 cover letter.
2. Patch 1/3 was accidentally overwritten with a broken diff instead of theremote_partition_enable()hunk from v2 1/2:(quoted diff) +2
-1@@ -1591,10 +1591,11 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs, * above it or remote partition root underneath it is not allowed. */ compute_excpus(cs, tmp->new_cpus); - WARN_ON_ONCE(cpumask_intersects(tmp->new_cpus, subpartitions_cpus)); if (!cpumask_intersects(tmp->new_cpus, cpu_active_mask) || cpumask_subset(top_cpuset.effective_cpus, tmp->new_cpus)) return PERR_INVCPUS; + if (cpumask_intersects(tmp->new_cpus, subpartitions_cpus)) + return PERR_NOCPUS;
Quoted 3 lines
Would it be OK if I post a clean, standalone [PATCH v4 0/3..3/3] series (as a new thread with a 0/3 cover letter listing the changelog and with the v2 1/2 diff restored in patch 1/3)?
That diff looks more correct. Please send out a new standalone v4 patch series with your patches.
Cheers,
Longman
Quoted 3 lines
Best regards,
Hui
在 2026/9/20 06:17, Hui Peng 写道:
Hello Hui,
Quoted 29 lines
Fix two partition validation bugs in
kernel/cgroup/cpuset.c:1. In
remote_partition_enable(), check whether the requested effective xcpus intersectparent->subpartitions_cpusso a remote partition cannot claim CPUs already delegated to a local child partition.
2. Invalidate_partition(), verify that enabling a local partition does not overlap CPUs already allocated to an active remote partition.Fixes aa7d3a56a20f ("cpuset: fix warning when disabling remote partition")Assisted-by LLMSigned-off-by Hui Peng <[email protected]>---
kernel/cgroup/cpuset.c +4
-1@@ -713,7 +713,10 @@ static inline bool cpus_excl_conflict(struct cpuset *trial, struct cpuset *siblireturn true;/* Exclusive_cpus cannot intersect */ -return cpumask_intersects (trial->exclusive_cpus, sibling->exclusive_cpus); +return cpumask_intersects (trial->exclusive_cpus, +cpumask_empty (sibling->exclusive_cpus) + ? sibling->effective_xcpus + : sibling->exclusive_cpus); }static inline bool mems_excl_conflict (struct cpuset *cs1,struct cpuset *cs2)
kernel/cgroup/cpuset.c static int remote_partition_enable(struct cpuset *cs, int new_prs, · line 1600
Quoted 8 lines
kernel/cgroup/cpuset.c +1
-1@@ -1591,8 +1594,8 @@ static int remote_partition_enable(struct cpuset *cs, int new_prs,* above it or remote partition root underneath it is not allowed. */ compute_excpus (cs, tmp->new_cpus); -WARN_ON_ONCE (cpumask_intersects (tmp->new_cpus, subpartitions_cpus));if (!cpumask_intersects (tmp->new_cpus, cpu_active_mask) || +cpumask_intersects (tmp->new_cpus, subpartitions_cpus) ||cpumask_subset (top_cpuset.effective_cpus, tmp->new_cpus))return PERR_INVCPUS ;
I have been going through this part of the code repeatedly recently, and I would like to share one thought.
Should this return PERR_NOCPUS instead?
+ if (cpumask_intersects(tmp->new_cpus, subpartitions_cpus))
+ return PERR_NOCPUS;
In the first reproducer you described in your reply to Ridong:
Minimized reproducer (Scenario 1):
#!/bin/sh
mkdir -p /tmp/cg1
mount -t cgroup2 none /tmp/cg1
echo "+cpuset" > /tmp/cg1/cgroup.subtree_control
mkdir /tmp/cg1/A
echo 1 > /tmp/cg1/A/cpuset.cpus
echo 1 > /tmp/cg1/A/cpuset.cpus.exclusive echo root > /tmp/cg1/A/cpuset.cpus.partition echo "+cpuset" > /tmp/cg1/A/cgroup.subtree_control
mkdir /tmp/cg1/A/B
echo 1 > /tmp/cg1/A/B/cpuset.cpus
echo 1 > /tmp/cg1/A/B/cpuset.cpus.exclusive echo "+cpuset" > /tmp/cg1/A/B/cgroup.subtree_control
mkdir /tmp/cg1/A/B/D
echo 1 > /tmp/cg1/A/B/D/cpuset.cpus
echo 1 > /tmp/cg1/A/B/D/cpuset.cpus.exclusive
echo root > /tmp/cg1/A/B/D/cpuset.cpus.partition
CPU 1 is active, and the requested CPU mask itself is valid. The failure happens because CPU 1 has already been allocated to partition A, so the top cpuset can no longer distribute it to the remote partition D.
remote_cpus_update() uses PERR_NOCPUS for the same kind of subpartitions_cpus conflict:
else if (cpumask_intersects(tmp->addmask, subpartitions_cpus) ||
cpumask_subset(top_cpuset.effective_cpus,
tmp->addmask))
WRITE_ONCE(cs->prs_err, PERR_NOCPUS);Would using PERR_NOCPUS here be more appropriate? This would keep the initial remote-partition enable path consistent with the CPU update path for an existing remote partition, and it also seems to describe the failure more accurately: the CPU itself is not invalid, but has already been allocated to another partition, so the parent can no longer distribute it downstream.
Thanks,
Guopeng
Quoted 21 lines
kernel/cgroup/cpuset.c +5
-1if (((new_prs == PRS_ISOLATED) &&@@ -2411,6 +2414,10 @@ static enum prs_errcode validate_partition(struct cpuset *cs, struct cpuset *triif (cpumask_empty (trialcs->effective_xcpus))return PERR_INVCPUS ; +if ((parent == &top_cpuset) && +cpumask_intersects (trialcs->effective_xcpus, subpartitions_cpus)) +return PERR_REMOTE ; +if (prstate_housekeeping_conflict (trialcs->partition_root_state, trialcs->effective_xcpus))return PERR_HKEEPING ;@@ -2970,7 +2977,7 @@ static int update_prstate(struct cpuset *cs, int new_prs)* local or remote partition. */ if ((parent == &top_cpuset) && -cpumask_intersects (cs->exclusive_cpus, subpartitions_cpus)) { +cpumask_intersects (user_xcpus (cs), subpartitions_cpus)) { err =PERR_REMOTE ; goto out; }
Hi Guopeng,
Quoted 5 lines · Guopeng Zhang
Should this return
PERR_NOCPUSinstead?(quoted diff) +2
-0+ if (cpumask_intersects(tmp->new_cpus, subpartitions_cpus)) + return PERR_NOCPUS;
...
Quoted 6 lines
Would using
PERR_NOCPUShere be more appropriate? This would keep the initial remote-partition enable path consistent with the CPU update path for an existing remote partition, and it also seems to describe the failure more accurately: the CPU itself is not invalid, but has already been allocated to another partition, so the parent can no longer distribute it downstream.
Agreed, returning PERR_NOCPUS when tmp->new_cpus intersects subpartitions_cpus is much more accurate and keeps remote_partition_enable() consistent with remote_cpus_update(). I have updated this in v2.
Thanks,
Hui