Skip to contentSkip to conversation

docs: cgroup-v2: Correct cpuset CPU descriptions

PATCH Superseded Guopeng Zhang 3 participants 9 messages Sep 18 – Sep 21 1 message not in the archive

Subsystems (2)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover docs: cgroup-v2: Correct cpuset CPU descriptions

From: Guopeng Zhang <[email protected]>

The cpuset interface documentation says cpuset.cpus.isolated is empty unless an isolated partition exists, omitting CPUs isolated at boot with isolcpus=domain. Its cpuset.cpus.effective description also assumes CPUs come from the immediate parent and are normally a subset of cpuset.cpus, which is not always true for valid non-root partition roots.

Update both descriptions to match the existing behavior.

Guopeng Zhang (2):
docs: cgroup-v2: Describe boot-isolated CPUs in cpuset.cpus.isolated
docs: cgroup-v2: Describe effective CPUs of partition roots

 Documentation/admin-guide/cgroup-v2.rst | 37 ++++++++++++++++---------
 1 file changed, 24 insertions(+), 13 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH 1/2 docs: cgroup-v2: Describe boot-isolated CPUs in cpuset.cpus.isolated

From: Guopeng Zhang <[email protected]>

cpuset.cpus.isolated is documented as empty when no isolated partition exists. This is not true when CPUs are isolated at boot with isolcpus=domain. cpuset_init() adds those CPUs to the isolated mask, and isolcpus=<cpu-list> uses domain isolation by default. They remain listed after a partition using them is released.

Document the boot-isolated CPUs as well.

---
 Documentation/admin-guide/cgroup-v2.rst | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
Documentation/admin-guide/cgroup-v2.rst +5 -3
index 2ec582985b5b..272e9f2432bd 100644
@@ -2625,9 +2625,11 @@ Cpuset Interface Files
   cpuset.cpus.isolated
 	A read-only and root cgroup only multiple values file.
 
-	This file shows the set of all isolated CPUs used in existing
-	isolated partitions. It will be empty if no isolated partition
-	is created.
+	This file shows CPUs isolated from scheduler domains by
+	isolated cpuset partitions or at boot with "isolcpus=domain"
+	(including the default "isolcpus=<cpu-list>" form).  CPUs
+	isolated at boot remain listed after any partition using them
+	is released.
 
   cpuset.cpus.partition
 	A read-write single value file which exists on non-root
Signature · 1 line
2.43.0

A message that is not in the archive (sent off-list or not yet received): [email protected]

#3 lore raw

Re: [PATCH 1/2] docs: cgroup-v2: Describe boot-isolated CPUs in cpuset.cpus.isolated

Quoted 56 lines · 在 2026/9/18 18:50, [email protected] 写道:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] cgroup/cpuset: boot-isolated CPUs lost from cpuset.cpus.isolated on partition release --

commit e1e9b698bdf04aebf82c92a40588b3e308527a2f
Author: Guopeng Zhang <[email protected]>

docs: cgroup-v2: Describe boot-isolated CPUs in cpuset.cpus.isolated

cpuset.cpus.isolated is documented as empty when no isolated partition exists. This is not true when CPUs are isolated at boot with isolcpus=domain. cpuset_init() adds those CPUs to the isolated mask, and isolcpus=<cpu-list> uses domain isolation by default. They remain listed after a partition using them is released.

Document the boot-isolated CPUs as well.

Quoted 6 lines
Documentation/admin-guide/cgroup-v2.rst +0 -0
index ebc683070521c..e666b9e15e1c1 100644
@@ -2650,9 +2650,11 @@ Cpuset Interface Files
   cpuset.cpus.isolated

Documentation/admin-guide/cgroup-v2.rst Cpuset Interface Files · line 2657

Quoted 10 lines
Documentation/admin-guide/cgroup-v2.rst +5 -3
 	A read-only and root cgroup only multiple values file.
 
-	This file shows the set of all isolated CPUs used in existing
-	isolated partitions. It will be empty if no isolated partition
-	is created.
+	This file shows CPUs isolated from scheduler domains by
+	isolated cpuset partitions or at boot with "isolcpus=domain"
+	(including the default "isolcpus=<cpu-list>" form).  CPUs
+	isolated at boot remain listed after any partition using them
+	is released.

[Severity: Medium]
Does the code actually behave as described here?

Looking at isolated_cpus_update() in kernel/cgroup/cpuset.c, when a partition is deactivated, it appears to unconditionally remove the released CPUs from the isolated_cpus mask:

kernel/cgroup/cpuset.c:isolated_cpus_update() {
    ...
	} else {
		if (!cpumask_intersects(xcpus, isolated_cpus))
			return;
		cpumask_andnot(isolated_cpus, isolated_cpus, xcpus);
	}
    ...
}

Because there is no logic to check if those CPUs were originally part of the boot-isolated set (i.e., outside HK_TYPE_DOMAIN_BOOT), they seem to be permanently purged from the isolated_cpus mask and thus disappear from cpuset.cpus.isolated.

Quoted 5 lines

If a user boots with isolcpus=domain for CPU 2, creates an isolated cpuset partition containing CPU 2, and then deactivates the partition, wouldn't CPU 2 be lost from cpuset.cpus.isolated? This appears to contradict the behavior explicitly promised by this documentation update.

The code quoted here is the old implementation. This was fixed by commit 6c37d7e074a4 ("cgroup/cpuset: Preserve boot-isolated CPUs on partition release") [1].

The current code checks HK_TYPE_DOMAIN_BOOT before clearing a CPU from isolated_cpus, so boot-isolated CPUs remain in cpuset.cpus.isolated after the partition is released.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=6c37d7e074a4be1ba8da59f4ed5df8977b3daa43

Thanks,
Guopeng

Reviewed-by on v1 1/2
#4 lore raw

Re: [PATCH 1/2] docs: cgroup-v2: Describe boot-isolated CPUs in cpuset.cpus.isolated

In reply to Guopeng Zhang
Quoted 24 lines · Guopeng Zhang

From: Guopeng Zhang <[email protected]>

cpuset.cpus.isolated is documented as empty when no isolated partition exists. This is not true when CPUs are isolated at boot with isolcpus=domain. cpuset_init() adds those CPUs to the isolated mask, and isolcpus=<cpu-list> uses domain isolation by default. They remain listed after a partition using them is released.

Document the boot-isolated CPUs as well.

---
  Documentation/admin-guide/cgroup-v2.rst | 8 +++++---
  1 file changed, 5 insertions(+), 3 deletions(-)
Documentation/admin-guide/cgroup-v2.rst +0 -1
index 2ec582985b5b..272e9f2432bd 100644
@@ -2625,9 +2625,11 @@ Cpuset Interface Files
    cpuset.cpus.isolated
  	A read-only and root cgroup only multiple values file.
 
-	This file shows the set of all isolated CPUs used in existing

Documentation/admin-guide/cgroup-v2.rst Cpuset Interface Files · line 2635

Quoted 10 lines
Documentation/admin-guide/cgroup-v2.rst +5 -2
-	isolated partitions. It will be empty if no isolated partition
-	is created.
+	This file shows CPUs isolated from scheduler domains by
+	isolated cpuset partitions or at boot with "isolcpus=domain"
+	(including the default "isolcpus=<cpu-list>" form).  CPUs
+	isolated at boot remain listed after any partition using them
+	is released.
 
    cpuset.cpus.partition
  	A read-write single value file which exists on non-root

Could we keep that terminology and avoid repeating "isolated"? Something like:

This file shows the set of all isolated CPUs used in existing isolated partitions or isolated at boot time via "isolcpus". The latter remain listed even after any partition using them is released.

With that:

Signature · 2 lines
Best regards
Ridong
#5 lore raw

Re: [PATCH 1/2] docs: cgroup-v2: Describe boot-isolated CPUs in cpuset.cpus.isolated

Quoted 38 lines · Ridong Chen
Quoted 24 lines · Guopeng Zhang

From: Guopeng Zhang <[email protected]>

cpuset.cpus.isolated is documented as empty when no isolated partition exists. This is not true when CPUs are isolated at boot with isolcpus=domain. cpuset_init() adds those CPUs to the isolated mask, and isolcpus=<cpu-list> uses domain isolation by default. They remain listed after a partition using them is released.

Document the boot-isolated CPUs as well.

---
  Documentation/admin-guide/cgroup-v2.rst | 8 +++++---

  1 file changed, 5 insertions(+), 3 deletions(-)
Documentation/admin-guide/cgroup-v2.rst +0 -0
index 2ec582985b5b..272e9f2432bd 100644
@@ -2625,9 +2625,11 @@ Cpuset Interface Files

    cpuset.cpus.isolated
      A read-only and root cgroup only multiple values file. -    This file shows the set of all isolated CPUs used in existing -    isolated partitions. It will be empty if no isolated partition

Quoted 8 lines

-    is created.
+    This file shows CPUs isolated from scheduler domains by +    isolated cpuset partitions or at boot with "isolcpus=domain" +    (including the default "isolcpus=<cpu-list>" form).  CPUs +    isolated at boot remain listed after any partition using them +    is released.
      cpuset.cpus.partition
      A read-write single value file which exists on non-root

Could we keep that terminology and avoid repeating "isolated"? Something like:

Quoted 9 lines

      This file shows the set of all isolated CPUs used in existing isolated partitions or isolated at boot time via "isolcpus". The latter remain listed even after any partition using them is released.

With that:

Thanks for the suggestion. That reads better.

I'll make one small change and use "the domain flag of isolcpus" here, since isolcpus also supports the nohz and managed_irq flags. I'll update the wording in v2 and add your Reviewed-by.

Thanks,
Guopeng

#6 lore raw

PATCH 2/2 docs: cgroup-v2: Describe effective CPUs of partition roots

In reply to Guopeng Zhang

From: Guopeng Zhang <[email protected]>

The cpuset.cpus.effective description says that the effective CPUs are granted by the parent and are normally a subset of cpuset.cpus. This does not always hold for a valid non-root partition root. Its effective CPUs are the active CPUs in cpuset.cpus.exclusive.effective after CPUs assigned to valid child partition roots are removed.

If cpuset.cpus.exclusive differs from cpuset.cpus, the effective CPUs may therefore be outside cpuset.cpus. Remote partitions also do not necessarily get their CPUs from their immediate parent.

---
 Documentation/admin-guide/cgroup-v2.rst | 29 ++++++++++++++++---------
 1 file changed, 19 insertions(+), 10 deletions(-)
Documentation/admin-guide/cgroup-v2.rst +19 -10
index 272e9f2432bd..49ce8deb4115 100644
@@ -2505,16 +2505,25 @@ Cpuset Interface Files
 	A read-only multiple values file which exists on all
 	cpuset-enabled cgroups.
 
-	It lists the onlined CPUs that are actually granted to this
-	cgroup by its parent.  These CPUs are allowed to be used by
-	tasks within the current cgroup.
-
-	If "cpuset.cpus" is empty, the "cpuset.cpus.effective" file shows
-	all the CPUs from the parent cgroup that can be available to
-	be used by this cgroup.  Otherwise, it should be a subset of
-	"cpuset.cpus" unless none of the CPUs listed in "cpuset.cpus"
-	can be granted.  In this case, it will be treated just like an
-	empty "cpuset.cpus".
+	It lists the active CPUs available to tasks in this cgroup.
+
+	For a cgroup that is not a valid partition root, an empty
+	"cpuset.cpus" makes "cpuset.cpus.effective" show all CPUs
+	available from the parent cgroup.  Otherwise, it is a subset
+	of "cpuset.cpus" unless none of the requested CPUs can be
+	granted.  In that case, it is treated like an empty
+	"cpuset.cpus".
+
+	For a valid non-root partition root, "cpuset.cpus.effective" contains
+	the active CPUs from "cpuset.cpus.exclusive.effective" except
+	those granted to valid child partition roots.  If
+	"cpuset.cpus.exclusive" is set to a value different from
+	"cpuset.cpus", the effective CPUs need not be a subset of
+	"cpuset.cpus".  For example, if CPUs 2-5 are active,
+	available, and not assigned to a valid child partition root,
+	a partition root with "cpuset.cpus" set to "0-3" and
+	"cpuset.cpus.exclusive" set to "2-5" has "2-5" in
+	"cpuset.cpus.effective".
 
 	Its value will be affected by CPU hotplug events.
 
Signature · 1 line
2.43.0
#7 lore raw

Re: [PATCH 2/2] docs: cgroup-v2: Describe effective CPUs of partition roots

Quoted 47 lines · Guopeng Zhang

From: Guopeng Zhang <[email protected]>

The cpuset.cpus.effective description says that the effective CPUs are granted by the parent and are normally a subset of cpuset.cpus. This does not always hold for a valid non-root partition root. Its effective CPUs are the active CPUs in cpuset.cpus.exclusive.effective after CPUs assigned to valid child partition roots are removed.

If cpuset.cpus.exclusive differs from cpuset.cpus, the effective CPUs may therefore be outside cpuset.cpus. Remote partitions also do not necessarily get their CPUs from their immediate parent.

---
  Documentation/admin-guide/cgroup-v2.rst | 29 ++++++++++++++++---------
  1 file changed, 19 insertions(+), 10 deletions(-)
Documentation/admin-guide/cgroup-v2.rst +12 -10
index 272e9f2432bd..49ce8deb4115 100644
@@ -2505,16 +2505,25 @@ Cpuset Interface Files
  	A read-only multiple values file which exists on all
  	cpuset-enabled cgroups.
 
-	It lists the onlined CPUs that are actually granted to this
-	cgroup by its parent.  These CPUs are allowed to be used by
-	tasks within the current cgroup.
-
-	If "cpuset.cpus" is empty, the "cpuset.cpus.effective" file shows
-	all the CPUs from the parent cgroup that can be available to
-	be used by this cgroup.  Otherwise, it should be a subset of
-	"cpuset.cpus" unless none of the CPUs listed in "cpuset.cpus"
-	can be granted.  In this case, it will be treated just like an
-	empty "cpuset.cpus".
+	It lists the active CPUs available to tasks in this cgroup.
+
+	For a cgroup that is not a valid partition root, an empty
+	"cpuset.cpus" makes "cpuset.cpus.effective" show all CPUs
+	available from the parent cgroup.  Otherwise, it is a subset
+	of "cpuset.cpus" unless none of the requested CPUs can be
+	granted.  In that case, it is treated like an empty
+	"cpuset.cpus".
+
+	For a valid non-root partition root, "cpuset.cpus.effective" contains
+	the active CPUs from "cpuset.cpus.exclusive.effective" except
+	those granted to valid child partition roots.  If

Documentation/admin-guide/cgroup-v2.rst Cpuset Interface Files · line 2529

Quoted 10 lines
Documentation/admin-guide/cgroup-v2.rst +7 -0
+	"cpuset.cpus.exclusive" is set to a value different from
+	"cpuset.cpus", the effective CPUs need not be a subset of
+	"cpuset.cpus".  For example, if CPUs 2-5 are active,
+	available, and not assigned to a valid child partition root,
+	a partition root with "cpuset.cpus" set to "0-3" and
+	"cpuset.cpus.exclusive" set to "2-5" has "2-5" in
+	"cpuset.cpus.effective".
 
  	Its value will be affected by CPU hotplug events.
 

Hi Guopeng,

Thank you for your effort.

May I suggest adding a simple diagram to help clarify the relationship? Similarly, could you add a simple diagram to illustrate the relationship? Something like:

```
  A(cpu,memory) - B(memory) - C()
                            \ D()
```

I mean adding one diagram for the valid root case and another for the others (member / invalid root), so it becomes easier to see how they differ. What do you think?

Signature · 2 lines
Best regards
Ridong
#8 lore raw

Re: [PATCH 2/2] docs: cgroup-v2: Describe effective CPUs of partition roots

Quoted 63 lines · Ridong Chen
Quoted 53 lines · Guopeng Zhang

From: Guopeng Zhang <[email protected]>

The cpuset.cpus.effective description says that the effective CPUs are granted by the parent and are normally a subset of cpuset.cpus. This does not always hold for a valid non-root partition root. Its effective CPUs are the active CPUs in cpuset.cpus.exclusive.effective after CPUs assigned to valid child partition roots are removed.

If cpuset.cpus.exclusive differs from cpuset.cpus, the effective CPUs may therefore be outside cpuset.cpus. Remote partitions also do not necessarily get their CPUs from their immediate parent.

---
  Documentation/admin-guide/cgroup-v2.rst | 29 ++++++++++++++++---------

  1 file changed, 19 insertions(+), 10 deletions(-)
Documentation/admin-guide/cgroup-v2.rst +0 -0
index 272e9f2432bd..49ce8deb4115 100644
@@ -2505,16 +2505,25 @@ Cpuset Interface Files

      A read-only multiple values file which exists on all cpuset-enabled cgroups.
  -    It lists the onlined CPUs that are actually granted to this -    cgroup by its parent.  These CPUs are allowed to be used by -    tasks within the current cgroup.

Documentation/admin-guide/cgroup-v2.rst +19 -7
-
-    If "cpuset.cpus" is empty, the "cpuset.cpus.effective" file shows
-    all the CPUs from the parent cgroup that can be available to
-    be used by this cgroup.  Otherwise, it should be a subset of
-    "cpuset.cpus" unless none of the CPUs listed in "cpuset.cpus"
-    can be granted.  In this case, it will be treated just like an
-    empty "cpuset.cpus".
+    It lists the active CPUs available to tasks in this cgroup.
+
+    For a cgroup that is not a valid partition root, an empty
+    "cpuset.cpus" makes "cpuset.cpus.effective" show all CPUs
+    available from the parent cgroup.  Otherwise, it is a subset
+    of "cpuset.cpus" unless none of the requested CPUs can be
+    granted.  In that case, it is treated like an empty
+    "cpuset.cpus".
+
+    For a valid non-root partition root, "cpuset.cpus.effective" contains
+    the active CPUs from "cpuset.cpus.exclusive.effective" except
+    those granted to valid child partition roots.  If
+    "cpuset.cpus.exclusive" is set to a value different from
+    "cpuset.cpus", the effective CPUs need not be a subset of
+    "cpuset.cpus".  For example, if CPUs 2-5 are active,
+    available, and not assigned to a valid child partition root,
+    a partition root with "cpuset.cpus" set to "0-3" and
+    "cpuset.cpus.exclusive" set to "2-5" has "2-5" in
+    "cpuset.cpus.effective".
Quoted 2 lines

        Its value will be affected by CPU hotplug events.

Hi Guopeng,

Thank you for your effort.

Quoted 9 lines

May I suggest adding a simple diagram to help clarify the relationship? Similarly, could you add a simple diagram to illustrate the relationship? Something like:

```
  A(cpu,memory) - B(memory) - C()
                            \ D()
```

I mean adding one diagram for the valid root case and another for the others (member / invalid root), so it becomes easier to see how they differ. What do you think?

Thanks for the suggestion. I agree that a simple diagram would make this easier to understand.

I'll add the diagrams in v2.

Thanks,
Guopeng

#9 lore raw
In reply to Guopeng Zhang
Quoted 9 lines · Guopeng Zhang

From: Guopeng Zhang <[email protected]>

The cpuset interface documentation says cpuset.cpus.isolated is empty unless an isolated partition exists, omitting CPUs isolated at boot with isolcpus=domain. Its cpuset.cpus.effective description also assumes CPUs come from the immediate parent and are normally a subset of cpuset.cpus, which is not always true for valid non-root partition roots.

Update both descriptions to match the existing behavior.

The behavior of cpuset partitions have evolved over time and the documentation doesn't fully reflect the current behavior. Your 2 docs patch look good so far and I am looking forward to your v2 patches.

Cheers,
Longman

Quoted 7 lines

Guopeng Zhang (2):
docs: cgroup-v2: Describe boot-isolated CPUs in cpuset.cpus.isolated
docs: cgroup-v2: Describe effective CPUs of partition roots

  Documentation/admin-guide/cgroup-v2.rst | 37 ++++++++++++++++---------
  1 file changed, 24 insertions(+), 13 deletions(-)