Skip to contentSkip to conversation

doc: Add CPU Isolation documentation

PATCH Superseded Frederic Weisbecker 6 participants 14 messages Aug 9, 2025 – Mar 25 Documentation

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 doc: Add CPU Isolation documentation

nohz_full was introduced in v3.10 in 2013, which means this documentation is overdue for 12 years.

The shoemaker's children always go barefoot. And working on timers hasn't made me arriving on time either.

Fortunately Paul wrote a part of the needed documentation a while ago, especially concerning nohz_full in Documentation/timers/no_hz.rst and also about per-CPU kthreads in Documentation/admin-guide/kernel-per-CPU-kthreads.rst

Introduce a new page that gives an overview of CPU isolation in general.

---
 Documentation/admin-guide/cpu-isolation.rst | 338 ++++++++++++++++++++
 Documentation/admin-guide/index.rst         |   1 +
 2 files changed, 339 insertions(+)
 create mode 100644 Documentation/admin-guide/cpu-isolation.rst
Documentation/admin-guide/cpu-isolation.rst new file +338 -0
new file mode 100644
index 000000000000..250027acf7b2
@@ -0,0 +1,338 @@
+=============
+CPU Isolation
+=============
+
+Introduction
+============
+
+"CPU Isolation" means leaving a CPU exclusive to a given userspace
+workload without any undesired code interference from the kernel.
+
+Those interferences, commonly pointed out as "noise", can be triggered
+by asynchronous events (interrupts, timers, scheduler preemption by
+workqueues and kthreads, ...) or synchronous events (syscalls and page
+faults).
+
+Such noise usually goes unnoticed. After all synchronous events are a
+component of the requested kernel service. And asynchronous events are
+either sufficiently well distributed by the scheduler when executed
+as tasks or reasonably fast when executed as interrupt. The timer
+interrupt can even execute 1024 times per seconds without
+significantly measurable impact most of the time.
+
+However some rare and extreme workloads can be quite sensitive to
+those kinds of noise. This is the case, for example, with high
+bandwidth network processing that can't afford losing a single packet
+or very low latency network processing. Typically those usecases
+involve DPDK, bypassing the kernel networking stack and performing
+direct access to the networking device from userscace.
+
+In order to run a CPU without or with limited kernel noise, the
+related housekeeping work needs to be either shutdown, migrated or
+offloaded.
+
+Housekeeping
+============
+
+In the CPU isolation terminology, housekeeping is the work, often
+asynchronous, that the kernel needs to process in order to maintain
+all its services. It matches the noises and disturbances enumerated
+above except when at least one CPU is isolated. Then housekeeping may
+make use of further coping mechanisms if CPU-tied work must be
+offloaded.
+
+Housekeeping CPUs are the non-isolated CPUs where the kernel noise
+is moved away from isolated CPUs.
+
+The isolation can be implemented in several ways depending on the
+nature of the noise:
+
+- Unbound work, where "unbound" means not tied to any CPU, can be
+  simply migrated away from isolated CPUs to housekeeping CPUs.
+  This is the case of unbound workqueues, kthreads and timers.
+
+- Bound work, where "bound" means tied to a specific CPU, usually
+  can't be moved away as-is by nature. Either:
+
+	- The work must switch to a locked implementation. Eg: This is
+	  the case of RCU with CONFIG_RCU_NOCB_CPU.
+
+	- The related feature must be shutdown and considered
+	  incompatible with isolated CPUs. Eg: Lockup watchdog,
+	  unreliable clocksources, etc...
+
+	- An elaborated and heavyweight coping mechanism stands as a
+	  replacement. Eg: the timer tick is shutdown on nohz_full but
+	  with the constraint of running a single task on the CPU. A
+	  significant cost penalty is added on kernel entry/exit and
+	  a residual 1Hz scheduler tick is offloaded to housekeeping
+	  CPUs.
+
+In any case, housekeeping work has to be handled, which is why there
+must be at least one housekeeping CPU in the system, preferrably more
+if the machine runs a lot of CPUs. For example one per node on NUMA
+systems.
+
+Also CPU isolation often means a tradeoff between noise-free isolated
+CPUs and added overhead on housekeeping CPUs, sometimes even on
+isolated CPUs entering the kernel.
+
+Isolation features
+==================
+
+Different levels of isolation can be configured in the kernel, each of
+which having their own drawbacks and tradeoffs.
+
+Scheduler domain isolation
+--------------------------
+
+This feature isolates a CPU from the scheduler topology. As a result,
+the target isn't part of the load balancing. Tasks won't migrate
+neither from nor to it unless affine explicitly.
+
+As a side effect the CPU is also isolated from unbound workqueues and
+unbound kthreads.
+
+Requirements
+~~~~~~~~~~~~
+
+- CONFIG_CPUSETS=y for the cpusets based interface
+
+Tradeoffs
+~~~~~~~~~
+
+By nature, the system load is overall less distributed since some CPUs
+are extracted from the global load balancing.
+
+Interface
+~~~~~~~~~
+
+- :ref:`Documentation/admin-guide/cgroup-v2.rst <Cpuset v2 "isolated"
+  partitions>`
+  are recommended because they are tunable at runtime.
+
+- The 'isolcpus=' kernel boot parameter with the 'domain' flag is a
+  less flexible alternative that doesn't allow for runtime
+  reconfiguration.
+
+IRQs isolation
+--------------
+
+Isolate the IRQs whenever possible, so that they don't fire on the
+target CPUs.
+
+Interface
+~~~~~~~~~
+
+- /proc/irq/*/smp_affinity as explained :ref:`Documentation/core-api/irq/irq-affinity.rst <here>` in detail.
+
+- The "irqaffinity=" kernel boot parameter for a default setting.
+
+- The "managed_irq" flag in the "isolcpus=" kernel boot parameter
+  tries a best effort affinity override for managed IRQs.
+
+Full Dynticks (aka nohz_full)
+-----------------------------
+
+Full dynticks extends the dynticks idle mode, which stop the tick when
+the CPU is idle, to CPUs running a single task in userspace. That is,
+the timer tick is stopped if the environment allows it.
+
+Global timer callbacks are also isolated from the nohz_full CPUs.
+
+Requirements
+~~~~~~~~~~~~
+
+- CONFIG_NO_HZ_FULL=y
+
+Constraints
+~~~~~~~~~~~
+
+- The isolated CPUs must run a single task only. Multitask requires
+  the tick to maintain preemption. This is usually fine since the
+  workload usually can't stand the latency of random context switches.
+
+- No call to the kernel from isolated CPUs, at the risk of triggering
+  random noise.
+
+- No use of posix CPU timers on isolated CPUs.
+
+- Architecture must have a stable and reliable clocksource (no
+  unreliable TSC that requires the watchdog).
+
+
+Tradeoffs
+~~~~~~~~~
+
+In terms of cost, this is the most invasive isolation feature. It is
+assumed to be used when the workload spends most of its time in
+userspace and doesn't rely on the kernel except for preparatory
+work because:
+
+- RCU is slower due to the locked, offloaded and threaded callbacks
+  processing (the same that would be obtained with "rcu_nocb=" boot
+  parameter).
+
+- Kernel entry/exit through syscalls, exceptions and IRQs are more
+  costly due to fully ordered RmW operations that maintain userspace
+  as RCU extended quiescent state. Also the CPU time is accounted on
+  kernel boundaries instead of periodically from the tick.
+
+- Housekeeping CPUs must run a 1Hz residual remote scheduler tick
+  on behalf of the isolated CPUs.
+
+Checklist
+=========
+
+You have set up each of the above isolation features but you still
+observe jitters that trash your workload? Make sure to check a few
+elements before proceeding.
+
+Some of these checklist items are similar to those of real time
+workloads:
+
+- Use mlock() to prevent your pages from being swapped away. Page
+  faults are usually not compatible with jitter sensitive workloads.
+
+- Avoid SMT to prevent your hardware thread from being "preempted"
+  by another one.
+
+- CPU frequency changes may induce subtle sorts of jitter in a
+  workload. Cpufreq should be used and tuned with caution.
+
+- Deep C-states may result in latency issues upon wake-up. If this
+  happens to be a problem, C-states can be limited via kernel boot
+  parameters such as processor.max_cstate or intel_idle.max_cstate.
+
+
+Full isolation example
+======================
+
+In this example, the system has 8 CPUs and the 8th is to be fully
+isolated. Since CPUs start from 0, the 8th CPU is CPU 7.
+
+Kernel parameters
+-----------------
+
+Set the following kernel boot parameters to disable SMT and setup tick
+and IRQ isolation:
+
+- Full dynticks: nohz_full=7
+
+- IRQs isolation: irqaffinity=0-6
+
+- Managed IRQs isolation: isolcpus=managed_irq,7
+
+- Prevent from SMT: nosmt
+
+The full command line is then:
+
+  nohz_full=7 irqaffinity=0-6 isolcpus=managed_irq,7 nosmt
+
+CPUSET configuration (cgroup v2)
+--------------------------------
+
+Assuming cgroup v2 is mounted to /sys/fs/cgroup, the following script
+isolates CPU 7 from scheduler domains.
+
+  cd /sys/fs/cgroup
+  # Activate the cpuset subsystem
+  echo +cpuset > cgroup.subtree_control
+  # Create partition to be isolated
+  mkdir test
+  cd test
+  echo +cpuset > cgroup.subtree_control
+  # Isolate CPU 7
+  echo 7 > cpuset.cpus
+  echo "isolated" > cpuset.cpus.partition
+
+The userspace workload
+----------------------
+
+Fake a pure userspace workload, the below program runs a dummy
+userspace loop on the isolated CPU 7.
+
+  #include <stdio.h>
+  #include <fcntl.h>
+  #include <unistd.h>
+  #include <errno.h>
+  int main(void)
+  {
+  	// Move the current task to the isolated cpuset (bind to CPU 7)
+  	int fd = open("/sys/fs/cgroup/test/cgroup.procs", O_WRONLY);
+  	if (fd < 0) {
+  		perror("Can't open cpuset file...\n");
+  		return 0;
+  	}
+
+  	write(fd, "0\n", 2);
+  	close(fd);
+
+  	// Run an endless dummy loop until the launcher kills us
+  	while (1)
+  	;
+
+  	return 0;
+  }
+
+Build it and save for later step:
+
+  # gcc user_loop.c -o user_loop
+
+The launcher
+------------
+
+The below launcher runs the above program for 10 seconds and traces
+the noise resulting from preempting tasks and IRQs.
+
+  TRACING=/sys/kernel/tracing/
+  # Make sure tracing is off for now
+  echo 0 > $TRACING/tracing_on
+  # Flush previous traces
+  echo > $TRACING/trace
+  # Record disturbance from other tasks
+  echo 1 > $TRACING/events/sched/sched_switch/enable
+  # Record disturbance from interrupts
+  echo 1 > $TRACING/events/irq_vectors/enable
+  # Now we can start tracing
+  echo 1 > $TRACING/tracing_on
+  # Run the dummy user_loop for 10 seconds on CPU 7
+  ./user_loop &
+  USER_LOOP_PID=$!
+  sleep 10
+  kill $USER_LOOP_PID
+  # Disable tracing and save traces from CPU 7 in a file
+  echo 0 > $TRACING/tracing_on
+  cat $TRACING/per_cpu/cpu7/trace > trace.7
+
+If no specific problem arose, the output of trace.7 should look like
+the following:
+
+  <idle>-0 [007] d..2. 1980.976624: sched_switch: prev_comm=swapper/7 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=user_loop next_pid=1553 next_prio=120
+  user_loop-1553 [007] d.h.. 1990.946593: reschedule_entry: vector=253
+  user_loop-1553 [007] d.h.. 1990.946593: reschedule_exit: vector=253
+
+That is, no specific noise triggered between the first trace and the
+second during 10 seconds when user_loop was running.
+
+Debugging
+=========
+
+Of course things are never so easy, especially on this matter.
+Chances are that actual noise will be observed in the aforementioned
+trace.7 file.
+
+The best way to investigate further is to enable finer grained
+tracepoints such as those of subsystems producing asynchronous
+events: workqueue, timer, irq_vector, etc... It also can be
+interesting to enable the tick_stop event to diagnose why the tick is
+retained when that happens.
+
+Some tools may also be useful for higher level analysis:
+
+- :ref:`Documentation/tools/rtla/rtla-osnoise.rst <rtla-osnoise>` runs a kernel
+  tracer that analyzes and output a
+  summary of the noises.
+
+- dynticks-testing does something similar but in userspace. It is available
+  at git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git
Documentation/admin-guide/index.rst +1 -0
index 259d79fbeb94..b5f1fc7d5290 100644
@@ -94,6 +94,7 @@ likely to be of interest on almost any system.
 
    cgroup-v2
    cgroup-v1/index
+   cpu-isolation
    cpu-load
    mm/index
    module-signing
Signature · 1 line
2.50.1
#1 lore raw

Review

Quoted 3 lines · Frederic Weisbecker

+- :ref:`Documentation/admin-guide/cgroup-v2.rst <Cpuset v2 "isolated"

(quoted diff) +2 -0
+  partitions>`
+  are recommended because they are tunable at runtime.

Anchor link target and name are mistakenly swapped so I have to correct them back to their appropriate places:

---- >8 ----

Documentation/admin-guide/cpu-isolation.rst +6 -7
index 250027acf7b26f..aef0b53b0ad5e6 100644
@@ -107,9 +107,8 @@ are extracted from the global load balancing.
 Interface
 ~~~~~~~~~
 
-- :ref:`Documentation/admin-guide/cgroup-v2.rst <Cpuset v2 "isolated"
-  partitions>`
-  are recommended because they are tunable at runtime.
+- :doc:`cgroup cpuset isolated partitions <cgroup-v2>` are recommended because
+  they are tunable at runtime.
 
 - The 'isolcpus=' kernel boot parameter with the 'domain' flag is a
   less flexible alternative that doesn't allow for runtime
@@ -124,7 +123,8 @@ target CPUs.
 Interface
 ~~~~~~~~~
 
-- /proc/irq/*/smp_affinity as explained :ref:`Documentation/core-api/irq/irq-affinity.rst <here>` in detail.
+- /proc/irq/\*/smp_affinity as explained in
+  Documentation/core-api/irq/irq-affinity.rst.
 
 - The "irqaffinity=" kernel boot parameter for a default setting.
 
@@ -330,9 +330,8 @@ retained when that happens.
 
 Some tools may also be useful for higher level analysis:
 
-- :ref:`Documentation/tools/rtla/rtla-osnoise.rst <rtla-osnoise>` runs a kernel
-  tracer that analyzes and output a
-  summary of the noises.
+- :doc:`rtla-osnoise </tools/rtla/rtla-osnoise>` runs a kernel tracer that
+  analyzes and output a summary of the noises.
 
 - dynticks-testing does something similar but in userspace. It is available
   at git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git
Quoted 76 lines
(quoted diff) +76 -0
+The full command line is then:
+
+  nohz_full=7 irqaffinity=0-6 isolcpus=managed_irq,7 nosmt
+
+CPUSET configuration (cgroup v2)
+--------------------------------
+
+Assuming cgroup v2 is mounted to /sys/fs/cgroup, the following script
+isolates CPU 7 from scheduler domains.
+
+  cd /sys/fs/cgroup
+  # Activate the cpuset subsystem
+  echo +cpuset > cgroup.subtree_control
+  # Create partition to be isolated
+  mkdir test
+  cd test
+  echo +cpuset > cgroup.subtree_control
+  # Isolate CPU 7
+  echo 7 > cpuset.cpus
+  echo "isolated" > cpuset.cpus.partition
+
+The userspace workload
+----------------------
+
+Fake a pure userspace workload, the below program runs a dummy
+userspace loop on the isolated CPU 7.
+
+  #include <stdio.h>
+  #include <fcntl.h>
+  #include <unistd.h>
+  #include <errno.h>
+  int main(void)
+  {
+  	// Move the current task to the isolated cpuset (bind to CPU 7)
+  	int fd = open("/sys/fs/cgroup/test/cgroup.procs", O_WRONLY);
+  	if (fd < 0) {
+  		perror("Can't open cpuset file...\n");
+  		return 0;
+  	}
+
+  	write(fd, "0\n", 2);
+  	close(fd);
+
+  	// Run an endless dummy loop until the launcher kills us
+  	while (1)
+  	;
+
+  	return 0;
+  }
+
+Build it and save for later step:
+
+  # gcc user_loop.c -o user_loop
+
+The launcher
+------------
+
+The below launcher runs the above program for 10 seconds and traces
+the noise resulting from preempting tasks and IRQs.
+
+  TRACING=/sys/kernel/tracing/
+  # Make sure tracing is off for now
+  echo 0 > $TRACING/tracing_on
+  # Flush previous traces
+  echo > $TRACING/trace
+  # Record disturbance from other tasks
+  echo 1 > $TRACING/events/sched/sched_switch/enable
+  # Record disturbance from interrupts
+  echo 1 > $TRACING/events/irq_vectors/enable
+  # Now we can start tracing
+  echo 1 > $TRACING/tracing_on
+  # Run the dummy user_loop for 10 seconds on CPU 7
+  ./user_loop &
+  USER_LOOP_PID=$!
+  sleep 10
+  kill $USER_LOOP_PID

Review

Quoted 10 lines
(quoted diff) +10 -0
+  # Disable tracing and save traces from CPU 7 in a file
+  echo 0 > $TRACING/tracing_on
+  cat $TRACING/per_cpu/cpu7/trace > trace.7
+
+If no specific problem arose, the output of trace.7 should look like
+the following:
+
+  <idle>-0 [007] d..2. 1980.976624: sched_switch: prev_comm=swapper/7 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=user_loop next_pid=1553 next_prio=120
+  user_loop-1553 [007] d.h.. 1990.946593: reschedule_entry: vector=253
+  user_loop-1553 [007] d.h.. 1990.946593: reschedule_exit: vector=253

Wrap these snippets above in literal code blocks.

Thanks.

Signature · 1 line
An old man doll... just what I always wanted! - Clara
#2 lore raw
Quoted 137 lines · Bagas Sanjaya

Review

Quoted 3 lines · Frederic Weisbecker

+- :ref:`Documentation/admin-guide/cgroup-v2.rst <Cpuset v2 "isolated"

(quoted diff) +2 -0
+  partitions>`
+  are recommended because they are tunable at runtime.

Anchor link target and name are mistakenly swapped so I have to correct them back to their appropriate places:

---- >8 ----

Documentation/admin-guide/cpu-isolation.rst +6 -7
index 250027acf7b26f..aef0b53b0ad5e6 100644
@@ -107,9 +107,8 @@ are extracted from the global load balancing.
 Interface
 ~~~~~~~~~
 
-- :ref:`Documentation/admin-guide/cgroup-v2.rst <Cpuset v2 "isolated"
-  partitions>`
-  are recommended because they are tunable at runtime.
+- :doc:`cgroup cpuset isolated partitions <cgroup-v2>` are recommended because
+  they are tunable at runtime.
 
 - The 'isolcpus=' kernel boot parameter with the 'domain' flag is a
   less flexible alternative that doesn't allow for runtime
@@ -124,7 +123,8 @@ target CPUs.
 Interface
 ~~~~~~~~~
 
-- /proc/irq/*/smp_affinity as explained :ref:`Documentation/core-api/irq/irq-affinity.rst <here>` in detail.
+- /proc/irq/\*/smp_affinity as explained in
+  Documentation/core-api/irq/irq-affinity.rst.
 
 - The "irqaffinity=" kernel boot parameter for a default setting.
 
@@ -330,9 +330,8 @@ retained when that happens.
 
 Some tools may also be useful for higher level analysis:
 
-- :ref:`Documentation/tools/rtla/rtla-osnoise.rst <rtla-osnoise>` runs a kernel
-  tracer that analyzes and output a
-  summary of the noises.
+- :doc:`rtla-osnoise </tools/rtla/rtla-osnoise>` runs a kernel tracer that
+  analyzes and output a summary of the noises.
 
 - dynticks-testing does something similar but in userspace. It is available
   at git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git
Quoted 77 lines
(quoted diff) +76 -0
+The full command line is then:
+
+  nohz_full=7 irqaffinity=0-6 isolcpus=managed_irq,7 nosmt
+
+CPUSET configuration (cgroup v2)
+--------------------------------
+
+Assuming cgroup v2 is mounted to /sys/fs/cgroup, the following script
+isolates CPU 7 from scheduler domains.
+
+  cd /sys/fs/cgroup
+  # Activate the cpuset subsystem
+  echo +cpuset > cgroup.subtree_control
+  # Create partition to be isolated
+  mkdir test
+  cd test
+  echo +cpuset > cgroup.subtree_control
+  # Isolate CPU 7
+  echo 7 > cpuset.cpus
+  echo "isolated" > cpuset.cpus.partition
+
+The userspace workload
+----------------------
+
+Fake a pure userspace workload, the below program runs a dummy
+userspace loop on the isolated CPU 7.
+
+  #include <stdio.h>
+  #include <fcntl.h>
+  #include <unistd.h>
+  #include <errno.h>
+  int main(void)
+  {
+  	// Move the current task to the isolated cpuset (bind to CPU 7)
+  	int fd = open("/sys/fs/cgroup/test/cgroup.procs", O_WRONLY);
+  	if (fd < 0) {
+  		perror("Can't open cpuset file...\n");
+  		return 0;
+  	}
+
+  	write(fd, "0\n", 2);
+  	close(fd);
+
+  	// Run an endless dummy loop until the launcher kills us
+  	while (1)
+  	;
+
+  	return 0;
+  }
+
+Build it and save for later step:
+
+  # gcc user_loop.c -o user_loop
+
+The launcher
+------------
+
+The below launcher runs the above program for 10 seconds and traces
+the noise resulting from preempting tasks and IRQs.
+
+  TRACING=/sys/kernel/tracing/
+  # Make sure tracing is off for now
+  echo 0 > $TRACING/tracing_on
+  # Flush previous traces
+  echo > $TRACING/trace
+  # Record disturbance from other tasks
+  echo 1 > $TRACING/events/sched/sched_switch/enable
+  # Record disturbance from interrupts
+  echo 1 > $TRACING/events/irq_vectors/enable
+  # Now we can start tracing
+  echo 1 > $TRACING/tracing_on
+  # Run the dummy user_loop for 10 seconds on CPU 7
+  ./user_loop &
+  USER_LOOP_PID=$!
+  sleep 10
+  kill $USER_LOOP_PID

Review

Quoted 10 lines
(quoted diff) +10 -0
+  # Disable tracing and save traces from CPU 7 in a file
+  echo 0 > $TRACING/tracing_on
+  cat $TRACING/per_cpu/cpu7/trace > trace.7
+
+If no specific problem arose, the output of trace.7 should look like
+the following:
+
+  <idle>-0 [007] d..2. 1980.976624: sched_switch: prev_comm=swapper/7 prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=user_loop next_pid=1553 next_prio=120
+  user_loop-1553 [007] d.h.. 1990.946593: reschedule_entry: vector=253
+  user_loop-1553 [007] d.h.. 1990.946593: reschedule_exit: vector=253

Wrap these snippets above in literal code blocks.

Quoted 1 line

Thanks.

Thanks, I'll do these modifications. I had a hard time building the documentation (still haven't managed so far) so I missed the actual rendering. Hopefully I'll figure it out at some point.

Signature · 2 lines
Frederic Weisbecker
SUSE Labs
#3 lore raw
In reply to Frederic Weisbecker
Quoted 3 lines · Frederic Weisbecker

nohz_full was introduced in v3.10 in 2013, which means this documentation is overdue for 12 years.

12 years is not that bad, it's not old enough to drink (legally) yet!

Quoted 23 lines

The shoemaker's children always go barefoot. And working on timers hasn't made me arriving on time either.

Fortunately Paul wrote a part of the needed documentation a while ago, especially concerning nohz_full in Documentation/timers/no_hz.rst and also about per-CPU kthreads in Documentation/admin-guide/kernel-per-CPU-kthreads.rst

Introduce a new page that gives an overview of CPU isolation in general.

---
 Documentation/admin-guide/cpu-isolation.rst | 338 ++++++++++++++++++++
 Documentation/admin-guide/index.rst         |   1 +
 2 files changed, 339 insertions(+)
 create mode 100644 Documentation/admin-guide/cpu-isolation.rst
Documentation/admin-guide/cpu-isolation.rst new file +0 -0
new file mode 100644
index 000000000000..250027acf7b2

Documentation/admin-guide/cpu-isolation.rst line 8

Quoted 8 lines
Documentation/admin-guide/cpu-isolation.rst new file +8 -0
@@ -0,0 +1,338 @@
+=============
+CPU Isolation
+=============
+
+Introduction
+============
+
+"CPU Isolation" means leaving a CPU exclusive to a given userspace
                                                            ^^^^^^^^^

Eh I'm being nitpicky, but this doesn't have to be userspace stuff right? "someone" could e.g. affine some IRQ to an isolated CPU to have the irqthread run undisturbed there, or somesuch.

Documentation/admin-guide/cpu-isolation.rst line 91

Quoted 7 lines
Documentation/admin-guide/cpu-isolation.rst new file +7 -0
+
+Scheduler domain isolation
+--------------------------
+
+This feature isolates a CPU from the scheduler topology. As a result,
+the target isn't part of the load balancing. Tasks won't migrate
+neither from nor to it unless affine explicitly.
                                 ^^^^^^

s/affine/affined/

Quoted 16 lines
Documentation/admin-guide/cpu-isolation.rst new file +15 -0
+As a side effect the CPU is also isolated from unbound workqueues and
+unbound kthreads.
 
+Checklist
+=========
+
+You have set up each of the above isolation features but you still
+observe jitters that trash your workload? Make sure to check a few
+elements before proceeding.
+
+Some of these checklist items are similar to those of real time
+workloads:
+
+- Use mlock() to prevent your pages from being swapped away. Page
+  faults are usually not compatible with jitter sensitive workloads.
+

Documentation/admin-guide/cpu-isolation.rst line 206

Quoted 10 lines
Documentation/admin-guide/cpu-isolation.rst new file +10 -0
+- Avoid SMT to prevent your hardware thread from being "preempted"
+  by another one.
+
+- CPU frequency changes may induce subtle sorts of jitter in a
+  workload. Cpufreq should be used and tuned with caution.
+
+- Deep C-states may result in latency issues upon wake-up. If this
+  happens to be a problem, C-states can be limited via kernel boot
+  parameters such as processor.max_cstate or intel_idle.max_cstate.
+

Nitpickery again, I know it's not an exhaustive listing, but I'd rather point to the sysfs cpuidle interface (or just mention it too), since that means deep C-states can be left enabled for HK CPUs.

Should we also mention BIOS/firmware fuckery like SMIs?

"""
- Your system may be subject to firmware-originating interrupts - x86 has System Management Interrupts (SMIs) for example. Check your system BIOS to disable such interference, and with some luck your vendor will have a BIOS tuning guidance for low-latency operations.
"""

Documentation/admin-guide/cpu-isolation.rst line 330

Quoted 13 lines
Documentation/admin-guide/cpu-isolation.rst new file +13 -0
+Debugging
+=========
+
+Of course things are never so easy, especially on this matter.
+Chances are that actual noise will be observed in the aforementioned
+trace.7 file.
+
+The best way to investigate further is to enable finer grained
+tracepoints such as those of subsystems producing asynchronous
+events: workqueue, timer, irq_vector, etc... It also can be
+interesting to enable the tick_stop event to diagnose why the tick is
+retained when that happens.
+

I'd also list the 'ipi_send*' family, although that's emitted from the HK CPU, not the disturbed isolated CPU.

Documentation/admin-guide/cpu-isolation.rst line 336

Quoted 6 lines
Documentation/admin-guide/cpu-isolation.rst new file +6 -0
+Some tools may also be useful for higher level analysis:
+
+- :ref:`Documentation/tools/rtla/rtla-osnoise.rst <rtla-osnoise>` runs a kernel
+  tracer that analyzes and output a
+  summary of the noises.
+

I'd want to point to hwnoise and timerlat as well, so maybe point to rtla.rst?

Quoted 16 lines

+- dynticks-testing does something similar but in userspace. It is available

Documentation/admin-guide/cpu-isolation.rst new file +1 -0
+  at git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git
Documentation/admin-guide/index.rst +1 -1
index 259d79fbeb94..b5f1fc7d5290 100644
@@ -94,6 +94,7 @@ likely to be of interest on almost any system.
 
    cgroup-v2
    cgroup-v1/index
+   cpu-isolation
    cpu-load
    mm/index
    module-signing
--

2.50.1

#4 lore raw
Quoted 6 lines · Valentin Schneider
Quoted 3 lines · Frederic Weisbecker

nohz_full was introduced in v3.10 in 2013, which means this documentation is overdue for 12 years.

12 years is not that bad, it's not old enough to drink (legally) yet!

;-)

Quoted 33 lines
Quoted 24 lines

The shoemaker's children always go barefoot. And working on timers hasn't made me arriving on time either.

Fortunately Paul wrote a part of the needed documentation a while ago, especially concerning nohz_full in Documentation/timers/no_hz.rst and also about per-CPU kthreads in Documentation/admin-guide/kernel-per-CPU-kthreads.rst

Introduce a new page that gives an overview of CPU isolation in general.

---
 Documentation/admin-guide/cpu-isolation.rst | 338 ++++++++++++++++++++
 Documentation/admin-guide/index.rst         |   1 +
 2 files changed, 339 insertions(+)
 create mode 100644 Documentation/admin-guide/cpu-isolation.rst
Documentation/admin-guide/cpu-isolation.rst new file +0 -0
new file mode 100644
index 000000000000..250027acf7b2

Documentation/admin-guide/cpu-isolation.rst line 8

Quoted 8 lines
Documentation/admin-guide/cpu-isolation.rst new file +8 -0
@@ -0,0 +1,338 @@
+=============
+CPU Isolation
+=============
+
+Introduction
+============
+
+"CPU Isolation" means leaving a CPU exclusive to a given userspace
                                                            ^^^^^^^^^
Quoted 3 lines

Eh I'm being nitpicky, but this doesn't have to be userspace stuff right? "someone" could e.g. affine some IRQ to an isolated CPU to have the irqthread run undisturbed there, or somesuch.

Good point!

Quoted 9 lines

Documentation/admin-guide/cpu-isolation.rst line 91

Quoted 7 lines
Documentation/admin-guide/cpu-isolation.rst new file +7 -0
+
+Scheduler domain isolation
+--------------------------
+
+This feature isolates a CPU from the scheduler topology. As a result,
+the target isn't part of the load balancing. Tasks won't migrate
+neither from nor to it unless affine explicitly.
                                 ^^^^^^

s/affine/affined/

Right.

Quoted 23 lines
Quoted 23 lines
Documentation/admin-guide/cpu-isolation.rst new file +21 -0
+As a side effect the CPU is also isolated from unbound workqueues and
+unbound kthreads.
 
+Checklist
+=========
+
+You have set up each of the above isolation features but you still
+observe jitters that trash your workload? Make sure to check a few
+elements before proceeding.
+
+Some of these checklist items are similar to those of real time
+workloads:
+
+- Use mlock() to prevent your pages from being swapped away. Page
+  faults are usually not compatible with jitter sensitive workloads.
+
+- Avoid SMT to prevent your hardware thread from being "preempted"
+  by another one.
+
+- CPU frequency changes may induce subtle sorts of jitter in a
+  workload. Cpufreq should be used and tuned with caution.
+
Quoted 8 lines

Documentation/admin-guide/cpu-isolation.rst line 206

Quoted 4 lines

+- Deep C-states may result in latency issues upon wake-up. If this

Documentation/admin-guide/cpu-isolation.rst new file +3 -0
+  happens to be a problem, C-states can be limited via kernel boot
+  parameters such as processor.max_cstate or intel_idle.max_cstate.
+

Nitpickery again, I know it's not an exhaustive listing, but I'd rather point to the sysfs cpuidle interface (or just mention it too), since that means deep C-states can be left enabled for HK CPUs.

Yes!

Quoted 10 lines

Should we also mention BIOS/firmware fuckery like SMIs?

"""
- Your system may be subject to firmware-originating interrupts - x86 has System Management Interrupts (SMIs) for example. Check your system BIOS to disable such interference, and with some luck your vendor will have a BIOS tuning guidance for low-latency operations.
"""

Definetely!

Quoted 17 lines
Quoted 4 lines
Documentation/admin-guide/cpu-isolation.rst new file +3 -0
+Debugging
+=========
+

Documentation/admin-guide/cpu-isolation.rst line 330

Quoted 10 lines
Documentation/admin-guide/cpu-isolation.rst new file +10 -0
+Of course things are never so easy, especially on this matter.
+Chances are that actual noise will be observed in the aforementioned
+trace.7 file.
+
+The best way to investigate further is to enable finer grained
+tracepoints such as those of subsystems producing asynchronous
+events: workqueue, timer, irq_vector, etc... It also can be
+interesting to enable the tick_stop event to diagnose why the tick is
+retained when that happens.
+

I'd also list the 'ipi_send*' family, although that's emitted from the HK CPU, not the disturbed isolated CPU.

Yeah I can do that.

Quoted 10 lines

Documentation/admin-guide/cpu-isolation.rst line 336

Quoted 7 lines
Documentation/admin-guide/cpu-isolation.rst new file +6 -0
+Some tools may also be useful for higher level analysis:
+
+- :ref:`Documentation/tools/rtla/rtla-osnoise.rst <rtla-osnoise>` runs a kernel
+  tracer that analyzes and output a
+  summary of the noises.
+

I'd want to point to hwnoise and timerlat as well, so maybe point to rtla.rst?

Good point.

Thanks!

Quoted 18 lines
Quoted 17 lines

+- dynticks-testing does something similar but in userspace. It is available

Documentation/admin-guide/cpu-isolation.rst new file +1 -0
+  at git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git
Documentation/admin-guide/index.rst +1 -1
index 259d79fbeb94..b5f1fc7d5290 100644
@@ -94,6 +94,7 @@ likely to be of interest on almost any system.
 
    cgroup-v2
    cgroup-v1/index
+   cpu-isolation
    cpu-load
    mm/index
    module-signing
--

2.50.1

Signature · 2 lines
Frederic Weisbecker
SUSE Labs
#5 lore raw
In reply to Frederic Weisbecker
Quoted 6 lines · Frederic Weisbecker

nohz_full was introduced in v3.10 in 2013, which means this documentation is overdue for 12 years.

The shoemaker's children always go barefoot. And working on timers hasn't made me arriving on time either.

Quoted 8 lines

Fortunately Paul wrote a part of the needed documentation a while ago, especially concerning nohz_full in Documentation/timers/no_hz.rst and also about per-CPU kthreads in Documentation/admin-guide/kernel-per-CPU-kthreads.rst

Introduce a new page that gives an overview of CPU isolation in general.

Thanks for working on this document. It will be really useful for those customers that need CPU isolation.

Quoted 23 lines
---
  Documentation/admin-guide/cpu-isolation.rst | 338 ++++++++++++++++++++
  Documentation/admin-guide/index.rst         |   1 +
  2 files changed, 339 insertions(+)

create mode 100644 Documentation/admin-guide/cpu-isolation.rst

Documentation/admin-guide/cpu-isolation.rst new file +11 -0
new file mode 100644
index 000000000000..250027acf7b2
@@ -0,0 +1,338 @@
+=============
+CPU Isolation
+=============
+
+Introduction
+============
+
+"CPU Isolation" means leaving a CPU exclusive to a given userspace
+workload without any undesired code interference from the kernel.
+
+Those interferences, commonly pointed out as "noise", can be triggered

Documentation/admin-guide/cpu-isolation.rst line 21

Quoted 10 lines
Documentation/admin-guide/cpu-isolation.rst new file +10 -0
+by asynchronous events (interrupts, timers, scheduler preemption by
+workqueues and kthreads, ...) or synchronous events (syscalls and page
+faults).
+
+Such noise usually goes unnoticed. After all synchronous events are a
+component of the requested kernel service. And asynchronous events are
+either sufficiently well distributed by the scheduler when executed
+as tasks or reasonably fast when executed as interrupt. The timer
+interrupt can even execute 1024 times per seconds without
+significantly measurable impact most of the time.

"significantly measurable" looks odd to me. Maybe "without a significant and measurable impact most of the time".

Documentation/admin-guide/cpu-isolation.rst line 174

Quoted 12 lines
Documentation/admin-guide/cpu-isolation.rst new file +12 -0
+
+Tradeoffs
+~~~~~~~~~
+
+In terms of cost, this is the most invasive isolation feature. It is
+assumed to be used when the workload spends most of its time in
+userspace and doesn't rely on the kernel except for preparatory
+work because:
+
+- RCU is slower due to the locked, offloaded and threaded callbacks
+  processing (the same that would be obtained with "rcu_nocb=" boot
+  parameter).

Should be "rcu_nocbs". The '=' is optional.

Cheers,
Longman

#6 lore raw
Quoted 48 lines · Waiman Long
Quoted 6 lines · Frederic Weisbecker

nohz_full was introduced in v3.10 in 2013, which means this documentation is overdue for 12 years.

The shoemaker's children always go barefoot. And working on timers hasn't made me arriving on time either.

Quoted 8 lines

Fortunately Paul wrote a part of the needed documentation a while ago, especially concerning nohz_full in Documentation/timers/no_hz.rst and also about per-CPU kthreads in Documentation/admin-guide/kernel-per-CPU-kthreads.rst

Introduce a new page that gives an overview of CPU isolation in general.

Thanks for working on this document. It will be really useful for those customers that need CPU isolation.

Quoted 28 lines
---
  Documentation/admin-guide/cpu-isolation.rst | 338 ++++++++++++++++++++
  Documentation/admin-guide/index.rst         |   1 +
  2 files changed, 339 insertions(+)

create mode 100644 Documentation/admin-guide/cpu-isolation.rst

Documentation/admin-guide/cpu-isolation.rst new file +16 -0
new file mode 100644
index 000000000000..250027acf7b2
@@ -0,0 +1,338 @@
+=============
+CPU Isolation
+=============
+
+Introduction
+============
+
+"CPU Isolation" means leaving a CPU exclusive to a given userspace
+workload without any undesired code interference from the kernel.
+
+Those interferences, commonly pointed out as "noise", can be triggered
+by asynchronous events (interrupts, timers, scheduler preemption by
+workqueues and kthreads, ...) or synchronous events (syscalls and page
+faults).
+
+Such noise usually goes unnoticed. After all synchronous events are a
Quoted 8 lines

Documentation/admin-guide/cpu-isolation.rst line 21

Quoted 5 lines
Documentation/admin-guide/cpu-isolation.rst new file +5 -0
+component of the requested kernel service. And asynchronous events are
+either sufficiently well distributed by the scheduler when executed
+as tasks or reasonably fast when executed as interrupt. The timer
+interrupt can even execute 1024 times per seconds without
+significantly measurable impact most of the time.

"significantly measurable" looks odd to me. Maybe "without a significant and measurable impact most of the time".

Makes sense.

Quoted 14 lines

Documentation/admin-guide/cpu-isolation.rst line 174

Quoted 13 lines
Documentation/admin-guide/cpu-isolation.rst new file +12 -0
+
+Tradeoffs
+~~~~~~~~~
+
+In terms of cost, this is the most invasive isolation feature. It is
+assumed to be used when the workload spends most of its time in
+userspace and doesn't rely on the kernel except for preparatory
+work because:
+
+- RCU is slower due to the locked, offloaded and threaded callbacks
+  processing (the same that would be obtained with "rcu_nocb=" boot
+  parameter).

Should be "rcu_nocbs". The '=' is optional.

Ok.

Thanks!

Quoted 4 lines

Cheers,
Longman

Signature · 2 lines
Frederic Weisbecker
SUSE Labs
#7 lore raw
In reply to Frederic Weisbecker

Documentation/admin-guide/cpu-isolation.rst

Quoted 3 lines · Frederic Weisbecker
Documentation/admin-guide/cpu-isolation.rst new file +0 -0
@@ -0,0 +1,338 @@

You could start with

.. SPDX-License-Identifier: GPL-2.0

at the top.

Quoted 3 lines
Documentation/admin-guide/cpu-isolation.rst new file +3 -0
+=============
+CPU Isolation
+=============

…

Documentation/admin-guide/cpu-isolation.rst line 111

Quoted 5 lines
Documentation/admin-guide/cpu-isolation.rst new file +5 -0
+Interface
+~~~~~~~~~
+
+- :ref:`Documentation/admin-guide/cgroup-v2.rst <Cpuset v2 "isolated"
+  partitions>`

I've been told by Jonathan once to just use the .rst file without anything around it and the HTML render will make the link on its own.

…

Documentation/admin-guide/cpu-isolation.rst line 174

Quoted 11 lines
Documentation/admin-guide/cpu-isolation.rst new file +11 -0
+Tradeoffs
+~~~~~~~~~
+
+In terms of cost, this is the most invasive isolation feature. It is
+assumed to be used when the workload spends most of its time in
+userspace and doesn't rely on the kernel except for preparatory
+work because:
+
+- RCU is slower due to the locked, offloaded and threaded callbacks
+  processing (the same that would be obtained with "rcu_nocb=" boot
+  parameter).

You mean the callback invocation is delayed? It shouldn't affect grace period handling and so on.

…

Quoted 13 lines
Documentation/admin-guide/cpu-isolation.rst new file +13 -0
+Checklist
+=========
+
+You have set up each of the above isolation features but you still
+observe jitters that trash your workload? Make sure to check a few
+elements before proceeding.
+
+Some of these checklist items are similar to those of real time
+workloads:
+
+- Use mlock() to prevent your pages from being swapped away. Page
+  faults are usually not compatible with jitter sensitive workloads.
+

Documentation/admin-guide/cpu-isolation.rst line 206

Quoted 10 lines
Documentation/admin-guide/cpu-isolation.rst new file +10 -0
+- Avoid SMT to prevent your hardware thread from being "preempted"
+  by another one.
+
+- CPU frequency changes may induce subtle sorts of jitter in a
+  workload. Cpufreq should be used and tuned with caution.
+
+- Deep C-states may result in latency issues upon wake-up. If this
+  happens to be a problem, C-states can be limited via kernel boot
+  parameters such as processor.max_cstate or intel_idle.max_cstate.
+

I intended to make a similar list similar to this for the real time part but it somehow faded away. There is now the idea to identify kernel options which are not so optimal which include CPU frequency for instance. The requirements here are should be the same.

Sebastian

#8 lore raw
Quoted 9 lines · Sebastian Andrzej Siewior

Documentation/admin-guide/cpu-isolation.rst

Quoted 3 lines · Frederic Weisbecker
Documentation/admin-guide/cpu-isolation.rst new file +0 -0
@@ -0,0 +1,338 @@

You could start with

.. SPDX-License-Identifier: GPL-2.0

at the top.

Ok.

Quoted 14 lines
Quoted 4 lines
Documentation/admin-guide/cpu-isolation.rst new file +3 -0
+=============
+CPU Isolation
+=============

…

Documentation/admin-guide/cpu-isolation.rst line 111

Quoted 5 lines
Documentation/admin-guide/cpu-isolation.rst new file +5 -0
+Interface
+~~~~~~~~~
+
+- :ref:`Documentation/admin-guide/cgroup-v2.rst <Cpuset v2 "isolated"
+  partitions>`

I've been told by Jonathan once to just use the .rst file without anything around it and the HTML render will make the link on its own.

Quoted 1 line

…

Hmm, how would that look like? Just this?

+- Documentation/admin-guide/cgroup-v2.rst

Quoted 16 lines

Documentation/admin-guide/cpu-isolation.rst line 174

Quoted 12 lines
Documentation/admin-guide/cpu-isolation.rst new file +11 -0
+Tradeoffs
+~~~~~~~~~
+
+In terms of cost, this is the most invasive isolation feature. It is
+assumed to be used when the workload spends most of its time in
+userspace and doesn't rely on the kernel except for preparatory
+work because:
+
+- RCU is slower due to the locked, offloaded and threaded callbacks
+  processing (the same that would be obtained with "rcu_nocb=" boot
+  parameter).

You mean the callback invocation is delayed? It shouldn't affect grace period handling and so on.

Quoted 1 line

…

Right.

Quoted 21 lines
Quoted 21 lines
Documentation/admin-guide/cpu-isolation.rst new file +20 -0
+Checklist
+=========
+
+You have set up each of the above isolation features but you still
+observe jitters that trash your workload? Make sure to check a few
+elements before proceeding.
+
+Some of these checklist items are similar to those of real time
+workloads:
+
+- Use mlock() to prevent your pages from being swapped away. Page
+  faults are usually not compatible with jitter sensitive workloads.
+
+- Avoid SMT to prevent your hardware thread from being "preempted"
+  by another one.
+
+- CPU frequency changes may induce subtle sorts of jitter in a
+  workload. Cpufreq should be used and tuned with caution.
+
+- Deep C-states may result in latency issues upon wake-up. If this
Quoted 8 lines

Documentation/admin-guide/cpu-isolation.rst line 206

Quoted 3 lines
Documentation/admin-guide/cpu-isolation.rst new file +3 -0
+  happens to be a problem, C-states can be limited via kernel boot
+  parameters such as processor.max_cstate or intel_idle.max_cstate.
+

I intended to make a similar list similar to this for the real time part but it somehow faded away. There is now the idea to identify kernel options which are not so optimal which include CPU frequency for instance. The requirements here are should be the same.

We should probably have this to a seperate file that both isolation and rt could link to?

Thanks.

Signature · 2 lines
Frederic Weisbecker
SUSE Labs
#9 lore raw
Quoted 9 lines · Frederic Weisbecker
Quoted 9 lines

Documentation/admin-guide/cpu-isolation.rst line 111

Quoted 5 lines
Documentation/admin-guide/cpu-isolation.rst new file +5 -0
+Interface
+~~~~~~~~~
+
+- :ref:`Documentation/admin-guide/cgroup-v2.rst <Cpuset v2 "isolated"
+  partitions>`

I've been told by Jonathan once to just use the .rst file without anything around it and the HTML render will make the link on its own.

Quoted 5 lines
Quoted 1 line

…

Hmm, how would that look like? Just this?

+- Documentation/admin-guide/cgroup-v2.rst

Yes. If you look at
Documentation/core-api/real-time/differences.rst

and look for "Sequence locks". If you compare this vs the render version
https://docs.kernel.org/core-api/real-time/differences.html#sequence-locks

then you see it got resolved as html link including description.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

+Checklist
+=========

…

Quoted 3 lines

We should probably have this to a seperate file that both isolation and rt could link to?

yeah. Either you look for a spot and start or keep it as-is and I rip it out and a reference to a common document once we have on.

Quoted 1 line

Thanks.

Sebastian

#10 lore raw
In reply to Frederic Weisbecker

On 2025-08-09 11:42:47 [+0200], Frederic Weisbecker wrote:
…

Quoted 3 lines
Documentation/admin-guide/cpu-isolation.rst new file +0 -0
@@ -0,0 +1,338 @@

…

Quoted 6 lines
Documentation/admin-guide/cpu-isolation.rst new file +6 -0
+CPUSET configuration (cgroup v2)
+--------------------------------
+
+Assuming cgroup v2 is mounted to /sys/fs/cgroup, the following script
+isolates CPU 7 from scheduler domains.
+

Documentation/admin-guide/cpu-isolation.rst line 247

Quoted 10 lines
Documentation/admin-guide/cpu-isolation.rst new file +10 -0
+  cd /sys/fs/cgroup
+  # Activate the cpuset subsystem
+  echo +cpuset > cgroup.subtree_control
+  # Create partition to be isolated
+  mkdir test
+  cd test
+  echo +cpuset > cgroup.subtree_control
+  # Isolate CPU 7
+  echo 7 > cpuset.cpus
+  echo "isolated" > cpuset.cpus.partition

while looking at this again, shouldn't you also do
echo 7 > cpuset.cpus.exclusive

to ensure the CPU isn't used somewhere else?

Sebastian

#11 lore raw
Quoted 25 lines · Sebastian Andrzej Siewior

On 2025-08-09 11:42:47 [+0200], Frederic Weisbecker wrote:
…

Quoted 3 lines
Documentation/admin-guide/cpu-isolation.rst new file +0 -0
@@ -0,0 +1,338 @@

…

Quoted 6 lines
Documentation/admin-guide/cpu-isolation.rst new file +6 -0
+CPUSET configuration (cgroup v2)
+--------------------------------
+
+Assuming cgroup v2 is mounted to /sys/fs/cgroup, the following script
+isolates CPU 7 from scheduler domains.
+

Documentation/admin-guide/cpu-isolation.rst line 247

Quoted 10 lines
Documentation/admin-guide/cpu-isolation.rst new file +10 -0
+  cd /sys/fs/cgroup
+  # Activate the cpuset subsystem
+  echo +cpuset > cgroup.subtree_control
+  # Create partition to be isolated
+  mkdir test
+  cd test
+  echo +cpuset > cgroup.subtree_control
+  # Isolate CPU 7
+  echo 7 > cpuset.cpus
+  echo "isolated" > cpuset.cpus.partition

while looking at this again, shouldn't you also do
echo 7 > cpuset.cpus.exclusive

Quoted 1 line

to ensure the CPU isn't used somewhere else?

For backport compatibility, the use of cpuset.cpus.exclusive is optional for creating a local partition underneath the cgroup root. The example should still work without setting cpuset.cpus.exclusive.

Cheers,
Longman

#12 lore raw
Quoted 8 lines · Waiman Long
Quoted 4 lines

while looking at this again, shouldn't you also do
echo 7 > cpuset.cpus.exclusive

to ensure the CPU isn't used somewhere else?

For backport compatibility, the use of cpuset.cpus.exclusive is optional for creating a local partition underneath the cgroup root. The example should still work without setting cpuset.cpus.exclusive.

I would have to double check but I think only after the cpuset.cpus.exclusive the CPU vanished from the cpumask of my current task.

Quoted 2 lines

Cheers,
Longman

Sebastian

#13 lore raw
Quoted 6 lines · To Waiman Long
Quoted 5 lines · Waiman Long
Quoted 4 lines

while looking at this again, shouldn't you also do
echo 7 > cpuset.cpus.exclusive

to ensure the CPU isn't used somewhere else?

Quoted 7 lines
Quoted 3 lines

For backport compatibility, the use of cpuset.cpus.exclusive is optional for creating a local partition underneath the cgroup root. The example should still work without setting cpuset.cpus.exclusive.

I would have to double check but I think only after the cpuset.cpus.exclusive the CPU vanished from the cpumask of my current task.

Let me take that back. After setting cpuset.cpus.partition the CPU mask of the remaining tasks gets updated. All good.

Quoted 3 lines
Quoted 2 lines

Cheers,
Longman

Sebastian