Skip to content

sched, steal_governor: Introduce preferred CPUs and steal-driven vCPU backoff

Shrikanth Hegde linux-doc lkml llvm sashiko-reviews virtualization 13 versions – Applied

Status from this message: The following commit has been merged into the sched/core branch of tip:

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v14 Latest13 patches 11 repliesApplied
  2. v1313 patches 15 repliesSuperseded
  3. v1213 patches 5 repliesSuperseded
Earlier versions · 10
VersionPostedPatchesRepliesStatus
v1112 patches10 repliesSuperseded
v1012 patches14 repliesSuperseded
v911 patches18 repliesSuperseded
v811 patches6 repliesSuperseded
v712 patches24 repliesSuperseded
v623 patches38 repliesSuperseded
v524 patches24 repliesApplied
v420 patches30 repliesSuperseded
v320 patches32 repliesSuperseded
v217 patches13 repliesSuperseded

Changes in v14 from the cover letter

- Make sched_non_preferred_cpu_push_stop look similar to __balance_push_cpu_stop. (Peter Zijlstra)
- Rename task_can_sched_on_preferred to task_can_migrate_to_preferred to make it read more natural. (Peter Zijlstra)
- Clarify that the new migration statistic is per-task and does not require a schedstat version change.
- Updated changelogs and Collected tags.

Patches in v14 Applied

#Subject and filesSizeRepliesApplied
1/13sched/cputime: Add kcpustat_field_total helperarch/s390/kernel/hiperdispatch.c fs/proc/uptime.c include/linux/kernel_stat.h+15 -120 replies
2/13cpumask: Introduce cpumask_intersects_andinclude/linux/bitmap.h include/linux/cpumask.h lib/bitmap.c+49 -00 replies
3/13sched/docs: Document cpu_preferred_mask and Preferred CPU conceptDocumentation/scheduler/index.rst Documentation/scheduler/sched-paravirt.rst+68 -00 replies
4/13cpumask: Introduce cpu_preferred_maskinclude/linux/cpumask.h kernel/Kconfig.preempt kernel/cpu.c +1 more+39 -06 replies
5/13sysfs: Add preferred CPU fileDocumentation/ABI/testing/sysfs-devices-system-cpu drivers/base/cpu.c+26 -00 replies
6/13sched/core: Try to use a preferred CPU in is_cpu_allowedkernel/sched/core.c+28 -21 reply
7/13sched/fair: Load balance only among preferred CPUskernel/sched/fair.c+3 -50 replies
8/13sched/core: Push current task from non preferred CPUkernel/sched/core.c kernel/sched/sched.h+92 -00 replies
9/13sched/debug: Add migration stats due to non preferred CPUsinclude/linux/sched.h kernel/sched/core.c kernel/sched/debug.c+9 -23 replies
10/13virt: Introduce steal governor driverDocumentation/driver-api/index.rst Documentation/driver-api/steal-governor.rst MAINTAINERS +1 more+239 -00 replies
11/13virt/steal_governor: Add control knobs for handling steal valuesdrivers/virt/steal_governor.c+71 -20 replies
12/13virt/steal_governor: Implement steal_governor policy loopdrivers/virt/steal_governor.c+149 -01 reply
13/13virt/steal_governor: Enable the driverdrivers/virt/Kconfig drivers/virt/Makefile+18 -00 replies

Reviews trailers per patch

3 of 13 patches have a Reviewed-by or Acked-by.

PatchFrederic WeisbeckerYury NorovMete Durlu
1/13A-b R-b R-b
2/13R-b
3/13
4/13
5/13
6/13
7/13R-b
8/13
9/13
10/13
11/13
12/13
13/13

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.

Read the v14 thread

Cover letter [PATCH v14 00/13] sched, steal_governor: Introduce preferred CPUs and steal-driven vCPU backoff

tl;dr

This patch series represents the result of multiple iterations, redesigns and community feedback. What started as an arch-specific RFC has evolved into a scheduler mechanism paired with a virtualization driver.

This feature is intended for workloads where vCPU preemption has costs beyond the lost CPU time, such as lock-holder preemption, critical sections and TLB/cache misses. Database workloads, for example, contain a mixture of OLTP/OLAP activity and are prone to these costs. Reducing vCPU preemption in these cases can help improve overall system throughput. It may regress slightly for pure CPU-time workloads.

I believe the series has now converged and can be considered for merging. The implementation has been deliberately kept simple. It performs well for real-life workloads on PowerPC, and earlier KVM testing also showed improvements on s390 and x86.

The following work is planned post the merge:
- Design for arch specific interface which could help to bring out additional performance gains.
- A test framework for the feature

Feature has been kept as generic solution for all arch which support paravirtualization and steal time accounting. If a need arises, arch specific gating could be introduced.

Special thanks to Yury Norov for the rigorous reviews that greatly improved the series and to everyone who have provided their review comments so far. (including sashiko) Really appreciated! _/\_

Rest of cover-letter is very similar since v8. Version history has been updated.

Detailed Background and Problem Statement
=========================================

As hardware scales, the density of physical CPUs (pCPUs) per server is increasing across many architectures. On these massive systems, deploying a single bare-metal OS for general workloads becomes increasingly difficult to manage if not impossible. The natural shift is to deploy Virtual Machines(VMs). For example, on IBM PowerPC architecture customers frequently deploy Shared Processor LPARs (SPLPARs) to maximize hardware ROI.

Typical enterprise workloads are combination of bursty and long running; their average CPU utilization is low, but they require high core counts during peak transactions. To accommodate this, customers often use CPU overcommit strategies i.e. configuring VMs with a large number of virtual CPUs (vCPUs) while backing them with a smaller, shared pool of physical CPUs (pCPUs). This achieves a high server consolidation and excellent cost efficiency.

However, when multiple such VMs have high utilization simultaneously, the shared pCPU pool becomes contended. The hypervisor is forced to preempt one vCPU to run another to maintain fairness. It maybe schedule vCPU of same VM or different VM. If a vCPU is preempted while holding a lock or irq disabled section etc, overall forward progress collapses. There are some mitigation strategies such as yielding the vCPU to lock-holder, but they don't cover all the cases. In addition there are hidden costs such as cache, tlb misses, cost of vCPU preemption, host scheduling overheads etc.

Under heavy contention, an effective mitigation strategy for workloads sensitive to vCPU preemption is for guests/VMs to voluntarily fold their workloads onto a smaller subset of vCPUs. By demanding fewer pCPUs, the VMs reduce overall host contention, which decreases vCPU preemption and improves total throughput for the system.

Limitations of Existing Approaches
==================================

CPU Hotplug, Isolated cpusets, cpuset:
- This is a heavy and administrative operation that requires topology rebuild. Crucially, it breaks userspace CPU affinities.

Explicit task affinity:
- Very difficult to manage for the users, if not impossible.

We need a fast, co-operative backoff mechanism inside the kernel that can dynamically react to contention without violating user/task affinity contracts. Since reacting to the contention is agnostic to the user it cannot violate user affinity contracts.

When there is high contention, fold the workload and use limited vCPUs and when there is no contention, use all the vCPUs again. This natural expansion/contraction gives the better performance to the users based on the underlying contention.

Proposed Architecture
=====================

Current design is built on basis that contention is effectively quantified by steal time as seen in guest kernel.
Steal time is already a well established construct today in para-virtualization world. All major archs support this feature. It is indication of the contention of physical CPU. It scales according to the amount of contention. Today it is used by administrative users for changing the VM configurations. During high contention the steal time shows up in each guest based on its configuration. The proposed solution works well when all VMs honor the hint and work in co-operative manner. Note there is still no inter-guest communication to achieve this co-operation. Read the section on best practises on how to get the best out of this solution.

This series introduces a dynamic vCPU backoff mechanism. It is separated into a core scheduler mechanism and a loadable virtualization policy module.

Layer A: The Scheduler Mechanism (preferred CPUs)
=================================================

Series introduces a new CPU state called preferred. It indicates that vCPU can be safely used and using that vCPU won't increase contention for underlying physical CPUs. This state info is made available via cpu_preferred_mask, which is strictly maintained as a subset of cpu_active_mask.

The scheduler uses this mask as a hint to fold workloads onto preferred CPUs using a few mechanisms.

1. Wakeup: is_cpu_allowed() checks if CPU is preferred. If not calls select_fallback_rq, which selects a preferred CPU if tasks's affinity permits.

2. The Tick (Push): During sched_tick(), if the current CPU is non-preferred, the scheduler actively pushes the running task onto a preferred CPU using a stopper thread.

3. Load Balance: sched_balance_rq restricts its domain span to cpu_preferred_mask, preventing tasks from being pulled toward non-preferred CPUs.

Design Constraint: The scheduler strictly respects user affinities. If a task is pinned exclusively to non-preferred CPUs, it will remain there. The kernel will not break user/task affinity contracts.

Layer B: The Policy Engine (virt/steal_governor)
================================================

The core scheduler should not dictate virtualization policy. Therefore, the policy is isolated into a new driver: steal_governor. (Can be selected by CONFIG_STEAL_GOVERNOR)
This module latches onto that concept that contention is quantified by steal time. It periodically samples the steal time values across the system and depending on high/low steal values, takes appropriate action.

When it sees high steal times, i.e. steal time exceeds high_threshold (default 5%), driver reduces the preferred CPUs by 1 core. When it sees Low Steal Times, i.e. steal time drops below low_threshold (default 2%), driver increases the preferred CPUs by 1 core.

This creates a dynamic, self-maintained stepwise loop. The guest automatically shrinks its pCPU footprint when the host is saturated, and expands it when the noise clears while requiring zero cross-VM communication.

Policy Design Constraints:
- Ensure at least one core is kept as preferred.
- Ensure preferred is always subset of active.

Best Practises
==============
1. Ensure all the VM run kernel which has the patches.

2. Keep CONFIG_STEAL_GOVERNOR=m. Build it as module, but don't load it by default. When the administrative user enables it in one VM, he/she will likely enable it in all VMs. Also module parameters can only be changed at module load. Having it as module also allows one to disable it to remove additional overhead it brings.

3. Keep the interval_ms=500 to 5000. I.e. between 500ms to 5 second. Though parameters allows slightly higher range.

4. Fine tune low and high threshold depending on your platform for best results. Even where is no contention, very small steal values might show up. So it might be better to keep low threshold higher than 0.

Baseline and Revision History
==============================

tip/sched/core at commit:
'627ea30aca3b ("sched/wait: Clarify WF_SYNC wakeup semantics")'

For the talk on the problem and discussion on this issue, one can also refer to the OSPM26 talk[1].

[1]: https://youtu.be/adxUKFPlOp0
[2]: https://www.ibm.com/support/pages/ibm-power-virtualization-best-practices-guide
[3]: https://www.ibm.com/docs/en/linux-on-systems?topic=bad-daytrader

v13->v14:
- Make sched_non_preferred_cpu_push_stop look similar to __balance_push_cpu_stop. (Peter Zijlstra)
- Rename task_can_sched_on_preferred to task_can_migrate_to_preferred to make it read more natural. (Peter Zijlstra)
- Clarify that the new migration statistic is per-task and does not require a schedstat version change.
- Updated changelogs and Collected tags.

v12->v13:
- Rename push_task_work_done to npc_push_work_pending. (Yury Norov)
- Move it around steal_* fields in rq. (Yury Norov)
It doesn't increase the struct rq's number of cachelines for both 128 byte and 64 byte cachelines.
- Put PREFERRED_CPU guard for npc_push_work_pending (Yury Norov)
- Update comments around rq lock's acquisition post select_fallback_rq (Yury Norov)

v11->v12:
- Simplify the ARM64's 32 bit task case where p->cpus_ptr may contain architecturally impossible CPUs. (Dietmar Eggemann, Vincent Guittot)
- Introduce cpumask_intersects_and for that simplification (Yury Norov)
- Keep the feature without any arch specific kconfig gating for now. (Vincent Guittot)
If a need arises, it can be brought in

v10->v11:
- Return active CPUs instead of possible CPUs for steal ratio denominator. This helps to avoid dilution of thresholds.
(Yury Norov, Ionut Nechita)
- Add XEN dom0 blocking check. (Ionut Nechita)
- Update documentation/changelogs.
- Multi arch framework has been deferred. Will be designed after this series gets merged upstream. It would depend on enhancement requests.
- Collected tags for patch 1
- Fix possible crash for 32-bit tasks on arm64 (sashiko)
- Fix possible race for task affinity due to abort in __migrate_task (sashiko)
- Fix unused variable for patch 1 (sashiko)
- Disable interrupt when taking rq lock in a small window (sashiko)
- Queue stopper iff migration is enabled. (sashiko)
- Document workload characteristics and the possible regression for pure CPU-time workloads.

v9->v10:
- Introduce kcpustat_field_total helper. (Yury Norov)
- Always do the design checks. This helps to avoid placing design constraints in core hotplug code.
- Remove cpu_preferred check in idle balancing. This helps to naturally take care update of nohz.next_balance.
- find_new_ilb changes are deferred as it isn't applicable for most common use cases.
- Move scheduler documentation to sched-paravirt.rst. (Yury Norov)
- Add details of limitation of default values in documentation. (Yury Norov)
- Remove task_can_sched_on_preferred out of sched.h (Mete Durlu)
- Updated suggested-by tags for few patches. (I know i should have done it earlier, sorry about that)
- Minor polish of all changelogs.

v8->v9:
- Move to simpler layout. Everything in drivers/virt/steal_governor.c (Yury Norov)
- Move design checks into a helper function (Yury Norov)
- Comments update (Yury Norov)
- Requeue work without further checks when steal ratio is within the low/high threshold window. (Yury Norov)
- Renamed sg_core_ctx to sg_ctx.

v7->v8:
- Rename to STEAL_GOVERNOR from STEAL_MONITOR.
- Remove additional defaults.c and move it to core.c (Yury Norov)
- Remove SM_DIR gating for direction control. (Yury Norov)
- Enforce design constraint and restore the state if not met (Yury Norov)
- Drop nohz_full tick enable patch.
- Move Kconfig patch as the last patch for enablement. (Yury Norov)
- Use disable_delayed_work_sync to avoid race condition during module unload. (Yury Norov)
- Add same kconfig dependency and fail to compile the driver (Yury Norov)
- Make low < high comparison during module init instead as they are dependent parameters (Sashiko)
- Update sysfs file helper section (Yury Norov)
- Make preferred sysfs file available only with CONFIG_PREFERRED_CPU=y (Yury Norov)
- A few documentation and comments fixes. (Randy Dunlap)
- Fix possible race in sched_push_current_non_preferred_cpu (Yury Norov)
- Move is_migration_disabled check just before actual migration.
- Make 100ms as minimal interval_ms from 10ms.
- Make helper functions static and remove from header file as there are no other callers.
- Collapse helper functions and periodic work into one patch.

Short summary on previous versions:
v6->v7:
- Consolidate new driver code to 4-5 patches.
- defer the arch specific interface.
- Use possible CPUs instead of active for steal value calculations.
- Simplify is_cpu_allowed.
- Make module parameters fixed at module load
- Define CONFIG_STEAL_MONITOR and Make it select CONFIG_PREFERRED_CPU

v5->v6:
- Drop the optimization of caching the preferred state in select_fallback_rq
- Drop wakeup patch

v4->v5:
- Move the computation of steal time and decide on preferred CPU state to a driver. i.e new driver called STEAL_MONITOR

v3->v4:
- Make preferred subset of active instead of online.
- Dropped RT patch and Defer sched_ext. Support only FAIR class.

v2->v3:
- Introduce a new config CONFIG_PREFERRED_CPU

v1->v2:
- A new name - Preferred CPUs and cpu_preferred_mask
- Arch independent code. Everything happens in scheduler.
- Steal time computation is gated with sched feature STEAL_MONITOR

RFC v3-> RFC v4:
- Introduced computation of steal time in arch/powerpc.

RFC PATCH v1:
- push task mechanism.
- No steal time computation. Manual sysfs hint for preferred CPUs

v1: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/#t
v3: https://lore.kernel.org/all/[email protected]/#r
v4: https://lore.kernel.org/all/[email protected]/#t
v5: https://lore.kernel.org/all/[email protected]/
v6: https://lore.kernel.org/all/[email protected]/#t
v7: https://lore.kernel.org/all/[email protected]/
v8: https://lore.kernel.org/all/[email protected]/
v9: https://lore.kernel.org/all/[email protected]/
v10: https://lore.kernel.org/all/[email protected]/
v11: https://lore.kernel.org/all/[email protected]/
v12: https://lore.kernel.org/all/[email protected]/
v13: https://lore.kernel.org/all/[email protected]/
Even earlier version:
https://lore.kernel.org/all/[email protected]/

========================================
Performance Numbers (powerpc, x86, s390)
========================================

PowerPC:
===================
VM1: 60VP/30EC and VM2: 30VP/20EC
Shared physical CPU pool size: 50 Cores. Each core is SMT8.
(VP - Virtual Core, EC - Entitles Core) - PowerVM terminologies of SPLPAR[2]

Default parameter values: 1000ms, 200 low threshold, 500 high threshold Both the VMs are running the same workload. Total throughput/time of VM1+VM2 is being mentioned in all cases.

Hackbench
baseline steal_governor steal_governor
disabled enabled
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

10 groups        5.20   |    5.40 (-3.85%)  |     4.65 (+10.58%)
20 groups       11.39   |   12.01 (-5.44%)  |     7.09 (+37.75%)
40 groups       20.32   |   19.80 (+2.56%)  |    11.31 (+44.34%)
10 groups(-p)    2.37   |    2.26 (+4.64%)  |     2.06 (+13.08%)
20 groups(-p)    3.34   |    3.28 (+1.80%)  |     3.20 (+4.19%)
40 groups(-p)    4.46   |    4.83 (-8.30%)  |     4.26 (+4.48%)

Remarks: Net improvement with steal_governor specially high load points.

schbench ( -L -n 0 -r 30 -s 0)
baseline steal_governor steal_governor
disabled enabled
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

-m 1 -t 128     2475162 |    2621246 (+5.90%)  |      2527299 (+2.11%)
-m 1 -t 256     1467350 |    1470032 (+0.18%)  |      1492372 (+1.71%)
-m 1 -t 512     1408813 |    1454687 (+3.26%)  |      1437605 (+2.04%)

Remarks: Effectively means no-improvements or regressions

kernbench baseline steal_governor steal_governor
(elapsed time) disabled enabled
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
-j nr_cpus 231 | 235 (-1.7%) | 199 (+14%)
Remarks: Net improvement in elapsed time.

Daytrader - A real life work which is a proxy for trading based on db2[3]
baseline steal_governor steal_governor
disabled enabled
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Load@30%	1x	|	0.96x	|	 1.53x			
Load@60%	1x	|	0.94x	|	 1.41x

Remarks: Very good improvement seen at different load points.

phoronix/memcached:
Cloned from github.com/phoronix-test-suite/phoronix-test-suite.git GET 1:1000 means 1 writer and 1000 readers.
baseline steal_governor steal_governor
disabled enabled
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
GET 1:1000 1x 0.99x 1.2x
Remarks: 20% improvement with read heavy variant.
Other variants such as GET 1:1 showed minor regressions. While inspecting it, it was found that steal time reported by PowerVM was not as expected even when only 1 VM was running. This is being looked into by PowerVM hypervisor team. So added a section in limitation that, this feature expects steal time values being accurate for right policy decisions.

When there is no steal time (such as dedicated LPAR) throughput was same with steal_governor enabled/disabled which indicates minimal overhead of steal_governor.

=======================================================================

Data from x86,s390 KVM which Ilya Leoshkevich carried out during OSPM26 time. *This was based on v2*. The core idea remains the same, although the implementation has changed and some overhead has been removed since then.
Note: Other variations of the benchmark shows no observable difference.

x86:
====
cascade-lake: 32 threads = 16 cores
Benchmark #VMs #CPUs/VM ΔRPS (%std)
===============================================

hackbench         8          16  90.73% ± 9.97%
hackbench         4          24  52.67% ± 7.43%
hackbench         4          16  37.96% ± 11.19%
hackbench         4          32  37.82% ± 4.38%
hackbench        12           8  36.90% ± 4.74%
hackbench         8           8  35.30% ± 3.61%
pgbench          16           4  31.77% ± 2.44%
hackbench         2          24  25.85% ± 8.63%
hackbench        16           8  24.87% ± 3.46%
pgbench          16           8  21.83% ± 2.20%
pgbench          12           8  21.35% ± 2.15%
pgbench           8           8  18.46% ± 1.01%
hackbench         2          32  15.56% ± 4.53%
pgbench          12           4  14.28% ± 2.04%
hackbench        16           4  14.07% ± 2.90%
hackbench        12           4  9.60% ± 3.49%
[...]
pgbench           4           8  -1.16% ± 3.60%
hackbench         4           4  -1.80% ± 9.55%
sysbench         12           4  -2.19% ± 0.78%
pgbench           4          24  -2.43% ± 4.38%
pgbench           4          32  -3.21% ± 0.79%
sysbench         16           4  -3.22% ± 1.09%

S390:
=====
z16: 16 threads = 8 cores (SMT-2)
Benchmark #VMs #CPUs/VM ΔRPS (std%)
===============================================

pgbench           2           8  73.50% ± 35.91%
pgbench          16           4  61.30% ± 4.09%
hackbench        16           4  54.11% ± 4.38%
hackbench        12           4  36.34% ± 4.63%
pgbench          12           4  34.83% ± 2.57%
hackbench         8           4  29.75% ± 5.86%
hackbench         8           8  25.98% ± 5.09%
pgbench           2           4  23.31% ± 33.44%
pgbench           2          16  19.95% ± 17.12%
hackbench         4           8  19.43% ± 9.33%
pgbench           8           4  19.32% ± 4.50%
[...]
schbench          8           8  -0.79% ± 0.33%
sysbench          8           8  -0.81% ± 0.39%
hackbench         4          16  -1.11% ± 5.82%
sysbench          8           4  -1.62% ± 0.49%
sysbench         16           4  -2.70% ± 0.58%
schbench         16           4  -2.73% ± 0.91%
sysbench         12           4  -2.91% ± 0.61%
hackbench         2          24  -4.99% ± 3.31%

Summary:
- Many improvement across archs specially with real life workloads.
- Overhead of steal_governor looks minimal when there is no steal time.
- Overhead when STEAL_GOVERNOR=n is negligible.
- Pure CPU-time workloads may see a small regression.

Testing and Validation
======================

Apart from performance, To ensure the robustness of the preferred CPU masking and push mechanisms, the following scenarios were tested:
- CPU Hotplug: bringing CPUs up/down change the preferred mask accordingly under no-contention and contention.
- Housekeeping cores: Verified with different combinations of nohz_full=<beginning, middle, end set of CPUs> to ensure that policy engine restricts to first housekeeping core in extreme cases.
- User Affinity: Confirmed that tasks explicitly pinned to non-preferred CPUs via taskset remain on their assigned CPUs.
- Affine Move: Confirmed the affinity move using "taskset -cp" happens on all combinations of non-preferred, preferred under contention.
- Affinity and hotplug: It works as expected. I.e affinity gets reset if all the CPUs of p->cpus_ptr go offline even if they are non-preferred CPUs.
- Extreme load and running threads: for example 4800 stress-ng threads on 480 CPU system and it still packs to preferred CPUs.

Known Limitations
=================

- The steal governor relies on accurate steal-time accounting. Incorrect steal-time values from the underlying architecture may lead to suboptimal policy decisions or performance regressions.

- If the workload is a pure CPU time based such as stress-ng --cpu=<> which doesn't suffer from vCPU preemption, it may not benefit from the steal governor.

Future work
===========
To keep this initial implementation clean and minimal, a few optimizations have been deferred:

- Design a selftest like framework to validate steal governor, at least for KVM setup where there is infra available to spin off guests.

- Multi arch interface where arch specific hints could be provided into the driver's helper functions.

- Push all tasks on rq: Currently, the stopper thread only pushes the current running task off a non-preferred CPU. Future optimizations may look into migrating all queued tasks on that runqueue.

- Sched Classes: This feature currently only works for the FAIR class. Real-time (RT) and sched_ext classes are deferred for now, as there is no need for it.

- NUMA Splicing: The steal_governor currently removes last active core based on CPU number. It does not yet do complex NUMA-aware splicing, expecting that CPUs are spread out uniformly across nodes in most cases.

Shrikanth Hegde (13):
sched/cputime: Add kcpustat_field_total helper
cpumask: Introduce cpumask_intersects_and
sched/docs: Document cpu_preferred_mask and Preferred CPU concept
cpumask: Introduce cpu_preferred_mask
sysfs: Add preferred CPU file
sched/core: Try to use a preferred CPU in is_cpu_allowed sched/fair: Load balance only among preferred CPUs sched/core: Push current task from non preferred CPU sched/debug: Add migration stats due to non preferred CPUs
virt: Introduce steal governor driver virt/steal_governor: Add control knobs for handling steal values virt/steal_governor: Implement steal_governor policy loop virt/steal_governor: Enable the driver

 .../ABI/testing/sysfs-devices-system-cpu      |  14 +
 Documentation/driver-api/index.rst            |   1 +
 Documentation/driver-api/steal-governor.rst   | 151 +++++++++
 Documentation/scheduler/index.rst             |   1 +
 Documentation/scheduler/sched-paravirt.rst    |  67 ++++
 MAINTAINERS                                   |   9 +
 arch/s390/kernel/hiperdispatch.c              |  10 +-
 drivers/base/cpu.c                            |  12 +
 drivers/virt/Kconfig                          |  17 +
 drivers/virt/Makefile                         |   1 +
 drivers/virt/steal_governor.c                 | 296 ++++++++++++++++++
 fs/proc/uptime.c                              |   6 +-
 include/linux/bitmap.h                        |  14 +
 include/linux/cpumask.h                       |  42 +++
 include/linux/kernel_stat.h                   |  11 +
 include/linux/sched.h                         |   1 +
 kernel/Kconfig.preempt                        |   4 +
 kernel/cpu.c                                  |   6 +
 kernel/sched/core.c                           | 123 +++++++-
 kernel/sched/debug.c                          |   1 +
 kernel/sched/fair.c                           |   8 +-
 kernel/sched/sched.h                          |   9 +
 lib/bitmap.c                                  |  17 +
 23 files changed, 802 insertions(+), 19 deletions(-)
 create mode 100644 Documentation/driver-api/steal-governor.rst
 create mode 100644 Documentation/scheduler/sched-paravirt.rst
 create mode 100644 drivers/virt/steal_governor.c
Signature · 1 line
2.52.0