Skip to content

xen: add core scheduling support

Juergen Gross xen-devel 6 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v6 Latest20 patches 4 repliesIn review
  2. v519 patches 13 repliesSuperseded
  3. v446 patches 24 repliesSuperseded
Earlier versions · 3
VersionPostedPatchesRepliesStatus
v347 patches113 repliesSuperseded
v248 patches77 repliesSuperseded
v160 patches70 repliesSuperseded

Changes in v6 from the cover letter

- only added new patch 20 (sent as v5 20/19 before)

Patches in v6 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/20xen/sched: add code to sync scheduling of all vcpus of a sched unitxen/arch/arm/domain.c xen/arch/x86/domain.c xen/common/schedule.c +4 more+294 -882 replies
2/20xen/sched: introduce unit_runnable_state()xen/common/domain.c xen/common/sched_arinc653.c xen/common/sched_credit.c +6 more+73 -300 replies
3/20xen/sched: add support for multiple vcpus per sched unit where missingxen/common/domain.c xen/common/schedule.c xen/include/xen/sched-if.h+24 -60 replies
4/20xen/sched: modify cpupool_domain_cpumask() to be an unit maskxen/common/cpupool.c xen/common/domain.c xen/common/domctl.c +7 more+55 -420 replies
5/20xen/sched: support allocating multiple vcpus into one sched unitxen/common/schedule.c+76 -210 replies
6/20xen/sched: add a percpu resource indexxen/common/schedule.c+9 -20 replies
7/20xen/sched: add fall back to idle vcpu when scheduling unitxen/arch/x86/domain.c xen/common/schedule.c xen/include/asm-arm/current.h +4 more+187 -630 replies
8/20xen/sched: make vcpu_wake() and vcpu_sleep() core scheduling awarexen/common/schedule.c xen/include/xen/sched-if.h xen/include/xen/sched.h+136 -90 replies
9/20xen/sched: move per-cpu variable scheduler to struct sched_resourcexen/common/sched_credit.c xen/common/sched_credit2.c xen/common/schedule.c +1 more+21 -170 replies
10/20xen/sched: move per-cpu variable cpupool to struct sched_resourcexen/common/cpupool.c xen/common/sched_credit.c xen/common/sched_rt.c +2 more+8 -100 replies
11/20xen/sched: reject switching smt on/off with core scheduling activexen/arch/x86/sysctl.c xen/common/schedule.c xen/include/xen/sched.h+7 -00 replies
12/20xen/sched: prepare per-cpupool scheduling granularityxen/common/cpupool.c xen/common/schedule.c xen/include/xen/sched-if.h+36 -110 replies
13/20xen/sched: split schedule_cpu_switch()xen/common/cpupool.c xen/common/schedule.c xen/include/xen/sched.h+78 -620 replies
14/20xen/sched: protect scheduling resource via rcuxen/common/cpupool.c xen/common/schedule.c xen/include/xen/sched-if.h+178 -200 replies
15/20xen/sched: support multiple cpus per scheduling resourcexen/common/cpupool.c xen/common/schedule.c xen/include/xen/sched-if.h+17 -60 replies
16/20xen/sched: support differing granularity in schedule_cpu_[add/rm]()xen/common/cpupool.c xen/common/schedule.c+204 -400 replies
17/20xen/sched: support core scheduling for moving cpus to/from cpupoolsxen/common/cpupool.c xen/common/schedule.c xen/include/xen/sched-if.h+76 -280 replies
18/20xen/sched: disable scheduling when entering ACPI deep sleep statesxen/arch/x86/acpi/power.c xen/common/schedule.c xen/include/xen/sched.h+35 -20 replies
19/20xen/sched: add scheduling granularity enumxen/arch/x86/Kconfig xen/common/Kconfig xen/common/cpupool.c+84 -00 replies
20/20docs: add "sched-gran" boot parameter documentationdocs/misc/xen-command-line.pandoc+28 -00 replies

Reviews trailers per patch

19 of 20 patches have a Reviewed-by or Acked-by.

PatchDario FaggioliJan BeulichJulien Grall
1/20R-b A-b
2/20R-b
3/20R-b A-b
4/20R-b
5/20R-b
6/20R-b
7/20R-b A-b A-b
8/20R-b
9/20R-b R-b
10/20R-b R-b
11/20A-b A-b
12/20R-b
13/20R-b
14/20R-b
15/20R-b
16/20R-b
17/20R-b
18/20R-b A-b
19/20R-b
20/20

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.

Read the v6 thread

Cover letter [Xen-devel] [PATCH v6 00/20] xen: add core scheduling support

Add support for core- and socket-scheduling in the Xen hypervisor.

Via boot parameter sched-gran=core (or sched-gran=socket) it is possible to change the scheduling granularity from cpu (the default) to either whole cores or even sockets.

All logical cpus (threads) of the core or socket are always scheduled together. This means that on a core always vcpus of the same domain will be active, and those vcpus will always be scheduled at the same time.

This is achieved by switching the scheduler to no longer see vcpus as the primary object to schedule, but "schedule units". Each schedule unit consists of as many vcpus as each core has threads on the current system. The vcpu->unit relation is fixed.

I have done some very basic performance testing: on a 4 cpu system (2 cores with 2 threads each) I did a "make -j 4" for building the Xen hypervisor. With This test has been run on dom0, once with no other guest active and once with another guest with 4 vcpus running the same test. The results are (always elapsed time, system time, user time):

sched-gran=cpu,    no other guest: 116.10 177.65 207.84
sched-gran=core,   no other guest: 114.04 175.47 207.45
sched-gran=cpu,    other guest:    202.30 334.21 384.63
sched-gran=core,   other guest:    207.24 293.04 371.37

The performance tests have been performed with credit2, the other schedulers are tested only briefly to be able to create a domain in a cpupool.

Cpupools have been moderately tested (cpu add/remove, create, destroy, move domain).

Cpu on-/offlining has been moderately tested, too.

Changes in V6:
- only added new patch 20 (sent as v5 20/19 before)

Changes in V5:
- dropped patches 1-27 as they already went in
- added comments in 2 patches

Changes in V4:
- comments addressed
- former patch 36 merged into patch 32

Changes in V3:
- comments addressed
- former patch 26 carved out and sent separately
- some minor bugs fixed

Changes in V2:
- comments addressed
- some patches merged into one
- idle scheduler related patches split off to own series
- some patches are already applied
- some bugs fixed (e.g. crashes when powering off)

Changes in V1:
- cpupools are working now
- cpu on-/offlining working now
- all schedulers working now
- renamed "items" to "units"
- introduction of "idle scheduler"
- several new patches (see individual patches, mostly splits of former patches or cpupool and cpu on-/offlining support)
- all review comments addressed
- some minor changes (see individual patches)

Changes in RFC V2:
- ARM is building now
- HVM domains are working now
- idling will always be done with idle_vcpu active
- other small changes see individual patches

Juergen Gross (20):
xen/sched: add code to sync scheduling of all vcpus of a sched unit xen/sched: introduce unit_runnable_state()
xen/sched: add support for multiple vcpus per sched unit where missing xen/sched: modify cpupool_domain_cpumask() to be an unit mask xen/sched: support allocating multiple vcpus into one sched unit xen/sched: add a percpu resource index
xen/sched: add fall back to idle vcpu when scheduling unit xen/sched: make vcpu_wake() and vcpu_sleep() core scheduling aware xen/sched: move per-cpu variable scheduler to struct sched_resource xen/sched: move per-cpu variable cpupool to struct sched_resource xen/sched: reject switching smt on/off with core scheduling active xen/sched: prepare per-cpupool scheduling granularity xen/sched: split schedule_cpu_switch()
xen/sched: protect scheduling resource via rcu
xen/sched: support multiple cpus per scheduling resource xen/sched: support differing granularity in schedule_cpu_[add/rm]() xen/sched: support core scheduling for moving cpus to/from cpupools xen/sched: disable scheduling when entering ACPI deep sleep states xen/sched: add scheduling granularity enum
docs: add "sched-gran" boot parameter documentation

 docs/misc/xen-command-line.pandoc |   28 +
 xen/arch/arm/domain.c             |    2 +-
 xen/arch/x86/Kconfig              |    1 +
 xen/arch/x86/acpi/power.c         |    4 +
 xen/arch/x86/domain.c             |   26 +-
 xen/arch/x86/sysctl.c             |    5 +
 xen/common/Kconfig                |    3 +
 xen/common/cpupool.c              |  232 +++++--
 xen/common/domain.c               |    8 +-
 xen/common/domctl.c               |    2 +-
 xen/common/sched_arinc653.c       |    4 +-
 xen/common/sched_credit.c         |   73 ++-
 xen/common/sched_credit2.c        |   32 +-
 xen/common/sched_null.c           |   11 +-
 xen/common/sched_rt.c             |   18 +-
 xen/common/schedule.c             | 1300 ++++++++++++++++++++++++++++++-------
 xen/common/softirq.c              |    6 +-
 xen/include/asm-arm/current.h     |    1 +
 xen/include/asm-x86/current.h     |   19 +-
 xen/include/asm-x86/smp.h         |    7 +
 xen/include/xen/sched-if.h        |   86 ++-
 xen/include/xen/sched.h           |   26 +-
 xen/include/xen/softirq.h         |    1 +
 23 files changed, 1532 insertions(+), 363 deletions(-)
Signature · 7 lines
2.16.4


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel