Skip to contentSkip to conversation

v2: add NUMA support to QEMU

PATCH In review Andre Przywara 2 participants 2 messages Dec 16, 2008

lore
Share & tools
lore
#0 lore raw

cover v2: add NUMA support to QEMU

Hi,

this is a reworked and repackaged version of the first NUMA try from last week. I integrated Anthony's comments. Documentation will follow.

I refrained from splitting the host pinning and guest NUMA topology
description on the command line, since it will not make the code easier
and I don't see the point of intentionally restricting the cmd line
interface.
If you want to separate it, write "-numa 2 -numa pin:1;2" (the second argument is missing the leading node number).

-------
The following patches add support for NUMA (Non-Uniform Memory Access) guests in QEMU. Since QEMU lacks real SMP support, this is mostly for debugging or research, but will improve performance in KVM (with additional KVM-only patches).

1/8: The user specifies a NUMA topology on the command line The command line syntax is:
-numa
<nrnodes>[,mem:size1[;size2..]][,cpu:cpu1[;cpu2..]]
Beside the number of nodes all other arguments are optional, so possible command lines are:
-numa 2
/* inject two NUMA nodes into the guest, distribute guest CPUs and memory equally over the two nodes, don't pin the memory to host nodes*/ -smp 4 -numa 3,mem:1536M;768M;768M,cpu:0-1;2;3
/* inject three nodes, distribute the memory and cpu as described:
node0: 1536M, CPUs 0,1; node1: 768M, CPU 2; node2: 768M, CPU 3 */

Please note that ; and * must be escaped on the shell.

2/8: push NUMA topology info to the BIOS using the QEMU firmware configuration interface. Defines three additional "channels"(?) to transport the information.

3/8: add 'info numa' monitor command to show the chosen NUMA topology. The output is based on Linux' numactl --hardware.

4/8: extend parser to parse host affinity option
This adds the "[,pin:node1[;node2]]" option to the numa command line -numa 2,pin:2;*
/* inject two nodes, allocate the memory for the first node from the host node 2, the second node has no affinity (all host nodes) */ For now it is not recommended to use host pinning in pure QEMU.

5/8: check for existence of libnuma in configure
libnuma is a Linux library wrapping around the NUMA kernel interface. It is usually contained within the numactl package. Support for this is optional, it will only affect the host pinning code (next two patches)

6/8: if stated on the command line, set the affinity of the guest memory to certain host nodes. Requires libnuma.

7/8: add "-numa pin" monitor command. This allows to change the host affinity of guest nodes during the guest's runtime. Requires libnuma. The syntax is the same as on the command line, but the leading number is omitted:

Quoted 1 line

numa pin:0;2

(no number means unchanged, * means all nodes, like: numa pin:;*)
Note that this will only affect future guest memory touches, if the guest has already used every page, no change will occur.

8/8:
For the sake of completeness a patch to build the ACPI SRAT table in the BIOS. This contains some code that is already upstream in BOCHS-cvs (from Gleb Natapov), I hope that Anthony will sync the BIOS soon. I will send a reworked patch then. If not, I can port over Gleb's changes first and base my patch on top of it.

Regards,
Andre.

Signature · 9 lines
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 277-84917
----to satisfy European Law for business letters:
AMD Saxony Limited Liability Company & Co. KG,
Wilschdorfer Landstr. 101, 01109 Dresden, Germany
Register Court Dresden: HRA 4896, General Partner authorized
to represent: AMD Saxony LLC (Wilmington, Delaware, US)
General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy
#1 lore raw
Quoted 4 lines · Andre Przywara

1/8: The user specifies a NUMA topology on the command line The command line syntax is:
-numa <nrnodes>[,mem:size1[;size2..]][,cpu:cpu1[;cpu2..]]

qemu uses '=' where you use ':' (for example, -drive file=...)

Signature · 1 line
error compiling committee.c: too many arguments to function