Skip to content

RFC Add support for NUMA on ARM64

Shannon Zhao qemu-devel 3 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 RFC Latest3 patches 4 repliesIn review
  2. v2 RFC1 patch 0 repliesSuperseded
  3. v1 RFC1 patch 0 repliesSuperseded

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/3hw/arm/virt: Use memory_region_allocate_system_memory to allocate memoryhw/arm/virt.c+2 -30 replies
2/3hw/arm/virt: Don't add memory node in creat_fdthw/arm/virt.c+0 -22 replies
3/3hw/arm/boot: Generate memory dtb according to NUMA topologyhw/arm/boot.c+77 -32 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v3 yet.

Read the v3 thread

Cover letter [Qemu-devel] [RFC PATCH v3 0/3] Add support for NUMA on ARM64

Add support for NUMA on ARM64. Tested successfully running a guest Linux kernel with the following patch applied:

- arm64:numa: adding numa support for arm64 platforms.
http://article.gmane.org/gmane.linux.ports.arm.kernel/382179

Changes v2 ... v3:
* update to use NUMA node property arm,associativity.

Changes v1 ... v2:
Take into account Peter's comments:
* rename virt_memory_init to arm_generate_memory_dtb
* move arm_generate_memory_dtb to boot.c and make it a common func
* use a struct numa_map to generate numa dtb

Example qemu command line:

qemu-system-aarch64 \
    -enable-kvm -smp 4\
    -kernel Image \
    -m 512 -machine virt,kernel_irqchip=on \
    -initrd guestfs.cpio.gz \
    -cpu host -nographic \
    -numa node,mem=256M,cpus=0-1,nodeid=0 \
    -numa node,mem=256M,cpus=2-3,nodeid=1 \
    -append "console=ttyAMA0 root=/dev/ram"

Todo:
1)The NUMA nodes information in DT is not finalized yet, so this patch might need to be further modified to follow any changes in it.

2)Consider IO-NUMA as well

Please refer to the following url for NUMA DT node details:

- Documentation: arm64/arm: dt bindings for numa.
http://article.gmane.org/gmane.linux.ports.arm.kernel/382180

Example: 2 Node system each having 2 CPUs and a Memory

        arm,associativity-reference-points = <0x0 0x1>;

        memory@40000000 {
                reg = <0x0 0x40000000 0x0 0xfffffff>;
                device_type = "memory";
                arm,associativity = <0x0 0x0 0xffff>;
        };
        memory@50000000 {
                reg = <0x0 0x50000000 0x0 0xfffffff>;
                device_type = "memory";
                arm,associativity = <0x1 0x0 0xffff>;
        };

        cpus {
                #size-cells = <0x0>;
                #address-cells = <0x1>;

                cpu@0 {
                        reg = <0x0>;
                        enable-method = "psci";
                        compatible = "arm,arm-v8";
                        device_type = "cpu";
                        arm,associativity = <0x0 0x0 0x0>;
                };

                cpu@1 {
                        reg = <0x1>;
                        enable-method = "psci";
                        compatible = "arm,arm-v8";
                        device_type = "cpu";
                        arm,associativity = <0x0 0x0 0x1>;
                };

                cpu@2 {
                        reg = <0x2>;
                        enable-method = "psci";
                        compatible = "arm,arm-v8";
                        device_type = "cpu";
                        arm,associativity = <0x1 0x0 0x2>;
                };
                cpu@3 {
                        reg = <0x3>;
                        enable-method = "psci";
                        compatible = "arm,arm-v8";
                        device_type = "cpu";
                        arm,associativity = <0x1 0x0 0x3>;
                };
        }

- arm,associativity

The mapping is done using arm,associativity device property. this property needs to be present in every device node which needs to to be mapped to numa nodes.

arm,associativity property is set of 32-bit integers. representing the board id, socket id and core id.

ex:

	/* board 0, socket 0, core 0 */
	arm,associativity = <0 0 0x000>;

	/* board 1, socket 0, core 8 */
	arm,associativity = <1 0 0x08>;

- arm,associativity-reference-points

This property is a set of 32-bit integers, each representing an index into the arm,associativity nodes. The first integer is the most significant NUMA boundary and the following are progressively less significant boundaries. There can be more than one level of NUMA.

Ex:
arm,associativity-reference-points = <0 1>;
The board Id(index 0) used first to calculate the associativity (node distance), then follows the socket id(index 1).

arm,associativity-reference-points = <1 0>;
The socket Id(index 1) used first to calculate the associativity, then follows the board id(index 0).

arm,associativity-reference-points = <0>;
Only the board Id(index 0) used to calculate the associativity.

arm,associativity-reference-points = <1>;
Only socket Id(index 1) used to calculate the associativity.

Shannon Zhao (3):
hw/arm/virt: Use memory_region_allocate_system_memory to allocate
memory
hw/arm/virt: Don't add memory node in creat_fdt hw/arm/boot: Add arm_generate_memory_dtb to generate memory dtb
according to NUMA topology

 hw/arm/boot.c |   80 ++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 hw/arm/virt.c |    7 +---
 2 files changed, 79 insertions(+), 8 deletions(-)