xen: vnuma introduction for pv guest
Subsystems (4)
v2 cover xen: vnuma introduction for pv guest
Xen vnuma introduction.
The patchset introduces vnuma to paravirtualized Xen guests runnning as domU.
Xen subop hypercall is used to retreive vnuma topology information. Bases on the retreived topology from Xen, NUMA number of nodes, memory ranges, distance table and cpumask is being set. If initialization is incorrect, sets 'dummy' node and unsets nodemask. vNUMA topology is constructed by Xen toolstack.
Example of vnuma enabled pv domain dmesg:
[ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x00001000-0x0009ffff] [ 0.000000] node 0: [mem 0x00100000-0xffffffff] [ 0.000000] node 1: [mem 0x100000000-0x1ffffffff] [ 0.000000] node 2: [mem 0x200000000-0x2ffffffff] [ 0.000000] node 3: [mem 0x300000000-0x3ffffffff] [ 0.000000] On node 0 totalpages: 1048479 [ 0.000000] DMA zone: 56 pages used for memmap [ 0.000000] DMA zone: 21 pages reserved [ 0.000000] DMA zone: 3999 pages, LIFO batch:0 [ 0.000000] DMA32 zone: 14280 pages used for memmap [ 0.000000] DMA32 zone: 1044480 pages, LIFO batch:31 [ 0.000000] On node 1 totalpages: 1048576 [ 0.000000] Normal zone: 14336 pages used for memmap [ 0.000000] Normal zone: 1048576 pages, LIFO batch:31 [ 0.000000] On node 2 totalpages: 1048576 [ 0.000000] Normal zone: 14336 pages used for memmap [ 0.000000] Normal zone: 1048576 pages, LIFO batch:31 [ 0.000000] On node 3 totalpages: 1048576 [ 0.000000] Normal zone: 14336 pages used for memmap [ 0.000000] Normal zone: 1048576 pages, LIFO batch:31 [ 0.000000] SFI: Simple Firmware Interface v0.81 http://simplefirmware.org [ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs [ 0.000000] No local APIC present [ 0.000000] APIC: disable apic facility [ 0.000000] APIC: switched to apic NOOP [ 0.000000] nr_irqs_gsi: 16 [ 0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000fffff] [ 0.000000] e820: cannot find a gap in the 32bit address range [ 0.000000] e820: PCI devices with unassigned 32bit BARs may break! [ 0.000000] e820: [mem 0x400100000-0x4004fffff] available for PCI devices [ 0.000000] Booting paravirtualized kernel on Xen [ 0.000000] Xen version: 4.4-unstable (preserve-AD) [ 0.000000] setup_percpu: NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:4 nr_node_ids:4 [ 0.000000] PERCPU: Embedded 28 pages/cpu @ffff8800ffc00000 s85376 r8192 d21120 u2097152 [ 0.000000] pcpu-alloc: s85376 r8192 d21120 u2097152 alloc=1*2097152
numactl output:
root@heatpipe:~# numactl --hardware available: 4 nodes (0-3) node 0 cpus: 0 node 0 size: 4031 MB node 0 free: 3997 MB node 1 cpus: 1 node 1 size: 4039 MB node 1 free: 4022 MB node 2 cpus: 2 node 2 size: 4039 MB node 2 free: 4023 MB node 3 cpus: 3 node 3 size: 3975 MB node 3 free: 3963 MB node distances: node 0 1 2 3 0: 10 20 20 20 1: 20 10 20 20 2: 20 20 10 20 3: 20 20 20 10
Current patchset is available at https://git.gitorious.org/xenvnuma/linuxvnuma.git:v4
[email protected]:xenvnuma/linuxvnuma.git:v4
Xen patchset is available at: https://git.gitorious.org/xenvnuma/xenvnuma.git:v3
TODO
* dom0, pvh and hvm vnuma support; * multiple memory ranges per node support; * benchmarking;
Elena Ufimtseva (2):
xen: vnuma support for PV guests running as domU
Subject: [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
arch/x86/include/asm/xen/vnuma.h | 12 ++++ arch/x86/mm/numa.c | 3 + arch/x86/xen/Makefile | 2 +- arch/x86/xen/setup.c | 6 +- arch/x86/xen/vnuma.c | 127 ++++++++++++++++++++++++++++++++++++++ include/xen/interface/memory.h | 43 +++++++++++++ 6 files changed, 191 insertions(+), 2 deletions(-) create mode 100644 arch/x86/include/asm/xen/vnuma.h create mode 100644 arch/x86/xen/vnuma.c
Signature · 1 line
1.7.10.4
PATCH RESEND v2 1/2 xen: vnuma support for PV guests running as domU
Issues Xen hypercall subop XENMEM_get_vnumainfo and sets the NUMA topology, otherwise sets dummy NUMA node and prevents numa_init from calling other numa initializators as they dont work with pv guests.
--- arch/x86/include/asm/xen/vnuma.h | 12 ++++ arch/x86/mm/numa.c | 3 + arch/x86/xen/Makefile | 2 +- arch/x86/xen/vnuma.c | 127 ++++++++++++++++++++++++++++++++++++++ include/xen/interface/memory.h | 43 +++++++++++++ 5 files changed, 186 insertions(+), 1 deletion(-) create mode 100644 arch/x86/include/asm/xen/vnuma.h create mode 100644 arch/x86/xen/vnuma.c
arch/x86/include/asm/xen/vnuma.h new file +12 -0
@@ -0,0 +1,12 @@ +#ifndef _ASM_X86_VNUMA_H +#define _ASM_X86_VNUMA_H + +#ifdef CONFIG_XEN +bool xen_vnuma_supported (void ); +int xen_numa_init (void ); +#else +static inline bool xen_vnuma_supported (void ) {return false; }; +static inline int xen_numa_init (void ) {return -1 ; }; +#endif + +#endif /* _ASM_X86_VNUMA_H */
arch/x86/mm/numa.c +3 -0
@@ -17,6 +17,7 @@#include <asm/dma.h> #include <asm/acpi.h> #include <asm/amd_nb.h> +#include "asm/xen/vnuma.h" #include "numa_internal.h"
@@ -632,6 +633,8 @@ static int __init dummy_numa_init(void)void __init x86_numa_init (void ) {if (!numa_off) { +if (!numa_init (xen_numa_init)) +return ;#ifdef CONFIG_X86_NUMAQ if (!numa_init (numaq_numa_init))return ;
arch/x86/xen/Makefile +1 -1
@@ -13,7 +13,7 @@ CFLAGS_mmu.o := $(nostackp) obj-y := enlighten.o setup.o multicalls.o mmu.o irq.o \ time.o xen-asm.o xen-asm_$(BITS).o \ grant-table.o suspend.o platform-pci-unplug.o \ - p2m.o + p2m.o vnuma.o obj-$(CONFIG_EVENT_TRACING) += trace.o
arch/x86/xen/vnuma.c new file +127 -0
127 lines not on this page · raw message
include/xen/interface/memory.h +43 -0
47 lines not on this page · raw message
Signature · 1 line
1.7.10.4
PATCH RESEND v2 1/2 xen: vnuma support for PV guests running as domU
In reply to Elena UfimtsevaIssues Xen hypercall subop XENMEM_get_vnumainfo and sets the NUMA topology, otherwise sets dummy NUMA node and prevents numa_init from calling other numa initializators as they dont work with pv guests.
--- arch/x86/include/asm/xen/vnuma.h | 12 ++++ arch/x86/mm/numa.c | 3 + arch/x86/xen/Makefile | 2 +- arch/x86/xen/vnuma.c | 127 ++++++++++++++++++++++++++++++++++++++ include/xen/interface/memory.h | 43 +++++++++++++ 5 files changed, 186 insertions(+), 1 deletion(-) create mode 100644 arch/x86/include/asm/xen/vnuma.h create mode 100644 arch/x86/xen/vnuma.c
arch/x86/include/asm/xen/vnuma.h new file +12 -0
@@ -0,0 +1,12 @@ +#ifndef _ASM_X86_VNUMA_H +#define _ASM_X86_VNUMA_H + +#ifdef CONFIG_XEN +bool xen_vnuma_supported (void ); +int xen_numa_init (void ); +#else +static inline bool xen_vnuma_supported (void ) {return false; }; +static inline int xen_numa_init (void ) {return -1 ; }; +#endif + +#endif /* _ASM_X86_VNUMA_H */
arch/x86/mm/numa.c +3 -0
@@ -17,6 +17,7 @@#include <asm/dma.h> #include <asm/acpi.h> #include <asm/amd_nb.h> +#include "asm/xen/vnuma.h" #include "numa_internal.h"
@@ -632,6 +633,8 @@ static int __init dummy_numa_init(void)void __init x86_numa_init (void ) {if (!numa_off) { +if (!numa_init (xen_numa_init)) +return ;#ifdef CONFIG_X86_NUMAQ if (!numa_init (numaq_numa_init))return ;
arch/x86/xen/Makefile +1 -1
@@ -13,7 +13,7 @@ CFLAGS_mmu.o := $(nostackp) obj-y := enlighten.o setup.o multicalls.o mmu.o irq.o \ time.o xen-asm.o xen-asm_$(BITS).o \ grant-table.o suspend.o platform-pci-unplug.o \ - p2m.o + p2m.o vnuma.o obj-$(CONFIG_EVENT_TRACING) += trace.o
arch/x86/xen/vnuma.c new file +127 -0
127 lines not on this page · raw message
include/xen/interface/memory.h +43 -0
47 lines not on this page · raw message
Signature · 1 line
1.7.10.4
Re: [PATCH RESEND v2 1/2] xen: vnuma support for PV guests running as domU
Quoted 4 lines · Elena Ufimtseva
Issues Xen hypercall subop
XENMEM_get_vnumainfoand sets the NUMA topology, otherwise sets dummy NUMA node and preventsnuma_initfrom calling other numa initializators as they dont work with pv guests.
[...]
arch/x86/include/asm/xen/vnuma.h line 9
Quoted 12 lines
arch/x86/include/asm/xen/vnuma.h new file +9
-0@@ -0,0 +1,12 @@ +#ifndef _ASM_X86_VNUMA_H +#define _ASM_X86_VNUMA_H + +#ifdef CONFIG_XEN +bool xen_vnuma_supported (void ); +int xen_numa_init (void ); +#else +static inline bool xen_vnuma_supported (void ) {return false; }; +static inline int xen_numa_init (void ) {return -1 ; };
Return a valid -ve errno here. e.g., -ENOSYS.
Quoted 29 lines
arch/x86/include/asm/xen/vnuma.h new file +3
-0+#endif + +#endif /* _ASM_X86_VNUMA_H */ arch/x86/mm/numa.c +3
-0@@ -17,6 +17,7 @@#include <asm/dma.h> #include <asm/acpi.h> #include <asm/amd_nb.h> +#include "asm/xen/vnuma.h" #include "numa_internal.h" @@ -632,6 +633,8 @@ static int __init dummy_numa_init(void)void __init x86_numa_init (void ) {if (!numa_off) { +if (!numa_init (xen_numa_init)) +return ;#ifdef CONFIG_X86_NUMAQ if (!numa_init (numaq_numa_init))return ;arch/x86/xen/Makefile +0
-0
arch/x86/xen/Makefile CFLAGS_mmu.o := $(nostackp) · line 16
Quoted 5 lines
arch/x86/xen/Makefile +1
-1@@ -13,7 +13,7 @@ CFLAGS_mmu.o := $(nostackp) obj-y := enlighten.o setup.o multicalls.o mmu.o irq.o \ time.o xen-asm.o xen-asm_$(BITS).o \ grant-table.o suspend.o platform-pci-unplug.o \ - p2m.o + p2m.o vnuma.o
obj-$(CONFIG_NUMA) += vnuma.o
I commented on this already, please take more care to address all comments before reposting.
Quoted 8 lines
arch/x86/xen/vnuma.c new file +5
-0@@ -0,0 +1,127 @@ +#include <linux/err.h> +#include <linux/memblock.h> +#include <xen/interface/xen.h> +#include <xen/interface/memory.h> +#include <asm/xen/interface.h>
arch/x86/xen/vnuma.c line 15
Quoted 10 lines
arch/x86/xen/vnuma.c new file +10
-0+#include <asm/xen/hypercall.h> +#include <asm/xen/vnuma.h> + +#ifdef CONFIG_NUMA + +/* Checks if hypercall is supported */ +bool xen_vnuma_supported (void ) +{ +return HYPERVISOR_memory_op (XENMEM_get_vnuma_info,NULL ) + == -ENOSYS ? false : true;
return HYPERVISOR_memory_op() != -ENOSYS;
But as I note in the other email, I think this function isn't needed.
include/xen/interface/memory.h struct xen_remove_from_physmap { · line 270
Quoted 11 lines
include/xen/interface/memory.h +5
-0@@ -263,4 +263,47 @@ struct xen_remove_from_physmap { };DEFINE_GUEST_HANDLE_STRUCT (xen_remove_from_physmap); +/* vNUMA structures */ +struct vmemrange { +uint64_t start, end; +/* reserved */ +uint64_t _padm;
If this field is reserved for future use, please name it _reserved and ensure that the hypervisor checks that it is 0.
include/xen/interface/memory.h struct xen_remove_from_physmap { · line 283
Quoted 10 lines
include/xen/interface/memory.h +10
-0+struct vnuma_topology_info { +/* OUT */ +domid_t domid; +uint32_t __pad; +/* IN */ +/* number of virtual numa nodes */ +union { +GUEST_HANDLE (uint) nr_nodes; +uint64_t _padn; + };
I don't think anonymous unions are allowed in the ABI. They're a c99 extension I think.
David
Re: [PATCH RESEND v2 1/2] xen: vnuma support for PV guests running as domU
In reply to Elena UfimtsevaQuoted 4 lines · Elena Ufimtseva
Issues Xen hypercall subop
XENMEM_get_vnumainfoand sets the NUMA topology, otherwise sets dummy NUMA node and preventsnuma_initfrom calling other numa initializators as they dont work with pv guests.
[...]
arch/x86/include/asm/xen/vnuma.h line 9
Quoted 12 lines
arch/x86/include/asm/xen/vnuma.h new file +9
-0@@ -0,0 +1,12 @@ +#ifndef _ASM_X86_VNUMA_H +#define _ASM_X86_VNUMA_H + +#ifdef CONFIG_XEN +bool xen_vnuma_supported (void ); +int xen_numa_init (void ); +#else +static inline bool xen_vnuma_supported (void ) {return false; }; +static inline int xen_numa_init (void ) {return -1 ; };
Return a valid -ve errno here. e.g., -ENOSYS.
Quoted 29 lines
arch/x86/include/asm/xen/vnuma.h new file +3
-0+#endif + +#endif /* _ASM_X86_VNUMA_H */ arch/x86/mm/numa.c +3
-0@@ -17,6 +17,7 @@#include <asm/dma.h> #include <asm/acpi.h> #include <asm/amd_nb.h> +#include "asm/xen/vnuma.h" #include "numa_internal.h" @@ -632,6 +633,8 @@ static int __init dummy_numa_init(void)void __init x86_numa_init (void ) {if (!numa_off) { +if (!numa_init (xen_numa_init)) +return ;#ifdef CONFIG_X86_NUMAQ if (!numa_init (numaq_numa_init))return ;arch/x86/xen/Makefile +0
-0
arch/x86/xen/Makefile CFLAGS_mmu.o := $(nostackp) · line 16
Quoted 5 lines
arch/x86/xen/Makefile +1
-1@@ -13,7 +13,7 @@ CFLAGS_mmu.o := $(nostackp) obj-y := enlighten.o setup.o multicalls.o mmu.o irq.o \ time.o xen-asm.o xen-asm_$(BITS).o \ grant-table.o suspend.o platform-pci-unplug.o \ - p2m.o + p2m.o vnuma.o
obj-$(CONFIG_NUMA) += vnuma.o
I commented on this already, please take more care to address all comments before reposting.
Quoted 8 lines
arch/x86/xen/vnuma.c new file +5
-0@@ -0,0 +1,127 @@ +#include <linux/err.h> +#include <linux/memblock.h> +#include <xen/interface/xen.h> +#include <xen/interface/memory.h> +#include <asm/xen/interface.h>
arch/x86/xen/vnuma.c line 15
Quoted 10 lines
arch/x86/xen/vnuma.c new file +10
-0+#include <asm/xen/hypercall.h> +#include <asm/xen/vnuma.h> + +#ifdef CONFIG_NUMA + +/* Checks if hypercall is supported */ +bool xen_vnuma_supported (void ) +{ +return HYPERVISOR_memory_op (XENMEM_get_vnuma_info,NULL ) + == -ENOSYS ? false : true;
return HYPERVISOR_memory_op() != -ENOSYS;
But as I note in the other email, I think this function isn't needed.
include/xen/interface/memory.h struct xen_remove_from_physmap { · line 270
Quoted 11 lines
include/xen/interface/memory.h +5
-0@@ -263,4 +263,47 @@ struct xen_remove_from_physmap { };DEFINE_GUEST_HANDLE_STRUCT (xen_remove_from_physmap); +/* vNUMA structures */ +struct vmemrange { +uint64_t start, end; +/* reserved */ +uint64_t _padm;
If this field is reserved for future use, please name it _reserved and ensure that the hypervisor checks that it is 0.
include/xen/interface/memory.h struct xen_remove_from_physmap { · line 283
Quoted 10 lines
include/xen/interface/memory.h +10
-0+struct vnuma_topology_info { +/* OUT */ +domid_t domid; +uint32_t __pad; +/* IN */ +/* number of virtual numa nodes */ +union { +GUEST_HANDLE (uint) nr_nodes; +uint64_t _padn; + };
I don't think anonymous unions are allowed in the ABI. They're a c99 extension I think.
David
PATCH RESEND v2 2/2 xen: enable vnuma for PV guest
In reply to Elena UfimtsevaEnables numa if vnuma topology hypercall is supported and it is domU.
--- arch/x86/xen/setup.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
arch/x86/xen/setup.c +5 -1
@@ -20,6 +20,7 @@#include <asm/numa.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <asm/xen/vnuma.h> #include <xen/xen.h> #include <xen/page.h>
@@ -598,6 +599,9 @@ void __init xen_arch_setup(void)WARN_ON (xen_set_default_idle ());fiddle_vdso ();#ifdef CONFIG_NUMA - numa_off =1 ; +if (!xen_initial_domain () &&xen_vnuma_supported ()) + numa_off =0 ; +else + numa_off =1 ;#endif }
Signature · 1 line
1.7.10.4
PATCH RESEND v2 2/2 xen: enable vnuma for PV guest
In reply to Elena UfimtsevaEnables numa if vnuma topology hypercall is supported and it is domU.
--- arch/x86/xen/setup.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
arch/x86/xen/setup.c +5 -1
@@ -20,6 +20,7 @@#include <asm/numa.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <asm/xen/vnuma.h> #include <xen/xen.h> #include <xen/page.h>
@@ -598,6 +599,9 @@ void __init xen_arch_setup(void)WARN_ON (xen_set_default_idle ());fiddle_vdso ();#ifdef CONFIG_NUMA - numa_off =1 ; +if (!xen_initial_domain () &&xen_vnuma_supported ()) + numa_off =0 ; +else + numa_off =1 ;#endif }
Signature · 1 line
1.7.10.4
Re: [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
Quoted 1 line · Elena Ufimtseva
Enables numa if vnuma topology hypercall is supported and it is domU.
[...]
Quoted 11 lines
arch/x86/xen/setup.c +1
-0@@ -20,6 +20,7 @@#include <asm/numa.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <asm/xen/vnuma.h> #include <xen/xen.h> #include <xen/page.h>
arch/x86/xen/setup.c void __init xen_arch_setup(void) · line 607
Quoted 10 lines
arch/x86/xen/setup.c +4
-1@@ -598,6 +599,9 @@ void __init xen_arch_setup(void)WARN_ON (xen_set_default_idle ());fiddle_vdso ();#ifdef CONFIG_NUMA - numa_off =1 ; +if (!xen_initial_domain () &&xen_vnuma_supported ()) + numa_off =0 ; +else + numa_off =1 ;#endif }
I think this whole #ifdef CONFIG_NUMA can be removed and hence xen_vnuma_supported() can be removed as well.
For any PV guest we can call the xen_numa_init() and it will do the right thing.
For dom0, the hypercall will either: return something sensible (if in the future Xen sets something up), or it will error.
If Xen does not have vnuma support, the hypercall will error.
In both error cases, the dummy numa node is setup as required.
If you do this, you can fold this change in with the previous patch.
David
Re: [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
Quoted 31 lines · David Vrabel
Quoted 1 line · Elena Ufimtseva
Enables numa if vnuma topology hypercall is supported and it is domU.
[...]
Quoted 11 lines
arch/x86/xen/setup.c +1
-0@@ -20,6 +20,7 @@#include <asm/numa.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <asm/xen/vnuma.h> #include <xen/xen.h> #include <xen/page.h> arch/x86/xen/setup.c void __init xen_arch_setup(void) · line 607
Quoted 10 lines
arch/x86/xen/setup.c +4
-1@@ -598,6 +599,9 @@ void __init xen_arch_setup(void)WARN_ON (xen_set_default_idle ());fiddle_vdso ();#ifdef CONFIG_NUMA - numa_off =1 ; +if (!xen_initial_domain () &&xen_vnuma_supported ()) + numa_off =0 ; +else + numa_off =1 ;#endif }I think this whole #ifdef
CONFIG_NUMAcan be removed and hencexen_vnuma_supported()can be removed as well.For any PV guest we can call the
xen_numa_init()and it will do the right thing.
Quoted 6 lines
For dom0, the hypercall will either: return something sensible (if in the future Xen sets something up), or it will error.
If Xen does not have vnuma support, the hypercall will error.
In both error cases, the dummy numa node is setup as required.
Incorrect. It will end up calling:
if (!numa_init(amd_numa_init))
which will crash dom0 (see 8d54db795 "xen/boot: Disable NUMA for PV guests.") as that amd_numa_init is called before the dummy node init.
Quoted 4 lines
If you do this, you can fold this change in with the previous patch.
David
Re: [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
In reply to David VrabelQuoted 31 lines · David Vrabel
Quoted 1 line · Elena Ufimtseva
Enables numa if vnuma topology hypercall is supported and it is domU.
[...]
Quoted 11 lines
arch/x86/xen/setup.c +1
-0@@ -20,6 +20,7 @@#include <asm/numa.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <asm/xen/vnuma.h> #include <xen/xen.h> #include <xen/page.h> arch/x86/xen/setup.c void __init xen_arch_setup(void) · line 607
Quoted 10 lines
arch/x86/xen/setup.c +4
-1@@ -598,6 +599,9 @@ void __init xen_arch_setup(void)WARN_ON (xen_set_default_idle ());fiddle_vdso ();#ifdef CONFIG_NUMA - numa_off =1 ; +if (!xen_initial_domain () &&xen_vnuma_supported ()) + numa_off =0 ; +else + numa_off =1 ;#endif }I think this whole #ifdef
CONFIG_NUMAcan be removed and hencexen_vnuma_supported()can be removed as well.For any PV guest we can call the
xen_numa_init()and it will do the right thing.
Quoted 6 lines
For dom0, the hypercall will either: return something sensible (if in the future Xen sets something up), or it will error.
If Xen does not have vnuma support, the hypercall will error.
In both error cases, the dummy numa node is setup as required.
Incorrect. It will end up calling:
if (!numa_init(amd_numa_init))
which will crash dom0 (see 8d54db795 "xen/boot: Disable NUMA for PV guests.") as that amd_numa_init is called before the dummy node init.
Quoted 4 lines
If you do this, you can fold this change in with the previous patch.
David
Re: [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
Quoted 37 lines · Konrad Rzeszutek Wilk
Quoted 36 lines · David Vrabel
Quoted 1 line · Elena Ufimtseva
Enables numa if vnuma topology hypercall is supported and it is domU.
[...]
Quoted 11 lines
arch/x86/xen/setup.c +1
-0@@ -20,6 +20,7 @@#include <asm/numa.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <asm/xen/vnuma.h> #include <xen/xen.h> #include <xen/page.h> arch/x86/xen/setup.c void __init xen_arch_setup(void) · line 607
Quoted 10 lines
arch/x86/xen/setup.c +4
-1@@ -598,6 +599,9 @@ void __init xen_arch_setup(void)WARN_ON (xen_set_default_idle ());fiddle_vdso ();#ifdef CONFIG_NUMA - numa_off =1 ; +if (!xen_initial_domain () &&xen_vnuma_supported ()) + numa_off =0 ; +else + numa_off =1 ;#endif }I think this whole #ifdef
CONFIG_NUMAcan be removed and hencexen_vnuma_supported()can be removed as well.For any PV guest we can call the
xen_numa_init()and it will do the right thing.For dom0, the hypercall will either: return something sensible (if in the future Xen sets something up), or it will error.
If Xen does not have vnuma support, the hypercall will error.
Quoted 8 lines
Quoted 1 line
In both error cases, the dummy numa node is setup as required.
Incorrect. It will end up calling:
if (!numa_init(amd_numa_init))which will crash dom0 (see
8d54db795"xen/boot: Disable NUMA for PV guests.") as thatamd_numa_initis called before the dummy node init.
No it won't. Any error path after the check for a PV guest will add the dummy node and return success, skipping any of the hardware-specific setup.
David
Re: [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
Quoted 42 lines · David Vrabel
Quoted 41 lines · Konrad Rzeszutek Wilk
Quoted 31 lines · David Vrabel
Quoted 1 line · Elena Ufimtseva
Enables numa if vnuma topology hypercall is supported and it is domU.
[...]
Quoted 11 lines
arch/x86/xen/setup.c +1
-0@@ -20,6 +20,7 @@#include <asm/numa.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <asm/xen/vnuma.h> #include <xen/xen.h> #include <xen/page.h> arch/x86/xen/setup.c void __init xen_arch_setup(void) · line 607
Quoted 10 lines
arch/x86/xen/setup.c +4
-1@@ -598,6 +599,9 @@ void __init xen_arch_setup(void)WARN_ON (xen_set_default_idle ());fiddle_vdso ();#ifdef CONFIG_NUMA - numa_off =1 ; +if (!xen_initial_domain () &&xen_vnuma_supported ()) + numa_off =0 ; +else + numa_off =1 ;#endif }I think this whole #ifdef
CONFIG_NUMAcan be removed and hencexen_vnuma_supported()can be removed as well.For any PV guest we can call the
xen_numa_init()and it will do the right thing.Quoted 6 lines
For dom0, the hypercall will either: return something sensible (if in the future Xen sets something up), or it will error.
If Xen does not have vnuma support, the hypercall will error.
In both error cases, the dummy numa node is setup as required.
Incorrect. It will end up calling:
Quoted 7 lines
Quoted 4 lines
if (!numa_init(amd_numa_init))which will crash dom0 (see
8d54db795"xen/boot: Disable NUMA for PV guests.") as thatamd_numa_initis called before the dummy node init.No it won't. Any error path after the check for a PV guest will add the dummy node and return success, skipping any of the hardware-specific setup.
Duh! I totally missed 'return' at the end of the check!
However, even with that (so the return), that means this part won't be called:
649 numa_init(dummy_numa_init);
Which means there won't be any dummy numa setup?
Re: [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
In reply to David VrabelQuoted 42 lines · David Vrabel
Quoted 41 lines · Konrad Rzeszutek Wilk
Quoted 31 lines · David Vrabel
Quoted 1 line · Elena Ufimtseva
Enables numa if vnuma topology hypercall is supported and it is domU.
[...]
Quoted 11 lines
arch/x86/xen/setup.c +1
-0@@ -20,6 +20,7 @@#include <asm/numa.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <asm/xen/vnuma.h> #include <xen/xen.h> #include <xen/page.h> arch/x86/xen/setup.c void __init xen_arch_setup(void) · line 607
Quoted 10 lines
arch/x86/xen/setup.c +4
-1@@ -598,6 +599,9 @@ void __init xen_arch_setup(void)WARN_ON (xen_set_default_idle ());fiddle_vdso ();#ifdef CONFIG_NUMA - numa_off =1 ; +if (!xen_initial_domain () &&xen_vnuma_supported ()) + numa_off =0 ; +else + numa_off =1 ;#endif }I think this whole #ifdef
CONFIG_NUMAcan be removed and hencexen_vnuma_supported()can be removed as well.For any PV guest we can call the
xen_numa_init()and it will do the right thing.Quoted 6 lines
For dom0, the hypercall will either: return something sensible (if in the future Xen sets something up), or it will error.
If Xen does not have vnuma support, the hypercall will error.
In both error cases, the dummy numa node is setup as required.
Incorrect. It will end up calling:
Quoted 7 lines
Quoted 4 lines
if (!numa_init(amd_numa_init))which will crash dom0 (see
8d54db795"xen/boot: Disable NUMA for PV guests.") as thatamd_numa_initis called before the dummy node init.No it won't. Any error path after the check for a PV guest will add the dummy node and return success, skipping any of the hardware-specific setup.
Duh! I totally missed 'return' at the end of the check!
However, even with that (so the return), that means this part won't be called:
649 numa_init(dummy_numa_init);
Which means there won't be any dummy numa setup?
Re: [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
Quoted 51 lines · Konrad Rzeszutek Wilk
Quoted 42 lines · David Vrabel
Quoted 41 lines · Konrad Rzeszutek Wilk
Quoted 31 lines · David Vrabel
Quoted 1 line · Elena Ufimtseva
Enables numa if vnuma topology hypercall is supported and it is domU.
[...]
Quoted 11 lines
arch/x86/xen/setup.c +1
-0@@ -20,6 +20,7 @@#include <asm/numa.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <asm/xen/vnuma.h> #include <xen/xen.h> #include <xen/page.h> arch/x86/xen/setup.c void __init xen_arch_setup(void) · line 607
Quoted 10 lines
arch/x86/xen/setup.c +4
-1@@ -598,6 +599,9 @@ void __init xen_arch_setup(void)WARN_ON (xen_set_default_idle ());fiddle_vdso ();#ifdef CONFIG_NUMA - numa_off =1 ; +if (!xen_initial_domain () &&xen_vnuma_supported ()) + numa_off =0 ; +else + numa_off =1 ;#endif }I think this whole #ifdef
CONFIG_NUMAcan be removed and hencexen_vnuma_supported()can be removed as well.For any PV guest we can call the
xen_numa_init()and it will do the right thing.Quoted 6 lines
For dom0, the hypercall will either: return something sensible (if in the future Xen sets something up), or it will error.
If Xen does not have vnuma support, the hypercall will error.
In both error cases, the dummy numa node is setup as required.
Incorrect. It will end up calling:
Quoted 7 lines
Quoted 4 lines
if (!numa_init(amd_numa_init))which will crash dom0 (see
8d54db795"xen/boot: Disable NUMA for PV guests.") as thatamd_numa_initis called before the dummy node init.No it won't. Any error path after the check for a PV guest will add the dummy node and return success, skipping any of the hardware-specific setup.
Quoted 8 lines
Duh! I totally missed 'return' at the end of the check!
However, even with that (so the return), that means this part won't be called:
649
numa_init(dummy_numa_init);Which means there won't be any dummy numa setup?
The relevant bits in dummy_numa_init are in the error path of xen_numa_init().
I do think this approach (using the provided API to setup the single (dummy) node), is preferable to calling dummy_numa_init().
If I thought the hypervisor ABI was finalized, I'd be happy with this series as-is -- the remaining issues are superficial.
David
Re: [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
Quoted 57 lines · David Vrabel
Quoted 56 lines · Konrad Rzeszutek Wilk
Quoted 42 lines · David Vrabel
Quoted 41 lines · Konrad Rzeszutek Wilk
Quoted 31 lines · David Vrabel
Quoted 1 line · Elena Ufimtseva
Enables numa if vnuma topology hypercall is supported and it is domU.
[...]
Quoted 11 lines
arch/x86/xen/setup.c +1
-0@@ -20,6 +20,7 @@#include <asm/numa.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <asm/xen/vnuma.h> #include <xen/xen.h> #include <xen/page.h> arch/x86/xen/setup.c void __init xen_arch_setup(void) · line 607
Quoted 10 lines
arch/x86/xen/setup.c +4
-1@@ -598,6 +599,9 @@ void __init xen_arch_setup(void)WARN_ON (xen_set_default_idle ());fiddle_vdso ();#ifdef CONFIG_NUMA - numa_off =1 ; +if (!xen_initial_domain () &&xen_vnuma_supported ()) + numa_off =0 ; +else + numa_off =1 ;#endif }I think this whole #ifdef
CONFIG_NUMAcan be removed and hencexen_vnuma_supported()can be removed as well.For any PV guest we can call the
xen_numa_init()and it will do the right thing.Quoted 6 lines
For dom0, the hypercall will either: return something sensible (if in the future Xen sets something up), or it will error.
If Xen does not have vnuma support, the hypercall will error.
In both error cases, the dummy numa node is setup as required.
Incorrect. It will end up calling:
Quoted 7 lines
Quoted 4 lines
if (!numa_init(amd_numa_init))which will crash dom0 (see
8d54db795"xen/boot: Disable NUMA for PV guests.") as thatamd_numa_initis called before the dummy node init.No it won't. Any error path after the check for a PV guest will add the dummy node and return success, skipping any of the hardware-specific setup.
Duh! I totally missed 'return' at the end of the check!
However, even with that (so the return), that means this part won't be called:
Quoted 6 lines
Quoted 3 lines
649
numa_init(dummy_numa_init);Which means there won't be any dummy numa setup?
The relevant bits in
dummy_numa_initare in the error path ofxen_numa_init().
That seems the wrong place to do it. The top layer calls in each of the numa implementations and then falls back to the dummy.
Calling from within the implementation on something that is eventually done on the upper level already is not right.
Quoted 3 lines
I do think this approach (using the provided API to setup the single (dummy) node), is preferable to calling
dummy_numa_init().
Doesn't it do the same thing? And also what about if you the user provides fakenuma?
Quoted 3 lines
If I thought the hypervisor ABI was finalized, I'd be happy with this series as-is -- the remaining issues are superficial.
That reads to me as an Ack, but I know you like to have it stated explicitly - so could you state the proper tag please?
Quoted 2 lines
David
Re: [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
In reply to David VrabelQuoted 57 lines · David Vrabel
Quoted 56 lines · Konrad Rzeszutek Wilk
Quoted 42 lines · David Vrabel
Quoted 41 lines · Konrad Rzeszutek Wilk
Quoted 31 lines · David Vrabel
Quoted 1 line · Elena Ufimtseva
Enables numa if vnuma topology hypercall is supported and it is domU.
[...]
Quoted 11 lines
arch/x86/xen/setup.c +1
-0@@ -20,6 +20,7 @@#include <asm/numa.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <asm/xen/vnuma.h> #include <xen/xen.h> #include <xen/page.h> arch/x86/xen/setup.c void __init xen_arch_setup(void) · line 607
Quoted 10 lines
arch/x86/xen/setup.c +4
-1@@ -598,6 +599,9 @@ void __init xen_arch_setup(void)WARN_ON (xen_set_default_idle ());fiddle_vdso ();#ifdef CONFIG_NUMA - numa_off =1 ; +if (!xen_initial_domain () &&xen_vnuma_supported ()) + numa_off =0 ; +else + numa_off =1 ;#endif }I think this whole #ifdef
CONFIG_NUMAcan be removed and hencexen_vnuma_supported()can be removed as well.For any PV guest we can call the
xen_numa_init()and it will do the right thing.Quoted 6 lines
For dom0, the hypercall will either: return something sensible (if in the future Xen sets something up), or it will error.
If Xen does not have vnuma support, the hypercall will error.
In both error cases, the dummy numa node is setup as required.
Incorrect. It will end up calling:
Quoted 7 lines
Quoted 4 lines
if (!numa_init(amd_numa_init))which will crash dom0 (see
8d54db795"xen/boot: Disable NUMA for PV guests.") as thatamd_numa_initis called before the dummy node init.No it won't. Any error path after the check for a PV guest will add the dummy node and return success, skipping any of the hardware-specific setup.
Duh! I totally missed 'return' at the end of the check!
However, even with that (so the return), that means this part won't be called:
Quoted 6 lines
Quoted 3 lines
649
numa_init(dummy_numa_init);Which means there won't be any dummy numa setup?
The relevant bits in
dummy_numa_initare in the error path ofxen_numa_init().
That seems the wrong place to do it. The top layer calls in each of the numa implementations and then falls back to the dummy.
Calling from within the implementation on something that is eventually done on the upper level already is not right.
Quoted 3 lines
I do think this approach (using the provided API to setup the single (dummy) node), is preferable to calling
dummy_numa_init().
Doesn't it do the same thing? And also what about if you the user provides fakenuma?
Quoted 3 lines
If I thought the hypervisor ABI was finalized, I'd be happy with this series as-is -- the remaining issues are superficial.
That reads to me as an Ack, but I know you like to have it stated explicitly - so could you state the proper tag please?
Quoted 2 lines
David
Re: [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
Quoted 7 lines · Konrad Rzeszutek Wilk
Quoted 2 lines · David Vrabel
The relevant bits in
dummy_numa_initare in the error path ofxen_numa_init().That seems the wrong place to do it. The top layer calls in each of the numa implementations and then falls back to the dummy.
Think of it as not the dummy, but Xen setting the NUMA configuration up with only a single node.
The useful bits in dummy_numa_init() are two calls to standard functions for use by *_numa_init() calls so it just seems easier all round to just call then directly than add a dependancy on dummy_numa_init().
Quoted 4 lines
Calling from within the implementation on something that is eventually done on the upper level already is not right.
From the point of view of the caller, it does the right thing. NUMA is
setup.
Quoted 5 lines
Quoted 2 lines
I do think this approach (using the provided API to setup the single (dummy) node), is preferable to calling
dummy_numa_init().Doesn't it do the same thing? And also what about if you the user provides fakenuma?
I don't know what "fakenuma" is refering to.
Quoted 5 lines
Quoted 2 lines
If I thought the hypervisor ABI was finalized, I'd be happy with this series as-is -- the remaining issues are superficial.
That reads to me as an Ack, but I know you like to have it stated explicitly - so could you state the proper tag please?
"If I thought the hypervisor ABI was finalized..." is a pretty big "if" so I have deliberately /not/ given an ack or a reviewed tag but I've tried to be clear than I think the Linux side is now good enough (except for any changes for any updates to the hypervisor ABI).
David
Re: [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
Quoted 11 lines · David Vrabel
Quoted 7 lines · Konrad Rzeszutek Wilk
Quoted 2 lines · David Vrabel
The relevant bits in
dummy_numa_initare in the error path ofxen_numa_init().That seems the wrong place to do it. The top layer calls in each of the numa implementations and then falls back to the dummy.
Think of it as not the dummy, but Xen setting the NUMA configuration up with only a single node.
Or with multiple nodes if numa=fake is used.
Quoted 4 lines
The useful bits in
dummy_numa_init()are two calls to standard functions for use by*_numa_init()calls so it just seems easier all round to just call then directly than add a dependancy ondummy_numa_init().
My worry is more of the numa_init not being completly executed.
Quoted 7 lines
Quoted 5 lines
Calling from within the implementation on something that is eventually done on the upper level already is not right.
From the point of view of the caller, it does the right thing. NUMA is
setup.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
I do think this approach (using the provided API to setup the single (dummy) node), is preferable to calling
dummy_numa_init().Doesn't it do the same thing? And also what about if you the user provides fakenuma?
I don't know what "fakenuma" is refering to.
numa=fake or see arch/x86/mm/numa_emulation.c
Which gets executed if you end up calling: numa_init(X) and 'X' returns a zero or positive value.
Which it won't as the first thing it the Xen PV NUMA code does is to see if the hypercall is supported. If not, it bails out - so we never get to do the 'numa=fake' if a user wanted to.
Re: [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
In reply to David VrabelQuoted 11 lines · David Vrabel
Quoted 7 lines · Konrad Rzeszutek Wilk
Quoted 2 lines · David Vrabel
The relevant bits in
dummy_numa_initare in the error path ofxen_numa_init().That seems the wrong place to do it. The top layer calls in each of the numa implementations and then falls back to the dummy.
Think of it as not the dummy, but Xen setting the NUMA configuration up with only a single node.
Or with multiple nodes if numa=fake is used.
Quoted 4 lines
The useful bits in
dummy_numa_init()are two calls to standard functions for use by*_numa_init()calls so it just seems easier all round to just call then directly than add a dependancy ondummy_numa_init().
My worry is more of the numa_init not being completly executed.
Quoted 7 lines
Quoted 5 lines
Calling from within the implementation on something that is eventually done on the upper level already is not right.
From the point of view of the caller, it does the right thing. NUMA is
setup.
Quoted 7 lines
Quoted 5 lines
Quoted 2 lines
I do think this approach (using the provided API to setup the single (dummy) node), is preferable to calling
dummy_numa_init().Doesn't it do the same thing? And also what about if you the user provides fakenuma?
I don't know what "fakenuma" is refering to.
numa=fake or see arch/x86/mm/numa_emulation.c
Which gets executed if you end up calling: numa_init(X) and 'X' returns a zero or positive value.
Which it won't as the first thing it the Xen PV NUMA code does is to see if the hypercall is supported. If not, it bails out - so we never get to do the 'numa=fake' if a user wanted to.
Re: [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
In reply to Konrad Rzeszutek WilkQuoted 7 lines · Konrad Rzeszutek Wilk
Quoted 2 lines · David Vrabel
The relevant bits in
dummy_numa_initare in the error path ofxen_numa_init().That seems the wrong place to do it. The top layer calls in each of the numa implementations and then falls back to the dummy.
Think of it as not the dummy, but Xen setting the NUMA configuration up with only a single node.
The useful bits in dummy_numa_init() are two calls to standard functions for use by *_numa_init() calls so it just seems easier all round to just call then directly than add a dependancy on dummy_numa_init().
Quoted 4 lines
Calling from within the implementation on something that is eventually done on the upper level already is not right.
From the point of view of the caller, it does the right thing. NUMA is
setup.
Quoted 5 lines
Quoted 2 lines
I do think this approach (using the provided API to setup the single (dummy) node), is preferable to calling
dummy_numa_init().Doesn't it do the same thing? And also what about if you the user provides fakenuma?
I don't know what "fakenuma" is refering to.
Quoted 5 lines
Quoted 2 lines
If I thought the hypervisor ABI was finalized, I'd be happy with this series as-is -- the remaining issues are superficial.
That reads to me as an Ack, but I know you like to have it stated explicitly - so could you state the proper tag please?
"If I thought the hypervisor ABI was finalized..." is a pretty big "if" so I have deliberately /not/ given an ack or a reviewed tag but I've tried to be clear than I think the Linux side is now good enough (except for any changes for any updates to the hypervisor ABI).
David
Re: [Xen-devel] [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
In reply to David VrabelQuoted 9 lines · David Vrabel
The relevant bits in
dummy_numa_initare in the error path ofxen_numa_init().I do think this approach (using the provided API to setup the single (dummy) node), is preferable to calling
dummy_numa_init().If I thought the hypervisor ABI was finalized, I'd be happy with this series as-is -- the remaining issues are superficial.
So, David, in this regard, what do you think about these (and the following messages in the thread?
http://bugs.xenproject.org/xen/mid/%[email protected]%3E
http://bugs.xenproject.org/xen/mid/%3C1384871712.19880.90.camel@Abyss%3E
http://bugs.xenproject.org/xen/mid/%[email protected]%3E
http://bugs.xenproject.org/xen/mid/%3C1384875772.15360.6.camel@Solace%3E
http://bugs.xenproject.org/xen/mid/%[email protected]%3E
I think Jan has a point about how we are right now retrieving and treating the number of vnodes...
Thanks and Regards,
Dario
Signature · 4 lines
<<This happens because I choose it to happen!>> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
Re: [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
In reply to David VrabelQuoted 9 lines · David Vrabel
The relevant bits in
dummy_numa_initare in the error path ofxen_numa_init().I do think this approach (using the provided API to setup the single (dummy) node), is preferable to calling
dummy_numa_init().If I thought the hypervisor ABI was finalized, I'd be happy with this series as-is -- the remaining issues are superficial.
So, David, in this regard, what do you think about these (and the following messages in the thread?
http://bugs.xenproject.org/xen/mid/%[email protected]%3E
http://bugs.xenproject.org/xen/mid/%3C1384871712.19880.90.camel@Abyss%3E
http://bugs.xenproject.org/xen/mid/%[email protected]%3E
http://bugs.xenproject.org/xen/mid/%3C1384875772.15360.6.camel@Solace%3E
http://bugs.xenproject.org/xen/mid/%[email protected]%3E
I think Jan has a point about how we are right now retrieving and treating the number of vnodes...
Thanks and Regards,
Dario
Signature · 4 lines
<<This happens because I choose it to happen!>> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel
Re: [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
In reply to Konrad Rzeszutek WilkQuoted 51 lines · Konrad Rzeszutek Wilk
Quoted 42 lines · David Vrabel
Quoted 41 lines · Konrad Rzeszutek Wilk
Quoted 31 lines · David Vrabel
Quoted 1 line · Elena Ufimtseva
Enables numa if vnuma topology hypercall is supported and it is domU.
[...]
Quoted 11 lines
arch/x86/xen/setup.c +1
-0@@ -20,6 +20,7 @@#include <asm/numa.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <asm/xen/vnuma.h> #include <xen/xen.h> #include <xen/page.h> arch/x86/xen/setup.c void __init xen_arch_setup(void) · line 607
Quoted 10 lines
arch/x86/xen/setup.c +4
-1@@ -598,6 +599,9 @@ void __init xen_arch_setup(void)WARN_ON (xen_set_default_idle ());fiddle_vdso ();#ifdef CONFIG_NUMA - numa_off =1 ; +if (!xen_initial_domain () &&xen_vnuma_supported ()) + numa_off =0 ; +else + numa_off =1 ;#endif }I think this whole #ifdef
CONFIG_NUMAcan be removed and hencexen_vnuma_supported()can be removed as well.For any PV guest we can call the
xen_numa_init()and it will do the right thing.Quoted 6 lines
For dom0, the hypercall will either: return something sensible (if in the future Xen sets something up), or it will error.
If Xen does not have vnuma support, the hypercall will error.
In both error cases, the dummy numa node is setup as required.
Incorrect. It will end up calling:
Quoted 7 lines
Quoted 4 lines
if (!numa_init(amd_numa_init))which will crash dom0 (see
8d54db795"xen/boot: Disable NUMA for PV guests.") as thatamd_numa_initis called before the dummy node init.No it won't. Any error path after the check for a PV guest will add the dummy node and return success, skipping any of the hardware-specific setup.
Quoted 8 lines
Duh! I totally missed 'return' at the end of the check!
However, even with that (so the return), that means this part won't be called:
649
numa_init(dummy_numa_init);Which means there won't be any dummy numa setup?
The relevant bits in dummy_numa_init are in the error path of xen_numa_init().
I do think this approach (using the provided API to setup the single (dummy) node), is preferable to calling dummy_numa_init().
If I thought the hypervisor ABI was finalized, I'd be happy with this series as-is -- the remaining issues are superficial.
David
Re: [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
In reply to Konrad Rzeszutek WilkQuoted 37 lines · Konrad Rzeszutek Wilk
Quoted 36 lines · David Vrabel
Quoted 1 line · Elena Ufimtseva
Enables numa if vnuma topology hypercall is supported and it is domU.
[...]
Quoted 11 lines
arch/x86/xen/setup.c +1
-0@@ -20,6 +20,7 @@#include <asm/numa.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <asm/xen/vnuma.h> #include <xen/xen.h> #include <xen/page.h> arch/x86/xen/setup.c void __init xen_arch_setup(void) · line 607
Quoted 10 lines
arch/x86/xen/setup.c +4
-1@@ -598,6 +599,9 @@ void __init xen_arch_setup(void)WARN_ON (xen_set_default_idle ());fiddle_vdso ();#ifdef CONFIG_NUMA - numa_off =1 ; +if (!xen_initial_domain () &&xen_vnuma_supported ()) + numa_off =0 ; +else + numa_off =1 ;#endif }I think this whole #ifdef
CONFIG_NUMAcan be removed and hencexen_vnuma_supported()can be removed as well.For any PV guest we can call the
xen_numa_init()and it will do the right thing.For dom0, the hypercall will either: return something sensible (if in the future Xen sets something up), or it will error.
If Xen does not have vnuma support, the hypercall will error.
Quoted 8 lines
Quoted 1 line
In both error cases, the dummy numa node is setup as required.
Incorrect. It will end up calling:
if (!numa_init(amd_numa_init))which will crash dom0 (see
8d54db795"xen/boot: Disable NUMA for PV guests.") as thatamd_numa_initis called before the dummy node init.
No it won't. Any error path after the check for a PV guest will add the dummy node and return success, skipping any of the hardware-specific setup.
David
Re: [PATCH RESEND v2 2/2] xen: enable vnuma for PV guest
In reply to Elena UfimtsevaQuoted 1 line · Elena Ufimtseva
Enables numa if vnuma topology hypercall is supported and it is domU.
[...]
Quoted 11 lines
arch/x86/xen/setup.c +1
-0@@ -20,6 +20,7 @@#include <asm/numa.h> #include <asm/xen/hypervisor.h> #include <asm/xen/hypercall.h> +#include <asm/xen/vnuma.h> #include <xen/xen.h> #include <xen/page.h>
arch/x86/xen/setup.c void __init xen_arch_setup(void) · line 607
Quoted 10 lines
arch/x86/xen/setup.c +4
-1@@ -598,6 +599,9 @@ void __init xen_arch_setup(void)WARN_ON (xen_set_default_idle ());fiddle_vdso ();#ifdef CONFIG_NUMA - numa_off =1 ; +if (!xen_initial_domain () &&xen_vnuma_supported ()) + numa_off =0 ; +else + numa_off =1 ;#endif }
I think this whole #ifdef CONFIG_NUMA can be removed and hence xen_vnuma_supported() can be removed as well.
For any PV guest we can call the xen_numa_init() and it will do the right thing.
For dom0, the hypercall will either: return something sensible (if in the future Xen sets something up), or it will error.
If Xen does not have vnuma support, the hypercall will error.
In both error cases, the dummy numa node is setup as required.
If you do this, you can fold this change in with the previous patch.
David
Quoted 10 lines · Elena Ufimtseva
Xen vnuma introduction.
The patchset introduces vnuma to paravirtualized Xen guests runnning as domU.
Xen subop hypercall is used to retreive vnuma topology information. Bases on the retreived topology from Xen, NUMA number of nodes, memory ranges, distance table and cpumask is being set. If initialization is incorrect, sets 'dummy' node and unsets nodemask. vNUMA topology is constructed by Xen toolstack.
Quoted 2 lines
Example of vnuma enabled pv domain dmesg:
Mmm... So, why the resend? When you do things like this, you usually tell the reason, if only, to let people know what happened and which series they should actually look at and review. :-)
Regards,
Dario
Signature · 4 lines
<<This happens because I choose it to happen!>> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
Quoted 10 lines · Elena Ufimtseva
Xen vnuma introduction.
The patchset introduces vnuma to paravirtualized Xen guests runnning as domU.
Xen subop hypercall is used to retreive vnuma topology information. Bases on the retreived topology from Xen, NUMA number of nodes, memory ranges, distance table and cpumask is being set. If initialization is incorrect, sets 'dummy' node and unsets nodemask. vNUMA topology is constructed by Xen toolstack.
Quoted 2 lines
Example of vnuma enabled pv domain dmesg:
Mmm... So, why the resend? When you do things like this, you usually tell the reason, if only, to let people know what happened and which series they should actually look at and review. :-)
Regards,
Dario
Signature · 4 lines
<<This happens because I choose it to happen!>> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel