numa: Introduce MachineClass::auto_enable_numa for implicit NUMA node
PATCH 1/1 numa: Introduce MachineClass::auto_enable_numa for implicit NUMA node
Add MachineClass::auto_enable_numa field. When it is true, a NUMA node is expected to be created implicitly.
---
This patch has a dependency on
https://patchwork.kernel.org/cover/11063235/
--- hw/core/numa.c | 9 +++++++-- hw/ppc/spapr.c | 9 +-------- include/hw/boards.h | 1 + 3 files changed, 9 insertions(+), 10 deletions(-)
hw/core/numa.c +7 -2
@@ -580,9 +580,14 @@ void numa_complete_configuration(MachineState *ms)* guest tries to use it with that drivers. * * Enable NUMA implicitly by adding a new NUMA node automatically. +* +* Or if MachineClass::auto_enable_numa is true and no NUMA nodes, +* assume there is just one node with whole RAM. */ -if (ms->ram_slots >0 && ms->numa_state->num_nodes ==0 && - mc->auto_enable_numa_with_memhp) { +if (ms->numa_state->num_nodes ==0 && + ((ms->ram_slots >0 && + mc->auto_enable_numa_with_memhp) || + mc->auto_enable_numa)) {NumaNodeOptions node = { };parse_numa_node (ms, &node, &error_abort); }
hw/ppc/spapr.c +1 -8
@@ -400,14 +400,6 @@ static int spapr_populate_memory(SpaprMachineState *spapr, void *fdt)hwaddr mem_start, node_size;int i, nb_nodes = machine->numa_state->num_nodes;NodeInfo *nodes = machine->numa_state->nodes; -NodeInfo ramnode; - -/* No NUMA nodes, assume there is just one node with whole RAM */ -if (!nb_nodes) { - nb_nodes =1 ; - ramnode.node_mem = machine->ram_size; - nodes = &ramnode; - }for (i =0 , mem_start =0 ; i < nb_nodes; ++i) {if (!nodes[i].node_mem) {
@@ -4369,6 +4361,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)*/ mc->numa_mem_align_shift =28 ; mc->numa_mem_supported = true; + mc->auto_enable_numa = true; smc->default_caps.caps[SPAPR_CAP_HTM ] =SPAPR_CAP_OFF ; smc->default_caps.caps[SPAPR_CAP_VSX ] =SPAPR_CAP_ON ;
include/hw/boards.h +1 -0
@@ -220,6 +220,7 @@ struct MachineClass {bool smbus_no_migration_support;bool nvdimm_supported;bool numa_mem_supported; +bool auto_enable_numa; HotplugHandler *(*get_hotplug_handler)(MachineState *machine, DeviceState *dev);
Signature · 1 line
2.20.1
Quoted 58 lines · On Mon, 5 Aug 2019 15:13:02 +0800 Tao Xu <[email protected]> wrote:
Add
MachineClass::auto_enable_numafield. When it is true, a NUMA node is expected to be created implicitly.Acked-by David Gibson <[email protected]>Suggested-by Igor Mammedov <[email protected]>Suggested-by Eduardo Habkost <[email protected]>Signed-off-by Tao Xu <[email protected]>---
This patch has a dependency on
https://patchwork.kernel.org/cover/11063235/--- hw/core/numa.c | 9 +++++++-- hw/ppc/spapr.c | 9 +-------- include/hw/boards.h | 1 + 3 files changed, 9 insertions(+), 10 deletions(-)hw/core/numa.c +7
-2@@ -580,9 +580,14 @@ void numa_complete_configuration(MachineState *ms)* guest tries to use it with that drivers. * * Enable NUMA implicitly by adding a new NUMA node automatically. +* +* Or if MachineClass::auto_enable_numa is true and no NUMA nodes, +* assume there is just one node with whole RAM. */ -if (ms->ram_slots >0 && ms->numa_state->num_nodes ==0 && - mc->auto_enable_numa_with_memhp) { +if (ms->numa_state->num_nodes ==0 && + ((ms->ram_slots >0 && + mc->auto_enable_numa_with_memhp) || + mc->auto_enable_numa)) {NumaNodeOptions node = { };parse_numa_node (ms, &node, &error_abort); }hw/ppc/spapr.c +0
-8@@ -400,14 +400,6 @@ static int spapr_populate_memory(SpaprMachineState *spapr, void *fdt)hwaddr mem_start, node_size;int i, nb_nodes = machine->numa_state->num_nodes;NodeInfo *nodes = machine->numa_state->nodes; -NodeInfo ramnode; - -/* No NUMA nodes, assume there is just one node with whole RAM */ -if (!nb_nodes) { - nb_nodes =1 ; - ramnode.node_mem = machine->ram_size; - nodes = &ramnode; - }for (i =0 , mem_start =0 ; i < nb_nodes; ++i) {if (!nodes[i].node_mem) {
hw/ppc/spapr.c static void spapr_machine_class_init(ObjectClass *oc, void *data) · line 4364
Quoted 4 lines
hw/ppc/spapr.c +1
-0@@ -4369,6 +4361,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)*/ mc->numa_mem_align_shift =28 ; mc->numa_mem_supported = true; + mc->auto_enable_numa = true;
this will always create a numa node (that will affect not only RAM but also all other components that depends on numa state (like CPUs)), where as spapr_populate_memory() was only faking numa node in DT for RAM. It makes non-numa configuration impossible.
Seeing David's ACK on the patch it might be fine, but I believe commit message should capture that and explain why the change in behavior is fine.
Quoted 14 lines
hw/ppc/spapr.c +0
-0smc->default_caps.caps[SPAPR_CAP_HTM ] =SPAPR_CAP_OFF ; smc->default_caps.caps[SPAPR_CAP_VSX ] =SPAPR_CAP_ON ;include/hw/boards.h +1
-0@@ -220,6 +220,7 @@ struct MachineClass {bool smbus_no_migration_support;bool nvdimm_supported;bool numa_mem_supported; +bool auto_enable_numa; HotplugHandler *(*get_hotplug_handler)(MachineState *machine, DeviceState *dev);
Quoted 10 lines · Igor Mammedov
Quoted 7 lines · On Mon, 5 Aug 2019 15:13:02 +0800 Tao Xu <[email protected]> wrote:
Add
MachineClass::auto_enable_numafield. When it is true, a NUMA node is expected to be created implicitly.Acked-by David Gibson <[email protected]>Suggested-by Igor Mammedov <[email protected]>Suggested-by Eduardo Habkost <[email protected]>Signed-off-by Tao Xu <[email protected]>
[...]
Quoted 9 lines
Quoted 1 line
+
mc->auto_enable_numa= true;this will always create a numa node (that will affect not only RAM but also all other components that depends on numa state (like CPUs)), where as
spapr_populate_memory()was only faking numa node in DT for RAM. It makes non-numa configuration impossible.
Seeing David's ACK on the patch it might be fine, but I believe commit message should capture that and explain why the change in behavior is fine.
After a quick look, all spapr code seems to have the same behavior when nb_numa_nodes==0 and nb_numa_nodes==1, but I'd like to be sure.
David and/or Tao Xu: do you confirm there's no ABI change at all on spapr after implicitly creating a NUMA node?
Quoted 16 lines
Quoted 15 lines
smc->default_caps.caps[SPAPR_CAP_HTM] = SPAPR_CAP_OFF; smc->default_caps.caps[SPAPR_CAP_VSX] = SPAPR_CAP_ON;include/hw/boards.h +1
-0@@ -220,6 +220,7 @@ struct MachineClass {bool smbus_no_migration_support;bool nvdimm_supported;bool numa_mem_supported; +bool auto_enable_numa; HotplugHandler *(*get_hotplug_handler)(MachineState *machine, DeviceState *dev);
Signature · 1 line
Eduardo
Quoted 22 lines · Eduardo Habkost
Quoted 10 lines · Igor Mammedov
Quoted 7 lines · On Mon, 5 Aug 2019 15:13:02 +0800 Tao Xu <[email protected]> wrote:
Add
MachineClass::auto_enable_numafield. When it is true, a NUMA node is expected to be created implicitly.Acked-by David Gibson <[email protected]>Suggested-by Igor Mammedov <[email protected]>Suggested-by Eduardo Habkost <[email protected]>Signed-off-by Tao Xu <[email protected]>[...]
Quoted 9 lines
Quoted 1 line
+
mc->auto_enable_numa= true;this will always create a numa node (that will affect not only RAM but also all other components that depends on numa state (like CPUs)), where as
spapr_populate_memory()was only faking numa node in DT for RAM. It makes non-numa configuration impossible.
Seeing David's ACK on the patch it might be fine, but I believe commit message should capture that and explain why the change in behavior is fine.
Quoted 7 lines
After a quick look, all spapr code seems to have the same behavior when nb_numa_nodes==0 and nb_numa_nodes==1, but I'd like to be sure.
David and/or Tao Xu: do you confirm there's no ABI change at all on spapr after implicitly creating a NUMA node?
Even without this patch and HMAT patch, if without numa configuration, global nb_numa_nodes is always existing and default is 0, so nb_nodes will be auto set to 1, so from my point of view, this patch will not change ABI.
And I would also want to listen David's opinion.
Quoted 17 lines
Quoted 16 lines
Quoted 15 lines
smc->default_caps.caps[SPAPR_CAP_HTM] = SPAPR_CAP_OFF; smc->default_caps.caps[SPAPR_CAP_VSX] = SPAPR_CAP_ON;include/hw/boards.h +1
-0@@ -220,6 +220,7 @@ struct MachineClass {bool smbus_no_migration_support;bool nvdimm_supported;bool numa_mem_supported; +bool auto_enable_numa; HotplugHandler *(*get_hotplug_handler)(MachineState *machine, DeviceState *dev);
Quoted 22 lines · Eduardo Habkost
Quoted 10 lines · Igor Mammedov
Quoted 7 lines · On Mon, 5 Aug 2019 15:13:02 +0800 Tao Xu <[email protected]> wrote:
Add
MachineClass::auto_enable_numafield. When it is true, a NUMA node is expected to be created implicitly.Acked-by David Gibson <[email protected]>Suggested-by Igor Mammedov <[email protected]>Suggested-by Eduardo Habkost <[email protected]>Signed-off-by Tao Xu <[email protected]>[...]
Quoted 9 lines
Quoted 1 line
+
mc->auto_enable_numa= true;this will always create a numa node (that will affect not only RAM but also all other components that depends on numa state (like CPUs)), where as
spapr_populate_memory()was only faking numa node in DT for RAM. It makes non-numa configuration impossible.
Seeing David's ACK on the patch it might be fine, but I believe commit message should capture that and explain why the change in behavior is fine.
Quoted 3 lines
After a quick look, all spapr code seems to have the same behavior when nb_numa_nodes==0 and nb_numa_nodes==1, but I'd like to be sure.
That's certainly the intention. If there are cases where it doesn't behave that way, it's a bug - although possible one we have to maintainer for machine compatibility.
Quoted 2 lines
David and/or Tao Xu: do you confirm there's no ABI change at all on spapr after implicitly creating a NUMA node?
I don't believe there is, no.
Quoted 18 lines
Quoted 17 lines
Quoted 16 lines
smc->default_caps.caps[SPAPR_CAP_HTM] = SPAPR_CAP_OFF; smc->default_caps.caps[SPAPR_CAP_VSX] = SPAPR_CAP_ON;include/hw/boards.h +1
-0@@ -220,6 +220,7 @@ struct MachineClass {bool smbus_no_migration_support;bool nvdimm_supported;bool numa_mem_supported; +bool auto_enable_numa; HotplugHandler *(*get_hotplug_handler)(MachineState *machine, DeviceState *dev);
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
Quoted 27 lines · David Gibson
Quoted 22 lines · Eduardo Habkost
Quoted 10 lines · Igor Mammedov
Quoted 7 lines · On Mon, 5 Aug 2019 15:13:02 +0800 Tao Xu <[email protected]> wrote:
Add
MachineClass::auto_enable_numafield. When it is true, a NUMA node is expected to be created implicitly.Acked-by David Gibson <[email protected]>Suggested-by Igor Mammedov <[email protected]>Suggested-by Eduardo Habkost <[email protected]>Signed-off-by Tao Xu <[email protected]>[...]
Quoted 9 lines
Quoted 1 line
+
mc->auto_enable_numa= true;this will always create a numa node (that will affect not only RAM but also all other components that depends on numa state (like CPUs)), where as
spapr_populate_memory()was only faking numa node in DT for RAM. It makes non-numa configuration impossible.
Seeing David's ACK on the patch it might be fine, but I believe commit message should capture that and explain why the change in behavior is fine.Quoted 3 lines
After a quick look, all spapr code seems to have the same behavior when nb_numa_nodes==0 and nb_numa_nodes==1, but I'd like to be sure.
Quoted 8 lines
That's certainly the intention. If there are cases where it doesn't behave that way, it's a bug - although possible one we have to maintainer for machine compatibility.
Quoted 2 lines
David and/or Tao Xu: do you confirm there's no ABI change at all on spapr after implicitly creating a NUMA node?
I don't believe there is, no.
Oh, FWIW, the PAPR interface which is what defines the guest environment has no notion of "non NUMA" except in the sense of a system with exactly one NUMA node.
Signature · 4 lines
David Gibson | I'll have my music baroque, and my code david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_ | _way_ _around_! http://www.ozlabs.org/~dgibson
Igor Mammedov
Reviewed-byQuoted 23 lines · On Thu, 8 Aug 2019 16:35:00 +1000 David Gibson <[email protected]>…
Quoted 22 lines · Eduardo Habkost
Quoted 10 lines · Igor Mammedov
Quoted 7 lines · On Mon, 5 Aug 2019 15:13:02 +0800 Tao Xu <[email protected]> wrote:
Add
MachineClass::auto_enable_numafield. When it is true, a NUMA node is expected to be created implicitly.Acked-by David Gibson <[email protected]>Suggested-by Igor Mammedov <[email protected]>Suggested-by Eduardo Habkost <[email protected]>Signed-off-by Tao Xu <[email protected]>[...]
Quoted 9 lines
Quoted 1 line
+
mc->auto_enable_numa= true;this will always create a numa node (that will affect not only RAM but also all other components that depends on numa state (like CPUs)), where as
spapr_populate_memory()was only faking numa node in DT for RAM. It makes non-numa configuration impossible.
Seeing David's ACK on the patch it might be fine, but I believe commit message should capture that and explain why the change in behavior is fine.
Quoted 7 lines
Quoted 3 lines
After a quick look, all spapr code seems to have the same behavior when nb_numa_nodes==0 and nb_numa_nodes==1, but I'd like to be sure.
That's certainly the intention. If there are cases where it doesn't behave that way, it's a bug - although possible one we have to maintainer for machine compatibility.
considering DT is firmware we typically do not add any compat code for the later.
Quoted 5 lines
Quoted 3 lines
David and/or Tao Xu: do you confirm there's no ABI change at all on spapr after implicitly creating a NUMA node?
I don't believe there is, no.
Also seeing your next reply, it seems there is no non-numa usecase is spec so it would be a bug to begin with, hence:
Quoted 19 lines
Quoted 18 lines
Quoted 17 lines
Quoted 16 lines
smc->default_caps.caps[SPAPR_CAP_HTM] = SPAPR_CAP_OFF; smc->default_caps.caps[SPAPR_CAP_VSX] = SPAPR_CAP_ON;include/hw/boards.h +1
-0@@ -220,6 +220,7 @@ struct MachineClass {bool smbus_no_migration_support;bool nvdimm_supported;bool numa_mem_supported; +bool auto_enable_numa; HotplugHandler *(*get_hotplug_handler)(MachineState *machine, DeviceState *dev);
Quoted 7 lines · Tao Xu
Add
MachineClass::auto_enable_numafield. When it is true, a NUMA node is expected to be created implicitly.Acked-by David Gibson <[email protected]>Suggested-by Igor Mammedov <[email protected]>Suggested-by Eduardo Habkost <[email protected]>Signed-off-by Tao Xu <[email protected]>
This introduces spurious warnings when running qemu-system-ppc64.
See: https://lore.kernel.org/qemu-devel/CAFEAcA-AvFS2cbDH-t5SxgY9hA=LGL81_8dn-vh193vtV9W1Lg@mail.gmail.com/
To reproduce it, just run 'qemu-system-ppc64 -machine pseries' without any -numa arguments.
I have removed this patch from machine-next so it won't block the existing pull request.
Signature · 1 line
Eduardo
Quoted 9 lines · Eduardo Habkost
Quoted 7 lines · Tao Xu
Add
MachineClass::auto_enable_numafield. When it is true, a NUMA node is expected to be created implicitly.Acked-by David Gibson <[email protected]>Suggested-by Igor Mammedov <[email protected]>Suggested-by Eduardo Habkost <[email protected]>Signed-off-by Tao Xu <[email protected]>
Quoted 9 lines
This introduces spurious warnings when running qemu-system-ppc64.
See: https://lore.kernel.org/qemu-devel/CAFEAcA-AvFS2cbDH-t5SxgY9hA=LGL81_8dn-vh193vtV9W1Lg@mail.gmail.com/To reproduce it, just run 'qemu-system-ppc64 -machine pseries' without any -numa arguments.
I have removed this patch from machine-next so it won't block the existing pull request.
I got it. If default splitting of RAM between nodes is
deprecated, this patch can't reuse the splitting code. I agree with droping this patch.
Quoted 16 lines · Tao Xu
Quoted 15 lines · Eduardo Habkost
Quoted 7 lines · Tao Xu
Add
MachineClass::auto_enable_numafield. When it is true, a NUMA node is expected to be created implicitly.Acked-by David Gibson <[email protected]>Suggested-by Igor Mammedov <[email protected]>Suggested-by Eduardo Habkost <[email protected]>Signed-off-by Tao Xu <[email protected]>This introduces spurious warnings when running qemu-system-ppc64.
See: https://lore.kernel.org/qemu-devel/CAFEAcA-AvFS2cbDH-t5SxgY9hA=LGL81_8dn-vh193vtV9W1Lg@mail.gmail.com/To reproduce it, just run 'qemu-system-ppc64 -machine pseries' without any -numa arguments.
Quoted 6 lines
Quoted 3 lines
I have removed this patch from machine-next so it won't block the existing pull request.
I got it. If default splitting of RAM between nodes is deprecated, this patch can't reuse the splitting code. I agree with droping this patch.
Probably all we need to fix this issue is to replace
NumaNodeOptions node = { };
with
NumaNodeOptions node = { .size = ram_size };
in the auto_enable_numa block.
Do you plan to send v2?
Signature · 1 line
Eduardo
Quoted 24 lines · Eduardo Habkost
Quoted 16 lines · Tao Xu
Quoted 15 lines · Eduardo Habkost
Quoted 7 lines · Tao Xu
Add
MachineClass::auto_enable_numafield. When it is true, a NUMA node is expected to be created implicitly.Acked-by David Gibson <[email protected]>Suggested-by Igor Mammedov <[email protected]>Suggested-by Eduardo Habkost <[email protected]>Signed-off-by Tao Xu <[email protected]>This introduces spurious warnings when running qemu-system-ppc64.
See: https://lore.kernel.org/qemu-devel/CAFEAcA-AvFS2cbDH-t5SxgY9hA=LGL81_8dn-vh193vtV9W1Lg@mail.gmail.com/To reproduce it, just run 'qemu-system-ppc64 -machine pseries' without any -numa arguments.
Quoted 6 lines
Quoted 3 lines
I have removed this patch from machine-next so it won't block the existing pull request.
I got it. If default splitting of RAM between nodes is deprecated, this patch can't reuse the splitting code. I agree with droping this patch.
Quoted 8 lines
Probably all we need to fix this issue is to replace
NumaNodeOptions node = { };
with
NumaNodeOptions node = { .size =ram_size};
in theauto_enable_numablock.Do you plan to send v2?
OK, thank you for your suggestion. I will fix it and send v2.