Skip to contentSkip to conversation

numa: s390x has no NUMA

PATCH In review David Hildenbrand 4 participants 10 messages Feb 23, 2018 – Feb 26, 2018

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 numa: s390x has no NUMA

Right now it is possible to crash QEMU for s390x by providing e.g.

    -numa node,nodeid=0,cpus=0-1

Problem is, that numa.c uses mc->cpu_index_to_instance_props as an indicator whether NUMA is supported by a machine type. We don't implement NUMA on s390x (and that concept also doesn't really exist). We need mc->cpu_index_to_instance_props for query-cpus.

So let's fix this case.

qemu-system-s390x: -numa node,nodeid=0,cpus=0-1: NUMA is not supported by
this machine-type

---
 numa.c | 6 ++++++
 1 file changed, 6 insertions(+)
numa.c +6 -0
index 7e0e789b02..3b9be613d9 100644
@@ -80,10 +80,16 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
         return;
     }
 
+#ifdef TARGET_S390X
+    /* s390x provides cpu_index_to_instance_props but has no NUMA */
+    error_report("NUMA is not supported by this machine-type");
+    exit(1);
+#else
     if (!mc->cpu_index_to_instance_props) {
         error_report("NUMA is not supported by this machine-type");
         exit(1);
     }
+#endif
     for (cpus = node->cpus; cpus; cpus = cpus->next) {
         CpuInstanceProperties props;
         if (cpus->value >= max_cpus) {
Signature · 1 line
2.14.3
Reviewed-by on v1 1/1
#1 lore raw
Quoted 29 lines · On Fri, 23 Feb 2018 18:36:57 +0100 David Hildenbrand <[email protected]> wrote:

Right now it is possible to crash QEMU for s390x by providing e.g.

    -numa node,nodeid=0,cpus=0-1

Problem is, that numa.c uses mc->cpu_index_to_instance_props as an indicator whether NUMA is supported by a machine type. We don't implement NUMA on s390x (and that concept also doesn't really exist). We need mc->cpu_index_to_instance_props for query-cpus.

So let's fix this case.

qemu-system-s390x: -numa node,nodeid=0,cpus=0-1: NUMA is not supported by this machine-type

---
 numa.c | 6 ++++++
 1 file changed, 6 insertions(+)
numa.c +3 -0
index 7e0e789b02..3b9be613d9 100644
@@ -80,10 +80,16 @@ static void parse_numa_node(MachineState *ms,
 NumaNodeOptions *node, return;
     }
 
+#ifdef TARGET_S390X
+    /* s390x provides cpu_index_to_instance_props but has no NUMA */
+    error_report("NUMA is not supported by this machine-type");

numa.c static void parse_numa_node(MachineState *ms, · line 95

Quoted 10 lines
numa.c +3 -0
+    exit(1);
+#else
     if (!mc->cpu_index_to_instance_props) {
         error_report("NUMA is not supported by this machine-type");
         exit(1);
     }
+#endif
     for (cpus = node->cpus; cpus; cpus = cpus->next) {
         CpuInstanceProperties props;
         if (cpus->value >= max_cpus) {

seems straightforward

#2 lore raw
In reply to David Hildenbrand
Quoted 7 lines · David Hildenbrand

Right now it is possible to crash QEMU for s390x by providing e.g.

    -numa node,nodeid=0,cpus=0-1

Problem is, that numa.c uses mc->cpu_index_to_instance_props as an indicator whether NUMA is supported by a machine type. We don't implement NUMA on s390x (and that concept also doesn't really exist). We need mc->cpu_index_to_instance_props for query-cpus.

Looks like we assert because of
machine->possible_cpus == 0.

Later during boot this is created in s390_possible_cpu_arch_ids. (via s390_init_cpus). What we (in the future) actually could provide is a cpu topology.

So something like this also fixes the bug

hw/s390x/s390-virtio-ccw.c +11 -0
index fd5bfcdaa5..d981335ca9 100644
@@ -13,6 +13,7 @@
 #include "qemu/osdep.h"
 #include "qapi/error.h"
 #include "cpu.h"
+#include "sysemu/numa.h"
 #include "hw/boards.h"
 #include "exec/address-spaces.h"
 #include "hw/s390x/s390-virtio-hcall.h"
@@ -393,11 +394,20 @@ static void s390_machine_device_unplug_request(HotplugHandler *hotplug_dev,
 static CpuInstanceProperties s390_cpu_index_to_props(MachineState *machine,
                                                      unsigned cpu_index)
 {
+    MachineClass *mc = MACHINE_GET_CLASS(machine);
+
+    /* make sure possible_cpu are intialized */
+    mc->possible_cpu_arch_ids(machine);
     g_assert(machine->possible_cpus && cpu_index < machine->possible_cpus->len);
 
     return machine->possible_cpus->cpus[cpu_index].props;
 }
 
+static int64_t s390_get_default_cpu_node_id(const MachineState *ms, int idx)
+{
+    return idx / smp_cpus % nb_numa_nodes;
+}
+
 static const CPUArchIdList *s390_possible_cpu_arch_ids(MachineState *ms)
 {
     int i;
@@ -473,6 +483,7 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
     mc->get_hotplug_handler = s390_get_hotplug_handler;
     mc->cpu_index_to_instance_props = s390_cpu_index_to_props;
     mc->possible_cpu_arch_ids = s390_possible_cpu_arch_ids;
+    mc->get_default_cpu_node_id = s390_get_default_cpu_node_id;
     /* it is overridden with 'host' cpu *in kvm_arch_init* */
     mc->default_cpu_type = S390_CPU_TYPE_NAME("qemu");
     hc->plug = s390_machine_device_plug;

and it would allow us to extend things later on. On the other hand, my fix does not implement anything so your fix is "more correct".

Quoted 33 lines

So let's fix this case.

qemu-system-s390x: -numa node,nodeid=0,cpus=0-1: NUMA is not supported by
this machine-type

---
 numa.c | 6 ++++++
 1 file changed, 6 insertions(+)
numa.c +6 -0
index 7e0e789b02..3b9be613d9 100644
@@ -80,10 +80,16 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
         return;
     }
 
+#ifdef TARGET_S390X
+    /* s390x provides cpu_index_to_instance_props but has no NUMA */
+    error_report("NUMA is not supported by this machine-type");
+    exit(1);
+#else
     if (!mc->cpu_index_to_instance_props) {
         error_report("NUMA is not supported by this machine-type");
         exit(1);
     }
+#endif
     for (cpus = node->cpus; cpus; cpus = cpus->next) {
         CpuInstanceProperties props;
         if (cpus->value >= max_cpus) {
#3 lore raw
Quoted 11 lines · Christian Borntraeger
Quoted 7 lines · David Hildenbrand

Right now it is possible to crash QEMU for s390x by providing e.g.

    -numa node,nodeid=0,cpus=0-1

Problem is, that numa.c uses mc->cpu_index_to_instance_props as an indicator whether NUMA is supported by a machine type. We don't implement NUMA on s390x (and that concept also doesn't really exist). We need mc->cpu_index_to_instance_props for query-cpus.

Quoted 8 lines

Looks like we assert because of
machine->possible_cpus == 0.

Later during boot this is created in s390_possible_cpu_arch_ids. (via s390_init_cpus). What we (in the future) actually could provide is a cpu topology.

So something like this also fixes the bug

Yes, but I decided to not go this way because we don't support NUMA as of now. -numa has to bail out (just as it did before I implemented proper query-cpus support).

What you propose is something for future support - one we have cpu topology information exposed.

Signature · 3 lines
Thanks,

David / dhildenb
#4 lore raw
In reply to David Hildenbrand
Quoted 7 lines · On Fri, 23 Feb 2018 18:36:57 +0100 David Hildenbrand <[email protected]> wrote:

Right now it is possible to crash QEMU for s390x by providing e.g.

    -numa node,nodeid=0,cpus=0-1

Problem is, that numa.c uses mc->cpu_index_to_instance_props as an indicator whether NUMA is supported by a machine type. We don't implement NUMA on s390x (and that concept also doesn't really exist). We need mc->cpu_index_to_instance_props for query-cpus.

Is existence of cpu_index_to_instance_probs the correct indicator for numa, then?

OTOH, your patch is straightforward...

Quoted 32 lines

So let's fix this case.

qemu-system-s390x: -numa node,nodeid=0,cpus=0-1: NUMA is not supported by
this machine-type

---
 numa.c | 6 ++++++
 1 file changed, 6 insertions(+)
numa.c +6 -0
index 7e0e789b02..3b9be613d9 100644
@@ -80,10 +80,16 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
         return;
     }
 
+#ifdef TARGET_S390X
+    /* s390x provides cpu_index_to_instance_props but has no NUMA */
+    error_report("NUMA is not supported by this machine-type");
+    exit(1);
+#else
     if (!mc->cpu_index_to_instance_props) {
         error_report("NUMA is not supported by this machine-type");
         exit(1);
     }
+#endif
     for (cpus = node->cpus; cpus; cpus = cpus->next) {
         CpuInstanceProperties props;
         if (cpus->value >= max_cpus) {
#5 lore raw
Quoted 11 lines · Cornelia Huck
Quoted 7 lines · On Fri, 23 Feb 2018 18:36:57 +0100 David Hildenbrand <[email protected]> wrote:

Right now it is possible to crash QEMU for s390x by providing e.g.

    -numa node,nodeid=0,cpus=0-1

Problem is, that numa.c uses mc->cpu_index_to_instance_props as an indicator whether NUMA is supported by a machine type. We don't implement NUMA on s390x (and that concept also doesn't really exist). We need mc->cpu_index_to_instance_props for query-cpus.

Quoted 4 lines

Is existence of cpu_index_to_instance_probs the correct indicator for numa, then?

OTOH, your patch is straightforward...

Maybe it is get_default_cpu_node_id as Christian discovered?

Signature · 3 lines
Thanks,

David / dhildenb
#6 lore raw
Quoted 12 lines · On Mon, 26 Feb 2018 11:28:26 +0100 David Hildenbrand <[email protected]> wrote:
Quoted 11 lines · Cornelia Huck
Quoted 7 lines · On Fri, 23 Feb 2018 18:36:57 +0100 David Hildenbrand <[email protected]> wrote:

Right now it is possible to crash QEMU for s390x by providing e.g.

    -numa node,nodeid=0,cpus=0-1

Problem is, that numa.c uses mc->cpu_index_to_instance_props as an indicator whether NUMA is supported by a machine type. We don't implement NUMA on s390x (and that concept also doesn't really exist). We need mc->cpu_index_to_instance_props for query-cpus.

Quoted 6 lines
Quoted 4 lines

Is existence of cpu_index_to_instance_probs the correct indicator for numa, then?

OTOH, your patch is straightforward...

Maybe it is get_default_cpu_node_id as Christian discovered?

Yes, that seems like a better candidate for checking.

#7 lore raw
Quoted 15 lines · Cornelia Huck
Quoted 12 lines · On Mon, 26 Feb 2018 11:28:26 +0100 David Hildenbrand <[email protected]> wrote:
Quoted 11 lines · Cornelia Huck
Quoted 7 lines · On Fri, 23 Feb 2018 18:36:57 +0100 David Hildenbrand <[email protected]> wrote:

Right now it is possible to crash QEMU for s390x by providing e.g.

    -numa node,nodeid=0,cpus=0-1

Problem is, that numa.c uses mc->cpu_index_to_instance_props as an indicator whether NUMA is supported by a machine type. We don't implement NUMA on s390x (and that concept also doesn't really exist). We need mc->cpu_index_to_instance_props for query-cpus.

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

Is existence of cpu_index_to_instance_probs the correct indicator for numa, then?

OTOH, your patch is straightforward...

Maybe it is get_default_cpu_node_id as Christian discovered?

Yes, that seems like a better candidate for checking.

Agreed.
As everybody else calls possible_cpu_arch_ids in cpu_index_to_props I am asking myself if we should do that as well anyway?

#8 lore raw
Quoted 23 lines · Christian Borntraeger
Quoted 20 lines · Cornelia Huck
Quoted 17 lines · On Mon, 26 Feb 2018 11:28:26 +0100 David Hildenbrand <[email protected]> wrote:
Quoted 15 lines · Cornelia Huck
Quoted 7 lines · On Fri, 23 Feb 2018 18:36:57 +0100 David Hildenbrand <[email protected]> wrote:

Right now it is possible to crash QEMU for s390x by providing e.g.

    -numa node,nodeid=0,cpus=0-1

Problem is, that numa.c uses mc->cpu_index_to_instance_props as an indicator whether NUMA is supported by a machine type. We don't implement NUMA on s390x (and that concept also doesn't really exist). We need mc->cpu_index_to_instance_props for query-cpus.

Is existence of cpu_index_to_instance_probs the correct indicator for numa, then?

OTOH, your patch is straightforward...

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Maybe it is get_default_cpu_node_id as Christian discovered?

Yes, that seems like a better candidate for checking.

Agreed.
As everybody else calls possible_cpu_arch_ids in cpu_index_to_props I am asking myself if we should do that as well anyway?

Well, it found a BUG :)

Signature · 3 lines
Thanks,

David / dhildenb
#9 lore raw
In reply to Christian Borntraeger
Quoted 21 lines · On Mon, 26 Feb 2018 12:07:43 +0100 Christian Borntraeger…
Quoted 20 lines · Cornelia Huck
Quoted 17 lines · On Mon, 26 Feb 2018 11:28:26 +0100 David Hildenbrand <[email protected]> wrote:
Quoted 15 lines · Cornelia Huck
Quoted 7 lines · On Fri, 23 Feb 2018 18:36:57 +0100 David Hildenbrand <[email protected]> wrote:

Right now it is possible to crash QEMU for s390x by providing e.g.

    -numa node,nodeid=0,cpus=0-1

Problem is, that numa.c uses mc->cpu_index_to_instance_props as an indicator whether NUMA is supported by a machine type. We don't implement NUMA on s390x (and that concept also doesn't really exist). We need mc->cpu_index_to_instance_props for query-cpus.

Is existence of cpu_index_to_instance_probs the correct indicator for numa, then?

OTOH, your patch is straightforward...

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Maybe it is get_default_cpu_node_id as Christian discovered?

Yes, that seems like a better candidate for checking.

Agreed.
As everybody else calls possible_cpu_arch_ids in cpu_index_to_props I am asking myself if we should do that as well anyway?

Making the behaviour consistent with other archs sounds like a good idea.