Skip to contentSkip to conversation

numa: Parse initiator= attribute before cpus= attribute

PATCH Applied Michal Privoznik 5 participants 9 messages Jul 7, 2021 – Jul 9, 2021

lore
Share & tools
lore
#0 lore raw

cover numa: Parse initiator= attribute before cpus= attribute

See 2/2 for explanation. The first patch is just cosmetics.

Michal Privoznik (2):
numa: Report expected initiator
numa: Parse initiator= attribute before cpus= attribute

 hw/core/machine.c |  3 ++-
 hw/core/numa.c    | 45 +++++++++++++++++++++++----------------------
 2 files changed, 25 insertions(+), 23 deletions(-)
Signature · 1 line
2.31.1
#1 lore raw

PATCH 1/2 numa: Report expected initiator

When setting up NUMA with HMAT enabled there's a check performed in machine_set_cpu_numa_node() that reports an error when a NUMA node has a CPU but the node's initiator is not itself. The error message reported contains only the expected value and not the actual value (which is different because an error is being reported). Report both values in the error message.

---
 hw/core/machine.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
hw/core/machine.c +2 -1
index 57c18f909a..6f59fb0b7f 100644
@@ -728,7 +728,8 @@ void machine_set_cpu_numa_node(MachineState *machine,
             if ((numa_info[props->node_id].initiator < MAX_NODES) &&
                 (props->node_id != numa_info[props->node_id].initiator)) {
                 error_setg(errp, "The initiator of CPU NUMA node %" PRId64
-                        " should be itself", props->node_id);
+                           " should be itself (got %" PRIu16 ")",
+                           props->node_id, numa_info[props->node_id].initiator);
                 return;
             }
             numa_info[props->node_id].has_cpu = true;
Signature · 1 line
2.31.1
Reviewed-by on v1 1/2
#2 lore raw

Re: [PATCH 1/2] numa: Report expected initiator

Quoted 8 lines · On Wed, 7 Jul 2021 15:40:29 +0200 Michal Privoznik <[email protected]> wrote:

When setting up NUMA with HMAT enabled there's a check performed in machine_set_cpu_numa_node() that reports an error when a NUMA node has a CPU but the node's initiator is not itself. The error message reported contains only the expected value and not the actual value (which is different because an error is being reported). Report both values in the error message.

Quoted 18 lines
---
 hw/core/machine.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
hw/core/machine.c +2 -1
index 57c18f909a..6f59fb0b7f 100644
@@ -728,7 +728,8 @@ void machine_set_cpu_numa_node(MachineState *machine,
             if ((numa_info[props->node_id].initiator < MAX_NODES) &&
                 (props->node_id != numa_info[props->node_id].initiator)) {
                 error_setg(errp, "The initiator of CPU NUMA node %" PRId64
-                        " should be itself", props->node_id);
+                           " should be itself (got %" PRIu16 ")",
+                           props->node_id, numa_info[props->node_id].initiator);
                 return;
             }
             numa_info[props->node_id].has_cpu = true;
Reviewed-by on v1 1/2
#3 lore raw

Re: [PATCH 1/2] numa: Report expected initiator

In reply to Michal Privoznik
Quoted 17 lines

When setting up NUMA with HMAT enabled there's a check performed in machine_set_cpu_numa_node() that reports an error when a NUMA node has a CPU but the node's initiator is not itself. The error message reported contains only the expected value and not the actual value (which is different because an error is being reported). Report both values in the error message.

---
 hw/core/machine.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
hw/core/machine.c +0 -0
index 57c18f909a..6f59fb0b7f 100644

hw/core/machine.c void machine_set_cpu_numa_node(MachineState *machine, · line 735

Quoted 9 lines
hw/core/machine.c +2 -1
@@ -728,7 +728,8 @@ void machine_set_cpu_numa_node(MachineState *machine,
             if ((numa_info[props->node_id].initiator < MAX_NODES) &&
                 (props->node_id != numa_info[props->node_id].initiator)) {
                 error_setg(errp, "The initiator of CPU NUMA node %" PRId64
-                        " should be itself", props->node_id);
+                           " should be itself (got %" PRIu16 ")",
+                           props->node_id, numa_info[props->node_id].initiator);
                 return;
             }
             numa_info[props->node_id].has_cpu = true;
#4 lore raw

PATCH 2/2 numa: Parse initiator= attribute before cpus= attribute

In reply to Michal Privoznik

When parsing cpus= attribute of -numa object couple of checks is performed, such as correct initiator setting (see the if() statement at the end of for() loop in machine_set_cpu_numa_node()).

However, with the current code cpus= attribute is parsed before initiator= attribute and thus the check may fail even though it is not obvious why. But since parsing the initiator= attribute does not depend on the cpus= attribute we can swap the order of the two.

It's fairly easy to reproduce with the following command line (snippet of an actual cmd line):

  -smp 4,sockets=4,cores=1,threads=1 \
  -object '{"qom-type":"memory-backend-ram","id":"ram-node0","size":2147483648}' \
  -numa node,nodeid=0,cpus=0-1,initiator=0,memdev=ram-node0 \
  -object '{"qom-type":"memory-backend-ram","id":"ram-node1","size":2147483648}' \
  -numa node,nodeid=1,cpus=2-3,initiator=1,memdev=ram-node1 \
  -numa hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-latency,latency=5 \
  -numa hmat-lb,initiator=0,target=0,hierarchy=first-level,data-type=access-latency,latency=10 \
  -numa hmat-lb,initiator=1,target=1,hierarchy=memory,data-type=access-latency,latency=5 \
  -numa hmat-lb,initiator=1,target=1,hierarchy=first-level,data-type=access-latency,latency=10 \
  -numa hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-bandwidth,bandwidth=204800K \
  -numa hmat-lb,initiator=0,target=0,hierarchy=first-level,data-type=access-bandwidth,bandwidth=208896K \
  -numa hmat-lb,initiator=1,target=1,hierarchy=memory,data-type=access-bandwidth,bandwidth=204800K \
  -numa hmat-lb,initiator=1,target=1,hierarchy=first-level,data-type=access-bandwidth,bandwidth=208896K \
  -numa hmat-cache,node-id=0,size=10K,level=1,associativity=direct,policy=write-back,line=8 \
  -numa hmat-cache,node-id=1,size=10K,level=1,associativity=direct,policy=write-back,line=8 \
---
 hw/core/numa.c | 45 +++++++++++++++++++++++----------------------
 1 file changed, 23 insertions(+), 22 deletions(-)
hw/core/numa.c +23 -22
index 1058d3697b..510d096a88 100644
@@ -88,6 +88,29 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
         return;
     }
 
+    /*
+     * If not set the initiator, set it to MAX_NODES. And if
+     * HMAT is enabled and this node has no cpus, QEMU will raise error.
+     */
+    numa_info[nodenr].initiator = MAX_NODES;
+    if (node->has_initiator) {
+        if (!ms->numa_state->hmat_enabled) {
+            error_setg(errp, "ACPI Heterogeneous Memory Attribute Table "
+                       "(HMAT) is disabled, enable it with -machine hmat=on "
+                       "before using any of hmat specific options");
+            return;
+        }
+
+        if (node->initiator >= MAX_NODES) {
+            error_report("The initiator id %" PRIu16 " expects an integer "
+                         "between 0 and %d", node->initiator,
+                         MAX_NODES - 1);
+            return;
+        }
+
+        numa_info[nodenr].initiator = node->initiator;
+    }
+
     for (cpus = node->cpus; cpus; cpus = cpus->next) {
         CpuInstanceProperties props;
         if (cpus->value >= max_cpus) {
@@ -142,28 +165,6 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
         numa_info[nodenr].node_memdev = MEMORY_BACKEND(o);
     }
 
-    /*
-     * If not set the initiator, set it to MAX_NODES. And if
-     * HMAT is enabled and this node has no cpus, QEMU will raise error.
-     */
-    numa_info[nodenr].initiator = MAX_NODES;
-    if (node->has_initiator) {
-        if (!ms->numa_state->hmat_enabled) {
-            error_setg(errp, "ACPI Heterogeneous Memory Attribute Table "
-                       "(HMAT) is disabled, enable it with -machine hmat=on "
-                       "before using any of hmat specific options");
-            return;
-        }
-
-        if (node->initiator >= MAX_NODES) {
-            error_report("The initiator id %" PRIu16 " expects an integer "
-                         "between 0 and %d", node->initiator,
-                         MAX_NODES - 1);
-            return;
-        }
-
-        numa_info[nodenr].initiator = node->initiator;
-    }
     numa_info[nodenr].present = true;
     max_numa_nodeid = MAX(max_numa_nodeid, nodenr + 1);
     ms->numa_state->num_nodes++;
Signature · 1 line
2.31.1
Reviewed-by on v1 2/2
#5 lore raw
Quoted 10 lines · On Wed, 7 Jul 2021 15:40:30 +0200 Michal Privoznik <[email protected]> wrote:

When parsing cpus= attribute of -numa object couple of checks is performed, such as correct initiator setting (see the if() statement at the end of for() loop in machine_set_cpu_numa_node()).

However, with the current code cpus= attribute is parsed before initiator= attribute and thus the check may fail even though it is not obvious why. But since parsing the initiator= attribute does not depend on the cpus= attribute we can swap the order of the two.

FYI:
I'm planning to deprecate '-numa node,cpus=' in favor of '-numa cpu'.

Quoted 87 lines

It's fairly easy to reproduce with the following command line (snippet of an actual cmd line):

  -smp 4,sockets=4,cores=1,threads=1 \
  -object '{"qom-type":"memory-backend-ram","id":"ram-node0","size":2147483648}' \
  -numa node,nodeid=0,cpus=0-1,initiator=0,memdev=ram-node0 \
  -object '{"qom-type":"memory-backend-ram","id":"ram-node1","size":2147483648}' \
  -numa node,nodeid=1,cpus=2-3,initiator=1,memdev=ram-node1 \
  -numa hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-latency,latency=5 \
  -numa hmat-lb,initiator=0,target=0,hierarchy=first-level,data-type=access-latency,latency=10 \
  -numa hmat-lb,initiator=1,target=1,hierarchy=memory,data-type=access-latency,latency=5 \
  -numa hmat-lb,initiator=1,target=1,hierarchy=first-level,data-type=access-latency,latency=10 \
  -numa hmat-lb,initiator=0,target=0,hierarchy=memory,data-type=access-bandwidth,bandwidth=204800K \
  -numa hmat-lb,initiator=0,target=0,hierarchy=first-level,data-type=access-bandwidth,bandwidth=208896K \
  -numa hmat-lb,initiator=1,target=1,hierarchy=memory,data-type=access-bandwidth,bandwidth=204800K \
  -numa hmat-lb,initiator=1,target=1,hierarchy=first-level,data-type=access-bandwidth,bandwidth=208896K \
  -numa hmat-cache,node-id=0,size=10K,level=1,associativity=direct,policy=write-back,line=8 \
  -numa hmat-cache,node-id=1,size=10K,level=1,associativity=direct,policy=write-back,line=8 \
---
 hw/core/numa.c | 45 +++++++++++++++++++++++----------------------
 1 file changed, 23 insertions(+), 22 deletions(-)
hw/core/numa.c +23 -22
index 1058d3697b..510d096a88 100644
@@ -88,6 +88,29 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
         return;
     }
 
+    /*
+     * If not set the initiator, set it to MAX_NODES. And if
+     * HMAT is enabled and this node has no cpus, QEMU will raise error.
+     */
+    numa_info[nodenr].initiator = MAX_NODES;
+    if (node->has_initiator) {
+        if (!ms->numa_state->hmat_enabled) {
+            error_setg(errp, "ACPI Heterogeneous Memory Attribute Table "
+                       "(HMAT) is disabled, enable it with -machine hmat=on "
+                       "before using any of hmat specific options");
+            return;
+        }
+
+        if (node->initiator >= MAX_NODES) {
+            error_report("The initiator id %" PRIu16 " expects an integer "
+                         "between 0 and %d", node->initiator,
+                         MAX_NODES - 1);
+            return;
+        }
+
+        numa_info[nodenr].initiator = node->initiator;
+    }
+
     for (cpus = node->cpus; cpus; cpus = cpus->next) {
         CpuInstanceProperties props;
         if (cpus->value >= max_cpus) {
@@ -142,28 +165,6 @@ static void parse_numa_node(MachineState *ms, NumaNodeOptions *node,
         numa_info[nodenr].node_memdev = MEMORY_BACKEND(o);
     }
 
-    /*
-     * If not set the initiator, set it to MAX_NODES. And if
-     * HMAT is enabled and this node has no cpus, QEMU will raise error.
-     */
-    numa_info[nodenr].initiator = MAX_NODES;
-    if (node->has_initiator) {
-        if (!ms->numa_state->hmat_enabled) {
-            error_setg(errp, "ACPI Heterogeneous Memory Attribute Table "
-                       "(HMAT) is disabled, enable it with -machine hmat=on "
-                       "before using any of hmat specific options");
-            return;
-        }
-
-        if (node->initiator >= MAX_NODES) {
-            error_report("The initiator id %" PRIu16 " expects an integer "
-                         "between 0 and %d", node->initiator,
-                         MAX_NODES - 1);
-            return;
-        }
-
-        numa_info[nodenr].initiator = node->initiator;
-    }
     numa_info[nodenr].present = true;
     max_numa_nodeid = MAX(max_numa_nodeid, nodenr + 1);
     ms->numa_state->num_nodes++;
#6 lore raw
Quoted 14 lines · Igor Mammedov
Quoted 10 lines · On Wed, 7 Jul 2021 15:40:30 +0200 Michal Privoznik <[email protected]> wrote:

When parsing cpus= attribute of -numa object couple of checks is performed, such as correct initiator setting (see the if() statement at the end of for() loop in machine_set_cpu_numa_node()).

However, with the current code cpus= attribute is parsed before initiator= attribute and thus the check may fail even though it is not obvious why. But since parsing the initiator= attribute does not depend on the cpus= attribute we can swap the order of the two.

Quoted 4 lines

FYI:
I'm planning to deprecate '-numa node,cpus=' in favor of '-numa cpu'.

Yes, I'm aware of that and I remember I posted some patches for libvirt that you reviewed. I need to return to them rather sooner than later. Meanwhile, this patch helps.

Thanks,
Michal

Reviewed-by on the v1 series
#7 lore raw
In reply to Michal Privoznik
Quoted 6 lines · Michal Privoznik

See 2/2 for explanation. The first patch is just cosmetics.

Michal Privoznik (2):
numa: Report expected initiator
numa: Parse initiator= attribute before cpus= attribute

Quoted 6 lines

hw/core/machine.c | 3 ++-
hw/core/numa.c | 45 +++++++++++++++++++++++----------------------

2 files changed, 25 insertions(+), 23 deletions(-)

--
2.31.1

For the series:

#8 lore raw
In reply to Michal Privoznik
Quoted 5 lines · Michal Privoznik

See 2/2 for explanation. The first patch is just cosmetics.

Michal Privoznik (2):
numa: Report expected initiator
numa: Parse initiator= attribute before cpus= attribute

Queued, thanks!

Signature · 1 line
Eduardo