Skip to contentSkip to conversation

Introduce cpu die topology and enable CPUID.1F for i386

PATCH New Like Xu 1 participant 4 messages Jun 20, 2019

lore
Share & tools
lore

Like Xu

author
#0 lore raw

v4 cover Introduce cpu die topology and enable CPUID.1F for i386

This patch series introduces a new cpu topolgy 'die' for PCMachine, which extends virtual cpu topology to the socket/die/core/thread model, allowing the setting of dies number per one socket via -smp qemu command.

For i386, it upgrades APIC-IDs generation and reversion functions with a new exposed leaf called CPUID.1F, which is a preferred superset to leaf 0BH. The CPUID.1F spec is on the latest Inetl SDM, 3-190 Vol 2A.

Guest system could discover multi-die/package topology through CPUID.1F. and its benefit is primarily for _reporting_ of the guest CPU topology. The guest kernel with multi-die/package support have no impact on its cache topology, NUMA topology, Linux scheduler, or system performance.

==changelog==

v4:

- base commit: 22fa84da on github.com/ehabkost/qemu.git:machine-next
- refine comments for pc_smp_parse()
- remove the use of cpu->enable_cpuid_0x1f
- apply new logic for cpuid_min_level adjustment and drop the legacy one
- refine the way of MachineState casting in pc_smp_parse()
- [QUEUED] move test_topo_bits to the previous patch for bisectability

v3: https://patchwork.kernel.org/cover/10989013/

- add a MachineClass::smp_parse function pointer
- place the PC-specific function inside hw/i386/pc.c
- introduce die_id in a separate patch with default value 0
- set env->nr_dies in pc_new_cpu() and pc_cpu_pre_plug()
- fix a circular dependency between target/i386/cpu.c and hw/i386/pc.c
- fix cpu->die_id check in pc_cpu_pre_plug()
- Based on "[PATCH v3 00/10] Refactor cpu topo into machine properties"
- Rebase to commit 219dca61ebf41625831d4f96a720852baf44b762

v2: https://patchwork.kernel.org/cover/10953191/

- Enable cpu die-level topolgy only for PCMachine and X86CPU
- Minimize cpuid.0.eax to the setting value actually used by guest
- Update cmd line -smps docs for die-level configurations
- Refactoring topo-bit tests for x86_apicid_from_cpu_idx() with nr_dies
- Based on "[PATCH v3 00/10] Refactor cpu topo into machine properties"
- Rebase to commit 2259637b95bef3116cc262459271de08e038cc66

v1: https://patchwork.kernel.org/cover/10876667/

Like Xu (3):
target/i386: Add CPUID.1F generation support for multi-dies PCMachine
machine: Refactor smp_parse() in vl.c as MachineClass::smp_parse() vl.c: Add -smp, dies=* command line support and update doc

 hw/core/machine.c    | 76 ++++++++++++++++++++++++++++++++++++++++
 hw/i386/pc.c         | 83 ++++++++++++++++++++++++++++++++++++++++++++
 include/hw/boards.h  |  5 +++
 include/hw/i386/pc.h |  1 +
 qemu-options.hx      | 17 ++++-----
 target/i386/cpu.c    | 41 ++++++++++++++++++++++
 target/i386/cpu.h    |  1 +
 target/i386/kvm.c    | 12 +++++++
 vl.c                 | 78 +++--------------------------------------
 9 files changed, 233 insertions(+), 81 deletions(-)
Signature · 1 line
2.21.0
author
#1 lore raw

PATCH v4 1/3 target/i386: Add CPUID.1F generation support for multi-dies PCMachine

The CPUID.1F as Intel V2 Extended Topology Enumeration Leaf would be exposed if guests want to emulate multiple software-visible die within each package. Per Intel's SDM, the 0x1f is a superset of 0xb, thus they can be generated by almost same code as 0xb except die_offset setting.

If the number of dies per package is greater than 1, the cpuid_min_level would be adjusted to 0x1f regardless of whether the host supports CPUID.1F. Likewise, the CPUID.1F wouldn't be exposed if env->nr_dies < 2.

---
 target/i386/cpu.c | 41 +++++++++++++++++++++++++++++++++++++++++
 target/i386/cpu.h |  1 +
 target/i386/kvm.c | 12 ++++++++++++
 3 files changed, 54 insertions(+)
target/i386/cpu.c +41 -0
index 88908a6373..efcbe6a2b2 100644
@@ -4439,6 +4439,42 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
             *ecx |= CPUID_TOPOLOGY_LEVEL_INVALID;
         }
 
+        assert(!(*eax & ~0x1f));
+        *ebx &= 0xffff; /* The count doesn't need to be reliable. */
+        break;
+    case 0x1F:
+        /* V2 Extended Topology Enumeration Leaf */
+        if (env->nr_dies < 2) {
+            *eax = *ebx = *ecx = *edx = 0;
+            break;
+        }
+
+        *ecx = count & 0xff;
+        *edx = cpu->apic_id;
+        switch (count) {
+        case 0:
+            *eax = apicid_core_offset(env->nr_dies, cs->nr_cores,
+                                                    cs->nr_threads);
+            *ebx = cs->nr_threads;
+            *ecx |= CPUID_TOPOLOGY_LEVEL_SMT;
+            break;
+        case 1:
+            *eax = apicid_die_offset(env->nr_dies, cs->nr_cores,
+                                                   cs->nr_threads);
+            *ebx = cs->nr_cores * cs->nr_threads;
+            *ecx |= CPUID_TOPOLOGY_LEVEL_CORE;
+            break;
+        case 2:
+            *eax = apicid_pkg_offset(env->nr_dies, cs->nr_cores,
+                                                   cs->nr_threads);
+            *ebx = env->nr_dies * cs->nr_cores * cs->nr_threads;
+            *ecx |= CPUID_TOPOLOGY_LEVEL_DIE;
+            break;
+        default:
+            *eax = 0;
+            *ebx = 0;
+            *ecx |= CPUID_TOPOLOGY_LEVEL_INVALID;
+        }
         assert(!(*eax & ~0x1f));
         *ebx &= 0xffff; /* The count doesn't need to be reliable. */
         break;
@@ -5116,6 +5152,11 @@ static void x86_cpu_expand_features(X86CPU *cpu, Error **errp)
             x86_cpu_adjust_level(cpu, &cpu->env.cpuid_min_level, 0x14);
         }
 
+        /* CPU topology with multi-dies support requires CPUID[0x1F] */
+        if (env->nr_dies > 1) {
+            x86_cpu_adjust_level(cpu, &env->cpuid_min_level, 0x1F);
+        }
+
         /* SVM requires CPUID[0x8000000A] */
         if (env->features[FEAT_8000_0001_ECX] & CPUID_EXT3_SVM) {
             x86_cpu_adjust_level(cpu, &env->cpuid_min_xlevel, 0x8000000A);
target/i386/cpu.h +1 -0
index 46dd81f6b7..eec6e4b7b7 100644
@@ -726,6 +726,7 @@ typedef uint32_t FeatureWordArray[FEATURE_WORDS];
 #define CPUID_TOPOLOGY_LEVEL_INVALID  (0U << 8)
 #define CPUID_TOPOLOGY_LEVEL_SMT      (1U << 8)
 #define CPUID_TOPOLOGY_LEVEL_CORE     (2U << 8)
+#define CPUID_TOPOLOGY_LEVEL_DIE      (5U << 8)
 
 /* MSR Feature Bits */
 #define MSR_ARCH_CAP_RDCL_NO    (1U << 0)
target/i386/kvm.c +12 -0
index 6899061b4e..5deb4248ac 100644
@@ -1080,6 +1080,10 @@ int kvm_arch_init_vcpu(CPUState *cs)
             }
             break;
         }
+        case 0x1f:
+            if (env->nr_dies < 2) {
+                break;
+            }
         case 4:
         case 0xb:
         case 0xd:
@@ -1087,6 +1091,11 @@ int kvm_arch_init_vcpu(CPUState *cs)
                 if (i == 0xd && j == 64) {
                     break;
                 }
+
+                if (i == 0x1f && j == 64) {
+                    break;
+                }
+
                 c->function = i;
                 c->flags = KVM_CPUID_FLAG_SIGNIFCANT_INDEX;
                 c->index = j;
@@ -1098,6 +1107,9 @@ int kvm_arch_init_vcpu(CPUState *cs)
                 if (i == 0xb && !(c->ecx & 0xff00)) {
                     break;
                 }
+                if (i == 0x1f && !(c->ecx & 0xff00)) {
+                    break;
+                }
                 if (i == 0xd && c->eax == 0) {
                     continue;
                 }
Signature · 1 line
2.21.0
author
#2 lore raw

PATCH v4 2/3 machine: Refactor smp_parse() in vl.c as MachineClass::smp_parse()

In reply to Like Xu

To make smp_parse() more flexible and expansive, a smp_parse function pointer is added to MachineClass that machine types could override.

The generic smp_parse() code in vl.c is moved to hw/core/machine.c, and become the default implementation of MachineClass::smp_parse. A PC-specific function called pc_smp_parse() has been added to hw/i386/pc.c, which in this patch changes nothing against the default one .

---
 hw/core/machine.c    | 76 ++++++++++++++++++++++++++++++++++++++++++
 hw/i386/pc.c         | 79 ++++++++++++++++++++++++++++++++++++++++++++
 include/hw/boards.h  |  5 +++
 include/hw/i386/pc.h |  1 +
 vl.c                 | 75 ++---------------------------------------
 5 files changed, 163 insertions(+), 73 deletions(-)
hw/core/machine.c +76 -0
index 8b8d263afe..36a838f1cb 100644
@@ -11,6 +11,9 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu/option.h"
+#include "qapi/qmp/qerror.h"
+#include "sysemu/replay.h"
 #include "qemu/units.h"
 #include "hw/boards.h"
 #include "qapi/error.h"
@@ -728,6 +731,78 @@ void machine_set_cpu_numa_node(MachineState *machine,
     }
 }
 
+static void smp_parse(MachineState *ms, QemuOpts *opts)
+{
+    if (opts) {
+        unsigned cpus    = qemu_opt_get_number(opts, "cpus", 0);
+        unsigned sockets = qemu_opt_get_number(opts, "sockets", 0);
+        unsigned cores   = qemu_opt_get_number(opts, "cores", 0);
+        unsigned threads = qemu_opt_get_number(opts, "threads", 0);
+
+        /* compute missing values, prefer sockets over cores over threads */
+        if (cpus == 0 || sockets == 0) {
+            cores = cores > 0 ? cores : 1;
+            threads = threads > 0 ? threads : 1;
+            if (cpus == 0) {
+                sockets = sockets > 0 ? sockets : 1;
+                cpus = cores * threads * sockets;
+            } else {
+                ms->smp.max_cpus =
+                        qemu_opt_get_number(opts, "maxcpus", cpus);
+                sockets = ms->smp.max_cpus / (cores * threads);
+            }
+        } else if (cores == 0) {
+            threads = threads > 0 ? threads : 1;
+            cores = cpus / (sockets * threads);
+            cores = cores > 0 ? cores : 1;
+        } else if (threads == 0) {
+            threads = cpus / (cores * sockets);
+            threads = threads > 0 ? threads : 1;
+        } else if (sockets * cores * threads < cpus) {
+            error_report("cpu topology: "
+                         "sockets (%u) * cores (%u) * threads (%u) < "
+                         "smp_cpus (%u)",
+                         sockets, cores, threads, cpus);
+            exit(1);
+        }
+
+        ms->smp.max_cpus =
+                qemu_opt_get_number(opts, "maxcpus", cpus);
+
+        if (ms->smp.max_cpus < cpus) {
+            error_report("maxcpus must be equal to or greater than smp");
+            exit(1);
+        }
+
+        if (sockets * cores * threads > ms->smp.max_cpus) {
+            error_report("cpu topology: "
+                         "sockets (%u) * cores (%u) * threads (%u) > "
+                         "maxcpus (%u)",
+                         sockets, cores, threads,
+                         ms->smp.max_cpus);
+            exit(1);
+        }
+
+        if (sockets * cores * threads != ms->smp.max_cpus) {
+            warn_report("Invalid CPU topology deprecated: "
+                        "sockets (%u) * cores (%u) * threads (%u) "
+                        "!= maxcpus (%u)",
+                        sockets, cores, threads,
+                        ms->smp.max_cpus);
+        }
+
+        ms->smp.cpus = cpus;
+        ms->smp.cores = cores;
+        ms->smp.threads = threads;
+    }
+
+    if (ms->smp.cpus > 1) {
+        Error *blocker = NULL;
+        error_setg(&blocker, QERR_REPLAY_NOT_SUPPORTED, "smp");
+        replay_add_blocker(blocker);
+    }
+}
+
 static void machine_class_init(ObjectClass *oc, void *data)
 {
     MachineClass *mc = MACHINE_CLASS(oc);
@@ -735,6 +810,7 @@ static void machine_class_init(ObjectClass *oc, void *data)
     /* Default 128 MB as guest ram size */
     mc->default_ram_size = 128 * MiB;
     mc->rom_file_has_mr = true;
+    mc->smp_parse = smp_parse;
 
     /* numa node memory size aligned on 8MB by default.
      * On Linux, each node's border has to be 8MB aligned
hw/i386/pc.c +79 -0

97 lines not on this page · raw message

include/hw/boards.h +5 -0
index 04f52c1335..6dd0761562 100644
@@ -159,6 +159,10 @@ typedef struct {
  * @kvm_type:
  *    Return the type of KVM corresponding to the kvm-type string option or
  *    computed based on other criteria such as the host kernel capabilities.
+ * @smp_parse:
+ *    The function pointer to hook different machine specific functions for
+ *    parsing "smp-opts" from QemuOpts to MachineState::CpuTopology and more
+ *    machine specific topology fields, such as smp_dies for PCMachine.
  */
 struct MachineClass {
     /*< private >*/
@@ -175,6 +179,7 @@ struct MachineClass {
     void (*reset)(MachineState *state);
     void (*hot_add_cpu)(MachineState *state, const int64_t id, Error **errp);
     int (*kvm_type)(MachineState *machine, const char *arg);
+    void (*smp_parse)(MachineState *ms, QemuOpts *opts);
 
     BlockInterfaceType block_default_type;
     int units_per_default_bus;
include/hw/i386/pc.h +1 -0
index 2ae86e202d..e96e449cef 100644
@@ -189,6 +189,7 @@ void pc_acpi_smi_interrupt(void *opaque, int irq, int level);
 
 void pc_cpus_init(PCMachineState *pcms);
 void pc_hot_add_cpu(MachineState *ms, const int64_t id, Error **errp);
+void pc_smp_parse(MachineState *ms, QemuOpts *opts);
 
 void pc_guest_info_init(PCMachineState *pcms);
 
vl.c +2 -73

87 lines not on this page · raw message

Signature · 1 line
2.21.0
author
#3 lore raw

PATCH v4 3/3 vl.c: Add -smp, dies=* command line support and update doc

In reply to Like Xu

For PC target, users could configure the number of dies per one package via command line with this patch, such as "-smp dies=2,cores=4".

The parsing rules of new cpu-topology model obey the same restrictions/logic as the legacy socket/core/thread model especially on missing values computing.

---
 hw/i386/pc.c    | 30 +++++++++++++++++-------------
 qemu-options.hx | 17 +++++++++--------
 vl.c            |  3 +++
 3 files changed, 29 insertions(+), 21 deletions(-)
hw/i386/pc.c +17 -13
index 092bd10d4d..2ed1b3f8de 100644
@@ -1548,9 +1548,12 @@ static void pc_new_cpu(PCMachineState *pcms, int64_t apic_id, Error **errp)
  */
 void pc_smp_parse(MachineState *ms, QemuOpts *opts)
 {
+    PCMachineState *pcms = PC_MACHINE(ms);
+
     if (opts) {
         unsigned cpus    = qemu_opt_get_number(opts, "cpus", 0);
         unsigned sockets = qemu_opt_get_number(opts, "sockets", 0);
+        unsigned dies = qemu_opt_get_number(opts, "dies", 1);
         unsigned cores   = qemu_opt_get_number(opts, "cores", 0);
         unsigned threads = qemu_opt_get_number(opts, "threads", 0);
 
@@ -1560,24 +1563,24 @@ void pc_smp_parse(MachineState *ms, QemuOpts *opts)
             threads = threads > 0 ? threads : 1;
             if (cpus == 0) {
                 sockets = sockets > 0 ? sockets : 1;
-                cpus = cores * threads * sockets;
+                cpus = cores * threads * dies * sockets;
             } else {
                 ms->smp.max_cpus =
                         qemu_opt_get_number(opts, "maxcpus", cpus);
-                sockets = ms->smp.max_cpus / (cores * threads);
+                sockets = ms->smp.max_cpus / (cores * threads * dies);
             }
         } else if (cores == 0) {
             threads = threads > 0 ? threads : 1;
-            cores = cpus / (sockets * threads);
+            cores = cpus / (sockets * dies * threads);
             cores = cores > 0 ? cores : 1;
         } else if (threads == 0) {
-            threads = cpus / (cores * sockets);
+            threads = cpus / (cores * dies * sockets);
             threads = threads > 0 ? threads : 1;
-        } else if (sockets * cores * threads < cpus) {
+        } else if (sockets * dies * cores * threads < cpus) {
             error_report("cpu topology: "
-                         "sockets (%u) * cores (%u) * threads (%u) < "
+                         "sockets (%u) * dies (%u) * cores (%u) * threads (%u) < "
                          "smp_cpus (%u)",
-                         sockets, cores, threads, cpus);
+                         sockets, dies, cores, threads, cpus);
             exit(1);
         }
 
@@ -1589,26 +1592,27 @@ void pc_smp_parse(MachineState *ms, QemuOpts *opts)
             exit(1);
         }
 
-        if (sockets * cores * threads > ms->smp.max_cpus) {
+        if (sockets * dies * cores * threads > ms->smp.max_cpus) {
             error_report("cpu topology: "
-                         "sockets (%u) * cores (%u) * threads (%u) > "
+                         "sockets (%u) * dies (%u) * cores (%u) * threads (%u) > "
                          "maxcpus (%u)",
-                         sockets, cores, threads,
+                         sockets, dies, cores, threads,
                          ms->smp.max_cpus);
             exit(1);
         }
 
-        if (sockets * cores * threads != ms->smp.max_cpus) {
+        if (sockets * dies * cores * threads != ms->smp.max_cpus) {
             warn_report("Invalid CPU topology deprecated: "
-                        "sockets (%u) * cores (%u) * threads (%u) "
+                        "sockets (%u) * dies (%u) * cores (%u) * threads (%u) "
                         "!= maxcpus (%u)",
-                        sockets, cores, threads,
+                        sockets, dies, cores, threads,
                         ms->smp.max_cpus);
         }
 
         ms->smp.cpus = cpus;
         ms->smp.cores = cores;
         ms->smp.threads = threads;
+        pcms->smp_dies = dies;
     }
 
     if (ms->smp.cpus > 1) {
qemu-options.hx +9 -8
index 0d8beb4afd..a5b314a448 100644
@@ -138,25 +138,26 @@ no incompatible TCG features have been enabled (e.g. icount/replay).
 ETEXI
 
 DEF("smp", HAS_ARG, QEMU_OPTION_smp,
-    "-smp [cpus=]n[,maxcpus=cpus][,cores=cores][,threads=threads][,sockets=sockets]\n"
+    "-smp [cpus=]n[,maxcpus=cpus][,cores=cores][,threads=threads][,dies=dies][,sockets=sockets]\n"
     "                set the number of CPUs to 'n' [default=1]\n"
     "                maxcpus= maximum number of total cpus, including\n"
     "                offline CPUs for hotplug, etc\n"
-    "                cores= number of CPU cores on one socket\n"
+    "                cores= number of CPU cores on one socket (for PC, it's on one die)\n"
     "                threads= number of threads on one CPU core\n"
+    "                dies= number of CPU dies on one socket (for PC only)\n"
     "                sockets= number of discrete sockets in the system\n",
         QEMU_ARCH_ALL)
 STEXI
-@item -smp [cpus=]@var{n}[,cores=@var{cores}][,threads=@var{threads}][,sockets=@var{sockets}][,maxcpus=@var{maxcpus}]
+@item -smp [cpus=]@var{n}[,cores=@var{cores}][,threads=@var{threads}][,dies=dies][,sockets=@var{sockets}][,maxcpus=@var{maxcpus}]
 @findex -smp
 Simulate an SMP system with @var{n} CPUs. On the PC target, up to 255
 CPUs are supported. On Sparc32 target, Linux limits the number of usable CPUs
 to 4.
-For the PC target, the number of @var{cores} per socket, the number
-of @var{threads} per cores and the total number of @var{sockets} can be
-specified. Missing values will be computed. If any on the three values is
-given, the total number of CPUs @var{n} can be omitted. @var{maxcpus}
-specifies the maximum number of hotpluggable CPUs.
+For the PC target, the number of @var{cores} per die, the number of @var{threads}
+per cores, the number of @var{dies} per packages and the total number of
+@var{sockets} can be specified. Missing values will be computed.
+If any on the three values is given, the total number of CPUs @var{n} can be omitted.
+@var{maxcpus} specifies the maximum number of hotpluggable CPUs.
 ETEXI
 
 DEF("numa", HAS_ARG, QEMU_OPTION_numa,
vl.c +3 -0
index 9218a718f3..e3908caa47 100644
@@ -1233,6 +1233,9 @@ static QemuOptsList qemu_smp_opts = {
         }, {
             .name = "sockets",
             .type = QEMU_OPT_NUMBER,
+        }, {
+            .name = "dies",
+            .type = QEMU_OPT_NUMBER,
         }, {
             .name = "cores",
             .type = QEMU_OPT_NUMBER,
Signature · 1 line
2.21.0