Introduce smp.modules for x86 in QEMU
v9 cover Introduce smp.modules for x86 in QEMU
From: Zhao Liu <[email protected]>
Hi list,
This is the our v9 patch series, rebased on the master branch at the commit 03d496a992d9 ("Merge tag 'pull-qapi-2024-02-26' of
https://repo.or.cz/qemu/armbru into staging").
Compared with v8 [1], v9 mainly added more module description in commit message and added missing smp.modules description/documentation.
With the general introduction (with origial cluster level) of this secries in v7 [2] cover letter, the following sections are mainly about the description of the newly added smp.modules (since v8, changed x86 cluster support to module) as supplement.
Since v4 [3], we've dropped the original L2 cache command line option (to configure L2 cache topology) and now we have the new RFC [4] to support the general cache topology configuration (as the supplement to this series).
Welcome your comments!
Why We Need a New CPU Topology Level
====================================
For the discussion in v7 about whether we should reuse current smp.clusters for x86 module, the core point is what's the essential differences between x86 module and general cluster.
Since, cluster (for ARM/riscv) lacks a comprehensive and rigorous hardware definition, and judging from the description of smp.clusters
[5] when it was introduced by QEMU, x86 module is very similar to general smp.clusters: they are all a layer above existing core level to organize the physical cores and share L2 cache.
But there are following reasons that drive us to introduce the new smp.modules:
* As the CPU topology abstraction in device tree [6], cluster supports nesting (though currently QEMU hasn't support that). In contrast, (x86) module does not support nesting.
* Due to nesting, there is great flexibility in sharing resources on cluster, rather than narrowing cluster down to sharing L2 (and L3 tags) as the lowest topology level that contains cores.
* Flexible nesting of cluster allows it to correspond to any level between the x86 package and core.
* In Linux kernel, x86's cluster only represents the L2 cache domain but QEMU's smp.clusters is the CPU topology level. Linux kernel will also expose module level topology information in sysfs for x86. To avoid cluster ambiguity and keep a consistent CPU topology naming style with the Linux kernel, we introduce module level for x86.
Based on the above considerations, and in order to eliminate the naming confusion caused by the mapping between general cluster and x86 module, we now formally introduce smp.modules as the new topology level.
Where to Place Module in Existing Topology Levels
=================================================
The module is, in existing hardware practice, the lowest layer that contains the core, while the cluster is able to have a higher topological scope than the module due to its nesting.
Therefore, we place the module between the cluster and the core:
drawer/book/socket/die/cluster/module/core/thread
Additional Consideration on CPU Topology
========================================
Beyond this patchset, nowadays, different arches have different topology requirements, and maintaining arch-agnostic general topology in SMP becomes to be an increasingly difficult thing due to differences in sharing resources and special flexibility (e.g., nesting):
* It becomes difficult to put together all CPU topology hierarchies of different arches to define complete topology order.
* It also becomes complex to ensure the correctness of the topology calculations.
- Now the max_cpus is calculated by multiplying all topology levels, and too many topology levels can easily cause omissions.
Maybe we should consider implementing arch-specfic topology hierarchies.
[1]: https://lore.kernel.org/qemu-devel/[email protected]/
[2]: https://lore.kernel.org/qemu-devel/[email protected]/
[3]: https://lore.kernel.org/qemu-devel/[email protected]/
[4]: https://lore.kernel.org/qemu-devel/[email protected]/
[5]: https://lore.kernel.org/qemu-devel/[email protected]/
[6]: https://www.kernel.org/doc/Documentation/devicetree/bindings/cpu/cpu-topology.txt
Thanks and Best Regards,
Zhao
---
Changelog:
Changes since v8:
* Add the reason of why a new module level is needed in commit message. (Markus).
* Add the description about how Linux kernel supports x86 module level in commit message. (Daniel)
* Add module description in qemu_smp_opts.
* Add missing "modules" parameter of -smp example in documentation.
* Add Philippe's reviewed-by tag.
Changes since v7 (main changes):
* Introduced smp.modules as a new CPU topology level. (Xiaoyao)
* Fixed calculations of cache_info_passthrough case in the patch "i386/cpu: Use APIC ID info to encode cache topo in CPUID[4]". (Xiaoyao)
* Moved the patch "i386/cpu: Use APIC ID info get NumSharingCache for CPUID[0x8000001D].EAX[bits 25:14]" after CPUID[4]'s similar change ("i386/cpu: Use APIC ID offset to encode cache topo in CPUID[4]"). (Xiaoyao)
* Introduced a bitmap in CPUX86State to cache available CPU topology levels.
* Refactored the encode_topo_cpuid1f() to use traversal to search the encoded level and avoid using static variables.
* Mapped x86 module to smp module instead of cluster.
* Dropped Michael/Babu's ACKed/Tested tags for most patches since the code change.
Changes since v6:
* Updated the comment when check cluster-id. Since there's no v8.2, the cluster-id support should at least start from v9.0.
* Rebased on commit d328fef93ae7 ("Merge tag 'pull-20231230' of
https://gitlab.com/rth7680/qemu into staging").
Changes since v5:
* The first four patches of v5 [1] have been merged, v6 contains the remaining patches.
* Reabsed on the latest master.
* Updated the comment when check cluster-id. Since current QEMU is v8.2, the cluster-id support should at least start from v8.3.
Changes since v4:
* Dropped the "x-l2-cache-topo" option. (Michael)
* Added A/R/T tags.
Changes since v3 (main changes):
* Exposed module level in CPUID[0x1F].
* Fixed compile warnings. (Babu)
* Fixed cache topology uninitialization bugs for some AMD CPUs. (Babu)
Changes since v2:
* Added "Tested-by", "Reviewed-by" and "ACKed-by" tags.
* Used newly added wrapped helper to get cores per socket in qemu_init_vcpu().
Changes since v1:
* Reordered patches. (Yanan)
* Deprecated the patch to fix comment of machine_parse_smp_config(). (Yanan)
* Renamed test-x86-cpuid.c to test-x86-topo.c. (Yanan)
* Split the intel's l1 cache topology fix into a new separate patch. (Yanan)
* Combined module_id and APIC ID for module level support into one patch. (Yanan)
* Made cache_into_passthrough case of cpuid 0x04 leaf in
* cpu_x86_cpuid() used max_processor_ids_for_cache() and max_core_ids_in_package() to encode CPUID[4]. (Yanan)
* Added the prefix "CPU_TOPO_LEVEL_*" for CPU topology level names. (Yanan)
---
Zhao Liu (20):
hw/core/machine: Introduce the module as a CPU topology level hw/core/machine: Support modules in -smp
hw/core: Introduce module-id as the topology subindex hw/core: Support module-id in numa configuration i386/cpu: Fix i/d-cache topology to core level for Intel CPU i386/cpu: Use APIC ID info to encode cache topo in CPUID[4] i386/cpu: Use APIC ID info get NumSharingCache for
CPUID[0x8000001D].EAX[bits 25:14]
i386/cpu: Consolidate the use of topo_info in cpu_x86_cpuid()
i386/cpu: Introduce bitmap to cache available CPU topology levels
i386: Split topology types of CPUID[0x1F] from the definitions of
CPUID[0xB]
i386/cpu: Decouple CPUID[0x1F] subleaf with specific topology level
i386: Introduce module level cpu topology to CPUX86State
i386: Support modules_per_die in X86CPUTopoInfo
i386: Expose module level in CPUID[0x1F]
i386: Support module_id in X86CPUTopoIDs
i386/cpu: Introduce module-id to X86CPU
hw/i386/pc: Support smp.modules for x86 PC machine
i386: Add cache topology info in CPUCacheInfo
i386/cpu: Use CPUCacheInfo.share_level to encode CPUID[4]
i386/cpu: Use CPUCacheInfo.share_level to encode
CPUID[0x8000001D].EAX[bits 25:14]Zhuocheng Ding (1):
tests: Add test case of APIC ID for module level parsing
hw/core/machine-hmp-cmds.c | 4 + hw/core/machine-smp.c | 41 +++-- hw/core/machine.c | 18 +++ hw/i386/pc.c | 1 + hw/i386/x86.c | 67 ++++++-- include/hw/boards.h | 4 + include/hw/i386/topology.h | 60 +++++++- qapi/machine.json | 7 + qemu-options.hx | 18 ++- system/vl.c | 3 + target/i386/cpu.c | 304 +++++++++++++++++++++++++++++-------- target/i386/cpu.h | 29 +++- target/i386/kvm/kvm.c | 3 +- tests/unit/test-x86-topo.c | 56 ++++--- 14 files changed, 489 insertions(+), 126 deletions(-)
Signature · 1 line
2.34.1
PATCH v9 1/21 hw/core/machine: Introduce the module as a CPU topology level
From: Zhao Liu <[email protected]>
In x86, module is the topology level above core, which contains a set of cores that share certain resources (in current products, the resource usually includes L2 cache, as well as module scoped features and MSRs).
Though smp.clusters could also share the L2 cache resource [1], there are following reasons that drive us to introduce the new smp.modules:
* As the CPU topology abstraction in device tree [2], cluster supports nesting (though currently QEMU hasn't support that). In contrast, (x86) module does not support nesting.
* Due to nesting, there is great flexibility in sharing resources on cluster, rather than narrowing cluster down to sharing L2 (and L3 tags) as the lowest topology level that contains cores.
* Flexible nesting of cluster allows it to correspond to any level between the x86 package and core.
* In Linux kernel, x86's cluster only represents the L2 cache domain but QEMU's smp.clusters is the CPU topology level. Linux kernel will also expose module level topology information in sysfs for x86. To avoid cluster ambiguity and keep a consistent CPU topology naming style with the Linux kernel, we introduce module level for x86.
The module is, in existing hardware practice, the lowest layer that contains the core, while the cluster is able to have a higher topological scope than the module due to its nesting.
Therefore, place the module between the cluster and the core:
drawer/book/socket/die/cluster/module/core/thread
With the above topological hierarchy order, introduce module level support in MachineState and MachineClass.
[1]: https://lore.kernel.org/qemu-devel/[email protected]/
[2]: https://www.kernel.org/doc/Documentation/devicetree/bindings/cpu/cpu-topology.txt
---
Changes since v8:
* Add the reason of why a new module level is needed in commit message. (Markus).
* Add the description about how Linux kernel supports x86 module level. (Daniel)
Changes since v7:
* New commit to introduce module level in -smp.
--- hw/core/machine-smp.c | 2 +- hw/core/machine.c | 1 + include/hw/boards.h | 4 ++++ 3 files changed, 6 insertions(+), 1 deletion(-)
hw/core/machine-smp.c +1 -1
@@ -234,7 +234,7 @@ void machine_parse_smp_config(MachineState *ms,unsigned int machine_topo_get_cores_per_socket (const MachineState *ms) { -return ms->smp.cores * ms->smp.clusters * ms->smp.dies; +return ms->smp.cores * ms->smp.modules * ms->smp.clusters * ms->smp.dies; }unsigned int machine_topo_get_threads_per_socket (const MachineState *ms)
hw/core/machine.c +1 -0
@@ -1148,6 +1148,7 @@ static void machine_initfn(Object *obj) ms->smp.sockets =1 ; ms->smp.dies =1 ; ms->smp.clusters =1 ; + ms->smp.modules =1 ; ms->smp.cores =1 ; ms->smp.threads =1 ;
include/hw/boards.h +4 -0
@@ -143,6 +143,7 @@ typedef struct {* provided SMP configuration * @books_supported - whether books are supported by the machine * @drawers_supported - whether drawers are supported by the machine +* @modules_supported - whether modules are supported by the machine */ typedef struct { bool prefer_sockets;
@@ -151,6 +152,7 @@ typedef struct {bool has_clusters;bool books_supported;bool drawers_supported; +bool modules_supported; } SMPCompatProps;/**
@@ -338,6 +340,7 @@ typedef struct DeviceMemoryState {* @sockets: the number of sockets in one book * @dies: the number of dies in one socket * @clusters: the number of clusters in one die +* @modules: the number of modules in one cluster * @cores: the number of cores in one cluster * @threads: the number of threads in one core * @max_cpus: the maximum number of logical processors on the machine
@@ -349,6 +352,7 @@ typedef struct CpuTopology {unsigned int sockets;unsigned int dies;unsigned int clusters; +unsigned int modules;unsigned int cores;unsigned int threads;unsigned int max_cpus;
Signature · 1 line
2.34.1
PATCH v9 2/21 hw/core/machine: Support modules in -smp
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
Add "modules" parameter parsing support in -smp.
---
Changes since v8:
* Add module description in qemu_smp_opts.
Changes since v7:
* New commit to introduce module level in -smp.
--- hw/core/machine-smp.c | 39 +++++++++++++++++++++++++++++---------- hw/core/machine.c | 1 + qapi/machine.json | 3 +++ system/vl.c | 3 +++ 4 files changed, 36 insertions(+), 10 deletions(-)
hw/core/machine-smp.c +29 -10
@@ -51,6 +51,10 @@ static char *cpu_hierarchy_to_string(MachineState *ms)g_string_append_printf (s," * clusters (%u)" , ms->smp.clusters); } +if (mc->smp_props.modules_supported) { +g_string_append_printf (s," * modules (%u)" , ms->smp.clusters); + } +g_string_append_printf (s," * cores (%u)" , ms->smp.cores);g_string_append_printf (s," * threads (%u)" , ms->smp.threads);
@@ -88,6 +92,7 @@ void machine_parse_smp_config(MachineState *ms,unsigned sockets = config->has_sockets ? config->sockets :0 ;unsigned dies = config->has_dies ? config->dies :0 ;unsigned clusters = config->has_clusters ? config->clusters :0 ; +unsigned modules = config->has_modules ? config->modules :0 ;unsigned cores = config->has_cores ? config->cores :0 ;unsigned threads = config->has_threads ? config->threads :0 ;unsigned maxcpus = config->has_maxcpus ? config->maxcpus :0 ;
@@ -102,6 +107,7 @@ void machine_parse_smp_config(MachineState *ms, (config->has_sockets && config->sockets ==0 ) || (config->has_dies && config->dies ==0 ) || (config->has_clusters && config->clusters ==0 ) || + (config->has_modules && config->modules ==0 ) || (config->has_cores && config->cores ==0 ) || (config->has_threads && config->threads ==0 ) || (config->has_maxcpus && config->maxcpus == 0)) {
@@ -117,12 +123,12 @@ void machine_parse_smp_config(MachineState *ms,error_setg (errp,"dies not supported by this machine's CPU topology" );return ; } + dies = dies >0 ? dies :1 ; +if (!mc->smp_props.clusters_supported && clusters >1 ) {error_setg (errp,"clusters not supported by this machine's CPU topology" );return ; } - - dies = dies >0 ? dies :1 ; clusters = clusters >0 ? clusters :1 ;if (!mc->smp_props.books_supported && books >1 ) {
@@ -138,6 +144,13 @@ void machine_parse_smp_config(MachineState *ms, } drawers = drawers >0 ? drawers :1 ; +if (!mc->smp_props.modules_supported && modules >1 ) { +error_setg (errp,"modules not supported by this " +"machine's CPU topology" ); +return ; + } + modules = modules >0 ? modules :1 ; +/* compute missing values based on the provided ones */ if (cpus ==0 && maxcpus ==0 ) { sockets = sockets >0 ? sockets :1 ;
@@ -152,11 +165,13 @@ void machine_parse_smp_config(MachineState *ms, cores = cores >0 ? cores :1 ; threads = threads >0 ? threads :1 ; sockets = maxcpus / - (drawers * books * dies * clusters * cores * threads); + (drawers * books * dies * clusters * + modules * cores * threads); }else if (cores ==0 ) { threads = threads >0 ? threads :1 ; cores = maxcpus / - (drawers * books * sockets * dies * clusters * threads); + (drawers * books * sockets * dies * + clusters * modules * threads); } }else {/* prefer cores over sockets since 6.2 */
@@ -164,23 +179,26 @@ void machine_parse_smp_config(MachineState *ms, sockets = sockets >0 ? sockets :1 ; threads = threads >0 ? threads :1 ; cores = maxcpus / - (drawers * books * sockets * dies * clusters * threads); + (drawers * books * sockets * dies * + clusters * modules * threads); }else if (sockets ==0 ) { threads = threads >0 ? threads :1 ; sockets = maxcpus / - (drawers * books * dies * clusters * cores * threads); + (drawers * books * dies * clusters * + modules * cores * threads); } }/* try to calculate omitted threads at last */ if (threads ==0 ) { threads = maxcpus / - (drawers * books * sockets * dies * clusters * cores); + (drawers * books * sockets * dies * + clusters * modules * cores); } } maxcpus = maxcpus >0 ? maxcpus : drawers * books * sockets * dies * - clusters * cores * threads; + clusters * modules * cores * threads; cpus = cpus >0 ? cpus : maxcpus; ms->smp.cpus = cpus;
@@ -189,6 +207,7 @@ void machine_parse_smp_config(MachineState *ms, ms->smp.sockets = sockets; ms->smp.dies = dies; ms->smp.clusters = clusters; + ms->smp.modules = modules; ms->smp.cores = cores; ms->smp.threads = threads; ms->smp.max_cpus = maxcpus;
@@ -196,8 +215,8 @@ void machine_parse_smp_config(MachineState *ms, mc->smp_props.has_clusters = config->has_clusters;/* sanity-check of the computed topology */ -if (drawers * books * sockets * dies * clusters * cores * threads != - maxcpus) { +if (drawers * books * sockets * dies * clusters * modules * cores * + threads != maxcpus) { g_autofree char *topo_msg = cpu_hierarchy_to_string(ms);error_setg (errp,"Invalid CPU topology: " "product of the hierarchy must match maxcpus: "
hw/core/machine.c +1 -0
@@ -872,6 +872,7 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name, .has_sockets = true, .sockets = ms->smp.sockets, .has_dies = true, .dies = ms->smp.dies, .has_clusters = true, .clusters = ms->smp.clusters, + .has_modules = true, .modules = ms->smp.modules, .has_cores = true, .cores = ms->smp.cores, .has_threads = true, .threads = ms->smp.threads, .has_maxcpus = true, .maxcpus = ms->smp.max_cpus,
qapi/machine.json +3 -0
@@ -1640,6 +1640,8 @@ # # @clusters: number of clusters per parent container (since 7.0) # +# @modules: number of modules per parent container (since 9.0) +# # @cores: number of cores per parent container # # @threads: number of threads per core
@@ -1653,6 +1655,7 @@ '*sockets': 'int', '*dies': 'int', '*clusters': 'int', + '*modules': 'int', '*cores': 'int', '*threads': 'int', '*maxcpus': 'int' } }
system/vl.c +3 -0
@@ -741,6 +741,9 @@ static QemuOptsList qemu_smp_opts = { }, { .name = "clusters", .type = QEMU_OPT_NUMBER, + }, { + .name = "modules", + .type = QEMU_OPT_NUMBER, }, { .name = "cores", .type = QEMU_OPT_NUMBER,
Signature · 1 line
2.34.1
Re: [PATCH v9 02/21] hw/core/machine: Support modules in -smp
Quoted 7 lines · Zhao Liu
From: Zhao Liu <[email protected]>
Add "modules" parameter parsing support in -smp.
Suggested-by Xiaoyao Li <[email protected]>Tested-by Yongwei Ma <[email protected]>Signed-off-by Zhao Liu <[email protected]>
QAPI schema
Re: [PATCH v9 02/21] hw/core/machine: Support modules in -smp
Quoted 4 lines · Markus Armbruster
Quoted 2 lines · Zhao Liu
From: Zhao Liu <[email protected]>
Quoted 8 lines
Quoted 5 lines
Add "modules" parameter parsing support in -smp.
Suggested-by Xiaoyao Li <[email protected]>Tested-by Yongwei Ma <[email protected]>Signed-off-by Zhao Liu <[email protected]>QAPI schema
I missed something. The patch added @modules without updating "The ordering from ...":
##
# @SMPConfiguration:
#
# Schema for CPU topology configuration. A missing value lets QEMU
# figure out a suitable value based on the ones that are provided.
#
# The members other than @cpus and @maxcpus define a topology of
# containers.
#--> # The ordering from highest/coarsest to lowest/finest is: @drawers, --> # @books, @sockets, @dies, @clusters, @cores, @threads.
Where does it go in this list?
The order below suggests between @clusters and @modules.
#
# Different architectures support different subsets of topology
# containers.
#
# For example, s390x does not have clusters and dies, and the socket
# is the parent container of cores.
#
# @cpus: number of virtual CPUs in the virtual machine
#
# @maxcpus: maximum number of hotpluggable virtual CPUs in the virtual
# machine
#
# @drawers: number of drawers in the CPU topology (since 8.2)
#
# @books: number of books in the CPU topology (since 8.2)
#
# @sockets: number of sockets per parent container
#
# @dies: number of dies per parent container
#
# @clusters: number of clusters per parent container (since 7.0)
#
# @modules: number of modules per parent container (since 9.1)
#
# @cores: number of cores per parent container
#
# @threads: number of threads per core
#
# Since: 6.1
##Re: [PATCH v9 02/21] hw/core/machine: Support modules in -smp
Quoted 31 lines · Markus Armbruster
Date: Thu, 09 Oct 2025 13:40:36 +0200
From: Markus Armbruster <[email protected]>
Subject: Re: [PATCH v9 02/21] hw/core/machine: Support modules in -smpQuoted 4 lines · Markus Armbruster
Quoted 2 lines · Zhao Liu
From: Zhao Liu <[email protected]>
Quoted 8 lines
Quoted 5 lines
Add "modules" parameter parsing support in -smp.
Suggested-by Xiaoyao Li <[email protected]>Tested-by Yongwei Ma <[email protected]>Signed-off-by Zhao Liu <[email protected]>QAPI schema
I missed something. The patch added @modules without updating "The ordering from ...":
## # @SMPConfiguration: # # Schema for CPU topology configuration. A missing value lets QEMU # figure out a suitable value based on the ones that are provided. # # The members other than @cpus and @maxcpus define a topology of # containers. #
Quoted 6 lines
--> # The ordering from highest/coarsest to lowest/finest is: @drawers, --> # @books, @sockets, @dies, @clusters, @cores, @threads.
Where does it go in this list?
The order below suggests between @clusters and @modules.
Thanks Markus! I missed this case... sorry for that.
And you're right, @module is between @clusters and @modules.
I'll submit a patch to fix this doc!
Regards,
Zhao
Re: [PATCH v9 02/21] hw/core/machine: Support modules in -smp
In reply to Zhao LiuQuoted 25 lines · Zhao Liu
From: Zhao Liu <[email protected]>
Add "modules" parameter parsing support in -smp.
Suggested-by Xiaoyao Li <[email protected]>Tested-by Yongwei Ma <[email protected]>Signed-off-by Zhao Liu <[email protected]>---
Changes since v8:
* Add module description inqemu_smp_opts.Changes since v7:
* New commit to introduce module level in -smp.--- hw/core/machine-smp.c | 39 +++++++++++++++++++++++++++++---------- hw/core/machine.c | 1 + qapi/machine.json | 3 +++ system/vl.c | 3 +++ 4 files changed, 36 insertions(+), 10 deletions(-)hw/core/machine-smp.c +0
-0
hw/core/machine-smp.c static char *cpu_hierarchy_to_string(MachineState *ms) · line 56
Quoted 6 lines
hw/core/machine-smp.c +3
-0@@ -51,6 +51,10 @@ static char *cpu_hierarchy_to_string(MachineState *ms)g_string_append_printf (s," * clusters (%u)" , ms->smp.clusters); } +if (mc->smp_props.modules_supported) { +g_string_append_printf (s," * modules (%u)" , ms->smp.clusters); + }
smp.clusters -> smp.modules?
Quoted 162 lines
hw/core/machine-smp.c +26
-10+g_string_append_printf (s," * cores (%u)" , ms->smp.cores);g_string_append_printf (s," * threads (%u)" , ms->smp.threads);@@ -88,6 +92,7 @@ void machine_parse_smp_config(MachineState *ms,unsigned sockets = config->has_sockets ? config->sockets :0 ;unsigned dies = config->has_dies ? config->dies :0 ;unsigned clusters = config->has_clusters ? config->clusters :0 ; +unsigned modules = config->has_modules ? config->modules :0 ;unsigned cores = config->has_cores ? config->cores :0 ;unsigned threads = config->has_threads ? config->threads :0 ;unsigned maxcpus = config->has_maxcpus ? config->maxcpus :0 ;@@ -102,6 +107,7 @@ void machine_parse_smp_config(MachineState *ms, (config->has_sockets && config->sockets ==0 ) || (config->has_dies && config->dies ==0 ) || (config->has_clusters && config->clusters ==0 ) || + (config->has_modules && config->modules ==0 ) || (config->has_cores && config->cores ==0 ) || (config->has_threads && config->threads ==0 ) || (config->has_maxcpus && config->maxcpus == 0)) {@@ -117,12 +123,12 @@ void machine_parse_smp_config(MachineState *ms,error_setg (errp,"dies not supported by this machine's CPU topology" );return ; } + dies = dies >0 ? dies :1 ; +if (!mc->smp_props.clusters_supported && clusters >1 ) {error_setg (errp,"clusters not supported by this machine's CPU topology" );return ; } - - dies = dies >0 ? dies :1 ; clusters = clusters >0 ? clusters :1 ;if (!mc->smp_props.books_supported && books >1 ) {@@ -138,6 +144,13 @@ void machine_parse_smp_config(MachineState *ms, } drawers = drawers >0 ? drawers :1 ; +if (!mc->smp_props.modules_supported && modules >1 ) { +error_setg (errp,"modules not supported by this " +"machine's CPU topology" ); +return ; + } + modules = modules >0 ? modules :1 ; +/* compute missing values based on the provided ones */ if (cpus ==0 && maxcpus ==0 ) { sockets = sockets >0 ? sockets :1 ;@@ -152,11 +165,13 @@ void machine_parse_smp_config(MachineState *ms, cores = cores >0 ? cores :1 ; threads = threads >0 ? threads :1 ; sockets = maxcpus / - (drawers * books * dies * clusters * cores * threads); + (drawers * books * dies * clusters * + modules * cores * threads); }else if (cores ==0 ) { threads = threads >0 ? threads :1 ; cores = maxcpus / - (drawers * books * sockets * dies * clusters * threads); + (drawers * books * sockets * dies * + clusters * modules * threads); } }else {/* prefer cores over sockets since 6.2 */ @@ -164,23 +179,26 @@ void machine_parse_smp_config(MachineState *ms, sockets = sockets >0 ? sockets :1 ; threads = threads >0 ? threads :1 ; cores = maxcpus / - (drawers * books * sockets * dies * clusters * threads); + (drawers * books * sockets * dies * + clusters * modules * threads); }else if (sockets ==0 ) { threads = threads >0 ? threads :1 ; sockets = maxcpus / - (drawers * books * dies * clusters * cores * threads); + (drawers * books * dies * clusters * + modules * cores * threads); } }/* try to calculate omitted threads at last */ if (threads ==0 ) { threads = maxcpus / - (drawers * books * sockets * dies * clusters * cores); + (drawers * books * sockets * dies * + clusters * modules * cores); } } maxcpus = maxcpus >0 ? maxcpus : drawers * books * sockets * dies * - clusters * cores * threads; + clusters * modules * cores * threads; cpus = cpus >0 ? cpus : maxcpus; ms->smp.cpus = cpus;@@ -189,6 +207,7 @@ void machine_parse_smp_config(MachineState *ms, ms->smp.sockets = sockets; ms->smp.dies = dies; ms->smp.clusters = clusters; + ms->smp.modules = modules; ms->smp.cores = cores; ms->smp.threads = threads; ms->smp.max_cpus = maxcpus;@@ -196,8 +215,8 @@ void machine_parse_smp_config(MachineState *ms, mc->smp_props.has_clusters = config->has_clusters;/* sanity-check of the computed topology */ -if (drawers * books * sockets * dies * clusters * cores * threads != - maxcpus) { +if (drawers * books * sockets * dies * clusters * modules * cores * + threads != maxcpus) { g_autofree char *topo_msg = cpu_hierarchy_to_string(ms);error_setg (errp,"Invalid CPU topology: " "product of the hierarchy must match maxcpus: "hw/core/machine.c +1
-0@@ -872,6 +872,7 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name, .has_sockets = true, .sockets = ms->smp.sockets, .has_dies = true, .dies = ms->smp.dies, .has_clusters = true, .clusters = ms->smp.clusters, + .has_modules = true, .modules = ms->smp.modules, .has_cores = true, .cores = ms->smp.cores, .has_threads = true, .threads = ms->smp.threads, .has_maxcpus = true, .maxcpus = ms->smp.max_cpus,qapi/machine.json +3
-0@@ -1640,6 +1640,8 @@ # # @clusters: number of clusters per parent container (since 7.0) # +# @modules: number of modules per parent container (since 9.0) +# # @cores: number of cores per parent container # # @threads: number of threads per core@@ -1653,6 +1655,7 @@ '*sockets': 'int', '*dies': 'int', '*clusters': 'int', + '*modules': 'int', '*cores': 'int', '*threads': 'int', '*maxcpus': 'int' } }system/vl.c +3
-0@@ -741,6 +741,9 @@ static QemuOptsList qemu_smp_opts = { }, { .name = "clusters", .type = QEMU_OPT_NUMBER, + }, { + .name = "modules", + .type = QEMU_OPT_NUMBER, }, { .name = "cores", .type = QEMU_OPT_NUMBER,
Re: [PATCH v9 02/21] hw/core/machine: Support modules in -smp
Quoted 9 lines
hw/core/machine-smp.c static char *cpu_hierarchy_to_string(MachineState *ms) · line 55
Quoted 6 lines
hw/core/machine-smp.c +3
-0@@ -51,6 +51,10 @@ static char *cpu_hierarchy_to_string(MachineState *ms)g_string_append_printf (s," * clusters (%u)" , ms->smp.clusters); } +if (mc->smp_props.modules_supported) { +g_string_append_printf (s," * modules (%u)" , ms->smp.clusters); + }smp.clusters -> smp.modules?
Good catch! Thanks!
-Zhao
PATCH v9 3/21 hw/core: Introduce module-id as the topology subindex
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
Add module-id in CpuInstanceProperties, to locate the CPU with module level.
---
Changes since v7:
* New commit to introduce module_id to locate the CPU with module level.
--- hw/core/machine-hmp-cmds.c | 4 ++++ qapi/machine.json | 4 ++++ 2 files changed, 8 insertions(+)
hw/core/machine-hmp-cmds.c +4 -0
@@ -87,6 +87,10 @@ void hmp_hotpluggable_cpus(Monitor *mon, const QDict *qdict)monitor_printf (mon," cluster-id: \"%" PRIu64"\"\n" , c->cluster_id); } +if (c->has_module_id) { +monitor_printf (mon," module-id: \"%" PRIu64"\"\n" , + c->module_id); + }if (c->has_core_id) {monitor_printf (mon," core-id: \"%" PRIu64"\"\n" , c->core_id); }
qapi/machine.json +4 -0
@@ -933,6 +933,9 @@ # @cluster-id: cluster number within the parent container the CPU # belongs to (since 7.1) # +# @module-id: module number within the parent container the CPU +# belongs to (since 9.0) +# # @core-id: core number within the parent container the CPU # belongs to #
@@ -951,6 +954,7 @@ '*socket-id': 'int', '*die-id': 'int', '*cluster-id': 'int', + '*module-id': 'int', '*core-id': 'int', '*thread-id': 'int' }
Signature · 1 line
2.34.1
Re: [PATCH v9 03/21] hw/core: Introduce module-id as the topology subindex
Quoted 8 lines · Zhao Liu
From: Zhao Liu <[email protected]>
Add module-id in CpuInstanceProperties, to locate the CPU with module level.
Suggested-by Xiaoyao Li <[email protected]>Tested-by Yongwei Ma <[email protected]>Signed-off-by Zhao Liu <[email protected]>
QAPI schema
PATCH v9 4/21 hw/core: Support module-id in numa configuration
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
Module is a level above the core, thereby supporting numa configuration on the module level can bring user more numa flexibility.
This is the natural further support for module level.
Add module level support in numa configuration.
---
Changes since v7:
* New commit to support module level.
--- hw/core/machine.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
hw/core/machine.c +16 -0
@@ -791,6 +791,11 @@ void machine_set_cpu_numa_node(MachineState *machine,return ; } +if (props->has_module_id && !slot->props.has_module_id) { +error_setg (errp,"module-id is not supported" ); +return ; + } +if (props->has_cluster_id && !slot->props.has_cluster_id) {error_setg (errp,"cluster-id is not supported" );return ;
@@ -815,6 +820,11 @@ void machine_set_cpu_numa_node(MachineState *machine,continue ; } +if (props->has_module_id && + props->module_id != slot->props.module_id) { +continue ; + } +if (props->has_cluster_id && props->cluster_id != slot->props.cluster_id) {continue ;
@@ -1212,6 +1222,12 @@ static char *cpu_slot_to_string(const CPUArchId *cpu) }g_string_append_printf (s,"cluster-id: %" PRId64, cpu->props.cluster_id); } +if (cpu->props.has_module_id) { +if (s->len) { +g_string_append_printf (s,", " ); + } +g_string_append_printf (s,"module-id: %" PRId64, cpu->props.module_id); + }if (cpu->props.has_core_id) {if (s->len) {g_string_append_printf (s,", " );
Signature · 1 line
2.34.1
PATCH v9 5/21 i386/cpu: Fix i/d-cache topology to core level for Intel CPU
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
For i-cache and d-cache, current QEMU hardcodes the maximum IDs for CPUs sharing cache (CPUID.04H.00H:EAX[bits 25:14] and CPUID.04H.01H:EAX[bits 25:14]) to 0, and this means i-cache and d-cache are shared in the SMT level.
This is correct if there's single thread per core, but is wrong for the hyper threading case (one core contains multiple threads) since the i-cache and d-cache are shared in the core level other than SMT level.
For AMD CPU, commit 8f4202fb1080 ("i386: Populate AMD Processor Cache Information for cpuid 0x8000001D") has already introduced i/d cache topology as core level by default.
Therefore, in order to be compatible with both multi-threaded and single-threaded situations, we should set i-cache and d-cache be shared at the core level by default.
This fix changes the default i/d cache topology from per-thread to per-core. Potentially, this change in L1 cache topology may affect the performance of the VM if the user does not specifically specify the topology or bind the vCPU. However, the way to achieve optimal performance should be to create a reasonable topology and set the appropriate vCPU affinity without relying on QEMU's default topology structure.
---
Changes since v3:
* Changed the description of current i/d cache encoding status to avoid misleading to "architectural rules". (Xiaoyao)
Changes since v1:
* Split this fix from the patch named "i386/cpu: Fix number of addressable IDs in CPUID.04H".
* Added the explanation of the impact on performance. (Xiaoyao)
--- target/i386/cpu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
target/i386/cpu.c +2 -2
@@ -6113,12 +6113,12 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,switch (count) {case 0 :/* L1 dcache info */ encode_cache_cpuid4 (env->cache_info_cpuid4.l1d_cache, -1 , cs->nr_cores, + cs->nr_threads, cs->nr_cores, eax, ebx, ecx, edx);break ;case 1 :/* L1 icache info */ encode_cache_cpuid4 (env->cache_info_cpuid4.l1i_cache, -1 , cs->nr_cores, + cs->nr_threads, cs->nr_cores, eax, ebx, ecx, edx);break ;case 2 :/* L2 cache info */
Signature · 1 line
2.34.1
PATCH v9 6/21 i386/cpu: Use APIC ID info to encode cache topo in CPUID[4]
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
Refer to the fixes of cache_info_passthrough ([1], [2]) and SDM, the CPUID.04H:EAX[bits 25:14] and CPUID.04H:EAX[bits 31:26] should use the nearest power-of-2 integer.
The nearest power-of-2 integer can be calculated by pow2ceil() or by using APIC ID offset/width (like L3 topology using 1 << die_offset [3]).
But in fact, CPUID.04H:EAX[bits 25:14] and CPUID.04H:EAX[bits 31:26] are associated with APIC ID. For example, in linux kernel, the field "num_threads_sharing" (Bits 25 - 14) is parsed with APIC ID. And for another example, on Alder Lake P, the CPUID.04H:EAX[bits 31:26] is not matched with actual core numbers and it's calculated by:
"(1 << (pkg_offset - core_offset)) - 1".
Therefore the topology information of APIC ID should be preferred to calculate nearest power-of-2 integer for CPUID.04H:EAX[bits 25:14] and CPUID.04H:EAX[bits 31:26]:
1. d/i cache is shared in a core, 1 << core_offset should be used instead of "cs->nr_threads" in encode_cache_cpuid4() for CPUID.04H.00H:EAX[bits 25:14] and CPUID.04H.01H:EAX[bits 25:14].
2. L2 cache is supposed to be shared in a core as for now, thereby 1 << core_offset should also be used instead of "cs->nr_threads" in encode_cache_cpuid4() for CPUID.04H.02H:EAX[bits 25:14].
3. Similarly, the value for CPUID.04H:EAX[bits 31:26] should also be calculated with the bit width between the package and SMT levels in the APIC ID (1 << (pkg_offset - core_offset) - 1).
In addition, use APIC ID bits calculations to replace "pow2ceil()" for cache_info_passthrough case.
[1]: efb3934adf9e ("x86: cpu: make sure number of addressable IDs for processor cores meets the spec") [2]: d7caf13b5fcf ("x86: cpu: fixup number of addressable IDs for logical processors sharing cache") [3]: d65af288a84d ("i386: Update new x86_apicid parsing rules with die_offset support")
---
Changes since v7:
* Fixed calculations in cache_info_passthrough case. (Xiaoyao)
* Renamed variables as *_width. (Xiaoyao)
* Unified variable names for encoding cache_info_passthrough case and non-cache_info_passthrough case as addressable_cores_width and addressable_threads_width.
* Fixed typos in commit message. (Xiaoyao)
* Dropped Michael/Babu's ACKed/Tested tags since the code change.
* Re-added Yongwei's Tested tag For his re-testing.
Changes since v3:
* Fixed compile warnings. (Babu)
* Fixed spelling typo.
Changes since v1:
* Used APIC ID offset to replace "pow2ceil()" for cache_info_passthrough case. (Yanan)
* Split the L1 cache fix into a separate patch.
* Renamed the title of this patch (the original is "i386/cpu: Fix number of addressable IDs in CPUID.04H").
--- target/i386/cpu.c | 37 ++++++++++++++++++++++++++++--------- 1 file changed, 28 insertions(+), 9 deletions(-)
target/i386/cpu.c +28 -9
@@ -6014,7 +6014,6 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, {X86CPU *cpu =env_archcpu (env);CPUState *cs =env_cpu (env); -uint32_t die_offset;uint32_t limit;uint32_t signature[3 ];X86CPUTopoInfo topo_info;
@@ -6086,7 +6085,10 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, (cpuid2_cache_descriptor (env->cache_info_cpuid2.l1i_cache) <<8 ) | (cpuid2_cache_descriptor (env->cache_info_cpuid2.l2_cache));break ; -case 4 : +case 4 : { +int addressable_cores_width; +int addressable_threads_width; +/* cache info: needed for Core compatibility */ if (cpu->cache_info_passthrough) {x86_cpu_get_cache_cpuid (index, count, eax, ebx, ecx, edx);
@@ -6098,39 +6100,55 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,int host_vcpus_per_cache =1 + ((*eax &0x3FFC000 ) >>14 );int vcpus_per_socket = cs->nr_cores * cs->nr_threads;if (cs->nr_cores >1 ) { + addressable_cores_width =apicid_pkg_offset (&topo_info) - +apicid_core_offset (&topo_info); + *eax &= ~0xFC000000 ; - *eax |= (pow2ceil (cs->nr_cores) -1 ) <<26 ; + *eax |= ((1 << addressable_cores_width) -1 ) <<26 ; }if (host_vcpus_per_cache > vcpus_per_socket) { +/* Share the cache at package level. */ + addressable_threads_width =apicid_pkg_offset (&topo_info); + *eax &= ~0x3FFC000 ; - *eax |= (pow2ceil (vcpus_per_socket) -1 ) <<14 ; + *eax |= ((1 << addressable_threads_width) -1 ) <<14 ; } } }else if (cpu->vendor_cpuid_only &&IS_AMD_CPU (env)) { *eax = *ebx = *ecx = *edx =0 ; }else { *eax =0 ; + addressable_cores_width =apicid_pkg_offset (&topo_info) - +apicid_core_offset (&topo_info); +switch (count) {case 0 :/* L1 dcache info */ + addressable_threads_width =apicid_core_offset (&topo_info);encode_cache_cpuid4 (env->cache_info_cpuid4.l1d_cache, - cs->nr_threads, cs->nr_cores, + (1 << addressable_threads_width), + (1 << addressable_cores_width), eax, ebx, ecx, edx);break ;case 1 :/* L1 icache info */ + addressable_threads_width =apicid_core_offset (&topo_info);encode_cache_cpuid4 (env->cache_info_cpuid4.l1i_cache, - cs->nr_threads, cs->nr_cores, + (1 << addressable_threads_width), + (1 << addressable_cores_width), eax, ebx, ecx, edx);break ;case 2 :/* L2 cache info */ + addressable_threads_width =apicid_core_offset (&topo_info);encode_cache_cpuid4 (env->cache_info_cpuid4.l2_cache, - cs->nr_threads, cs->nr_cores, + (1 << addressable_threads_width), + (1 << addressable_cores_width), eax, ebx, ecx, edx);break ;case 3 :/* L3 cache info */ - die_offset =apicid_die_offset (&topo_info);if (cpu->enable_l3_cache) { + addressable_threads_width =apicid_die_offset (&topo_info);encode_cache_cpuid4 (env->cache_info_cpuid4.l3_cache, - (1 << die_offset), cs->nr_cores, + (1 << addressable_threads_width), + (1 << addressable_cores_width), eax, ebx, ecx, edx);break ; }
@@ -6141,6 +6159,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, } }break ; + }case 5 :/* MONITOR/MWAIT Leaf */ *eax = cpu->mwait.eax;/* Smallest monitor-line size in bytes */
Signature · 1 line
2.34.1
Re: [PATCH v9 06/21] i386/cpu: Use APIC ID info to encode cache topo in CPUID[4]
Quoted 135 lines · Zhao Liu
From: Zhao Liu <[email protected]>
Refer to the fixes of
cache_info_passthrough([1], [2]) and SDM, the CPUID.04H:EAX[bits 25:14] and CPUID.04H:EAX[bits 31:26] should use the nearest power-of-2 integer.The nearest power-of-2 integer can be calculated by
pow2ceil()or by using APIC ID offset/width (like L3 topology using 1 <<die_offset[3]).But in fact, CPUID.04H:EAX[bits 25:14] and CPUID.04H:EAX[bits 31:26] are associated with APIC ID. For example, in linux kernel, the field "
num_threads_sharing" (Bits 25 - 14) is parsed with APIC ID. And for another example, on Alder Lake P, the CPUID.04H:EAX[bits 31:26] is not matched with actual core numbers and it's calculated by:
"(1 << (pkg_offset-core_offset)) - 1".Therefore the topology information of APIC ID should be preferred to calculate nearest power-of-2 integer for CPUID.04H:EAX[bits 25:14] and CPUID.04H:EAX[bits 31:26]:
1. d/i cache is shared in a core, 1 <<core_offsetshould be usedinstead of "cs->nr_threads" in encode_cache_cpuid4() for CPUID.04H.00H:EAX[bits 25:14] and CPUID.04H.01H:EAX[bits 25:14]. 2. L2 cache is supposed to be shared in a core as for now, thereby 1 << core_offset should also be used instead of "cs->nr_threads" in encode_cache_cpuid4() for CPUID.04H.02H:EAX[bits 25:14].3. Similarly, the value for CPUID.04H:EAX[bits 31:26] should also be calculated with the bit width between the package and SMT levels in the APIC ID (1 << (
pkg_offset-core_offset) - 1).In addition, use APIC ID bits calculations to replace "
pow2ceil()" forcache_info_passthroughcase.[1]:
efb3934adf9e("x86: cpu: make sure number of addressable IDs for processor cores meets the spec") [2]:d7caf13b5fcf("x86: cpu: fixup number of addressable IDs for logical processors sharing cache") [3]:d65af288a84d("i386: Update newx86_apicidparsing rules withdie_offsetsupport")Fixes 7e3482f82480 ("i386: Helpers to encode cache information consistently")Suggested-by Robert Hoo <[email protected]>Tested-by Yongwei Ma <[email protected]>Signed-off-by Zhao Liu <[email protected]>---
Changes since v7:
* Fixed calculations incache_info_passthroughcase. (Xiaoyao)
* Renamed variables as*_width. (Xiaoyao)
* Unified variable names for encodingcache_info_passthroughcase and non-cache_info_passthrough case asaddressable_cores_widthandaddressable_threads_width.
* Fixed typos in commit message. (Xiaoyao)
* Dropped Michael/Babu's ACKed/Tested tags since the code change.
* Re-added Yongwei's Tested tag For his re-testing.Changes since v3:
* Fixed compile warnings. (Babu)
* Fixed spelling typo.Changes since v1:
* Used APIC ID offset to replace "pow2ceil()" forcache_info_passthroughcase. (Yanan)
* Split the L1 cache fix into a separate patch.
* Renamed the title of this patch (the original is "i386/cpu: Fix number of addressable IDs in CPUID.04H").--- target/i386/cpu.c | 37 ++++++++++++++++++++++++++++--------- 1 file changed, 28 insertions(+), 9 deletions(-)target/i386/cpu.c +22
-6@@ -6014,7 +6014,6 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, {X86CPU *cpu =env_archcpu (env);CPUState *cs =env_cpu (env); -uint32_t die_offset;uint32_t limit;uint32_t signature[3 ];X86CPUTopoInfo topo_info;@@ -6086,7 +6085,10 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, (cpuid2_cache_descriptor (env->cache_info_cpuid2.l1i_cache) <<8 ) | (cpuid2_cache_descriptor (env->cache_info_cpuid2.l2_cache));break ; -case 4 : +case 4 : { +int addressable_cores_width; +int addressable_threads_width; +/* cache info: needed for Core compatibility */ if (cpu->cache_info_passthrough) {x86_cpu_get_cache_cpuid (index, count, eax, ebx, ecx, edx);@@ -6098,39 +6100,55 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,int host_vcpus_per_cache =1 + ((*eax &0x3FFC000 ) >>14 );int vcpus_per_socket = cs->nr_cores * cs->nr_threads;if (cs->nr_cores >1 ) { + addressable_cores_width =apicid_pkg_offset (&topo_info) - +apicid_core_offset (&topo_info); + *eax &= ~0xFC000000 ; - *eax |= (pow2ceil (cs->nr_cores) -1 ) <<26 ; + *eax |= ((1 << addressable_cores_width) -1 ) <<26 ; }if (host_vcpus_per_cache > vcpus_per_socket) { +/* Share the cache at package level. */ + addressable_threads_width =apicid_pkg_offset (&topo_info); + *eax &= ~0x3FFC000 ; - *eax |= (pow2ceil (vcpus_per_socket) -1 ) <<14 ; + *eax |= ((1 << addressable_threads_width) -1 ) <<14 ; } } }else if (cpu->vendor_cpuid_only &&IS_AMD_CPU (env)) { *eax = *ebx = *ecx = *edx =0 ; }else { *eax =0 ; + addressable_cores_width =apicid_pkg_offset (&topo_info) - +apicid_core_offset (&topo_info); +switch (count) {case 0 :/* L1 dcache info */ + addressable_threads_width =apicid_core_offset (&topo_info);encode_cache_cpuid4 (env->cache_info_cpuid4.l1d_cache, - cs->nr_threads, cs->nr_cores, + (1 << addressable_threads_width), + (1 << addressable_cores_width), eax, ebx, ecx, edx);break ;case 1 :/* L1 icache info */ + addressable_threads_width =apicid_core_offset (&topo_info);encode_cache_cpuid4 (env->cache_info_cpuid4.l1i_cache, - cs->nr_threads, cs->nr_cores, + (1 << addressable_threads_width), + (1 << addressable_cores_width), eax, ebx, ecx, edx);break ;case 2 :/* L2 cache info */ + addressable_threads_width =apicid_core_offset (&topo_info);
target/i386/cpu.c void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, · line 6148
Quoted 10 lines
target/i386/cpu.c +3
-2encode_cache_cpuid4 (env->cache_info_cpuid4.l2_cache, - cs->nr_threads, cs->nr_cores, + (1 << addressable_threads_width), + (1 << addressable_cores_width), eax, ebx, ecx, edx);break ;case 3 :/* L3 cache info */ - die_offset =apicid_die_offset (&topo_info);if (cpu->enable_l3_cache) { + addressable_threads_width =apicid_die_offset (&topo_info);
Please get rid of the local variable @addressable_threads_width.
It is truly confusing. In this patch, it is assigned to
- apicid_pkg_offset(&topo_info);
- apicid_core_offset(&topo_info);
- apicid_die_offset(&topo_info);
in different cases.
I only suggested the name of addressable_core_width in v7, which is apicid_pkg_offset(&topo_info) - apicid_core_offset(&topo_info);
And it is straightforward that it means the number of bits in x2APICID to encode different addressable cores.
But it is not similar to addressable_threads_width, the semantic changes per different cache level. In fact, you want something like bit_width_of_addressable_threads_sharing_this_level_of_cache.
So I suggest stop using the variable of "address_therads_width". Instead just apicid_*_offset() directly.
With above fixed, feel free to add my
Quoted 15 lines
target/i386/cpu.c +3
-1encode_cache_cpuid4 (env->cache_info_cpuid4.l3_cache, - (1 << die_offset), cs->nr_cores, + (1 << addressable_threads_width), + (1 << addressable_cores_width), eax, ebx, ecx, edx);break ; }@@ -6141,6 +6159,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, } }break ; + }case 5 :/* MONITOR/MWAIT Leaf */ *eax = cpu->mwait.eax;/* Smallest monitor-line size in bytes */
Re: [PATCH v9 06/21] i386/cpu: Use APIC ID info to encode cache topo in CPUID[4]
Hi Xiaoyao,
Quoted 8 lines
target/i386/cpu.c void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, · line 6148
Quoted 4 lines
case 3: /* L3 cache info */target/i386/cpu.c +1
-1- die_offset =apicid_die_offset (&topo_info);if (cpu->enable_l3_cache) { + addressable_threads_width =apicid_die_offset (&topo_info);Please get rid of the local variable @addressable_threads_width.
It is truly confusing.
There're several reasons for this:
1. This commit is trying to use APIC ID topology layout to decode 2 cache topology fields in CPUID[4], CPUID.04H:EAX[bits 25:14] and CPUID.04H:EAX[bits 31:26]. When there's a addressable_cores_width to map to CPUID.04H:EAX[bits 31:26], it's more clear to also map CPUID.04H:EAX[bits 25:14] to another variable.
2. All these 2 variables are temporary in this commit, and they will be replaed by 2 helpers in follow-up cleanup of this series.
3. Similarly, to make it easier to clean up later with the helper and for more people to review, it's neater to explicitly indicate the CPUID.04H:EAX[bits 25:14] with a variable here.
4. I call this field as addressable_threads_width since it's "Maximum number of addressable IDs for logical processors sharing this cache".
Its own name is long, but given the length, only individual words could be selected as names.
TBH, "addressable" deserves more emphasis than "sharing". The former emphasizes the fact that the number of threads chosen here is based on the APIC ID layout and does not necessarily correspond to actual threads.
Therefore, this variable is needed here.
Thanks,
Zhao
Re: [PATCH v9 06/21] i386/cpu: Use APIC ID info to encode cache topo in CPUID[4]
Hi Xiaoyao,
Did the following reason convince you? Could I take your r/b tag with current code? ;-)
Thanks,
Zhao
Quoted 46 lines · Zhao Liu
Date: Sun, 10 Mar 2024 21:38:19 +0800
From: Zhao Liu <[email protected]>
Subject: Re: [PATCH v9 06/21] i386/cpu: Use APIC ID info to encode cache
topo in CPUID[4]Hi Xiaoyao,
Quoted 8 lines
target/i386/cpu.c void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, · line 6148
Quoted 4 lines
case 3: /* L3 cache info */target/i386/cpu.c +1
-1- die_offset =apicid_die_offset (&topo_info);if (cpu->enable_l3_cache) { + addressable_threads_width =apicid_die_offset (&topo_info);Please get rid of the local variable @addressable_threads_width.
It is truly confusing.
There're several reasons for this:
1. This commit is trying to use APIC ID topology layout to decode 2 cache topology fields in CPUID[4], CPUID.04H:EAX[bits 25:14] and CPUID.04H:EAX[bits 31:26]. When there's a
addressable_cores_widthto map to CPUID.04H:EAX[bits 31:26], it's more clear to also map CPUID.04H:EAX[bits 25:14] to another variable.2. All these 2 variables are temporary in this commit, and they will be replaed by 2 helpers in follow-up cleanup of this series.
3. Similarly, to make it easier to clean up later with the helper and for more people to review, it's neater to explicitly indicate the CPUID.04H:EAX[bits 25:14] with a variable here.
4. I call this field as
addressable_threads_widthsince it's "Maximum number of addressable IDs for logical processors sharing this cache".Its own name is long, but given the length, only individual words could be selected as names.
TBH, "addressable" deserves more emphasis than "sharing". The former emphasizes the fact that the number of threads chosen here is based on the APIC ID layout and does not necessarily correspond to actual threads.
Therefore, this variable is needed here.
Thanks,
Zhao
Re: [PATCH v9 06/21] i386/cpu: Use APIC ID info to encode cache topo in CPUID[4]
In reply to Zhao LiuQuoted 11 lines · Zhao Liu
Hi Xiaoyao,
Quoted 8 lines
target/i386/cpu.c void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, · line 6148
Quoted 4 lines
case 3: /* L3 cache info */target/i386/cpu.c +1
-1- die_offset =apicid_die_offset (&topo_info);if (cpu->enable_l3_cache) { + addressable_threads_width =apicid_die_offset (&topo_info);Please get rid of the local variable @addressable_threads_width.
It is truly confusing.
Quoted 7 lines
There're several reasons for this:
1. This commit is trying to use APIC ID topology layout to decode 2 cache topology fields in CPUID[4], CPUID.04H:EAX[bits 25:14] and CPUID.04H:EAX[bits 31:26]. When there's a
addressable_cores_widthto map to CPUID.04H:EAX[bits 31:26], it's more clear to also map CPUID.04H:EAX[bits 25:14] to another variable.
I don't dislike using a variable. I dislike the name of that variable since it's misleading
Quoted 2 lines
2. All these 2 variables are temporary in this commit, and they will be replaed by 2 helpers in follow-up cleanup of this series.
you mean patch 20?
I don't see how removing the local variable @addressable_threads_width conflicts with patch 20. As a con, it introduces code churn.
Quoted 3 lines
3. Similarly, to make it easier to clean up later with the helper and for more people to review, it's neater to explicitly indicate the CPUID.04H:EAX[bits 25:14] with a variable here.
If you do want keeping the variable. Please add a comment above it to explain the meaning.
Quoted 15 lines
4. I call this field as
addressable_threads_widthsince it's "Maximum number of addressable IDs for logical processors sharing this cache".Its own name is long, but given the length, only individual words could be selected as names.
TBH, "addressable" deserves more emphasis than "sharing". The former emphasizes the fact that the number of threads chosen here is based on the APIC ID layout and does not necessarily correspond to actual threads.
Therefore, this variable is needed here.
Thanks,
Zhao
Re: [PATCH v9 06/21] i386/cpu: Use APIC ID info to encode cache topo in CPUID[4]
Quoted 19 lines · Xiaoyao Li
Date: Mon, 11 Mar 2024 17:03:02 +0800
From: Xiaoyao Li <[email protected]>
Subject: Re: [PATCH v9 06/21] i386/cpu: Use APIC ID info to encode cache
topo in CPUID[4]Quoted 13 lines · Zhao Liu
Hi Xiaoyao,
Quoted 8 lines
target/i386/cpu.c void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, · line 6148
Quoted 4 lines
case 3: /* L3 cache info */target/i386/cpu.c +1
-1- die_offset =apicid_die_offset (&topo_info);if (cpu->enable_l3_cache) { + addressable_threads_width =apicid_die_offset (&topo_info);Please get rid of the local variable @addressable_threads_width.
It is truly confusing.
There're several reasons for this:
Quoted 8 lines
Quoted 5 lines
1. This commit is trying to use APIC ID topology layout to decode 2 cache topology fields in CPUID[4], CPUID.04H:EAX[bits 25:14] and CPUID.04H:EAX[bits 31:26]. When there's a
addressable_cores_widthto map to CPUID.04H:EAX[bits 31:26], it's more clear to also map CPUID.04H:EAX[bits 25:14] to another variable.I don't dislike using a variable. I dislike the name of that variable since it's misleading
Names are hard to choose...
Quoted 8 lines
Quoted 3 lines
2. All these 2 variables are temporary in this commit, and they will be replaed by 2 helpers in follow-up cleanup of this series.
you mean patch 20?
I don't see how removing the local variable @addressable_threads_width conflicts with patch 20. As a con, it introduces code churn.
Yes...I prefer to wrap it in variables in advance, then the meaning of the fields is clearer I think.
Quoted 7 lines
Quoted 3 lines
3. Similarly, to make it easier to clean up later with the helper and for more people to review, it's neater to explicitly indicate the CPUID.04H:EAX[bits 25:14] with a variable here.
If you do want keeping the variable. Please add a comment above it to explain the meaning.
OK, I'll add comments for both 2 variables. Thanks!
PATCH v9 7/21 i386/cpu: Use APIC ID info get NumSharingCache for CPUID[0x8000001D].EAX[bits 25:14]
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
The commit 8f4202fb1080 ("i386: Populate AMD Processor Cache Information for cpuid 0x8000001D") adds the cache topology for AMD CPU by encoding the number of sharing threads directly.
From AMD's APM, NumSharingCache (CPUID[0x8000001D].EAX[bits 25:14]) means [1]:
The number of logical processors sharing this cache is the value of this field incremented by 1. To determine which logical processors are sharing a cache, determine a Share Id for each processor as follows:
ShareId = LocalApicId >> log2(NumSharingCache+1)
Logical processors with the same ShareId then share a cache. If NumSharingCache+1 is not a power of two, round it up to the next power of two.
From the description above, the calculation of this field should be same as CPUID[4].EAX[bits 25:14] for Intel CPUs. So also use the offsets of APIC ID to calculate this field.
[1]: APM, vol.3, appendix.E.4.15 Function 8000_001Dh--Cache Topology
Information
---
Changes since v7:
* Moved this patch after CPUID[4]'s similar change ("i386/cpu: Use APIC ID offset to encode cache topo in CPUID[4]"). (Xiaoyao)
* Dropped Michael/Babu's Acked/Reviewed/Tested tags since the code change due to the rebase.
* Re-added Yongwei's Tested tag For his re-testing (compilation on Intel platforms).
Changes since v3:
* Rewrote the subject. (Babu)
* Deleted the original "comment/help" expression, as this behavior is confirmed for AMD CPUs. (Babu)
* Renamed "num_apic_ids" (v3) to "num_sharing_cache" to match spec definition. (Babu)
Changes since v1:
* Renamed "l3_threads" to "num_apic_ids" in encode_cache_cpuid8000001d(). (Yanan)
* Added the description of the original commit and add Cc.
--- target/i386/cpu.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
target/i386/cpu.c +4 -4
@@ -331,7 +331,7 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx) { - uint32_t l3_threads; + uint32_t num_sharing_cache;assert (cache->size == cache->line_size * cache->associativity * cache->partitions * cache->sets);
@@ -340,11 +340,11 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache,/* L3 is shared among multiple cores */ if (cache->level ==3 ) { - l3_threads = topo_info->cores_per_die * topo_info->threads_per_core; - *eax |= (l3_threads -1 ) <<14 ; + num_sharing_cache =1 <<apicid_die_offset (topo_info); }else { - *eax |= ((topo_info->threads_per_core -1 ) <<14 ); + num_sharing_cache =1 <<apicid_core_offset (topo_info); } + *eax |= (num_sharing_cache -1 ) <<14 ;assert (cache->line_size >0 );assert (cache->partitions >0 );
Signature · 1 line
2.34.1
Re: [PATCH v9 07/21] i386/cpu: Use APIC ID info get NumSharingCache for CPUID[0x8000001D].EAX[bits 25:14]
Quoted 23 lines · Zhao Liu
From: Zhao Liu <[email protected]>
The commit
8f4202fb1080("i386: Populate AMD Processor Cache Information for cpuid0x8000001D") adds the cache topology for AMD CPU by encoding the number of sharing threads directly.From AMD's APM, NumSharingCache (CPUID[0x8000001D].EAX[bits 25:14]) means [1]:
The number of logical processors sharing this cache is the value of this field incremented by 1. To determine which logical processors are sharing a cache, determine a Share Id for each processor as follows:
ShareId = LocalApicId >> log2(NumSharingCache+1)
Logical processors with the same ShareId then share a cache. If NumSharingCache+1 is not a power of two, round it up to the next power of two.
From the description above, the calculation of this field should be same as CPUID[4].EAX[bits 25:14] for Intel CPUs. So also use the offsets of APIC ID to calculate this field.
Quoted 6 lines
[1]: APM, vol.3, appendix.E.4.15 Function 8000_001Dh--Cache Topology
InformationCc Babu Moger <[email protected]>Tested-by Yongwei Ma <[email protected]>Signed-off-by Zhao Liu <[email protected]>
Quoted 52 lines
---
Changes since v7:
* Moved this patch after CPUID[4]'s similar change ("i386/cpu: Use APIC ID offset to encode cache topo in CPUID[4]"). (Xiaoyao)
* Dropped Michael/Babu's Acked/Reviewed/Tested tags since the code change due to the rebase.
* Re-added Yongwei's Tested tag For his re-testing (compilation on Intel platforms).Changes since v3:
* Rewrote the subject. (Babu)
* Deleted the original "comment/help" expression, as this behavior is confirmed for AMD CPUs. (Babu)
* Renamed "num_apic_ids" (v3) to "num_sharing_cache" to match spec definition. (Babu)Changes since v1:
* Renamed "l3_threads" to "num_apic_ids" inencode_cache_cpuid8000001d(). (Yanan)
* Added the description of the original commit and add Cc.--- target/i386/cpu.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)target/i386/cpu.c +4
-4@@ -331,7 +331,7 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx) { - uint32_t l3_threads; + uint32_t num_sharing_cache;assert (cache->size == cache->line_size * cache->associativity * cache->partitions * cache->sets);@@ -340,11 +340,11 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache,/* L3 is shared among multiple cores */ if (cache->level ==3 ) { - l3_threads = topo_info->cores_per_die * topo_info->threads_per_core; - *eax |= (l3_threads -1 ) <<14 ; + num_sharing_cache =1 <<apicid_die_offset (topo_info); }else { - *eax |= ((topo_info->threads_per_core -1 ) <<14 ); + num_sharing_cache =1 <<apicid_core_offset (topo_info); } + *eax |= (num_sharing_cache -1 ) <<14 ;assert (cache->line_size >0 );assert (cache->partitions >0 );
Signature · 2 lines
Thanks Babu Moger
Re: [PATCH v9 07/21] i386/cpu: Use APIC ID info get NumSharingCache for CPUID[0x8000001D].EAX[bits 25:14]
In reply to Zhao LiuQuoted 23 lines · Zhao Liu
From: Zhao Liu <[email protected]>
The commit
8f4202fb1080("i386: Populate AMD Processor Cache Information for cpuid0x8000001D") adds the cache topology for AMD CPU by encoding the number of sharing threads directly.From AMD's APM, NumSharingCache (CPUID[0x8000001D].EAX[bits 25:14])
means [1]:The number of logical processors sharing this cache is the value of this field incremented by 1. To determine which logical processors are sharing a cache, determine a Share Id for each processor as follows:
ShareId = LocalApicId >> log2(NumSharingCache+1)
Logical processors with the same ShareId then share a cache. If NumSharingCache+1 is not a power of two, round it up to the next power of two.
From the description above, the calculation of this field should be same
as CPUID[4].EAX[bits 25:14] for Intel CPUs. So also use the offsets of APIC ID to calculate this field.
Quoted 6 lines
[1]: APM, vol.3, appendix.E.4.15 Function 8000_001Dh--Cache Topology
InformationCc Babu Moger <[email protected]>Tested-by Yongwei Ma <[email protected]>Signed-off-by Zhao Liu <[email protected]>
Quoted 52 lines
---
Changes since v7:
* Moved this patch after CPUID[4]'s similar change ("i386/cpu: Use APIC ID offset to encode cache topo in CPUID[4]"). (Xiaoyao)
* Dropped Michael/Babu's Acked/Reviewed/Tested tags since the code change due to the rebase.
* Re-added Yongwei's Tested tag For his re-testing (compilation on Intel platforms).Changes since v3:
* Rewrote the subject. (Babu)
* Deleted the original "comment/help" expression, as this behavior is confirmed for AMD CPUs. (Babu)
* Renamed "num_apic_ids" (v3) to "num_sharing_cache" to match spec definition. (Babu)Changes since v1:
* Renamed "l3_threads" to "num_apic_ids" inencode_cache_cpuid8000001d(). (Yanan) * Added the description of the original commit and add Cc. --- target/i386/cpu.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)target/i386/cpu.c +4
-4@@ -331,7 +331,7 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx) { - uint32_t l3_threads; + uint32_t num_sharing_cache;assert (cache->size == cache->line_size * cache->associativity * cache->partitions * cache->sets);@@ -340,11 +340,11 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache,/* L3 is shared among multiple cores */ if (cache->level ==3 ) { - l3_threads = topo_info->cores_per_die * topo_info->threads_per_core; - *eax |= (l3_threads -1 ) <<14 ; + num_sharing_cache =1 <<apicid_die_offset (topo_info); }else { - *eax |= ((topo_info->threads_per_core -1 ) <<14 ); + num_sharing_cache =1 <<apicid_core_offset (topo_info); } + *eax |= (num_sharing_cache -1 ) <<14 ;assert (cache->line_size >0 );assert (cache->partitions >0 );
PATCH v9 8/21 i386/cpu: Consolidate the use of topo_info in cpu_x86_cpuid()
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
In cpu_x86_cpuid(), there are many variables in representing the cpu topology, e.g., topo_info, cs->nr_cores and cs->nr_threads.
Since the names of cs->nr_cores/cs->nr_threads does not accurately represent its meaning, the use of cs->nr_cores/cs->nr_threads is prone to confusion and mistakes.
And the structure X86CPUTopoInfo names its members clearly, thus the variable "topo_info" should be preferred.
In addition, in cpu_x86_cpuid(), to uniformly use the topology variable, replace env->dies with topo_info.dies_per_pkg as well.
---
Changes since v8:
* Add Philippe's reviewed-by tag.
Changes since v7:
* Renamed cpus_per_pkg to threads_per_pkg. (Xiaoyao)
* Dropped Michael/Babu's Acked/Tested tags since the code change.
* Re-added Yongwei's Tested tag For his re-testing.
* Added Xiaoyao's Reviewed tag.
Changes since v3:
* Fixed typo. (Babu)
Changes since v1:
* Extracted cores_per_socket from the code block and use it as a local variable for cpu_x86_cpuid(). (Yanan)
* Removed vcpus_per_socket variable and use cpus_per_pkg directly. (Yanan)
* Replaced env->dies with topo_info.dies_per_pkg in cpu_x86_cpuid().
--- target/i386/cpu.c | 31 ++++++++++++++++++------------- 1 file changed, 18 insertions(+), 13 deletions(-)
target/i386/cpu.c +18 -13
@@ -6017,11 +6017,16 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,uint32_t limit;uint32_t signature[3 ];X86CPUTopoInfo topo_info; +uint32_t cores_per_pkg; +uint32_t threads_per_pkg; topo_info.dies_per_pkg = env->nr_dies; topo_info.cores_per_die = cs->nr_cores / env->nr_dies; topo_info.threads_per_core = cs->nr_threads; + cores_per_pkg = topo_info.cores_per_die * topo_info.dies_per_pkg; + threads_per_pkg = cores_per_pkg * topo_info.threads_per_core; +/* Calculate & apply limits for different index ranges */ if (index >=0xC0000000 ) { limit = env->cpuid_xlevel2;
@@ -6057,8 +6062,8 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, *ecx |=CPUID_EXT_OSXSAVE ; } *edx = env->features[FEAT_1_EDX ]; -if (cs->nr_cores * cs->nr_threads >1 ) { - *ebx |= (cs->nr_cores * cs->nr_threads) <<16 ; +if (threads_per_pkg >1 ) { + *ebx |= threads_per_pkg <<16 ; *edx |=CPUID_HT ; }if (!cpu->enable_pmu) {
@@ -6098,15 +6103,15 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,*/ if (*eax &31 ) {int host_vcpus_per_cache =1 + ((*eax &0x3FFC000 ) >>14 ); -int vcpus_per_socket = cs->nr_cores * cs->nr_threads; -if (cs->nr_cores >1 ) { + +if (cores_per_pkg >1 ) { addressable_cores_width =apicid_pkg_offset (&topo_info) -apicid_core_offset (&topo_info); *eax &= ~0xFC000000 ; *eax |= ((1 << addressable_cores_width) -1 ) <<26 ; } -if (host_vcpus_per_cache > vcpus_per_socket) { +if (host_vcpus_per_cache > threads_per_pkg) {/* Share the cache at package level. */ addressable_threads_width =apicid_pkg_offset (&topo_info);
@@ -6252,12 +6257,12 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,switch (count) {case 0 : *eax =apicid_core_offset (&topo_info); - *ebx = cs->nr_threads; + *ebx = topo_info.threads_per_core; *ecx |=CPUID_TOPOLOGY_LEVEL_SMT ;break ;case 1 : *eax =apicid_pkg_offset (&topo_info); - *ebx = cs->nr_cores * cs->nr_threads; + *ebx = threads_per_pkg; *ecx |=CPUID_TOPOLOGY_LEVEL_CORE ;break ;default :
@@ -6277,7 +6282,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,break ;case 0x1F :/* V2 Extended Topology Enumeration Leaf */ -if (env->nr_dies <2 ) { +if (topo_info.dies_per_pkg <2 ) { *eax = *ebx = *ecx = *edx =0 ;break ; }
@@ -6287,7 +6292,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,switch (count) {case 0 : *eax =apicid_core_offset (&topo_info); - *ebx = cs->nr_threads; + *ebx = topo_info.threads_per_core; *ecx |=CPUID_TOPOLOGY_LEVEL_SMT ;break ;case 1 :
@@ -6297,7 +6302,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,break ;case 2 : *eax =apicid_pkg_offset (&topo_info); - *ebx = cs->nr_cores * cs->nr_threads; + *ebx = threads_per_pkg; *ecx |=CPUID_TOPOLOGY_LEVEL_DIE ;break ;default :
@@ -6525,7 +6530,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,* discards multiple thread information if it is set. * So don't set it here for Intel to make Linux guests happy. */ -if (cs->nr_cores * cs->nr_threads >1 ) { +if (threads_per_pkg >1 ) {if (env->cpuid_vendor1 !=CPUID_VENDOR_INTEL_1 || env->cpuid_vendor2 !=CPUID_VENDOR_INTEL_2 || env->cpuid_vendor3 !=CPUID_VENDOR_INTEL_3 ) {
@@ -6591,7 +6596,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, *eax |= (cpu_x86_virtual_addr_width (env) <<8 ); } *ebx = env->features[FEAT_8000_0008_EBX ]; -if (cs->nr_cores * cs->nr_threads >1 ) { +if (threads_per_pkg >1 ) {/* * Bits 15:12 is "The number of bits in the initial * Core::X86::Apic::ApicId[ApicId] value that indicate
@@ -6599,7 +6604,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,* Bits 7:0 is "The number of threads in the package is NC+1" */ *ecx = (apicid_pkg_offset (&topo_info) <<12 ) | - ((cs->nr_cores * cs->nr_threads) -1 ); + (threads_per_pkg -1 ); }else { *ecx =0 ; }
Signature · 1 line
2.34.1
Re: [PATCH v9 08/21] i386/cpu: Consolidate the use of topo_info in cpu_x86_cpuid()
Quoted 6 lines · Zhao Liu
From: Zhao Liu<[email protected]>
In
cpu_x86_cpuid(), there are many variables in representing the cpu topology, e.g.,topo_info,cs->nr_coresandcs->nr_threads.Since the names of cs->nr_cores/cs->nr_threads does not accurately
Again as in v7, please changes to "cs->nr_cores and cs->nr_threads",
"cs->nr_cores/cs->nr_threads" looks like a single variable of division
Quoted 8 lines
represent its meaning, the use of cs->nr_cores/cs->nr_threads is prone to confusion and mistakes.
And the structure X86CPUTopoInfo names its members clearly, thus the variable "
topo_info" should be preferred.In addition, in
cpu_x86_cpuid(), to uniformly use the topology variable, replaceenv->dieswith topo_info.dies_per_pkg as well.
Re: [PATCH v9 08/21] i386/cpu: Consolidate the use of topo_info in cpu_x86_cpuid()
Hi Xiaoyao,
Quoted 8 lines · Xiaoyao Li
Date: Sat, 9 Mar 2024 21:48:34 +0800
From: Xiaoyao Li <[email protected]>
Subject: Re: [PATCH v9 08/21] i386/cpu: Consolidate the use oftopo_infoin
cpu_x86_cpuid()Quoted 2 lines · Zhao Liu
From: Zhao Liu<[email protected]>
Quoted 9 lines
Quoted 4 lines
In
cpu_x86_cpuid(), there are many variables in representing the cpu topology, e.g.,topo_info,cs->nr_coresandcs->nr_threads.Since the names of cs->nr_cores/cs->nr_threads does not accurately
Again as in v7, please changes to "
cs->nr_coresandcs->nr_threads","cs->nr_cores/cs->nr_threads" looks like a single variable of division
I missed to change this.
Though, TBH, legitimate division operations are going to add spaces around "/". It's also common practice to represent "or" by "/" w/t spaces. But I'll change it in the next version to make you happy!
Thanks,
Zhao
PATCH v9 9/21 i386/cpu: Introduce bitmap to cache available CPU topology levels
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
Currently, QEMU checks the specify number of topology domains to detect if there's extended topology levels (e.g., checking nr_dies).
With this bitmap, the extended CPU topology (the levels other than SMT, core and package) could be easier to detect without touching the topology details.
This is also in preparation for the follow-up to decouple CPUID[0x1F] subleaf with specific topology level.
---
Changes since v7:
* New commit to response Xiaoyao's suggestion about the gloabl variable to cache topology levels. (Xiaoyao)
--- hw/i386/x86.c | 5 ++++- include/hw/i386/topology.h | 23 +++++++++++++++++++++++ target/i386/cpu.c | 18 +++++++++++++++--- target/i386/cpu.h | 4 ++++ target/i386/kvm/kvm.c | 3 ++- 5 files changed, 48 insertions(+), 5 deletions(-)
hw/i386/x86.c +4 -1
@@ -313,7 +313,10 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,init_topo_info (&topo_info, x86ms); - env->nr_dies = ms->smp.dies; +if (ms->smp.dies >1 ) { + env->nr_dies = ms->smp.dies; +set_bit (CPU_TOPO_LEVEL_DIE , env->avail_cpu_topo); + }/* * If APIC ID is not set,
include/hw/i386/topology.h +23 -0
@@ -60,6 +60,21 @@ typedef struct X86CPUTopoInfo {unsigned threads_per_core; } X86CPUTopoInfo; +/* +* CPUTopoLevel is the general i386 topology hierarchical representation, +* ordered by increasing hierarchical relationship. +* Its enumeration value is not bound to the type value of Intel (CPUID[0x1F]) +* or AMD (CPUID[0x80000026]). +*/ +enum CPUTopoLevel { +CPU_TOPO_LEVEL_INVALID , +CPU_TOPO_LEVEL_SMT , +CPU_TOPO_LEVEL_CORE , +CPU_TOPO_LEVEL_DIE , +CPU_TOPO_LEVEL_PACKAGE , +CPU_TOPO_LEVEL_MAX , +}; +/* Return the bit width needed for 'count' IDs */ static unsigned apicid_bitwidth_for_count (unsigned count) {
@@ -168,4 +183,12 @@ static inline apic_id_t x86_apicid_from_cpu_idx(X86CPUTopoInfo *topo_info, return x86_apicid_from_topo_ids(topo_info, &topo_ids); } +/* +* Check whether there's extended topology level (die)? +*/ +static inline bool x86_has_extended_topo (unsigned long *topo_bitmap) +{ +return test_bit (CPU_TOPO_LEVEL_DIE , topo_bitmap); +} +#endif /* HW_I386_TOPOLOGY_H */
target/i386/cpu.c +15 -3
@@ -6282,7 +6282,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,break ;case 0x1F :/* V2 Extended Topology Enumeration Leaf */ -if (topo_info.dies_per_pkg <2 ) { +if (!x86_has_extended_topo (env->avail_cpu_topo)) { *eax = *ebx = *ecx = *edx =0 ;break ; }
@@ -7114,7 +7114,7 @@ void x86_cpu_expand_features(X86CPU *cpu, Error **errp)* cpu->vendor_cpuid_only has been unset for compatibility with older * machine types. */ -if ((env->nr_dies >1 ) && +if (x86_has_extended_topo (env->avail_cpu_topo) && (IS_INTEL_CPU (env) || !cpu->vendor_cpuid_only)) {x86_cpu_adjust_level (cpu, &env->cpuid_min_level,0x1F ); }
@@ -7620,13 +7620,25 @@ static void x86_cpu_post_initfn(Object *obj)accel_cpu_instance_init (CPU (obj)); } +static void x86_cpu_init_default_topo (X86CPU *cpu) +{ +CPUX86State *env = &cpu->env; + + env->nr_dies =1 ; + +/* SMT, core and package levels are set by default. */ +set_bit (CPU_TOPO_LEVEL_SMT , env->avail_cpu_topo); +set_bit (CPU_TOPO_LEVEL_CORE , env->avail_cpu_topo); +set_bit (CPU_TOPO_LEVEL_PACKAGE , env->avail_cpu_topo); +} +static void x86_cpu_initfn (Object *obj) {X86CPU *cpu =X86_CPU (obj);X86CPUClass *xcc =X86_CPU_GET_CLASS (obj);CPUX86State *env = &cpu->env; - env->nr_dies =1 ; +x86_cpu_init_default_topo (cpu); object_property_add(obj, "feature-words", "X86CPUFeatureWordInfo", x86_cpu_get_feature_words,
target/i386/cpu.h +4 -0
@@ -24,6 +24,7 @@#include "cpu-qom.h" #include "kvm/hyperv-proto.h" #include "exec/cpu-defs.h" +#include "hw/i386/topology.h" #include "qapi/qapi-types-common.h" #include "qemu/cpu-float.h" #include "qemu/timer.h"
@@ -1892,6 +1893,9 @@ typedef struct CPUArchState {/* Number of dies within this CPU package. */ unsigned nr_dies; + +/* Bitmap of available CPU topology levels for this CPU. */ +DECLARE_BITMAP (avail_cpu_topo,CPU_TOPO_LEVEL_MAX ); } CPUX86State;struct kvm_msrs ;
target/i386/kvm/kvm.c +2 -1
@@ -50,6 +50,7 @@#include "hw/i386/apic_internal.h" #include "hw/i386/apic-msidef.h" #include "hw/i386/intel_iommu.h" +#include "hw/i386/topology.h" #include "hw/i386/x86-iommu.h" #include "hw/i386/e820_memory_layout.h"
@@ -1913,7 +1914,7 @@ int kvm_arch_init_vcpu(CPUState *cs)break ; }case 0x1f : -if (env->nr_dies <2 ) { +if (!x86_has_extended_topo (env->avail_cpu_topo)) { cpuid_i--;break ; }
Signature · 1 line
2.34.1
Re: [PATCH v9 09/21] i386/cpu: Introduce bitmap to cache available CPU topology levels
Quoted 9 lines · Zhao Liu
From: Zhao Liu <[email protected]>
Currently, QEMU checks the specify number of topology domains to detect if there's extended topology levels (e.g., checking
nr_dies).With this bitmap, the extended CPU topology (the levels other than SMT, core and package) could be easier to detect without touching the topology details.
Quoted 5 lines
This is also in preparation for the follow-up to decouple CPUID[0x1F] subleaf with specific topology level.
Tested-by Yongwei Ma <[email protected]>Signed-off-by Zhao Liu <[email protected]>
Re: [PATCH v9 09/21] i386/cpu: Introduce bitmap to cache available CPU topology levels
Quoted 15 lines · Xiaoyao Li
Date: Mon, 11 Mar 2024 14:28:17 +0800
From: Xiaoyao Li <[email protected]>
Subject: Re: [PATCH v9 09/21] i386/cpu: Introduce bitmap to cache available
CPU topology levelsQuoted 9 lines · Zhao Liu
From: Zhao Liu <[email protected]>
Currently, QEMU checks the specify number of topology domains to detect if there's extended topology levels (e.g., checking
nr_dies).With this bitmap, the extended CPU topology (the levels other than SMT, core and package) could be easier to detect without touching the topology details.
Quoted 8 lines
Quoted 5 lines
This is also in preparation for the follow-up to decouple CPUID[0x1F] subleaf with specific topology level.
Tested-by Yongwei Ma <[email protected]>Signed-off-by Zhao Liu <[email protected]>Reviewed-by Xiaoyao Li <[email protected]>
Thanks for your review!
Regrads,
Zhao
PATCH v9 10/21 i386: Split topology types of CPUID[0x1F] from the definitions of CPUID[0xB]
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
CPUID[0xB] defines SMT, Core and Invalid types, and this leaf is shared by Intel and AMD CPUs.
But for extended topology levels, Intel CPU (in CPUID[0x1F]) and AMD CPU (in CPUID[0x80000026]) have the different definitions with different enumeration values.
Though CPUID[0x80000026] hasn't been implemented in QEMU, to avoid possible misunderstanding, split topology types of CPUID[0x1F] from the definitions of CPUID[0xB] and introduce CPUID[0x1F]-specific topology types.
---
Changes since v8:
* Add Philippe's reviewed-by tag.
Changes since v3:
* New commit to prepare to refactor CPUID[0x1F] encoding.
--- target/i386/cpu.c | 14 +++++++------- target/i386/cpu.h | 13 +++++++++---- 2 files changed, 16 insertions(+), 11 deletions(-)
target/i386/cpu.c +7 -7
@@ -6258,17 +6258,17 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,case 0 : *eax =apicid_core_offset (&topo_info); *ebx = topo_info.threads_per_core; - *ecx |=CPUID_TOPOLOGY_LEVEL_SMT ; + *ecx |=CPUID_B_ECX_TOPO_LEVEL_SMT <<8 ;break ;case 1 : *eax =apicid_pkg_offset (&topo_info); *ebx = threads_per_pkg; - *ecx |=CPUID_TOPOLOGY_LEVEL_CORE ; + *ecx |=CPUID_B_ECX_TOPO_LEVEL_CORE <<8 ;break ;default : *eax =0 ; *ebx =0 ; - *ecx |=CPUID_TOPOLOGY_LEVEL_INVALID ; + *ecx |=CPUID_B_ECX_TOPO_LEVEL_INVALID <<8 ; }assert (!(*eax & ~0x1f ));
@@ -6293,22 +6293,22 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,case 0 : *eax =apicid_core_offset (&topo_info); *ebx = topo_info.threads_per_core; - *ecx |=CPUID_TOPOLOGY_LEVEL_SMT ; + *ecx |=CPUID_1F_ECX_TOPO_LEVEL_SMT <<8 ;break ;case 1 : *eax =apicid_die_offset (&topo_info); *ebx = topo_info.cores_per_die * topo_info.threads_per_core; - *ecx |=CPUID_TOPOLOGY_LEVEL_CORE ; + *ecx |=CPUID_1F_ECX_TOPO_LEVEL_CORE <<8 ;break ;case 2 : *eax =apicid_pkg_offset (&topo_info); *ebx = threads_per_pkg; - *ecx |=CPUID_TOPOLOGY_LEVEL_DIE ; + *ecx |=CPUID_1F_ECX_TOPO_LEVEL_DIE <<8 ;break ;default : *eax =0 ; *ebx =0 ; - *ecx |=CPUID_TOPOLOGY_LEVEL_INVALID ; + *ecx |=CPUID_1F_ECX_TOPO_LEVEL_INVALID <<8 ; }assert (!(*eax & ~0x1f )); *ebx &=0xffff ;/* The count doesn't need to be reliable. */
target/i386/cpu.h +9 -4
@@ -1017,10 +1017,15 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,#define CPUID_MWAIT_EMX (1U << 0)/* enumeration supported */ /* CPUID[0xB].ECX level types */ -#define CPUID_TOPOLOGY_LEVEL_INVALID (0U << 8) -#define CPUID_TOPOLOGY_LEVEL_SMT (1U << 8) -#define CPUID_TOPOLOGY_LEVEL_CORE (2U << 8) -#define CPUID_TOPOLOGY_LEVEL_DIE (5U << 8) +#define CPUID_B_ECX_TOPO_LEVEL_INVALID 0 +#define CPUID_B_ECX_TOPO_LEVEL_SMT 1 +#define CPUID_B_ECX_TOPO_LEVEL_CORE 2 + +/* COUID[0x1F].ECX level types */ +#define CPUID_1F_ECX_TOPO_LEVEL_INVALID CPUID_B_ECX_TOPO_LEVEL_INVALID +#define CPUID_1F_ECX_TOPO_LEVEL_SMT CPUID_B_ECX_TOPO_LEVEL_SMT +#define CPUID_1F_ECX_TOPO_LEVEL_CORE CPUID_B_ECX_TOPO_LEVEL_CORE +#define CPUID_1F_ECX_TOPO_LEVEL_DIE 5/* MSR Feature Bits */ #define MSR_ARCH_CAP_RDCL_NO (1U << 0)
Signature · 1 line
2.34.1
PATCH v9 11/21 i386/cpu: Decouple CPUID[0x1F] subleaf with specific topology level
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
At present, the subleaf 0x02 of CPUID[0x1F] is bound to the "die" level.
In fact, the specific topology level exposed in 0x1F depends on the platform's support for extension levels (module, tile and die).
To help expose "module" level in 0x1F, decouple CPUID[0x1F] subleaf with specific topology level.
---
Changes since v7:
* Refactored the encode_topo_cpuid1f() to use traversal to search the encoded level and avoid using static variables. (Xiaoyao)
- Since the total number of levels in the bitmap is not too large, the overhead of traversing is supposed to be acceptable.
* Renamed the variable num_cpus_next_level to num_threads_next_level. (Xiaoyao)
* Renamed the helper num_cpus_by_topo_level() to num_threads_by_topo_level(). (Xiaoyao)
* Dropped Michael/Babu's Acked/Tested tags since the code change.
* Re-added Yongwei's Tested tag For his re-testing.
Changes since v3:
* New patch to prepare to expose module level in 0x1F.
* Moved the CPUTopoLevel enumeration definition from "i386: Add cache topology info in CPUCacheInfo" to this patch. Note, to align with topology types in SDM, revert the name of CPU_TOPO_LEVEL_UNKNOW to CPU_TOPO_LEVEL_INVALID.
--- target/i386/cpu.c | 138 +++++++++++++++++++++++++++++++++++++--------- 1 file changed, 113 insertions(+), 25 deletions(-)
target/i386/cpu.c +113 -25
@@ -269,6 +269,118 @@ static void encode_cache_cpuid4(CPUCacheInfo *cache, (cache->complex_indexing ?CACHE_COMPLEX_IDX :0 ); } +static uint32_t num_threads_by_topo_level (X86CPUTopoInfo *topo_info, +enum CPUTopoLevel topo_level) +{ +switch (topo_level) { +case CPU_TOPO_LEVEL_SMT : +return 1 ; +case CPU_TOPO_LEVEL_CORE : +return topo_info->threads_per_core; +case CPU_TOPO_LEVEL_DIE : +return topo_info->threads_per_core * topo_info->cores_per_die; +case CPU_TOPO_LEVEL_PACKAGE : +return topo_info->threads_per_core * topo_info->cores_per_die * + topo_info->dies_per_pkg; +default : +g_assert_not_reached (); + } +return 0 ; +} + +static uint32_t apicid_offset_by_topo_level (X86CPUTopoInfo *topo_info, +enum CPUTopoLevel topo_level) +{ +switch (topo_level) { +case CPU_TOPO_LEVEL_SMT : +return 0 ; +case CPU_TOPO_LEVEL_CORE : +return apicid_core_offset (topo_info); +case CPU_TOPO_LEVEL_DIE : +return apicid_die_offset (topo_info); +case CPU_TOPO_LEVEL_PACKAGE : +return apicid_pkg_offset (topo_info); +default : +g_assert_not_reached (); + } +return 0 ; +} + +static uint32_t cpuid1f_topo_type (enum CPUTopoLevel topo_level) +{ +switch (topo_level) { +case CPU_TOPO_LEVEL_INVALID : +return CPUID_1F_ECX_TOPO_LEVEL_INVALID ; +case CPU_TOPO_LEVEL_SMT : +return CPUID_1F_ECX_TOPO_LEVEL_SMT ; +case CPU_TOPO_LEVEL_CORE : +return CPUID_1F_ECX_TOPO_LEVEL_CORE ; +case CPU_TOPO_LEVEL_DIE : +return CPUID_1F_ECX_TOPO_LEVEL_DIE ; +default : +/* Other types are not supported in QEMU. */ +g_assert_not_reached (); + } +return 0 ; +} + +static void encode_topo_cpuid1f (CPUX86State *env,uint32_t count, +X86CPUTopoInfo *topo_info, +uint32_t *eax,uint32_t *ebx, +uint32_t *ecx,uint32_t *edx) +{ +X86CPU *cpu =env_archcpu (env); +unsigned long level; +uint32_t num_threads_next_level, offset_next_level; + +assert (count +1 <CPU_TOPO_LEVEL_MAX ); + +/* +* Find the No.count topology levels in avail_cpu_topo bitmap. +* Start from bit 0 (CPU_TOPO_LEVEL_INVALID). +*/ + level =CPU_TOPO_LEVEL_INVALID ; +for (int i =0 ; i <= count; i++) { + level =find_next_bit (env->avail_cpu_topo, +CPU_TOPO_LEVEL_PACKAGE , + level +1 ); + +/* +* CPUID[0x1f] doesn't explicitly encode the package level, +* and it just encode the invalid level (all fields are 0) +* into the last subleaf of 0x1f. +*/ +if (level ==CPU_TOPO_LEVEL_PACKAGE ) { + level =CPU_TOPO_LEVEL_INVALID ; +break ; + } + } + +if (level ==CPU_TOPO_LEVEL_INVALID ) { + num_threads_next_level =0 ; + offset_next_level =0 ; + }else { +unsigned long next_level; + + next_level =find_next_bit (env->avail_cpu_topo, +CPU_TOPO_LEVEL_PACKAGE , + level +1 ); + num_threads_next_level =num_threads_by_topo_level (topo_info, + next_level); + offset_next_level =apicid_offset_by_topo_level (topo_info, + next_level); + } + + *eax = offset_next_level; + *ebx = num_threads_next_level; + *ebx &=0xffff ;/* The count doesn't need to be reliable. */ + *ecx = count &0xff ; + *ecx |=cpuid1f_topo_type (level) <<8 ; + *edx = cpu->apic_id; + +assert (!(*eax & ~0x1f )); +} +/* Encode cache info for CPUID[0x80000005].ECX or CPUID[0x80000005].EDX */ static uint32_t encode_cache_cpuid80000005 (CPUCacheInfo *cache) {
@@ -6287,31 +6399,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,break ; } - *ecx = count &0xff ; - *edx = cpu->apic_id; -switch (count) { -case 0 : - *eax =apicid_core_offset (&topo_info); - *ebx = topo_info.threads_per_core; - *ecx |=CPUID_1F_ECX_TOPO_LEVEL_SMT <<8 ; -break ; -case 1 : - *eax =apicid_die_offset (&topo_info); - *ebx = topo_info.cores_per_die * topo_info.threads_per_core; - *ecx |=CPUID_1F_ECX_TOPO_LEVEL_CORE <<8 ; -break ; -case 2 : - *eax =apicid_pkg_offset (&topo_info); - *ebx = threads_per_pkg; - *ecx |=CPUID_1F_ECX_TOPO_LEVEL_DIE <<8 ; -break ; -default : - *eax =0 ; - *ebx =0 ; - *ecx |=CPUID_1F_ECX_TOPO_LEVEL_INVALID <<8 ; - } -assert (!(*eax & ~0x1f )); - *ebx &=0xffff ;/* The count doesn't need to be reliable. */ +encode_topo_cpuid1f (env, count, &topo_info, eax, ebx, ecx, edx);break ;case 0xD : {/* Processor Extended State */
Signature · 1 line
2.34.1
Re: [PATCH v9 11/21] i386/cpu: Decouple CPUID[0x1F] subleaf with specific topology level
Quoted 4 lines · Zhao Liu
From: Zhao Liu <[email protected]>
At present, the subleaf
0x02of CPUID[0x1F] is bound to the "die" level.
Quoted 8 lines
In fact, the specific topology level exposed in
0x1Fdepends on the platform's support for extension levels (module, tile and die).To help expose "module" level in
0x1F, decouple CPUID[0x1F] subleaf with specific topology level.Tested-by Yongwei Ma <[email protected]>Signed-off-by Zhao Liu <[email protected]>
Besides, some nits below.
Quoted 90 lines
---
Changes since v7:
* Refactored theencode_topo_cpuid1f()to use traversal to search the encoded level and avoid using static variables. (Xiaoyao)
- Since the total number of levels in the bitmap is not too large, the overhead of traversing is supposed to be acceptable.
* Renamed the variablenum_cpus_next_leveltonum_threads_next_level. (Xiaoyao)
* Renamed the helpernum_cpus_by_topo_level()tonum_threads_by_topo_level(). (Xiaoyao)
* Dropped Michael/Babu's Acked/Tested tags since the code change.
* Re-added Yongwei's Tested tag For his re-testing.Changes since v3:
* New patch to prepare to expose module level in0x1F.
* Moved the CPUTopoLevel enumeration definition from "i386: Add cache topology info in CPUCacheInfo" to this patch. Note, to align with topology types in SDM, revert the name ofCPU_TOPO_LEVEL_UNKNOWtoCPU_TOPO_LEVEL_INVALID.--- target/i386/cpu.c | 138 +++++++++++++++++++++++++++++++++++++--------- 1 file changed, 113 insertions(+), 25 deletions(-)target/i386/cpu.c +59
-0@@ -269,6 +269,118 @@ static void encode_cache_cpuid4(CPUCacheInfo *cache, (cache->complex_indexing ?CACHE_COMPLEX_IDX :0 ); } +static uint32_t num_threads_by_topo_level (X86CPUTopoInfo *topo_info, +enum CPUTopoLevel topo_level) +{ +switch (topo_level) { +case CPU_TOPO_LEVEL_SMT : +return 1 ; +case CPU_TOPO_LEVEL_CORE : +return topo_info->threads_per_core; +case CPU_TOPO_LEVEL_DIE : +return topo_info->threads_per_core * topo_info->cores_per_die; +case CPU_TOPO_LEVEL_PACKAGE : +return topo_info->threads_per_core * topo_info->cores_per_die * + topo_info->dies_per_pkg; +default : +g_assert_not_reached (); + } +return 0 ; +} + +static uint32_t apicid_offset_by_topo_level (X86CPUTopoInfo *topo_info, +enum CPUTopoLevel topo_level) +{ +switch (topo_level) { +case CPU_TOPO_LEVEL_SMT : +return 0 ; +case CPU_TOPO_LEVEL_CORE : +return apicid_core_offset (topo_info); +case CPU_TOPO_LEVEL_DIE : +return apicid_die_offset (topo_info); +case CPU_TOPO_LEVEL_PACKAGE : +return apicid_pkg_offset (topo_info); +default : +g_assert_not_reached (); + } +return 0 ; +} + +static uint32_t cpuid1f_topo_type (enum CPUTopoLevel topo_level) +{ +switch (topo_level) { +case CPU_TOPO_LEVEL_INVALID : +return CPUID_1F_ECX_TOPO_LEVEL_INVALID ; +case CPU_TOPO_LEVEL_SMT : +return CPUID_1F_ECX_TOPO_LEVEL_SMT ; +case CPU_TOPO_LEVEL_CORE : +return CPUID_1F_ECX_TOPO_LEVEL_CORE ; +case CPU_TOPO_LEVEL_DIE : +return CPUID_1F_ECX_TOPO_LEVEL_DIE ; +default : +/* Other types are not supported in QEMU. */ +g_assert_not_reached (); + } +return 0 ; +} + +static void encode_topo_cpuid1f (CPUX86State *env,uint32_t count, +X86CPUTopoInfo *topo_info, +uint32_t *eax,uint32_t *ebx, +uint32_t *ecx,uint32_t *edx)
target/i386/cpu.c static void encode_cache_cpuid4(CPUCacheInfo *cache, · line 340
Quoted 10 lines
target/i386/cpu.c +10
-0+{ +X86CPU *cpu =env_archcpu (env); +unsigned long level; +uint32_t num_threads_next_level, offset_next_level; + +assert (count +1 <CPU_TOPO_LEVEL_MAX ); + +/* +* Find the No.count topology levels in avail_cpu_topo bitmap. +* Start from bit 0 (CPU_TOPO_LEVEL_INVALID).
AFAICS, it starts from bit 1 (CPU_TOPO_LEVEL_SMT). Because the initial value of level is CPU_TOPO_LEVEL_INVALID, but the first round of the loop is to find the valid bit starting from (level + 1).
target/i386/cpu.c static void encode_cache_cpuid4(CPUCacheInfo *cache, · line 352
Quoted 12 lines
target/i386/cpu.c +12
-0+*/ + level =CPU_TOPO_LEVEL_INVALID ; +for (int i =0 ; i <= count; i++) { + level =find_next_bit (env->avail_cpu_topo, +CPU_TOPO_LEVEL_PACKAGE , + level +1 ); + +/* +* CPUID[0x1f] doesn't explicitly encode the package level, +* and it just encode the invalid level (all fields are 0) +* into the last subleaf of 0x1f. +*/
QEMU will never set bit CPU_TOPO_LEVEL_PACKAGE in env->avail_cpu_topo.
So I think we should assert() it instead of fixing it silently.
target/i386/cpu.c static void encode_cache_cpuid4(CPUCacheInfo *cache, · line 363
Quoted 11 lines
target/i386/cpu.c +11
-0+if (level ==CPU_TOPO_LEVEL_PACKAGE ) { + level =CPU_TOPO_LEVEL_INVALID ; +break ; + } + } + +if (level ==CPU_TOPO_LEVEL_INVALID ) { + num_threads_next_level =0 ; + offset_next_level =0 ; + }else { +unsigned long next_level;
please define it at the beginning of the function. e.g.,
target/i386/cpu.c static void encode_cache_cpuid4(CPUCacheInfo *cache, · line 376
Quoted 12 lines
target/i386/cpu.c +12
-0+ next_level =find_next_bit (env->avail_cpu_topo, +CPU_TOPO_LEVEL_PACKAGE , + level +1 ); + num_threads_next_level =num_threads_by_topo_level (topo_info, + next_level); + offset_next_level =apicid_offset_by_topo_level (topo_info, + next_level); + } + + *eax = offset_next_level; + *ebx = num_threads_next_level; + *ebx &=0xffff ;/* The count doesn't need to be reliable. */
we can combine them together. e.g.,
*ebx = num_threads_next_level & 0xffff; /* ... */
target/i386/cpu.c static void encode_cache_cpuid4(CPUCacheInfo *cache, · line 378
Quoted 2 lines
target/i386/cpu.c +2
-0+ *ecx = count &0xff ; + *ecx |=cpuid1f_topo_type (level) <<8 ;
Ditto,
*ecx = count & 0xff | cpuid1f_topo_type(level) << 8;
Quoted 41 lines
target/i386/cpu.c +6
-25+ *edx = cpu->apic_id; + +assert (!(*eax & ~0x1f )); +} +/* Encode cache info for CPUID[0x80000005].ECX or CPUID[0x80000005].EDX */ static uint32_t encode_cache_cpuid80000005 (CPUCacheInfo *cache) {@@ -6287,31 +6399,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,break ; } - *ecx = count &0xff ; - *edx = cpu->apic_id; -switch (count) { -case 0 : - *eax =apicid_core_offset (&topo_info); - *ebx = topo_info.threads_per_core; - *ecx |=CPUID_1F_ECX_TOPO_LEVEL_SMT <<8 ; -break ; -case 1 : - *eax =apicid_die_offset (&topo_info); - *ebx = topo_info.cores_per_die * topo_info.threads_per_core; - *ecx |=CPUID_1F_ECX_TOPO_LEVEL_CORE <<8 ; -break ; -case 2 : - *eax =apicid_pkg_offset (&topo_info); - *ebx = threads_per_pkg; - *ecx |=CPUID_1F_ECX_TOPO_LEVEL_DIE <<8 ; -break ; -default : - *eax =0 ; - *ebx =0 ; - *ecx |=CPUID_1F_ECX_TOPO_LEVEL_INVALID <<8 ; - } -assert (!(*eax & ~0x1f )); - *ebx &=0xffff ;/* The count doesn't need to be reliable. */ +encode_topo_cpuid1f (env, count, &topo_info, eax, ebx, ecx, edx);break ;case 0xD : {/* Processor Extended State */
Re: [PATCH v9 11/21] i386/cpu: Decouple CPUID[0x1F] subleaf with specific topology level
Quoted 13 lines · Xiaoyao Li
Date: Mon, 11 Mar 2024 16:45:41 +0800
From: Xiaoyao Li <[email protected]>
Subject: Re: [PATCH v9 11/21] i386/cpu: Decouple CPUID[0x1F] subleaf with
specific topology levelQuoted 7 lines · Zhao Liu
From: Zhao Liu <[email protected]>
At present, the subleaf
0x02of CPUID[0x1F] is bound to the "die" level.In fact, the specific topology level exposed in
0x1Fdepends on the platform's support for extension levels (module, tile and die).
Quoted 7 lines
Quoted 5 lines
To help expose "module" level in
0x1F, decouple CPUID[0x1F] subleaf with specific topology level.Tested-by Yongwei Ma <[email protected]>Signed-off-by Zhao Liu <[email protected]>Reviewed-by Xiaoyao Li <[email protected]>
Thanks!
Quoted 2 lines
Besides, some nits below.
[snip]
Quoted 10 lines
Quoted 10 lines
target/i386/cpu.c +10
-0+static void encode_topo_cpuid1f (CPUX86State *env,uint32_t count, +X86CPUTopoInfo *topo_info, +uint32_t *eax,uint32_t *ebx, +uint32_t *ecx,uint32_t *edx) +{ +X86CPU *cpu =env_archcpu (env); +unsigned long level; +uint32_t num_threads_next_level, offset_next_level; + +assert (count +1 <CPU_TOPO_LEVEL_MAX );
Quoted 8 lines
target/i386/cpu.c static void encode_cache_cpuid4(CPUCacheInfo *cache, · line 340
Quoted 4 lines
target/i386/cpu.c +4
-0+ +/* +* Find the No.count topology levels in avail_cpu_topo bitmap. +* Start from bit 0 (CPU_TOPO_LEVEL_INVALID). AFAICS, it starts from bit 1 (
CPU_TOPO_LEVEL_SMT). Because the initial value of level isCPU_TOPO_LEVEL_INVALID, but the first round of the loop is to find the valid bit starting from (level + 1).
Yes, this description is much clearer.
Quoted 13 lines
target/i386/cpu.c static void encode_cache_cpuid4(CPUCacheInfo *cache, · line 352
Quoted 12 lines
target/i386/cpu.c +12
-0+*/ + level =CPU_TOPO_LEVEL_INVALID ; +for (int i =0 ; i <= count; i++) { + level =find_next_bit (env->avail_cpu_topo, +CPU_TOPO_LEVEL_PACKAGE , + level +1 ); + +/* +* CPUID[0x1f] doesn't explicitly encode the package level, +* and it just encode the invalid level (all fields are 0) +* into the last subleaf of 0x1f. +*/
Quoted 1 line
QEMU will never set bit
CPU_TOPO_LEVEL_PACKAGEinenv->avail_cpu_topo.
In the patch 9 [1], I set the CPU_TOPO_LEVEL_PACKAGE in bitmap. This level is a basic topology level in general, so it's worth being set.
Only in Intel's 0x1F, it doesn't have a corresponding type, and where I use it as a termination condition for 0x1F encoding (not an error case).
[1]: https://lore.kernel.org/qemu-devel/[email protected]/
Quoted 14 lines
So I think we should
assert()it instead of fixing it silently.target/i386/cpu.c static void encode_cache_cpuid4(CPUCacheInfo *cache, · line 363
Quoted 11 lines
target/i386/cpu.c +11
-0+if (level ==CPU_TOPO_LEVEL_PACKAGE ) { + level =CPU_TOPO_LEVEL_INVALID ; +break ; + } + } + +if (level ==CPU_TOPO_LEVEL_INVALID ) { + num_threads_next_level =0 ; + offset_next_level =0 ; + }else { +unsigned long next_level;
Quoted 1 line
please define it at the beginning of the function. e.g.,
Okay, I'll put the declaration of "next_level" at the beginning of this function with a current variable "level".
Quoted 16 lines
target/i386/cpu.c static void encode_cache_cpuid4(CPUCacheInfo *cache, · line 376
Quoted 13 lines
target/i386/cpu.c +12
-0+ next_level =find_next_bit (env->avail_cpu_topo, +CPU_TOPO_LEVEL_PACKAGE , + level +1 ); + num_threads_next_level =num_threads_by_topo_level (topo_info, + next_level); + offset_next_level =apicid_offset_by_topo_level (topo_info, + next_level); + } + + *eax = offset_next_level; + *ebx = num_threads_next_level; + *ebx &=0xffff ;/* The count doesn't need to be reliable. */ we can combine them together. e.g.,
Quoted 8 lines
*ebx =
num_threads_next_level&0xffff; /* ... */target/i386/cpu.c static void encode_cache_cpuid4(CPUCacheInfo *cache, · line 378
Quoted 2 lines
target/i386/cpu.c +2
-0+ *ecx = count &0xff ; + *ecx |=cpuid1f_topo_type (level) <<8 ;Ditto,
*ecx = count &
0xff|cpuid1f_topo_type(level)<< 8;
OK, will combine these.
Quoted 5 lines
Quoted 5 lines
target/i386/cpu.c +5
-0+ *edx = cpu->apic_id; + +assert (!(*eax & ~0x1f )); +} +
PATCH v9 12/21 i386: Introduce module level cpu topology to CPUX86State
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
Intel CPUs implement module level on hybrid client products (e.g., ADL-N, MTL, etc) and E-core server products.
A module contains a set of cores that share certain resources (in current products, the resource usually includes L2 cache, as well as module scoped features and MSRs).
Module level support is the prerequisite for L2 cache topology on module level. With module level, we can implement the Guest's CPU topology and future cache topology to be consistent with the Host's on Intel hybrid client/E-core server platforms.
---
Changes since v7:
* Mapped x86 module to smp module instead of cluster.
* Re-wrote the commit message to explain the reason why we needs module level.
* Dropped Michael/Babu's ACKed/Tested tags since the code change.
* Re-added Yongwei's Tested tag For his re-testing.
Changes since v1:
* The background of the introduction of the "cluster" parameter and its exact meaning were revised according to Yanan's explanation. (Yanan)
--- hw/i386/x86.c | 5 +++++ target/i386/cpu.c | 1 + target/i386/cpu.h | 3 +++ 3 files changed, 9 insertions(+)
hw/i386/x86.c +5 -0
@@ -313,6 +313,11 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,init_topo_info (&topo_info, x86ms); +if (ms->smp.modules >1 ) { + env->nr_modules = ms->smp.modules; +/* TODO: Expose module level in CPUID[0x1F]. */ + } +if (ms->smp.dies >1 ) { env->nr_dies = ms->smp.dies;set_bit (CPU_TOPO_LEVEL_DIE , env->avail_cpu_topo);
target/i386/cpu.c +1 -0
@@ -7712,6 +7712,7 @@ static void x86_cpu_init_default_topo(X86CPU *cpu) {CPUX86State *env = &cpu->env; + env->nr_modules =1 ; env->nr_dies =1 ;/* SMT, core and package levels are set by default. */
target/i386/cpu.h +3 -0
@@ -1899,6 +1899,9 @@ typedef struct CPUArchState {/* Number of dies within this CPU package. */ unsigned nr_dies; +/* Number of modules within one die. */ +unsigned nr_modules; +/* Bitmap of available CPU topology levels for this CPU. */ DECLARE_BITMAP (avail_cpu_topo,CPU_TOPO_LEVEL_MAX ); } CPUX86State;
Signature · 1 line
2.34.1
PATCH v9 13/21 i386: Support modules_per_die in X86CPUTopoInfo
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
Support module level in i386 cpu topology structure "X86CPUTopoInfo".
Since x86 does not yet support the "modules" parameter in "-smp", X86CPUTopoInfo.modules_per_die is currently always 1.
Therefore, the module level width in APIC ID, which can be calculated by "apicid_bitwidth_for_count(topo_info->modules_per_die)", is always 0 for now, so we can directly add APIC ID related helpers to support module level parsing.
In addition, update topology structure in test-x86-topo.c.
---
Changes since v7:
* Mapped x86 module to smp module instead of cluster.
* Dropped Michael/Babu's ACKed/Tested tags since the code change.
* Re-added Yongwei's Tested tag For his re-testing.
Changes since v3:
* Droped the description about not exposing module level in commit message.
* Updated topology related calculation in newly added helpers:
num_cpus_by_topo_level() and apicid_offset_by_topo_level().
Changes since v1:
* Included module level related helpers (apicid_module_width() and apicid_module_offset()) in this patch. (Yanan)
--- hw/i386/x86.c | 9 +++++++- include/hw/i386/topology.h | 22 +++++++++++++++---- target/i386/cpu.c | 13 ++++++----- tests/unit/test-x86-topo.c | 45 ++++++++++++++++++++------------------ 4 files changed, 58 insertions(+), 31 deletions(-)
hw/i386/x86.c +8 -1
@@ -72,7 +72,14 @@ static void init_topo_info(X86CPUTopoInfo *topo_info,MachineState *ms =MACHINE (x86ms); topo_info->dies_per_pkg = ms->smp.dies; - topo_info->cores_per_die = ms->smp.cores; +/* +* Though smp.modules means the number of modules in one cluster, +* i386 doesn't support cluster level so that the smp.clusters +* always defaults to 1, therefore using smp.modules directly is +* fine here. +*/ + topo_info->modules_per_die = ms->smp.modules; + topo_info->cores_per_module = ms->smp.cores; topo_info->threads_per_core = ms->smp.threads; }
include/hw/i386/topology.h +18 -4
@@ -56,7 +56,8 @@ typedef struct X86CPUTopoIDs {typedef struct X86CPUTopoInfo {unsigned dies_per_pkg; -unsigned cores_per_die; +unsigned modules_per_die; +unsigned cores_per_module;unsigned threads_per_core; }X86CPUTopoInfo ;
@@ -92,7 +93,13 @@ static inline unsigned apicid_smt_width(X86CPUTopoInfo *topo_info)/* Bit width of the Core_ID field */ static inline unsigned apicid_core_width (X86CPUTopoInfo *topo_info) { -return apicid_bitwidth_for_count (topo_info->cores_per_die); +return apicid_bitwidth_for_count (topo_info->cores_per_module); +} + +/* Bit width of the Module_ID field */ +static inline unsigned apicid_module_width (X86CPUTopoInfo *topo_info) +{ +return apicid_bitwidth_for_count (topo_info->modules_per_die); }/* Bit width of the Die_ID field */
@@ -107,10 +114,16 @@ static inline unsigned apicid_core_offset(X86CPUTopoInfo *topo_info)return apicid_smt_width (topo_info); } +/* Bit offset of the Module_ID field */ +static inline unsigned apicid_module_offset (X86CPUTopoInfo *topo_info) +{ +return apicid_core_offset (topo_info) +apicid_core_width (topo_info); +} +/* Bit offset of the Die_ID field */ static inline unsigned apicid_die_offset (X86CPUTopoInfo *topo_info) { -return apicid_core_offset (topo_info) +apicid_core_width (topo_info); +return apicid_module_offset (topo_info) +apicid_module_width (topo_info); }/* Bit offset of the Pkg_ID (socket ID) field */
@@ -142,7 +155,8 @@ static inline void x86_topo_ids_from_idx(X86CPUTopoInfo *topo_info, X86CPUTopoIDs *topo_ids) {unsigned nr_dies = topo_info->dies_per_pkg; -unsigned nr_cores = topo_info->cores_per_die; +unsigned nr_cores = topo_info->cores_per_module * + topo_info->modules_per_die;unsigned nr_threads = topo_info->threads_per_core; topo_ids->pkg_id = cpu_index / (nr_dies * nr_cores * nr_threads);
target/i386/cpu.c +8 -5
@@ -278,10 +278,11 @@ static uint32_t num_threads_by_topo_level(X86CPUTopoInfo *topo_info,case CPU_TOPO_LEVEL_CORE :return topo_info->threads_per_core;case CPU_TOPO_LEVEL_DIE : -return topo_info->threads_per_core * topo_info->cores_per_die; +return topo_info->threads_per_core * topo_info->cores_per_module * + topo_info->modules_per_die;case CPU_TOPO_LEVEL_PACKAGE : -return topo_info->threads_per_core * topo_info->cores_per_die * - topo_info->dies_per_pkg; +return topo_info->threads_per_core * topo_info->cores_per_module * + topo_info->modules_per_die * topo_info->dies_per_pkg;default :g_assert_not_reached (); }
@@ -6133,10 +6134,12 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,uint32_t threads_per_pkg; topo_info.dies_per_pkg = env->nr_dies; - topo_info.cores_per_die = cs->nr_cores / env->nr_dies; + topo_info.modules_per_die = env->nr_modules; + topo_info.cores_per_module = cs->nr_cores / env->nr_dies / env->nr_modules; topo_info.threads_per_core = cs->nr_threads; - cores_per_pkg = topo_info.cores_per_die * topo_info.dies_per_pkg; + cores_per_pkg = topo_info.cores_per_module * topo_info.modules_per_die * + topo_info.dies_per_pkg; threads_per_pkg = cores_per_pkg * topo_info.threads_per_core;/* Calculate & apply limits for different index ranges */
tests/unit/test-x86-topo.c +24 -21
94 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v9 14/21 i386: Expose module level in CPUID[0x1F]
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
Linux kernel (from v6.4, with commit edc0a2b595765 ("x86/topology: Fix erroneous smp_num_siblings on Intel Hybrid platforms") is able to handle platforms with Module level enumerated via CPUID.1F.
Expose the module level in CPUID[0x1F] if the machine has more than 1 modules.
---
Changes since v7:
* Mapped x86 module to smp module instead of cluster.
* Dropped Michael/Babu's ACKed/Tested tags since the code change.
* Re-added Yongwei's Tested tag For his re-testing.
Changes since v3:
* New patch to expose module level in 0x1F.
* Added Tested-by tag from Yongwei.
--- hw/i386/x86.c | 2 +- include/hw/i386/topology.h | 6 ++++-- target/i386/cpu.c | 6 ++++++ target/i386/cpu.h | 1 + 4 files changed, 12 insertions(+), 3 deletions(-)
hw/i386/x86.c +1 -1
@@ -322,7 +322,7 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,if (ms->smp.modules >1 ) { env->nr_modules = ms->smp.modules; -/* TODO: Expose module level in CPUID[0x1F]. */ +set_bit (CPU_TOPO_LEVEL_MODULE , env->avail_cpu_topo); }if (ms->smp.dies >1 ) {
include/hw/i386/topology.h +4 -2
@@ -71,6 +71,7 @@ enum CPUTopoLevel {CPU_TOPO_LEVEL_INVALID ,CPU_TOPO_LEVEL_SMT ,CPU_TOPO_LEVEL_CORE , +CPU_TOPO_LEVEL_MODULE ,CPU_TOPO_LEVEL_DIE ,CPU_TOPO_LEVEL_PACKAGE , CPU_TOPO_LEVEL_MAX,
@@ -198,11 +199,12 @@ static inline apic_id_t x86_apicid_from_cpu_idx(X86CPUTopoInfo *topo_info, }/* -* Check whether there's extended topology level (die)? +* Check whether there's extended topology level (module or die)? */ static inline bool x86_has_extended_topo (unsigned long *topo_bitmap) { -return test_bit (CPU_TOPO_LEVEL_DIE , topo_bitmap); +return test_bit (CPU_TOPO_LEVEL_MODULE , topo_bitmap) || +test_bit (CPU_TOPO_LEVEL_DIE , topo_bitmap); }#endif /* HW_I386_TOPOLOGY_H */
target/i386/cpu.c +6 -0
@@ -277,6 +277,8 @@ static uint32_t num_threads_by_topo_level(X86CPUTopoInfo *topo_info,return 1 ;case CPU_TOPO_LEVEL_CORE :return topo_info->threads_per_core; +case CPU_TOPO_LEVEL_MODULE : +return topo_info->threads_per_core * topo_info->cores_per_module;case CPU_TOPO_LEVEL_DIE :return topo_info->threads_per_core * topo_info->cores_per_module * topo_info->modules_per_die;
@@ -297,6 +299,8 @@ static uint32_t apicid_offset_by_topo_level(X86CPUTopoInfo *topo_info,return 0 ;case CPU_TOPO_LEVEL_CORE :return apicid_core_offset (topo_info); +case CPU_TOPO_LEVEL_MODULE : +return apicid_module_offset (topo_info);case CPU_TOPO_LEVEL_DIE :return apicid_die_offset (topo_info);case CPU_TOPO_LEVEL_PACKAGE :
@@ -316,6 +320,8 @@ static uint32_t cpuid1f_topo_type(enum CPUTopoLevel topo_level)return CPUID_1F_ECX_TOPO_LEVEL_SMT ;case CPU_TOPO_LEVEL_CORE :return CPUID_1F_ECX_TOPO_LEVEL_CORE ; +case CPU_TOPO_LEVEL_MODULE : +return CPUID_1F_ECX_TOPO_LEVEL_MODULE ;case CPU_TOPO_LEVEL_DIE :return CPUID_1F_ECX_TOPO_LEVEL_DIE ;default :
target/i386/cpu.h +1 -0
@@ -1025,6 +1025,7 @@ uint64_t x86_cpu_get_supported_feature_word(FeatureWord w,#define CPUID_1F_ECX_TOPO_LEVEL_INVALID CPUID_B_ECX_TOPO_LEVEL_INVALID#define CPUID_1F_ECX_TOPO_LEVEL_SMT CPUID_B_ECX_TOPO_LEVEL_SMT#define CPUID_1F_ECX_TOPO_LEVEL_CORE CPUID_B_ECX_TOPO_LEVEL_CORE +#define CPUID_1F_ECX_TOPO_LEVEL_MODULE 3#define CPUID_1F_ECX_TOPO_LEVEL_DIE 5/* MSR Feature Bits */
Signature · 1 line
2.34.1
PATCH v9 15/21 i386: Support module_id in X86CPUTopoIDs
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
Add module_id member in X86CPUTopoIDs.
module_id can be parsed from APIC ID, so also update APIC ID parsing rule to support module level. With this support, the conversions with module level between X86CPUTopoIDs, X86CPUTopoInfo and APIC ID are completed.
module_id can be also generated from cpu topology, and before i386 supports "modules" in smp, the default "modules per die" (modules * clusters) is only 1, thus the module_id generated in this way is 0, so that it will not conflict with the module_id generated by APIC ID.
---
Changes since v7:
* Mapped x86 module to the smp module instead of cluster.
* Dropped Michael/Babu's ACKed/Tested tags since the code change.
* Re-added Yongwei's Tested tag For his re-testing.
Changes since v1:
* Merged the patch "i386: Update APIC ID parsing rule to support module level" into this one. (Yanan)
* Moved the apicid_module_width() and apicid_module_offset() support into the previous modules_per_die related patch. (Yanan)
--- hw/i386/x86.c | 31 +++++++++++++++++++++---------- include/hw/i386/topology.h | 17 +++++++++++++---- 2 files changed, 34 insertions(+), 14 deletions(-)
hw/i386/x86.c +21 -10
@@ -332,12 +332,9 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,/* * If APIC ID is not set, -* set it based on socket/die/core/thread properties. +* set it based on socket/die/module/core/thread properties. */ if (cpu->apic_id ==UNASSIGNED_APIC_ID ) { -int max_socket = (ms->smp.max_cpus -1 ) / - smp_threads / smp_cores / ms->smp.dies; -/* * die-id was optional in QEMU 4.0 and older, so keep it optional * if there's only one die per socket.
@@ -349,9 +346,9 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev,if (cpu->socket_id <0 ) {error_setg (errp,"CPU socket-id is not set" );return ; - }else if (cpu->socket_id > max_socket) { + }else if (cpu->socket_id > ms->smp.sockets -1 ) {error_setg (errp,"Invalid CPU socket-id: %u must be in range 0:%u" , - cpu->socket_id, max_socket); + cpu->socket_id, ms->smp.sockets -1 );return ; }if (cpu->die_id <0 ) {
@@ -383,17 +380,27 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev, topo_ids.die_id = cpu->die_id; topo_ids.core_id = cpu->core_id; topo_ids.smt_id = cpu->thread_id; + +/* +* TODO: This is the temporary initialization for topo_ids.module_id to +* avoid "maybe-uninitialized" compilation errors. Will remove when +* X86CPU supports module_id. +*/ + topo_ids.module_id =0 ; + cpu->apic_id =x86_apicid_from_topo_ids (&topo_info, &topo_ids); } cpu_slot =x86_find_cpu_slot (MACHINE (x86ms), cpu->apic_id, &idx);if (!cpu_slot) {x86_topo_ids_from_apicid (cpu->apic_id, &topo_info, &topo_ids); +error_setg (errp, -"Invalid CPU [socket: %u, die: %u, core: %u, thread: %u] with" -" APIC ID %" PRIu32", valid index range 0:%d" , - topo_ids.pkg_id, topo_ids.die_id, topo_ids.core_id, topo_ids.smt_id, - cpu->apic_id, ms->possible_cpus->len -1 ); +"Invalid CPU [socket: %u, die: %u, module: %u, core: %u, thread: %u]" +" with APIC ID %" PRIu32", valid index range 0:%d" , + topo_ids.pkg_id, topo_ids.die_id, topo_ids.module_id, + topo_ids.core_id, topo_ids.smt_id, cpu->apic_id, + ms->possible_cpus->len -1 );return ; }
@@ -519,6 +526,10 @@ const CPUArchIdList *x86_possible_cpu_arch_ids(MachineState *ms) ms->possible_cpus->cpus[i].props.has_die_id = true; ms->possible_cpus->cpus[i].props.die_id = topo_ids.die_id; } +if (ms->smp.modules >1 ) { + ms->possible_cpus->cpus[i].props.has_module_id = true; + ms->possible_cpus->cpus[i].props.module_id = topo_ids.module_id; + } ms->possible_cpus->cpus[i].props.has_core_id = true; ms->possible_cpus->cpus[i].props.core_id = topo_ids.core_id; ms->possible_cpus->cpus[i].props.has_thread_id = true;
include/hw/i386/topology.h +13 -4
@@ -50,6 +50,7 @@ typedef uint32_t apic_id_t;typedef struct X86CPUTopoIDs {unsigned pkg_id;unsigned die_id; +unsigned module_id;unsigned core_id;unsigned smt_id; }X86CPUTopoIDs ;
@@ -143,6 +144,7 @@ static inline apic_id_t x86_apicid_from_topo_ids(X86CPUTopoInfo *topo_info, {return (topo_ids->pkg_id <<apicid_pkg_offset (topo_info)) | (topo_ids->die_id <<apicid_die_offset (topo_info)) | + (topo_ids->module_id <<apicid_module_offset (topo_info)) | (topo_ids->core_id <<apicid_core_offset (topo_info)) | topo_ids->smt_id; }
@@ -156,12 +158,16 @@ static inline void x86_topo_ids_from_idx(X86CPUTopoInfo *topo_info, X86CPUTopoIDs *topo_ids) {unsigned nr_dies = topo_info->dies_per_pkg; -unsigned nr_cores = topo_info->cores_per_module * - topo_info->modules_per_die; +unsigned nr_modules = topo_info->modules_per_die; +unsigned nr_cores = topo_info->cores_per_module;unsigned nr_threads = topo_info->threads_per_core; - topo_ids->pkg_id = cpu_index / (nr_dies * nr_cores * nr_threads); - topo_ids->die_id = cpu_index / (nr_cores * nr_threads) % nr_dies; + topo_ids->pkg_id = cpu_index / (nr_dies * nr_modules * + nr_cores * nr_threads); + topo_ids->die_id = cpu_index / (nr_modules * nr_cores * + nr_threads) % nr_dies; + topo_ids->module_id = cpu_index / (nr_cores * nr_threads) % + nr_modules; topo_ids->core_id = cpu_index / nr_threads % nr_cores; topo_ids->smt_id = cpu_index % nr_threads; }
@@ -179,6 +185,9 @@ static inline void x86_topo_ids_from_apicid(apic_id_t apicid, topo_ids->core_id = (apicid >>apicid_core_offset (topo_info)) & ~(0xFFFFFFFFUL <<apicid_core_width (topo_info)); + topo_ids->module_id = + (apicid >>apicid_module_offset (topo_info)) & + ~(0xFFFFFFFFUL <<apicid_module_width (topo_info)); topo_ids->die_id = (apicid >>apicid_die_offset (topo_info)) & ~(0xFFFFFFFFUL <<apicid_die_width (topo_info));
Signature · 1 line
2.34.1
PATCH v9 16/21 i386/cpu: Introduce module-id to X86CPU
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
Introduce module-id to be consistent with the module-id field in CpuInstanceProperties.
Following the legacy smp check rules, also add the module_id validity into x86_cpu_pre_plug().
---
Changes since v7:
* Introduced module_id instead of cluster_id.
* Dropped Michael/Babu's ACKed/Tested tags since the code change.
* Re-added Yongwei's Tested tag For his re-testing.
Changes since v6:
* Updated the comment when check cluster-id. Since there's no v8.2, the cluster-id support should at least start from v9.0.
Changes since v5:
* Updated the comment when check cluster-id. Since current QEMU is v8.2, the cluster-id support should at least start from v8.3.
Changes since v3:
* Used the imperative in the commit message. (Babu)
--- hw/i386/x86.c | 33 +++++++++++++++++++++++++-------- target/i386/cpu.c | 2 ++ target/i386/cpu.h | 1 + 3 files changed, 28 insertions(+), 8 deletions(-)
hw/i386/x86.c +25 -8
@@ -343,6 +343,14 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev, cpu->die_id =0 ; } +/* +* module-id was optional in QEMU 9.0 and older, so keep it optional +* if there's only one module per die. +*/ +if (cpu->module_id <0 && ms->smp.modules ==1 ) { + cpu->module_id =0 ; + } +if (cpu->socket_id <0 ) {error_setg (errp,"CPU socket-id is not set" );return ;
@@ -359,6 +367,14 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev, cpu->die_id, ms->smp.dies - 1);return ; } +if (cpu->module_id <0 ) { +error_setg (errp,"CPU module-id is not set" ); +return ; + }else if (cpu->module_id > ms->smp.modules -1 ) { +error_setg (errp,"Invalid CPU module-id: %u must be in range 0:%u" , + cpu->module_id, ms->smp.modules -1 ); +return ; + }if (cpu->core_id <0 ) {error_setg (errp,"CPU core-id is not set" );return ;
@@ -378,16 +394,9 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev, topo_ids.pkg_id = cpu->socket_id; topo_ids.die_id = cpu->die_id; + topo_ids.module_id = cpu->module_id; topo_ids.core_id = cpu->core_id; topo_ids.smt_id = cpu->thread_id; - -/* -* TODO: This is the temporary initialization for topo_ids.module_id to -* avoid "maybe-uninitialized" compilation errors. Will remove when -* X86CPU supports module_id. -*/ - topo_ids.module_id =0 ; - cpu->apic_id =x86_apicid_from_topo_ids (&topo_info, &topo_ids); }
@@ -432,6 +441,14 @@ void x86_cpu_pre_plug(HotplugHandler *hotplug_dev, } cpu->die_id = topo_ids.die_id; +if (cpu->module_id !=-1 && cpu->module_id != topo_ids.module_id) { +error_setg (errp,"property module-id: %u doesn't match set apic-id:" +" 0x%x (module-id: %u)" , cpu->module_id, cpu->apic_id, + topo_ids.module_id); +return ; + } + cpu->module_id = topo_ids.module_id; +if (cpu->core_id !=-1 && cpu->core_id != topo_ids.core_id) { error_setg(errp, "property core-id: %u doesn't match set apic-id:" " 0x%x (core-id: %u)", cpu->core_id, cpu->apic_id,
target/i386/cpu.c +2 -0
@@ -7940,12 +7940,14 @@ static Property x86_cpu_properties[] = {DEFINE_PROP_UINT32 ("apic-id" ,X86CPU , apic_id,0 ),DEFINE_PROP_INT32 ("thread-id" ,X86CPU , thread_id,0 ),DEFINE_PROP_INT32 ("core-id" ,X86CPU , core_id,0 ), +DEFINE_PROP_INT32 ("module-id" ,X86CPU , module_id,0 ),DEFINE_PROP_INT32 ("die-id" ,X86CPU , die_id,0 ),DEFINE_PROP_INT32 ("socket-id" ,X86CPU , socket_id,0 ), #elseDEFINE_PROP_UINT32 ("apic-id" ,X86CPU , apic_id,UNASSIGNED_APIC_ID ),DEFINE_PROP_INT32 ("thread-id" ,X86CPU , thread_id,-1 ),DEFINE_PROP_INT32 ("core-id" ,X86CPU , core_id,-1 ), +DEFINE_PROP_INT32 ("module-id" ,X86CPU , module_id,-1 ),DEFINE_PROP_INT32 ("die-id" ,X86CPU , die_id,-1 ), DEFINE_PROP_INT32("socket-id", X86CPU, socket_id, -1), #endif
target/i386/cpu.h +1 -0
@@ -2051,6 +2051,7 @@ struct ArchCPU {int32_t node_id;/* NUMA node this CPU belongs to */ int32_t socket_id;int32_t die_id; +int32_t module_id;int32_t core_id;int32_t thread_id;
Signature · 1 line
2.34.1
PATCH v9 17/21 tests: Add test case of APIC ID for module level parsing
In reply to Zhao LiuFrom: Zhuocheng Ding <[email protected]>
After i386 supports module level, it's time to add the test for module level's parsing.
--- tests/unit/test-x86-topo.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)
tests/unit/test-x86-topo.c +15 -4
@@ -37,6 +37,7 @@ static void test_topo_bits(void) topo_info = (X86CPUTopoInfo ) {1 ,1 ,1 ,1 }; g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 0); g_assert_cmpuint(apicid_core_width(&topo_info), ==, 0); + g_assert_cmpuint(apicid_module_width(&topo_info), ==, 0); g_assert_cmpuint(apicid_die_width(&topo_info), ==, 0); topo_info = (X86CPUTopoInfo ) {1 ,1 ,1 ,1 };
@@ -74,13 +75,22 @@ static void test_topo_bits(void) topo_info = (X86CPUTopoInfo ) {1 ,1 ,33 ,2 }; g_assert_cmpuint(apicid_core_width(&topo_info), ==, 6); - topo_info = (X86CPUTopoInfo ) {1 ,1 ,30 ,2 }; + topo_info = (X86CPUTopoInfo ) {1 ,6 ,30 ,2 }; + g_assert_cmpuint(apicid_module_width(&topo_info), ==, 3); + topo_info = (X86CPUTopoInfo ) {1 ,7 ,30 ,2 }; + g_assert_cmpuint(apicid_module_width(&topo_info), ==, 3); + topo_info = (X86CPUTopoInfo ) {1 ,8 ,30 ,2 }; + g_assert_cmpuint(apicid_module_width(&topo_info), ==, 3); + topo_info = (X86CPUTopoInfo ) {1 ,9 ,30 ,2 }; + g_assert_cmpuint(apicid_module_width(&topo_info), ==, 4); + + topo_info = (X86CPUTopoInfo ) {1 ,6 ,30 ,2 }; g_assert_cmpuint(apicid_die_width(&topo_info), ==, 0); - topo_info = (X86CPUTopoInfo ) {2 ,1 ,30 ,2 }; + topo_info = (X86CPUTopoInfo ) {2 ,6 ,30 ,2 }; g_assert_cmpuint(apicid_die_width(&topo_info), ==, 1); - topo_info = (X86CPUTopoInfo ) {3 ,1 ,30 ,2 }; + topo_info = (X86CPUTopoInfo ) {3 ,6 ,30 ,2 }; g_assert_cmpuint(apicid_die_width(&topo_info), ==, 2); - topo_info = (X86CPUTopoInfo ) {4 ,1 ,30 ,2 }; + topo_info = (X86CPUTopoInfo ) {4 ,6 ,30 ,2 }; g_assert_cmpuint(apicid_die_width(&topo_info), ==, 2);/* build a weird topology and see if IDs are calculated correctly
@@ -91,6 +101,7 @@ static void test_topo_bits(void) topo_info = (X86CPUTopoInfo ) {1 ,1 ,6 ,3 }; g_assert_cmpuint(apicid_smt_width(&topo_info), ==, 2); g_assert_cmpuint(apicid_core_offset(&topo_info), ==, 2); + g_assert_cmpuint(apicid_module_offset(&topo_info), ==, 5); g_assert_cmpuint(apicid_die_offset(&topo_info), ==, 5); g_assert_cmpuint(apicid_pkg_offset(&topo_info), ==, 5);
Signature · 1 line
2.34.1
PATCH v9 18/21 hw/i386/pc: Support smp.modules for x86 PC machine
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
As module-level topology support is added to X86CPU, now we can enable the support for the modules parameter on PC machines. With this support, we can define a 5-level x86 CPU topology with "-smp":
-smp cpus=*,maxcpus=*,sockets=*,dies=*,modules=*,cores=*,threads=*.
Additionally, add the 5-level topology example in description of "-smp".
---
Changes since v8:
* Add missing "modules" parameter in -smp example.
Changes since v7:
* Supported modules instead of clusters for PC.
* Dropped Michael/Babu/Yanan's ACKed/Tested/Reviewed tags since the code change.
* Re-added Yongwei's Tested tag For his re-testing.
--- hw/i386/pc.c | 1 + qemu-options.hx | 18 ++++++++++-------- 2 files changed, 11 insertions(+), 8 deletions(-)
hw/i386/pc.c +1 -0
@@ -1830,6 +1830,7 @@ static void pc_machine_class_init(ObjectClass *oc, void *data) mc->default_cpu_type =TARGET_DEFAULT_CPU_TYPE ; mc->nvdimm_supported = true; mc->smp_props.dies_supported = true; + mc->smp_props.modules_supported = true; mc->default_ram_id ="pc.ram" ; pcmc->default_smbios_ep_type =SMBIOS_ENTRY_POINT_TYPE_64 ;
qemu-options.hx +10 -8
@@ -281,7 +281,8 @@ ERST DEF("smp", HAS_ARG, QEMU_OPTION_smp, "-smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books][,sockets=sockets]\n" - " [,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n" + " [,dies=dies][,clusters=clusters][,modules=modules][,cores=cores]\n" + " [,threads=threads]\n" " set the number of initial CPUs to 'n' [default=1]\n" " maxcpus= maximum number of total CPUs, including\n" " offline CPUs for hotplug, etc\n"
@@ -290,7 +291,8 @@ DEF("smp", HAS_ARG, QEMU_OPTION_smp, " sockets= number of sockets in one book\n" " dies= number of dies in one socket\n" " clusters= number of clusters in one die\n" - " cores= number of cores in one cluster\n" + " modules= number of modules in one cluster\n" + " cores= number of cores in one module\n" " threads= number of threads in one core\n" "Note: Different machines may have different subsets of the CPU topology\n" " parameters supported, so the actual meaning of the supported parameters\n"
@@ -306,7 +308,7 @@ DEF("smp", HAS_ARG, QEMU_OPTION_smp, " must be set as 1 in the purpose of correct parsing.\n", QEMU_ARCH_ALL) SRST -``-smp [[cpus=]n][,maxcpus=maxcpus][,sockets=sockets][,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]`` +``-smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books][,sockets=sockets][,dies=dies][,clusters=clusters][,modules=modules][,cores=cores][,threads=threads]`` Simulate a SMP system with '\ ``n``\ ' CPUs initially present on the machine type board. On boards supporting CPU hotplug, the optional '\ ``maxcpus``\ ' parameter can be set to enable further CPUs to be
@@ -345,14 +347,14 @@ SRST -smp 8,sockets=2,cores=2,threads=2,maxcpus=8 The following sub-option defines a CPU topology hierarchy (2 sockets - totally on the machine, 2 dies per socket, 2 cores per die, 2 threads - per core) for PC machines which support sockets/dies/cores/threads. - Some members of the option can be omitted but their values will be - automatically computed: + totally on the machine, 2 dies per socket, 2 modules per die, 2 cores per + module, 2 threads per core) for PC machines which support sockets/dies + /modules/cores/threads. Some members of the option can be omitted but + their values will be automatically computed: :: - -smp 16,sockets=2,dies=2,cores=2,threads=2,maxcpus=16 + -smp 32,sockets=2,dies=2,modules=2,cores=2,threads=2,maxcpus=32 The following sub-option defines a CPU topology hierarchy (2 sockets totally on the machine, 2 clusters per socket, 2 cores per cluster,
Signature · 1 line
2.34.1
Re: [PATCH v9 18/21] hw/i386/pc: Support smp.modules for x86 PC machine
Hi Zhao,
Quoted 65 lines · Zhao Liu
From: Zhao Liu <[email protected]>
As module-level topology support is added to X86CPU, now we can enable the support for the modules parameter on PC machines. With this support, we can define a 5-level x86 CPU topology with "-smp":
-smp cpus=*,maxcpus=*,sockets=*,dies=*,modules=*,cores=*,threads=*.
Additionally, add the 5-level topology example in description of "-smp".
Tested-by Yongwei Ma <[email protected]>Co-developed-by Zhuocheng Ding <[email protected]>Signed-off-by Zhuocheng Ding <[email protected]>Signed-off-by Zhao Liu <[email protected]>---
Changes since v8:
* Add missing "modules" parameter in -smp example.Changes since v7:
* Supported modules instead of clusters for PC.
* Dropped Michael/Babu/Yanan's ACKed/Tested/Reviewed tags since the code change.
* Re-added Yongwei's Tested tag For his re-testing.--- hw/i386/pc.c | 1 + qemu-options.hx | 18 ++++++++++-------- 2 files changed, 11 insertions(+), 8 deletions(-)hw/i386/pc.c +1
-0@@ -1830,6 +1830,7 @@ static void pc_machine_class_init(ObjectClass *oc, void *data) mc->default_cpu_type =TARGET_DEFAULT_CPU_TYPE ; mc->nvdimm_supported = true; mc->smp_props.dies_supported = true; + mc->smp_props.modules_supported = true; mc->default_ram_id ="pc.ram" ; pcmc->default_smbios_ep_type =SMBIOS_ENTRY_POINT_TYPE_64 ;qemu-options.hx +4
-2@@ -281,7 +281,8 @@ ERST DEF("smp", HAS_ARG, QEMU_OPTION_smp, "-smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books][,sockets=sockets]\n" - " [,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n" + " [,dies=dies][,clusters=clusters][,modules=modules][,cores=cores]\n" + " [,threads=threads]\n" " set the number of initial CPUs to 'n' [default=1]\n" " maxcpus= maximum number of total CPUs, including\n" " offline CPUs for hotplug, etc\n"@@ -290,7 +291,8 @@ DEF("smp", HAS_ARG, QEMU_OPTION_smp, " sockets= number of sockets in one book\n" " dies= number of dies in one socket\n" " clusters= number of clusters in one die\n" - " cores= number of cores in one cluster\n" + " modules= number of modules in one cluster\n" + " cores= number of cores in one module\n" " threads= number of threads in one core\n" "Note: Different machines may have different subsets of the CPU topology\n" " parameters supported, so the actual meaning of the supported parameters\n"
qemu-options.hx DEF("smp", HAS_ARG, QEMU_OPTION_smp, · line 311
Quoted 5 lines
qemu-options.hx +1
-1@@ -306,7 +308,7 @@ DEF("smp", HAS_ARG, QEMU_OPTION_smp, " must be set as 1 in the purpose of correct parsing.\n", QEMU_ARCH_ALL) SRST -``-smp [[cpus=]n][,maxcpus=maxcpus][,sockets=sockets][,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]`` +``-smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books][,sockets=sockets][,dies=dies][,clusters=clusters][,modules=modules][,cores=cores][,threads=threads]``
You have added drawers, books here. Were they missing before?
Quoted 23 lines
qemu-options.hx +5
-5Simulate a SMP system with '\ ``n``\ ' CPUs initially present on the machine type board. On boards supporting CPU hotplug, the optional '\ ``maxcpus``\ ' parameter can be set to enable further CPUs to be@@ -345,14 +347,14 @@ SRST -smp 8,sockets=2,cores=2,threads=2,maxcpus=8 The following sub-option defines a CPU topology hierarchy (2 sockets - totally on the machine, 2 dies per socket, 2 cores per die, 2 threads - per core) for PC machines which support sockets/dies/cores/threads. - Some members of the option can be omitted but their values will be - automatically computed: + totally on the machine, 2 dies per socket, 2 modules per die, 2 cores per + module, 2 threads per core) for PC machines which support sockets/dies + /modules/cores/threads. Some members of the option can be omitted but + their values will be automatically computed: :: - -smp 16,sockets=2,dies=2,cores=2,threads=2,maxcpus=16 + -smp 32,sockets=2,dies=2,modules=2,cores=2,threads=2,maxcpus=32 The following sub-option defines a CPU topology hierarchy (2 sockets totally on the machine, 2 clusters per socket, 2 cores per cluster,
Signature · 2 lines
Thanks Babu Moger
Re: [PATCH v9 18/21] hw/i386/pc: Support smp.modules for x86 PC machine
Hi Babu,
Quoted 2 lines
Quoted 2 lines
DEF("smp",
HAS_ARG,QEMU_OPTION_smp,"-smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books][,sockets=sockets]\n"
Here the "drawers" and "books" are listed...
Quoted 16 lines
Quoted 16 lines
qemu-options.hx +4
-2- " [,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n" + " [,dies=dies][,clusters=clusters][,modules=modules][,cores=cores]\n" + " [,threads=threads]\n" " set the number of initial CPUs to 'n' [default=1]\n" " maxcpus= maximum number of total CPUs, including\n" " offline CPUs for hotplug, etc\n"@@ -290,7 +291,8 @@ DEF("smp", HAS_ARG, QEMU_OPTION_smp, " sockets= number of sockets in one book\n" " dies= number of dies in one socket\n" " clusters= number of clusters in one die\n" - " cores= number of cores in one cluster\n" + " modules= number of modules in one cluster\n" + " cores= number of cores in one module\n" " threads= number of threads in one core\n" "Note: Different machines may have different subsets of the CPU topology\n" " parameters supported, so the actual meaning of the supported parameters\n"
Quoted 9 lines
qemu-options.hx DEF("smp", HAS_ARG, QEMU_OPTION_smp, · line 311
Quoted 6 lines
qemu-options.hx +1
-1@@ -306,7 +308,7 @@ DEF("smp", HAS_ARG, QEMU_OPTION_smp, " must be set as 1 in the purpose of correct parsing.\n", QEMU_ARCH_ALL) SRST -``-smp [[cpus=]n][,maxcpus=maxcpus][,sockets=sockets][,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]`` +``-smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books][,sockets=sockets][,dies=dies][,clusters=clusters][,modules=modules][,cores=cores][,threads=threads]``You have added drawers, books here. Were they missing before?
...so yes, I think those 2 parameters are missed at this place.
Thank you for reviewing this.
Regards,
Zhao
Moger, Babu
Reviewed-byRe: [PATCH v9 18/21] hw/i386/pc: Support smp.modules for x86 PC machine
Quoted 33 lines · Zhao Liu
Hi Babu,
Quoted 2 lines
Quoted 2 lines
DEF("smp",
HAS_ARG,QEMU_OPTION_smp,"-smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books][,sockets=sockets]\n"Here the "drawers" and "books" are listed...
Quoted 16 lines
Quoted 16 lines
qemu-options.hx +4
-2- " [,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n" + " [,dies=dies][,clusters=clusters][,modules=modules][,cores=cores]\n" + " [,threads=threads]\n" " set the number of initial CPUs to 'n' [default=1]\n" " maxcpus= maximum number of total CPUs, including\n" " offline CPUs for hotplug, etc\n"@@ -290,7 +291,8 @@ DEF("smp", HAS_ARG, QEMU_OPTION_smp, " sockets= number of sockets in one book\n" " dies= number of dies in one socket\n" " clusters= number of clusters in one die\n" - " cores= number of cores in one cluster\n" + " modules= number of modules in one cluster\n" + " cores= number of cores in one module\n" " threads= number of threads in one core\n" "Note: Different machines may have different subsets of the CPU topology\n" " parameters supported, so the actual meaning of the supported parameters\n"Quoted 9 lines
qemu-options.hx DEF("smp", HAS_ARG, QEMU_OPTION_smp, · line 311
Quoted 6 lines
qemu-options.hx +1
-1@@ -306,7 +308,7 @@ DEF("smp", HAS_ARG, QEMU_OPTION_smp, " must be set as 1 in the purpose of correct parsing.\n", QEMU_ARCH_ALL) SRST -``-smp [[cpus=]n][,maxcpus=maxcpus][,sockets=sockets][,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]`` +``-smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books][,sockets=sockets][,dies=dies][,clusters=clusters][,modules=modules][,cores=cores][,threads=threads]``You have added drawers, books here. Were they missing before?
Quoted 1 line
...so yes, I think those 2 parameters are missed at this place.
ok. If there is another revision then add a line about this change in the commit message. Otherwise it is fine.
Quoted 6 lines
Thank you for reviewing this.
Regards,
Zhao
Signature · 2 lines
Thanks Babu Moger
Re: [PATCH v9 18/21] hw/i386/pc: Support smp.modules for x86 PC machine
Quoted 10 lines
Quoted 4 lines
Quoted 2 lines
You have added drawers, books here. Were they missing before?
...so yes, I think those 2 parameters are missed at this place.
ok. If there is another revision then add a line about this change in the commit message. Otherwise it is fine.
Reviewed-by Babu Moger <[email protected]>
Sure! Thank you.
-Zhao
PATCH v9 19/21 i386: Add cache topology info in CPUCacheInfo
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
Currently, by default, the cache topology is encoded as:
1. i/d cache is shared in one core.
2. L2 cache is shared in one core.
3. L3 cache is shared in one die.
This default general setting has caused a misunderstanding, that is, the cache topology is completely equated with a specific cpu topology, such as the connection between L2 cache and core level, and the connection between L3 cache and die level.
In fact, the settings of these topologies depend on the specific platform and are not static. For example, on Alder Lake-P, every four Atom cores share the same L2 cache.
Thus, we should explicitly define the corresponding cache topology for different cache models to increase scalability.
Except legacy_l2_cache_cpuid2 (its default topo level is CPU_TOPO_LEVEL_UNKNOW), explicitly set the corresponding topology level for all other cache models. In order to be compatible with the existing cache topology, set the CPU_TOPO_LEVEL_CORE level for the i/d cache, set the CPU_TOPO_LEVEL_CORE level for L2 cache, and set the CPU_TOPO_LEVEL_DIE level for L3 cache.
The field for CPUID[4].EAX[bits 25:14] or CPUID[0x8000001D].EAX[bits 25:14] will be set based on CPUCacheInfo.share_level.
---
Changes since v3:
* Fixed cache topology uninitialization bugs for some AMD CPUs. (Babu)
* Moved the CPUTopoLevel enumeration definition to the previous 0x1f rework patch.
Changes since v1:
* Added the prefix "CPU_TOPO_LEVEL_*" for CPU topology level names. (Yanan)
* (Revert, pls refer "i386: Decouple CPUID[0x1F] subleaf with specific topology level") Renamed the "INVALID" level to CPU_TOPO_LEVEL_UNKNOW. (Yanan)
--- target/i386/cpu.c | 36 ++++++++++++++++++++++++++++++++++++ target/i386/cpu.h | 7 +++++++ 2 files changed, 43 insertions(+)
target/i386/cpu.c +36 -0
@@ -554,6 +554,7 @@ static CPUCacheInfo legacy_l1d_cache = { .sets = 64, .partitions = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, };/*FIXME: CPUID leaf 0x80000005 is inconsistent with leaves 2 & 4 */
@@ -568,6 +569,7 @@ static CPUCacheInfo legacy_l1d_cache_amd = { .partitions = 1, .lines_per_tag = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, };/* L1 instruction cache: */
@@ -581,6 +583,7 @@ static CPUCacheInfo legacy_l1i_cache = { .sets = 64, .partitions = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, };/*FIXME: CPUID leaf 0x80000005 is inconsistent with leaves 2 & 4 */
@@ -595,6 +598,7 @@ static CPUCacheInfo legacy_l1i_cache_amd = { .partitions = 1, .lines_per_tag = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, };/* Level 2 unified cache: */
@@ -608,6 +612,7 @@ static CPUCacheInfo legacy_l2_cache = { .sets = 4096, .partitions = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, };/*FIXME: CPUID leaf 2 descriptor is inconsistent with CPUID leaf 4 */
@@ -617,6 +622,7 @@ static CPUCacheInfo legacy_l2_cache_cpuid2 = { .size = 2 * MiB, .line_size = 64, .associativity = 8, + .share_level = CPU_TOPO_LEVEL_INVALID, };
@@ -630,6 +636,7 @@ static CPUCacheInfo legacy_l2_cache_amd = { .associativity = 16, .sets = 512, .partitions = 1, + .share_level = CPU_TOPO_LEVEL_CORE, };/* Level 3 unified cache: */
@@ -645,6 +652,7 @@ static CPUCacheInfo legacy_l3_cache = { .self_init = true, .inclusive = true, .complex_indexing = true, + .share_level = CPU_TOPO_LEVEL_DIE, };/* TLB definitions: */
@@ -1943,6 +1951,7 @@ static const CPUCaches epyc_cache_info = { .lines_per_tag = 1, .self_init = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l1i_cache = &(CPUCacheInfo ) { .type =INSTRUCTION_CACHE ,
@@ -1955,6 +1964,7 @@ static const CPUCaches epyc_cache_info = { .lines_per_tag = 1, .self_init = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l2_cache = &(CPUCacheInfo ) { .type =UNIFIED_CACHE ,
@@ -1965,6 +1975,7 @@ static const CPUCaches epyc_cache_info = { .partitions = 1, .sets = 1024, .lines_per_tag = 1, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l3_cache = &(CPUCacheInfo ) { .type =UNIFIED_CACHE ,
@@ -1978,6 +1989,7 @@ static const CPUCaches epyc_cache_info = { .self_init = true, .inclusive = true, .complex_indexing = true, + .share_level = CPU_TOPO_LEVEL_DIE, }, };
@@ -1993,6 +2005,7 @@ static CPUCaches epyc_v4_cache_info = { .lines_per_tag = 1, .self_init = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l1i_cache = &(CPUCacheInfo ) { .type =INSTRUCTION_CACHE ,
@@ -2005,6 +2018,7 @@ static CPUCaches epyc_v4_cache_info = { .lines_per_tag = 1, .self_init = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l2_cache = &(CPUCacheInfo ) { .type =UNIFIED_CACHE ,
@@ -2015,6 +2029,7 @@ static CPUCaches epyc_v4_cache_info = { .partitions = 1, .sets = 1024, .lines_per_tag = 1, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l3_cache = &(CPUCacheInfo ) { .type =UNIFIED_CACHE ,
@@ -2028,6 +2043,7 @@ static CPUCaches epyc_v4_cache_info = { .self_init = true, .inclusive = true, .complex_indexing = false, + .share_level = CPU_TOPO_LEVEL_DIE, }, };
@@ -2043,6 +2059,7 @@ static const CPUCaches epyc_rome_cache_info = { .lines_per_tag = 1, .self_init = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l1i_cache = &(CPUCacheInfo ) { .type =INSTRUCTION_CACHE ,
@@ -2055,6 +2072,7 @@ static const CPUCaches epyc_rome_cache_info = { .lines_per_tag = 1, .self_init = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l2_cache = &(CPUCacheInfo ) { .type =UNIFIED_CACHE ,
@@ -2065,6 +2083,7 @@ static const CPUCaches epyc_rome_cache_info = { .partitions = 1, .sets = 1024, .lines_per_tag = 1, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l3_cache = &(CPUCacheInfo ) { .type =UNIFIED_CACHE ,
@@ -2078,6 +2097,7 @@ static const CPUCaches epyc_rome_cache_info = { .self_init = true, .inclusive = true, .complex_indexing = true, + .share_level = CPU_TOPO_LEVEL_DIE, }, };
@@ -2093,6 +2113,7 @@ static const CPUCaches epyc_rome_v3_cache_info = { .lines_per_tag = 1, .self_init = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l1i_cache = &(CPUCacheInfo ) { .type =INSTRUCTION_CACHE ,
@@ -2105,6 +2126,7 @@ static const CPUCaches epyc_rome_v3_cache_info = { .lines_per_tag = 1, .self_init = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l2_cache = &(CPUCacheInfo ) { .type =UNIFIED_CACHE ,
@@ -2115,6 +2137,7 @@ static const CPUCaches epyc_rome_v3_cache_info = { .partitions = 1, .sets = 1024, .lines_per_tag = 1, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l3_cache = &(CPUCacheInfo ) { .type =UNIFIED_CACHE ,
@@ -2128,6 +2151,7 @@ static const CPUCaches epyc_rome_v3_cache_info = { .self_init = true, .inclusive = true, .complex_indexing = false, + .share_level = CPU_TOPO_LEVEL_DIE, }, };
@@ -2143,6 +2167,7 @@ static const CPUCaches epyc_milan_cache_info = { .lines_per_tag = 1, .self_init = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l1i_cache = &(CPUCacheInfo ) { .type =INSTRUCTION_CACHE ,
@@ -2155,6 +2180,7 @@ static const CPUCaches epyc_milan_cache_info = { .lines_per_tag = 1, .self_init = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l2_cache = &(CPUCacheInfo ) { .type =UNIFIED_CACHE ,
@@ -2165,6 +2191,7 @@ static const CPUCaches epyc_milan_cache_info = { .partitions = 1, .sets = 1024, .lines_per_tag = 1, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l3_cache = &(CPUCacheInfo ) { .type =UNIFIED_CACHE ,
@@ -2178,6 +2205,7 @@ static const CPUCaches epyc_milan_cache_info = { .self_init = true, .inclusive = true, .complex_indexing = true, + .share_level = CPU_TOPO_LEVEL_DIE, }, };
@@ -2193,6 +2221,7 @@ static const CPUCaches epyc_milan_v2_cache_info = { .lines_per_tag = 1, .self_init = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l1i_cache = &(CPUCacheInfo ) { .type =INSTRUCTION_CACHE ,
@@ -2205,6 +2234,7 @@ static const CPUCaches epyc_milan_v2_cache_info = { .lines_per_tag = 1, .self_init = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l2_cache = &(CPUCacheInfo ) { .type =UNIFIED_CACHE ,
@@ -2215,6 +2245,7 @@ static const CPUCaches epyc_milan_v2_cache_info = { .partitions = 1, .sets = 1024, .lines_per_tag = 1, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l3_cache = &(CPUCacheInfo ) { .type =UNIFIED_CACHE ,
@@ -2228,6 +2259,7 @@ static const CPUCaches epyc_milan_v2_cache_info = { .self_init = true, .inclusive = true, .complex_indexing = false, + .share_level = CPU_TOPO_LEVEL_DIE, }, };
@@ -2243,6 +2275,7 @@ static const CPUCaches epyc_genoa_cache_info = { .lines_per_tag = 1, .self_init = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l1i_cache = &(CPUCacheInfo ) { .type =INSTRUCTION_CACHE ,
@@ -2255,6 +2288,7 @@ static const CPUCaches epyc_genoa_cache_info = { .lines_per_tag = 1, .self_init = 1, .no_invd_sharing = true, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l2_cache = &(CPUCacheInfo ) { .type =UNIFIED_CACHE ,
@@ -2265,6 +2299,7 @@ static const CPUCaches epyc_genoa_cache_info = { .partitions = 1, .sets = 2048, .lines_per_tag = 1, + .share_level = CPU_TOPO_LEVEL_CORE, }, .l3_cache = &(CPUCacheInfo ) { .type =UNIFIED_CACHE ,
@@ -2278,6 +2313,7 @@ static const CPUCaches epyc_genoa_cache_info = { .self_init = true, .inclusive = true, .complex_indexing = false, + .share_level = CPU_TOPO_LEVEL_DIE, }, };
target/i386/cpu.h +7 -0
@@ -1590,6 +1590,13 @@ typedef struct CPUCacheInfo {* address bits. CPUID[4].EDX[bit 2]. */ bool complex_indexing; + +/* +* Cache Topology. The level that cache is shared in. +* Used to encode CPUID[4].EAX[bits 25:14] or +* CPUID[0x8000001D].EAX[bits 25:14]. +*/ +enum CPUTopoLevel share_level; } CPUCacheInfo;
Signature · 1 line
2.34.1
PATCH v9 20/21 i386/cpu: Use CPUCacheInfo.share_level to encode CPUID[4]
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
CPUID[4].EAX[bits 25:14] is used to represent the cache topology for Intel CPUs.
After cache models have topology information, we can use CPUCacheInfo.share_level to decide which topology level to be encoded into CPUID[4].EAX[bits 25:14].
And since with the helper max_processor_ids_for_cache(), the filed CPUID[4].EAX[bits 25:14] (original virable "num_apic_ids") is parsed based on cpu topology levels, which are verified when parsing -smp, it's no need to check this value by "assert(num_apic_ids > 0)" again, so remove this assert().
Additionally, wrap the encoding of CPUID[4].EAX[bits 31:26] into a helper to make the code cleaner.
---
Changes since v7:
* Renamed max_processor_ids_for_cache() to max_thread_ids_for_cache(). (Xiaoyao)
* Dropped Michael/Babu's ACKed/Tested tags since the code change.
* Re-added Yongwei's Tested tag For his re-testing.
Changes since v1:
* Used "enum CPUTopoLevel share_level" as the parameter in max_processor_ids_for_cache().
* Made cache_into_passthrough case also use max_processor_ids_for_cache() and max_core_ids_in_package() to encode CPUID[4]. (Yanan)
* Renamed the title of this patch (the original is "i386: Use CPUCacheInfo.share_level to encode CPUID[4].EAX[bits 25:14]").
--- target/i386/cpu.c | 76 ++++++++++++++++++++++++++++------------------- 1 file changed, 45 insertions(+), 31 deletions(-)
target/i386/cpu.c +45 -31
@@ -235,22 +235,53 @@ static uint8_t cpuid2_cache_descriptor(CPUCacheInfo *cache) ((t) == UNIFIED_CACHE) ? CACHE_TYPE_UNIFIED : \ 0 /* Invalid value */) +static uint32_t max_thread_ids_for_cache (X86CPUTopoInfo *topo_info, +enum CPUTopoLevel share_level) +{ +uint32_t num_ids =0 ; + +switch (share_level) { +case CPU_TOPO_LEVEL_CORE : + num_ids =1 <<apicid_core_offset (topo_info); +break ; +case CPU_TOPO_LEVEL_DIE : + num_ids =1 <<apicid_die_offset (topo_info); +break ; +case CPU_TOPO_LEVEL_PACKAGE : + num_ids =1 <<apicid_pkg_offset (topo_info); +break ; +default : +/* +* Currently there is no use case for SMT and MODULE, so use +* assert directly to facilitate debugging. +*/ +g_assert_not_reached (); + } + +return num_ids -1 ; +} + +static uint32_t max_core_ids_in_package (X86CPUTopoInfo *topo_info) +{ +uint32_t num_cores =1 << (apicid_pkg_offset (topo_info) - +apicid_core_offset (topo_info)); +return num_cores -1 ; +}/* Encode cache info for CPUID[4] */ static void encode_cache_cpuid4 (CPUCacheInfo *cache, -int num_apic_ids,int num_cores, +X86CPUTopoInfo *topo_info,uint32_t *eax,uint32_t *ebx,uint32_t *ecx,uint32_t *edx) {assert (cache->size == cache->line_size * cache->associativity * cache->partitions * cache->sets); -assert (num_apic_ids >0 ); *eax =CACHE_TYPE (cache->type) |CACHE_LEVEL (cache->level) | (cache->self_init ?CACHE_SELF_INIT_LEVEL :0 ) | - ((num_cores -1 ) <<26 ) | - ((num_apic_ids -1 ) <<14 ); + (max_core_ids_in_package (topo_info) <<26 ) | + (max_thread_ids_for_cache (topo_info, cache->share_level) <<14 );assert (cache->line_size >0 );assert (cache->partitions >0 );
@@ -6247,10 +6278,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, (cpuid2_cache_descriptor (env->cache_info_cpuid2.l1i_cache) <<8 ) | (cpuid2_cache_descriptor (env->cache_info_cpuid2.l2_cache));break ; -case 4 : { -int addressable_cores_width; -int addressable_threads_width; - +case 4 :/* cache info: needed for Core compatibility */ if (cpu->cache_info_passthrough) {x86_cpu_get_cache_cpuid (index, count, eax, ebx, ecx, edx);
@@ -6262,55 +6290,42 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,int host_vcpus_per_cache =1 + ((*eax &0x3FFC000 ) >>14 );if (cores_per_pkg >1 ) { - addressable_cores_width =apicid_pkg_offset (&topo_info) - -apicid_core_offset (&topo_info); - *eax &= ~0xFC000000 ; - *eax |= ((1 << addressable_cores_width) -1 ) <<26 ; + *eax |=max_core_ids_in_package (&topo_info) <<26 ; }if (host_vcpus_per_cache > threads_per_pkg) { -/* Share the cache at package level. */ - addressable_threads_width =apicid_pkg_offset (&topo_info); - *eax &= ~0x3FFC000 ; - *eax |= ((1 << addressable_threads_width) -1 ) <<14 ; + +/* Share the cache at package level. */ + *eax |=max_thread_ids_for_cache (&topo_info, +CPU_TOPO_LEVEL_PACKAGE ) <<14 ; } } }else if (cpu->vendor_cpuid_only &&IS_AMD_CPU (env)) { *eax = *ebx = *ecx = *edx =0 ; }else { *eax =0 ; - addressable_cores_width =apicid_pkg_offset (&topo_info) - -apicid_core_offset (&topo_info);switch (count) {case 0 :/* L1 dcache info */ - addressable_threads_width =apicid_core_offset (&topo_info);encode_cache_cpuid4 (env->cache_info_cpuid4.l1d_cache, - (1 << addressable_threads_width), - (1 << addressable_cores_width), + &topo_info, eax, ebx, ecx, edx);break ;case 1 :/* L1 icache info */ - addressable_threads_width =apicid_core_offset (&topo_info);encode_cache_cpuid4 (env->cache_info_cpuid4.l1i_cache, - (1 << addressable_threads_width), - (1 << addressable_cores_width), + &topo_info, eax, ebx, ecx, edx);break ;case 2 :/* L2 cache info */ - addressable_threads_width =apicid_core_offset (&topo_info);encode_cache_cpuid4 (env->cache_info_cpuid4.l2_cache, - (1 << addressable_threads_width), - (1 << addressable_cores_width), + &topo_info, eax, ebx, ecx, edx);break ;case 3 :/* L3 cache info */ if (cpu->enable_l3_cache) { - addressable_threads_width =apicid_die_offset (&topo_info);encode_cache_cpuid4 (env->cache_info_cpuid4.l3_cache, - (1 << addressable_threads_width), - (1 << addressable_cores_width), + &topo_info, eax, ebx, ecx, edx);break ; }
@@ -6321,7 +6336,6 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count, } }break ; - }case 5 :/* MONITOR/MWAIT Leaf */ *eax = cpu->mwait.eax;/* Smallest monitor-line size in bytes */
Signature · 1 line
2.34.1
PATCH v9 21/21 i386/cpu: Use CPUCacheInfo.share_level to encode CPUID[0x8000001D].EAX[bits 25:14]
In reply to Zhao LiuFrom: Zhao Liu <[email protected]>
CPUID[0x8000001D].EAX[bits 25:14] NumSharingCache: number of logical processors sharing cache.
The number of logical processors sharing this cache is NumSharingCache + 1.
After cache models have topology information, we can use CPUCacheInfo.share_level to decide which topology level to be encoded into CPUID[0x8000001D].EAX[bits 25:14].
---
Changes since v7:
* Renamed max_processor_ids_for_cache() to max_thread_ids_for_cache().
* Dropped Michael/Babu's ACKed/Tested tags since the code change.
* Re-added Yongwei's Tested tag For his re-testing.
Changes since v3:
* Explained what "CPUID[0x8000001D].EAX[bits 25:14]" means in the commit message. (Babu)
Changes since v1:
* Used cache->share_level as the parameter in max_processor_ids_for_cache().
--- target/i386/cpu.c | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-)
target/i386/cpu.c +1 -9
@@ -481,20 +481,12 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx) { - uint32_t num_sharing_cache; assert(cache->size == cache->line_size * cache->associativity * cache->partitions * cache->sets); *eax =CACHE_TYPE (cache->type) |CACHE_LEVEL (cache->level) | (cache->self_init ?CACHE_SELF_INIT_LEVEL :0 ); - -/* L3 is shared among multiple cores */ -if (cache->level ==3 ) { - num_sharing_cache =1 <<apicid_die_offset (topo_info); - }else { - num_sharing_cache =1 <<apicid_core_offset (topo_info); - } - *eax |= (num_sharing_cache -1 ) <<14 ; + *eax |=max_thread_ids_for_cache (topo_info, cache->share_level) <<14 ;assert (cache->line_size >0 );assert (cache->partitions >0 );
Signature · 1 line
2.34.1
Re: [PATCH v9 21/21] i386/cpu: Use CPUCacheInfo.share_level to encode CPUID[0x8000001D].EAX[bits 25:14]
Quoted 8 lines · Zhao Liu
From: Zhao Liu <[email protected]>
CPUID[0x8000001D].EAX[bits 25:14] NumSharingCache: number of logical processors sharing cache.
The number of logical processors sharing this cache is NumSharingCache + 1.
Quoted 7 lines
After cache models have topology information, we can use CPUCacheInfo.share_level to decide which topology level to be encoded into CPUID[0x8000001D].EAX[bits 25:14].
Cc Babu Moger <[email protected]>Tested-by Yongwei Ma <[email protected]>Signed-off-by Zhao Liu <[email protected]>
Quoted 43 lines
---
Changes since v7:
* Renamedmax_processor_ids_for_cache()tomax_thread_ids_for_cache().
* Dropped Michael/Babu's ACKed/Tested tags since the code change.
* Re-added Yongwei's Tested tag For his re-testing.Changes since v3:
* Explained what "CPUID[0x8000001D].EAX[bits 25:14]" means in the commit message. (Babu)Changes since v1:
* Usedcache->share_levelas the parameter inmax_processor_ids_for_cache().--- target/i386/cpu.c | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-)target/i386/cpu.c +1
-9@@ -481,20 +481,12 @@ static void encode_cache_cpuid8000001d(CPUCacheInfo *cache, uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx) { - uint32_t num_sharing_cache; assert(cache->size == cache->line_size * cache->associativity * cache->partitions * cache->sets); *eax =CACHE_TYPE (cache->type) |CACHE_LEVEL (cache->level) | (cache->self_init ?CACHE_SELF_INIT_LEVEL :0 ); - -/* L3 is shared among multiple cores */ -if (cache->level ==3 ) { - num_sharing_cache =1 <<apicid_die_offset (topo_info); - }else { - num_sharing_cache =1 <<apicid_core_offset (topo_info); - } - *eax |= (num_sharing_cache -1 ) <<14 ; + *eax |=max_thread_ids_for_cache (topo_info, cache->share_level) <<14 ;assert (cache->line_size >0 );assert (cache->partitions >0 );
Signature · 2 lines
Thanks Babu Moger
Hi Paolo & Michael,
BTW, may I ask if this series could land in v9.0? ;-)
Thanks,
Zhao
Quoted 226 lines · Zhao Liu
Date: Tue, 27 Feb 2024 18:32:10 +0800
From: Zhao Liu <[email protected]>
Subject: [PATCH v9 00/21] Introduce smp.modules for x86 in QEMU
X-Mailer: git-send-email 2.34.1From: Zhao Liu <[email protected]>
Hi list,
This is the our v9 patch series, rebased on the master branch at the commit
03d496a992d9("Merge tag 'pull-qapi-2024-02-26' of
https://repo.or.cz/qemu/armbru into staging").Compared with v8 [1], v9 mainly added more module description in commit message and added missing smp.modules description/documentation.
With the general introduction (with origial cluster level) of this secries in v7 [2] cover letter, the following sections are mainly about the description of the newly added smp.modules (since v8, changed x86 cluster support to module) as supplement.
Since v4 [3], we've dropped the original L2 cache command line option (to configure L2 cache topology) and now we have the new RFC [4] to support the general cache topology configuration (as the supplement to this series).
Welcome your comments!
Why We Need a New CPU Topology Level
====================================For the discussion in v7 about whether we should reuse current smp.clusters for x86 module, the core point is what's the essential differences between x86 module and general cluster.
Since, cluster (for ARM/riscv) lacks a comprehensive and rigorous hardware definition, and judging from the description of smp.clusters
[5] when it was introduced by QEMU, x86 module is very similar to general smp.clusters: they are all a layer above existing core level to organize the physical cores and share L2 cache.But there are following reasons that drive us to introduce the new smp.modules:
* As the CPU topology abstraction in device tree [6], cluster supports nesting (though currently QEMU hasn't support that). In contrast, (x86) module does not support nesting.
* Due to nesting, there is great flexibility in sharing resources on cluster, rather than narrowing cluster down to sharing L2 (and L3 tags) as the lowest topology level that contains cores.
* Flexible nesting of cluster allows it to correspond to any level between the x86 package and core.
* In Linux kernel, x86's cluster only represents the L2 cache domain but QEMU's smp.clusters is the CPU topology level. Linux kernel will also expose module level topology information in sysfs for x86. To avoid cluster ambiguity and keep a consistent CPU topology naming style with the Linux kernel, we introduce module level for x86.
Based on the above considerations, and in order to eliminate the naming confusion caused by the mapping between general cluster and x86 module, we now formally introduce smp.modules as the new topology level.
Where to Place Module in Existing Topology Levels
=================================================The module is, in existing hardware practice, the lowest layer that contains the core, while the cluster is able to have a higher topological scope than the module due to its nesting.
Therefore, we place the module between the cluster and the core:
drawer/book/socket/die/cluster/module/core/thread
Additional Consideration on CPU Topology
========================================Beyond this patchset, nowadays, different arches have different topology requirements, and maintaining arch-agnostic general topology in SMP becomes to be an increasingly difficult thing due to differences in sharing resources and special flexibility (e.g., nesting):
* It becomes difficult to put together all CPU topology hierarchies of different arches to define complete topology order.
* It also becomes complex to ensure the correctness of the topology calculations.
- Now themax_cpusis calculated by multiplying all topology levels, and too many topology levels can easily cause omissions.Maybe we should consider implementing arch-specfic topology hierarchies.
[1]: https://lore.kernel.org/qemu-devel/[email protected]/
[2]: https://lore.kernel.org/qemu-devel/[email protected]/
[3]: https://lore.kernel.org/qemu-devel/[email protected]/
[4]: https://lore.kernel.org/qemu-devel/[email protected]/
[5]: https://lore.kernel.org/qemu-devel/[email protected]/
[6]: https://www.kernel.org/doc/Documentation/devicetree/bindings/cpu/cpu-topology.txtThanks and Best Regards,
Zhao
---
Changelog:Changes since v8:
* Add the reason of why a new module level is needed in commit message. (Markus).
* Add the description about how Linux kernel supports x86 module level in commit message. (Daniel)
* Add module description inqemu_smp_opts.
* Add missing "modules" parameter of -smp example in documentation.
* Add Philippe's reviewed-by tag.Changes since v7 (main changes):
* Introduced smp.modules as a new CPU topology level. (Xiaoyao)
* Fixed calculations ofcache_info_passthroughcase in the patch "i386/cpu: Use APIC ID info to encode cache topo in CPUID[4]". (Xiaoyao)
* Moved the patch "i386/cpu: Use APIC ID info get NumSharingCache for CPUID[0x8000001D].EAX[bits 25:14]" after CPUID[4]'s similar change ("i386/cpu: Use APIC ID offset to encode cache topo in CPUID[4]"). (Xiaoyao)
* Introduced a bitmap in CPUX86State to cache available CPU topology levels.
* Refactored theencode_topo_cpuid1f()to use traversal to search the encoded level and avoid using static variables.
* Mapped x86 module to smp module instead of cluster.
* Dropped Michael/Babu's ACKed/Tested tags for most patches since the code change.Changes since v6:
* Updated the comment when check cluster-id. Since there's no v8.2, the cluster-id support should at least start from v9.0.
* Rebased on commitd328fef93ae7("Merge tag 'pull-20231230' of
https://gitlab.com/rth7680/qemu into staging").Changes since v5:
* The first four patches of v5 [1] have been merged, v6 contains the remaining patches.
* Reabsed on the latest master.
* Updated the comment when check cluster-id. Since current QEMU is v8.2, the cluster-id support should at least start from v8.3.Changes since v4:
* Dropped the "x-l2-cache-topo" option. (Michael)
* Added A/R/T tags.Changes since v3 (main changes):
* Exposed module level in CPUID[0x1F].
* Fixed compile warnings. (Babu)
* Fixed cache topology uninitialization bugs for some AMD CPUs. (Babu)Changes since v2:
* Added "Tested-by", "Reviewed-by" and "ACKed-by" tags.
* Used newly added wrapped helper to get cores per socket inqemu_init_vcpu().Changes since v1:
* Reordered patches. (Yanan)
* Deprecated the patch to fix comment ofmachine_parse_smp_config(). (Yanan)
* Renamedtest-x86-cpuid.ctotest-x86-topo.c. (Yanan)
* Split the intel's l1 cache topology fix into a new separate patch. (Yanan)
* Combinedmodule_idand APIC ID for module level support into one patch. (Yanan)
* Madecache_into_passthroughcase of cpuid0x04leaf in
*cpu_x86_cpuid()usedmax_processor_ids_for_cache()andmax_core_ids_in_package()to encode CPUID[4]. (Yanan)
* Added the prefix "CPU_TOPO_LEVEL_*" for CPU topology level names. (Yanan)
---
Zhao Liu (20):
hw/core/machine: Introduce the module as a CPU topology level hw/core/machine: Support modules in -smp
hw/core: Introduce module-id as the topology subindex hw/core: Support module-id in numa configuration i386/cpu: Fix i/d-cache topology to core level for Intel CPU i386/cpu: Use APIC ID info to encode cache topo in CPUID[4] i386/cpu: Use APIC ID info get NumSharingCache forCPUID[0x8000001D].EAX[bits 25:14] i386/cpu: Consolidate the use of topo_info in cpu_x86_cpuid() i386/cpu: Introduce bitmap to cache available CPU topology levels i386: Split topology types of CPUID[0x1F] from the definitions of CPUID[0xB] i386/cpu: Decouple CPUID[0x1F] subleaf with specific topology level i386: Introduce module level cpu topology to CPUX86State i386: Support modules_per_die in X86CPUTopoInfo i386: Expose module level in CPUID[0x1F] i386: Support module_id in X86CPUTopoIDs i386/cpu: Introduce module-id to X86CPU hw/i386/pc: Support smp.modules for x86 PC machine i386: Add cache topology info in CPUCacheInfo i386/cpu: Use CPUCacheInfo.share_level to encode CPUID[4] i386/cpu: Use CPUCacheInfo.share_level to encode CPUID[0x8000001D].EAX[bits 25:14]Zhuocheng Ding (1):
tests: Add test case of APIC ID for module level parsinghw/core/machine-hmp-cmds.c | 4 + hw/core/machine-smp.c | 41 +++-- hw/core/machine.c | 18 +++ hw/i386/pc.c | 1 + hw/i386/x86.c | 67 ++++++-- include/hw/boards.h | 4 + include/hw/i386/topology.h | 60 +++++++- qapi/machine.json | 7 + qemu-options.hx | 18 ++- system/vl.c | 3 + target/i386/cpu.c | 304 +++++++++++++++++++++++++++++-------- target/i386/cpu.h | 29 +++- target/i386/kvm/kvm.c | 3 +- tests/unit/test-x86-topo.c | 56 ++++--- 14 files changed, 489 insertions(+), 126 deletions(-)Signature · 1 line
2.34.1
Hi Paolo & Michael,
Just a kindly ping, could you please take time to review this series when you're free?
(This is also the foundation support of hybrid topology).
Many thanks,
Zhao
Quoted 239 lines · Zhao Liu
Date: Tue, 27 Feb 2024 18:41:36 +0800
From: Zhao Liu <[email protected]>
Subject: Re: [PATCH v9 00/21] Introduce smp.modules for x86 in QEMUHi Paolo & Michael,
BTW, may I ask if this series could land in v9.0? ;-)
Thanks,
ZhaoQuoted 226 lines · Zhao Liu
Date: Tue, 27 Feb 2024 18:32:10 +0800
From: Zhao Liu <[email protected]>
Subject: [PATCH v9 00/21] Introduce smp.modules for x86 in QEMU
X-Mailer: git-send-email 2.34.1From: Zhao Liu <[email protected]>
Hi list,
This is the our v9 patch series, rebased on the master branch at the commit
03d496a992d9("Merge tag 'pull-qapi-2024-02-26' of
https://repo.or.cz/qemu/armbru into staging").Compared with v8 [1], v9 mainly added more module description in commit message and added missing smp.modules description/documentation.
With the general introduction (with origial cluster level) of this secries in v7 [2] cover letter, the following sections are mainly about the description of the newly added smp.modules (since v8, changed x86 cluster support to module) as supplement.
Since v4 [3], we've dropped the original L2 cache command line option (to configure L2 cache topology) and now we have the new RFC [4] to support the general cache topology configuration (as the supplement to this series).
Welcome your comments!
Why We Need a New CPU Topology Level
====================================For the discussion in v7 about whether we should reuse current smp.clusters for x86 module, the core point is what's the essential differences between x86 module and general cluster.
Since, cluster (for ARM/riscv) lacks a comprehensive and rigorous hardware definition, and judging from the description of smp.clusters
[5] when it was introduced by QEMU, x86 module is very similar to general smp.clusters: they are all a layer above existing core level to organize the physical cores and share L2 cache.But there are following reasons that drive us to introduce the new smp.modules:
* As the CPU topology abstraction in device tree [6], cluster supports nesting (though currently QEMU hasn't support that). In contrast, (x86) module does not support nesting.
* Due to nesting, there is great flexibility in sharing resources on cluster, rather than narrowing cluster down to sharing L2 (and L3 tags) as the lowest topology level that contains cores.
* Flexible nesting of cluster allows it to correspond to any level between the x86 package and core.
* In Linux kernel, x86's cluster only represents the L2 cache domain but QEMU's smp.clusters is the CPU topology level. Linux kernel will also expose module level topology information in sysfs for x86. To avoid cluster ambiguity and keep a consistent CPU topology naming style with the Linux kernel, we introduce module level for x86.
Based on the above considerations, and in order to eliminate the naming confusion caused by the mapping between general cluster and x86 module, we now formally introduce smp.modules as the new topology level.
Where to Place Module in Existing Topology Levels
=================================================The module is, in existing hardware practice, the lowest layer that contains the core, while the cluster is able to have a higher topological scope than the module due to its nesting.
Therefore, we place the module between the cluster and the core:
drawer/book/socket/die/cluster/module/core/thread
Additional Consideration on CPU Topology
========================================Beyond this patchset, nowadays, different arches have different topology requirements, and maintaining arch-agnostic general topology in SMP becomes to be an increasingly difficult thing due to differences in sharing resources and special flexibility (e.g., nesting):
* It becomes difficult to put together all CPU topology hierarchies of different arches to define complete topology order.
* It also becomes complex to ensure the correctness of the topology calculations.
- Now themax_cpusis calculated by multiplying all topology levels, and too many topology levels can easily cause omissions.Maybe we should consider implementing arch-specfic topology hierarchies.
[1]: https://lore.kernel.org/qemu-devel/[email protected]/
[2]: https://lore.kernel.org/qemu-devel/[email protected]/
[3]: https://lore.kernel.org/qemu-devel/[email protected]/
[4]: https://lore.kernel.org/qemu-devel/[email protected]/
[5]: https://lore.kernel.org/qemu-devel/[email protected]/
[6]: https://www.kernel.org/doc/Documentation/devicetree/bindings/cpu/cpu-topology.txtThanks and Best Regards,
Zhao
---
Changelog:Changes since v8:
* Add the reason of why a new module level is needed in commit message. (Markus).
* Add the description about how Linux kernel supports x86 module level in commit message. (Daniel)
* Add module description inqemu_smp_opts.
* Add missing "modules" parameter of -smp example in documentation.
* Add Philippe's reviewed-by tag.Changes since v7 (main changes):
* Introduced smp.modules as a new CPU topology level. (Xiaoyao)
* Fixed calculations ofcache_info_passthroughcase in the patch "i386/cpu: Use APIC ID info to encode cache topo in CPUID[4]". (Xiaoyao)
* Moved the patch "i386/cpu: Use APIC ID info get NumSharingCache for CPUID[0x8000001D].EAX[bits 25:14]" after CPUID[4]'s similar change ("i386/cpu: Use APIC ID offset to encode cache topo in CPUID[4]"). (Xiaoyao)
* Introduced a bitmap in CPUX86State to cache available CPU topology levels.
* Refactored theencode_topo_cpuid1f()to use traversal to search the encoded level and avoid using static variables.
* Mapped x86 module to smp module instead of cluster.
* Dropped Michael/Babu's ACKed/Tested tags for most patches since the code change.Changes since v6:
* Updated the comment when check cluster-id. Since there's no v8.2, the cluster-id support should at least start from v9.0.
* Rebased on commitd328fef93ae7("Merge tag 'pull-20231230' of
https://gitlab.com/rth7680/qemu into staging").Changes since v5:
* The first four patches of v5 [1] have been merged, v6 contains the remaining patches.
* Reabsed on the latest master.
* Updated the comment when check cluster-id. Since current QEMU is v8.2, the cluster-id support should at least start from v8.3.Changes since v4:
* Dropped the "x-l2-cache-topo" option. (Michael)
* Added A/R/T tags.Changes since v3 (main changes):
* Exposed module level in CPUID[0x1F].
* Fixed compile warnings. (Babu)
* Fixed cache topology uninitialization bugs for some AMD CPUs. (Babu)Changes since v2:
* Added "Tested-by", "Reviewed-by" and "ACKed-by" tags.
* Used newly added wrapped helper to get cores per socket inqemu_init_vcpu().Changes since v1:
* Reordered patches. (Yanan)
* Deprecated the patch to fix comment ofmachine_parse_smp_config(). (Yanan)
* Renamedtest-x86-cpuid.ctotest-x86-topo.c. (Yanan)
* Split the intel's l1 cache topology fix into a new separate patch. (Yanan)
* Combinedmodule_idand APIC ID for module level support into one patch. (Yanan)
* Madecache_into_passthroughcase of cpuid0x04leaf in
*cpu_x86_cpuid()usedmax_processor_ids_for_cache()andmax_core_ids_in_package()to encode CPUID[4]. (Yanan)
* Added the prefix "CPU_TOPO_LEVEL_*" for CPU topology level names. (Yanan)
---
Zhao Liu (20):
hw/core/machine: Introduce the module as a CPU topology level hw/core/machine: Support modules in -smp
hw/core: Introduce module-id as the topology subindex hw/core: Support module-id in numa configuration i386/cpu: Fix i/d-cache topology to core level for Intel CPU i386/cpu: Use APIC ID info to encode cache topo in CPUID[4] i386/cpu: Use APIC ID info get NumSharingCache forCPUID[0x8000001D].EAX[bits 25:14] i386/cpu: Consolidate the use of topo_info in cpu_x86_cpuid() i386/cpu: Introduce bitmap to cache available CPU topology levels i386: Split topology types of CPUID[0x1F] from the definitions of CPUID[0xB] i386/cpu: Decouple CPUID[0x1F] subleaf with specific topology level i386: Introduce module level cpu topology to CPUX86State i386: Support modules_per_die in X86CPUTopoInfo i386: Expose module level in CPUID[0x1F] i386: Support module_id in X86CPUTopoIDs i386/cpu: Introduce module-id to X86CPU hw/i386/pc: Support smp.modules for x86 PC machine i386: Add cache topology info in CPUCacheInfo i386/cpu: Use CPUCacheInfo.share_level to encode CPUID[4] i386/cpu: Use CPUCacheInfo.share_level to encode CPUID[0x8000001D].EAX[bits 25:14]Zhuocheng Ding (1):
tests: Add test case of APIC ID for module level parsinghw/core/machine-hmp-cmds.c | 4 + hw/core/machine-smp.c | 41 +++-- hw/core/machine.c | 18 +++ hw/i386/pc.c | 1 + hw/i386/x86.c | 67 ++++++-- include/hw/boards.h | 4 + include/hw/i386/topology.h | 60 +++++++- qapi/machine.json | 7 + qemu-options.hx | 18 ++- system/vl.c | 3 + target/i386/cpu.c | 304 +++++++++++++++++++++++++++++-------- target/i386/cpu.h | 29 +++- target/i386/kvm/kvm.c | 3 +- tests/unit/test-x86-topo.c | 56 ++++--- 14 files changed, 489 insertions(+), 126 deletions(-)Signature · 1 line
2.34.1
Sanity tested on AMD machine. Looks good.
Quoted 217 lines · Zhao Liu
From: Zhao Liu <[email protected]>
Hi list,
This is the our v9 patch series, rebased on the master branch at the commit
03d496a992d9("Merge tag 'pull-qapi-2024-02-26' of
https://repo.or.cz/qemu/armbru into staging").Compared with v8 [1], v9 mainly added more module description in commit message and added missing smp.modules description/documentation.
With the general introduction (with origial cluster level) of this secries in v7 [2] cover letter, the following sections are mainly about the description of the newly added smp.modules (since v8, changed x86 cluster support to module) as supplement.
Since v4 [3], we've dropped the original L2 cache command line option (to configure L2 cache topology) and now we have the new RFC [4] to support the general cache topology configuration (as the supplement to this series).
Welcome your comments!
Why We Need a New CPU Topology Level
====================================For the discussion in v7 about whether we should reuse current smp.clusters for x86 module, the core point is what's the essential differences between x86 module and general cluster.
Since, cluster (for ARM/riscv) lacks a comprehensive and rigorous hardware definition, and judging from the description of smp.clusters
[5] when it was introduced by QEMU, x86 module is very similar to general smp.clusters: they are all a layer above existing core level to organize the physical cores and share L2 cache.But there are following reasons that drive us to introduce the new smp.modules:
* As the CPU topology abstraction in device tree [6], cluster supports nesting (though currently QEMU hasn't support that). In contrast, (x86) module does not support nesting.
* Due to nesting, there is great flexibility in sharing resources on cluster, rather than narrowing cluster down to sharing L2 (and L3 tags) as the lowest topology level that contains cores.
* Flexible nesting of cluster allows it to correspond to any level between the x86 package and core.
* In Linux kernel, x86's cluster only represents the L2 cache domain but QEMU's smp.clusters is the CPU topology level. Linux kernel will also expose module level topology information in sysfs for x86. To avoid cluster ambiguity and keep a consistent CPU topology naming style with the Linux kernel, we introduce module level for x86.
Based on the above considerations, and in order to eliminate the naming confusion caused by the mapping between general cluster and x86 module, we now formally introduce smp.modules as the new topology level.
Where to Place Module in Existing Topology Levels
=================================================The module is, in existing hardware practice, the lowest layer that contains the core, while the cluster is able to have a higher topological scope than the module due to its nesting.
Therefore, we place the module between the cluster and the core:
drawer/book/socket/die/cluster/module/core/thread
Additional Consideration on CPU Topology
========================================Beyond this patchset, nowadays, different arches have different topology requirements, and maintaining arch-agnostic general topology in SMP becomes to be an increasingly difficult thing due to differences in sharing resources and special flexibility (e.g., nesting):
* It becomes difficult to put together all CPU topology hierarchies of different arches to define complete topology order.
* It also becomes complex to ensure the correctness of the topology calculations.
- Now themax_cpusis calculated by multiplying all topology levels, and too many topology levels can easily cause omissions.Maybe we should consider implementing arch-specfic topology hierarchies.
[1]: https://lore.kernel.org/qemu-devel/[email protected]/
[2]: https://lore.kernel.org/qemu-devel/[email protected]/
[3]: https://lore.kernel.org/qemu-devel/[email protected]/
[4]: https://lore.kernel.org/qemu-devel/[email protected]/
[5]: https://lore.kernel.org/qemu-devel/[email protected]/
[6]: https://www.kernel.org/doc/Documentation/devicetree/bindings/cpu/cpu-topology.txtThanks and Best Regards,
Zhao
---
Changelog:Changes since v8:
* Add the reason of why a new module level is needed in commit message. (Markus).
* Add the description about how Linux kernel supports x86 module level in commit message. (Daniel)
* Add module description inqemu_smp_opts.
* Add missing "modules" parameter of -smp example in documentation.
* Add Philippe's reviewed-by tag.Changes since v7 (main changes):
* Introduced smp.modules as a new CPU topology level. (Xiaoyao)
* Fixed calculations ofcache_info_passthroughcase in the patch "i386/cpu: Use APIC ID info to encode cache topo in CPUID[4]". (Xiaoyao)
* Moved the patch "i386/cpu: Use APIC ID info get NumSharingCache for CPUID[0x8000001D].EAX[bits 25:14]" after CPUID[4]'s similar change ("i386/cpu: Use APIC ID offset to encode cache topo in CPUID[4]"). (Xiaoyao)
* Introduced a bitmap in CPUX86State to cache available CPU topology levels.
* Refactored theencode_topo_cpuid1f()to use traversal to search the encoded level and avoid using static variables.
* Mapped x86 module to smp module instead of cluster.
* Dropped Michael/Babu's ACKed/Tested tags for most patches since the code change.Changes since v6:
* Updated the comment when check cluster-id. Since there's no v8.2, the cluster-id support should at least start from v9.0.
* Rebased on commitd328fef93ae7("Merge tag 'pull-20231230' of
https://gitlab.com/rth7680/qemu into staging").Changes since v5:
* The first four patches of v5 [1] have been merged, v6 contains the remaining patches.
* Reabsed on the latest master.
* Updated the comment when check cluster-id. Since current QEMU is v8.2, the cluster-id support should at least start from v8.3.Changes since v4:
* Dropped the "x-l2-cache-topo" option. (Michael)
* Added A/R/T tags.Changes since v3 (main changes):
* Exposed module level in CPUID[0x1F].
* Fixed compile warnings. (Babu)
* Fixed cache topology uninitialization bugs for some AMD CPUs. (Babu)Changes since v2:
* Added "Tested-by", "Reviewed-by" and "ACKed-by" tags.
* Used newly added wrapped helper to get cores per socket inqemu_init_vcpu().Changes since v1:
* Reordered patches. (Yanan)
* Deprecated the patch to fix comment ofmachine_parse_smp_config(). (Yanan)
* Renamedtest-x86-cpuid.ctotest-x86-topo.c. (Yanan)
* Split the intel's l1 cache topology fix into a new separate patch. (Yanan)
* Combinedmodule_idand APIC ID for module level support into one patch. (Yanan)
* Madecache_into_passthroughcase of cpuid0x04leaf in
*cpu_x86_cpuid()usedmax_processor_ids_for_cache()andmax_core_ids_in_package()to encode CPUID[4]. (Yanan)
* Added the prefix "CPU_TOPO_LEVEL_*" for CPU topology level names. (Yanan)
---
Zhao Liu (20):
hw/core/machine: Introduce the module as a CPU topology level hw/core/machine: Support modules in -smp
hw/core: Introduce module-id as the topology subindex hw/core: Support module-id in numa configuration i386/cpu: Fix i/d-cache topology to core level for Intel CPU i386/cpu: Use APIC ID info to encode cache topo in CPUID[4] i386/cpu: Use APIC ID info get NumSharingCache forCPUID[0x8000001D].EAX[bits 25:14] i386/cpu: Consolidate the use of topo_info in cpu_x86_cpuid() i386/cpu: Introduce bitmap to cache available CPU topology levels i386: Split topology types of CPUID[0x1F] from the definitions of CPUID[0xB] i386/cpu: Decouple CPUID[0x1F] subleaf with specific topology level i386: Introduce module level cpu topology to CPUX86State i386: Support modules_per_die in X86CPUTopoInfo i386: Expose module level in CPUID[0x1F] i386: Support module_id in X86CPUTopoIDs i386/cpu: Introduce module-id to X86CPU hw/i386/pc: Support smp.modules for x86 PC machine i386: Add cache topology info in CPUCacheInfo i386/cpu: Use CPUCacheInfo.share_level to encode CPUID[4] i386/cpu: Use CPUCacheInfo.share_level to encode CPUID[0x8000001D].EAX[bits 25:14]Zhuocheng Ding (1):
tests: Add test case of APIC ID for module level parsinghw/core/machine-hmp-cmds.c | 4 + hw/core/machine-smp.c | 41 +++-- hw/core/machine.c | 18 +++ hw/i386/pc.c | 1 + hw/i386/x86.c | 67 ++++++-- include/hw/boards.h | 4 + include/hw/i386/topology.h | 60 +++++++- qapi/machine.json | 7 + qemu-options.hx | 18 ++- system/vl.c | 3 + target/i386/cpu.c | 304 +++++++++++++++++++++++++++++-------- target/i386/cpu.h | 29 +++- target/i386/kvm/kvm.c | 3 +- tests/unit/test-x86-topo.c | 56 ++++--- 14 files changed, 489 insertions(+), 126 deletions(-)
Signature · 2 lines
Thanks Babu Moger
Quoted 6 lines · Zhao Liu
---
Zhao Liu (20):
hw/core/machine: Introduce the module as a CPU topology level hw/core/machine: Support modules in -smp
hw/core: Introduce module-id as the topology subindex hw/core: Support module-id in numa configuration
Patches 1-4 queued, thanks!
Quoted 6 lines · Philippe Mathieu-Daudé
Date: Fri, 8 Mar 2024 17:36:38 +0100
From: Philippe Mathieu-Daudé <[email protected]>
Subject: Re: [PATCH v9 00/21] Introduce smp.modules for x86 in QEMUOn 27/2/24 11:32, Zhao Liu wrote:
Quoted 8 lines
Quoted 6 lines
---
Zhao Liu (20):
hw/core/machine: Introduce the module as a CPU topology level hw/core/machine: Support modules in -smp
hw/core: Introduce module-id as the topology subindex hw/core: Support module-id in numa configurationPatches 1-4 queued, thanks!
Thanks Philippe!
Hi Zhao,
Quoted 14 lines · Zhao Liu
Quoted 13 lines · Philippe Mathieu-Daudé
Date: Fri, 8 Mar 2024 17:36:38 +0100
From: Philippe Mathieu-Daudé <[email protected]>
Subject: Re: [PATCH v9 00/21] Introduce smp.modules for x86 in QEMUQuoted 6 lines · Zhao Liu
---
Zhao Liu (20):
hw/core/machine: Introduce the module as a CPU topology level hw/core/machine: Support modules in -smp
hw/core: Introduce module-id as the topology subindex hw/core: Support module-id in numa configuration
Quoted 3 lines
Quoted 1 line
Patches 1-4 queued, thanks!
Thanks Philippe!
I dropped this 4 patches in favor of "Cleanup on SMP and its test"
v2 (https://lore.kernel.org/qemu-devel/[email protected]/)
which seems more important, to get the "parameter=0" deprecation
in the next release. (Patch 2 diverged).
Regards,
Phil.
Quoted 5 lines
I dropped this 4 patches in favor of "Cleanup on SMP and its test"
v2 (https://lore.kernel.org/qemu-devel/[email protected]/)
which seems more important, to get the "parameter=0" deprecation
in the next release. (Patch 2 diverged).
Thanks! This series will continue to be refreshed and rebased on the new master code base.
Regards,
Zhao