s390x: CPU Topology
Subsystems (2)
v21 cover s390x: CPU Topology
Hi,
What is new:
- changed name of target/s390/kvm/cpu_topology to target/s390/kvm/stsi-topology
- moved the TLE list head from a global to a local in insert_stsi_15_1_x()
- cleaning and merging some of the avocado tests
- modification patch "query-cpu-fast" due to upstream modifications I removed the RB but the changes are minimal.
Implementation discussions
==========================
CPU models
----------
Since the facility 11, S390_FEAT_CONFIGURATION_TOPOLOGY is already in the CPU model for old QEMU we could not activate it as usual from KVM but needed a KVM capability: KVM_CAP_S390_CPU_TOPOLOGY. Checking and enabling this capability enables facility 11, S390_FEAT_CONFIGURATION_TOPOLOGY.
It is the responsibility of the admin to ensure the same CPU model for source and target host in a migration.
Migration
---------
When the target guest is started, the Multi-processor Topology Change Report (MTCR) bit is set during the creation of the vCPU by KVM. We do not need to migrate its state, in the worst case, the target guest will see the MTCR and actualize its view of the topology without necessity, but this will be done only one time.
Reset
-----
Reseting the topology is done during subsystem reset, the polarization is reset to horizontal polarization.
Topology attributes
-------------------
The topology attributes are carried by the CPU object and defined on object creation.
In the case the new attributes, socket, book, drawer, dedicated, entitlement are not provided QEMU provides defaults values.
- Geometry defaults
The geometry default are based on the core-id of the core to fill the geometry in a monotone way starting with drawer 0, book 0, and filling socket 0 with the number of cores per socket, then filling socket 1, socket 2 ... etc until the book is complete and all books until the first drawer is complete before starting with the next drawer.
This allows to keep existing start scripts and Libvirt existing interface until it is extended.
- Modifiers defaults
Default entitlement is medium
Default dedication is not dedicated.
- Machine polarization default to horizontal
Dynamic topology modification
-----------------------------
QAPI interface is extended with:
- a command: 'set-cpu-topology'
- a query: 'query-cpu-polarization'
- a query: extension of qmp 'query-cpus-fast'
- a query: extension of hmp 'hotpluggable-cpus'
- an event: 'CPU_POLARITY_CHANGE'
New command and interface are specified as unstable.
The admin may use query-cpus-fast to verify the topology provided to the guest and set-cpu-topology to modify it.
The event CPU_POLARITY_CHANGE is sent when the guest successfuly uses the PTF(2) instruction to request a polarization change. In that case, the admin is supposed to modify the CPU provisioning accordingly.
Testing
=======
To use the QEMU patches, you will need Linux V6-rc1 or newer, or use the following Linux mainline patches:
f5ecfee94493 2022-07-20 KVM: s390: resetting the Topology-Change-Report 24fe0195bc19 2022-07-20 KVM: s390: guest support for topology function 0130337ec45b 2022-07-20 KVM: s390: Cleanup ipte lock access and SIIF fac..
Currently this code is for KVM only, I have no idea if it is interesting to provide a TCG patch. If ever it will be done in another series.
This series provide 12 avocado tests using Fedora-35 kernel and initrd image.
Documentation
=============
To have a better understanding of the S390x CPU Topology and its implementation in QEMU you can have a look at the documentation in the last patch of this series.
The admin will want to match the host and the guest topology, taking into account that the guest does not recognize multithreading. Consequently, two vCPU assigned to threads of the same real CPU should preferably be assigned to the same socket of the guest machine.
Regards,
Pierre
Pierre Morel (20):
s390x/cpu topology: add s390 specifics to CPU topology s390x/cpu topology: add topology entries on CPU hotplug target/s390x/cpu topology: handle STSI(15) and build the SYSIB s390x/sclp: reporting the maximum nested topology entries s390x/cpu topology: resetting the Topology-Change-Report s390x/cpu topology: interception of PTF instruction target/s390x/cpu topology: activate CPU topology qapi/s390x/cpu topology: set-cpu-topology qmp command
machine: adding s390 topology to query-cpu-fast
machine: adding s390 topology to info hotpluggable-cpus qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event qapi/s390x/cpu topology: query-cpu-polarization qmp command docs/s390x/cpu topology: document s390x cpu topology tests/avocado: s390x cpu topology core tests/avocado: s390x cpu topology polarisation tests/avocado: s390x cpu topology entitlement tests tests/avocado: s390x cpu topology test dedicated CPU tests/avocado: s390x cpu topology test socket full tests/avocado: s390x cpu topology dedicated errors tests/avocado: s390x cpu topology bad move
MAINTAINERS | 10 + docs/devel/index-internals.rst | 1 + docs/devel/s390-cpu-topology.rst | 170 ++++++++++ docs/system/s390x/cpu-topology.rst | 240 ++++++++++++++ docs/system/target-s390x.rst | 1 + qapi/machine-common.json | 22 ++ qapi/machine-target.json | 113 +++++++ qapi/machine.json | 30 +- include/hw/boards.h | 10 +- include/hw/qdev-properties-system.h | 4 + include/hw/s390x/cpu-topology.h | 80 +++++ include/hw/s390x/s390-virtio-ccw.h | 6 + include/hw/s390x/sclp.h | 4 +- target/s390x/cpu.h | 83 +++++ target/s390x/kvm/kvm_s390x.h | 1 + hw/core/machine-hmp-cmds.c | 6 + hw/core/machine-smp.c | 48 ++- hw/core/machine.c | 4 + hw/core/qdev-properties-system.c | 13 + hw/s390x/cpu-topology.c | 475 ++++++++++++++++++++++++++++ hw/s390x/s390-virtio-ccw.c | 27 +- hw/s390x/sclp.c | 5 + softmmu/vl.c | 6 + target/s390x/cpu-sysemu.c | 13 + target/s390x/cpu.c | 11 + target/s390x/cpu_models.c | 1 + target/s390x/kvm/kvm.c | 42 ++- target/s390x/kvm/stsi-topology.c | 310 ++++++++++++++++++ hw/s390x/meson.build | 1 + qapi/meson.build | 1 + qemu-options.hx | 7 +- target/s390x/kvm/meson.build | 3 +- tests/avocado/s390_topology.py | 415 ++++++++++++++++++++++++ 33 files changed, 2143 insertions(+), 20 deletions(-) create mode 100644 docs/devel/s390-cpu-topology.rst create mode 100644 docs/system/s390x/cpu-topology.rst create mode 100644 qapi/machine-common.json create mode 100644 include/hw/s390x/cpu-topology.h create mode 100644 hw/s390x/cpu-topology.c create mode 100644 target/s390x/kvm/stsi-topology.c create mode 100644 tests/avocado/s390_topology.py
Signature · 290 lines
2.31.1 Since v20: - changed name of target/s390/kvm/cpu_topology to target/s390/kvm/stsi-topology (Thomas, Cedric) - moved the TLE list head from a global to a local in insert_stsi_15_1_x() (Nina) - cleaning and merging some of the avocado tests (Nina) - Several cleanings (Cedric, Thomas, Nina) - moved setting of entitlement and dedicated from disapeared cpustate_to_cpuinfo_s390() to new s390_query_cpu_fast() - small changes to documentation to reflect last changes using enum instead of int for polarization. 0 -> horizontal and 1 -> vertical Since v19: - use enum to specify the entitlement (Nina) - Change default entitlement to "auto" - suppress skip_basis in avocado tests (after comment from Cedric) - Correction of the documentation (Cedric) - better code organization for s390_topology_add_core_to_socket and s390_socket_nb (Cedric) - Changed Copyright to respect IBM policy (Nina) - set vertical_polarization back into s390_topology (Nina) Since v18: - Changed default entitlement to S390_CPU_ENTITLEMENT__MAX because no default can be correct. - added polarization entry to the S390CcwMachineState which allow to suppress shadow entitlement - Suppress shadow entitlement (Nina) - Added query-cpu-polarization - Added more avocado tests - modified cpu_hierarchy_to_string to look better (Nina) Since v17: - bug correction in handling PTF - added avocado tests (Thomas) - Change comments on QEMU machine to 8.1 Since v16: - documentation, split, bug correction and rephrasing (Nina, Thomas) - create machine-common.json - use of entitlement_shadow to keep track of the entitlement - adding drawers and books to query-hotpluggable-cpus - keep hmp interface for set-cpu-topology for a future series Since v15: - Use Enum for polarity and entitlement (Nina) - move kvm_vm_enable_cap(KVM_CAP_S390_CPU_TOPOLOGY) to kvm_arch_init() (Thomas) - Make all CPU attributes optional for set-cpu-topology monitor command (Thomas, Nina) - Change use of the prefix "x-" to the use of feature unstable to declare set-cpu-topology as unstable. (Nina) - Make CPU_POLARITY_CHANGE even as unstable (Nina) - Documentation update (Thomas, Nina) Since v14: - move the ordering of TLE to just before filling the SYSIB, optimize TLE ordering to be done on need only. (Cedric in previous series) - remove 'query-topology' and instead extend 'query-cpus-fast' (Daniel) - rename POLARITY_CHANGE to CPU_POLARITY_CHANGE (Thomas) - Divers bugs correction and doc changes (Thomas, Nina) - Separate topology and entitlement, simplify pft handling (Nina) - add the resetting of all CPU to horizontal polarity once implementing PTF interpretation Since v13: - Suppress the topology device to simplify the code (Cedric) - moved reset of MTCR from device reset into subsystem reset and removed previous reviewed-by from Nico and Janis - No need for Migration - No need for machine dependencies (Christian, Thomas) - Adding all features, drawer/book and dynamic (Cedric) - since v12 - suppress new CPU flag "disable-topology" just use ctop - no use of special fields in CCW machine or in CPU - modifications in documentation - insert documentation in tree (Cedric) - moved cpu-topology.c from target/s390 to target/s390/kvm to compile smoothly (without topology) for TCG (Cedric) - since v11 - new CPU flag "disable-topology" I would have take "topology" if I was able to have it false on default. (Christian, Thomas) - Build the topology during the interception of the STSI instruction. (Cedric) - return CC3 in case the calculated SYSIB length is greater than 4096. (Janis) - minor corections on documentation - since v10 - change machine attribute "topology-disable" to "topology" (Cedric) - Add preliminary patch for machine properties (Cedric) - Use next machine as 7.2 (Cedric / Connie) - Remove unecessary mutex (Thomas) - use ENOTSUP return value for kvm_s390_topology_set_mtcr() (Cedric) - Add explanation on container and cpu TLEs (Thomas) - use again cpu and socket count in topology structure (Cedric) - Suppress the S390TopoTLE structure and integrate the TLE masks to the socket structure. (-) - the STSI instruction now finds the topology from the machine (Cedric) - since v9 - remove books and drawers - remove thread denying and replace with a merge of cores * threads to specify the CPUs available to the guest - add a class option to avoid topology on older machines (Cedric) - Allocate a SYSIB buffer of the maximal length to avoid overflow. (Nico, Janis) - suppress redundancy of smp parameters in topology and use directly the machine smp structure - Early check for topology support (Cedric) - since v8 - Linux patches are now mainline - simplification of the implementation (Janis) - Migration, new machine definition (Thomas) - Documentation - since v7 - Coherence with the Linux patch series changes for MTCR get (Pierre) - check return values during new CPU creation (Thomas) - Improving codding style and argument usages (Thomas) - since v6 - Changes on smp args in qemu-options (Daniel) - changed comments in machine.jason (Daniel) - Added reset (Janosch) - since v5 - rebasing on newer QEMU version - reworked most lines above 80 characters. - since v4 - Added drawer and books to topology - Added numa topology - Added documentation - since v3 - Added migration (Thomas) - Separated STSI instruction from KVM to prepare TCG (Thomas) - Take care of endianess to prepare TCG (Thomas) - Added comments on STSI CPU container and PFT instruction (Thomas) - Moved enabling the instructions as the last patch (Thomas)
PATCH v21 1/20 s390x/cpu topology: add s390 specifics to CPU topology
S390 adds two new SMP levels, drawers and books to the CPU topology.
The S390 CPU have specific topology features like dedication and entitlement to give to the guest indications on the host vCPUs scheduling and help the guest take the best decisions on the scheduling of threads on the vCPUs.
Let us provide the SMP properties with books and drawers levels and S390 CPU with dedication and entitlement,
--- qapi/machine-common.json | 22 +++++++++++++ qapi/machine.json | 21 ++++++++++--- include/hw/boards.h | 10 +++++- include/hw/qdev-properties-system.h | 4 +++ target/s390x/cpu.h | 6 ++++ hw/core/machine-smp.c | 48 ++++++++++++++++++++++++----- hw/core/machine.c | 4 +++ hw/core/qdev-properties-system.c | 13 ++++++++ hw/s390x/s390-virtio-ccw.c | 2 ++ softmmu/vl.c | 6 ++++ target/s390x/cpu.c | 7 +++++ qapi/meson.build | 1 + qemu-options.hx | 7 +++-- 13 files changed, 137 insertions(+), 14 deletions(-) create mode 100644 qapi/machine-common.json
qapi/machine-common.json new file +22 -0
@@ -0,0 +1,22 @@ +# -*- Mode: Python -*- +# vim: filetype=python +# +# This work is licensed under the terms of the GNU GPL, version 2 or later. +# See the COPYING file in the top-level directory. + +## +# = Machines S390 data types +## + +## +# @CpuS390Entitlement: +# +# An enumeration of cpu entitlements that can be assumed by a virtual +# S390 CPU +# +# Since: 8.1 +## +{ 'enum': 'CpuS390Entitlement', + 'prefix': 'S390_CPU_ENTITLEMENT', + 'data': [ 'auto', 'low', 'medium', 'high' ] } +
qapi/machine.json +17 -4
@@ -9,6 +9,7 @@ ## { 'include': 'common.json' } +{ 'include': 'machine-common.json' } ## # @SysEmuTarget:
@@ -71,8 +72,8 @@ # # @thread-id: ID of the underlying host thread # -# @props: properties describing to which node/socket/core/thread -# virtual CPU belongs to, provided if supported by board +# @props: properties describing to which node/drawer/book/socket/core/thread +# virtual CPU belongs to, provided if supported by board # # @target: the QEMU system emulation target, which determines which # additional fields will be listed (since 3.0)
@@ -901,7 +902,11 @@ # # @node-id: NUMA node ID the CPU belongs to # -# @socket-id: socket number within node/board the CPU belongs to +# @drawer-id: drawer number within node/board the CPU belongs to (since 8.1) +# +# @book-id: book number within drawer/node/board the CPU belongs to (since 8.1) +# +# @socket-id: socket number within book/node/board the CPU belongs to # # @die-id: die number within socket the CPU belongs to (since 4.1) #
@@ -912,7 +917,7 @@ # # @thread-id: thread number within core the CPU belongs to # -# Note: currently there are 6 properties that could be present but +# Note: currently there are 8 properties that could be present but # management should be prepared to pass through other properties # with device_add command to allow for future interface extension. # This also requires the filed names to be kept in sync with the
@@ -922,6 +927,8 @@ ## { 'struct': 'CpuInstanceProperties', 'data': { '*node-id': 'int', + '*drawer-id': 'int', + '*book-id': 'int', '*socket-id': 'int', '*die-id': 'int', '*cluster-id': 'int',
@@ -1480,6 +1487,10 @@ # # @cpus: number of virtual CPUs in the virtual machine # +# @drawers: number of drawers in the CPU topology (since 8.1) +# +# @books: number of books in the CPU topology (since 8.1) +# # @sockets: number of sockets in the CPU topology # # @dies: number of dies per socket in the CPU topology
@@ -1498,6 +1509,8 @@ ## { 'struct': 'SMPConfiguration', 'data': { '*cpus': 'int', + '*drawers': 'int', + '*books': 'int', '*sockets': 'int', '*dies': 'int', '*clusters': 'int',
include/hw/boards.h +9 -1
@@ -131,12 +131,16 @@ typedef struct {* @clusters_supported - whether clusters are supported by the machine * @has_clusters - whether clusters are explicitly specified in the user * provided SMP configuration +* @books_supported - whether books are supported by the machine +* @drawers_supported - whether drawers are supported by the machine */ typedef struct {bool prefer_sockets;bool dies_supported;bool clusters_supported;bool has_clusters; +bool books_supported; +bool drawers_supported; }SMPCompatProps ;/**
@@ -305,7 +309,9 @@ typedef struct DeviceMemoryState {/** * CpuTopology: * @cpus: the number of present logical processors on the machine -* @sockets: the number of sockets on the machine +* @drawers: the number of drawers on the machine +* @books: the number of books in one drawer +* @sockets: the number of sockets in one book * @dies: the number of dies in one socket * @clusters: the number of clusters in one die * @cores: the number of cores in one cluster
@@ -314,6 +320,8 @@ typedef struct DeviceMemoryState {*/ typedef struct CpuTopology { unsigned int cpus; + unsigned int drawers; + unsigned int books; unsigned int sockets; unsigned int dies; unsigned int clusters;
include/hw/qdev-properties-system.h +4 -0
@@ -22,6 +22,7 @@ extern const PropertyInfo qdev_prop_audiodev;extern const PropertyInfo qdev_prop_off_auto_pcibar;extern const PropertyInfo qdev_prop_pcie_link_speed;extern const PropertyInfo qdev_prop_pcie_link_width; +extern const PropertyInfo qdev_prop_cpus390entitlement;#define DEFINE_PROP_PCI_DEVFN (_n, _s, _f, _d) \ DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_pci_devfn, int32_t)
@@ -73,5 +74,8 @@ extern const PropertyInfo qdev_prop_pcie_link_width;#define DEFINE_PROP_UUID_NODEFAULT (_name, _state, _field) \ DEFINE_PROP(_name, _state, _field, qdev_prop_uuid, QemuUUID) +#define DEFINE_PROP_CPUS390ENTITLEMENT (_n, _s, _f, _d) \ + DEFINE_PROP_SIGNED(_n, _s, _f, _d, qdev_prop_cpus390entitlement, \ + CpuS390Entitlement)#endif
target/s390x/cpu.h +6 -0
@@ -30,6 +30,7 @@#include "exec/cpu-defs.h" #include "qemu/cpu-float.h" #include "tcg/tcg_s390x.h" +#include "qapi/qapi-types-machine-common.h" #define ELF_MACHINE_UNAME "S390X"
@@ -130,6 +131,11 @@ struct CPUArchState {#if !defined(CONFIG_USER_ONLY )uint32_t core_id;/* PoP "CPU address", same as cpu_index */ +int32_t socket_id; +int32_t book_id; +int32_t drawer_id; +bool dedicated; +CpuS390Entitlement entitlement;/* Used only for vertical polarization */ uint64_t cpuid;#endif
hw/core/machine-smp.c +41 -7
105 lines not on this page · raw message
hw/core/machine.c +4 -0
@@ -839,6 +839,8 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name,MachineState *ms =MACHINE (obj);SMPConfiguration *config = &(SMPConfiguration ){ .has_cpus = true, .cpus = ms->smp.cpus, + .has_drawers = true, .drawers = ms->smp.drawers, + .has_books = true, .books = ms->smp.books, .has_sockets = true, .sockets = ms->smp.sockets, .has_dies = true, .dies = ms->smp.dies, .has_clusters = true, .clusters = ms->smp.clusters,
@@ -1104,6 +1106,8 @@ static void machine_initfn(Object *obj)/* default to mc->default_cpus */ ms->smp.cpus = mc->default_cpus; ms->smp.max_cpus = mc->default_cpus; + ms->smp.drawers =1 ; + ms->smp.books =1 ; ms->smp.sockets =1 ; ms->smp.dies =1 ; ms->smp.clusters =1 ;
hw/core/qdev-properties-system.c +13 -0
@@ -1147,3 +1147,16 @@ const PropertyInfo qdev_prop_uuid = { .set = set_uuid, .set_default_value = set_default_uuid_auto, }; + +/* --- s390 cpu entitlement policy --- */ + +QEMU_BUILD_BUG_ON (sizeof (CpuS390Entitlement) !=sizeof (int )); + +const PropertyInfo qdev_prop_cpus390entitlement = { + .name ="CpuS390Entitlement" , + .description ="low/medium (default)/high" , + .enum_table = &CpuS390Entitlement_lookup, + .get = qdev_propinfo_get_enum, + .set = qdev_propinfo_set_enum, + .set_default_value = qdev_propinfo_set_default_value_enum, +};
hw/s390x/s390-virtio-ccw.c +2 -0
@@ -737,6 +737,8 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data) mc->no_sdcard =1 ; mc->max_cpus =S390_MAX_CPUS ; mc->has_hotpluggable_cpus = true; + mc->smp_props.books_supported = true; + mc->smp_props.drawers_supported = true;assert (!mc->get_hotplug_handler); mc->get_hotplug_handler = s390_get_hotplug_handler; mc->cpu_index_to_instance_props = s390_cpu_index_to_props;
softmmu/vl.c +6 -0
@@ -725,6 +725,12 @@ static QemuOptsList qemu_smp_opts = { { .name = "cpus", .type = QEMU_OPT_NUMBER, + }, { + .name = "drawers", + .type = QEMU_OPT_NUMBER, + }, { + .name = "books", + .type = QEMU_OPT_NUMBER, }, { .name = "sockets", .type = QEMU_OPT_NUMBER,
target/s390x/cpu.c +7 -0
@@ -31,6 +31,7 @@#include "qapi/qapi-types-machine.h" #include "sysemu/hw_accel.h" #include "hw/qdev-properties.h" +#include "hw/qdev-properties-system.h" #include "fpu/softfloat-helpers.h" #include "disas/capstone.h" #include "sysemu/tcg.h"
@@ -292,6 +293,12 @@ static gchar *s390_gdb_arch_name(CPUState *cs)static Property s390x_cpu_properties[] = { #if !defined(CONFIG_USER_ONLY)DEFINE_PROP_UINT32 ("core-id" ,S390CPU , env.core_id,0 ), +DEFINE_PROP_INT32 ("socket-id" ,S390CPU , env.socket_id,-1 ), +DEFINE_PROP_INT32 ("book-id" ,S390CPU , env.book_id,-1 ), +DEFINE_PROP_INT32 ("drawer-id" ,S390CPU , env.drawer_id,-1 ), +DEFINE_PROP_BOOL ("dedicated" ,S390CPU , env.dedicated, false), +DEFINE_PROP_CPUS390ENTITLEMENT ("entitlement" ,S390CPU , env.entitlement, +S390_CPU_ENTITLEMENT_AUTO ), #endifDEFINE_PROP_END_OF_LIST () };
qapi/meson.build +1 -0
@@ -36,6 +36,7 @@ qapi_all_modules = [ 'error', 'introspect', 'job', + 'machine-common', 'machine', 'machine-target', 'migration',
qemu-options.hx +5 -2
@@ -257,11 +257,14 @@ SRST ERST DEF("smp", HAS_ARG, QEMU_OPTION_smp, - "-smp [[cpus=]n][,maxcpus=maxcpus][,sockets=sockets][,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n" + "-smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books][,sockets=sockets]\n" + " [,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n" " set the number of initial CPUs to 'n' [default=1]\n" " maxcpus= maximum number of total CPUs, including\n" " offline CPUs for hotplug, etc\n" - " sockets= number of sockets on the machine board\n" + " drawers= number of drawers on the machine board\n" + " books= number of books in one drawer\n" + " sockets= number of sockets in one book\n" " dies= number of dies in one socket\n" " clusters= number of clusters in one die\n" " cores= number of cores in one cluster\n"
Signature · 1 line
2.31.1
Re: [PATCH v21 01/20] s390x/cpu topology: add s390 specifics to CPU topology
Some notes below.
The s390x/ prefix in the title might suggest that this patch is s390 specific, but it touches common files.
Quoted 3 lines · Pierre Morel
S390 adds two new SMP levels, drawers and books to the CPU topology.
The S390 CPU have specific topology features like dedication
S390 CPUs have specific topology features like dedication and entitlement. These indicate to the guest information on host vCPU scheduling and help the guest make better scheduling decisions.
Quoted 37 lines
and entitlement to give to the guest indications on the host vCPUs scheduling and help the guest take the best decisions on the scheduling of threads on the vCPUs.
Let us provide the SMP properties with books and drawers levels and S390 CPU with dedication and entitlement,
Signed-off-by Pierre Morel <[email protected]>---
qapi/machine-common.json | 22 +++++++++++++ qapi/machine.json | 21 ++++++++++--- include/hw/boards.h | 10 +++++-include/hw/qdev-properties-system.h| 4 +++ target/s390x/cpu.h | 6 ++++ hw/core/machine-smp.c | 48 ++++++++++++++++++++++++--- --
hw/core/machine.c | 4 +++ hw/core/qdev-properties-system.c | 13 ++++++++ hw/s390x/s390-virtio-ccw.c | 2 ++
softmmu/vl.c | 6 ++++ target/s390x/cpu.c | 7 +++++ qapi/meson.build | 1 + qemu-options.hx | 7 +++--13 files changed, 137 insertions(+), 14 deletions(-)create mode 100644
qapi/machine-common.jsonqapi/machine-common.json new file +4
-0@@ -0,0 +1,22 @@ +# -*- Mode: Python -*- +# vim: filetype=python +# +# This work is licensed under the terms of the GNU GPL, version 2 orlater.
qapi/machine-common.json line 8
Quoted 4 lines
qapi/machine-common.json new file +4
-0+# See the COPYING file in the top-level directory. + +## +# = Machines S390 data types
Common definitions for machine.json and machine-target.json
[...]
Re: [PATCH v21 01/20] s390x/cpu topology: add s390 specifics to CPU topology
Quoted 6 lines · Nina Schoetterl-Glausch
Reviewed-by Nina Schoetterl-Glausch <[email protected]>Some notes below.
The s390x/ prefix in the title might suggest that this patch is s390 specific, but it touches common files.
Right.
What do you suggest?
I can cut it in two or squash it with patch number 2.
The first idea was to separate the patch to ease the review but the functionality introduced in patch 1 do only make sense with patch 2.
So I would be for squashing the first two patches.
?
Quoted 54 lines · Pierre Morel
Quoted 3 lines
S390 adds two new SMP levels, drawers and books to the CPU topology.
The S390 CPU have specific topology features like dedicationS390 CPUs have specific topology features like dedication and entitlement. These indicate to the guest information on host vCPU scheduling and help the guest make better scheduling decisions.
Quoted 37 lines
and entitlement to give to the guest indications on the host vCPUs scheduling and help the guest take the best decisions on the scheduling of threads on the vCPUs.
Let us provide the SMP properties with books and drawers levels and S390 CPU with dedication and entitlement,
Signed-off-by Pierre Morel <[email protected]>--- qapi/machine-common.json | 22 +++++++++++++ qapi/machine.json | 21 ++++++++++--- include/hw/boards.h | 10 +++++- include/hw/qdev-properties-system.h | 4 +++ target/s390x/cpu.h | 6 ++++ hw/core/machine-smp.c | 48 ++++++++++++++++++++++++--- -- hw/core/machine.c | 4 +++ hw/core/qdev-properties-system.c | 13 ++++++++ hw/s390x/s390-virtio-ccw.c | 2 ++ softmmu/vl.c | 6 ++++ target/s390x/cpu.c | 7 +++++ qapi/meson.build | 1 + qemu-options.hx | 7 +++-- 13 files changed, 137 insertions(+), 14 deletions(-)create mode 100644
qapi/machine-common.jsonqapi/machine-common.json new file +4
-0@@ -0,0 +1,22 @@ +# -*- Mode: Python -*- +# vim: filetype=python +# +# This work is licensed under the terms of the GNU GPL, version 2 orlater.
qapi/machine-common.json line 8
Quoted 4 lines
qapi/machine-common.json new file +4
-0+# See the COPYING file in the top-level directory. + +## +# = Machines S390 data typesCommon definitions for machine.json and machine-target.json
[...]
Re: [PATCH v21 01/20] s390x/cpu topology: add s390 specifics to CPU topology
Quoted 14 lines · Pierre Morel
Quoted 6 lines · Nina Schoetterl-Glausch
Reviewed-by Nina Schoetterl-Glausch <[email protected]>Some notes below.
The s390x/ prefix in the title might suggest that this patch is s390 specific, but it touches common files.
Right.
What do you suggest?
Quoted 8 lines
I can cut it in two or squash it with patch number 2.
The first idea was to separate the patch to ease the review but the functionality introduced in patch 1 do only make sense with patch 2.
So I would be for squashing the first two patches.
?
Oh, I'd just change the title.
CPU topology: extend with s390 specifics
or similar, it was just a nit not to create the impression that the patch only touches s390 stuff.
Quoted 59 lines
Quoted 59 lines · Pierre Morel
Quoted 3 lines
S390 adds two new SMP levels, drawers and books to the CPU topology.
The S390 CPU have specific topology features like dedicationS390 CPUs have specific topology features like dedication and entitlement. These indicate to the guest information on host vCPU scheduling and help the guest make better scheduling decisions.
Quoted 39 lines
and entitlement to give to the guest indications on the host vCPUs scheduling and help the guest take the best decisions on the scheduling of threads on the vCPUs.
Let us provide the SMP properties with books and drawers levels and S390 CPU with dedication and entitlement,
Signed-off-by Pierre Morel <[email protected]>---
qapi/machine-common.json | 22 +++++++++++++ qapi/machine.json | 21 ++++++++++--- include/hw/boards.h | 10 +++++-include/hw/qdev-properties-system.h| 4 +++ target/s390x/cpu.h | 6 ++++ hw/core/machine-smp.c | 48
++++++++++++++++++++++++---
--
hw/core/machine.c | 4 +++ hw/core/qdev-properties-system.c | 13 ++++++++ hw/s390x/s390-virtio-ccw.c | 2 ++ softmmu/vl.c | 6 ++++ target/s390x/cpu.c | 7 +++++ qapi/meson.build | 1 + qemu-options.hx | 7 +++--13 files changed, 137 insertions(+), 14 deletions(-)create mode 100644
qapi/machine-common.jsonqapi/machine-common.json new file +4
-0@@ -0,0 +1,22 @@ +# -*- Mode: Python -*- +# vim: filetype=python +# +# This work is licensed under the terms of the GNU GPL, version2 or
later.qapi/machine-common.json line 8
Quoted 4 lines
qapi/machine-common.json new file +4
-0+# See the COPYING file in the top-level directory. + +## +# = Machines S390 data typesCommon definitions for machine.json and machine-target.json
[...]
Re: [PATCH v21 01/20] s390x/cpu topology: add s390 specifics to CPU topology
Quoted 20 lines · Nina Schoetterl-Glausch
Quoted 19 lines · Pierre Morel
Quoted 6 lines · Nina Schoetterl-Glausch
Reviewed-by Nina Schoetterl-Glausch <[email protected]>Some notes below.
The s390x/ prefix in the title might suggest that this patch is s390 specific, but it touches common files.
Right.
What do you suggest?
I can cut it in two or squash it with patch number 2.
The first idea was to separate the patch to ease the review but the functionality introduced in patch 1 do only make sense with patch 2.
So I would be for squashing the first two patches.
Quoted 7 lines
Quoted 1 line
?
Oh, I'd just change the title.
CPU topology: extend with s390 specifics
or similar, it was just a nit not to create the impression that the patch only touches s390 stuff.
OK thanks
Pierre
PATCH v21 2/20 s390x/cpu topology: add topology entries on CPU hotplug
In reply to Pierre MorelThe topology information are attributes of the CPU and are specified during the CPU device creation.
On hot plug we:
- calculate the default values for the topology for drawers, books and sockets in the case they are not specified.
- verify the CPU attributes
- check that we have still room on the desired socket
The possibility to insert a CPU in a mask is dependent on the number of cores allowed in a socket, a book or a drawer, the checking is done during the hot plug of the CPU to have an immediate answer.
If the complete topology is not specified, the core is added in the physical topology based on its core ID and it gets defaults values for the modifier attributes.
This way, starting QEMU without specifying the topology can still get some advantage of the CPU topology.
--- MAINTAINERS | 6 + include/hw/s390x/cpu-topology.h | 54 +++++++ hw/s390x/cpu-topology.c | 264 ++++++++++++++++++++++++++++++++ hw/s390x/s390-virtio-ccw.c | 22 ++- hw/s390x/meson.build | 1 + 5 files changed, 345 insertions(+), 2 deletions(-) create mode 100644 include/hw/s390x/cpu-topology.h create mode 100644 hw/s390x/cpu-topology.c
MAINTAINERS +6 -0
@@ -1697,6 +1697,12 @@ F: hw/s390x/event-facility.c F: hw/s390x/sclp*.c L: [email protected] +S390 CPU topology +M: Pierre Morel <[email protected]> +S: Supported +F: include/hw/s390x/cpu-topology.h +F: hw/s390x/cpu-topology.c + X86 Machines ------------ PC
include/hw/s390x/cpu-topology.h new file +54 -0
54 lines not on this page · raw message
hw/s390x/cpu-topology.c new file +264 -0
264 lines not on this page · raw message
hw/s390x/s390-virtio-ccw.c +20 -2
@@ -45,6 +45,7 @@#include "hw/s390x/pv.h" #include "migration/blocker.h" #include "qapi/visitor.h" +#include "hw/s390x/cpu-topology.h" static Error *pv_mig_blocker;
@@ -312,10 +313,18 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev, {MachineState *ms =MACHINE (hotplug_dev);S390CPU *cpu =S390_CPU (dev); +ERRP_GUARD ();g_assert (!ms->possible_cpus->cpus[cpu->env.core_id].cpu); ms->possible_cpus->cpus[cpu->env.core_id].cpu =OBJECT (dev); +if (s390_has_topology ()) { +s390_topology_setup_cpu (ms, cpu, errp); +if (*errp) { +return ; + } + } +if (dev->hotplugged) {raise_irq_cpu_hotplug (); }
@@ -555,11 +564,20 @@ static const CPUArchIdList *s390_possible_cpu_arch_ids(MachineState *ms) sizeof(CPUArchId) * max_cpus); ms->possible_cpus->len = max_cpus;for (i =0 ; i < ms->possible_cpus->len; i++) { +CpuInstanceProperties *props = &ms->possible_cpus->cpus[i].props; + ms->possible_cpus->cpus[i].type = ms->cpu_type; ms->possible_cpus->cpus[i].vcpus_count =1 ; ms->possible_cpus->cpus[i].arch_id = i; - ms->possible_cpus->cpus[i].props.has_core_id = true; - ms->possible_cpus->cpus[i].props.core_id = i; + + props->has_core_id = true; + props->core_id = i; + props->has_socket_id = true; + props->socket_id =s390_std_socket (i, &ms->smp); + props->has_book_id = true; + props->book_id =s390_std_book (i, &ms->smp); + props->has_drawer_id = true; + props->drawer_id =s390_std_drawer (i, &ms->smp); }return ms->possible_cpus;
hw/s390x/meson.build +1 -0
@@ -24,6 +24,7 @@ s390x_ss.add(when: 'CONFIG_KVM', if_true: files( 's390-stattrib-kvm.c', 'pv.c', 's390-pci-kvm.c', + 'cpu-topology.c', )) s390x_ss.add(when: 'CONFIG_TCG', if_true: files( 'tod-tcg.c',
Signature · 1 line
2.31.1
Re: [PATCH v21 02/20] s390x/cpu topology: add topology entries on CPU hotplug
Quoted 23 lines · Pierre Morel
The topology information are attributes of the CPU and are specified during the CPU device creation.
On hot plug we:
- calculate the default values for the topology for drawers, books and sockets in the case they are not specified.
- verify the CPU attributes
- check that we have still room on the desired socketThe possibility to insert a CPU in a mask is dependent on the number of cores allowed in a socket, a book or a drawer, the checking is done during the hot plug of the CPU to have an immediate answer.
If the complete topology is not specified, the core is added in the physical topology based on its core ID and it gets defaults values for the modifier attributes.
This way, starting QEMU without specifying the topology can still get some advantage of the CPU topology.
Signed-off-by Pierre Morel <[email protected]>---
...
include/hw/s390x/cpu-topology.h line 5
Quoted 11 lines
include/hw/s390x/cpu-topology.h new file +5
-0@@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* CPU Topology +* +* Copyright IBM Corp. 2022,2023
Nit: Add a space after the comma ?
Quoted 5 lines
include/hw/s390x/cpu-topology.h new file +5
-0+* Author(s): Pierre Morel <[email protected]> +* +*/ +#ifndef HW_S390X_CPU_TOPOLOGY_H +#define HW_S390X_CPU_TOPOLOGY_H
include/hw/s390x/cpu-topology.h line 20
Quoted 10 lines
include/hw/s390x/cpu-topology.h new file +10
-0+ +#ifndef CONFIG_USER_ONLY + +#include "qemu/queue.h" +#include "hw/boards.h" +#include "qapi/qapi-types-machine-target.h" + +typedef struct S390Topology { +uint8_t *cores_per_socket; +}S390Topology ;
So S390Topology has only one entry, "cores_per_socket" here...
...
Quoted 28 lines
hw/s390x/cpu-topology.c new file +22
-0@@ -0,0 +1,264 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* CPU Topology +* +* Copyright IBM Corp. 2022,2023 +* Author(s): Pierre Morel <[email protected]> +* +* S390 topology handling can be divided in two parts: +* +* - The first part in this file is taking care of all common functions +* used by KVM and TCG to create and modify the topology. +* +* - The second part, building the topology information data for the +* guest with CPU and KVM specificity will be implemented inside +* the target/s390/kvm sub tree. +*/ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "hw/qdev-properties.h" +#include "hw/boards.h"
hw/s390x/cpu-topology.c line 32
Quoted 10 lines
hw/s390x/cpu-topology.c new file +10
-0+#include "target/s390x/cpu.h" +#include "hw/s390x/s390-virtio-ccw.h" +#include "hw/s390x/cpu-topology.h" + +/* +* s390_topology is used to keep the topology information. +* .cores_per_socket: tracks information on the count of cores +* per socket. +* .smp: keeps track of the machine topology. +*
... but the description talks about ".smp" here, too, which never seems to be added. Leftover from a previous iteration? (also: please remove the empty line at the end of the comment)
Quoted 4 lines
hw/s390x/cpu-topology.c new file +4
-0+*/ +S390Topology s390_topology = { +/* will be initialized after the cpu model is realized */ + .cores_per_socket =NULL ,
hw/s390x/cpu-topology.c line 46
Quoted 10 lines
hw/s390x/cpu-topology.c new file +10
-0+}; + +/** +* s390_socket_nb: +* @cpu: s390x CPU +* +* Returns the socket number used inside the cores_per_socket array +* for a topology tree entry +*/ +static int __s390_socket_nb (int drawer_id,int book_id,int socket_id)
Please don't use function names starting with double underscores. This namespace is reserved by the C standard. Maybe "s390_socket_nb_from_ids" ?
Quoted 12 lines
hw/s390x/cpu-topology.c new file +12
-0+{ +return (drawer_id * current_machine->smp.books + book_id) * + current_machine->smp.sockets + socket_id; +} + +/** +* s390_socket_nb: +* @cpu: s390x CPU +* +* Returns the socket number used inside the cores_per_socket array +* for a cpu. +*/
hw/s390x/cpu-topology.c line 68
Quoted 10 lines
hw/s390x/cpu-topology.c new file +10
-0+static int s390_socket_nb (S390CPU *cpu) +{ +return __s390_socket_nb (cpu->env.drawer_id, cpu->env.book_id, + cpu->env.socket_id); +} + +/** +* s390_has_topology: +* +* Return value: if the topology is supported by the machine.
"Return: true if the topology is supported by the machine"
(QEMU uses kerneldoc style, so it's just "Return:" and not "Return value:", see e.g. https://www.kernel.org/doc/html/latest/doc-guide/kernel-doc.html )
Quoted 58 lines
hw/s390x/cpu-topology.c new file +58
-0+*/ +bool s390_has_topology (void ) +{ +return false; +} + +/** +* s390_topology_init: +* @ms: the machine state where the machine topology is defined +* +* Keep track of the machine topology. +* +* Allocate an array to keep the count of cores per socket. +* The index of the array starts at socket 0 from book 0 and +* drawer 0 up to the maximum allowed by the machine topology. +*/ +static void s390_topology_init (MachineState *ms) +{ +CpuTopology *smp = &ms->smp; + + s390_topology.cores_per_socket =g_new0 (uint8_t, smp->sockets * + smp->books * smp->drawers); +} + +/** +* s390_topology_cpu_default: +* @cpu: pointer to a S390CPU +* @errp: Error pointer +* +* Setup the default topology if no attributes are already set. +* Passing a CPU with some, but not all, attributes set is considered +* an error. +* +* The function calculates the (drawer_id, book_id, socket_id) +* topology by filling the cores starting from the first socket +* (0, 0, 0) up to the last (smp->drawers, smp->books, smp->sockets). +* +* CPU type and dedication have defaults values set in the +* s390x_cpu_properties, entitlement must be adjust depending on the +* dedication. +* +* Returns false if it is impossible to setup a default topology +* true otherwise. +*/ +static bool s390_topology_cpu_default (S390CPU *cpu,Error **errp) +{ +CpuTopology *smp = ¤t_machine->smp; +CPUS390XState *env = &cpu->env; + +/* All geometry topology attributes must be set or all unset */ +if ((env->socket_id <0 || env->book_id <0 || env->drawer_id <0 ) && + (env->socket_id >=0 || env->book_id >=0 || env->drawer_id >=0 )) { +error_setg (errp, +"Please define all or none of the topology geometry attributes" ); +return false; + } + +/* Check if one of the geometry topology is unset */
hw/s390x/cpu-topology.c line 136
Quoted 10 lines
hw/s390x/cpu-topology.c new file +10
-0+if (env->socket_id <0 ) { +/* Calculate default geometry topology attributes */ + env->socket_id =s390_std_socket (env->core_id, smp); + env->book_id =s390_std_book (env->core_id, smp); + env->drawer_id =s390_std_drawer (env->core_id, smp); + } + +/* +* When the user specifies the entitlement as 'auto' on the command line, +* qemu will set the entitlement as:
s/qemu/QEMU/
Quoted 44 lines
hw/s390x/cpu-topology.c new file +44
-0+* Medium when the CPU is not dedicated. +* High when dedicated is true. +*/ +if (env->entitlement ==S390_CPU_ENTITLEMENT_AUTO ) { +if (env->dedicated) { + env->entitlement =S390_CPU_ENTITLEMENT_HIGH ; + }else { + env->entitlement =S390_CPU_ENTITLEMENT_MEDIUM ; + } + } +return true; +} + +/** +* s390_topology_check: +* @socket_id: socket to check +* @book_id: book to check +* @drawer_id: drawer to check +* @entitlement: entitlement to check +* @dedicated: dedication to check +* @errp: Error pointer +* +* The function checks if the topology +* attributes fits inside the system topology. +* +* Returns false if the specified topology does not match with +* the machine topology. +*/ +static bool s390_topology_check (uint16_t socket_id,uint16_t book_id, +uint16_t drawer_id,uint16_t entitlement, +bool dedicated,Error **errp) +{ +CpuTopology *smp = ¤t_machine->smp; +ERRP_GUARD (); + +if (socket_id >= smp->sockets) { +error_setg (errp,"Unavailable socket: %d" , socket_id); +return false; + } +if (book_id >= smp->books) { +error_setg (errp,"Unavailable book: %d" , book_id); +return false; + } +if (drawer_id >= smp->drawers) {
hw/s390x/cpu-topology.c line 190
Quoted 10 lines
hw/s390x/cpu-topology.c new file +10
-0+error_setg (errp,"Unavailable drawer: %d" , drawer_id); +return false; + } +if (entitlement >=S390_CPU_ENTITLEMENT__MAX ) { +error_setg (errp,"Unknown entitlement: %d" , entitlement); +return false; + } +if (dedicated && (entitlement ==S390_CPU_ENTITLEMENT_LOW || + entitlement ==S390_CPU_ENTITLEMENT_MEDIUM )) { +error_setg (errp,"A dedicated cpu implies high entitlement" );
s/cpu/CPU/ ?
Quoted 31 lines
hw/s390x/cpu-topology.c new file +31
-0+return false; + } +return true; +} + +/** +* s390_update_cpu_props: +* @ms: the machine state +* @cpu: the CPU for which to update the properties from the environment. +* +*/ +static void s390_update_cpu_props (MachineState *ms,S390CPU *cpu) +{ +CpuInstanceProperties *props; + + props = &ms->possible_cpus->cpus[cpu->env.core_id].props; + + props->socket_id = cpu->env.socket_id; + props->book_id = cpu->env.book_id; + props->drawer_id = cpu->env.drawer_id; +} + +/** +* s390_topology_setup_cpu: +* @ms: MachineState used to initialize the topology structure on +* first call. +* @cpu: the new S390CPU to insert in the topology structure +* @errp: the error pointer +* +* Called from CPU hotplug to check and setup the CPU attributes +* before the CPU is inserted in the topology.
hw/s390x/cpu-topology.c line 231
Quoted 10 lines
hw/s390x/cpu-topology.c new file +10
-0+* There is no need to update the MTCR explicitly here because it +* will be updated by KVM on creation of the new CPU. +*/ +void s390_topology_setup_cpu (MachineState *ms,S390CPU *cpu,Error **errp) +{ +ERRP_GUARD (); +int entry; + +/* +* We do not want to initialize the topology if the cpu model
s/cpu/CPU/
Quoted 23 lines
hw/s390x/cpu-topology.c new file +23
-0+* does not support topology, consequently, we have to wait for +* the first CPU to be realized, which realizes the CPU model +* to initialize the topology structures. +* +* s390_topology_setup_cpu() is called from the cpu hotplug. +*/ +if (!s390_topology.cores_per_socket) { +s390_topology_init (ms); + } + +if (!s390_topology_cpu_default (cpu, errp)) { +return ; + } + +if (!s390_topology_check (cpu->env.socket_id, cpu->env.book_id, + cpu->env.drawer_id, cpu->env.entitlement, + cpu->env.dedicated, errp)) { +return ; + } + +/* Do we still have space in the socket */ + entry =s390_socket_nb (cpu); +if (s390_topology.cores_per_socket[entry] >= current_machine->smp.cores) {
hw/s390x/cpu-topology.c line 264
Quoted 10 lines
hw/s390x/cpu-topology.c new file +10
-0+error_setg (errp,"No more space on this socket" ); +return ; + } + +/* Update the count of cores in sockets */ + s390_topology.cores_per_socket[entry] +=1 ; + +/* topology tree is reflected in props */ +s390_update_cpu_props (ms, cpu); +}
Thomas
Re: [PATCH v21 02/20] s390x/cpu topology: add topology entries on CPU hotplug
Quoted 25 lines · Thomas Huth
Quoted 23 lines · Pierre Morel
The topology information are attributes of the CPU and are specified during the CPU device creation.
On hot plug we:
- calculate the default values for the topology for drawers, books and sockets in the case they are not specified.
- verify the CPU attributes
- check that we have still room on the desired socketThe possibility to insert a CPU in a mask is dependent on the number of cores allowed in a socket, a book or a drawer, the checking is done during the hot plug of the CPU to have an immediate answer.
If the complete topology is not specified, the core is added in the physical topology based on its core ID and it gets defaults values for the modifier attributes.
This way, starting QEMU without specifying the topology can still get some advantage of the CPU topology.
Signed-off-by Pierre Morel <[email protected]>---
...
Quoted 13 lines
include/hw/s390x/cpu-topology.h line 5
Quoted 11 lines
include/hw/s390x/cpu-topology.h new file +5
-0@@ -0,0 +1,54 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* CPU Topology +* +* Copyright IBM Corp. 2022,2023 Nit: Add a space after the comma ?
thx
Quoted 53 lines
Quoted 6 lines
include/hw/s390x/cpu-topology.h new file +5
-0+* Author(s): Pierre Morel <[email protected]> +* +*/ +#ifndef HW_S390X_CPU_TOPOLOGY_H +#define HW_S390X_CPU_TOPOLOGY_H include/hw/s390x/cpu-topology.h line 20
Quoted 10 lines
include/hw/s390x/cpu-topology.h new file +10
-0+ +#ifndef CONFIG_USER_ONLY + +#include "qemu/queue.h" +#include "hw/boards.h" +#include "qapi/qapi-types-machine-target.h" + +typedef struct S390Topology { + uint8_t *cores_per_socket; +}S390Topology ;So S390Topology has only one entry, "
cores_per_socket" here......
Quoted 33 lines
hw/s390x/cpu-topology.c new file +27
-0@@ -0,0 +1,264 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* CPU Topology +* +* Copyright IBM Corp. 2022,2023 +* Author(s): Pierre Morel <[email protected]> +* +* S390 topology handling can be divided in two parts: +* +* - The first part in this file is taking care of all common functions +* used by KVM and TCG to create and modify the topology. +* +* - The second part, building the topology information data for the +* guest with CPU and KVM specificity will be implemented inside +* the target/s390/kvm sub tree. +*/ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "hw/qdev-properties.h" +#include "hw/boards.h" +#include "target/s390x/cpu.h" +#include "hw/s390x/s390-virtio-ccw.h" +#include "hw/s390x/cpu-topology.h" + +/*
Quoted 8 lines
hw/s390x/cpu-topology.c line 32
Quoted 5 lines
hw/s390x/cpu-topology.c new file +5
-0+* s390_topology is used to keep the topology information. +* .cores_per_socket: tracks information on the count of cores +* per socket. +* .smp: keeps track of the machine topology. +* ... but the description talks about ".smp" here, too, which never seems to be added. Leftover from a previous iteration? (also: please remove the empty line at the end of the comment)
right, thx
Quoted 10 lines
Quoted 10 lines
hw/s390x/cpu-topology.c new file +9
-0+*/ +S390Topology s390_topology = { + /* will be initialized after the cpu model is realized */ + .cores_per_socket = NULL, +}; + +/** +* s390_socket_nb: +* @cpu: s390x CPU
Quoted 8 lines
hw/s390x/cpu-topology.c line 46
Quoted 5 lines
hw/s390x/cpu-topology.c new file +5
-0+* +* Returns the socket number used inside the cores_per_socket array +* for a topology tree entry +*/ +static int __s390_socket_nb (int drawer_id,int book_id,int socket_id)Please don't use function names starting with double underscores. This namespace is reserved by the C standard. Maybe "
s390_socket_nb_from_ids" ?
OK
Quoted 26 lines
Quoted 13 lines
hw/s390x/cpu-topology.c new file +12
-0+{ + return (drawer_id * current_machine->smp.books + book_id) * + current_machine->smp.sockets + socket_id; +} + +/** +* s390_socket_nb: +* @cpu: s390x CPU +* +* Returns the socket number used inside the cores_per_socket array +* for a cpu. +*/ hw/s390x/cpu-topology.c line 68
Quoted 10 lines
hw/s390x/cpu-topology.c new file +10
-0+static int s390_socket_nb (S390CPU *cpu) +{ + return __s390_socket_nb(cpu->env.drawer_id, cpu->env.book_id, + cpu->env.socket_id); +} + +/** +* s390_has_topology: +* +* Return value: if the topology is supported by the machine. "Return: true if the topology is supported by the machine"
Quoted 1 line
(QEMU uses kerneldoc style, so it's just "Return:" and not "Return value:", see e.g. https://www.kernel.org/doc/html/latest/doc-guide/kernel-doc.html )
OK, thx
Quoted 67 lines
Quoted 67 lines
hw/s390x/cpu-topology.c new file +51
-0+*/ +bool s390_has_topology (void ) +{ + return false; +} + +/** +* s390_topology_init: +* @ms: the machine state where the machine topology is defined +* +* Keep track of the machine topology. +* +* Allocate an array to keep the count of cores per socket. +* The index of the array starts at socket 0 from book 0 and +* drawer 0 up to the maximum allowed by the machine topology. +*/ +static void s390_topology_init (MachineState *ms) +{ + CpuTopology *smp = &ms->smp; + + s390_topology.cores_per_socket = g_new0(uint8_t, smp->sockets * + smp->books * smp->drawers); +} + +/** +* s390_topology_cpu_default: +* @cpu: pointer to a S390CPU +* @errp: Error pointer +* +* Setup the default topology if no attributes are already set. +* Passing a CPU with some, but not all, attributes set is considered +* an error. +* +* The function calculates the (drawer_id, book_id, socket_id) +* topology by filling the cores starting from the first socket +* (0, 0, 0) up to the last (smp->drawers, smp->books, smp->sockets). +* +* CPU type and dedication have defaults values set in the +* s390x_cpu_properties, entitlement must be adjust depending on the +* dedication. +* +* Returns false if it is impossible to setup a default topology +* true otherwise. +*/ +static bool s390_topology_cpu_default(S390CPU *cpu, Error **errp) +{ + CpuTopology *smp = ¤t_machine->smp; + CPUS390XState *env = &cpu->env; + + /* All geometry topology attributes must be set or all unset */ + if ((env->socket_id < 0 || env->book_id < 0 || env->drawer_id <0) &&
+ (env->socket_id>= 0 ||env->book_id>= 0 ||env->drawer_idQuoted 1 line
= 0)) {
+ error_setg(errp,
+ "Please define all or none of the topology geometry attributes");
+ return false;
+ }
+
+ /* Check if one of the geometry topology is unset */ + if (env->socket_id< 0) {
+ /* Calculate default geometry topology attributes */ +env->socket_id=s390_std_socket(env->core_id, smp); +env->book_id=s390_std_book(env->core_id, smp); +env->drawer_id=s390_std_drawer(env->core_id, smp);
Quoted 8 lines
Quoted 6 lines
+ }
+
+ /*
+ * When the user specifies the entitlement as 'auto' on the command line,
+ * qemu will set the entitlement as:s/qemu/QEMU/
ok
Quoted 50 lines
Quoted 50 lines
+ * Medium when the CPU is not dedicated.
+ * High when dedicated is true.
+ */
+ if (env->entitlement==S390_CPU_ENTITLEMENT_AUTO) { + if (env->dedicated) {
+env->entitlement=S390_CPU_ENTITLEMENT_HIGH; + } else {
+env->entitlement=S390_CPU_ENTITLEMENT_MEDIUM; + }
+ }
+ return true;hw/s390x/cpu-topology.c new file +19
-0+} + +/** +* s390_topology_check: +* @socket_id: socket to check +* @book_id: book to check +* @drawer_id: drawer to check +* @entitlement: entitlement to check +* @dedicated: dedication to check +* @errp: Error pointer +* +* The function checks if the topology +* attributes fits inside the system topology. +* +* Returns false if the specified topology does not match with +* the machine topology. +*/ +static bool s390_topology_check (uint16_t socket_id,uint16_t book_id, + uint16_t drawer_id, uint16_tentitlement,
+ bool dedicated, Error **errp)hw/s390x/cpu-topology.c new file +17
-0+{ + CpuTopology *smp = ¤t_machine->smp; + ERRP_GUARD(); + + if (socket_id >= smp->sockets) { + error_setg(errp, "Unavailable socket: %d", socket_id); + return false; + } + if (book_id >= smp->books) { + error_setg(errp, "Unavailable book: %d", book_id); + return false; + } + if (drawer_id >= smp->drawers) { + error_setg(errp, "Unavailable drawer: %d", drawer_id); + return false; + } + if (entitlement >= S390_CPU_ENTITLEMENT__MAX) {
Quoted 8 lines
Quoted 6 lines
+
error_setg(errp, "Unknown entitlement: %d", entitlement); + return false;
+ }
+ if (dedicated && (entitlement ==S390_CPU_ENTITLEMENT_LOW|| + entitlement ==S390_CPU_ENTITLEMENT_MEDIUM)) { +error_setg(errp, "A dedicated cpu implies high entitlement");s/cpu/CPU/ ?
yes OK
Quoted 38 lines
Quoted 38 lines
+ return false;
+ }
+ return true;hw/s390x/cpu-topology.c new file +6
-0+} + +/** +* s390_update_cpu_props: +* @ms: the machine state +* @cpu: the CPU for which to update the properties from the environment.
+ *
+ */hw/s390x/cpu-topology.c new file +24
-0+static void s390_update_cpu_props (MachineState *ms,S390CPU *cpu) +{ + CpuInstanceProperties *props; + + props = &ms->possible_cpus->cpus[cpu->env.core_id].props; + + props->socket_id = cpu->env.socket_id; + props->book_id = cpu->env.book_id; + props->drawer_id = cpu->env.drawer_id; +} + +/** +* s390_topology_setup_cpu: +* @ms: MachineState used to initialize the topology structure on +* first call. +* @cpu: the new S390CPU to insert in the topology structure +* @errp: the error pointer +* +* Called from CPU hotplug to check and setup the CPU attributes +* before the CPU is inserted in the topology. +* There is no need to update the MTCR explicitly here because it +* will be updated by KVM on creation of the new CPU. +*/ +void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error**errp)
Quoted 8 lines
hw/s390x/cpu-topology.c line 231
Quoted 6 lines
hw/s390x/cpu-topology.c new file +6
-0+{ + ERRP_GUARD(); + int entry; + + /* +* We do not want to initialize the topology if the cpu model s/cpu/CPU/
ok
Quoted 29 lines
Quoted 29 lines
+ * does not support topology, consequently, we have to wait for + * the first CPU to be realized, which realizes the CPU model + * to initialize the topology structures.
+ *
+ *s390_topology_setup_cpu()is called from the cpu hotplug. + */
+ if (!s390_topology.cores_per_socket) {
+s390_topology_init(ms);
+ }hw/s390x/cpu-topology.c new file +14
-0+ + if (!s390_topology_cpu_default(cpu, errp)) { + return; + } + + if (!s390_topology_check(cpu->env.socket_id, cpu->env.book_id, + cpu->env.drawer_id, cpu->env.entitlement, + cpu->env.dedicated, errp)) { + return; + } + + /* Do we still have space in the socket */ + entry = s390_socket_nb(cpu); + if (s390_topology.cores_per_socket[entry] >=
current_machine->smp.cores) {
+error_setg(errp, "No more space on this socket"); + return;
+ }
+
Quoted 10 lines
hw/s390x/cpu-topology.c line 264
Quoted 6 lines
+ /* Update the count of cores in sockets */
+ s390_topology.cores_per_socket[entry] += 1;hw/s390x/cpu-topology.c new file +4
-0+ + /* topology tree is reflected in props */ + s390_update_cpu_props(ms, cpu); +}Thomas
Thanks, I make the changes.
Regards,
Pierre
Re: [PATCH v21 02/20] s390x/cpu topology: add topology entries on CPU hotplug
In reply to Pierre MorelQuoted 14 lines · Pierre Morel
The topology information are attributes of the CPU and are specified during the CPU device creation.
On hot plug we:
- calculate the default values for the topology for drawers, books and sockets in the case they are not specified.
- verify the CPU attributes
- check that we have still room on the desired socketThe possibility to insert a CPU in a mask is dependent on the number of cores allowed in a socket, a book or a drawer, the checking is done during the hot plug of the CPU to have an immediate answer.
Quoted 8 lines
If the complete topology is not specified, the core is added in the physical topology based on its core ID and it gets defaults values for the modifier attributes.
This way, starting QEMU without specifying the topology can still get some advantage of the CPU topology.
Signed-off-by Pierre Morel <[email protected]>
Thomas' comments.
Re: [PATCH v21 02/20] s390x/cpu topology: add topology entries on CPU hotplug
Quoted 19 lines · Nina Schoetterl-Glausch
Quoted 18 lines · Pierre Morel
The topology information are attributes of the CPU and are specified during the CPU device creation.
On hot plug we:
- calculate the default values for the topology for drawers, books and sockets in the case they are not specified.
- verify the CPU attributes
- check that we have still room on the desired socketThe possibility to insert a CPU in a mask is dependent on the number of cores allowed in a socket, a book or a drawer, the checking is done during the hot plug of the CPU to have an immediate answer.
If the complete topology is not specified, the core is added in the physical topology based on its core ID and it gets defaults values for the modifier attributes.
Quoted 7 lines
Quoted 4 lines
This way, starting QEMU without specifying the topology can still get some advantage of the CPU topology.
Signed-off-by Pierre Morel <[email protected]>Thomas' comments.
Thanks,
regards
Pierre
PATCH v21 3/20 target/s390x/cpu topology: handle STSI(15) and build the SYSIB
In reply to Pierre MorelOn interception of STSI(15.1.x) the System Information Block (SYSIB) is built from the list of pre-ordered topology entries.
--- MAINTAINERS | 1 + qapi/machine-target.json | 14 ++ include/hw/s390x/cpu-topology.h | 25 +++ include/hw/s390x/sclp.h | 1 + target/s390x/cpu.h | 76 ++++++++ hw/s390x/cpu-topology.c | 4 +- target/s390x/kvm/kvm.c | 5 +- target/s390x/kvm/stsi-topology.c | 310 +++++++++++++++++++++++++++++++ target/s390x/kvm/meson.build | 3 +- 9 files changed, 436 insertions(+), 3 deletions(-) create mode 100644 target/s390x/kvm/stsi-topology.c
MAINTAINERS +1 -0
@@ -1702,6 +1702,7 @@ M: Pierre Morel <[email protected]> S: Supported F: include/hw/s390x/cpu-topology.h F: hw/s390x/cpu-topology.c +F: target/s390x/kvm/stsi-topology.c X86 Machines ------------
qapi/machine-target.json +14 -0
@@ -361,3 +361,17 @@ 'TARGET_MIPS', 'TARGET_LOONGARCH64', 'TARGET_RISCV' ] } } + +## +# @CpuS390Polarization: +# +# An enumeration of cpu polarization that can be assumed by a virtual +# S390 CPU +# +# Since: 8.1 +## +{ 'enum': 'CpuS390Polarization', + 'prefix': 'S390_CPU_POLARIZATION', + 'data': [ 'horizontal', 'vertical' ], + 'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] } +}
include/hw/s390x/cpu-topology.h +25 -0
@@ -15,10 +15,35 @@#include "hw/boards.h" #include "qapi/qapi-types-machine-target.h" +#define S390_TOPOLOGY_CPU_IFL 0x03 + +typedef union s390_topology_id { +uint64_t id; +struct { +uint8_t sentinel; +uint8_t drawer; +uint8_t book; +uint8_t socket; +uint8_t dedicated; +uint8_t entitlement; +uint8_t type; +uint8_t origin; + }; +}s390_topology_id ; + +typedef struct S390TopologyEntry { +QTAILQ_ENTRY (S390TopologyEntry) next; +s390_topology_id id; +uint64_t mask; +}S390TopologyEntry ; +typedef struct S390Topology {uint8_t *cores_per_socket; +bool polarization; }S390Topology ; +typedef QTAILQ_HEAD(, S390TopologyEntry) S390TopologyList; +#ifdef CONFIG_KVM bool s390_has_topology (void );void s390_topology_setup_cpu (MachineState *ms,S390CPU *cpu,Error **errp);
include/hw/s390x/sclp.h +1 -0
@@ -112,6 +112,7 @@ typedef struct CPUEntry { }QEMU_PACKED CPUEntry;#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128 +#define SCLP_READ_SCP_INFO_MNEST 2 typedef struct ReadInfo { SCCBHeader h; uint16_t rnmax;
target/s390x/cpu.h +76 -0
91 lines not on this page · raw message
hw/s390x/cpu-topology.c +3 -1
@@ -29,11 +29,13 @@* .cores_per_socket: tracks information on the count of cores * per socket. * .smp: keeps track of the machine topology. -* +* .list: queue the topology entries inside which +* we keep the information on the CPU topology. */ S390Topology s390_topology = {/* will be initialized after the cpu model is realized */ .cores_per_socket =NULL , + .polarization =S390_CPU_POLARIZATION_HORIZONTAL , };/**
target/s390x/kvm/kvm.c +4 -1
@@ -1919,9 +1919,12 @@ static int handle_stsi(S390CPU *cpu)if (run->s390_stsi.sel1 !=2 || run->s390_stsi.sel2 !=2 ) {return 0 ; } -/* Only sysib 3.2.2 needs post-handling for now. */ insert_stsi_3_2_2 (cpu, run->s390_stsi.addr, run->s390_stsi.ar);return 0 ; +case 15 : +insert_stsi_15_1_x (cpu, run->s390_stsi.sel2, run->s390_stsi.addr, + run->s390_stsi.ar); +return 0 ;default :return 0 ; }
target/s390x/kvm/stsi-topology.c new file +310 -0
310 lines not on this page · raw message
target/s390x/kvm/meson.build +2 -1
@@ -1,6 +1,7 @@ s390x_ss.add(when: 'CONFIG_KVM', if_true: files( - 'kvm.c' + 'kvm.c', + 'stsi-topology.c' ), if_false: files( 'stubs.c' ))
Signature · 1 line
2.31.1
Re: [PATCH v21 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
Quoted 5 lines · Pierre Morel
On interception of STSI(15.1.x) the System Information Block (SYSIB) is built from the list of pre-ordered topology entries.
Signed-off-by Pierre Morel <[email protected]>---
...
Quoted 67 lines
target/s390x/cpu.h +49
-0@@ -569,6 +569,29 @@ typedef struct SysIB_322 { } SysIB_322;QEMU_BUILD_BUG_ON (sizeof (SysIB_322) !=4096 ); +/* +* Topology Magnitude fields (MAG) indicates the maximum number of +* topology list entries (TLE) at the corresponding nesting level. +*/ +#define S390_TOPOLOGY_MAG 6 +#define S390_TOPOLOGY_MAG6 0 +#define S390_TOPOLOGY_MAG5 1 +#define S390_TOPOLOGY_MAG4 2 +#define S390_TOPOLOGY_MAG3 3 +#define S390_TOPOLOGY_MAG2 4 +#define S390_TOPOLOGY_MAG1 5 +/* Configuration topology */ +typedef struct SysIB_151x { +uint8_t reserved0[2 ]; +uint16_t length; +uint8_t mag[S390_TOPOLOGY_MAG ]; +uint8_t reserved1; +uint8_t mnest; +uint32_t reserved2; +char tle[]; +}SysIB_151x ; +QEMU_BUILD_BUG_ON (sizeof (SysIB_151x) !=16 ); + typedef union SysIB { SysIB_111 sysib_111; SysIB_121 sysib_121;@@ -576,9 +599,62 @@ typedef union SysIB {SysIB_221 sysib_221;SysIB_222 sysib_222;SysIB_322 sysib_322; +SysIB_151x sysib_151x; } SysIB;QEMU_BUILD_BUG_ON (sizeof (SysIB) !=4096 ); +/* +* CPU Topology List provided by STSI with fc=15 provides a list +* of two different Topology List Entries (TLE) types to specify +* the topology hierarchy. +* +* - Container Topology List Entry +* Defines a container to contain other Topology List Entries +* of any type, nested containers or CPU. +* - CPU Topology List Entry +* Specifies the CPUs position, type, entitlement and polarization +* of the CPUs contained in the last Container TLE. +* +* There can be theoretically up to five levels of containers, QEMU +* uses only three levels, the drawer's, book's and socket's level. +* +* A container with a nesting level (NL) greater than 1 can only +* contain another container of nesting level NL-1. +* +* A container of nesting level 1 (socket), contains as many CPU TLE +* as needed to describe the position and qualities of all CPUs inside +* the container. +* The qualities of a CPU are polarization, entitlement and type. +* +* The CPU TLE defines the position of the CPUs of identical qualities +* using a 64bits mask which first bit has its offset defined by
target/s390x/cpu.h typedef union SysIB { · line 640
Quoted 10 lines
target/s390x/cpu.h +10
-0+* the CPU address orgin field of the CPU TLE like in: +* CPU address = origin * 64 + bit position within the mask +* +*/ +/* Container type Topology List Entry */ +typedef struct SysIBTl_container { +uint8_t nl; +uint8_t reserved[6 ]; +uint8_t id; +}SysIBTl_container ;
Why mixing CamelCase with underscore-style here? SysIBTlContainer would look more natural, I think?
target/s390x/cpu.h typedef union SysIB { · line 653
Quoted 13 lines
target/s390x/cpu.h +13
-0+QEMU_BUILD_BUG_ON (sizeof (SysIBTl_container) !=8 ); + +/* CPU type Topology List Entry */ +typedef struct SysIBTl_cpu { +uint8_t nl; +uint8_t reserved0[3 ]; +#define SYSIB_TLE_POLARITY_MASK 0x03 +#define SYSIB_TLE_DEDICATED 0x04 +uint8_t flags; +uint8_t type; +uint16_t origin; +uint64_t mask; +}SysIBTl_cpu ;
dito, maybe better SysIBTlCpu ?
Quoted 12 lines
target/s390x/cpu.h +4
-0+QEMU_BUILD_BUG_ON (sizeof (SysIBTl_cpu) !=16 ); + +void insert_stsi_15_1_x (S390CPU *cpu,int sel2,uint64_t addr,uint8_t ar); +/* MMU defines */ #define ASCE_ORIGIN (~0xfffULL)/* segment table origin */ #define ASCE_SUBSPACE 0x200/* subspace group control */ hw/s390x/cpu-topology.c +0
-0
hw/s390x/cpu-topology.c line 33
Quoted 6 lines
hw/s390x/cpu-topology.c +2
-1@@ -29,11 +29,13 @@* .cores_per_socket: tracks information on the count of cores * per socket. * .smp: keeps track of the machine topology. -* +* .list: queue the topology entries inside which +* we keep the information on the CPU topology.
The comment does not match the code below.
Quoted 72 lines
hw/s390x/cpu-topology.c +1
-0*/ S390Topology s390_topology = {/* will be initialized after the cpu model is realized */ .cores_per_socket =NULL , + .polarization =S390_CPU_POLARIZATION_HORIZONTAL , };/** target/s390x/kvm/kvm.c +4
-1@@ -1919,9 +1919,12 @@ static int handle_stsi(S390CPU *cpu)if (run->s390_stsi.sel1 !=2 || run->s390_stsi.sel2 !=2 ) {return 0 ; } -/* Only sysib 3.2.2 needs post-handling for now. */ insert_stsi_3_2_2 (cpu, run->s390_stsi.addr, run->s390_stsi.ar);return 0 ; +case 15 : +insert_stsi_15_1_x (cpu, run->s390_stsi.sel2, run->s390_stsi.addr, + run->s390_stsi.ar); +return 0 ;default :return 0 ; }target/s390x/kvm/stsi-topology.c new file +40
-0@@ -0,0 +1,310 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* QEMU S390x CPU Topology +* +* Copyright IBM Corp. 2022,2023 +* Author(s): Pierre Morel <[email protected]> +* +*/ +#include "qemu/osdep.h" +#include "cpu.h" +#include "hw/s390x/pv.h" +#include "hw/sysbus.h" +#include "hw/s390x/sclp.h" +#include "hw/s390x/cpu-topology.h" + +/** +* fill_container: +* @p: The address of the container TLE to fill +* @level: The level of nesting for this container +* @id: The container receives a unique ID inside its own container +* +* Returns the next free TLE entry. +*/ +static char *fill_container (char *p,int level,int id) +{ +SysIBTl_container *tle = (SysIBTl_container *)p; + + tle->nl = level; + tle->id = id; +return p +sizeof (*tle); +} + +/** +* fill_tle_cpu: +* @p: The address of the CPU TLE to fill +* @entry: a pointer to the S390TopologyEntry defining this +* CPU container. +* +* Returns the next free TLE entry. +*/
target/s390x/kvm/stsi-topology.c line 50
Quoted 10 lines
target/s390x/kvm/stsi-topology.c new file +10
-0+static char *fill_tle_cpu (char *p,S390TopologyEntry *entry) +{ +SysIBTl_cpu *tle = (SysIBTl_cpu *)p; +s390_topology_id topology_id = entry->id; + + tle->nl =0 ; +if (topology_id.dedicated) { + tle->flags =SYSIB_TLE_DEDICATED ; + } + tle->flags |= topology_id.entitlement;
If it was not dedicated, the "|=" might operate with a random value here. Better pre-initialize flags with 0, or change the order of the operations here:
tle->flags = topology_id.entitlement;
if (topology_id.dedicated) {
tle->flags |= SYSIB_TLE_DEDICATED;
}Quoted 110 lines
target/s390x/kvm/stsi-topology.c new file +110
-0+ tle->type = topology_id.type; + tle->origin =cpu_to_be16 (topology_id.origin *64 ); + tle->mask =cpu_to_be64 (entry->mask); +return p +sizeof (*tle); +} + +/* +* Macro to check that the size of data after increment +* will not get bigger than the size of the SysIB. +*/ +#define SYSIB_GUARD (data, x) do { \ + data += x; \ + if (data > sizeof(SysIB)) { \ + return 0; \ + } \ + } while (0) + +/** +* stsi_topology_fill_sysib: +* @p: A pointer to the position of the first TLE +* @level: The nested level wanted by the guest +* +* Fill the SYSIB with the topology information as described in +* the PoP, nesting containers as appropriate, with the maximum +* nesting limited by @level. +* +* Return value: +* On success: the size of the SysIB_15x after being filled with TLE. +* On error: 0 in the case we would overrun the end of the SysIB. +*/ +static int stsi_topology_fill_sysib (S390TopologyList *topology_list, +char *p,int level) +{ +S390TopologyEntry *entry; +int last_drawer =-1 ; +int last_book =-1 ; +int last_socket =-1 ; +int drawer_id =0 ; +int book_id =0 ; +int socket_id =0 ; +int n =sizeof (SysIB_151x); + + QTAILQ_FOREACH(entry, topology_list, next) { +bool drawer_change = last_drawer != entry->id.drawer; +bool book_change = drawer_change || last_book != entry->id.book; +bool socket_change = book_change || last_socket != entry->id.socket; + +/* If we reach the sentinel get out */ +if (entry->id.sentinel) { +break ; + } + +if (level >3 && drawer_change) { +SYSIB_GUARD (n,sizeof (SysIBTl_container)); + p =fill_container (p,3 , drawer_id++); + book_id =0 ; + } +if (level >2 && book_change) { +SYSIB_GUARD (n,sizeof (SysIBTl_container)); + p =fill_container (p,2 , book_id++); + socket_id =0 ; + } +if (socket_change) { +SYSIB_GUARD (n,sizeof (SysIBTl_container)); + p =fill_container (p,1 , socket_id++); + } + +SYSIB_GUARD (n,sizeof (SysIBTl_cpu)); + p =fill_tle_cpu (p, entry); + last_drawer = entry->id.drawer; + last_book = entry->id.book; + last_socket = entry->id.socket; + } + +return n; +} + +/** +* setup_stsi: +* sysib: pointer to a SysIB to be filled with SysIB_151x data +* level: Nested level specified by the guest +* +* Setup the SYSIB for STSI 15.1, the header as well as the description +* of the topology. +*/ +static int setup_stsi (S390TopologyList *topology_list,SysIB_151x *sysib, +int level) +{ + sysib->mnest = level; +switch (level) { +case 4 : + sysib->mag[S390_TOPOLOGY_MAG4 ] = current_machine->smp.drawers; + sysib->mag[S390_TOPOLOGY_MAG3 ] = current_machine->smp.books; + sysib->mag[S390_TOPOLOGY_MAG2 ] = current_machine->smp.sockets; + sysib->mag[S390_TOPOLOGY_MAG1 ] = current_machine->smp.cores; +break ; +case 3 : + sysib->mag[S390_TOPOLOGY_MAG3 ] = current_machine->smp.drawers * + current_machine->smp.books; + sysib->mag[S390_TOPOLOGY_MAG2 ] = current_machine->smp.sockets; + sysib->mag[S390_TOPOLOGY_MAG1 ] = current_machine->smp.cores; +break ; +case 2 : + sysib->mag[S390_TOPOLOGY_MAG2 ] = current_machine->smp.drawers * + current_machine->smp.books * + current_machine->smp.sockets; + sysib->mag[S390_TOPOLOGY_MAG1 ] = current_machine->smp.cores; +break ; + } +
target/s390x/kvm/stsi-topology.c line 170
Quoted 10 lines
target/s390x/kvm/stsi-topology.c new file +10
-0+return stsi_topology_fill_sysib (topology_list, sysib->tle, level); +} + +/** +* s390_topology_add_cpu_to_entry: +* @entry: Topology entry to setup +* @cpu: the S390CPU to add +* +* Set the core bit inside the topology mask and +* increments the number of cores for the socket.
The second part of the sentence does not seem to be done here anymore?
Quoted 24 lines
target/s390x/kvm/stsi-topology.c new file +24
-0+*/ +static void s390_topology_add_cpu_to_entry (S390TopologyEntry *entry, +S390CPU *cpu) +{ +set_bit (63 - (cpu->env.core_id %64 ), &entry->mask); +} + +/** +* s390_topology_from_cpu: +* @cpu: The S390CPU +* +* Initialize the topology id from the CPU environment. +*/ +static s390_topology_id s390_topology_from_cpu (S390CPU *cpu) +{ +s390_topology_id topology_id = {0 }; + + topology_id.drawer = cpu->env.drawer_id; + topology_id.book = cpu->env.book_id; + topology_id.socket = cpu->env.socket_id; + topology_id.origin = cpu->env.core_id /64 ; + topology_id.type =S390_TOPOLOGY_CPU_IFL ; + topology_id.dedicated = cpu->env.dedicated; +
target/s390x/kvm/stsi-topology.c line 204
Quoted 10 lines
target/s390x/kvm/stsi-topology.c new file +10
-0+if (s390_topology.polarization ==S390_CPU_POLARIZATION_VERTICAL ) { + topology_id.entitlement = cpu->env.entitlement; + } + +return topology_id; +} + +/** +* s390_topology_insert: +* @cpu: s390CPU insert.
"s390CPU insert" sounds like a weird description for @cpu
Quoted 57 lines
target/s390x/kvm/stsi-topology.c new file +57
-0+* +* Parse the topology list to find if the entry already +* exist and add the core in it. +* If it does not exist, allocate a new entry and insert +* it in the queue from lower id to greater id. +*/ +static void s390_topology_insert (S390TopologyList *topology_list,S390CPU *cpu) +{ +s390_topology_id id =s390_topology_from_cpu (cpu); +S390TopologyEntry *entry =NULL ; +S390TopologyEntry *tmp =NULL ; + + QTAILQ_FOREACH(tmp, topology_list, next) { +if (id.id == tmp->id.id) { +s390_topology_add_cpu_to_entry (tmp, cpu); +return ; + }else if (id.id < tmp->id.id) { + entry =g_malloc0 (sizeof (S390TopologyEntry)); + entry->id.id = id.id; +s390_topology_add_cpu_to_entry (entry, cpu); +QTAILQ_INSERT_BEFORE (tmp, entry, next); +return ; + } + } +} + +/** +* s390_topology_fill_list_sorted: +* +* Loop over all CPU and insert it at the right place +* inside the TLE entry list. +* Fill the S390Topology list with entries according to the order +* specified by the PoP. +*/ +static void s390_topology_fill_list_sorted (S390TopologyList *topology_list) +{ +CPUState *cs; + +CPU_FOREACH (cs) { +s390_topology_insert (topology_list,S390_CPU (cs)); + } +} + +/** +* s390_topology_empty_list: +* +* Clear all entries in the S390Topology list except the sentinel. +*/ +static void s390_topology_empty_list (S390TopologyList *topology_list) +{ +S390TopologyEntry *entry =NULL ; +S390TopologyEntry *tmp =NULL ; + + QTAILQ_FOREACH_SAFE(entry, topology_list, next, tmp) { +QTAILQ_REMOVE (topology_list, entry, next); +g_free (entry); + }
target/s390x/kvm/stsi-topology.c line 271
Quoted 10 lines
target/s390x/kvm/stsi-topology.c new file +10
-0+} + +/** +* insert_stsi_15_1_x: +* cpu: the CPU doing the call for which we set CC +* sel2: the selector 2, containing the nested level +* addr: Guest logical address of the guest SysIB +* ar: the access register number +* +* Create a list head for the Topology entries and initialize it.
s/Topology/topology/
target/s390x/kvm/stsi-topology.c line 272
Quoted 1 line
target/s390x/kvm/stsi-topology.c new file +1
-0+* Insert the first entry as a sentinelle.
s/sentinelle/sentinel/
Quoted 24 lines
target/s390x/kvm/stsi-topology.c new file +24
-0+* Emulate STSI 15.1.x, that is, perform all necessary checks and +* fill the SYSIB. +* In case the topology description is too long to fit into the SYSIB, +* set CC=3 and abort without writing the SYSIB. +*/ +void insert_stsi_15_1_x (S390CPU *cpu,int sel2,uint64_t addr,uint8_t ar) +{ +S390TopologyList topology_list; +S390TopologyEntry *entry; +SysIB sysib = {0 }; +int length; + +if (!s390_has_topology () || sel2 <2 || sel2 >SCLP_READ_SCP_INFO_MNEST ) { +setcc (cpu,3 ); +return ; + } + +QTAILQ_INIT (&topology_list); + entry =g_malloc0 (sizeof (S390TopologyEntry)); + entry->id.sentinel =0xff ; +QTAILQ_INSERT_HEAD (&topology_list, entry, next); + +s390_topology_fill_list_sorted (&topology_list); +
target/s390x/kvm/stsi-topology.c line 307
Quoted 10 lines
target/s390x/kvm/stsi-topology.c new file +10
-0+ length =setup_stsi (&topology_list, &sysib.sysib_151x, sel2); + +if (!length) { +setcc (cpu,3 ); +return ; + } + + sysib.sysib_151x.length =cpu_to_be16 (length); +s390_cpu_virt_mem_write (cpu, addr, ar, &sysib, length); +setcc (cpu,0 );
s390_cpu_virt_mem_write() could fail, I think in that case you should not change the CC value here?
Also, what about protected virtualization? Do you have to use s390_cpu_pv_mem_write() in case PV is enabled?
Quoted 8 lines
target/s390x/kvm/stsi-topology.c new file +3
-0+ +s390_topology_empty_list (&topology_list); +}target/s390x/kvm/meson.build +0
-0
target/s390x/kvm/meson.build line 7
Quoted 8 lines
target/s390x/kvm/meson.build +2
-1@@ -1,6 +1,7 @@ s390x_ss.add(when: 'CONFIG_KVM', if_true: files( - 'kvm.c' + 'kvm.c', + 'stsi-topology.c' ), if_false: files( 'stubs.c' ))
Thomas
Re: [PATCH v21 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
On 04/07/2023 13.40, Thomas Huth wrote:
...
Quoted 1 line
Also, what about protected virtualization? Do you have to use
s390_cpu_pv_mem_write()in case PV is enabled?
Never mind, I keep forgetting that CPU topology can't be used together with PV (I just noticed after reading patch 07/20). Not sure ... but maybe a comment here in the code would help?
Thomas
Re: [PATCH v21 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
In reply to Thomas HuthQuoted 76 lines · Thomas Huth
Quoted 5 lines · Pierre Morel
On interception of STSI(15.1.x) the System Information Block (SYSIB) is built from the list of pre-ordered topology entries.
Signed-off-by Pierre Morel <[email protected]>---
...
Quoted 69 lines
target/s390x/cpu.h +0
-0@@ -569,6 +569,29 @@ typedef struct SysIB_322 { } SysIB_322; QEMU_BUILD_BUG_ON(sizeof(SysIB_322) != 4096);+/*
+ * Topology Magnitude fields (MAG) indicates the maximum number of
+ * topology list entries (TLE) at the corresponding nesting level.
+ */target/s390x/cpu.h +19
-0+#define S390_TOPOLOGY_MAG 6 +#define S390_TOPOLOGY_MAG6 0 +#define S390_TOPOLOGY_MAG5 1 +#define S390_TOPOLOGY_MAG4 2 +#define S390_TOPOLOGY_MAG3 3 +#define S390_TOPOLOGY_MAG2 4 +#define S390_TOPOLOGY_MAG1 5 +/* Configuration topology */ +typedef struct SysIB_151x { + uint8_t reserved0[2]; + uint16_t length; + uint8_t mag[S390_TOPOLOGY_MAG]; + uint8_t reserved1; + uint8_t mnest; + uint32_t reserved2; + char tle[]; +}SysIB_151x ; +QEMU_BUILD_BUG_ON (sizeof (SysIB_151x) !=16 ); +typedef union SysIB {
SysIB_111sysib_111;
SysIB_121sysib_121;target/s390x/cpu.h +1
-0@@ -576,9 +599,62 @@ typedef union SysIB { SysIB_221 sysib_221; SysIB_222 sysib_222; SysIB_322 sysib_322; + SysIB_151x sysib_151x; } SysIB; QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096);+/*
+ * CPU Topology List provided by STSI with fc=15 provides a list
+ * of two different Topology List Entries (TLE) types to specify
+ * the topology hierarchy.
+ *
+ * - Container Topology List Entry
+ * Defines a container to contain other Topology List Entries
+ * of any type, nested containers or CPU.
+ * - CPU Topology List Entry
+ * Specifies the CPUs position, type, entitlement and polarization
+ * of the CPUs contained in the last Container TLE.
+ *
+ * There can be theoretically up to five levels of containers, QEMU
+ * uses only three levels, the drawer's, book's and socket's level.
+ *
+ * A container with a nesting level (NL) greater than 1 can only
+ * contain another container of nesting level NL-1.
+ *
+ * A container of nesting level 1 (socket), contains as many CPU TLE
+ * as needed to describe the position and qualities of all CPUs inside
+ * the container.
+ * The qualities of a CPU are polarization, entitlement and type.
+ *
+ * The CPU TLE defines the position of the CPUs of identical qualities
+ * using a 64bits mask which first bit has its offset defined by
+ * the CPU address orgin field of the CPU TLE like in:
+ * CPU address = origin * 64 + bit position within the mask
+ *
+ */
Quoted 8 lines
target/s390x/cpu.h typedef union SysIB { · line 640
Quoted 6 lines
target/s390x/cpu.h +6
-0+/* Container type Topology List Entry */ +typedef struct SysIBTl_container { + uint8_t nl; + uint8_t reserved[6]; + uint8_t id; +}SysIBTl_container ;Why mixing CamelCase with underscore-style here? SysIBTlContainer would look more natural, I think?
OK, what about SYSIBContainerListEntry ?
Quoted 16 lines
Quoted 4 lines
target/s390x/cpu.h +3
-0+QEMU_BUILD_BUG_ON (sizeof (SysIBTl_container) !=8 ); + +/* CPU type Topology List Entry */ target/s390x/cpu.h typedef union SysIB { · line 653
Quoted 10 lines
target/s390x/cpu.h +10
-0+typedef struct SysIBTl_cpu { + uint8_t nl; + uint8_t reserved0[3]; +#define SYSIB_TLE_POLARITY_MASK 0x03 +#define SYSIB_TLE_DEDICATED 0x04 + uint8_t flags; + uint8_t type; + uint16_t origin; + uint64_t mask; +}SysIBTl_cpu ;dito, maybe better SysIBTlCpu ?
What about SysIBCPUListEntry ?
Quoted 14 lines
Quoted 14 lines
target/s390x/cpu.h +3
-0+QEMU_BUILD_BUG_ON (sizeof (SysIBTl_cpu) !=16 ); + +void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr,
uint8_tar);
+
/* MMU defines */
#define ASCE_ORIGIN (~0xfffULL) /* segment table origin */ #define ASCE_SUBSPACE 0x200 /* subspace group control */hw/s390x/cpu-topology.c +0
-0@@ -29,11 +29,13 @@
Quoted 8 lines
hw/s390x/cpu-topology.c line 33
Quoted 6 lines
hw/s390x/cpu-topology.c +2
-1* .cores_per_socket: tracks information on the count of cores * per socket. * .smp: keeps track of the machine topology. - * + * .list: queue the topology entries inside which + * we keep the information on the CPU topology.The comment does not match the code below.
a left over I modify this. Thanks.
Quoted 86 lines
Quoted 73 lines
hw/s390x/cpu-topology.c +1
-0*/ S390Topology s390_topology = { /* will be initialized after the cpu model is realized */ .cores_per_socket = NULL, + .polarization = S390_CPU_POLARIZATION_HORIZONTAL, }; /**target/s390x/kvm/kvm.c +4
-1@@ -1919,9 +1919,12 @@ static int handle_stsi(S390CPU *cpu) if (run->s390_stsi.sel1 != 2 || run->s390_stsi.sel2 != 2) { return 0; } - /* Only sysib 3.2.2 needs post-handling for now. */ insert_stsi_3_2_2(cpu, run->s390_stsi.addr, run->s390_stsi.ar); return 0; + case 15: + insert_stsi_15_1_x(cpu, run->s390_stsi.sel2, run->s390_stsi.addr, + run->s390_stsi.ar); + return 0;default:
return 0;
}target/s390x/kvm/stsi-topology.c new file +40
-0@@ -0,0 +1,310 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* +* QEMU S390x CPU Topology +* +* Copyright IBM Corp. 2022,2023 +* Author(s): Pierre Morel <[email protected]> +* +*/ +#include "qemu/osdep.h" +#include "cpu.h" +#include "hw/s390x/pv.h" +#include "hw/sysbus.h" +#include "hw/s390x/sclp.h" +#include "hw/s390x/cpu-topology.h" + +/** +* fill_container: +* @p: The address of the container TLE to fill +* @level: The level of nesting for this container +* @id: The container receives a unique ID inside its own container +* +* Returns the next free TLE entry. +*/ +static char *fill_container (char *p,int level,int id) +{ + SysIBTl_container *tle = (SysIBTl_container *)p; + + tle->nl = level; + tle->id = id; + return p + sizeof(*tle); +} + +/** +* fill_tle_cpu: +* @p: The address of the CPU TLE to fill +* @entry: a pointer to the S390TopologyEntry defining this +* CPU container. +* +* Returns the next free TLE entry. +*/ target/s390x/kvm/stsi-topology.c line 50
Quoted 10 lines
target/s390x/kvm/stsi-topology.c new file +10
-0+static char *fill_tle_cpu (char *p,S390TopologyEntry *entry) +{ + SysIBTl_cpu *tle = (SysIBTl_cpu *)p; + s390_topology_id topology_id = entry->id; + + tle->nl = 0; + if (topology_id.dedicated) { + tle->flags = SYSIB_TLE_DEDICATED; + } + tle->flags |= topology_id.entitlement;If it was not dedicated, the "|=" might operate with a random value here. Better pre-initialize flags with 0, or change the order of the operations here:
Quoted 4 lines
tle->flags= topology_id.entitlement;
if (topology_id.dedicated) {
tle->flags|=SYSIB_TLE_DEDICATED;
}
oh, yes.
Quoted 118 lines
Quoted 118 lines
target/s390x/kvm/stsi-topology.c new file +45
-0+ tle->type = topology_id.type; + tle->origin = cpu_to_be16(topology_id.origin * 64); + tle->mask = cpu_to_be64(entry->mask); + return p + sizeof(*tle); +} + +/* +* Macro to check that the size of data after increment +* will not get bigger than the size of the SysIB. +*/ +#define SYSIB_GUARD (data, x) do { \ + data += x; \ + if (data > sizeof(SysIB)) { \ + return 0; \ + } \ + } while (0) + +/** +* stsi_topology_fill_sysib: +* @p: A pointer to the position of the first TLE +* @level: The nested level wanted by the guest +* +* Fill the SYSIB with the topology information as described in +* the PoP, nesting containers as appropriate, with the maximum +* nesting limited by @level. +* +* Return value: +* On success: the size of the SysIB_15x after being filled with TLE. +* On error: 0 in the case we would overrun the end of the SysIB. +*/ +static int stsi_topology_fill_sysib (S390TopologyList *topology_list, + char *p, int level) +{ + S390TopologyEntry *entry; + int last_drawer = -1; + int last_book = -1; + int last_socket = -1; + int drawer_id = 0; + int book_id = 0; + int socket_id = 0; + int n = sizeof(SysIB_151x); + + QTAILQ_FOREACH(entry, topology_list, next) { + bool drawer_change = last_drawer != entry->id.drawer; + bool book_change = drawer_change || last_book !=
entry->id.book;
+ boolsocket_change=book_change||last_socket!=entry->id.socket;target/s390x/kvm/stsi-topology.c new file +40
-0+ + /* If we reach the sentinel get out */ + if (entry->id.sentinel) { + break; + } + + if (level > 3 && drawer_change) { + SYSIB_GUARD(n, sizeof(SysIBTl_container)); + p = fill_container(p, 3, drawer_id++); + book_id = 0; + } + if (level > 2 && book_change) { + SYSIB_GUARD(n, sizeof(SysIBTl_container)); + p = fill_container(p, 2, book_id++); + socket_id = 0; + } + if (socket_change) { + SYSIB_GUARD(n, sizeof(SysIBTl_container)); + p = fill_container(p, 1, socket_id++); + } + + SYSIB_GUARD(n, sizeof(SysIBTl_cpu)); + p = fill_tle_cpu(p, entry); + last_drawer = entry->id.drawer; + last_book = entry->id.book; + last_socket = entry->id.socket; + } + + return n; +} + +/** +* setup_stsi: +* sysib: pointer to a SysIB to be filled with SysIB_151x data +* level: Nested level specified by the guest +* +* Setup the SYSIB for STSI 15.1, the header as well as the description +* of the topology. +*/ +static int setup_stsi(S390TopologyList *topology_list, SysIB_151x*sysib,
+ int level)target/s390x/kvm/stsi-topology.c new file +27
-0+{ + sysib->mnest = level; + switch (level) { + case 4: + sysib->mag[S390_TOPOLOGY_MAG4] = current_machine->smp.drawers; + sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.books; + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets; + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores; + break; + case 3: + sysib->mag[S390_TOPOLOGY_MAG3] = current_machine->smp.drawers * + current_machine->smp.books; + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.sockets; + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores; + break; + case 2: + sysib->mag[S390_TOPOLOGY_MAG2] = current_machine->smp.drawers * + current_machine->smp.books * + current_machine->smp.sockets; + sysib->mag[S390_TOPOLOGY_MAG1] = current_machine->smp.cores; + break; + } + + return stsi_topology_fill_sysib(topology_list, sysib->tle, level); +} + +/**
Quoted 8 lines
Quoted 6 lines
+ *
s390_topology_add_cpu_to_entry:
+ * @entry: Topology entry to setup
+ * @cpu: the S390CPU to add
+ *
+ * Set the core bit inside the topology mask and
+ * increments the number of cores for the socket.The second part of the sentence does not seem to be done here anymore?
Right, it is not needed here anymore since this is doe during the STSI interception.
It was needed during the setup of the topology or during changes, hotplug or moving a CPU,
which is done in hw/s390/cpu.topology.c
Quoted 29 lines
Quoted 29 lines
+ */
target/s390x/kvm/stsi-topology.c new file +27
-0+static void s390_topology_add_cpu_to_entry (S390TopologyEntry *entry, + S390CPU *cpu) +{ + set_bit(63 - (cpu->env.core_id % 64), &entry->mask); +} + +/** +* s390_topology_from_cpu: +* @cpu: The S390CPU +* +* Initialize the topology id from the CPU environment. +*/ +static s390_topology_id s390_topology_from_cpu (S390CPU *cpu) +{ + s390_topology_id topology_id = {0}; + + topology_id.drawer = cpu->env.drawer_id; + topology_id.book = cpu->env.book_id; + topology_id.socket = cpu->env.socket_id; + topology_id.origin = cpu->env.core_id / 64; + topology_id.type = S390_TOPOLOGY_CPU_IFL; + topology_id.dedicated = cpu->env.dedicated; + + if (s390_topology.polarization == S390_CPU_POLARIZATION_VERTICAL) { + topology_id.entitlement = cpu->env.entitlement; + } +
Quoted 8 lines
target/s390x/kvm/stsi-topology.c line 204
Quoted 6 lines
+ return
topology_id;target/s390x/kvm/stsi-topology.c new file +5
-0+} + +/** +* s390_topology_insert: +* @cpu: s390CPU insert. "s390CPU insert" sounds like a weird description for @cpu
yes I think "S390CPU to be inserted in the topology list" is better
Quoted 64 lines
Quoted 64 lines
+ *
+ * Parse the topology list to find if the entry already
+ * exist and add the core in it.
+ * If it does not exist, allocate a new entry and insert
+ * it in the queue from lower id to greater id.
+ */
+static voids390_topology_insert(S390TopologyList *topology_list, S390CPU *cpu)target/s390x/kvm/stsi-topology.c new file +28
-0+{ + s390_topology_id id = s390_topology_from_cpu(cpu); + S390TopologyEntry *entry = NULL; + S390TopologyEntry *tmp = NULL; + + QTAILQ_FOREACH(tmp, topology_list, next) { + if (id.id == tmp->id.id) { + s390_topology_add_cpu_to_entry(tmp, cpu); + return; + } else if (id.id < tmp->id.id) { + entry = g_malloc0(sizeof(S390TopologyEntry)); + entry->id.id = id.id; + s390_topology_add_cpu_to_entry(entry, cpu); + QTAILQ_INSERT_BEFORE(tmp, entry, next); + return; + } + } +} + +/** +* s390_topology_fill_list_sorted: +* +* Loop over all CPU and insert it at the right place +* inside the TLE entry list. +* Fill the S390Topology list with entries according to the order +* specified by the PoP. +*/ +static void s390_topology_fill_list_sorted(S390TopologyList
*topology_list)target/s390x/kvm/stsi-topology.c new file +26
-0+{ + CPUState *cs; + + CPU_FOREACH(cs) { + s390_topology_insert(topology_list, S390_CPU(cs)); + } +} + +/** +* s390_topology_empty_list: +* +* Clear all entries in the S390Topology list except the sentinel. +*/ +static void s390_topology_empty_list (S390TopologyList *topology_list) +{ + S390TopologyEntry *entry = NULL; + S390TopologyEntry *tmp = NULL; + + QTAILQ_FOREACH_SAFE(entry, topology_list, next, tmp) { + QTAILQ_REMOVE(topology_list, entry, next); + g_free(entry); + } +} + +/** +* insert_stsi_15_1_x:
Quoted 8 lines
Quoted 6 lines
+ * cpu: the CPU doing the call for which we set CC
+ * sel2: the selector 2, containing the nested level
+ * addr: Guest logical address of the guest SysIB
+ * ar: the access register number
+ *
+ * Create a list head for the Topology entries and initialize it.s/Topology/topology/
OK
Quoted 4 lines
Quoted 2 lines
+ * Insert the first entry as a sentinelle.
s/sentinelle/sentinel/
thx
Quoted 23 lines
Quoted 23 lines
+ * Emulate STSI 15.1.x, that is, perform all necessary checks and
+ * fill the SYSIB.
+ * In case the topology description is too long to fit into the SYSIB,
+ * set CC=3 and abort without writing the SYSIB.
+ */
+voidinsert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar)target/s390x/kvm/stsi-topology.c new file +7
-0+{ + S390TopologyList topology_list; + S390TopologyEntry *entry; + SysIB sysib = {0}; + int length; + + if (!s390_has_topology() || sel2 < 2 || sel2 >
SCLP_READ_SCP_INFO_MNEST) {
+setcc(cpu, 3);
+ return;
+ }
+
+QTAILQ_INIT(&topology_list);
+ entry = g_malloc0(sizeof(S390TopologyEntry)); +entry->id.sentinel=0xff;
Quoted 8 lines
target/s390x/kvm/stsi-topology.c line 301
Quoted 8 lines
+
QTAILQ_INSERT_HEAD(&topology_list, entry, next);target/s390x/kvm/stsi-topology.c new file +7
-0+ + s390_topology_fill_list_sorted(&topology_list); + + length = setup_stsi(&topology_list, &sysib.sysib_151x, sel2); + + if (!length) { + setcc(cpu, 3);
I forgot to free the buffers here... will be added like
if (!length) {
s390_topology_empty_list(&topology_list);
setcc(cpu, 3);
return;
}
Quoted 8 lines
Quoted 6 lines
+ return;
+ }
+
+ sysib.sysib_151x.length =cpu_to_be16(length); +s390_cpu_virt_mem_write(cpu, addr, ar, &sysib, length); +setcc(cpu, 0);
s390_cpu_virt_mem_write()could fail, I think in that case you should not change the CC value here?
right, thanks.
Regards,
Pierre
Re: [PATCH v21 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
Quoted 88 lines · Pierre Morel
Quoted 76 lines · Thomas Huth
Quoted 5 lines · Pierre Morel
On interception of STSI(15.1.x) the System Information Block (SYSIB) is built from the list of pre-ordered topology entries.
Signed-off-by Pierre Morel <[email protected]>---
...
Quoted 69 lines
target/s390x/cpu.h +0
-0@@ -569,6 +569,29 @@ typedef struct SysIB_322 { } SysIB_322; QEMU_BUILD_BUG_ON(sizeof(SysIB_322) != 4096);+/*
+ * Topology Magnitude fields (MAG) indicates the maximum number of
+ * topology list entries (TLE) at the corresponding nesting level.
+ */target/s390x/cpu.h +19
-0+#define S390_TOPOLOGY_MAG 6 +#define S390_TOPOLOGY_MAG6 0 +#define S390_TOPOLOGY_MAG5 1 +#define S390_TOPOLOGY_MAG4 2 +#define S390_TOPOLOGY_MAG3 3 +#define S390_TOPOLOGY_MAG2 4 +#define S390_TOPOLOGY_MAG1 5 +/* Configuration topology */ +typedef struct SysIB_151x { + uint8_t reserved0[2]; + uint16_t length; + uint8_t mag[S390_TOPOLOGY_MAG]; + uint8_t reserved1; + uint8_t mnest; + uint32_t reserved2; + char tle[]; +}SysIB_151x ; +QEMU_BUILD_BUG_ON (sizeof (SysIB_151x) !=16 ); +typedef union SysIB {
SysIB_111sysib_111;
SysIB_121sysib_121;target/s390x/cpu.h +1
-0@@ -576,9 +599,62 @@ typedef union SysIB { SysIB_221 sysib_221; SysIB_222 sysib_222; SysIB_322 sysib_322; + SysIB_151x sysib_151x; } SysIB; QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096);+/*
+ * CPU Topology List provided by STSI with fc=15 provides a list
+ * of two different Topology List Entries (TLE) types to specify
+ * the topology hierarchy.
+ *
+ * - Container Topology List Entry
+ * Defines a container to contain other Topology List Entries
+ * of any type, nested containers or CPU.
+ * - CPU Topology List Entry
+ * Specifies the CPUs position, type, entitlement and polarization
+ * of the CPUs contained in the last Container TLE.
+ *
+ * There can be theoretically up to five levels of containers, QEMU
+ * uses only three levels, the drawer's, book's and socket's level.
+ *
+ * A container with a nesting level (NL) greater than 1 can only
+ * contain another container of nesting level NL-1.
+ *
+ * A container of nesting level 1 (socket), contains as many CPU TLE
+ * as needed to describe the position and qualities of all CPUs inside
+ * the container.
+ * The qualities of a CPU are polarization, entitlement and type.
+ *
+ * The CPU TLE defines the position of the CPUs of identical qualities
+ * using a 64bits mask which first bit has its offset defined by
+ * the CPU address orgin field of the CPU TLE like in:
+ * CPU address = origin * 64 + bit position within the mask
+ *
+ */Quoted 8 lines
target/s390x/cpu.h typedef union SysIB { · line 640
Quoted 6 lines
target/s390x/cpu.h +6
-0+/* Container type Topology List Entry */ +typedef struct SysIBTl_container { + uint8_t nl; + uint8_t reserved[6]; + uint8_t id; +}SysIBTl_container ;Why mixing CamelCase with underscore-style here? SysIBTlContainer would look more natural, I think?
Quoted 1 line
OK, what about SYSIBContainerListEntry ?
Sounds fine!
Quoted 19 lines
Quoted 17 lines
Quoted 5 lines
target/s390x/cpu.h +3
-0+QEMU_BUILD_BUG_ON (sizeof (SysIBTl_container) !=8 ); + +/* CPU type Topology List Entry */ target/s390x/cpu.h typedef union SysIB { · line 653
Quoted 10 lines
target/s390x/cpu.h +10
-0+typedef struct SysIBTl_cpu { + uint8_t nl; + uint8_t reserved0[3]; +#define SYSIB_TLE_POLARITY_MASK 0x03 +#define SYSIB_TLE_DEDICATED 0x04 + uint8_t flags; + uint8_t type; + uint16_t origin; + uint64_t mask; +}SysIBTl_cpu ;dito, maybe better SysIBTlCpu ?
Quoted 1 line
What about SysIBCPUListEntry ?
Ack.
Thomas
Re: [PATCH v21 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
In reply to Pierre MorelQuoted 16 lines · Pierre Morel
On interception of STSI(15.1.x) the System Information Block (SYSIB) is built from the list of pre-ordered topology entries.
Signed-off-by Pierre Morel <[email protected]>--- MAINTAINERS | 1 + qapi/machine-target.json | 14 ++ include/hw/s390x/cpu-topology.h | 25 +++ include/hw/s390x/sclp.h | 1 + target/s390x/cpu.h | 76 ++++++++ hw/s390x/cpu-topology.c | 4 +- target/s390x/kvm/kvm.c | 5 +- target/s390x/kvm/stsi-topology.c | 310 +++++++++++++++++++++++++++++++ target/s390x/kvm/meson.build | 3 +- 9 files changed, 436 insertions(+), 3 deletions(-) create mode 100644 target/s390x/kvm/stsi-topology.c
[...]
Quoted 3 lines
typedef struct S390Topology { uint8_t *cores_per_socket;+ bool polarization;
You don't use this as a bool and since it's no longer called vertical_polarization, it's not longer entirely clear what the value means so I think this should be a CpuS390Polarization.
That also makes the assignment in patch 12 clearer since it assigns the same type.
[...]
Quoted 5 lines
S390Topology
s390_topology= {
/* will be initialized after the cpu model is realized */ .cores_per_socket = NULL,
+ .polarization =S390_CPU_POLARIZATION_HORIZONTAL,
};
[...]
Quoted 17 lines
target/s390x/kvm/stsi-topology.c new file +17
-0+static s390_topology_id s390_topology_from_cpu (S390CPU *cpu) +{ +s390_topology_id topology_id = {0 }; + + topology_id.drawer = cpu->env.drawer_id; + topology_id.book = cpu->env.book_id; + topology_id.socket = cpu->env.socket_id; + topology_id.origin = cpu->env.core_id /64 ; + topology_id.type =S390_TOPOLOGY_CPU_IFL ; + topology_id.dedicated = cpu->env.dedicated; + +if (s390_topology.polarization ==S390_CPU_POLARIZATION_VERTICAL ) { + topology_id.entitlement = cpu->env.entitlement; + } + +return topology_id; +}
[...]
Re: [PATCH v21 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
Quoted 22 lines · Nina Schoetterl-Glausch
Quoted 16 lines · Pierre Morel
On interception of STSI(15.1.x) the System Information Block (SYSIB) is built from the list of pre-ordered topology entries.
Signed-off-by Pierre Morel <[email protected]>--- MAINTAINERS | 1 + qapi/machine-target.json | 14 ++ include/hw/s390x/cpu-topology.h | 25 +++ include/hw/s390x/sclp.h | 1 + target/s390x/cpu.h | 76 ++++++++ hw/s390x/cpu-topology.c | 4 +- target/s390x/kvm/kvm.c | 5 +- target/s390x/kvm/stsi-topology.c | 310 +++++++++++++++++++++++++++++++ target/s390x/kvm/meson.build | 3 +- 9 files changed, 436 insertions(+), 3 deletions(-)create mode 100644
target/s390x/kvm/stsi-topology.c[...]
Quoted 3 lines
typedef struct S390Topology { uint8_t *cores_per_socket;+ bool polarization;
Quoted 7 lines
You don't use this as a bool and since it's no longer called
vertical_polarization, it's not longer entirely clear what the value means so I think this should be a CpuS390Polarization.
That also makes the assignment in patch 12 clearer since it assigns the same type.[...]
right, I make the change
Thanks
Pierre
Re: [PATCH v21 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
In reply to Pierre MorelQuoted 62 lines · Pierre Morel
On interception of STSI(15.1.x) the System Information Block (SYSIB) is built from the list of pre-ordered topology entries.
Signed-off-by Pierre Morel <[email protected]>--- MAINTAINERS | 1 + qapi/machine-target.json | 14 ++ include/hw/s390x/cpu-topology.h | 25 +++ include/hw/s390x/sclp.h | 1 + target/s390x/cpu.h | 76 ++++++++ hw/s390x/cpu-topology.c | 4 +- target/s390x/kvm/kvm.c | 5 +- target/s390x/kvm/stsi-topology.c | 310 +++++++++++++++++++++++++++++++ target/s390x/kvm/meson.build | 3 +- 9 files changed, 436 insertions(+), 3 deletions(-) create mode 100644 target/s390x/kvm/stsi-topology.cMAINTAINERS +1
-0@@ -1702,6 +1702,7 @@ M: Pierre Morel <[email protected]> S: Supported F: include/hw/s390x/cpu-topology.h F: hw/s390x/cpu-topology.c +F: target/s390x/kvm/stsi-topology.c X86 Machines ------------qapi/machine-target.json +14
-0@@ -361,3 +361,17 @@ 'TARGET_MIPS', 'TARGET_LOONGARCH64', 'TARGET_RISCV' ] } } + +## +# @CpuS390Polarization: +# +# An enumeration of cpu polarization that can be assumed by a virtual +# S390 CPU +# +# Since: 8.1 +## +{ 'enum': 'CpuS390Polarization', + 'prefix': 'S390_CPU_POLARIZATION', + 'data': [ 'horizontal', 'vertical' ], + 'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] } +}include/hw/s390x/cpu-topology.h +3
-0@@ -15,10 +15,35 @@#include "hw/boards.h" #include "qapi/qapi-types-machine-target.h" +#define S390_TOPOLOGY_CPU_IFL 0x03 + +typedef union s390_topology_id {
include/hw/s390x/cpu-topology.h line 30
Quoted 10 lines
include/hw/s390x/cpu-topology.h +10
-0+uint64_t id; +struct { +uint8_t sentinel; +uint8_t drawer; +uint8_t book; +uint8_t socket; +uint8_t dedicated; +uint8_t entitlement; +uint8_t type; +uint8_t origin;
The order here is not quite right according to the PoP. Type should be higher, after socket, such that all cpus of the same type in a socket are stored as a block.
Also entitlement and dedication need to be inverted, e.g such that dedicated cpus are shown before non dedicated ones.
Quoted 3 lines
include/hw/s390x/cpu-topology.h +3
-0+ }; +} s390_topology_id; +
[...]
Quoted 4 lines
+/**
+ *s390_topology_empty_list:
+ *
+ * Clear all entries in the S390Topology list except the sentinel.
The comment is out of date.
Quoted 72 lines
+ */
include/hw/s390x/cpu-topology.h +58
-0+static void s390_topology_empty_list (S390TopologyList *topology_list) +{ +S390TopologyEntry *entry =NULL ; +S390TopologyEntry *tmp =NULL ; + + QTAILQ_FOREACH_SAFE(entry, topology_list, next, tmp) { +QTAILQ_REMOVE (topology_list, entry, next); +g_free (entry); + } +} + +/** +* insert_stsi_15_1_x: +* cpu: the CPU doing the call for which we set CC +* sel2: the selector 2, containing the nested level +* addr: Guest logical address of the guest SysIB +* ar: the access register number +* +* Create a list head for the Topology entries and initialize it. +* Insert the first entry as a sentinelle. +* +* Emulate STSI 15.1.x, that is, perform all necessary checks and +* fill the SYSIB. +* In case the topology description is too long to fit into the SYSIB, +* set CC=3 and abort without writing the SYSIB. +*/ +void insert_stsi_15_1_x (S390CPU *cpu,int sel2,uint64_t addr,uint8_t ar) +{ +S390TopologyList topology_list; +S390TopologyEntry *entry; +SysIB sysib = {0 }; +int length; + +if (!s390_has_topology () || sel2 <2 || sel2 >SCLP_READ_SCP_INFO_MNEST ) { +setcc (cpu,3 ); +return ; + } + +QTAILQ_INIT (&topology_list); + entry =g_malloc0 (sizeof (S390TopologyEntry)); + entry->id.sentinel =0xff ; +QTAILQ_INSERT_HEAD (&topology_list, entry, next); + +s390_topology_fill_list_sorted (&topology_list); + + length =setup_stsi (&topology_list, &sysib.sysib_151x, sel2); + +if (!length) { +setcc (cpu,3 ); +return ; + } + + sysib.sysib_151x.length =cpu_to_be16 (length); +s390_cpu_virt_mem_write (cpu, addr, ar, &sysib, length); +setcc (cpu,0 ); + +s390_topology_empty_list (&topology_list); +}target/s390x/kvm/meson.build +2
-1@@ -1,6 +1,7 @@ s390x_ss.add(when: 'CONFIG_KVM', if_true: files( - 'kvm.c' + 'kvm.c', + 'stsi-topology.c' ), if_false: files( 'stubs.c' ))
PATCH v21 4/20 s390x/sclp: reporting the maximum nested topology entries
In reply to Pierre MorelThe maximum nested topology entries is used by the guest to know how many nested topology are available on the machine.
Let change the MNEST value from 2 to 4 in the SCLP READ INFO structure now that we support books and drawers.
--- include/hw/s390x/sclp.h | 5 +++-- hw/s390x/sclp.c | 5 +++++ 2 files changed, 8 insertions(+), 2 deletions(-)
include/hw/s390x/sclp.h +3 -2
@@ -112,12 +112,13 @@ typedef struct CPUEntry { }QEMU_PACKED CPUEntry;#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128 -#define SCLP_READ_SCP_INFO_MNEST 2 +#define SCLP_READ_SCP_INFO_MNEST 4 typedef struct ReadInfo { SCCBHeader h; uint16_t rnmax; uint8_t rnsize; - uint8_t _reserved1[16 - 11]; /* 11-15 */ + uint8_t _reserved1[15 - 11]; /* 11-14 */ + uint8_t stsi_parm; /* 15-15 */ uint16_t entries_cpu; /* 16-17 */ uint16_t offset_cpu; /* 18-19 */ uint8_t _reserved2[24 - 20]; /* 20-23 */
hw/s390x/sclp.c +5 -0
@@ -20,6 +20,7 @@#include "hw/s390x/event-facility.h" #include "hw/s390x/s390-pci-bus.h" #include "hw/s390x/ipl.h" +#include "hw/s390x/cpu-topology.h" static inline SCLPDevice *get_sclp_device (void ) {
@@ -123,6 +124,10 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb)return ; } +if (s390_has_topology ()) { + read_info->stsi_parm =SCLP_READ_SCP_INFO_MNEST ; + } +/* CPU information */ prepare_cpu_entries (machine, entries_start, &cpu_count); read_info->entries_cpu =cpu_to_be16 (cpu_count);
Signature · 1 line
2.31.1
PATCH v21 5/20 s390x/cpu topology: resetting the Topology-Change-Report
In reply to Pierre MorelDuring a subsystem reset the Topology-Change-Report is cleared by the machine.
Let's ask KVM to clear the Modified Topology Change Report (MTCR) bit of the SCA in the case of a subsystem reset.
--- include/hw/s390x/cpu-topology.h | 1 + target/s390x/cpu.h | 1 + target/s390x/kvm/kvm_s390x.h | 1 + hw/s390x/cpu-topology.c | 11 +++++++++++ hw/s390x/s390-virtio-ccw.c | 3 +++ target/s390x/cpu-sysemu.c | 13 +++++++++++++ target/s390x/kvm/kvm.c | 17 +++++++++++++++++ 7 files changed, 47 insertions(+)
include/hw/s390x/cpu-topology.h +1 -0
@@ -58,6 +58,7 @@ static inline void s390_topology_setup_cpu(MachineState *ms,#endif extern S390Topology s390_topology; +void s390_topology_reset (void );static inline int s390_std_socket (int n,CpuTopology *smp) {
target/s390x/cpu.h +1 -0
@@ -654,6 +654,7 @@ typedef struct SysIBTl_cpu {QEMU_BUILD_BUG_ON (sizeof (SysIBTl_cpu) !=16 );void insert_stsi_15_1_x (S390CPU *cpu,int sel2,uint64_t addr,uint8_t ar); +void s390_cpu_topology_set_changed (bool changed);/* MMU defines */ #define ASCE_ORIGIN (~0xfffULL)/* segment table origin */
target/s390x/kvm/kvm_s390x.h +1 -0
@@ -47,5 +47,6 @@ void kvm_s390_crypto_reset(void);void kvm_s390_restart_interrupt (S390CPU *cpu);void kvm_s390_stop_interrupt (S390CPU *cpu);void kvm_s390_set_diag318 (CPUState *cs,uint64_t diag318_info); +int kvm_s390_topology_set_mtcr (uint64_t attr);#endif /* KVM_S390X_H */
hw/s390x/cpu-topology.c +11 -0
@@ -92,6 +92,17 @@ static void s390_topology_init(MachineState *ms) smp->books * smp->drawers); } +/** +* s390_topology_reset: +* +* Generic reset for CPU topology, calls s390_topology_reset() +* to reset the kernel Modified Topology Change Record. +*/ +void s390_topology_reset (void ) +{ +s390_cpu_topology_set_changed (false); +} +/** * s390_topology_cpu_default: * @cpu: pointer to a S390CPU
hw/s390x/s390-virtio-ccw.c +3 -0
@@ -123,6 +123,9 @@ static void subsystem_reset(void)device_cold_reset (dev); } } +if (s390_has_topology ()) { +s390_topology_reset (); + } }static int virtio_ccw_hcall_notify (const uint64_t *args)
target/s390x/cpu-sysemu.c +13 -0
@@ -307,3 +307,16 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg)kvm_s390_set_diag318 (cs, arg.host_ulong); } } + +void s390_cpu_topology_set_changed (bool changed) +{ +int ret; + +if (kvm_enabled ()) { + ret =kvm_s390_topology_set_mtcr (changed); +if (ret) { +error_report ("Failed to set Modified Topology Change Report: %s" , +strerror (-ret)); + } + } +}
target/s390x/kvm/kvm.c +17 -0
@@ -2592,6 +2592,23 @@ int kvm_s390_get_zpci_op(void)return cap_zpci_op; } +int kvm_s390_topology_set_mtcr (uint64_t attr) +{ +struct kvm_device_attr attribute = { + .group =KVM_S390_VM_CPU_TOPOLOGY , + .attr = attr, + }; + +if (!s390_has_feat (S390_FEAT_CONFIGURATION_TOPOLOGY )) { +return 0 ; + } +if (!kvm_vm_check_attr (kvm_state,KVM_S390_VM_CPU_TOPOLOGY , attr)) { +return -ENOTSUP ; + } + +return kvm_vm_ioctl (kvm_state,KVM_SET_DEVICE_ATTR , &attribute); +} +void kvm_arch_accel_class_init (ObjectClass *oc) { }
Signature · 1 line
2.31.1
Re: [PATCH v21 05/20] s390x/cpu topology: resetting the Topology-Change-Report
Quoted 7 lines · Pierre Morel
During a subsystem reset the Topology-Change-Report is cleared by the machine.
Let's ask KVM to clear the Modified Topology Change Report (MTCR) bit of the SCA in the case of a subsystem reset.Signed-off-by Pierre Morel <[email protected]>Reviewed-by Thomas Huth <[email protected]>
PATCH v21 6/20 s390x/cpu topology: interception of PTF instruction
In reply to Pierre MorelWhen the host supports the CPU topology facility, the PTF instruction with function code 2 is interpreted by the SIE, provided that the userland hypervisor activates the interpretation by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension.
The PTF instructions with function code 0 and 1 are intercepted and must be emulated by the userland hypervisor.
During RESET all CPU of the configuration are placed in horizontal polarity.
--- include/hw/s390x/s390-virtio-ccw.h | 6 ++++ hw/s390x/cpu-topology.c | 54 ++++++++++++++++++++++++++++++ target/s390x/kvm/kvm.c | 11 ++++++ 3 files changed, 71 insertions(+)
include/hw/s390x/s390-virtio-ccw.h +6 -0
@@ -30,6 +30,12 @@ struct S390CcwMachineState {uint8_t loadparm[8 ]; }; +#define S390_PTF_REASON_NONE (0x00 << 8) +#define S390_PTF_REASON_DONE (0x01 << 8) +#define S390_PTF_REASON_BUSY (0x02 << 8) +#define S390_TOPO_FC_MASK 0xffUL +void s390_handle_ptf (S390CPU *cpu,uint8_t r1,uintptr_t ra); +struct S390CcwMachineClass {/*< private >*/ MachineClass parent_class;
hw/s390x/cpu-topology.c +54 -0
66 lines not on this page · raw message
target/s390x/kvm/kvm.c +11 -0
@@ -96,6 +96,7 @@#define PRIV_B9_EQBS 0x9c#define PRIV_B9_CLP 0xa0 +#define PRIV_B9_PTF 0xa2#define PRIV_B9_PCISTG 0xd0#define PRIV_B9_PCILG 0xd2#define PRIV_B9_RPCIT 0xd3
@@ -1464,6 +1465,13 @@ static int kvm_mpcifc_service_call(S390CPU *cpu, struct kvm_run *run) } } +static void kvm_handle_ptf (S390CPU *cpu,struct kvm_run *run) +{ +uint8_t r1 = (run->s390_sieic.ipb >>20 ) &0x0f ; + +s390_handle_ptf (cpu, r1,RA_IGNORED ); +} +static int handle_b9 (S390CPU *cpu,struct kvm_run *run,uint8_t ipa1) {int r =0 ;
@@ -1481,6 +1489,9 @@ static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)case PRIV_B9_RPCIT : r =kvm_rpcit_service_call (cpu, run);break ; +case PRIV_B9_PTF : +kvm_handle_ptf (cpu, run); +break ;case PRIV_B9_EQBS :/* just inject exception */ r =-1 ;
Signature · 1 line
2.31.1
Re: [PATCH v21 06/20] s390x/cpu topology: interception of PTF instruction
Quoted 11 lines · Pierre Morel
When the host supports the CPU topology facility, the PTF instruction with function code 2 is interpreted by the SIE, provided that the userland hypervisor activates the interpretation by using the
KVM_CAP_S390_CPU_TOPOLOGYKVM extension.The PTF instructions with function code 0 and 1 are intercepted and must be emulated by the userland hypervisor.
During RESET all CPU of the configuration are placed in horizontal polarity.
Quoted 7 lines
Signed-off-by Pierre Morel <[email protected]>Reviewed-by Nina Schoetterl-Glausch <[email protected]>--- include/hw/s390x/s390-virtio-ccw.h | 6 ++++ hw/s390x/cpu-topology.c | 54 ++++++++++++++++++++++++++++++ target/s390x/kvm/kvm.c | 11 ++++++ 3 files changed, 71 insertions(+)
Re: [PATCH v21 06/20] s390x/cpu topology: interception of PTF instruction
Quoted 21 lines · Thomas Huth
Quoted 11 lines · Pierre Morel
When the host supports the CPU topology facility, the PTF instruction with function code 2 is interpreted by the SIE, provided that the userland hypervisor activates the interpretation by using the
KVM_CAP_S390_CPU_TOPOLOGYKVM extension.The PTF instructions with function code 0 and 1 are intercepted and must be emulated by the userland hypervisor.
During RESET all CPU of the configuration are placed in horizontal polarity.
Quoted 7 lines
Signed-off-by Pierre Morel <[email protected]>Reviewed-by Nina Schoetterl-Glausch <[email protected]>---
include/hw/s390x/s390-virtio-ccw.h| 6 ++++ hw/s390x/cpu-topology.c | 54 ++++++++++++++++++++++++++++++ target/s390x/kvm/kvm.c | 11 ++++++3 files changed, 71 insertions(+)
Quoted 2 lines
Reviewed-by Thomas Huth <[email protected]>
Thanks,
Pierre
PATCH v21 7/20 target/s390x/cpu topology: activate CPU topology
In reply to Pierre MorelThe KVM capability KVM_CAP_S390_CPU_TOPOLOGY is used to activate the S390_FEAT_CONFIGURATION_TOPOLOGY feature and the topology facility in the host CPU model for the guest in the case the topology is available in QEMU and in KVM.
The feature is disabled by default and fenced for SE (secure execution).
--- hw/s390x/cpu-topology.c | 2 +- target/s390x/cpu_models.c | 1 + target/s390x/kvm/kvm.c | 9 +++++++++ 3 files changed, 11 insertions(+), 1 deletion(-)
hw/s390x/cpu-topology.c +1 -1
@@ -71,7 +71,7 @@ static int s390_socket_nb(S390CPU *cpu)*/ bool s390_has_topology (void ) { -return false; +return s390_has_feat (S390_FEAT_CONFIGURATION_TOPOLOGY ); }/**
target/s390x/cpu_models.c +1 -0
@@ -255,6 +255,7 @@ bool s390_has_feat(S390Feat feat)case S390_FEAT_SIE_CMMA :case S390_FEAT_SIE_PFMFI :case S390_FEAT_SIE_IBS : +case S390_FEAT_CONFIGURATION_TOPOLOGY :return false;break ;default :
target/s390x/kvm/kvm.c +9 -0
@@ -371,6 +371,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)kvm_vm_enable_cap (s,KVM_CAP_S390_USER_SIGP ,0 );kvm_vm_enable_cap (s,KVM_CAP_S390_VECTOR_REGISTERS ,0 );kvm_vm_enable_cap (s,KVM_CAP_S390_USER_STSI ,0 ); +kvm_vm_enable_cap (s,KVM_CAP_S390_CPU_TOPOLOGY ,0 );if (ri_allowed ()) {if (kvm_vm_enable_cap (s,KVM_CAP_S390_RI ,0 ) ==0 ) { cap_ri =1 ;
@@ -2470,6 +2471,14 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp)set_bit (S390_FEAT_UNPACK , model->features); } +/* +* If we have kernel support for CPU Topology indicate the +* configuration-topology facility. +*/ +if (kvm_check_extension (kvm_state,KVM_CAP_S390_CPU_TOPOLOGY )) { +set_bit (S390_FEAT_CONFIGURATION_TOPOLOGY , model->features); + } +/* We emulate a zPCI bus and AEN, therefore we don't need HW support */ set_bit (S390_FEAT_ZPCI , model->features);set_bit (S390_FEAT_ADAPTER_EVENT_NOTIFICATION , model->features);
Signature · 1 line
2.31.1
PATCH v21 8/20 qapi/s390x/cpu topology: set-cpu-topology qmp command
In reply to Pierre MorelThe modification of the CPU attributes are done through a monitor command.
It allows to move the core inside the topology tree to optimize the cache usage in the case the host's hypervisor previously moved the CPU.
The same command allows to modify the CPU attributes modifiers like polarization entitlement and the dedicated attribute to notify the guest if the host admin modified scheduling or dedication of a vCPU.
With this knowledge the guest has the possibility to optimize the usage of the vCPUs.
The command has a feature unstable for the moment.
--- qapi/machine-target.json | 37 +++++++++++ hw/s390x/cpu-topology.c | 134 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 171 insertions(+)
qapi/machine-target.json +37 -0
@@ -4,6 +4,8 @@ # This work is licensed under the terms of the GNU GPL, version 2 or later. # See the COPYING file in the top-level directory. +{ 'include': 'machine-common.json' } + ## # @CpuModelInfo: #
@@ -375,3 +377,38 @@ 'data': [ 'horizontal', 'vertical' ], 'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] } } + +## +# @set-cpu-topology: +# +# @core-id: the vCPU ID to be moved +# @socket-id: optional destination socket where to move the vCPU +# @book-id: optional destination book where to move the vCPU +# @drawer-id: optional destination drawer where to move the vCPU +# @entitlement: optional entitlement +# @dedicated: optional, if the vCPU is dedicated to a real CPU +# +# Features: +# @unstable: This command may still be modified. +# +# Modifies the topology by moving the CPU inside the topology +# tree or by changing a modifier attribute of a CPU. +# Default value for optional parameter is the current value +# used by the CPU. +# +# Returns: Nothing on success, the reason on failure. +# +# Since: 8.1 +## +{ 'command': 'set-cpu-topology', + 'data': { + 'core-id': 'uint16', + '*socket-id': 'uint16', + '*book-id': 'uint16', + '*drawer-id': 'uint16', + '*entitlement': 'CpuS390Entitlement', + '*dedicated': 'bool' + }, + 'features': [ 'unstable' ], + 'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] } +}
hw/s390x/cpu-topology.c +134 -0
149 lines not on this page · raw message
Signature · 1 line
2.31.1
Re: [PATCH v21 08/20] qapi/s390x/cpu topology: set-cpu-topology qmp command
Quoted 18 lines · Pierre Morel
The modification of the CPU attributes are done through a monitor command.
It allows to move the core inside the topology tree to optimize the cache usage in the case the host's hypervisor previously moved the CPU.
The same command allows to modify the CPU attributes modifiers like polarization entitlement and the dedicated attribute to notify the guest if the host admin modified scheduling or dedication of a vCPU.
With this knowledge the guest has the possibility to optimize the usage of the vCPUs.
The command has a feature unstable for the moment.
Signed-off-by Pierre Morel <[email protected]>---
...
Quoted 90 lines
hw/s390x/cpu-topology.c +68
-0@@ -23,6 +23,7 @@#include "target/s390x/cpu.h" #include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/cpu-topology.h" +#include "qapi/qapi-commands-machine-target.h" /* * s390_topology is used to keep the topology information. @@ -260,6 +261,29 @@ static bool s390_topology_check(uint16_t socket_id, uint16_t book_id,return true; } +/** +* s390_topology_need_report +* @cpu: Current cpu +* @drawer_id: future drawer ID +* @book_id: future book ID +* @socket_id: future socket ID +* @entitlement: future entitlement +* @dedicated: future dedicated +* +* A modified topology change report is needed if the topology +* tree or the topology attributes change. +*/ +static bool s390_topology_need_report (S390CPU *cpu,int drawer_id, +int book_id,int socket_id, +uint16_t entitlement,bool dedicated) +{ +return cpu->env.drawer_id != drawer_id || + cpu->env.book_id != book_id || + cpu->env.socket_id != socket_id || + cpu->env.entitlement != entitlement || + cpu->env.dedicated != dedicated; +} +/** * s390_update_cpu_props: * @ms: the machine state @@ -329,3 +353,113 @@ void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp)/* topology tree is reflected in props */ s390_update_cpu_props (ms, cpu); } + +static void s390_change_topology (uint16_t core_id, +bool has_socket_id,uint16_t socket_id, +bool has_book_id,uint16_t book_id, +bool has_drawer_id,uint16_t drawer_id, +bool has_entitlement, +CpuS390Entitlement entitlement, +bool has_dedicated,bool dedicated, +Error **errp) +{ +MachineState *ms = current_machine; +int old_socket_entry; +int new_socket_entry; +bool report_needed; +S390CPU *cpu; +ERRP_GUARD (); + + cpu =s390_cpu_addr2state (core_id); +if (!cpu) { +error_setg (errp,"Core-id %d does not exist!" , core_id); +return ; + } + +/* Get attributes not provided from cpu and verify the new topology */ +if (!has_socket_id) { + socket_id = cpu->env.socket_id; + } +if (!has_book_id) { + book_id = cpu->env.book_id; + } +if (!has_drawer_id) { + drawer_id = cpu->env.drawer_id; + } +if (!has_dedicated) { + dedicated = cpu->env.dedicated; + } + +/* +* When the user specifies the entitlement as 'auto' on the command line, +* qemu will set the entitlement as: +* Medium when the CPU is not dedicated. +* High when dedicated is true. +*/ +if (!has_entitlement || (entitlement ==S390_CPU_ENTITLEMENT_AUTO )) {
hw/s390x/cpu-topology.c void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp) · line 409
Quoted 10 lines
hw/s390x/cpu-topology.c +10
-0+if (dedicated) { + entitlement =S390_CPU_ENTITLEMENT_HIGH ; + }else { + entitlement =S390_CPU_ENTITLEMENT_MEDIUM ; + } + } + +if (!s390_topology_check (socket_id, book_id, drawer_id, + entitlement, dedicated, errp)) +return ;
Missing curly braces here.
Not sure why this isn't caught by checkpatch.pl properly :-(
Thomas
Quoted 55 lines
hw/s390x/cpu-topology.c +55
-0+/* Check for space on new socket */ + old_socket_entry =s390_socket_nb (cpu); + new_socket_entry =__s390_socket_nb (drawer_id, book_id, socket_id); + +if (new_socket_entry != old_socket_entry) { +if (s390_topology.cores_per_socket[new_socket_entry] >= + ms->smp.cores) { +error_setg (errp,"No more space on this socket" ); +return ; + } +/* Update the count of cores in sockets */ + s390_topology.cores_per_socket[new_socket_entry] +=1 ; + s390_topology.cores_per_socket[old_socket_entry] -=1 ; + } + +/* Check if we will need to report the modified topology */ + report_needed =s390_topology_need_report (cpu, drawer_id, book_id, + socket_id, entitlement, + dedicated); + +/* All checks done, report new topology into the vCPU */ + cpu->env.drawer_id = drawer_id; + cpu->env.book_id = book_id; + cpu->env.socket_id = socket_id; + cpu->env.dedicated = dedicated; + cpu->env.entitlement = entitlement; + +/* topology tree is reflected in props */ +s390_update_cpu_props (ms, cpu); + +/* Advertise the topology change */ +if (report_needed) { +s390_cpu_topology_set_changed (true); + } +} + +void qmp_set_cpu_topology (uint16_t core, +bool has_socket,uint16_t socket, +bool has_book,uint16_t book, +bool has_drawer,uint16_t drawer, +bool has_entitlement,CpuS390Entitlement entitlement, +bool has_dedicated,bool dedicated, +Error **errp) +{ +ERRP_GUARD (); + +if (!s390_has_topology ()) { +error_setg (errp,"This machine doesn't support topology" ); +return ; + } + +s390_change_topology (core, has_socket, socket, has_book, book, + has_drawer, drawer, has_entitlement, entitlement, + has_dedicated, dedicated, errp); +}
Re: [PATCH v21 08/20] qapi/s390x/cpu topology: set-cpu-topology qmp command
Quoted 130 lines · Thomas Huth
Quoted 18 lines · Pierre Morel
The modification of the CPU attributes are done through a monitor command.
It allows to move the core inside the topology tree to optimize the cache usage in the case the host's hypervisor previously moved the CPU.
The same command allows to modify the CPU attributes modifiers like polarization entitlement and the dedicated attribute to notify the guest if the host admin modified scheduling or dedication of a vCPU.
With this knowledge the guest has the possibility to optimize the usage of the vCPUs.
The command has a feature unstable for the moment.
Signed-off-by Pierre Morel <[email protected]>---
...
Quoted 98 lines
hw/s390x/cpu-topology.c +1
-0@@ -23,6 +23,7 @@ #include "target/s390x/cpu.h" #include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/cpu-topology.h" +#include "qapi/qapi-commands-machine-target.h" /** s390_topology is used to keep the topology information. @@ -260,6 +261,29 @@ static bool s390_topology_check(uint16_t socket_id, uint16_t book_id, return true; }+/**
+ *s390_topology_need_report
+ * @cpu: Current cpu
+ * @drawer_id: future drawer ID
+ * @book_id: future book ID
+ * @socket_id: future socket ID
+ * @entitlement: future entitlement
+ * @dedicated: future dedicated
+ *
+ * A modified topology change report is needed if the topology
+ * tree or the topology attributes change.
+ */
+static bool s390_topology_need_report(S390CPU *cpu, intdrawer_id, + intbook_id, intsocket_id, +uint16_tentitlement, bool dedicated)hw/s390x/cpu-topology.c +8
-0+{ + return cpu->env.drawer_id != drawer_id || + cpu->env.book_id != book_id || + cpu->env.socket_id != socket_id || + cpu->env.entitlement != entitlement || + cpu->env.dedicated != dedicated; +} +/**
*s390_update_cpu_props:
* @ms: the machine statehw/s390x/cpu-topology.c +0
-0@@ -329,3 +353,113 @@ void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp) /* topology tree is reflected in props */ s390_update_cpu_props(ms, cpu);}
hw/s390x/cpu-topology.c +3
-0+ +static void s390_change_topology (uint16_t core_id, + bool has_socket_id, uint16_t
socket_id,
+ boolhas_book_id,uint16_tbook_id, + boolhas_drawer_id,uint16_tdrawer_id,
+ boolhas_entitlement,
+ CpuS390Entitlement entitlement, + boolhas_dedicated, bool dedicated, + Error **errp)hw/s390x/cpu-topology.c +15
-0+{ + MachineState *ms = current_machine; + int old_socket_entry; + int new_socket_entry; + bool report_needed; + S390CPU *cpu; + ERRP_GUARD(); + + cpu = s390_cpu_addr2state(core_id); + if (!cpu) { + error_setg(errp, "Core-id %d does not exist!", core_id); + return; + } + + /* Get attributes not provided from cpu and verify the newtopology */
+ if (!has_socket_id) {
+socket_id=cpu->env.socket_id;
+ }
+ if (!has_book_id) {
+book_id=cpu->env.book_id;
+ }
+ if (!has_drawer_id) {
+drawer_id=cpu->env.drawer_id;
+ }
+ if (!has_dedicated) {
+ dedicated =cpu->env.dedicated;
+ }
+
+ /*
+ * When the user specifies the entitlement as 'auto' on the command line,
+ * qemu will set the entitlement as:
+ * Medium when the CPU is not dedicated.
+ * High when dedicated is true.
+ */
+ if (!has_entitlement || (entitlement ==S390_CPU_ENTITLEMENT_AUTO)) {
+ if (dedicated) {
+ entitlement =S390_CPU_ENTITLEMENT_HIGH;Quoted 8 lines
+ } else {
+ entitlement =S390_CPU_ENTITLEMENT_MEDIUM; + }
+ }
+
+ if (!s390_topology_check(socket_id,book_id,drawer_id, + entitlement, dedicated, errp)) + return;Missing curly braces here.
Not sure why this isn't caught by checkpatch.pl properly :-(
Quoted 2 lines
Thomas
Yes indeed.
I add the curly braces.
Thanks
Pierre
Re: [PATCH v21 08/20] qapi/s390x/cpu topology: set-cpu-topology qmp command
In reply to Pierre MorelQuoted 12 lines · Pierre Morel
The modification of the CPU attributes are done through a monitor command.
It allows to move the core inside the topology tree to optimize the cache usage in the case the host's hypervisor previously moved the CPU.
The same command allows to modify the CPU attributes modifiers like polarization entitlement and the dedicated attribute to notify the guest if the host admin modified scheduling or dedication of a vCPU.
Quoted 6 lines
With this knowledge the guest has the possibility to optimize the usage of the vCPUs.
The command has a feature unstable for the moment.
Signed-off-by Pierre Morel <[email protected]>
Re: [PATCH v21 08/20] qapi/s390x/cpu topology: set-cpu-topology qmp command
Quoted 13 lines · Nina Schoetterl-Glausch
Quoted 12 lines · Pierre Morel
The modification of the CPU attributes are done through a monitor command.
It allows to move the core inside the topology tree to optimize the cache usage in the case the host's hypervisor previously moved the CPU.
The same command allows to modify the CPU attributes modifiers like polarization entitlement and the dedicated attribute to notify the guest if the host admin modified scheduling or dedication of a vCPU.
Quoted 7 lines
Quoted 6 lines
With this knowledge the guest has the possibility to optimize the usage of the vCPUs.
The command has a feature unstable for the moment.
Signed-off-by Pierre Morel <[email protected]>Reviewed-by Nina Schoetterl-Glausch <[email protected]>
Thanks,
Pierre
PATCH v21 9/20 machine: adding s390 topology to query-cpu-fast
In reply to Pierre MorelS390x provides two more topology attributes, entitlement and dedication.
Let's add these CPU attributes to the QAPI command query-cpu-fast.
--- qapi/machine.json | 9 ++++++++- target/s390x/cpu.c | 4 ++++ 2 files changed, 12 insertions(+), 1 deletion(-)
qapi/machine.json +8 -1
@@ -56,10 +56,17 @@ # Additional information about a virtual S390 CPU # # @cpu-state: the virtual CPU's state +# @dedicated: the virtual CPU's dedication (since 8.1) +# @entitlement: the virtual CPU's entitlement (since 8.1) # # Since: 2.12 ## -{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } } +{ 'struct': 'CpuInfoS390', + 'data': { 'cpu-state': 'CpuS390State', + 'dedicated': 'bool', + 'entitlement': 'CpuS390Entitlement' + } +} ## # @CpuInfoFast:
target/s390x/cpu.c +4 -0
@@ -146,6 +146,10 @@ static void s390_query_cpu_fast(CPUState *cpu, CpuInfoFast *value)S390CPU *s390_cpu =S390_CPU (cpu); value->u.s390x.cpu_state = s390_cpu->env.cpu_state; +#if !defined(CONFIG_USER_ONLY ) + value->u.s390x.dedicated = s390_cpu->env.dedicated; + value->u.s390x.entitlement = s390_cpu->env.entitlement; +#endif }/* S390CPUClass::reset() */
Signature · 1 line
2.31.1
Re: [PATCH v21 09/20] machine: adding s390 topology to query-cpu-fast
Quoted 18 lines · Pierre Morel
S390x provides two more topology attributes, entitlement and dedication.
Let's add these CPU attributes to the QAPI command query-cpu-fast.
Signed-off-by Pierre Morel <[email protected]>--- qapi/machine.json | 9 ++++++++- target/s390x/cpu.c | 4 ++++ 2 files changed, 12 insertions(+), 1 deletion(-)qapi/machine.json +0
-0@@ -56,10 +56,17 @@ # Additional information about a virtual S390 CPU # # @cpu-state: the virtual CPU's state
qapi/machine.json line 67
Quoted 10 lines
qapi/machine.json +6
-1+# @dedicated: the virtual CPU's dedication (since 8.1) +# @entitlement: the virtual CPU's entitlement (since 8.1) # # Since: 2.12 ## -{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } } +{ 'struct': 'CpuInfoS390', + 'data': { 'cpu-state': 'CpuS390State', + 'dedicated': 'bool', + 'entitlement': 'CpuS390Entitlement'
Would it make sense to make them optional and only report those if the topology feature is enabled?
Thomas
Quoted 20 lines
qapi/machine.json +2
-0+ } +} ## # @CpuInfoFast:target/s390x/cpu.c +4
-0@@ -146,6 +146,10 @@ static void s390_query_cpu_fast(CPUState *cpu, CpuInfoFast *value)S390CPU *s390_cpu =S390_CPU (cpu); value->u.s390x.cpu_state = s390_cpu->env.cpu_state; +#if !defined(CONFIG_USER_ONLY ) + value->u.s390x.dedicated = s390_cpu->env.dedicated; + value->u.s390x.entitlement = s390_cpu->env.entitlement; +#endif }/* S390CPUClass::reset() */
Re: [PATCH v21 09/20] machine: adding s390 topology to query-cpu-fast
Quoted 32 lines · Thomas Huth
Quoted 18 lines · Pierre Morel
S390x provides two more topology attributes, entitlement and dedication.
Let's add these CPU attributes to the QAPI command query-cpu-fast.
Signed-off-by Pierre Morel <[email protected]>---
qapi/machine.json | 9 ++++++++-
target/s390x/cpu.c| 4 ++++2 files changed, 12 insertions(+), 1 deletion(-)qapi/machine.json +0
-0@@ -56,10 +56,17 @@ # Additional information about a virtual S390 CPU # # @cpu-state: the virtual CPU's stateqapi/machine.json line 67
Quoted 10 lines
qapi/machine.json +6
-1+# @dedicated: the virtual CPU's dedication (since 8.1) +# @entitlement: the virtual CPU's entitlement (since 8.1) # # Since: 2.12 ## -{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } } +{ 'struct': 'CpuInfoS390', + 'data': { 'cpu-state': 'CpuS390State', + 'dedicated': 'bool', + 'entitlement': 'CpuS390Entitlement'Would it make sense to make them optional and only report those if the topology feature is enabled?
Quoted 2 lines
Thomas
I think you are right, I make them optional, it has no sense if the architecture does support topology.
Thanks,
Pierre
Quoted 20 lines
Quoted 19 lines
qapi/machine.json +2
-0+ } +} ## # @CpuInfoFast:target/s390x/cpu.c +4
-0@@ -146,6 +146,10 @@ static void s390_query_cpu_fast(CPUState *cpu, CpuInfoFast *value) S390CPU *s390_cpu = S390_CPU(cpu); value->u.s390x.cpu_state = s390_cpu->env.cpu_state; +#if !defined(CONFIG_USER_ONLY ) + value->u.s390x.dedicated = s390_cpu->env.dedicated; + value->u.s390x.entitlement = s390_cpu->env.entitlement; +#endif } /* S390CPUClass::reset() */
PATCH v21 10/20 machine: adding s390 topology to info hotpluggable-cpus
In reply to Pierre MorelS390 topology adds books and drawers topology containers. Let's add these to the HMP information for hotpluggable cpus.
--- hw/core/machine-hmp-cmds.c | 6 ++++++ 1 file changed, 6 insertions(+)
hw/core/machine-hmp-cmds.c +6 -0
@@ -71,6 +71,12 @@ void hmp_hotpluggable_cpus(Monitor *mon, const QDict *qdict)if (c->has_node_id) {monitor_printf (mon," node-id: \"%" PRIu64"\"\n" , c->node_id); } +if (c->has_drawer_id) { +monitor_printf (mon," drawer-id: \"%" PRIu64"\"\n" , c->drawer_id); + } +if (c->has_book_id) { +monitor_printf (mon," book-id: \"%" PRIu64"\"\n" , c->book_id); + }if (c->has_socket_id) {monitor_printf (mon," socket-id: \"%" PRIu64"\"\n" , c->socket_id); }
Signature · 1 line
2.31.1
Re: [PATCH v21 10/20] machine: adding s390 topology to info hotpluggable-cpus
Quoted 14 lines · Pierre Morel
S390 topology adds books and drawers topology containers. Let's add these to the HMP information for hotpluggable cpus.
Signed-off-by Pierre Morel <[email protected]>Reviewed-by Nina Schoetterl-Glausch <[email protected]>--- hw/core/machine-hmp-cmds.c | 6 ++++++ 1 file changed, 6 insertions(+)hw/core/machine-hmp-cmds.c +0
-0
hw/core/machine-hmp-cmds.c void hmp_hotpluggable_cpus(Monitor *mon, const QDict *qdict) · line 78
Quoted 8 lines
hw/core/machine-hmp-cmds.c +5
-0@@ -71,6 +71,12 @@ void hmp_hotpluggable_cpus(Monitor *mon, const QDict *qdict)if (c->has_node_id) {monitor_printf (mon," node-id: \"%" PRIu64"\"\n" , c->node_id); } +if (c->has_drawer_id) { +monitor_printf (mon," drawer-id: \"%" PRIu64"\"\n" , c->drawer_id); + } +if (c->has_book_id) { +monitor_printf (mon," book-id: \"%" PRIu64"\"\n" , c->book_id);
I think the output should be left-aligned (with four spaces at the beginning), not right aligned to the colons?
Thomas
Quoted 4 lines
hw/core/machine-hmp-cmds.c +1
-0+ }if (c->has_socket_id) {monitor_printf (mon," socket-id: \"%" PRIu64"\"\n" , c->socket_id); }
Re: [PATCH v21 10/20] machine: adding s390 topology to info hotpluggable-cpus
Quoted 29 lines · Thomas Huth
Quoted 15 lines · Pierre Morel
S390 topology adds books and drawers topology containers. Let's add these to the HMP information for hotpluggable cpus.
Signed-off-by Pierre Morel <[email protected]>Reviewed-by Nina Schoetterl-Glausch <[email protected]>---
hw/core/machine-hmp-cmds.c| 6 ++++++1 file changed, 6 insertions(+)hw/core/machine-hmp-cmds.c +0
-0@@ -71,6 +71,12 @@ void hmp_hotpluggable_cpus(Monitor *mon, const QDict *qdict)hw/core/machine-hmp-cmds.c void hmp_hotpluggable_cpus(Monitor *mon, const · line 81
Quoted 10 lines
hw/core/machine-hmp-cmds.c +5
-0if (c->has_node_id) { monitor_printf(mon, " node-id: \"%" PRIu64 "\"\n", c->node_id); } + if (c->has_drawer_id) { + monitor_printf(mon, " drawer-id: \"%" PRIu64 "\"\n", c->drawer_id); + } + if (c->has_book_id) { + monitor_printf(mon, " book-id: \"%" PRIu64 "\"\n", c->book_id);I think the output should be left-aligned (with four spaces at the beginning), not right aligned to the colons?
Quoted 2 lines
Thomas
I think you are right, core-id is not aligned on the colons.
thanks,
Pierre
Quoted 7 lines
Quoted 5 lines
hw/core/machine-hmp-cmds.c +1
-0+ }if (
c->has_socket_id) {
monitor_printf(mon, " socket-id: \"%" PRIu64 "\"\n", c->socket_id); }
PATCH v21 11/20 qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event
In reply to Pierre MorelWhen the guest asks to change the polarization this change is forwarded to the upper layer using QAPI.
The upper layer is supposed to take according decisions concerning CPU provisioning.
--- qapi/machine-target.json | 33 +++++++++++++++++++++++++++++++++ hw/s390x/cpu-topology.c | 2 ++ 2 files changed, 35 insertions(+)
qapi/machine-target.json +33 -0
@@ -412,3 +412,36 @@ 'features': [ 'unstable' ], 'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] } } + +## +# @CPU_POLARIZATION_CHANGE: +# +# Emitted when the guest asks to change the polarization. +# +# @polarization: polarization specified by the guest +# +# Features: +# @unstable: This command may still be modified. +# +# The guest can tell the host (via the PTF instruction) whether the +# CPUs should be provisioned using horizontal or vertical polarization. +# +# On horizontal polarization the host is expected to provision all vCPUs +# equally. +# On vertical polarization the host can provision each vCPU differently. +# The guest will get information on the details of the provisioning +# the next time it uses the STSI(15) instruction. +# +# Since: 8.1 +# +# Example: +# +# <- { "event": "CPU_POLARIZATION_CHANGE", +# "data": { "polarization": "horizontal" }, +# "timestamp": { "seconds": 1401385907, "microseconds": 422329 } } +## +{ 'event': 'CPU_POLARIZATION_CHANGE', + 'data': { 'polarization': 'CpuS390Polarization' }, + 'features': [ 'unstable' ], + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] } +}
hw/s390x/cpu-topology.c +2 -0
@@ -24,6 +24,7 @@#include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/cpu-topology.h" #include "qapi/qapi-commands-machine-target.h" +#include "qapi/qapi-events-machine-target.h" /* * s390_topology is used to keep the topology information.
@@ -137,6 +138,7 @@ void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra) }else { s390_topology.polarization = polarization;s390_cpu_topology_set_changed (true); +qapi_event_send_cpu_polarization_change (polarization);setcc (cpu,0 ); }break ;
Signature · 1 line
2.31.1
Re: [PATCH v21 11/20] qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event
Quoted 10 lines · Pierre Morel
When the guest asks to change the polarization this change is forwarded to the upper layer using QAPI.
The upper layer is supposed to take according decisions concerning CPU provisioning.Signed-off-by Pierre Morel <[email protected]>--- qapi/machine-target.json | 33 +++++++++++++++++++++++++++++++++ hw/s390x/cpu-topology.c | 2 ++ 2 files changed, 35 insertions(+)
Re: [PATCH v21 11/20] qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event
Quoted 6 lines · Thomas Huth
Quoted 5 lines · Pierre Morel
When the guest asks to change the polarization this change is forwarded to the upper layer using QAPI.
The upper layer is supposed to take according decisions concerning CPU provisioning.
Quoted 8 lines
Quoted 5 lines
Signed-off-by Pierre Morel <[email protected]>---
qapi/machine-target.json| 33 +++++++++++++++++++++++++++++++++ hw/s390x/cpu-topology.c | 2 ++2 files changed, 35 insertions(+)Reviewed-by Thomas Huth <[email protected]>
Thanks,
Pierre
Re: [PATCH v21 11/20] qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event
In reply to Pierre MorelQuoted 6 lines · Pierre Morel
When the guest asks to change the polarization this change is forwarded to the upper layer using QAPI.
The upper layer is supposed to take according decisions concerning CPU provisioning.Signed-off-by Pierre Morel <[email protected]>
Re: [PATCH v21 11/20] qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event
Quoted 9 lines · Nina Schoetterl-Glausch
Quoted 6 lines · Pierre Morel
When the guest asks to change the polarization this change is forwarded to the upper layer using QAPI.
The upper layer is supposed to take according decisions concerning CPU provisioning.Signed-off-by Pierre Morel <[email protected]>Reviewed-by Nina Schoetterl-Glausch <[email protected]>
Thanks,
Pierre
PATCH v21 12/20 qapi/s390x/cpu topology: query-cpu-polarization qmp command
In reply to Pierre MorelThe query-cpu-polarization qmp command returns the current CPU polarization of the machine.
--- qapi/machine-target.json | 29 +++++++++++++++++++++++++++++ hw/s390x/cpu-topology.c | 8 ++++++++ 2 files changed, 37 insertions(+)
qapi/machine-target.json +29 -0
@@ -445,3 +445,32 @@ 'features': [ 'unstable' ], 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] } } + +## +# @CpuPolarizationInfo: +# +# The result of a cpu polarization +# +# @polarization: the CPU polarization +# +# Since: 8.1 +## +{ 'struct': 'CpuPolarizationInfo', + 'data': { 'polarization': 'CpuS390Polarization' }, + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] } +} + +## +# @query-cpu-polarization: +# +# Features: +# @unstable: This command may still be modified. +# +# Returns: the machine polarization +# +# Since: 8.1 +## +{ 'command': 'query-cpu-polarization', 'returns': 'CpuPolarizationInfo', + 'features': [ 'unstable' ], + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] } +}
hw/s390x/cpu-topology.c +8 -0
@@ -465,3 +465,11 @@ void qmp_set_cpu_topology(uint16_t core, has_drawer, drawer, has_entitlement, entitlement, has_dedicated, dedicated, errp); } + +CpuPolarizationInfo *qmp_query_cpu_polarization (Error **errp) +{ +CpuPolarizationInfo *info =g_new0 (CpuPolarizationInfo,1 ); + + info->polarization = s390_topology.polarization; +return info; +}
Signature · 1 line
2.31.1
Re: [PATCH v21 12/20] qapi/s390x/cpu topology: query-cpu-polarization qmp command
Quoted 33 lines · Pierre Morel
The query-cpu-polarization qmp command returns the current CPU polarization of the machine.
Signed-off-by Pierre Morel <[email protected]>--- qapi/machine-target.json | 29 +++++++++++++++++++++++++++++ hw/s390x/cpu-topology.c | 8 ++++++++ 2 files changed, 37 insertions(+)qapi/machine-target.json +16
-0@@ -445,3 +445,32 @@ 'features': [ 'unstable' ], 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] } } + +## +# @CpuPolarizationInfo: +# +# The result of a cpu polarization +# +# @polarization: the CPU polarization +# +# Since: 8.1 +## +{ 'struct': 'CpuPolarizationInfo', + 'data': { 'polarization': 'CpuS390Polarization' }, + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] } +} + +##
qapi/machine-target.json line 473
Quoted 10 lines
qapi/machine-target.json +10
-0+# @query-cpu-polarization: +# +# Features: +# @unstable: This command may still be modified. +# +# Returns: the machine polarization +# +# Since: 8.1 +## +{ 'command': 'query-cpu-polarization', 'returns': 'CpuPolarizationInfo',
Since this is very specific to s390x, I wonder whether we want to have a "s390x" in the command name? 'query-s390x-cpu-polarization'? ... or is this getting too long already?
Anyway,
Re: [PATCH v21 12/20] qapi/s390x/cpu topology: query-cpu-polarization qmp command
Quoted 48 lines · Thomas Huth
Quoted 43 lines · Pierre Morel
The query-cpu-polarization qmp command returns the current CPU polarization of the machine.
Signed-off-by Pierre Morel <[email protected]>---
qapi/machine-target.json| 29 +++++++++++++++++++++++++++++ hw/s390x/cpu-topology.c | 8 ++++++++2 files changed, 37 insertions(+)qapi/machine-target.json +26
-0@@ -445,3 +445,32 @@ 'features': [ 'unstable' ], 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] } } + +## +# @CpuPolarizationInfo: +# +# The result of a cpu polarization +# +# @polarization: the CPU polarization +# +# Since: 8.1 +## +{ 'struct': 'CpuPolarizationInfo', + 'data': { 'polarization': 'CpuS390Polarization' }, + 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] } +} + +## +# @query-cpu-polarization: +# +# Features: +# @unstable: This command may still be modified. +# +# Returns: the machine polarization +# +# Since: 8.1 +## +{ 'command': 'query-cpu-polarization', 'returns':Quoted 1 line
'CpuPolarizationInfo',
Since this is very specific to s390x, I wonder whether we want to have a "s390x" in the command name? 'query-s390x-cpu-polarization'? ... or is this getting too long already?
Quoted 3 lines
Anyway,
Reviewed-by Thomas Huth <[email protected]>
I do not know.I prefer short commands but the interface will mostly be used by a program, so...
I let it like that unless there is more pressure to change it.
Thanks,
Pierre
Re: [PATCH v21 12/20] qapi/s390x/cpu topology: query-cpu-polarization qmp command
In reply to Pierre MorelQuoted 4 lines · Pierre Morel
The query-cpu-polarization qmp command returns the current CPU polarization of the machine.
Signed-off-by Pierre Morel <[email protected]>
[...]
PATCH v21 13/20 docs/s390x/cpu topology: document s390x cpu topology
In reply to Pierre MorelAdd some basic examples for the definition of cpu topology in s390x.
--- MAINTAINERS | 2 + docs/devel/index-internals.rst | 1 + docs/devel/s390-cpu-topology.rst | 170 ++++++++++++++++++++ docs/system/s390x/cpu-topology.rst | 240 +++++++++++++++++++++++++++++ docs/system/target-s390x.rst | 1 + 5 files changed, 414 insertions(+) create mode 100644 docs/devel/s390-cpu-topology.rst create mode 100644 docs/system/s390x/cpu-topology.rst
MAINTAINERS +2 -0
@@ -1703,6 +1703,8 @@ S: Supported F: include/hw/s390x/cpu-topology.h F: hw/s390x/cpu-topology.c F: target/s390x/kvm/stsi-topology.c +F: docs/devel/s390-cpu-topology.rst +F: docs/system/s390x/cpu-topology.rst X86 Machines ------------
docs/devel/index-internals.rst +1 -0
@@ -14,6 +14,7 @@ Details about QEMU's various subsystems including how to add features to them. migration multi-process reset + s390-cpu-topology s390-dasd-ipl tracing vfio-migration
docs/devel/s390-cpu-topology.rst new file +170 -0
170 lines not on this page · raw message
docs/system/s390x/cpu-topology.rst new file +240 -0
240 lines not on this page · raw message
docs/system/target-s390x.rst +1 -0
@@ -34,3 +34,4 @@ Architectural features .. toctree:: s390x/bootdevices s390x/protvirt + s390x/cpu-topology
Signature · 1 line
2.31.1
Re: [PATCH v21 13/20] docs/s390x/cpu topology: document s390x cpu topology
Quoted 45 lines · Pierre Morel
Add some basic examples for the definition of cpu topology in s390x.
Signed-off-by Pierre Morel <[email protected]>--- MAINTAINERS | 2 + docs/devel/index-internals.rst | 1 + docs/devel/s390-cpu-topology.rst | 170 ++++++++++++++++++++ docs/system/s390x/cpu-topology.rst | 240 +++++++++++++++++++++++++++++ docs/system/target-s390x.rst | 1 + 5 files changed, 414 insertions(+)create mode 100644
docs/devel/s390-cpu-topology.rstcreate mode 100644docs/system/s390x/cpu-topology.rstMAINTAINERS +2
-0@@ -1703,6 +1703,8 @@ S: Supported F: include/hw/s390x/cpu-topology.h F: hw/s390x/cpu-topology.c F: target/s390x/kvm/stsi-topology.c +F: docs/devel/s390-cpu-topology.rst +F: docs/system/s390x/cpu-topology.rst X86 Machines ------------docs/devel/index-internals.rst +1
-0@@ -14,6 +14,7 @@ Details about QEMU's various subsystems including how to add features to them. migration multi-process reset + s390-cpu-topology s390-dasd-ipl tracing vfio-migrationdocs/devel/s390-cpu-topology.rst new file +0
-0
docs/devel/s390-cpu-topology.rst line 4
Quoted 4 lines
docs/devel/s390-cpu-topology.rst new file +4
-0@@ -0,0 +1,170 @@ +QAPI interface for S390 CPU topology +==================================== + +Let's start QEMU with the following command defining 4 CPUs,
Maybe better something like this:
The following sections will explain the S390 CPU topology with the help of exemplary output. For this, let's assume that QEMU has been started with the following command, defining 4 CPUs.
?
Quoted 15 lines
docs/devel/s390-cpu-topology.rst new file +15
-0+CPU[0] defined by the -smp argument will have default values: + +.. code-block:: bash + + qemu-system-s390x \ + -enable-kvm \ + -cpu z14,ctop=on \ + -smp 1,drawers=3,books=3,sockets=2,cores=2,maxcpus=36 \ + \ + -device z14-s390x-cpu,core-id=19,entitlement=high \ + -device z14-s390x-cpu,core-id=11,entitlement=low \ + -device z14-s390x-cpu,core-id=112,entitlement=high \ + ... + +and see the result when using the QAPI interface.
...
Quoted 17 lines
docs/devel/s390-cpu-topology.rst new file +17
-0+QAPI command: set-cpu-topology +------------------------------ + +The command set-cpu-topology allows to modify the topology tree +or the topology modifiers of a vCPU in the configuration. + +.. code-block:: QMP + + { "execute": "set-cpu-topology", + "arguments": { + "core-id": 11, + "socket-id": 0, + "book-id": 0, + "drawer-id": 0, + "entitlement": "low", + "dedicated": false + }
docs/devel/s390-cpu-topology.rst line 122
Quoted 10 lines
docs/devel/s390-cpu-topology.rst new file +10
-0+ } + {"return": {}} + +The core-id parameter is the only non optional parameter and every +unspecified parameter keeps its previous value. + +QAPI event CPU_POLARIZATION_CHANGE +---------------------------------- + +When a guest is requests a modification of the polarization,
Scratch the word "is".
Quoted 1 line
+QEMU sends a
CPU_POLARIZATION_CHANGEevent.
...
docs/system/s390x/cpu-topology.rst line 6
Quoted 12 lines
docs/system/s390x/cpu-topology.rst new file +6
-0@@ -0,0 +1,240 @@ +CPU topology on s390x +===================== + +Since QEMU 8.1, CPU topology on s390x provides up to 3 levels of +topology containers: drawers, books, sockets, defining a tree shaped +hierarchy.
"drawers, books and sockets. They define a tree-shaped hierarchy."
?
docs/system/s390x/cpu-topology.rst line 8
Quoted 1 line
docs/system/s390x/cpu-topology.rst new file +1
-0+The socket container contains one or more CPU entries.
"The socket container has one or more CPU entries." ?
docs/system/s390x/cpu-topology.rst line 21
Quoted 13 lines
docs/system/s390x/cpu-topology.rst new file +13
-0+Each of these CPU entries consists of a bitmap and three CPU attributes: + +- CPU type +- entitlement +- dedication + +Each bit set in the bitmap correspond to the core-id of a vCPU with +matching the three attribute. + +This documentation provide general information on S390 CPU topology, +how to enable it and on the new CPU attributes. +For information on how to modify the S390 CPU topology and on how to +monitor the polarization change see ``Developer Information``.
It would be nicer to have a proper link here instead. See commit d6359e150dbdf84f67add786473fd277a9a442bb for example how to do this in our .rst files.
Quoted 25 lines
docs/system/s390x/cpu-topology.rst new file +25
-0+Prerequisites +------------- + +To use the CPU topology, you need to run with KVM on a s390x host that +uses the Linux kernel v6.0 or newer (which provide the so-called +``KVM_CAP_S390_CPU_TOPOLOGY`` capability that allows QEMU to signal the +CPU topology facility via the so-called STFLE bit 11 to the VM). + +Enabling CPU topology +--------------------- + +Currently, CPU topology is only enabled in the host model by default. + +Enabling CPU topology in a CPU model is done by setting the CPU flag +``ctop`` to ``on`` like in: + +.. code-block:: bash + + -cpu gen16b,ctop=on + +Having the topology disabled by default allows migration between +old and new QEMU without adding new flags. + +Default topology usage +----------------------
docs/system/s390x/cpu-topology.rst line 57
Quoted 10 lines
docs/system/s390x/cpu-topology.rst new file +10
-0+ +The CPU topology can be specified on the QEMU command line +with the ``-smp`` or the ``-device`` QEMU command arguments. + +Note also that since 7.2 threads are no longer supported in the topology +and the ``-smp`` command line argument accepts only ``threads=1``. + +If none of the containers attributes (drawers, books, sockets) are +specified for the ``-smp`` flag, the number of these containers +is ``1`` .
"Thus the following two options will result in the same topology, for example:" ?
Quoted 29 lines
docs/system/s390x/cpu-topology.rst new file +29
-0+.. code-block:: bash + + -smp cpus=5,drawer=1,books=1,sockets=8,cores=4,maxcpus=32 + +or + +.. code-block:: bash + + -smp cpus=5,sockets=8,cores=4,maxcpus=32 + +When a CPU is defined by the ``-smp`` command argument, its position +inside the topology is calculated by adding the CPUs to the topology +based on the core-id starting with core-0 at position 0 of socket-0, +book-0, drawer-0 and filling all CPUs of socket-0 before to fill socket-1 +of book-0 and so on up to the last socket of the last book of the last +drawer. + +When a CPU is defined by the ``-device`` command argument, the +tree topology attributes must be all defined or all not defined. + +.. code-block:: bash + + -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1 + +or + +.. code-block:: bash + + -device gen16b-s390x-cpu,core-id=1,dedicated=true
docs/system/s390x/cpu-topology.rst line 97
Quoted 10 lines
docs/system/s390x/cpu-topology.rst new file +10
-0+ +If none of the tree attributes (drawer, book, sockets), are specified +for the ``-device`` argument, as for all CPUs defined with the ``-smp`` +command argument the topology tree attributes will be set by simply +adding the CPUs to the topology based on the core-id starting with +core-0 at position 0 of socket-0, book-0, drawer-0. + +QEMU will not try to solve collisions and will report an error if the +CPU topology, explicitly or implicitly defined on a ``-device`` +argument collides with the definition of a CPU implicitely defined
s/implicitely/implicitly/
Quoted 60 lines
docs/system/s390x/cpu-topology.rst new file +60
-0+on the ``-smp`` argument. + +When the topology modifier attributes are not defined for the +``-device`` command argument they takes following default values: + +- dedicated: ``false`` +- entitlement: ``medium`` + + +Hot plug +++++++++ + +New CPUs can be plugged using the device_add hmp command as in: + +.. code-block:: bash + + (qemu) device_add gen16b-s390x-cpu,core-id=9 + +The same placement of the CPU is derived from the core-id as described above. + +The topology can of course be fully defined: + +.. code-block:: bash + + (qemu) device_add gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1 + + +Examples +++++++++ + +In the following machine we define 8 sockets with 4 cores each. + +.. code-block:: bash + + $ qemu-system-s390x -m 2G \ + -cpu gen16b,ctop=on \ + -smp cpus=5,sockets=8,cores=4,maxcpus=32 \ + -device host-s390x-cpu,core-id=14 \ + +A new CPUs can be plugged using the device_add hmp command as before: + +.. code-block:: bash + + (qemu) device_add gen16b-s390x-cpu,core-id=9 + +The core-id defines the placement of the core in the topology by +starting with core 0 in socket 0 up to maxcpus. + +In the example above: + +* There are 5 CPUs provided to the guest with the ``-smp`` command line + They will take the core-ids 0,1,2,3,4 + As we have 4 cores in a socket, we have 4 CPUs provided + to the guest in socket 0, with core-ids 0,1,2,3. + The last cpu, with core-id 4, will be on socket 1. + +* the core with ID 14 provided by the ``-device`` command line will + be placed in socket 3, with core-id 14 + +* the core with ID 9 provided by the ``device_add`` qmp command will
docs/system/s390x/cpu-topology.rst line 167
Quoted 10 lines
docs/system/s390x/cpu-topology.rst new file +10
-0+ be placed in socket 2, with core-id 9 + + +Polarization, entitlement and dedication +---------------------------------------- + +Polarization +++++++++++++ + +The polarization is an indication given by the ``guest`` to the host
Why quoting guest, but not host? I'd remove the quotes from guest here.
docs/system/s390x/cpu-topology.rst line 171
Quoted 4 lines
docs/system/s390x/cpu-topology.rst new file +4
-0+that it is able to make use of CPU provisioning information. +The guest indicates the polarization by using the PTF instruction. + +Polarization is define two models of CPU provisioning: horizontal
"Polarization defines ..." ? ... or "Polarization is defined by ..." ?
docs/system/s390x/cpu-topology.rst line 175
Quoted 4 lines
docs/system/s390x/cpu-topology.rst new file +4
-0+and vertical. + +The horizontal polarization is the default model on boot and after +subsystem reset in which the guest considers all vCPUs being having
scratch "being" ?
Quoted 19 lines
docs/system/s390x/cpu-topology.rst new file +19
-0+an equal provisioning of CPUs by the host. + +In the vertical polarization model the guest can make use of the +vCPU entitlement information provided by the host to optimize +kernel thread scheduling. + +A subsystem reset puts all vCPU of the configuration into the +horizontal polarization. + +Entitlement ++++++++++++ + +The vertical polarization specifies that the guest's vCPU can get +different real CPU provisions: + +- a vCPU with vertical high entitlement specifies that this + vCPU gets 100% of the real CPU provisioning. + +- a vCPU with vertical medium entitlement specifies that this
docs/system/s390x/cpu-topology.rst line 204
Quoted 10 lines
docs/system/s390x/cpu-topology.rst new file +10
-0+ vCPU shares the real CPU with other vCPUs. + +- a vCPU with vertical low entitlement specifies that this + vCPU only gets real CPU provisioning when no other vCPUs needs it. + +In the case a vCPU with vertical high entitlement does not use +the real CPU, the unused "slack" can be dispatched to other vCPU +with medium or low entitlement. + +The upper level specifies a vCPU as ``dedicated`` when the vCPU is
Using `` quotes will print "dedicated" in monotyped font ... is that what you wanted here? AFAIK we're mainly doing that for things that can be typed in the terminal, e.g. command line options. So should this use normal quotes instead?
Quoted 41 lines
docs/system/s390x/cpu-topology.rst new file +36
-0+fully dedicated to a single real CPU. + +The dedicated bit is an indication of affinity of a vCPU for a real CPU +while the entitlement indicates the sharing or exclusivity of use. + +Defining the topology on command line +------------------------------------- + +The topology can entirely be defined using -device cpu statements, +with the exception of CPU 0 which must be defined with the -smp +argument. + +For example, here we set the position of the cores 1,2,3 to +drawer 1, book 1, socket 2 and cores 0,9 and 14 to drawer 0, +book 0, socket 0 without defining entitlement or dedication. +The core 4, will be set on its default position on socket 1 +(since we have 4 core per socket) and we define it as dedicated and +with vertical high entitlement. + +.. code-block:: bash + + $ qemu-system-s390x -m 2G \ + -cpu gen16b,ctop=on \ + -smp cpus=1,sockets=8,cores=4,maxcpus=32 \ + \ + -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1 \ + -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=2 \ + -device gen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=3 \ + \ + -device gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=9 \ + -device gen16b-s390x-cpu,drawer-id=0,book-id=0,socket-id=0,core-id=14 \ + \ + -device gen16b-s390x-cpu,core-id=4,dedicated=on,entitlement=high + +The entitlement defined for the CPU 4, will only be used after the guest +successfully enables vertical polarization by using the PTF instruction.docs/system/target-s390x.rst +0
-0
docs/system/target-s390x.rst Architectural features · line 37
Quoted 4 lines
docs/system/target-s390x.rst +1
-0@@ -34,3 +34,4 @@ Architectural features .. toctree:: s390x/bootdevices s390x/protvirt + s390x/cpu-topology
Thomas
Re: [PATCH v21 13/20] docs/s390x/cpu topology: document s390x cpu topology
Quoted 53 lines · Thomas Huth
Quoted 45 lines · Pierre Morel
Add some basic examples for the definition of cpu topology in s390x.
Signed-off-by Pierre Morel <[email protected]>---
MAINTAINERS | 2 + docs/devel/index-internals.rst | 1 + docs/devel/s390-cpu-topology.rst | 170 ++++++++++++++++++++docs/system/s390x/cpu-topology.rst| 240 +++++++++++++++++++++++++++++ docs/system/target-s390x.rst | 1 +5 files changed, 414 insertions(+)create mode 100644
docs/devel/s390-cpu-topology.rstcreate mode 100644docs/system/s390x/cpu-topology.rstMAINTAINERS +0
-0@@ -1703,6 +1703,8 @@ S: SupportedF:
include/hw/s390x/cpu-topology.h
F:hw/s390x/cpu-topology.c
F:target/s390x/kvm/stsi-topology.cMAINTAINERS +2
-0+F: docs/devel/s390-cpu-topology.rst +F: docs/system/s390x/cpu-topology.rstX86 Machines
------------docs/devel/index-internals.rst +0
-0@@ -14,6 +14,7 @@ Details about QEMU's various subsystems includinghow to add features to them.
migration
multi-process
reset
+ s390-cpu-topology
s390-dasd-ipl
tracing
vfio-migrationdocs/devel/s390-cpu-topology.rst new file +0
-0docs/devel/s390-cpu-topology.rst line 4
Quoted 4 lines
docs/devel/s390-cpu-topology.rst new file +4
-0@@ -0,0 +1,170 @@ +QAPI interface for S390 CPU topology +==================================== + +Let's start QEMU with the following command defining 4 CPUs,Maybe better something like this:
Quoted 3 lines
The following sections will explain the S390 CPU topology with the help of exemplary output. For this, let's assume that QEMU has been started with the following command, defining 4 CPUs.
?
Absolutely better.
Thanks
Quoted 38 lines
Quoted 16 lines
docs/devel/s390-cpu-topology.rst new file +15
-0+CPU[0] defined by the -smp argument will have default values: + +.. code-block:: bash + + qemu-system-s390x \ + -enable-kvm \ + -cpu z14,ctop=on \ + -smp 1,drawers=3,books=3,sockets=2,cores=2,maxcpus=36 \ + \ + -device z14-s390x-cpu,core-id=19,entitlement=high \ + -device z14-s390x-cpu,core-id=11,entitlement=low \ + -device z14-s390x-cpu,core-id=112,entitlement=high \ + ... + +and see the result when using the QAPI interface....
Quoted 21 lines
docs/devel/s390-cpu-topology.rst new file +21
-0+QAPI command: set-cpu-topology +------------------------------ + +The command set-cpu-topology allows to modify the topology tree +or the topology modifiers of a vCPU in the configuration. + +.. code-block:: QMP + + { "execute": "set-cpu-topology", + "arguments": { + "core-id": 11, + "socket-id": 0, + "book-id": 0, + "drawer-id": 0, + "entitlement": "low", + "dedicated": false + } + } + {"return": {}} + +The core-id parameter is the only non optional parameter and every
Quoted 8 lines
docs/devel/s390-cpu-topology.rst line 122
Quoted 6 lines
docs/devel/s390-cpu-topology.rst new file +6
-0+unspecified parameter keeps its previous value. + +QAPI event CPU_POLARIZATION_CHANGE +---------------------------------- + +When a guest is requests a modification of the polarization,Scratch the word "is".
yes
Quoted 18 lines
Quoted 2 lines
+QEMU sends a
CPU_POLARIZATION_CHANGEevent....
docs/system/s390x/cpu-topology.rst line 6
Quoted 12 lines
docs/system/s390x/cpu-topology.rst new file +6
-0@@ -0,0 +1,240 @@ +CPU topology on s390x +===================== + +Since QEMU 8.1, CPU topology on s390x provides up to 3 levels of +topology containers: drawers, books, sockets, defining a tree shaped +hierarchy."drawers, books and sockets. They define a tree-shaped hierarchy."
Quoted 2 lines
?
yes, thx
Quoted 3 lines
docs/system/s390x/cpu-topology.rst line 8
Quoted 1 line
docs/system/s390x/cpu-topology.rst new file +1
-0+The socket container contains one or more CPU entries."The socket container has one or more CPU entries." ?
yes thx
Quoted 48 lines
Quoted 5 lines
docs/system/s390x/cpu-topology.rst new file +1
-0+Each of these CPU entries consists of a bitmap and three CPUattributes:
docs/system/s390x/cpu-topology.rst new file +2
-0+ +- CPU typedocs/system/s390x/cpu-topology.rst line 21
Quoted 10 lines
docs/system/s390x/cpu-topology.rst new file +10
-0+- entitlement +- dedication + +Each bit set in the bitmap correspond to the core-id of a vCPU with +matching the three attribute. + +This documentation provide general information on S390 CPU topology, +how to enable it and on the new CPU attributes. +For information on how to modify the S390 CPU topology and on how to +monitor the polarization change see ``Developer Information``.It would be nicer to have a proper link here instead. See commit
d6359e150dbdf84f67add786473fd277a9a442bbfor example how to do this in our .rst files.Quoted 30 lines
docs/system/s390x/cpu-topology.rst new file +30
-0+Prerequisites +------------- + +To use the CPU topology, you need to run with KVM on a s390x host that +uses the Linux kernel v6.0 or newer (which provide the so-called +``KVM_CAP_S390_CPU_TOPOLOGY`` capability that allows QEMU to signal the +CPU topology facility via the so-called STFLE bit 11 to the VM). + +Enabling CPU topology +--------------------- + +Currently, CPU topology is only enabled in the host model by default. + +Enabling CPU topology in a CPU model is done by setting the CPU flag +``ctop`` to ``on`` like in: + +.. code-block:: bash + + -cpu gen16b,ctop=on + +Having the topology disabled by default allows migration between +old and new QEMU without adding new flags. + +Default topology usage +---------------------- + +The CPU topology can be specified on the QEMU command line +with the ``-smp`` or the ``-device`` QEMU command arguments. + +Note also that since 7.2 threads are no longer supported in the
Quoted 8 lines
docs/system/s390x/cpu-topology.rst line 57
Quoted 6 lines
topology
docs/system/s390x/cpu-topology.rst new file +5
-0+and the ``-smp`` command line argument accepts only ``threads=1``. + +If none of the containers attributes (drawers, books, sockets) are +specified for the ``-smp`` flag, the number of these containers +is ``1`` ."Thus the following two options will result in the same topology, for example:" ?
Yes thanks
Quoted 36 lines
Quoted 36 lines
+.. code-block:: bash
docs/system/s390x/cpu-topology.rst new file +13
-0+ + -smp cpus=5,drawer=1,books=1,sockets=8,cores=4,maxcpus=32 + +or + +.. code-block:: bash + + -smp cpus=5,sockets=8,cores=4,maxcpus=32 + +When a CPU is defined by the ``-smp`` command argument, its position +inside the topology is calculated by adding the CPUs to the topology +based on the core-id starting with core-0 at position 0 of socket-0, +book-0, drawer-0 and filling all CPUs of socket-0 before to fillsocket-1
docs/system/s390x/cpu-topology.rst new file +9
-0+of book-0 and so on up to the last socket of the last book of the last +drawer. + +When a CPU is defined by the ``-device`` command argument, the +tree topology attributes must be all defined or all not defined. + +.. code-block:: bash + + -devicegen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1
docs/system/s390x/cpu-topology.rst new file +10
-0+ +or + +.. code-block:: bash + + -device gen16b-s390x-cpu,core-id=1,dedicated=true + +If none of the tree attributes (drawer, book, sockets), are specified +for the ``-device`` argument, as for all CPUs defined with the ``-smp`` +command argument the topology tree attributes will be set by simply
Quoted 8 lines
docs/system/s390x/cpu-topology.rst line 97
Quoted 6 lines
docs/system/s390x/cpu-topology.rst new file +6
-0+adding the CPUs to the topology based on the core-id starting with +core-0 at position 0 of socket-0, book-0, drawer-0. + +QEMU will not try to solve collisions and will report an error if the +CPU topology, explicitly or implicitly defined on a ``-device`` +argument collides with the definition of a CPU implicitely defineds/implicitely/implicitly/
thx
Quoted 67 lines
Quoted 67 lines
docs/system/s390x/cpu-topology.rst new file +19
-0+on the ``-smp`` argument. + +When the topology modifier attributes are not defined for the +``-device`` command argument they takes following default values: + +- dedicated: ``false`` +- entitlement: ``medium`` + + +Hot plug +++++++++ + +New CPUs can be plugged using the device_add hmp command as in: + +.. code-block:: bash + + (qemu) device_add gen16b-s390x-cpu,core-id=9 + +The same placement of the CPU is derived from the core-id asdescribed above.
docs/system/s390x/cpu-topology.rst new file +6
-0+ +The topology can of course be fully defined: + +.. code-block:: bash + + (qemu) device_addgen16b-s390x-cpu,drawer-id=1,book-id=1,socket-id=2,core-id=1
docs/system/s390x/cpu-topology.rst new file +39
-0+ + +Examples +++++++++ + +In the following machine we define 8 sockets with 4 cores each. + +.. code-block:: bash + + $ qemu-system-s390x -m 2G \ + -cpu gen16b,ctop=on \ + -smp cpus=5,sockets=8,cores=4,maxcpus=32 \ + -device host-s390x-cpu,core-id=14 \ + +A new CPUs can be plugged using the device_add hmp command as before: + +.. code-block:: bash + + (qemu) device_add gen16b-s390x-cpu,core-id=9 + +The core-id defines the placement of the core in the topology by +starting with core 0 in socket 0 up to maxcpus. + +In the example above: + +* There are 5 CPUs provided to the guest with the ``-smp`` command line + They will take the core-ids 0,1,2,3,4 + As we have 4 cores in a socket, we have 4 CPUs provided + to the guest in socket 0, with core-ids 0,1,2,3. + The last cpu, with core-id 4, will be on socket 1. + +* the core with ID 14 provided by the ``-device`` command line will + be placed in socket 3, with core-id 14 + +* the core with ID 9 provided by the ``device_add`` qmp command will + be placed in socket 2, with core-id 9 + + +Polarization, entitlement and dedication
Quoted 8 lines
docs/system/s390x/cpu-topology.rst line 167
Quoted 6 lines
+----------------------------------------
docs/system/s390x/cpu-topology.rst new file +5
-0+ +Polarization +++++++++++++ + +The polarization is an indication given by the ``guest`` to the hostWhy quoting guest, but not host? I'd remove the quotes from guest here.
OK
Quoted 7 lines
docs/system/s390x/cpu-topology.rst line 171
Quoted 5 lines
docs/system/s390x/cpu-topology.rst new file +4
-0+that it is able to make use of CPU provisioning information. +The guest indicates the polarization by using the PTF instruction. + +Polarization is define two models of CPU provisioning: horizontal"Polarization defines ..." ? ... or "Polarization is defined by ..." ?
thx
Quoted 7 lines
docs/system/s390x/cpu-topology.rst line 175
Quoted 5 lines
docs/system/s390x/cpu-topology.rst new file +4
-0+and vertical. + +The horizontal polarization is the default model on boot and after +subsystem reset in which the guest considers all vCPUs being havingscratch "being" ?
Thx
Quoted 24 lines
Quoted 24 lines
docs/system/s390x/cpu-topology.rst new file +23
-0+an equal provisioning of CPUs by the host. + +In the vertical polarization model the guest can make use of the +vCPU entitlement information provided by the host to optimize +kernel thread scheduling. + +A subsystem reset puts all vCPU of the configuration into the +horizontal polarization. + +Entitlement ++++++++++++ + +The vertical polarization specifies that the guest's vCPU can get +different real CPU provisions: + +- a vCPU with vertical high entitlement specifies that this + vCPU gets 100% of the real CPU provisioning. + +- a vCPU with vertical medium entitlement specifies that this + vCPU shares the real CPU with other vCPUs. + +- a vCPU with vertical low entitlement specifies that this + vCPU only gets real CPU provisioning when no other vCPUs needs it.
Quoted 8 lines
docs/system/s390x/cpu-topology.rst line 204
Quoted 6 lines
docs/system/s390x/cpu-topology.rst new file +6
-0+ +In the case a vCPU with vertical high entitlement does not use +the real CPU, the unused "slack" can be dispatched to other vCPU +with medium or low entitlement. + +The upper level specifies a vCPU as ``dedicated`` when the vCPU isUsing `` quotes will print "dedicated" in monotyped font ... is that what you wanted here? AFAIK we're mainly doing that for things that can be typed in the terminal, e.g. command line options. So should this use normal quotes instead?
No, I think we do not need quotes here.
Thanks
regards,
Pierre
PATCH v21 14/20 tests/avocado: s390x cpu topology core
In reply to Pierre MorelIntroduction of the s390x cpu topology core functions and basic tests.
We test the corelation between the command line and the QMP results in query-cpus-fast for various CPU topology.
--- MAINTAINERS | 1 + tests/avocado/s390_topology.py | 196 +++++++++++++++++++++++++++++++++ 2 files changed, 197 insertions(+) create mode 100644 tests/avocado/s390_topology.py
MAINTAINERS +1 -0
@@ -1705,6 +1705,7 @@ F: hw/s390x/cpu-topology.c F: target/s390x/kvm/stsi-topology.c F: docs/devel/s390-cpu-topology.rst F: docs/system/s390x/cpu-topology.rst +F: tests/avocado/s390_topology.py X86 Machines ------------
tests/avocado/s390_topology.py new file +196 -0
196 lines not on this page · raw message
Signature · 1 line
2.31.1
Re: [PATCH v21 14/20] tests/avocado: s390x cpu topology core
Quoted 53 lines · Pierre Morel
Introduction of the s390x cpu topology core functions and basic tests.
We test the corelation between the command line and the QMP results in query-cpus-fast for various CPU topology.
Signed-off-by Pierre Morel <[email protected]>--- MAINTAINERS | 1 + tests/avocado/s390_topology.py | 196 +++++++++++++++++++++++++++++++++ 2 files changed, 197 insertions(+)create mode 100644
tests/avocado/s390_topology.pyMAINTAINERS +1
-0@@ -1705,6 +1705,7 @@ F: hw/s390x/cpu-topology.c F: target/s390x/kvm/stsi-topology.c F: docs/devel/s390-cpu-topology.rst F: docs/system/s390x/cpu-topology.rst +F: tests/avocado/s390_topology.py X86 Machines ------------tests/avocado/s390_topology.py new file +22
-0@@ -0,0 +1,196 @@ +# Functional test that boots a Linux kernel and checks the console +# +# Copyright IBM Corp. 2023 +# +# Author: +# Pierre Morel <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import os +import shutil +import time + +from avocado_qemu import QemuSystemTest +from avocado_qemu import exec_command +from avocado_qemu import exec_command_and_wait_for_pattern +from avocado_qemu import interrupt_interactive_console_until_pattern +from avocado_qemu import wait_for_console_pattern +from avocado.utils import process +from avocado.utils import archive +
tests/avocado/s390_topology.py line 32
Quoted 10 lines
tests/avocado/s390_topology.py new file +10
-0+ +class S390CPUTopology(QemuSystemTest): + """ + S390x CPU topology consist of 4 topology layers, from bottom to top, + the cores, sockets, books and drawers and 2 modifiers attributes, + the entitlement and the dedication. + See: docs/system/s390x/cpu-topology.rst. + + S390x CPU topology is setup in different ways: + - implicitely from the '-smp' argument by completing each topology
implicitly
tests/avocado/s390_topology.py line 33
Quoted 1 line
tests/avocado/s390_topology.py new file +1
-0+ level one after the other begining with drawer 0, book 0 and socket 0.
beginning
tests/avocado/s390_topology.py line 34
Quoted 1 line
tests/avocado/s390_topology.py new file +1
-0+ - explicitely from the '-device' argument on the QEMU command line
explicitly
tests/avocado/s390_topology.py line 35
Quoted 1 line
tests/avocado/s390_topology.py new file +1
-0+ - explicitely by hotplug of a new CPU using QMP or HMP
explicitly
Thomas
Re: [PATCH v21 14/20] tests/avocado: s390x cpu topology core
Quoted 74 lines · Thomas Huth
Quoted 56 lines · Pierre Morel
Introduction of the s390x cpu topology core functions and basic tests.
We test the corelation between the command line and the QMP results in query-cpus-fast for various CPU topology.
Signed-off-by Pierre Morel <[email protected]>---
MAINTAINERS | 1 +tests/avocado/s390_topology.py| 196 +++++++++++++++++++++++++++++++++2 files changed, 197 insertions(+)create mode 100644
tests/avocado/s390_topology.pyMAINTAINERS +0
-0@@ -1705,6 +1705,7 @@ F: hw/s390x/cpu-topology.cF:
target/s390x/kvm/stsi-topology.c
F:docs/devel/s390-cpu-topology.rst
F:docs/system/s390x/cpu-topology.rst
+F:tests/avocado/s390_topology.py
X86 Machines
------------tests/avocado/s390_topology.py new file +26
-0@@ -0,0 +1,196 @@ +# Functional test that boots a Linux kernel and checks the console +# +# Copyright IBM Corp. 2023 +# +# Author: +# Pierre Morel <[email protected]> +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +import os +import shutil +import time + +from avocado_qemu import QemuSystemTest +from avocado_qemu import exec_command +from avocado_qemu import exec_command_and_wait_for_pattern +from avocado_qemu import interrupt_interactive_console_until_pattern +from avocado_qemu import wait_for_console_pattern +from avocado.utils import process +from avocado.utils import archive + + +class S390CPUTopology(QemuSystemTest): + """ + S390x CPU topology consist of 4 topology layers, from bottom toQuoted 7 lines
top,
+ the cores, sockets, books and drawers and 2 modifiers attributes, + the entitlement and the dedication.
+ See:docs/system/s390x/cpu-topology.rst.
+
+ S390x CPU topology is setup in different ways:
+ - implicitely from the '-smp' argument by completing each topologyimplicitly
Quoted 2 lines
+ level one after the other begining with drawer 0, book 0 and socket 0.
beginning
Quoted 1 line
+ - explicitely from the '-device' argument on the QEMU command line
Quoted 8 lines
explicitly
Quoted 1 line
+ - explicitely by hotplug of a new CPU using QMP or HMP
explicitly
Thomas
Thanks, I make the corrections.
Pierre
Re: [PATCH v21 14/20] tests/avocado: s390x cpu topology core
In reply to Pierre MorelQuoted 4 lines · Pierre Morel
Introduction of the s390x cpu topology core functions and basic tests.
We test the corelation between the command line and
corRelation
Quoted 3 lines
the QMP results in query-cpus-fast for various CPU topology.
Signed-off-by Pierre Morel <[email protected]>
Quoted 1 line
---
[...]
Quoted 7 lines
tests/avocado/s390_topology.py new file +0
-0@@ -0,0 +1,196 @@
[...]
Quoted 3 lines
+class S390CPUTopology(QemuSystemTest):
+ """
+ S390x CPU topology consist of 4 topology layers, from bottom to
consistS
Quoted 20 lines
top,
+ the cores, sockets, books and drawers and 2 modifiers attributes,
+ the entitlement and the dedication.
+ See:docs/system/s390x/cpu-topology.rst.
+
+ S390x CPU topology is setup in different ways:
+ - implicitely from the '-smp' argument by completing each topology
+ level one after the other begining with drawer 0, book 0 and socket 0.
+ - explicitely from the '-device' argument on the QEMU command line
+ - explicitely by hotplug of a new CPU using QMP or HMP + - it is modified by using QMP 'set-cpu-topology'
+
+ The S390x modifier attribute entitlement depends on the machine + polarization, which can be horizontal or vertical.
+ The polarization is changed on a request from the guest. + """
[...]
Re: [PATCH v21 14/20] tests/avocado: s390x cpu topology core
Quoted 11 lines · Nina Schoetterl-Glausch
Quoted 4 lines · Pierre Morel
Introduction of the s390x cpu topology core functions and basic tests.
We test the corelation between the command line and
corRelation
Quoted 3 lines
the QMP results in query-cpus-fast for various CPU topology.
Signed-off-by Pierre Morel <[email protected]>Reviewed-by Nina Schoetterl-Glausch <[email protected]>
Thanks,
I did change the two spelling errors.
regards,
Pierre
PATCH v21 15/20 tests/avocado: s390x cpu topology polarisation
In reply to Pierre MorelPolarization is changed on a request from the guest. Let's verify the polarization is accordingly set by QEMU.
--- tests/avocado/s390_topology.py | 46 ++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+)
tests/avocado/s390_topology.py +46 -0
@@ -40,6 +40,7 @@ class S390CPUTopology(QemuSystemTest): The polarization is changed on a request from the guest. """ timeout = 90 + event_timeout = 1 KERNEL_COMMON_COMMAND_LINE = ('printk.time=0 ' 'root=/dev/ram '
@@ -99,6 +100,15 @@ def kernel_init(self): '-initrd', initrd_path, '-append', kernel_command_line) + def system_init(self): + self.log.info("System init") + exec_command(self, 'mount proc -t proc /proc') + time.sleep(0.2) + exec_command(self, 'mount sys -t sysfs /sys') + time.sleep(0.2) + exec_command_and_wait_for_pattern(self, + '/bin/cat /sys/devices/system/cpu/dispatching', '0') + def test_single(self): """ This test checks the simplest topology with a single CPU.
@@ -194,3 +204,39 @@ def test_hotplug_full(self): self.check_topology(3, 1, 1, 1, 'high', False) self.check_topology(4, 1, 1, 1, 'medium', False) self.check_topology(5, 2, 1, 1, 'high', True) + + + def guest_set_dispatching(self, dispatching): + exec_command(self, + f'echo {dispatching} > /sys/devices/system/cpu/dispatching') + self.vm.event_wait('CPU_POLARIZATION_CHANGE', self.event_timeout) + exec_command_and_wait_for_pattern(self, + '/bin/cat /sys/devices/system/cpu/dispatching', dispatching) + + + def test_polarisation(self): + """ + This test verifies that QEMU modifies the entitlement change after + several guest polarization change requests. + + :avocado: tags=arch:s390x + :avocado: tags=machine:s390-ccw-virtio + """ + self.kernel_init() + self.vm.launch() + self.wait_until_booted() + + self.system_init() + res = self.vm.qmp('query-cpu-polarization') + self.assertEqual(res['return']['polarization'], 'horizontal') + self.check_topology(0, 0, 0, 0, 'medium', False) + + self.guest_set_dispatching('1'); + res = self.vm.qmp('query-cpu-polarization') + self.assertEqual(res['return']['polarization'], 'vertical') + self.check_topology(0, 0, 0, 0, 'medium', False) + + self.guest_set_dispatching('0'); + res = self.vm.qmp('query-cpu-polarization') + self.assertEqual(res['return']['polarization'], 'horizontal') + self.check_topology(0, 0, 0, 0, 'medium', False)
Signature · 1 line
2.31.1
Re: [PATCH v21 15/20] tests/avocado: s390x cpu topology polarisation
Quoted 13 lines · Pierre Morel
Polarization is changed on a request from the guest. Let's verify the polarization is accordingly set by QEMU.
Signed-off-by Pierre Morel <[email protected]>--- tests/avocado/s390_topology.py | 46 ++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+)tests/avocado/s390_topology.py +0
-0
tests/avocado/s390_topology.py class S390CPUTopology(QemuSystemTest): · line 43
Quoted 4 lines
tests/avocado/s390_topology.py +1
-0@@ -40,6 +40,7 @@ class S390CPUTopology(QemuSystemTest): The polarization is changed on a request from the guest. """ timeout = 90 + event_timeout = 1
When running tests in CI and the machines are very loaded, the tests can be stalled easily by multiple seconds. So using a timeout of 1 seconds sounds way too low for me. Please use at least 5 seconds, or maybe even 10.
tests/avocado/s390_topology.py def kernel_init(self): · line 108
Quoted 12 lines
tests/avocado/s390_topology.py +6
-0KERNEL_COMMON_COMMAND_LINE = ('printk.time=0 ' 'root=/dev/ram '@@ -99,6 +100,15 @@ def kernel_init(self): '-initrd', initrd_path, '-append', kernel_command_line) + def system_init(self): + self.log.info("System init") + exec_command(self, 'mount proc -t proc /proc') + time.sleep(0.2) + exec_command(self, 'mount sys -t sysfs /sys') + time.sleep(0.2)
Hard coded sleeps are ugly... they are prone to race conditions (e.g. on loaded test systems), and they artificially slow down the test duration.
What about doing all three commands in one statement instead:
exec_command_and_wait_for_pattern(self,
"""mount proc -t proc /proc ;
mount sys -t sysfs /sys ;
/bin/cat /sys/devices/system/cpu/dispatching""",
'0')?
Quoted 46 lines
tests/avocado/s390_topology.py +39
-0+ exec_command_and_wait_for_pattern(self, + '/bin/cat /sys/devices/system/cpu/dispatching', '0') + def test_single(self): """ This test checks the simplest topology with a single CPU.@@ -194,3 +204,39 @@ def test_hotplug_full(self): self.check_topology(3, 1, 1, 1, 'high', False) self.check_topology(4, 1, 1, 1, 'medium', False) self.check_topology(5, 2, 1, 1, 'high', True) + + + def guest_set_dispatching(self, dispatching): + exec_command(self, + f'echo {dispatching} > /sys/devices/system/cpu/dispatching') + self.vm.event_wait('CPU_POLARIZATION_CHANGE', self.event_timeout) + exec_command_and_wait_for_pattern(self, + '/bin/cat /sys/devices/system/cpu/dispatching', dispatching) + + + def test_polarisation(self): + """ + This test verifies that QEMU modifies the entitlement change after + several guest polarization change requests. + + :avocado: tags=arch:s390x + :avocado: tags=machine:s390-ccw-virtio + """ + self.kernel_init() + self.vm.launch() + self.wait_until_booted() + + self.system_init() + res = self.vm.qmp('query-cpu-polarization') + self.assertEqual(res['return']['polarization'], 'horizontal') + self.check_topology(0, 0, 0, 0, 'medium', False) + + self.guest_set_dispatching('1'); + res = self.vm.qmp('query-cpu-polarization') + self.assertEqual(res['return']['polarization'], 'vertical') + self.check_topology(0, 0, 0, 0, 'medium', False) + + self.guest_set_dispatching('0'); + res = self.vm.qmp('query-cpu-polarization') + self.assertEqual(res['return']['polarization'], 'horizontal') + self.check_topology(0, 0, 0, 0, 'medium', False)
Re: [PATCH v21 15/20] tests/avocado: s390x cpu topology polarisation
Quoted 9 lines · Thomas Huth
Quoted 8 lines · Pierre Morel
Polarization is changed on a request from the guest. Let's verify the polarization is accordingly set by QEMU.
Signed-off-by Pierre Morel <[email protected]>---
tests/avocado/s390_topology.py| 46 ++++++++++++++++++++++++++++++++++1 file changed, 46 insertions(+)
Quoted 11 lines
Quoted 9 lines
tests/avocado/s390_topology.py +0
-0@@ -40,6 +40,7 @@ class S390CPUTopology(QemuSystemTest):The polarization is changed on a request from the guest. """
timeout = 90
+event_timeout= 1When running tests in CI and the machines are very loaded, the tests can be stalled easily by multiple seconds. So using a timeout of 1 seconds sounds way too low for me. Please use at least 5 seconds, or maybe even 10.
OK
Quoted 17 lines
Quoted 6 lines
KERNEL_COMMON_COMMAND_LINE= ('printk.time=0 ' 'root=/dev/ram 'tests/avocado/s390_topology.py +0
-0@@ -99,6 +100,15 @@ def kernel_init(self): '-initrd', initrd_path, '-append', kernel_command_line)Quoted 6 lines
+ def
system_init(self):
+self.log.info("System init")
+exec_command(self, 'mount proc -t proc /proc')+time.sleep(0.2)
+exec_command(self, 'mount sys -t sysfs /sys')+time.sleep(0.2)Hard coded sleeps are ugly... they are prone to race conditions (e.g. on loaded test systems), and they artificially slow down the test duration.
What about doing all three commands in one statement instead:
Quoted 8 lines
exec_command_and_wait_for_pattern(self,
"""mount proc -t proc /proc ;
mount sys -t sysfs /sys ;
/bin/cat/sys/devices/system/cpu/dispatching""", '0')?
OK , I use this. thx.
Regards,
Pierre
PATCH v21 16/20 tests/avocado: s390x cpu topology entitlement tests
In reply to Pierre MorelThis test takes care to check the changes on different entitlements when the guest requests a polarization change.
--- tests/avocado/s390_topology.py | 47 ++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+)
tests/avocado/s390_topology.py +47 -0
@@ -240,3 +240,50 @@ def test_polarisation(self): res = self.vm.qmp('query-cpu-polarization') self.assertEqual(res['return']['polarization'], 'horizontal') self.check_topology(0, 0, 0, 0, 'medium', False) + + def test_entitlement(self): + """ + This test verifies that QEMU modifies the polarization + after a guest request. + + :avocado: tags=arch:s390x + :avocado: tags=machine:s390-ccw-virtio + """ + self.kernel_init() + self.vm.add_args('-smp', + '4,drawers=2,books=2,sockets=3,cores=2,maxcpus=24') + self.vm.launch() + self.wait_until_booted() + + self.system_init() + + res = self.vm.qmp('set-cpu-topology', + {'core-id': 0, 'entitlement': 'low'}) + self.assertEqual(res['return'], {}) + res = self.vm.qmp('set-cpu-topology', + {'core-id': 1, 'entitlement': 'medium'}) + self.assertEqual(res['return'], {}) + res = self.vm.qmp('set-cpu-topology', + {'core-id': 2, 'entitlement': 'high'}) + self.assertEqual(res['return'], {}) + res = self.vm.qmp('set-cpu-topology', + {'core-id': 3, 'entitlement': 'high'}) + self.assertEqual(res['return'], {}) + self.check_topology(0, 0, 0, 0, 'low', False) + self.check_topology(1, 0, 0, 0, 'medium', False) + self.check_topology(2, 1, 0, 0, 'high', False) + self.check_topology(3, 1, 0, 0, 'high', False) + + self.guest_set_dispatching('1'); + + self.check_topology(0, 0, 0, 0, 'low', False) + self.check_topology(1, 0, 0, 0, 'medium', False) + self.check_topology(2, 1, 0, 0, 'high', False) + self.check_topology(3, 1, 0, 0, 'high', False) + + self.guest_set_dispatching('0'); + + self.check_topology(0, 0, 0, 0, 'low', False) + self.check_topology(1, 0, 0, 0, 'medium', False) + self.check_topology(2, 1, 0, 0, 'high', False) + self.check_topology(3, 1, 0, 0, 'high', False)
Signature · 1 line
2.31.1
Re: [PATCH v21 16/20] tests/avocado: s390x cpu topology entitlement tests
Quoted 21 lines · Pierre Morel
This test takes care to check the changes on different entitlements when the guest requests a polarization change.
Signed-off-by Pierre Morel <[email protected]>--- tests/avocado/s390_topology.py | 47 ++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+)tests/avocado/s390_topology.py +5
-0@@ -240,3 +240,50 @@ def test_polarisation(self): res = self.vm.qmp('query-cpu-polarization') self.assertEqual(res['return']['polarization'], 'horizontal') self.check_topology(0, 0, 0, 0, 'medium', False) + + def test_entitlement(self): + """ + This test verifies that QEMU modifies the polarization + after a guest request.
...
Quoted 8 lines
tests/avocado/s390_topology.py +8
-0+ self.check_topology(0, 0, 0, 0, 'low', False) + self.check_topology(1, 0, 0, 0, 'medium', False) + self.check_topology(2, 1, 0, 0, 'high', False) + self.check_topology(3, 1, 0, 0, 'high', False) + + self.guest_set_dispatching('1'); + + self.check_topology(0, 0, 0, 0, 'low', False)
tests/avocado/s390_topology.py def test_polarisation(self): · line 289
Quoted 10 lines
tests/avocado/s390_topology.py +10
-0+ self.check_topology(1, 0, 0, 0, 'medium', False) + self.check_topology(2, 1, 0, 0, 'high', False) + self.check_topology(3, 1, 0, 0, 'high', False) + + self.guest_set_dispatching('0'); + + self.check_topology(0, 0, 0, 0, 'low', False) + self.check_topology(1, 0, 0, 0, 'medium', False) + self.check_topology(2, 1, 0, 0, 'high', False) + self.check_topology(3, 1, 0, 0, 'high', False)
Sorry, I think I'm too blind to see it, but what has changed after the guest changed the polarization?
Thomas
Re: [PATCH v21 16/20] tests/avocado: s390x cpu topology entitlement tests
Quoted 40 lines · Thomas Huth
Quoted 24 lines · Pierre Morel
This test takes care to check the changes on different entitlements when the guest requests a polarization change.
Signed-off-by Pierre Morel <[email protected]>---
tests/avocado/s390_topology.py| 47
++++++++++++++++++++++++++++++++++1 file changed, 47 insertions(+)tests/avocado/s390_topology.py +0
-0@@ -240,3 +240,50 @@ def test_polarisation(self): res = self.vm.qmp('query-cpu-polarization') self.assertEqual(res['return']['polarization'], 'horizontal')
self.check_topology(0, 0, 0, 0, 'medium', False)+
+ deftest_entitlement(self):
+ """
+ This test verifies that QEMU modifies the polarization + after a guest request....
Quoted 14 lines
+
self.check_topology(0, 0, 0, 0, 'low', False)+self.check_topology(1, 0, 0, 0, 'medium', False)+self.check_topology(2, 1, 0, 0, 'high', False)+self.check_topology(3, 1, 0, 0, 'high', False)tests/avocado/s390_topology.py +10
-0+ + self.guest_set_dispatching('1'); + + self.check_topology(0, 0, 0, 0, 'low', False) + self.check_topology(1, 0, 0, 0, 'medium', False) + self.check_topology(2, 1, 0, 0, 'high', False) + self.check_topology(3, 1, 0, 0, 'high', False) + + self.guest_set_dispatching('0'); +
Quoted 8 lines
Quoted 4 lines
+
self.check_topology(0, 0, 0, 0, 'low', False)+self.check_topology(1, 0, 0, 0, 'medium', False)+self.check_topology(2, 1, 0, 0, 'high', False)+self.check_topology(3, 1, 0, 0, 'high', False)Sorry, I think I'm too blind to see it, but what has changed after the guest
changed the polarization?
Nothing, the values are retained, they're just not active. The guest will see a horizontal polarization until it changes back to vertical.
Quoted 3 lines
Thomas
Re: [PATCH v21 16/20] tests/avocado: s390x cpu topology entitlement tests
Quoted 50 lines · Nina Schoetterl-Glausch
Quoted 40 lines · Thomas Huth
Quoted 24 lines · Pierre Morel
This test takes care to check the changes on different entitlements when the guest requests a polarization change.
Signed-off-by Pierre Morel <[email protected]>--- tests/avocado/s390_topology.py | 47 ++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+)tests/avocado/s390_topology.py +5
-0@@ -240,3 +240,50 @@ def test_polarisation(self): res = self.vm.qmp('query-cpu-polarization') self.assertEqual(res['return']['polarization'], 'horizontal') self.check_topology(0, 0, 0, 0, 'medium', False) + + def test_entitlement(self): + """ + This test verifies that QEMU modifies the polarization + after a guest request....
Quoted 14 lines
+
self.check_topology(0, 0, 0, 0, 'low', False)+self.check_topology(1, 0, 0, 0, 'medium', False)+self.check_topology(2, 1, 0, 0, 'high', False)+self.check_topology(3, 1, 0, 0, 'high', False)tests/avocado/s390_topology.py +10
-0+ + self.guest_set_dispatching('1'); + + self.check_topology(0, 0, 0, 0, 'low', False) + self.check_topology(1, 0, 0, 0, 'medium', False) + self.check_topology(2, 1, 0, 0, 'high', False) + self.check_topology(3, 1, 0, 0, 'high', False) + + self.guest_set_dispatching('0'); +Quoted 8 lines
Quoted 4 lines
+
self.check_topology(0, 0, 0, 0, 'low', False)+self.check_topology(1, 0, 0, 0, 'medium', False)+self.check_topology(2, 1, 0, 0, 'high', False)+self.check_topology(3, 1, 0, 0, 'high', False)Sorry, I think I'm too blind to see it, but what has changed after the guest
changed the polarization?
Quoted 3 lines
Nothing, the values are retained, they're just not active. The guest will see a horizontal polarization until it changes back to vertical.
But then the comment in front of it ("This test verifies that QEMU *modifies* the polarization...") does not quite match, does it?
Thomas
Re: [PATCH v21 16/20] tests/avocado: s390x cpu topology entitlement tests
Quoted 53 lines · Thomas Huth
Quoted 52 lines · Nina Schoetterl-Glausch
Quoted 50 lines · Thomas Huth
Quoted 25 lines · Pierre Morel
This test takes care to check the changes on different entitlements
when the guest requests a polarization change.Signed-off-by Pierre Morel <[email protected]>---
tests/avocado/s390_topology.py| 47
++++++++++++++++++++++++++++++++++1 file changed, 47 insertions(+)tests/avocado/s390_topology.py +0
-0@@ -240,3 +240,50 @@ def test_polarisation(self): res = self.vm.qmp('query-cpu-polarization') self.assertEqual(res['return']['polarization'], 'horizontal')
self.check_topology(0, 0, 0, 0, 'medium', False)+
+ deftest_entitlement(self):
+ """
+ This test verifies that QEMU modifies the polarization + after a guest request....
Quoted 8 lines
+
self.check_topology(0, 0, 0, 0, 'low', False)+self.check_topology(1, 0, 0, 0, 'medium', False)+self.check_topology(2, 1, 0, 0, 'high', False)+self.check_topology(3, 1, 0, 0, 'high', False)tests/avocado/s390_topology.py +4
-0+ + self.guest_set_dispatching('1'); + + self.check_topology(0, 0, 0, 0, 'low', False)tests/avocado/s390_topology.py def test_polarisation(self): · line 289
Quoted 10 lines
tests/avocado/s390_topology.py +10
-0+ self.check_topology(1, 0, 0, 0, 'medium', False) + self.check_topology(2, 1, 0, 0, 'high', False) + self.check_topology(3, 1, 0, 0, 'high', False) + + self.guest_set_dispatching('0'); + + self.check_topology(0, 0, 0, 0, 'low', False) + self.check_topology(1, 0, 0, 0, 'medium', False) + self.check_topology(2, 1, 0, 0, 'high', False) + self.check_topology(3, 1, 0, 0, 'high', False)Sorry, I think I'm too blind to see it, but what has changed after
the guest
changed the polarization?
Quoted 7 lines
Quoted 4 lines
Nothing, the values are retained, they're just not active. The guest will see a horizontal polarization until it changes back to
vertical.But then the comment in front of it ("This test verifies that QEMU *modifies* the polarization...") does not quite match, does it?
Yeah, it tests that QEMU reports it's own state changed when using set-cpu-topology.
I think it would be a good idea to get the guests view from the sysfs, also.
Quoted 4 lines
Thomas
Re: [PATCH v21 16/20] tests/avocado: s390x cpu topology entitlement tests
Quoted 56 lines · Nina Schoetterl-Glausch
Quoted 55 lines · Thomas Huth
Quoted 54 lines · Nina Schoetterl-Glausch
Quoted 41 lines · Thomas Huth
Quoted 25 lines · Pierre Morel
This test takes care to check the changes on different entitlements
when the guest requests a polarization change.Signed-off-by Pierre Morel <[email protected]>--- tests/avocado/s390_topology.py | 47 ++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+)tests/avocado/s390_topology.py +5
-0@@ -240,3 +240,50 @@ def test_polarisation(self): res = self.vm.qmp('query-cpu-polarization') self.assertEqual(res['return']['polarization'], 'horizontal') self.check_topology(0, 0, 0, 0, 'medium', False) + + def test_entitlement(self): + """ + This test verifies that QEMU modifies the polarization + after a guest request....
Quoted 14 lines
+
self.check_topology(0, 0, 0, 0, 'low', False)+self.check_topology(1, 0, 0, 0, 'medium', False)+self.check_topology(2, 1, 0, 0, 'high', False)+self.check_topology(3, 1, 0, 0, 'high', False)tests/avocado/s390_topology.py +10
-0+ + self.guest_set_dispatching('1'); + + self.check_topology(0, 0, 0, 0, 'low', False) + self.check_topology(1, 0, 0, 0, 'medium', False) + self.check_topology(2, 1, 0, 0, 'high', False) + self.check_topology(3, 1, 0, 0, 'high', False) + + self.guest_set_dispatching('0'); +Quoted 8 lines
Quoted 4 lines
+
self.check_topology(0, 0, 0, 0, 'low', False)+self.check_topology(1, 0, 0, 0, 'medium', False)+self.check_topology(2, 1, 0, 0, 'high', False)+self.check_topology(3, 1, 0, 0, 'high', False)Sorry, I think I'm too blind to see it, but what has changed after
the guest
changed the polarization?Nothing, the values are retained, they're just not active. The guest will see a horizontal polarization until it changes back to
vertical.
Quoted 10 lines
Quoted 2 lines
But then the comment in front of it ("This test verifies that QEMU *modifies* the polarization...") does not quite match, does it?
Yeah, it tests that QEMU reports it's own state changed when using set-cpu-topology.
I think it would be a good idea to get the guests view from the sysfs, also.Quoted 3 lines
Thomas
Yes, I think you are right, I rewrite this to check the guest view of the changes.
As you said the values are retained when not used by horizontal polarization so it is a non sense to check from host view.
Thanks
Pierre
Re: [PATCH v21 16/20] tests/avocado: s390x cpu topology entitlement tests
Quoted 74 lines · Pierre Morel
Quoted 59 lines · Nina Schoetterl-Glausch
Quoted 58 lines · Thomas Huth
Quoted 57 lines · Nina Schoetterl-Glausch
Quoted 43 lines · Thomas Huth
Quoted 27 lines · Pierre Morel
This test takes care to check the changes on different entitlements
when the guest requests a polarization change.Signed-off-by Pierre Morel <[email protected]>---
tests/avocado/s390_topology.py| 47
++++++++++++++++++++++++++++++++++1 file changed, 47 insertions(+)tests/avocado/s390_topology.py +0
-0@@ -240,3 +240,50 @@ def test_polarisation(self): res = self.vm.qmp('query-cpu-polarization') self.assertEqual(res['return']['polarization'], 'horizontal')
self.check_topology(0, 0, 0, 0, 'medium', False)
+
+ deftest_entitlement(self):
+ """
+ This test verifies that QEMU modifies the polarization
+ after a guest request....
Quoted 14 lines
+
self.check_topology(0, 0, 0, 0, 'low', False)+self.check_topology(1, 0, 0, 0, 'medium', False)+self.check_topology(2, 1, 0, 0, 'high', False)+self.check_topology(3, 1, 0, 0, 'high', False)tests/avocado/s390_topology.py +10
-0+ + self.guest_set_dispatching('1'); + + self.check_topology(0, 0, 0, 0, 'low', False) + self.check_topology(1, 0, 0, 0, 'medium', False) + self.check_topology(2, 1, 0, 0, 'high', False) + self.check_topology(3, 1, 0, 0, 'high', False) + + self.guest_set_dispatching('0'); +Quoted 8 lines
Quoted 4 lines
+
self.check_topology(0, 0, 0, 0, 'low', False)+self.check_topology(1, 0, 0, 0, 'medium', False)+self.check_topology(2, 1, 0, 0, 'high', False)+self.check_topology(3, 1, 0, 0, 'high', False)Sorry, I think I'm too blind to see it, but what has changed after
the guest
changed the polarization?Nothing, the values are retained, they're just not active. The guest will see a horizontal polarization until it changes back
to
vertical.Quoted 12 lines
Quoted 3 lines
But then the comment in front of it ("This test verifies that QEMU
*modifies* the polarization...") does not quite match, does it?Yeah, it tests that QEMU reports it's own state changed when using set-cpu-topology.
I think it would be a good idea to get the guests view from the sysfs,
also.Quoted 3 lines
Thomas
Quoted 5 lines
Yes, I think you are right, I rewrite this to check the guest view of the changes.
As you said the values are retained when not used by horizontal polarization so it is a non sense to check from host view.
I don't think it's bad to check the host view, you can do both.
Quoted 5 lines
Thanks
Pierre
Re: [PATCH v21 16/20] tests/avocado: s390x cpu topology entitlement tests
Quoted 73 lines · Nina Schoetterl-Glausch
Quoted 72 lines · Pierre Morel
Quoted 71 lines · Nina Schoetterl-Glausch
Quoted 53 lines · Thomas Huth
Quoted 52 lines · Nina Schoetterl-Glausch
Quoted 51 lines · Thomas Huth
Quoted 27 lines · Pierre Morel
This test takes care to check the changes on different entitlements
when the guest requests a polarization change.Signed-off-by Pierre Morel <[email protected]>--- tests/avocado/s390_topology.py | 47 ++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+)tests/avocado/s390_topology.py +0
-0@@ -240,3 +240,50 @@ def test_polarisation(self): res = self.vm.qmp('query-cpu-polarization') self.assertEqual(res['return']['polarization'], 'horizontal') self.check_topology(0, 0, 0, 0, 'medium',False)
+
+ deftest_entitlement(self):
+ """
+ This test verifies that QEMU modifies the polarization
+ after a guest request....
Quoted 8 lines
+
self.check_topology(0, 0, 0, 0, 'low', False)+self.check_topology(1, 0, 0, 0, 'medium', False)+self.check_topology(2, 1, 0, 0, 'high', False)+self.check_topology(3, 1, 0, 0, 'high', False)tests/avocado/s390_topology.py +4
-0+ + self.guest_set_dispatching('1'); + + self.check_topology(0, 0, 0, 0, 'low', False)tests/avocado/s390_topology.py
Quoted 10 lines
tests/avocado/s390_topology.py +10
-0+ self.check_topology(1, 0, 0, 0, 'medium', False) + self.check_topology(2, 1, 0, 0, 'high', False) + self.check_topology(3, 1, 0, 0, 'high', False) + + self.guest_set_dispatching('0'); + + self.check_topology(0, 0, 0, 0, 'low', False) + self.check_topology(1, 0, 0, 0, 'medium', False) + self.check_topology(2, 1, 0, 0, 'high', False) + self.check_topology(3, 1, 0, 0, 'high', False)Sorry, I think I'm too blind to see it, but what has changed after
the guest
changed the polarization?Quoted 8 lines
Quoted 5 lines
Nothing, the values are retained, they're just not active. The guest will see a horizontal polarization until it changes back
to
vertical.But then the comment in front of it ("This test verifies that QEMU
*modifies* the polarization...") does not quite match, does it?Yeah, it tests that QEMU reports it's own state changed when using set-cpu-topology.
I think it would be a good idea to get the guests view from the sysfs,
also.Quoted 3 lines
Thomas
Quoted 7 lines
Quoted 5 lines
Yes, I think you are right, I rewrite this to check the guest view of the changes.
As you said the values are retained when not used by horizontal polarization so it is a non sense to check from host view.
I don't think it's bad to check the host view, you can do both.
OK, then I do it.
Thanks,
Pierre
PATCH v21 17/20 tests/avocado: s390x cpu topology test dedicated CPU
In reply to Pierre MorelA dedicated CPU in vertical polarization can only have a high entitlement.
Let's check this.
--- tests/avocado/s390_topology.py | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)
tests/avocado/s390_topology.py +28 -0
@@ -287,3 +287,31 @@ def test_entitlement(self): self.check_topology(1, 0, 0, 0, 'medium', False) self.check_topology(2, 1, 0, 0, 'high', False) self.check_topology(3, 1, 0, 0, 'high', False) + + def test_dedicated(self): + """ + This test verifies that QEMU modifies the entitlement change correctly + for a dedicated CPU after several guest polarization change requests. + + :avocado: tags=arch:s390x + :avocado: tags=machine:s390-ccw-virtio + """ + self.kernel_init() + self.vm.launch() + self.wait_until_booted() + + self.system_init() + + res = self.vm.qmp('set-cpu-topology', + {'core-id': 0, 'dedicated': True}) + self.assertEqual(res['return'], {}) + + self.check_topology(0, 0, 0, 0, 'high', True) + + self.guest_set_dispatching('1'); + + self.check_topology(0, 0, 0, 0, 'high', True) + + self.guest_set_dispatching('0'); + + self.check_topology(0, 0, 0, 0, 'high', True)
Signature · 1 line
2.31.1
Re: [PATCH v21 17/20] tests/avocado: s390x cpu topology test dedicated CPU
Same as for the previous test.
I rewrite this to check the guest view.
Quoted 45 lines · Pierre Morel
A dedicated CPU in vertical polarization can only have a high entitlement.
Let's check this.Signed-off-by Pierre Morel <[email protected]>--- tests/avocado/s390_topology.py | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)tests/avocado/s390_topology.py +28
-0@@ -287,3 +287,31 @@ def test_entitlement(self): self.check_topology(1, 0, 0, 0, 'medium', False) self.check_topology(2, 1, 0, 0, 'high', False) self.check_topology(3, 1, 0, 0, 'high', False) + + def test_dedicated(self): + """ + This test verifies that QEMU modifies the entitlement change correctly + for a dedicated CPU after several guest polarization change requests. + + :avocado: tags=arch:s390x + :avocado: tags=machine:s390-ccw-virtio + """ + self.kernel_init() + self.vm.launch() + self.wait_until_booted() + + self.system_init() + + res = self.vm.qmp('set-cpu-topology', + {'core-id': 0, 'dedicated': True}) + self.assertEqual(res['return'], {}) + + self.check_topology(0, 0, 0, 0, 'high', True) + + self.guest_set_dispatching('1'); + + self.check_topology(0, 0, 0, 0, 'high', True) + + self.guest_set_dispatching('0'); + + self.check_topology(0, 0, 0, 0, 'high', True)
PATCH v21 18/20 tests/avocado: s390x cpu topology test socket full
In reply to Pierre MorelThis test verifies that QMP set-cpu-topology does not accept to overload a socket.
--- tests/avocado/s390_topology.py | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
tests/avocado/s390_topology.py +25 -0
@@ -315,3 +315,28 @@ def test_dedicated(self): self.guest_set_dispatching('0'); self.check_topology(0, 0, 0, 0, 'high', True) + + def test_socket_full(self): + """ + This test verifies that QEMU does not accept to overload a socket. + The socket-id 0 on book-id 0 already contains CPUs 0 and 1 and can + not accept any new CPU while socket-id 0 on book-id 1 is free. + + :avocado: tags=arch:s390x + :avocado: tags=machine:s390-ccw-virtio + """ + self.kernel_init() + self.vm.add_args('-smp', + '3,drawers=2,books=2,sockets=3,cores=2,maxcpus=24') + self.vm.launch() + self.wait_until_booted() + + self.system_init() + + res = self.vm.qmp('set-cpu-topology', + {'core-id': 2, 'socket-id': 0, 'book-id': 0}) + self.assertEqual(res['error']['class'], 'GenericError') + + res = self.vm.qmp('set-cpu-topology', + {'core-id': 2, 'socket-id': 0, 'book-id': 1}) + self.assertEqual(res['return'], {})
Signature · 1 line
2.31.1
Re: [PATCH v21 18/20] tests/avocado: s390x cpu topology test socket full
Quoted 31 lines · Pierre Morel
This test verifies that QMP set-cpu-topology does not accept to overload a socket.
Signed-off-by Pierre Morel <[email protected]>--- tests/avocado/s390_topology.py | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)tests/avocado/s390_topology.py +15
-0@@ -315,3 +315,28 @@ def test_dedicated(self): self.guest_set_dispatching('0'); self.check_topology(0, 0, 0, 0, 'high', True) + + def test_socket_full(self): + """ + This test verifies that QEMU does not accept to overload a socket. + The socket-id 0 on book-id 0 already contains CPUs 0 and 1 and can + not accept any new CPU while socket-id 0 on book-id 1 is free. + + :avocado: tags=arch:s390x + :avocado: tags=machine:s390-ccw-virtio + """ + self.kernel_init() + self.vm.add_args('-smp', + '3,drawers=2,books=2,sockets=3,cores=2,maxcpus=24') + self.vm.launch() + self.wait_until_booted()
tests/avocado/s390_topology.py def test_dedicated(self): · line 342
Quoted 10 lines
tests/avocado/s390_topology.py +10
-0+ + self.system_init() + + res = self.vm.qmp('set-cpu-topology', + {'core-id': 2, 'socket-id': 0, 'book-id': 0}) + self.assertEqual(res['error']['class'], 'GenericError') + + res = self.vm.qmp('set-cpu-topology', + {'core-id': 2, 'socket-id': 0, 'book-id': 1}) + self.assertEqual(res['return'], {})
Re: [PATCH v21 18/20] tests/avocado: s390x cpu topology test socket full
Quoted 37 lines · Thomas Huth
Quoted 36 lines · Pierre Morel
This test verifies that QMP set-cpu-topology does not accept to overload a socket.
Signed-off-by Pierre Morel <[email protected]>---
tests/avocado/s390_topology.py| 25 +++++++++++++++++++++++++1 file changed, 25 insertions(+)tests/avocado/s390_topology.py +4
-0@@ -315,3 +315,28 @@ def test_dedicated(self): self.guest_set_dispatching('0'); self.check_topology(0, 0, 0, 0, 'high', True) + + def test_socket_full(self): + """ + This test verifies that QEMU does not accept to overload asocket.
+ The socket-id 0 on book-id 0 already contains CPUs 0 and 1 and can
+ not accept any new CPU while socket-id 0 on book-id 1 is free.tests/avocado/s390_topology.py +13
-0+ + :avocado: tags=arch:s390x + :avocado: tags=machine:s390-ccw-virtio + """ + self.kernel_init() + self.vm.add_args('-smp', + '3,drawers=2,books=2,sockets=3,cores=2,maxcpus=24') + self.vm.launch() + self.wait_until_booted() + + self.system_init() + + res = self.vm.qmp('set-cpu-topology',
Quoted 9 lines
Quoted 6 lines
+ {'core-id': 2, 'socket-id': 0, 'book-id': 0}) +
self.assertEqual(res['error']['class'], 'GenericError')+
+ res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 2, 'socket-id': 0, 'book-id': 1}) +self.assertEqual(res['return'], {})Reviewed-by Thomas Huth <[email protected]>
Thanks,
Pierre
PATCH v21 19/20 tests/avocado: s390x cpu topology dedicated errors
In reply to Pierre MorelLet's test that QEMU refuses to setup a dedicated CPU with low or medium entitlement.
--- tests/avocado/s390_topology.py | 48 ++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+)
tests/avocado/s390_topology.py +48 -0
@@ -340,3 +340,51 @@ def test_socket_full(self): res = self.vm.qmp('set-cpu-topology', {'core-id': 2, 'socket-id': 0, 'book-id': 1}) self.assertEqual(res['return'], {}) + + def test_dedicated_error(self): + """ + This test verifies that QEMU refuses to lower the entitlement + of a dedicated CPU + + :avocado: tags=arch:s390x + :avocado: tags=machine:s390-ccw-virtio + """ + self.kernel_init() + self.vm.launch() + self.wait_until_booted() + + self.system_init() + + res = self.vm.qmp('set-cpu-topology', + {'core-id': 0, 'dedicated': True}) + self.assertEqual(res['return'], {}) + + self.check_topology(0, 0, 0, 0, 'high', True) + + self.guest_set_dispatching('1'); + + self.check_topology(0, 0, 0, 0, 'high', True) + + res = self.vm.qmp('set-cpu-topology', + {'core-id': 0, 'entitlement': 'low', 'dedicated': True}) + self.assertEqual(res['error']['class'], 'GenericError') + + res = self.vm.qmp('set-cpu-topology', + {'core-id': 0, 'entitlement': 'low'}) + self.assertEqual(res['error']['class'], 'GenericError') + + res = self.vm.qmp('set-cpu-topology', + {'core-id': 0, 'entitlement': 'medium', 'dedicated': True}) + self.assertEqual(res['error']['class'], 'GenericError') + + res = self.vm.qmp('set-cpu-topology', + {'core-id': 0, 'entitlement': 'medium'}) + self.assertEqual(res['error']['class'], 'GenericError') + + res = self.vm.qmp('set-cpu-topology', + {'core-id': 0, 'entitlement': 'low', 'dedicated': False}) + self.assertEqual(res['return'], {}) + + res = self.vm.qmp('set-cpu-topology', + {'core-id': 0, 'entitlement': 'medium', 'dedicated': False}) + self.assertEqual(res['return'], {})
Signature · 1 line
2.31.1
Re: [PATCH v21 19/20] tests/avocado: s390x cpu topology dedicated errors
Quoted 7 lines · Pierre Morel
Let's test that QEMU refuses to setup a dedicated CPU with low or medium entitlement.
Signed-off-by Pierre Morel <[email protected]>--- tests/avocado/s390_topology.py | 48 ++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+)
Re: [PATCH v21 19/20] tests/avocado: s390x cpu topology dedicated errors
Quoted 11 lines · Thomas Huth
Quoted 7 lines · Pierre Morel
Let's test that QEMU refuses to setup a dedicated CPU with low or medium entitlement.
Signed-off-by Pierre Morel <[email protected]>---
tests/avocado/s390_topology.py| 48 ++++++++++++++++++++++++++++++++++1 file changed, 48 insertions(+)Reviewed-by Thomas Huth <[email protected]>
Thanks,
Pierre
PATCH v21 20/20 tests/avocado: s390x cpu topology bad move
In reply to Pierre MorelThis test verifies that QEMU refuses to move a CPU to an unexistant location.
--- tests/avocado/s390_topology.py | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)
tests/avocado/s390_topology.py +25 -0
@@ -388,3 +388,28 @@ def test_dedicated_error(self): res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'entitlement': 'medium', 'dedicated': False}) self.assertEqual(res['return'], {}) + + def test_move_error(self): + """ + This test verifies that QEMU refuses to move a CPU to an + unexistant location + + :avocado: tags=arch:s390x + :avocado: tags=machine:s390-ccw-virtio + """ + self.kernel_init() + self.vm.launch() + self.wait_until_booted() + + self.system_init() + + res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'drawer-id': 1}) + self.assertEqual(res['error']['class'], 'GenericError') + + res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'book-id': 1}) + self.assertEqual(res['error']['class'], 'GenericError') + + res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'socket-id': 1}) + self.assertEqual(res['error']['class'], 'GenericError') + + self.check_topology(0, 0, 0, 0, 'medium', False)
Signature · 1 line
2.31.1
Re: [PATCH v21 20/20] tests/avocado: s390x cpu topology bad move
Quoted 2 lines · Pierre Morel
This test verifies that QEMU refuses to move a CPU to an unexistant location.
s/unexistant/nonexistent/ ?
Quoted 10 lines
Signed-off-by Pierre Morel <[email protected]>--- tests/avocado/s390_topology.py | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+)tests/avocado/s390_topology.py +0
-0
tests/avocado/s390_topology.py def test_dedicated_error(self): · line 395
Quoted 8 lines
tests/avocado/s390_topology.py +5
-0@@ -388,3 +388,28 @@ def test_dedicated_error(self): res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'entitlement': 'medium', 'dedicated': False}) self.assertEqual(res['return'], {}) + + def test_move_error(self): + """ + This test verifies that QEMU refuses to move a CPU to an + unexistant location
s/unexistant/nonexistent/ ?
With the words fixed:
Re: [PATCH v21 20/20] tests/avocado: s390x cpu topology bad move
Quoted 27 lines · Thomas Huth
Quoted 2 lines · Pierre Morel
This test verifies that QEMU refuses to move a CPU to an unexistant location.
s/unexistant/nonexistent/ ?
Quoted 10 lines
Signed-off-by Pierre Morel <[email protected]>---
tests/avocado/s390_topology.py| 25 +++++++++++++++++++++++++1 file changed, 25 insertions(+)tests/avocado/s390_topology.py +0
-0tests/avocado/s390_topology.py def test_dedicated_error(self): · line 395
Quoted 8 lines
tests/avocado/s390_topology.py +5
-0@@ -388,3 +388,28 @@ def test_dedicated_error(self): res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'entitlement': 'medium', 'dedicated': False}) self.assertEqual(res['return'], {}) + + def test_move_error(self): + """ + This test verifies that QEMU refuses to move a CPU to an + unexistant locations/unexistant/nonexistent/ ?
Quoted 3 lines
With the words fixed:
Reviewed-by Thomas Huth <[email protected]>
I fix it, thanks,
Regards,
Pierre
On 30/06/2023 11.17, Pierre Morel wrote:
...
Quoted 10 lines
Testing
=======To use the QEMU patches, you will need Linux V6-rc1 or newer, or use the following Linux mainline patches:
f5ecfee944932022-07-20 KVM: s390: resetting the Topology-Change-Report24fe0195bc192022-07-20 KVM: s390: guest support for topology function0130337ec45b2022-07-20 KVM: s390: Cleanup ipte lock access and SIIF fac..
Quoted 5 lines
Currently this code is for KVM only, I have no idea if it is interesting to provide a TCG patch. If ever it will be done in another series.
This series provide 12 avocado tests using Fedora-35 kernel and initrd image.
Hi Pierre,
the new avocado tests currently fail if you run them on a x86 host. Could you please add a check that they are properly skipped instead if the environment does not match? I guess a
self.require_accelerator('kvm')
should do the job...
Thomas
Quoted 20 lines · Thomas Huth
On 30/06/2023 11.17, Pierre Morel wrote:
...Quoted 10 lines
Testing
=======To use the QEMU patches, you will need Linux V6-rc1 or newer, or use the following Linux mainline patches:
f5ecfee944932022-07-20 KVM: s390: resetting the Topology-Change-Report24fe0195bc192022-07-20 KVM: s390: guest support for topology function0130337ec45b2022-07-20 KVM: s390: Cleanup ipte lock access and SIIF fac..Quoted 5 lines
Currently this code is for KVM only, I have no idea if it is interesting to provide a TCG patch. If ever it will be done in another series.
This series provide 12 avocado tests using Fedora-35 kernel and initrd image.
Hi Pierre,
Quoted 8 lines
the new avocado tests currently fail if you run them on a x86 host. Could you please add a check that they are properly skipped instead if the environment does not match? I guess a
self.require_accelerator('kvm')
should do the job...
Thomas
Yes, thanks, I add this during initialization of the VM.
Regards,
Pierre