Skip to contentSkip to conversation

s390x: CPU Topology

PATCH Superseded Pierre Morel 5 participants 36 messages Nov 3, 2022 – Nov 27, 2022 1 message not in the archive

Subsystems (2)

v11 of 21 Superseded Latest is v21 →

lore
Share & tools
lore
#0 lore raw

v11 cover s390x: CPU Topology

Hi,

The implementation of the CPU Topology in QEMU has been drastically modified since the last patch series and the number of LOCs has been greatly reduced.

0) Two new patches in front of the series:
- A preliminary patch to move the machine properties to the class_init routine
- The max thread machine class attribute patch.

Both patches could be taken upstream separatly from each other and from the rest of the series.

1) Unnecessary objects have been removed, only a single S390Topology object is created to support migration and reset.

2) The introduction of drawers and books is deferred to a later version.

3) A new machine property, topology, is added for new machines for test purpose and migration to/from a host without facility 11 from/to a host with the facility 11.

Also a documentation has been added to the series.

To use the QEMU patches, you will need Linux V6-rc1 or newer, or use the following Linux mainline patches:

f5ecfee94493 2022-07-20 KVM: s390: resetting the Topology-Change-Report 24fe0195bc19 2022-07-20 KVM: s390: guest support for topology function 0130337ec45b 2022-07-20 KVM: s390: Cleanup ipte lock access and SIIF fac..

Currently this code is for KVM only, I have no idea if it is interesting to provide a TCG patch. If ever it will be done in another series.

To have a better understanding of the S390x CPU Topology and its implementation in QEMU you can have a look at the documentation in the last patch of this series.

The admin will want to match the host and the guest topology, taking into account that the guest does not recognize multithreading. Consequently, two vCPU assigned to threads of the same real CPU should preferably be assigned to the same socket of the guest machine.

Regards,
Pierre

Pierre Morel (11):
s390x: Register TYPE_S390_CCW_MACHINE properties as class properties s390x/cpu topology: add max_threads machine class attribute s390x/cpu topology: core_id sets s390x CPU topology s390x/cpu topology: reporting the CPU topology to the guest s390x/cpu_topology: resetting the Topology-Change-Report s390x/cpu_topology: CPU topology migration
target/s390x: interception of PTF instruction
s390x/cpu topology: add topology_capable QEMU capability s390x/cpu topology: add topology machine property s390x/cpu_topology: activating CPU topology
docs/s390x: document s390x cpu topology

 docs/system/s390x/cpu-topology.rst |  80 ++++++++
 include/hw/boards.h                |   3 +
 include/hw/s390x/cpu-topology.h    |  45 +++++
 include/hw/s390x/s390-virtio-ccw.h |  10 +
 target/s390x/cpu.h                 |  81 ++++++++
 target/s390x/kvm/kvm_s390x.h       |   1 +
 hw/core/machine.c                  |   3 +
 hw/s390x/cpu-topology.c            | 286 +++++++++++++++++++++++++++++
 hw/s390x/s390-virtio-ccw.c         | 192 +++++++++++++------
 target/s390x/cpu-sysemu.c          |  21 +++
 target/s390x/cpu_topology.c        | 100 ++++++++++
 target/s390x/kvm/kvm.c             |  50 ++++-
 util/qemu-config.c                 |   4 +
 hw/s390x/meson.build               |   1 +
 qemu-options.hx                    |   6 +-
 target/s390x/meson.build           |   1 +
 16 files changed, 827 insertions(+), 57 deletions(-)
 create mode 100644 docs/system/s390x/cpu-topology.rst
 create mode 100644 include/hw/s390x/cpu-topology.h
 create mode 100644 hw/s390x/cpu-topology.c
 create mode 100644 target/s390x/cpu_topology.c
Signature · 112 lines
2.31.1

Changelog:

- since v10

- change machine attribute "topology-disable" to "topology"
  (Cedric)
- Add preliminary patch for machine properties
  (Cedric)
- Use next machine as 7.2
  (Cedric / Connie)
- Remove unecessary mutex
  (Thomas)
- use ENOTSUP return value for kvm_s390_topology_set_mtcr()
  (Cedric)
- Add explanation on container and cpu TLEs
  (Thomas)
- use again cpu and socket count in topology structure
  (Cedric)
- Suppress the S390TopoTLE structure and integrate
  the TLE masks to the socket structure.
  (-)
- the STSI instruction now finds the topology from the machine
  (Cedric)

- since v9

- remove books and drawers

- remove thread denying and replace with a merge
  of cores * threads to specify the CPUs available
  to the guest

- add a class option to avoid topology on older
  machines
  (Cedric)

- Allocate a SYSIB buffer of the maximal length to
  avoid overflow.
  (Nico, Janis)

- suppress redundancy of smp parameters in topology
  and use directly the machine smp structure

- Early check for topology support
  (Cedric)

- since v8

- Linux patches are now mainline

- simplification of the implementation
  (Janis)

- Migration, new machine definition
  (Thomas)

- Documentation

- since v7

- Coherence with the Linux patch series changes for MTCR get
  (Pierre)

- check return values during new CPU creation
  (Thomas)

- Improving codding style and argument usages
  (Thomas)

- since v6

- Changes on smp args in qemu-options
  (Daniel)
  
- changed comments in machine.jason
  (Daniel)
 
- Added reset
  (Janosch)

- since v5

- rebasing on newer QEMU version

- reworked most lines above 80 characters.

- since v4

- Added drawer and books to topology

- Added numa topology

- Added documentation

- since v3

- Added migration
  (Thomas)

- Separated STSI instruction from KVM to prepare TCG
  (Thomas)

- Take care of endianess to prepare TCG
  (Thomas)

- Added comments on STSI CPU container and PFT instruction
  (Thomas)

- Moved enabling the instructions as the last patch
  (Thomas)
#1 lore raw

PATCH v11 1/11 s390x: Register TYPE_S390_CCW_MACHINE properties as class properties

---
 hw/s390x/s390-virtio-ccw.c | 127 +++++++++++++++++++++----------------
 1 file changed, 72 insertions(+), 55 deletions(-)
hw/s390x/s390-virtio-ccw.c +72 -55
index 1cc20d8717..567498e780 100644
@@ -43,6 +43,7 @@
 #include "sysemu/sysemu.h"
 #include "hw/s390x/pv.h"
 #include "migration/blocker.h"
+#include "qapi/visitor.h"
 
 static Error *pv_mig_blocker;
 
@@ -589,38 +590,6 @@ static ram_addr_t s390_fixup_ram_size(ram_addr_t sz)
     return newsz;
 }
 
-static void ccw_machine_class_init(ObjectClass *oc, void *data)
-{
-    MachineClass *mc = MACHINE_CLASS(oc);
-    NMIClass *nc = NMI_CLASS(oc);
-    HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(oc);
-    S390CcwMachineClass *s390mc = S390_CCW_MACHINE_CLASS(mc);
-
-    s390mc->ri_allowed = true;
-    s390mc->cpu_model_allowed = true;
-    s390mc->css_migration_enabled = true;
-    s390mc->hpage_1m_allowed = true;
-    mc->init = ccw_init;
-    mc->reset = s390_machine_reset;
-    mc->block_default_type = IF_VIRTIO;
-    mc->no_cdrom = 1;
-    mc->no_floppy = 1;
-    mc->no_parallel = 1;
-    mc->no_sdcard = 1;
-    mc->max_cpus = S390_MAX_CPUS;
-    mc->has_hotpluggable_cpus = true;
-    assert(!mc->get_hotplug_handler);
-    mc->get_hotplug_handler = s390_get_hotplug_handler;
-    mc->cpu_index_to_instance_props = s390_cpu_index_to_props;
-    mc->possible_cpu_arch_ids = s390_possible_cpu_arch_ids;
-    /* it is overridden with 'host' cpu *in kvm_arch_init* */
-    mc->default_cpu_type = S390_CPU_TYPE_NAME("qemu");
-    hc->plug = s390_machine_device_plug;
-    hc->unplug_request = s390_machine_device_unplug_request;
-    nc->nmi_monitor_handler = s390_nmi;
-    mc->default_ram_id = "s390.ram";
-}
-
 static inline bool machine_get_aes_key_wrap(Object *obj, Error **errp)
 {
     S390CcwMachineState *ms = S390_CCW_MACHINE(obj);
@@ -710,19 +679,29 @@ bool hpage_1m_allowed(void)
     return get_machine_class()->hpage_1m_allowed;
 }
 
-static char *machine_get_loadparm(Object *obj, Error **errp)
+static void machine_get_loadparm(Object *obj, Visitor *v,
+                                 const char *name, void *opaque,
+                                 Error **errp)
 {
     S390CcwMachineState *ms = S390_CCW_MACHINE(obj);
+    char *str = g_strndup((char *) ms->loadparm, sizeof(ms->loadparm));
 
-    /* make a NUL-terminated string */
-    return g_strndup((char *) ms->loadparm, sizeof(ms->loadparm));
+    visit_type_str(v, name, &str, errp);
+    g_free(str);
 }
 
-static void machine_set_loadparm(Object *obj, const char *val, Error **errp)
+static void machine_set_loadparm(Object *obj, Visitor *v,
+                                 const char *name, void *opaque,
+                                 Error **errp)
 {
     S390CcwMachineState *ms = S390_CCW_MACHINE(obj);
+    char *val;
     int i;
 
+    if (!visit_type_str(v, name, &val, errp)) {
+        return;
+    }
+
     for (i = 0; i < sizeof(ms->loadparm) && val[i]; i++) {
         uint8_t c = qemu_toupper(val[i]); /* mimic HMC */
 
@@ -740,34 +719,72 @@ static void machine_set_loadparm(Object *obj, const char *val, Error **errp)
         ms->loadparm[i] = ' '; /* pad right with spaces */
     }
 }
-static inline void s390_machine_initfn(Object *obj)
+
+static void ccw_machine_class_init(ObjectClass *oc, void *data)
 {
-    object_property_add_bool(obj, "aes-key-wrap",
-                             machine_get_aes_key_wrap,
-                             machine_set_aes_key_wrap);
-    object_property_set_description(obj, "aes-key-wrap",
+    MachineClass *mc = MACHINE_CLASS(oc);
+    NMIClass *nc = NMI_CLASS(oc);
+    HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(oc);
+    S390CcwMachineClass *s390mc = S390_CCW_MACHINE_CLASS(mc);
+
+    s390mc->ri_allowed = true;
+    s390mc->cpu_model_allowed = true;
+    s390mc->css_migration_enabled = true;
+    s390mc->hpage_1m_allowed = true;
+    mc->init = ccw_init;
+    mc->reset = s390_machine_reset;
+    mc->block_default_type = IF_VIRTIO;
+    mc->no_cdrom = 1;
+    mc->no_floppy = 1;
+    mc->no_parallel = 1;
+    mc->no_sdcard = 1;
+    mc->max_cpus = S390_MAX_CPUS;
+    mc->has_hotpluggable_cpus = true;
+    assert(!mc->get_hotplug_handler);
+    mc->get_hotplug_handler = s390_get_hotplug_handler;
+    mc->cpu_index_to_instance_props = s390_cpu_index_to_props;
+    mc->possible_cpu_arch_ids = s390_possible_cpu_arch_ids;
+    /* it is overridden with 'host' cpu *in kvm_arch_init* */
+    mc->default_cpu_type = S390_CPU_TYPE_NAME("qemu");
+    hc->plug = s390_machine_device_plug;
+    hc->unplug_request = s390_machine_device_unplug_request;
+    nc->nmi_monitor_handler = s390_nmi;
+    mc->default_ram_id = "s390.ram";
+
+    object_class_property_add_bool(oc, "aes-key-wrap",
+                                   machine_get_aes_key_wrap,
+                                   machine_set_aes_key_wrap);
+    object_class_property_set_description(oc, "aes-key-wrap",
             "enable/disable AES key wrapping using the CPACF wrapping key");
-    object_property_set_bool(obj, "aes-key-wrap", true, NULL);
 
-    object_property_add_bool(obj, "dea-key-wrap",
-                             machine_get_dea_key_wrap,
-                             machine_set_dea_key_wrap);
-    object_property_set_description(obj, "dea-key-wrap",
+    object_class_property_add_bool(oc, "dea-key-wrap",
+                                   machine_get_dea_key_wrap,
+                                   machine_set_dea_key_wrap);
+    object_class_property_set_description(oc, "dea-key-wrap",
             "enable/disable DEA key wrapping using the CPACF wrapping key");
-    object_property_set_bool(obj, "dea-key-wrap", true, NULL);
-    object_property_add_str(obj, "loadparm",
-            machine_get_loadparm, machine_set_loadparm);
-    object_property_set_description(obj, "loadparm",
+
+    object_class_property_add(oc, "loadparm", "loadparm",
+                              machine_get_loadparm, machine_set_loadparm,
+                              NULL, NULL);
+    object_class_property_set_description(oc, "loadparm",
             "Up to 8 chars in set of [A-Za-z0-9. ] (lower case chars converted"
             " to upper case) to pass to machine loader, boot manager,"
             " and guest kernel");
 
-    object_property_add_bool(obj, "zpcii-disable",
-                             machine_get_zpcii_disable,
-                             machine_set_zpcii_disable);
-    object_property_set_description(obj, "zpcii-disable",
+    object_class_property_add_bool(oc, "zpcii-disable",
+                                   machine_get_zpcii_disable,
+                                   machine_set_zpcii_disable);
+    object_class_property_set_description(oc, "zpcii-disable",
             "disable zPCI interpretation facilties");
-    object_property_set_bool(obj, "zpcii-disable", false, NULL);
+}
+
+static inline void s390_machine_initfn(Object *obj)
+{
+    S390CcwMachineState *ms = S390_CCW_MACHINE(obj);
+
+    ms->aes_key_wrap = true;
+    ms->dea_key_wrap = true;
+    ms->zpcii_disable = false;
 }
 
 static const TypeInfo ccw_machine_info = {
Signature · 1 line
2.31.1
#2 lore raw

Re: [PATCH v11 01/11] s390x: Register TYPE_S390_CCW_MACHINE properties as class properties

Quoted 4 lines · Pierre Morel
---
  hw/s390x/s390-virtio-ccw.c | 127 +++++++++++++++++++++----------------
  1 file changed, 72 insertions(+), 55 deletions(-)

-EMISSINGPATCHDESCRIPTION

... please add some words *why* this is a good idea / necessary.

Thanks,
Thomas

#3 lore raw

Re: [PATCH v11 01/11] s390x: Register TYPE_S390_CCW_MACHINE properties as class properties

Quoted 9 lines · Thomas Huth
Quoted 4 lines · Pierre Morel

---
  hw/s390x/s390-virtio-ccw.c | 127 +++++++++++++++++++++----------------

  1 file changed, 72 insertions(+), 55 deletions(-)

-EMISSINGPATCHDESCRIPTION

... please add some words *why* this is a good idea / necessary.

I saw that the i386 patch had no description for the same patch so...

To be honest I do not know why it is necessary.
The only reason I see is to be in sync with the PC implementation.

So what about:
"
Register TYPE_S390_CCW_MACHINE properties as class properties to be conform with the X architectures
"
?

@Cédric , any official recommendation for doing that?

Quoted 5 lines

 Thanks,
  Thomas

Signature · 2 lines
Pierre Morel
IBM Lab Boeblingen
Reviewed-by
#4 lore raw

Re: [PATCH v11 01/11] s390x: Register TYPE_S390_CCW_MACHINE properties as class properties

Quoted 18 lines · Pierre Morel
Quoted 9 lines · Thomas Huth
Quoted 4 lines · Pierre Morel

---
  hw/s390x/s390-virtio-ccw.c | 127 +++++++++++++++++++++----------------

  1 file changed, 72 insertions(+), 55 deletions(-)

-EMISSINGPATCHDESCRIPTION

... please add some words *why* this is a good idea / necessary.

I saw that the i386 patch had no description for the same patch so...

To be honest I do not know why it is necessary.
The only reason I see is to be in sync with the PC implementation.

Quoted 8 lines

So what about:
"
Register TYPE_S390_CCW_MACHINE properties as class properties to be conform with the X architectures
"
?

@Cédric , any official recommendation for doing that?

There was a bunch of commits related to QOM in this series :

91def7b83 arm/virt: Register most properties as class properties f5730c69f0 i386: Register feature bit properties as class properties

which moved property definitions at the class level.

Then,

commit d8fb7d0969 ("vl: switch -M parsing to keyval")

changed machine_help_func() to use a machine class and not machine instance anymore.

I would use the same kind of commit log and add a Fixes tag to get it merged in 7.2

With that,

Thanks,

C.

#5 lore raw

Re: [PATCH v11 01/11] s390x: Register TYPE_S390_CCW_MACHINE properties as class properties

Quoted 46 lines · Cédric Le Goater
Quoted 19 lines · Pierre Morel
Quoted 10 lines · Thomas Huth
Quoted 5 lines · Pierre Morel

---
  hw/s390x/s390-virtio-ccw.c | 127
+++++++++++++++++++++----------------

  1 file changed, 72 insertions(+), 55 deletions(-)

-EMISSINGPATCHDESCRIPTION

... please add some words *why* this is a good idea / necessary.

I saw that the i386 patch had no description for the same patch so...

To be honest I do not know why it is necessary.
The only reason I see is to be in sync with the PC implementation.

Quoted 8 lines

So what about:
"
Register TYPE_S390_CCW_MACHINE properties as class properties to be conform with the X architectures
"
?

@Cédric , any official recommendation for doing that?

There was a bunch of commits related to QOM in this series :

  91def7b83 arm/virt: Register most properties as class properties f5730c69f0 i386: Register feature bit properties as class properties

which moved property definitions at the class level.

Then,

  commit d8fb7d0969 ("vl: switch -M parsing to keyval")

changed machine_help_func() to use a machine class and not machine instance anymore.

I would use the same kind of commit log and add a Fixes tag to get it merged in 7.2

Quoted 9 lines

With that,

Thanks,

C.

OK,
Thanks,

Pierre

Signature · 2 lines
Pierre Morel
IBM Lab Boeblingen
#6 lore raw

Re: [PATCH v11 01/11] s390x: Register TYPE_S390_CCW_MACHINE properties as class properties

In reply to Cédric Le Goater
Quoted 37 lines · Cédric Le Goater
Quoted 18 lines · Pierre Morel
Quoted 9 lines · Thomas Huth
Quoted 4 lines · Pierre Morel

---
  hw/s390x/s390-virtio-ccw.c | 127 +++++++++++++++++++++----------------

  1 file changed, 72 insertions(+), 55 deletions(-)

-EMISSINGPATCHDESCRIPTION

... please add some words *why* this is a good idea / necessary.

I saw that the i386 patch had no description for the same patch so...

To be honest I do not know why it is necessary.
The only reason I see is to be in sync with the PC implementation.

Quoted 8 lines

So what about:
"
Register TYPE_S390_CCW_MACHINE properties as class properties to be conform with the X architectures
"
?

@Cédric , any official recommendation for doing that?

There was a bunch of commits related to QOM in this series :

  91def7b83 arm/virt: Register most properties as class properties f5730c69f0 i386: Register feature bit properties as class properties

which moved property definitions at the class level.

Then,

Quoted 7 lines

  commit d8fb7d0969 ("vl: switch -M parsing to keyval")

changed machine_help_func() to use a machine class and not machine instance anymore.

I would use the same kind of commit log and add a Fixes tag to get it merged in 7.2

Ah, so this fixes the problem that running QEMU with " -M s390-ccw-virtio,help" does not show the s390x-specific properties anymore? ... that's certainly somethings that should be mentioned in the commit message! What about something like this:

"Currently, when running 'qemu-system-s390x -M -M s390-ccw-virtio,help' the s390x-specific properties are not listed anymore. This happens because since commit d8fb7d0969 ("vl: switch -M parsing to keyval") the properties have to be defined at the class level and not at the instance level anymore. Fix it on s390x now, too, by moving the registration of the properties to the class level"

Fixes d8fb7d0969 ("vl: switch -M parsing to keyval")

?

Thomas

#7 lore raw

Re: [PATCH v11 01/11] s390x: Register TYPE_S390_CCW_MACHINE properties as class properties

Quoted 49 lines · Thomas Huth
Quoted 38 lines · Cédric Le Goater
Quoted 19 lines · Pierre Morel
Quoted 10 lines · Thomas Huth
Quoted 5 lines · Pierre Morel

---
  hw/s390x/s390-virtio-ccw.c | 127
+++++++++++++++++++++----------------

  1 file changed, 72 insertions(+), 55 deletions(-)

-EMISSINGPATCHDESCRIPTION

... please add some words *why* this is a good idea / necessary.

I saw that the i386 patch had no description for the same patch so...

To be honest I do not know why it is necessary.
The only reason I see is to be in sync with the PC implementation.

Quoted 8 lines

So what about:
"
Register TYPE_S390_CCW_MACHINE properties as class properties to be conform with the X architectures
"
?

@Cédric , any official recommendation for doing that?

There was a bunch of commits related to QOM in this series :

   91def7b83 arm/virt: Register most properties as class properties f5730c69f0 i386: Register feature bit properties as class properties

which moved property definitions at the class level.

Then,

Quoted 7 lines

   commit d8fb7d0969 ("vl: switch -M parsing to keyval")

changed machine_help_func() to use a machine class and not machine instance anymore.

I would use the same kind of commit log and add a Fixes tag to get it merged in 7.2

Ah, so this fixes the problem that running QEMU with " -M s390-ccw-virtio,help" does not show the s390x-specific properties anymore? ... that's certainly somethings that should be mentioned in the commit message! What about something like this:

Quoted 8 lines

"Currently, when running 'qemu-system-s390x -M -M s390-ccw-virtio,help' the s390x-specific properties are not listed anymore. This happens because since commit d8fb7d0969 ("vl: switch -M parsing to keyval") the properties have to be defined at the class level and not at the instance level anymore. Fix it on s390x now, too, by moving the registration of the properties to the class level"

Fixes d8fb7d0969 ("vl: switch -M parsing to keyval")

?

 Thomas

That seems really good :)

Thank you Thomas!

Regards,
Pierre

Signature · 2 lines
Pierre Morel
IBM Lab Boeblingen
#8 lore raw

Re: [PATCH v11 01/11] s390x: Register TYPE_S390_CCW_MACHINE properties as class properties

Quoted 53 lines · Pierre Morel
Quoted 50 lines · Thomas Huth
Quoted 37 lines · Cédric Le Goater
Quoted 18 lines · Pierre Morel
Quoted 9 lines · Thomas Huth
Quoted 4 lines · Pierre Morel

---
  hw/s390x/s390-virtio-ccw.c | 127 +++++++++++++++++++++----------------

  1 file changed, 72 insertions(+), 55 deletions(-)

-EMISSINGPATCHDESCRIPTION

... please add some words *why* this is a good idea / necessary.

I saw that the i386 patch had no description for the same patch so...

To be honest I do not know why it is necessary.
The only reason I see is to be in sync with the PC implementation.

Quoted 8 lines

So what about:
"
Register TYPE_S390_CCW_MACHINE properties as class properties to be conform with the X architectures
"
?

@Cédric , any official recommendation for doing that?

There was a bunch of commits related to QOM in this series :

   91def7b83 arm/virt: Register most properties as class properties f5730c69f0 i386: Register feature bit properties as class properties

which moved property definitions at the class level.

Then,

Quoted 7 lines

   commit d8fb7d0969 ("vl: switch -M parsing to keyval")

changed machine_help_func() to use a machine class and not machine instance anymore.

I would use the same kind of commit log and add a Fixes tag to get it merged in 7.2

Ah, so this fixes the problem that running QEMU with " -M s390-ccw-virtio,help" does not show the s390x-specific properties anymore? ... that's certainly somethings that should be mentioned in the commit message! What about something like this:

"Currently, when running 'qemu-system-s390x -M -M s390-ccw-virtio,help' the s390x-specific properties are not listed anymore. This happens because since commit d8fb7d0969 ("vl: switch -M parsing to keyval") the properties have to be defined at the class level and not at the instance level anymore. Fix it on s390x now, too, by moving the registration of the properties to the class level"

Quoted 8 lines
Quoted 6 lines
Fixes d8fb7d0969 ("vl: switch -M parsing to keyval")

?

  Thomas

That seems really good :)

All right, I've queued this patch (with the updated commit description) and the next one on my s390x-branch for QEMU 7.2:

https://gitlab.com/thuth/qemu/-/commits/s390x-next/

Thomas

#9 lore raw

Re: [PATCH v11 01/11] s390x: Register TYPE_S390_CCW_MACHINE properties as class properties

Quoted 62 lines · Thomas Huth
Quoted 61 lines · Pierre Morel
Quoted 57 lines · Thomas Huth
Quoted 38 lines · Cédric Le Goater
Quoted 19 lines · Pierre Morel
Quoted 10 lines · Thomas Huth
Quoted 5 lines · Pierre Morel

---
  hw/s390x/s390-virtio-ccw.c | 127
+++++++++++++++++++++----------------

  1 file changed, 72 insertions(+), 55 deletions(-)

-EMISSINGPATCHDESCRIPTION

... please add some words *why* this is a good idea / necessary.

I saw that the i386 patch had no description for the same patch so...

To be honest I do not know why it is necessary.
The only reason I see is to be in sync with the PC implementation.

Quoted 8 lines

So what about:
"
Register TYPE_S390_CCW_MACHINE properties as class properties to be conform with the X architectures
"
?

@Cédric , any official recommendation for doing that?

There was a bunch of commits related to QOM in this series :

   91def7b83 arm/virt: Register most properties as class properties f5730c69f0 i386: Register feature bit properties as class properties

which moved property definitions at the class level.

Then,

Quoted 7 lines

   commit d8fb7d0969 ("vl: switch -M parsing to keyval")

changed machine_help_func() to use a machine class and not machine instance anymore.

I would use the same kind of commit log and add a Fixes tag to get it merged in 7.2

Ah, so this fixes the problem that running QEMU with " -M s390-ccw-virtio,help" does not show the s390x-specific properties anymore? ... that's certainly somethings that should be mentioned in the commit message! What about something like this:

"Currently, when running 'qemu-system-s390x -M -M s390-ccw-virtio,help' the s390x-specific properties are not listed anymore. This happens because since commit d8fb7d0969 ("vl: switch -M parsing to keyval") the properties have to be defined at the class level and not at the instance level anymore. Fix it on s390x now, too, by moving the registration of the properties to the class level"

Fixes d8fb7d0969 ("vl: switch -M parsing to keyval")

?

  Thomas

Quoted 9 lines
Quoted 1 line

That seems really good :)

All right, I've queued this patch (with the updated commit description) and the next one on my s390x-branch for QEMU 7.2:

 https://gitlab.com/thuth/qemu/-/commits/s390x-next/

 Thomas

Thank you!

Regards,
Pierre

Signature · 2 lines
Pierre Morel
IBM Lab Boeblingen
#10 lore raw

PATCH v11 2/11 s390x/cpu topology: add max_threads machine class attribute

In reply to Pierre Morel

The S390 CPU topology accepts the smp.threads argument while in reality it does not effectively allow multthreading.

Let's keep this behavior for machines older than 7.3 and refuse to use threads in newer machines until multithreading is really proposed to the guest by the machine.

---
 include/hw/s390x/s390-virtio-ccw.h |  1 +
 hw/s390x/s390-virtio-ccw.c         | 11 +++++++++++
 2 files changed, 12 insertions(+)
include/hw/s390x/s390-virtio-ccw.h +1 -0
index 8a0090a071..4f8a39abda 100644
@@ -40,6 +40,7 @@ struct S390CcwMachineClass {
     bool cpu_model_allowed;
     bool css_migration_enabled;
     bool hpage_1m_allowed;
+    int max_threads;
 };
 
 /* runtime-instrumentation allowed by the machine */
hw/s390x/s390-virtio-ccw.c +11 -0
index 567498e780..9ab91df5b1 100644
@@ -85,8 +85,15 @@ out:
 static void s390_init_cpus(MachineState *machine)
 {
     MachineClass *mc = MACHINE_GET_CLASS(machine);
+    S390CcwMachineClass *s390mc = S390_CCW_MACHINE_CLASS(mc);
     int i;
 
+    if (machine->smp.threads > s390mc->max_threads) {
+        error_report("S390 does not support more than %d threads.",
+                     s390mc->max_threads);
+        exit(1);
+    }
+
     /* initialize possible_cpus */
     mc->possible_cpu_arch_ids(machine);
 
@@ -731,6 +738,7 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
     s390mc->cpu_model_allowed = true;
     s390mc->css_migration_enabled = true;
     s390mc->hpage_1m_allowed = true;
+    s390mc->max_threads = 1;
     mc->init = ccw_init;
     mc->reset = s390_machine_reset;
     mc->block_default_type = IF_VIRTIO;
@@ -859,8 +867,11 @@ static void ccw_machine_7_1_instance_options(MachineState *machine)
 
 static void ccw_machine_7_1_class_options(MachineClass *mc)
 {
+    S390CcwMachineClass *s390mc = S390_CCW_MACHINE_CLASS(mc);
+
     ccw_machine_7_2_class_options(mc);
     compat_props_add(mc->compat_props, hw_compat_7_1, hw_compat_7_1_len);
+    s390mc->max_threads = S390_MAX_CPUS;
 }
 DEFINE_CCW_MACHINE(7_1, "7.1", false);
 
Signature · 1 line
2.31.1
#11 lore raw

PATCH v11 3/11 s390x/cpu topology: core_id sets s390x CPU topology

In reply to Pierre Morel

In the S390x CPU topology the core_id specifies the CPU address and the position of the core withing the topology.

Let's build the topology based on the core_id.

---
 include/hw/s390x/cpu-topology.h    |  41 ++++++++++
 include/hw/s390x/s390-virtio-ccw.h |   1 +
 target/s390x/cpu.h                 |   2 +
 hw/s390x/cpu-topology.c            | 125 +++++++++++++++++++++++++++++
 hw/s390x/s390-virtio-ccw.c         |  23 ++++++
 hw/s390x/meson.build               |   1 +
 6 files changed, 193 insertions(+)
 create mode 100644 include/hw/s390x/cpu-topology.h
 create mode 100644 hw/s390x/cpu-topology.c
include/hw/s390x/cpu-topology.h new file +41 -0
new file mode 100644
index 0000000000..4e16a2153d
@@ -0,0 +1,41 @@
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+#ifndef HW_S390X_CPU_TOPOLOGY_H
+#define HW_S390X_CPU_TOPOLOGY_H
+
+#include "hw/qdev-core.h"
+#include "qom/object.h"
+
+#define S390_TOPOLOGY_CPU_IFL 0x03
+#define S390_TOPOLOGY_MAX_ORIGIN ((63 + S390_MAX_CPUS) / 64)
+
+typedef struct S390TopoSocket {
+    int active_count;
+    uint64_t mask[S390_TOPOLOGY_MAX_ORIGIN];
+} S390TopoSocket;
+
+struct S390Topology {
+    SysBusDevice parent_obj;
+    uint32_t nr_cpus;
+    uint32_t nr_sockets;
+    S390TopoSocket *socket;
+};
+
+#define TYPE_S390_CPU_TOPOLOGY "s390-topology"
+OBJECT_DECLARE_SIMPLE_TYPE(S390Topology, S390_CPU_TOPOLOGY)
+
+void s390_topology_new_cpu(S390CPU *cpu);
+
+static inline bool s390_has_topology(void)
+{
+    return false;
+}
+
+#endif
include/hw/s390x/s390-virtio-ccw.h +1 -0
index 4f8a39abda..23b472708d 100644
@@ -29,6 +29,7 @@ struct S390CcwMachineState {
     bool pv;
     bool zpcii_disable;
     uint8_t loadparm[8];
+    void *topology;
 };
 
 struct S390CcwMachineClass {
target/s390x/cpu.h +2 -0
index 7d6d01325b..c9066b2496 100644
@@ -175,6 +175,8 @@ struct ArchCPU {
     /* needed for live migration */
     void *irqstate;
     uint32_t irqstate_saved_size;
+    /* Topology this CPU belongs too */
+    void *topology;
 };
 
 
hw/s390x/cpu-topology.c new file +125 -0

125 lines not on this page · raw message

hw/s390x/s390-virtio-ccw.c +23 -0
index 9ab91df5b1..5776d3e58f 100644
@@ -44,6 +44,7 @@
 #include "hw/s390x/pv.h"
 #include "migration/blocker.h"
 #include "qapi/visitor.h"
+#include "hw/s390x/cpu-topology.h"
 
 static Error *pv_mig_blocker;
 
@@ -102,6 +103,19 @@ static void s390_init_cpus(MachineState *machine)
     }
 }
 
+static void s390_init_topology(MachineState *machine)
+{
+    DeviceState *dev;
+
+    if (s390_has_topology()) {
+        dev = qdev_new(TYPE_S390_CPU_TOPOLOGY);
+        object_property_add_child(&machine->parent_obj,
+                                  TYPE_S390_CPU_TOPOLOGY, OBJECT(dev));
+        sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
+        S390_CCW_MACHINE(machine)->topology = dev;
+    }
+}
+
 static const char *const reset_dev_types[] = {
     TYPE_VIRTUAL_CSS_BRIDGE,
     "s390-sclp-event-facility",
@@ -252,6 +266,9 @@ static void ccw_init(MachineState *machine)
     /* init memory + setup max page size. Required for the CPU model */
     s390_memory_init(machine->ram);
 
+    /* Adding the topology must be done before CPU initialization */
+    s390_init_topology(machine);
+
     /* init CPUs (incl. CPU model) early so s390_has_feature() works */
     s390_init_cpus(machine);
 
@@ -314,6 +331,12 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev,
     g_assert(!ms->possible_cpus->cpus[cpu->env.core_id].cpu);
     ms->possible_cpus->cpus[cpu->env.core_id].cpu = OBJECT(dev);
 
+    /* Inserting the CPU in the Topology can not fail */
+    if (S390_CCW_MACHINE(ms)->topology) {
+        cpu->topology = S390_CCW_MACHINE(ms)->topology;
+        s390_topology_new_cpu(cpu);
+    }
+
     if (dev->hotplugged) {
         raise_irq_cpu_hotplug();
     }
hw/s390x/meson.build +1 -0
index f291016fee..653f6ab488 100644
@@ -2,6 +2,7 @@ s390x_ss = ss.source_set()
 s390x_ss.add(files(
   'ap-bridge.c',
   'ap-device.c',
+  'cpu-topology.c',
   'ccw-device.c',
   'css-bridge.c',
   'css.c',
Signature · 1 line
2.31.1
#12 lore raw

Re: [PATCH v11 03/11] s390x/cpu topology: core_id sets s390x CPU topology

Hello Pierre,

Quoted 69 lines · Pierre Morel

In the S390x CPU topology the core_id specifies the CPU address and the position of the core withing the topology.

Let's build the topology based on the core_id.

---
  include/hw/s390x/cpu-topology.h    |  41 ++++++++++
  include/hw/s390x/s390-virtio-ccw.h |   1 +
  target/s390x/cpu.h                 |   2 +
  hw/s390x/cpu-topology.c            | 125 +++++++++++++++++++++++++++++
  hw/s390x/s390-virtio-ccw.c         |  23 ++++++
  hw/s390x/meson.build               |   1 +
  6 files changed, 193 insertions(+)

create mode 100644 include/hw/s390x/cpu-topology.h create mode 100644 hw/s390x/cpu-topology.c

include/hw/s390x/cpu-topology.h new file +41 -0
new file mode 100644
index 0000000000..4e16a2153d
@@ -0,0 +1,41 @@
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+#ifndef HW_S390X_CPU_TOPOLOGY_H
+#define HW_S390X_CPU_TOPOLOGY_H
+
+#include "hw/qdev-core.h"
+#include "qom/object.h"
+
+#define S390_TOPOLOGY_CPU_IFL 0x03
+#define S390_TOPOLOGY_MAX_ORIGIN ((63 + S390_MAX_CPUS) / 64)
+
+typedef struct S390TopoSocket {
+    int active_count;
+    uint64_t mask[S390_TOPOLOGY_MAX_ORIGIN];
+} S390TopoSocket;
+
+struct S390Topology {
+    SysBusDevice parent_obj;
+    uint32_t nr_cpus;
+    uint32_t nr_sockets;
+    S390TopoSocket *socket;
+};
+
+#define TYPE_S390_CPU_TOPOLOGY "s390-topology"
+OBJECT_DECLARE_SIMPLE_TYPE(S390Topology, S390_CPU_TOPOLOGY)
+
+void s390_topology_new_cpu(S390CPU *cpu);
+
+static inline bool s390_has_topology(void)
+{
+    return false;
+}
+
+#endif
include/hw/s390x/s390-virtio-ccw.h +0 -0
index 4f8a39abda..23b472708d 100644

include/hw/s390x/s390-virtio-ccw.h struct S390CcwMachineState { · line 32

Quoted 4 lines
include/hw/s390x/s390-virtio-ccw.h +1 -0
@@ -29,6 +29,7 @@ struct S390CcwMachineState {
      bool pv;
      bool zpcii_disable;
      uint8_t loadparm[8];
+    void *topology;

I think it is safe to use 'DeviceState *' or 'Object *' for the pointer under machine. What is most practical.

target/s390x/cpu.h struct ArchCPU { · line 179

Quoted 13 lines
include/hw/s390x/s390-virtio-ccw.h +0 -0
  };
 
  struct S390CcwMachineClass {
target/s390x/cpu.h +2 -0
index 7d6d01325b..c9066b2496 100644
@@ -175,6 +175,8 @@ struct ArchCPU {
      /* needed for live migration */
      void *irqstate;
      uint32_t irqstate_saved_size;
+    /* Topology this CPU belongs too */
+    void *topology;

However, under the CPU, we don't know what the future changes reserve for us and I would call the attribute 'opaque' or 'machine_data'.

For now, it only holds a reference to the S390Topology device model but it could become a struct with time.

Quoted 40 lines
target/s390x/cpu.h +0 -0
  };
 
 
hw/s390x/cpu-topology.c new file +31 -0
new file mode 100644
index 0000000000..6af41d3d7b
@@ -0,0 +1,125 @@
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022
+ * Author(s): Pierre Morel <[email protected]>
+
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+#include "hw/sysbus.h"
+#include "hw/qdev-properties.h"
+#include "hw/boards.h"
+#include "qemu/typedefs.h"
+#include "target/s390x/cpu.h"
+#include "hw/s390x/s390-virtio-ccw.h"
+#include "hw/s390x/cpu-topology.h"
+
+/*
+ * s390_topology_new_cpu:
+ * @cpu: a pointer to the new CPU
+ *
+ * The topology pointed by S390CPU, gives us the CPU topology
+ * established by the -smp QEMU aruments.
+ * The core-id is used to calculate the position of the CPU inside
+ * the topology:
+ *  - the socket, container TLE, containing the CPU, we have one socket

hw/s390x/cpu-topology.c line 41

Quoted 10 lines
hw/s390x/cpu-topology.c new file +10 -0
+ *    for every nr_cpus (nr_cpus = smp.cores * smp.threads)
+ *  - the CPU TLE inside the socket, we have potentionly up to 4 CPU TLE
+ *    in a container TLE with the assumption that all CPU are identical
+ *    with the same polarity and entitlement because we have maximum 256
+ *    CPUs and each TLE can hold up to 64 identical CPUs.
+ *  - the bit in the 64 bit CPU TLE core mask
+ */
+void s390_topology_new_cpu(S390CPU *cpu)
+{
+    S390Topology *topo = (S390Topology *)cpu->topology;

where is cpu->topology set ?

Quoted 33 lines
hw/s390x/cpu-topology.c new file +33 -0
+    int core_id = cpu->env.core_id;
+    int bit, origin;
+    int socket_id;
+
+    socket_id = core_id / topo->nr_cpus;
+
+    /*
+     * At the core level, each CPU is represented by a bit in a 64bit
+     * uint64_t which represent the presence of a CPU.
+     * The firmware assume that all CPU in a CPU TLE have the same
+     * type, polarization and are all dedicated or shared.
+     * In that case the origin variable represents the offset of the first
+     * CPU in the CPU container.
+     * More than 64 CPUs per socket are represented in several CPU containers
+     * inside the socket container.
+     * The only reason to have several S390TopologyCores inside a socket is
+     * to have more than 64 CPUs.
+     * In that case the origin variable represents the offset of the first CPU
+     * in the CPU container. More than 64 CPUs per socket are represented in
+     * several CPU containers inside the socket container.
+     */
+    bit = core_id;
+    origin = bit / 64;
+    bit %= 64;
+    bit = 63 - bit;
+
+    topo->socket[socket_id].active_count++;
+    set_bit(bit, &topo->socket[socket_id].mask[origin]);
+}
+
+/**
+ * s390_topology_realize:
+ * @dev: the device state

hw/s390x/cpu-topology.c line 84

Quoted 10 lines
hw/s390x/cpu-topology.c new file +10 -0
+ * @errp: the error pointer (not used)
+ *
+ * During realize the machine CPU topology is initialized with the
+ * QEMU -smp parameters.
+ * The maximum count of CPU TLE in the all Topology can not be greater
+ * than the maximum CPUs.
+ */
+static void s390_topology_realize(DeviceState *dev, Error **errp)
+{
+    MachineState *ms = MACHINE(qdev_get_machine());

hmm,

hw/s390x/cpu-topology.c line 88

Quoted 4 lines
hw/s390x/cpu-topology.c new file +4 -0
+    S390Topology *topo = S390_CPU_TOPOLOGY(dev);
+
+    topo->nr_cpus = ms->smp.cores * ms->smp.threads;
+    topo->nr_sockets = ms->smp.sockets;

These properties should be set in s390_init_topology() with :

object_property_set_int(OBJECT(dev), "num-cpus",

                          ms->smp.cores * ms->smp.threads, errp);

object_property_set_int(OBJECT(dev), "num-sockets",

                          ms->smp.sockets, errp);

before calling sysbus_realize_and_unref()

hw/s390x/cpu-topology.c line 89

Quoted 1 line
hw/s390x/cpu-topology.c new file +1 -0
+    topo->socket = g_new0(S390TopoSocket, topo->nr_sockets);

For consistency, you could add an unrealize handler to free the array.

hw/s390x/cpu-topology.c line 94

Quoted 5 lines
hw/s390x/cpu-topology.c new file +5 -0
+}
+
+static Property s390_topology_properties[] = {
+    DEFINE_PROP_UINT32("nr_cpus", S390Topology, nr_cpus, 1),
+    DEFINE_PROP_UINT32("nr_sockets", S390Topology, nr_sockets, 1),

A quick audit of the property names in QEMU code shows that a "num-" prefix is usually preferred.

Quoted 44 lines
hw/s390x/cpu-topology.c new file +31 -0
+    DEFINE_PROP_END_OF_LIST(),
+};
+
+/**
+ * topology_class_init:
+ * @oc: Object class
+ * @data: (not used)
+ *
+ * A very simple object we will need for reset and migration.
+ */
+static void topology_class_init(ObjectClass *oc, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(oc);
+
+    dc->realize = s390_topology_realize;
+    device_class_set_props(dc, s390_topology_properties);
+    set_bit(DEVICE_CATEGORY_MISC, dc->categories);
+}
+
+static const TypeInfo cpu_topology_info = {
+    .name          = TYPE_S390_CPU_TOPOLOGY,
+    .parent        = TYPE_SYS_BUS_DEVICE,
+    .instance_size = sizeof(S390Topology),
+    .class_init    = topology_class_init,
+};
+
+static void topology_register(void)
+{
+    type_register_static(&cpu_topology_info);
+}
+type_init(topology_register);
hw/s390x/s390-virtio-ccw.c +1 -0
index 9ab91df5b1..5776d3e58f 100644
@@ -44,6 +44,7 @@
  #include "hw/s390x/pv.h"
  #include "migration/blocker.h"
  #include "qapi/visitor.h"
+#include "hw/s390x/cpu-topology.h"
 
  static Error *pv_mig_blocker;
 

hw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 110

Quoted 8 lines
hw/s390x/s390-virtio-ccw.c +5 -0
@@ -102,6 +103,19 @@ static void s390_init_cpus(MachineState *machine)
      }
  }
 
+static void s390_init_topology(MachineState *machine)
+{
+    DeviceState *dev;
+
+    if (s390_has_topology()) {

I would move the s390_has_topology() check in the caller.

hw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 115

Quoted 5 lines
hw/s390x/s390-virtio-ccw.c +5 -0
+        dev = qdev_new(TYPE_S390_CPU_TOPOLOGY);
+        object_property_add_child(&machine->parent_obj,
+                                  TYPE_S390_CPU_TOPOLOGY, OBJECT(dev));
+        sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
+        S390_CCW_MACHINE(machine)->topology = dev;

and I would move the assignment in the caller also.

Quoted 17 lines
hw/s390x/s390-virtio-ccw.c +6 -0
+    }
+}
+
  static const char *const reset_dev_types[] = {
      TYPE_VIRTUAL_CSS_BRIDGE,
      "s390-sclp-event-facility",
@@ -252,6 +266,9 @@ static void ccw_init(MachineState *machine)
      /* init memory + setup max page size. Required for the CPU model */
      s390_memory_init(machine->ram);
 
+    /* Adding the topology must be done before CPU initialization */
+    s390_init_topology(machine);
+
      /* init CPUs (incl. CPU model) early so s390_has_feature() works */
      s390_init_cpus(machine);
 

hw/s390x/s390-virtio-ccw.c static void s390_cpu_plug(HotplugHandler *hotplug_dev, · line 336

Quoted 6 lines
hw/s390x/s390-virtio-ccw.c +3 -0
@@ -314,6 +331,12 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev,
      g_assert(!ms->possible_cpus->cpus[cpu->env.core_id].cpu);
      ms->possible_cpus->cpus[cpu->env.core_id].cpu = OBJECT(dev);
 
+    /* Inserting the CPU in the Topology can not fail */
+    if (S390_CCW_MACHINE(ms)->topology) {
+        cpu->topology = S390_CCW_MACHINE(ms)->topology;

Two QOM cast. One should be enough. Please introduce a local variable.

hw/s390x/s390-virtio-ccw.c static void s390_cpu_plug(HotplugHandler *hotplug_dev, · line 337

Quoted 1 line
hw/s390x/s390-virtio-ccw.c +1 -0
+        s390_topology_new_cpu(cpu);

I would pass the 'topology' object as a parameter of s390_topology_new_cpu() and do the cpu->topology assignment in the same routine.

May be rename it also to :

void s390_topology_add_cpu(S390Topology *topo, S390CPU *cpu)

Thanks,

C.

Quoted 17 lines
hw/s390x/s390-virtio-ccw.c +2 -0
+    }
+
      if (dev->hotplugged) {
          raise_irq_cpu_hotplug();
      }
hw/s390x/meson.build +1 -0
index f291016fee..653f6ab488 100644
@@ -2,6 +2,7 @@ s390x_ss = ss.source_set()
  s390x_ss.add(files(
    'ap-bridge.c',
    'ap-device.c',
+  'cpu-topology.c',
    'ccw-device.c',
    'css-bridge.c',
    'css.c',
#13 lore raw

Re: [PATCH v11 03/11] s390x/cpu topology: core_id sets s390x CPU topology

Quoted 72 lines · Cédric Le Goater

Hello Pierre,

Quoted 69 lines · Pierre Morel

In the S390x CPU topology the core_id specifies the CPU address and the position of the core withing the topology.

Let's build the topology based on the core_id.

---
  include/hw/s390x/cpu-topology.h    |  41 ++++++++++ include/hw/s390x/s390-virtio-ccw.h |   1 + target/s390x/cpu.h                 |   2 + hw/s390x/cpu-topology.c            | 125 +++++++++++++++++++++++++++++ hw/s390x/s390-virtio-ccw.c         |  23 ++++++ hw/s390x/meson.build               |   1 +

  6 files changed, 193 insertions(+)

  create mode 100644 include/hw/s390x/cpu-topology.h create mode 100644 hw/s390x/cpu-topology.c

include/hw/s390x/cpu-topology.h new file +6 -0
new file mode 100644
index 0000000000..4e16a2153d
@@ -0,0 +1,41 @@
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or 

(at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */

include/hw/s390x/cpu-topology.h new file +32 -0
+#ifndef HW_S390X_CPU_TOPOLOGY_H
+#define HW_S390X_CPU_TOPOLOGY_H
+
+#include "hw/qdev-core.h"
+#include "qom/object.h"
+
+#define S390_TOPOLOGY_CPU_IFL 0x03
+#define S390_TOPOLOGY_MAX_ORIGIN ((63 + S390_MAX_CPUS) / 64)
+
+typedef struct S390TopoSocket {
+    int active_count;
+    uint64_t mask[S390_TOPOLOGY_MAX_ORIGIN];
+} S390TopoSocket;
+
+struct S390Topology {
+    SysBusDevice parent_obj;
+    uint32_t nr_cpus;
+    uint32_t nr_sockets;
+    S390TopoSocket *socket;
+};
+
+#define TYPE_S390_CPU_TOPOLOGY "s390-topology"
+OBJECT_DECLARE_SIMPLE_TYPE(S390Topology, S390_CPU_TOPOLOGY)
+
+void s390_topology_new_cpu(S390CPU *cpu);
+
+static inline bool s390_has_topology(void)
+{
+    return false;
+}
+
+#endif
include/hw/s390x/s390-virtio-ccw.h +0 -0
index 4f8a39abda..23b472708d 100644
Quoted 8 lines

include/hw/s390x/s390-virtio-ccw.h struct S390CcwMachineState { · line 32

Quoted 5 lines
include/hw/s390x/s390-virtio-ccw.h +1 -0
@@ -29,6 +29,7 @@ struct S390CcwMachineState {
       bool pv;
       bool zpcii_disable;
       uint8_t loadparm[8];
+    void *topology;

I think it is safe to use 'DeviceState *' or 'Object *' for the pointer under machine. What is most practical.

OK, DeviceState bring one less cast..

Quoted 17 lines

target/s390x/cpu.h struct ArchCPU { · line 179

Quoted 13 lines
include/hw/s390x/s390-virtio-ccw.h +0 -0
   };
   struct S390CcwMachineClass {
target/s390x/cpu.h +2 -0
index 7d6d01325b..c9066b2496 100644
@@ -175,6 +175,8 @@ struct ArchCPU {
       /* needed for live migration */
       void *irqstate;
       uint32_t irqstate_saved_size;
+    /* Topology this CPU belongs too */
+    void *topology;

However, under the CPU, we don't know what the future changes reserve for us and I would call the attribute 'opaque' or 'machine_data'.

Quoted 2 lines

For now, it only holds a reference to the S390Topology device model but it could become a struct with time.

OK, I use machine_data

Quoted 45 lines
Quoted 45 lines
target/s390x/cpu.h +0 -0
   };
hw/s390x/cpu-topology.c new file +7 -0
new file mode 100644
index 0000000000..6af41d3d7b
@@ -0,0 +1,125 @@
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022
+ * Author(s): Pierre Morel <[email protected]>
+
+ * This work is licensed under the terms of the GNU GPL, version 2 or 

(at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */

hw/s390x/cpu-topology.c new file +25 -0
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+#include "hw/sysbus.h"
+#include "hw/qdev-properties.h"
+#include "hw/boards.h"
+#include "qemu/typedefs.h"
+#include "target/s390x/cpu.h"
+#include "hw/s390x/s390-virtio-ccw.h"
+#include "hw/s390x/cpu-topology.h"
+
+/*
+ * s390_topology_new_cpu:
+ * @cpu: a pointer to the new CPU
+ *
+ * The topology pointed by S390CPU, gives us the CPU topology
+ * established by the -smp QEMU aruments.
+ * The core-id is used to calculate the position of the CPU inside
+ * the topology:
+ *  - the socket, container TLE, containing the CPU, we have one socket
+ *    for every nr_cpus (nr_cpus = smp.cores * smp.threads)
+ *  - the CPU TLE inside the socket, we have potentionly up to 4 CPU TLE
+ *    in a container TLE with the assumption that all CPU are identical
+ *    with the same polarity and entitlement because we have maximum 256
Quoted 8 lines

hw/s390x/cpu-topology.c line 41

Quoted 6 lines

+ *    CPUs and each TLE can hold up to 64 identical CPUs.
+ *  - the bit in the 64 bit CPU TLE core mask
+ */

hw/s390x/cpu-topology.c new file +3 -0
+void s390_topology_new_cpu(S390CPU *cpu)
+{
+    S390Topology *topo = (S390Topology *)cpu->topology;

where is cpu->topology set ?

In the caller but this is reworked anyway due to a rearangement of the function.

Quoted 43 lines
Quoted 43 lines

+    int core_id = cpu->env.core_id;
+    int bit, origin;
+    int socket_id;

hw/s390x/cpu-topology.c new file +9 -0
+
+    socket_id = core_id / topo->nr_cpus;
+
+    /*
+     * At the core level, each CPU is represented by a bit in a 64bit
+     * uint64_t which represent the presence of a CPU.
+     * The firmware assume that all CPU in a CPU TLE have the same
+     * type, polarization and are all dedicated or shared.
+     * In that case the origin variable represents the offset of the 

first
+     * CPU in the CPU container.
+     * More than 64 CPUs per socket are represented in several CPU containers
+     * inside the socket container.
+     * The only reason to have several S390TopologyCores inside a socket is
+     * to have more than 64 CPUs.
+     * In that case the origin variable represents the offset of the first CPU
+     * in the CPU container. More than 64 CPUs per socket are represented in
+     * several CPU containers inside the socket container. +     */
+    bit = core_id;
+    origin = bit / 64;
+    bit %= 64;
+    bit = 63 - bit;

hw/s390x/cpu-topology.c new file +12 -0
+
+    topo->socket[socket_id].active_count++;
+    set_bit(bit, &topo->socket[socket_id].mask[origin]);
+}
+
+/**
+ * s390_topology_realize:
+ * @dev: the device state
+ * @errp: the error pointer (not used)
+ *
+ * During realize the machine CPU topology is initialized with the
+ * QEMU -smp parameters.
Quoted 8 lines

hw/s390x/cpu-topology.c line 84

Quoted 6 lines

+ * The maximum count of CPU TLE in the all Topology can not be greater
+ * than the maximum CPUs.
+ */

hw/s390x/cpu-topology.c new file +3 -0
+static void s390_topology_realize(DeviceState *dev, Error **errp)
+{
+    MachineState *ms = MACHINE(qdev_get_machine());

hmm,

will disappear

Quoted 8 lines
Quoted 5 lines

+    S390Topology *topo = S390_CPU_TOPOLOGY(dev); +
+    topo->nr_cpus = ms->smp.cores * ms->smp.threads; +    topo->nr_sockets = ms->smp.sockets;

These properties should be set in s390_init_topology() with :

Quoted 7 lines

  object_property_set_int(OBJECT(dev), "num-cpus", ms->smp.cores * ms->smp.threads, errp);

  object_property_set_int(OBJECT(dev), "num-sockets", ms->smp.sockets, errp);

before calling sysbus_realize_and_unref()

OK, done

Quoted 5 lines
Quoted 3 lines

+    topo->socket = g_new0(S390TopoSocket, topo->nr_sockets);

For consistency, you could add an unrealize handler to free the array.

yes

Quoted 9 lines

hw/s390x/cpu-topology.c line 94

Quoted 6 lines
hw/s390x/cpu-topology.c new file +5 -0
+}
+
+static Property s390_topology_properties[] = {
+    DEFINE_PROP_UINT32("nr_cpus", S390Topology, nr_cpus, 1),
+    DEFINE_PROP_UINT32("nr_sockets", S390Topology, nr_sockets, 1),

A quick audit of the property names in QEMU code shows that a "num-" prefix is usually preferred.

OK

Quoted 44 lines
Quoted 44 lines

+    DEFINE_PROP_END_OF_LIST(),

hw/s390x/cpu-topology.c new file +30 -0
+};
+
+/**
+ * topology_class_init:
+ * @oc: Object class
+ * @data: (not used)
+ *
+ * A very simple object we will need for reset and migration.
+ */
+static void topology_class_init(ObjectClass *oc, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(oc);
+
+    dc->realize = s390_topology_realize;
+    device_class_set_props(dc, s390_topology_properties);
+    set_bit(DEVICE_CATEGORY_MISC, dc->categories);
+}
+
+static const TypeInfo cpu_topology_info = {
+    .name          = TYPE_S390_CPU_TOPOLOGY,
+    .parent        = TYPE_SYS_BUS_DEVICE,
+    .instance_size = sizeof(S390Topology),
+    .class_init    = topology_class_init,
+};
+
+static void topology_register(void)
+{
+    type_register_static(&cpu_topology_info);
+}
+type_init(topology_register);
hw/s390x/s390-virtio-ccw.c +1 -0
index 9ab91df5b1..5776d3e58f 100644
@@ -44,6 +44,7 @@
   #include "hw/s390x/pv.h"
   #include "migration/blocker.h"
   #include "qapi/visitor.h"
+#include "hw/s390x/cpu-topology.h"
   static Error *pv_mig_blocker;
@@ -102,6 +103,19 @@ static void s390_init_cpus(MachineState *machine)
       }
Quoted 8 lines

hw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 110

Quoted 6 lines
hw/s390x/s390-virtio-ccw.c +5 -0
   }
+static void s390_init_topology(MachineState *machine)
+{
+    DeviceState *dev;
+
+    if (s390_has_topology()) {

I would move the s390_has_topology() check in the caller.

yes

Quoted 33 lines

hw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 115

Quoted 6 lines
hw/s390x/s390-virtio-ccw.c +5 -0
+        dev = qdev_new(TYPE_S390_CPU_TOPOLOGY);
+        object_property_add_child(&machine->parent_obj,
+                                  TYPE_S390_CPU_TOPOLOGY, OBJECT(dev));
+        sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);
+        S390_CCW_MACHINE(machine)->topology = dev;

and I would move the assignment in the caller also.

Quoted 15 lines
hw/s390x/s390-virtio-ccw.c +6 -0
+    }
+}
+
   static const char *const reset_dev_types[] = {
       TYPE_VIRTUAL_CSS_BRIDGE,
       "s390-sclp-event-facility",
@@ -252,6 +266,9 @@ static void ccw_init(MachineState *machine)
       /* init memory + setup max page size. Required for the CPU model */
       s390_memory_init(machine->ram);
+    /* Adding the topology must be done before CPU initialization */
+    s390_init_topology(machine);
+
       /* init CPUs (incl. CPU model) early so s390_has_feature() works */
       s390_init_cpus(machine);

hw/s390x/s390-virtio-ccw.c static void s390_cpu_plug(HotplugHandler · line 336

Quoted 6 lines
hw/s390x/s390-virtio-ccw.c +3 -0
@@ -314,6 +331,12 @@ static void s390_cpu_plug(HotplugHandler 
 *hotplug_dev,
       g_assert(!ms->possible_cpus->cpus[cpu->env.core_id].cpu);
       ms->possible_cpus->cpus[cpu->env.core_id].cpu = OBJECT(dev);
+    /* Inserting the CPU in the Topology can not fail */
+    if (S390_CCW_MACHINE(ms)->topology) {
+        cpu->topology = S390_CCW_MACHINE(ms)->topology;

Two QOM cast. One should be enough. Please introduce a local variable.

Quoted 8 lines

hw/s390x/s390-virtio-ccw.c static void s390_cpu_plug(HotplugHandler *hotplug_dev, · line 337

Quoted 1 line
hw/s390x/s390-virtio-ccw.c +1 -0
+        s390_topology_new_cpu(cpu);

I would pass the 'topology' object as a parameter of s390_topology_new_cpu() and do the cpu->topology assignment in the same routine.

May be rename it also to :

  void s390_topology_add_cpu(S390Topology *topo, S390CPU *cpu)

OK, better.

Quoted 5 lines

Thanks,

C.

Thanks a lot for your reviewing.

Regards,
Pierre

Signature · 2 lines
Pierre Morel
IBM Lab Boeblingen
#14 lore raw

PATCH v11 4/11 s390x/cpu topology: reporting the CPU topology to the guest

In reply to Pierre Morel

The guest can use the STSI instruction to get a buffer filled with the CPU topology description.

Let us implement the STSI instruction for the basis CPU topology level, level 2.

---
 include/hw/s390x/cpu-topology.h |   6 ++
 target/s390x/cpu.h              |  77 ++++++++++++++++++++++++
 hw/s390x/cpu-topology.c         |   1 -
 target/s390x/cpu_topology.c     | 100 ++++++++++++++++++++++++++++++++
 target/s390x/kvm/kvm.c          |   6 +-
 target/s390x/meson.build        |   1 +
 6 files changed, 189 insertions(+), 2 deletions(-)
 create mode 100644 target/s390x/cpu_topology.c
include/hw/s390x/cpu-topology.h +6 -0
index 4e16a2153d..6fec10e032 100644
@@ -16,6 +16,11 @@
 #define S390_TOPOLOGY_CPU_IFL 0x03
 #define S390_TOPOLOGY_MAX_ORIGIN ((63 + S390_MAX_CPUS) / 64)
 
+#define S390_TOPOLOGY_POLARITY_HORIZONTAL      0x00
+#define S390_TOPOLOGY_POLARITY_VERTICAL_LOW    0x01
+#define S390_TOPOLOGY_POLARITY_VERTICAL_MEDIUM 0x02
+#define S390_TOPOLOGY_POLARITY_VERTICAL_HIGH   0x03
+
 typedef struct S390TopoSocket {
     int active_count;
     uint64_t mask[S390_TOPOLOGY_MAX_ORIGIN];
@@ -26,6 +31,7 @@ struct S390Topology {
     uint32_t nr_cpus;
     uint32_t nr_sockets;
     S390TopoSocket *socket;
+    bool topology_needed;
 };
 
 #define TYPE_S390_CPU_TOPOLOGY "s390-topology"
target/s390x/cpu.h +77 -0

87 lines not on this page · raw message

hw/s390x/cpu-topology.c +0 -1
index 6af41d3d7b..9fa8ca1261 100644
@@ -44,7 +44,6 @@ void s390_topology_new_cpu(S390CPU *cpu)
     int socket_id;
 
     socket_id = core_id / topo->nr_cpus;
-
     /*
      * At the core level, each CPU is represented by a bit in a 64bit
      * uint64_t which represent the presence of a CPU.
target/s390x/cpu_topology.c new file +100 -0

100 lines not on this page · raw message

target/s390x/kvm/kvm.c +5 -1
index 3ac7ec9acf..7dc96f3663 100644
@@ -51,6 +51,7 @@
 #include "hw/s390x/s390-virtio-ccw.h"
 #include "hw/s390x/s390-virtio-hcall.h"
 #include "hw/s390x/pv.h"
+#include "hw/s390x/cpu-topology.h"
 
 #ifndef DEBUG_KVM
 #define DEBUG_KVM  0
@@ -1919,9 +1920,12 @@ static int handle_stsi(S390CPU *cpu)
         if (run->s390_stsi.sel1 != 2 || run->s390_stsi.sel2 != 2) {
             return 0;
         }
-        /* Only sysib 3.2.2 needs post-handling for now. */
         insert_stsi_3_2_2(cpu, run->s390_stsi.addr, run->s390_stsi.ar);
         return 0;
+    case 15:
+        insert_stsi_15_1_x(cpu, run->s390_stsi.sel2, run->s390_stsi.addr,
+                           run->s390_stsi.ar);
+        return 0;
     default:
         return 0;
     }
target/s390x/meson.build +1 -0
index 84c1402a6a..890ccfa789 100644
@@ -29,6 +29,7 @@ s390x_softmmu_ss.add(files(
   'sigp.c',
   'cpu-sysemu.c',
   'cpu_models_sysemu.c',
+  'cpu_topology.c',
 ))
 
 s390x_user_ss = ss.source_set()
Signature · 1 line
2.31.1
#15 lore raw

Re: [PATCH v11 04/11] s390x/cpu topology: reporting the CPU topology to the guest

Hello Pierre,

Quoted 34 lines · Pierre Morel

The guest can use the STSI instruction to get a buffer filled with the CPU topology description.

Let us implement the STSI instruction for the basis CPU topology level, level 2.

---
  include/hw/s390x/cpu-topology.h |   6 ++
  target/s390x/cpu.h              |  77 ++++++++++++++++++++++++
  hw/s390x/cpu-topology.c         |   1 -
  target/s390x/cpu_topology.c     | 100 ++++++++++++++++++++++++++++++++
  target/s390x/kvm/kvm.c          |   6 +-
  target/s390x/meson.build        |   1 +
  6 files changed, 189 insertions(+), 2 deletions(-)

create mode 100644 target/s390x/cpu_topology.c

include/hw/s390x/cpu-topology.h +5 -0
index 4e16a2153d..6fec10e032 100644
@@ -16,6 +16,11 @@
  #define S390_TOPOLOGY_CPU_IFL 0x03
  #define S390_TOPOLOGY_MAX_ORIGIN ((63 + S390_MAX_CPUS) / 64)
 
+#define S390_TOPOLOGY_POLARITY_HORIZONTAL      0x00
+#define S390_TOPOLOGY_POLARITY_VERTICAL_LOW    0x01
+#define S390_TOPOLOGY_POLARITY_VERTICAL_MEDIUM 0x02
+#define S390_TOPOLOGY_POLARITY_VERTICAL_HIGH   0x03
+
  typedef struct S390TopoSocket {
      int active_count;
      uint64_t mask[S390_TOPOLOGY_MAX_ORIGIN];

include/hw/s390x/cpu-topology.h struct S390Topology { · line 34

Quoted 4 lines
include/hw/s390x/cpu-topology.h +1 -0
@@ -26,6 +31,7 @@ struct S390Topology {
      uint32_t nr_cpus;
      uint32_t nr_sockets;
      S390TopoSocket *socket;
+    bool topology_needed;

This is unused in this patch. Introduced too soon ?

Quoted 101 lines
include/hw/s390x/cpu-topology.h +0 -0
  };
 
  #define TYPE_S390_CPU_TOPOLOGY "s390-topology"
target/s390x/cpu.h +77 -0
index c9066b2496..69a7523146 100644
@@ -567,6 +567,81 @@ typedef union SysIB {
  } SysIB;
  QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096);
 
+/*
+ * CPU Topology List provided by STSI with fc=15 provides a list
+ * of two different Topology List Entries (TLE) types to specify
+ * the topology hierarchy.
+ *
+ * - Container Topology List Entry
+ *   Defines a container to contain other Topology List Entries
+ *   of any type, nested containers or CPU.
+ * - CPU Topology List Entry
+ *   Specifies the CPUs position, type, entitlement and polarization
+ *   of the CPUs contained in the last Container TLE.
+ *
+ * There can be theoretically up to five levels of containers, QEMU
+ * uses only one level, the socket level.
+ *
+ * A container of with a nesting level (NL) greater than 1 can only
+ * contain another container of nesting level NL-1.
+ *
+ * A container of nesting level 1 (socket), contains as many CPU TLE
+ * as needed to describe the position and qualities of all CPUs inside
+ * the container.
+ * The qualities of a CPU are polarization, entitlement and type.
+ *
+ * The CPU TLE defines the position of the CPUs of identical qualities
+ * using a 64bits mask which first bit has its offset defined by
+ * the CPU address orgin field of the CPU TLE like in:
+ * CPU address = origin * 64 + bit position within the mask
+ *
+ */
+/* Container type Topology List Entry */
+typedef struct SysIBTl_container {
+        uint8_t nl;
+        uint8_t reserved[6];
+        uint8_t id;
+} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_container;
+QEMU_BUILD_BUG_ON(sizeof(SysIBTl_container) != 8);
+
+/* CPU type Topology List Entry */
+typedef struct SysIBTl_cpu {
+        uint8_t nl;
+        uint8_t reserved0[3];
+        uint8_t reserved1:5;
+        uint8_t dedicated:1;
+        uint8_t polarity:2;
+        uint8_t type;
+        uint16_t origin;
+        uint64_t mask;
+} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_cpu;
+QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
+
+#define S390_TOPOLOGY_MAG  6
+#define S390_TOPOLOGY_MAG6 0
+#define S390_TOPOLOGY_MAG5 1
+#define S390_TOPOLOGY_MAG4 2
+#define S390_TOPOLOGY_MAG3 3
+#define S390_TOPOLOGY_MAG2 4
+#define S390_TOPOLOGY_MAG1 5
+/* Configuration topology */
+typedef struct SysIB_151x {
+    uint8_t  reserved0[2];
+    uint16_t length;
+    uint8_t  mag[S390_TOPOLOGY_MAG];
+    uint8_t  reserved1;
+    uint8_t  mnest;
+    uint32_t reserved2;
+    char tle[0];
+} QEMU_PACKED QEMU_ALIGNED(8) SysIB_151x;
+QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16);
+
+/* Max size of a SYSIB structure is when all CPU are alone in a container */
+#define S390_TOPOLOGY_SYSIB_SIZE (sizeof(SysIB_151x) +                         \
+                                  S390_MAX_CPUS * (sizeof(SysIBTl_container) + \
+                                                   sizeof(SysIBTl_cpu)))
+
+
  /* MMU defines */
  #define ASCE_ORIGIN           (~0xfffULL) /* segment table origin             */
  #define ASCE_SUBSPACE         0x200       /* subspace group control           */
@@ -845,4 +920,6 @@ S390CPU *s390_cpu_addr2state(uint16_t cpu_addr);
 
  #include "exec/cpu-all.h"
 
+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar);
+
  #endif
hw/s390x/cpu-topology.c +0 -0
index 6af41d3d7b..9fa8ca1261 100644

hw/s390x/cpu-topology.c void s390_topology_new_cpu(S390CPU *cpu) · line 47

Quoted 4 lines
hw/s390x/cpu-topology.c +0 -1
@@ -44,7 +44,6 @@ void s390_topology_new_cpu(S390CPU *cpu)
      int socket_id;
 
      socket_id = core_id / topo->nr_cpus;
-

Unnecessary change.

Thanks,

C.

Quoted 147 lines
hw/s390x/cpu-topology.c +0 -0
      /*
       * At the core level, each CPU is represented by a bit in a 64bit
       * uint64_t which represent the presence of a CPU.
target/s390x/cpu_topology.c new file +100 -0
new file mode 100644
index 0000000000..a1179d8e95
@@ -0,0 +1,100 @@
+/*
+ * QEMU S390x CPU Topology
+ *
+ * Copyright IBM Corp. 2022
+ * Author(s): Pierre Morel <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "hw/s390x/pv.h"
+#include "hw/sysbus.h"
+#include "hw/s390x/cpu-topology.h"
+#include "hw/s390x/sclp.h"
+
+static char *fill_container(char *p, int level, int id)
+{
+    SysIBTl_container *tle = (SysIBTl_container *)p;
+
+    tle->nl = level;
+    tle->id = id;
+    return p + sizeof(*tle);
+}
+
+static char *fill_tle_cpu(char *p, uint64_t mask, int origin)
+{
+    SysIBTl_cpu *tle = (SysIBTl_cpu *)p;
+
+    tle->nl = 0;
+    tle->dedicated = 1;
+    tle->polarity = S390_TOPOLOGY_POLARITY_HORIZONTAL;
+    tle->type = S390_TOPOLOGY_CPU_IFL;
+    tle->origin = cpu_to_be64(origin * 64);
+    tle->mask = cpu_to_be64(mask);
+    return p + sizeof(*tle);
+}
+
+static char *s390_top_set_level2(S390Topology *topo, char *p)
+{
+    int i, origin;
+
+    for (i = 0; i < topo->nr_sockets; i++) {
+        if (!topo->socket[i].active_count) {
+            continue;
+        }
+        p = fill_container(p, 1, i);
+        for (origin = 0; origin < S390_TOPOLOGY_MAX_ORIGIN; origin++) {
+            uint64_t mask = 0L;
+
+            mask = topo->socket[i].mask[origin];
+            if (mask) {
+                p = fill_tle_cpu(p, mask, origin);
+            }
+        }
+    }
+    return p;
+}
+
+static int setup_stsi(S390CPU *cpu, SysIB_151x *sysib, int level)
+{
+    S390Topology *topo = (S390Topology *)cpu->topology;
+    char *p = sysib->tle;
+
+    sysib->mnest = level;
+    switch (level) {
+    case 2:
+        sysib->mag[S390_TOPOLOGY_MAG2] = topo->nr_sockets;
+        sysib->mag[S390_TOPOLOGY_MAG1] = topo->nr_cpus;
+        p = s390_top_set_level2(topo, p);
+        break;
+    }
+
+    return p - (char *)sysib;
+}
+
+#define S390_TOPOLOGY_MAX_MNEST 2
+
+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar)
+{
+    union {
+        char place_holder[S390_TOPOLOGY_SYSIB_SIZE];
+        SysIB_151x sysib;
+    } buffer QEMU_ALIGNED(8);
+    int len;
+
+    if (s390_is_pv() || !s390_has_topology() ||
+        sel2 < 2 || sel2 > S390_TOPOLOGY_MAX_MNEST) {
+        setcc(cpu, 3);
+        return;
+    }
+
+    len = setup_stsi(cpu, &buffer.sysib, sel2);
+
+    buffer.sysib.length = cpu_to_be16(len);
+    s390_cpu_virt_mem_write(cpu, addr, ar, &buffer.sysib, len);
+    setcc(cpu, 0);
+}
+
target/s390x/kvm/kvm.c +5 -1
index 3ac7ec9acf..7dc96f3663 100644
@@ -51,6 +51,7 @@
  #include "hw/s390x/s390-virtio-ccw.h"
  #include "hw/s390x/s390-virtio-hcall.h"
  #include "hw/s390x/pv.h"
+#include "hw/s390x/cpu-topology.h"
 
  #ifndef DEBUG_KVM
  #define DEBUG_KVM  0
@@ -1919,9 +1920,12 @@ static int handle_stsi(S390CPU *cpu)
          if (run->s390_stsi.sel1 != 2 || run->s390_stsi.sel2 != 2) {
              return 0;
          }
-        /* Only sysib 3.2.2 needs post-handling for now. */
          insert_stsi_3_2_2(cpu, run->s390_stsi.addr, run->s390_stsi.ar);
          return 0;
+    case 15:
+        insert_stsi_15_1_x(cpu, run->s390_stsi.sel2, run->s390_stsi.addr,
+                           run->s390_stsi.ar);
+        return 0;
      default:
          return 0;
      }
target/s390x/meson.build +1 -0
index 84c1402a6a..890ccfa789 100644
@@ -29,6 +29,7 @@ s390x_softmmu_ss.add(files(
    'sigp.c',
    'cpu-sysemu.c',
    'cpu_models_sysemu.c',
+  'cpu_topology.c',
  ))
 
  s390x_user_ss = ss.source_set()
#16 lore raw

Re: [PATCH v11 04/11] s390x/cpu topology: reporting the CPU topology to the guest

Quoted 35 lines · Cédric Le Goater

Hello Pierre,

Quoted 32 lines · Pierre Morel

The guest can use the STSI instruction to get a buffer filled with the CPU topology description.

Let us implement the STSI instruction for the basis CPU topology level, level 2.

---
  include/hw/s390x/cpu-topology.h |   6 ++ target/s390x/cpu.h              |  77 ++++++++++++++++++++++++ hw/s390x/cpu-topology.c         |   1 - target/s390x/cpu_topology.c     | 100 ++++++++++++++++++++++++++++++++ target/s390x/kvm/kvm.c          |   6 +- target/s390x/meson.build        |   1 +

  6 files changed, 189 insertions(+), 2 deletions(-)

  create mode 100644 target/s390x/cpu_topology.c

include/hw/s390x/cpu-topology.h +5 -0
index 4e16a2153d..6fec10e032 100644
@@ -16,6 +16,11 @@
   #define S390_TOPOLOGY_CPU_IFL 0x03
   #define S390_TOPOLOGY_MAX_ORIGIN ((63 + S390_MAX_CPUS) / 64)
+#define S390_TOPOLOGY_POLARITY_HORIZONTAL      0x00
+#define S390_TOPOLOGY_POLARITY_VERTICAL_LOW    0x01
+#define S390_TOPOLOGY_POLARITY_VERTICAL_MEDIUM 0x02
+#define S390_TOPOLOGY_POLARITY_VERTICAL_HIGH   0x03
+
   typedef struct S390TopoSocket {
       int active_count;
       uint64_t mask[S390_TOPOLOGY_MAX_ORIGIN];
Quoted 8 lines

include/hw/s390x/cpu-topology.h struct S390Topology { · line 34

Quoted 5 lines
include/hw/s390x/cpu-topology.h +1 -0
@@ -26,6 +31,7 @@ struct S390Topology {
       uint32_t nr_cpus;
       uint32_t nr_sockets;
       S390TopoSocket *socket;
+    bool topology_needed;

This is unused in this patch. Introduced too soon ?

right, thanks.

Quoted 99 lines
Quoted 99 lines
include/hw/s390x/cpu-topology.h +0 -0
   };
   #define TYPE_S390_CPU_TOPOLOGY "s390-topology"
target/s390x/cpu.h +70 -0
index c9066b2496..69a7523146 100644
@@ -567,6 +567,81 @@ typedef union SysIB {
   } SysIB;
   QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096);
+/*
+ * CPU Topology List provided by STSI with fc=15 provides a list
+ * of two different Topology List Entries (TLE) types to specify
+ * the topology hierarchy.
+ *
+ * - Container Topology List Entry
+ *   Defines a container to contain other Topology List Entries
+ *   of any type, nested containers or CPU.
+ * - CPU Topology List Entry
+ *   Specifies the CPUs position, type, entitlement and polarization
+ *   of the CPUs contained in the last Container TLE.
+ *
+ * There can be theoretically up to five levels of containers, QEMU
+ * uses only one level, the socket level.
+ *
+ * A container of with a nesting level (NL) greater than 1 can only
+ * contain another container of nesting level NL-1.
+ *
+ * A container of nesting level 1 (socket), contains as many CPU TLE
+ * as needed to describe the position and qualities of all CPUs inside
+ * the container.
+ * The qualities of a CPU are polarization, entitlement and type.
+ *
+ * The CPU TLE defines the position of the CPUs of identical qualities
+ * using a 64bits mask which first bit has its offset defined by
+ * the CPU address orgin field of the CPU TLE like in:
+ * CPU address = origin * 64 + bit position within the mask
+ *
+ */
+/* Container type Topology List Entry */
+typedef struct SysIBTl_container {
+        uint8_t nl;
+        uint8_t reserved[6];
+        uint8_t id;
+} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_container;
+QEMU_BUILD_BUG_ON(sizeof(SysIBTl_container) != 8);
+
+/* CPU type Topology List Entry */
+typedef struct SysIBTl_cpu {
+        uint8_t nl;
+        uint8_t reserved0[3];
+        uint8_t reserved1:5;
+        uint8_t dedicated:1;
+        uint8_t polarity:2;
+        uint8_t type;
+        uint16_t origin;
+        uint64_t mask;
+} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_cpu;
+QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
+
+#define S390_TOPOLOGY_MAG  6
+#define S390_TOPOLOGY_MAG6 0
+#define S390_TOPOLOGY_MAG5 1
+#define S390_TOPOLOGY_MAG4 2
+#define S390_TOPOLOGY_MAG3 3
+#define S390_TOPOLOGY_MAG2 4
+#define S390_TOPOLOGY_MAG1 5
+/* Configuration topology */
+typedef struct SysIB_151x {
+    uint8_t  reserved0[2];
+    uint16_t length;
+    uint8_t  mag[S390_TOPOLOGY_MAG];
+    uint8_t  reserved1;
+    uint8_t  mnest;
+    uint32_t reserved2;
+    char tle[0];
+} QEMU_PACKED QEMU_ALIGNED(8) SysIB_151x;
+QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16);
+
+/* Max size of a SYSIB structure is when all CPU are alone in a 

container */
+#define S390_TOPOLOGY_SYSIB_SIZE (sizeof(SysIB_151x)
+                         \
+                                  S390_MAX_CPUS * (sizeof(SysIBTl_container) + \
+                                                   sizeof(SysIBTl_cpu)))

target/s390x/cpu.h +2 -0
+
+

  /* MMU defines */
  #define ASCE_ORIGIN           (~0xfffULL) /* segment table origin             */ #define ASCE_SUBSPACE         0x200       /* subspace group control           */

target/s390x/cpu.h +2 -0
@@ -845,4 +920,6 @@ S390CPU *s390_cpu_addr2state(uint16_t cpu_addr);
   #include "exec/cpu-all.h"
+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar);
+
   #endif
hw/s390x/cpu-topology.c +0 -0
index 6af41d3d7b..9fa8ca1261 100644
Quoted 8 lines

hw/s390x/cpu-topology.c void s390_topology_new_cpu(S390CPU *cpu) · line 46

Quoted 6 lines
hw/s390x/cpu-topology.c +0 -1
@@ -44,7 +44,6 @@ void s390_topology_new_cpu(S390CPU *cpu)
       int socket_id;
       socket_id = core_id / topo->nr_cpus;
-

Unnecessary change.

Yes,

thanks,

regards,
Pierre

Signature · 2 lines
Pierre Morel
IBM Lab Boeblingen
#17 lore raw

Re: [PATCH v11 04/11] s390x/cpu topology: reporting the CPU topology to the guest

In reply to Pierre Morel
Quoted 201 lines · Pierre Morel

The guest can use the STSI instruction to get a buffer filled with the CPU topology description.

Let us implement the STSI instruction for the basis CPU topology level, level 2.

---
  include/hw/s390x/cpu-topology.h |   6 ++
  target/s390x/cpu.h              |  77 ++++++++++++++++++++++++
  hw/s390x/cpu-topology.c         |   1 -
  target/s390x/cpu_topology.c     | 100 ++++++++++++++++++++++++++++++++
  target/s390x/kvm/kvm.c          |   6 +-
  target/s390x/meson.build        |   1 +
  6 files changed, 189 insertions(+), 2 deletions(-)

create mode 100644 target/s390x/cpu_topology.c

include/hw/s390x/cpu-topology.h +6 -0
index 4e16a2153d..6fec10e032 100644
@@ -16,6 +16,11 @@
  #define S390_TOPOLOGY_CPU_IFL 0x03
  #define S390_TOPOLOGY_MAX_ORIGIN ((63 + S390_MAX_CPUS) / 64)
 
+#define S390_TOPOLOGY_POLARITY_HORIZONTAL      0x00
+#define S390_TOPOLOGY_POLARITY_VERTICAL_LOW    0x01
+#define S390_TOPOLOGY_POLARITY_VERTICAL_MEDIUM 0x02
+#define S390_TOPOLOGY_POLARITY_VERTICAL_HIGH   0x03
+
  typedef struct S390TopoSocket {
      int active_count;
      uint64_t mask[S390_TOPOLOGY_MAX_ORIGIN];
@@ -26,6 +31,7 @@ struct S390Topology {
      uint32_t nr_cpus;
      uint32_t nr_sockets;
      S390TopoSocket *socket;
+    bool topology_needed;
  };
 
  #define TYPE_S390_CPU_TOPOLOGY "s390-topology"
target/s390x/cpu.h +77 -0
index c9066b2496..69a7523146 100644
@@ -567,6 +567,81 @@ typedef union SysIB {
  } SysIB;
  QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096);
 
+/*
+ * CPU Topology List provided by STSI with fc=15 provides a list
+ * of two different Topology List Entries (TLE) types to specify
+ * the topology hierarchy.
+ *
+ * - Container Topology List Entry
+ *   Defines a container to contain other Topology List Entries
+ *   of any type, nested containers or CPU.
+ * - CPU Topology List Entry
+ *   Specifies the CPUs position, type, entitlement and polarization
+ *   of the CPUs contained in the last Container TLE.
+ *
+ * There can be theoretically up to five levels of containers, QEMU
+ * uses only one level, the socket level.
+ *
+ * A container of with a nesting level (NL) greater than 1 can only
+ * contain another container of nesting level NL-1.
+ *
+ * A container of nesting level 1 (socket), contains as many CPU TLE
+ * as needed to describe the position and qualities of all CPUs inside
+ * the container.
+ * The qualities of a CPU are polarization, entitlement and type.
+ *
+ * The CPU TLE defines the position of the CPUs of identical qualities
+ * using a 64bits mask which first bit has its offset defined by
+ * the CPU address orgin field of the CPU TLE like in:
+ * CPU address = origin * 64 + bit position within the mask
+ *
+ */
+/* Container type Topology List Entry */
+typedef struct SysIBTl_container {
+        uint8_t nl;
+        uint8_t reserved[6];
+        uint8_t id;
+} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_container;
+QEMU_BUILD_BUG_ON(sizeof(SysIBTl_container) != 8);
+
+/* CPU type Topology List Entry */
+typedef struct SysIBTl_cpu {
+        uint8_t nl;
+        uint8_t reserved0[3];
+        uint8_t reserved1:5;
+        uint8_t dedicated:1;
+        uint8_t polarity:2;
+        uint8_t type;
+        uint16_t origin;
+        uint64_t mask;
+} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_cpu;
+QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
+
+#define S390_TOPOLOGY_MAG  6
+#define S390_TOPOLOGY_MAG6 0
+#define S390_TOPOLOGY_MAG5 1
+#define S390_TOPOLOGY_MAG4 2
+#define S390_TOPOLOGY_MAG3 3
+#define S390_TOPOLOGY_MAG2 4
+#define S390_TOPOLOGY_MAG1 5
+/* Configuration topology */
+typedef struct SysIB_151x {
+    uint8_t  reserved0[2];
+    uint16_t length;
+    uint8_t  mag[S390_TOPOLOGY_MAG];
+    uint8_t  reserved1;
+    uint8_t  mnest;
+    uint32_t reserved2;
+    char tle[0];
+} QEMU_PACKED QEMU_ALIGNED(8) SysIB_151x;
+QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16);
+
+/* Max size of a SYSIB structure is when all CPU are alone in a container */
+#define S390_TOPOLOGY_SYSIB_SIZE (sizeof(SysIB_151x) +                         \
+                                  S390_MAX_CPUS * (sizeof(SysIBTl_container) + \
+                                                   sizeof(SysIBTl_cpu)))
+
+
  /* MMU defines */
  #define ASCE_ORIGIN           (~0xfffULL) /* segment table origin             */
  #define ASCE_SUBSPACE         0x200       /* subspace group control           */
@@ -845,4 +920,6 @@ S390CPU *s390_cpu_addr2state(uint16_t cpu_addr);
 
  #include "exec/cpu-all.h"
 
+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar);
+
  #endif
hw/s390x/cpu-topology.c +0 -1
index 6af41d3d7b..9fa8ca1261 100644
@@ -44,7 +44,6 @@ void s390_topology_new_cpu(S390CPU *cpu)
      int socket_id;
 
      socket_id = core_id / topo->nr_cpus;
-
      /*
       * At the core level, each CPU is represented by a bit in a 64bit
       * uint64_t which represent the presence of a CPU.
target/s390x/cpu_topology.c new file +49 -0
new file mode 100644
index 0000000000..a1179d8e95
@@ -0,0 +1,100 @@
+/*
+ * QEMU S390x CPU Topology
+ *
+ * Copyright IBM Corp. 2022
+ * Author(s): Pierre Morel <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "hw/s390x/pv.h"
+#include "hw/sysbus.h"
+#include "hw/s390x/cpu-topology.h"
+#include "hw/s390x/sclp.h"
+
+static char *fill_container(char *p, int level, int id)
+{
+    SysIBTl_container *tle = (SysIBTl_container *)p;
+
+    tle->nl = level;
+    tle->id = id;
+    return p + sizeof(*tle);
+}
+
+static char *fill_tle_cpu(char *p, uint64_t mask, int origin)
+{
+    SysIBTl_cpu *tle = (SysIBTl_cpu *)p;
+
+    tle->nl = 0;
+    tle->dedicated = 1;
+    tle->polarity = S390_TOPOLOGY_POLARITY_HORIZONTAL;
+    tle->type = S390_TOPOLOGY_CPU_IFL;
+    tle->origin = cpu_to_be64(origin * 64);
+    tle->mask = cpu_to_be64(mask);
+    return p + sizeof(*tle);
+}
+
+static char *s390_top_set_level2(S390Topology *topo, char *p)
+{
+    int i, origin;
+
+    for (i = 0; i < topo->nr_sockets; i++) {
+        if (!topo->socket[i].active_count) {
+            continue;
+        }
+        p = fill_container(p, 1, i);
+        for (origin = 0; origin < S390_TOPOLOGY_MAX_ORIGIN; origin++) {

target/s390x/cpu_topology.c line 59

Quoted 10 lines
target/s390x/cpu_topology.c new file +10 -0
+            uint64_t mask = 0L;
+
+            mask = topo->socket[i].mask[origin];
+            if (mask) {
+                p = fill_tle_cpu(p, mask, origin);
+            }
+        }
+    }
+    return p;
+}

Why is it not possible to compute this topo information at "runtime", when stsi is called, without maintaining state in an extra S390Topology object ? Couldn't we loop on the CPU list to gather the topology bits for the same result ?

It would greatly simplify the feature.

C.

Quoted 78 lines
target/s390x/cpu_topology.c new file +40 -0
+static int setup_stsi(S390CPU *cpu, SysIB_151x *sysib, int level)
+{
+    S390Topology *topo = (S390Topology *)cpu->topology;
+    char *p = sysib->tle;
+
+    sysib->mnest = level;
+    switch (level) {
+    case 2:
+        sysib->mag[S390_TOPOLOGY_MAG2] = topo->nr_sockets;
+        sysib->mag[S390_TOPOLOGY_MAG1] = topo->nr_cpus;
+        p = s390_top_set_level2(topo, p);
+        break;
+    }
+
+    return p - (char *)sysib;
+}
+
+#define S390_TOPOLOGY_MAX_MNEST 2
+
+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar)
+{
+    union {
+        char place_holder[S390_TOPOLOGY_SYSIB_SIZE];
+        SysIB_151x sysib;
+    } buffer QEMU_ALIGNED(8);
+    int len;
+
+    if (s390_is_pv() || !s390_has_topology() ||
+        sel2 < 2 || sel2 > S390_TOPOLOGY_MAX_MNEST) {
+        setcc(cpu, 3);
+        return;
+    }
+
+    len = setup_stsi(cpu, &buffer.sysib, sel2);
+
+    buffer.sysib.length = cpu_to_be16(len);
+    s390_cpu_virt_mem_write(cpu, addr, ar, &buffer.sysib, len);
+    setcc(cpu, 0);
+}
+
target/s390x/kvm/kvm.c +5 -1
index 3ac7ec9acf..7dc96f3663 100644
@@ -51,6 +51,7 @@
  #include "hw/s390x/s390-virtio-ccw.h"
  #include "hw/s390x/s390-virtio-hcall.h"
  #include "hw/s390x/pv.h"
+#include "hw/s390x/cpu-topology.h"
 
  #ifndef DEBUG_KVM
  #define DEBUG_KVM  0
@@ -1919,9 +1920,12 @@ static int handle_stsi(S390CPU *cpu)
          if (run->s390_stsi.sel1 != 2 || run->s390_stsi.sel2 != 2) {
              return 0;
          }
-        /* Only sysib 3.2.2 needs post-handling for now. */
          insert_stsi_3_2_2(cpu, run->s390_stsi.addr, run->s390_stsi.ar);
          return 0;
+    case 15:
+        insert_stsi_15_1_x(cpu, run->s390_stsi.sel2, run->s390_stsi.addr,
+                           run->s390_stsi.ar);
+        return 0;
      default:
          return 0;
      }
target/s390x/meson.build +1 -0
index 84c1402a6a..890ccfa789 100644
@@ -29,6 +29,7 @@ s390x_softmmu_ss.add(files(
    'sigp.c',
    'cpu-sysemu.c',
    'cpu_models_sysemu.c',
+  'cpu_topology.c',
  ))
 
  s390x_user_ss = ss.source_set()
#18 lore raw

Re: [PATCH v11 04/11] s390x/cpu topology: reporting the CPU topology to the guest

Quoted 216 lines · Cédric Le Goater
Quoted 199 lines · Pierre Morel

The guest can use the STSI instruction to get a buffer filled with the CPU topology description.

Let us implement the STSI instruction for the basis CPU topology level, level 2.

---
  include/hw/s390x/cpu-topology.h |   6 ++ target/s390x/cpu.h              |  77 ++++++++++++++++++++++++ hw/s390x/cpu-topology.c         |   1 - target/s390x/cpu_topology.c     | 100 ++++++++++++++++++++++++++++++++ target/s390x/kvm/kvm.c          |   6 +- target/s390x/meson.build        |   1 +

  6 files changed, 189 insertions(+), 2 deletions(-)

  create mode 100644 target/s390x/cpu_topology.c

include/hw/s390x/cpu-topology.h +6 -0
index 4e16a2153d..6fec10e032 100644
@@ -16,6 +16,11 @@
   #define S390_TOPOLOGY_CPU_IFL 0x03
   #define S390_TOPOLOGY_MAX_ORIGIN ((63 + S390_MAX_CPUS) / 64)
+#define S390_TOPOLOGY_POLARITY_HORIZONTAL      0x00
+#define S390_TOPOLOGY_POLARITY_VERTICAL_LOW    0x01
+#define S390_TOPOLOGY_POLARITY_VERTICAL_MEDIUM 0x02
+#define S390_TOPOLOGY_POLARITY_VERTICAL_HIGH   0x03
+
   typedef struct S390TopoSocket {
       int active_count;
       uint64_t mask[S390_TOPOLOGY_MAX_ORIGIN];
@@ -26,6 +31,7 @@ struct S390Topology {
       uint32_t nr_cpus;
       uint32_t nr_sockets;
       S390TopoSocket *socket;
+    bool topology_needed;
   };
   #define TYPE_S390_CPU_TOPOLOGY "s390-topology"
target/s390x/cpu.h +70 -0
index c9066b2496..69a7523146 100644
@@ -567,6 +567,81 @@ typedef union SysIB {
   } SysIB;
   QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096);
+/*
+ * CPU Topology List provided by STSI with fc=15 provides a list
+ * of two different Topology List Entries (TLE) types to specify
+ * the topology hierarchy.
+ *
+ * - Container Topology List Entry
+ *   Defines a container to contain other Topology List Entries
+ *   of any type, nested containers or CPU.
+ * - CPU Topology List Entry
+ *   Specifies the CPUs position, type, entitlement and polarization
+ *   of the CPUs contained in the last Container TLE.
+ *
+ * There can be theoretically up to five levels of containers, QEMU
+ * uses only one level, the socket level.
+ *
+ * A container of with a nesting level (NL) greater than 1 can only
+ * contain another container of nesting level NL-1.
+ *
+ * A container of nesting level 1 (socket), contains as many CPU TLE
+ * as needed to describe the position and qualities of all CPUs inside
+ * the container.
+ * The qualities of a CPU are polarization, entitlement and type.
+ *
+ * The CPU TLE defines the position of the CPUs of identical qualities
+ * using a 64bits mask which first bit has its offset defined by
+ * the CPU address orgin field of the CPU TLE like in:
+ * CPU address = origin * 64 + bit position within the mask
+ *
+ */
+/* Container type Topology List Entry */
+typedef struct SysIBTl_container {
+        uint8_t nl;
+        uint8_t reserved[6];
+        uint8_t id;
+} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_container;
+QEMU_BUILD_BUG_ON(sizeof(SysIBTl_container) != 8);
+
+/* CPU type Topology List Entry */
+typedef struct SysIBTl_cpu {
+        uint8_t nl;
+        uint8_t reserved0[3];
+        uint8_t reserved1:5;
+        uint8_t dedicated:1;
+        uint8_t polarity:2;
+        uint8_t type;
+        uint16_t origin;
+        uint64_t mask;
+} QEMU_PACKED QEMU_ALIGNED(8) SysIBTl_cpu;
+QEMU_BUILD_BUG_ON(sizeof(SysIBTl_cpu) != 16);
+
+#define S390_TOPOLOGY_MAG  6
+#define S390_TOPOLOGY_MAG6 0
+#define S390_TOPOLOGY_MAG5 1
+#define S390_TOPOLOGY_MAG4 2
+#define S390_TOPOLOGY_MAG3 3
+#define S390_TOPOLOGY_MAG2 4
+#define S390_TOPOLOGY_MAG1 5
+/* Configuration topology */
+typedef struct SysIB_151x {
+    uint8_t  reserved0[2];
+    uint16_t length;
+    uint8_t  mag[S390_TOPOLOGY_MAG];
+    uint8_t  reserved1;
+    uint8_t  mnest;
+    uint32_t reserved2;
+    char tle[0];
+} QEMU_PACKED QEMU_ALIGNED(8) SysIB_151x;
+QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16);
+
+/* Max size of a SYSIB structure is when all CPU are alone in a 

container */
+#define S390_TOPOLOGY_SYSIB_SIZE (sizeof(SysIB_151x)
+                         \
+                                  S390_MAX_CPUS * (sizeof(SysIBTl_container) + \
+                                                   sizeof(SysIBTl_cpu)))

target/s390x/cpu.h +2 -0
+
+

  /* MMU defines */
  #define ASCE_ORIGIN           (~0xfffULL) /* segment table origin             */ #define ASCE_SUBSPACE         0x200       /* subspace group control           */

target/s390x/cpu.h +2 -0
@@ -845,4 +920,6 @@ S390CPU *s390_cpu_addr2state(uint16_t cpu_addr);
   #include "exec/cpu-all.h"
+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar);
+
   #endif
hw/s390x/cpu-topology.c +0 -1
index 6af41d3d7b..9fa8ca1261 100644
@@ -44,7 +44,6 @@ void s390_topology_new_cpu(S390CPU *cpu)
       int socket_id;
       socket_id = core_id / topo->nr_cpus;
-
       /*
        * At the core level, each CPU is represented by a bit in a 64bit
        * uint64_t which represent the presence of a CPU.
target/s390x/cpu_topology.c new file +7 -0
new file mode 100644
index 0000000000..a1179d8e95
@@ -0,0 +1,100 @@
+/*
+ * QEMU S390x CPU Topology
+ *
+ * Copyright IBM Corp. 2022
+ * Author(s): Pierre Morel <[email protected]>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or 

(at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */

target/s390x/cpu_topology.c new file +39 -0
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "hw/s390x/pv.h"
+#include "hw/sysbus.h"
+#include "hw/s390x/cpu-topology.h"
+#include "hw/s390x/sclp.h"
+
+static char *fill_container(char *p, int level, int id)
+{
+    SysIBTl_container *tle = (SysIBTl_container *)p;
+
+    tle->nl = level;
+    tle->id = id;
+    return p + sizeof(*tle);
+}
+
+static char *fill_tle_cpu(char *p, uint64_t mask, int origin)
+{
+    SysIBTl_cpu *tle = (SysIBTl_cpu *)p;
+
+    tle->nl = 0;
+    tle->dedicated = 1;
+    tle->polarity = S390_TOPOLOGY_POLARITY_HORIZONTAL;
+    tle->type = S390_TOPOLOGY_CPU_IFL;
+    tle->origin = cpu_to_be64(origin * 64);
+    tle->mask = cpu_to_be64(mask);
+    return p + sizeof(*tle);
+}
+
+static char *s390_top_set_level2(S390Topology *topo, char *p)
+{
+    int i, origin;
+
+    for (i = 0; i < topo->nr_sockets; i++) {
+        if (!topo->socket[i].active_count) {
+            continue;
+        }
+        p = fill_container(p, 1, i);
+        for (origin = 0; origin < S390_TOPOLOGY_MAX_ORIGIN; origin++) {

target/s390x/cpu_topology.c line 59

Quoted 10 lines
target/s390x/cpu_topology.c new file +10 -0
+            uint64_t mask = 0L;
+
+            mask = topo->socket[i].mask[origin];
+            if (mask) {
+                p = fill_tle_cpu(p, mask, origin);
+            }
+        }
+    }
+    return p;
+}

Why is it not possible to compute this topo information at "runtime", when stsi is called, without maintaining state in an extra S390Topology object ? Couldn't we loop on the CPU list to gather the topology bits for the same result ?

Quoted 4 lines

It would greatly simplify the feature.

C.

The vCPU are not stored in order of creation in the CPU list and not in a topology order. To be able to build the SYSIB we need an intermediate structure to reorder the CPUs per container.

We can do this re-ordering during the STSI interception but the idea was to keep this instruction as fast as possible.

The second reason is to have a structure ready for the QEMU migration when we introduce vCPU migration from a socket to another socket, having then a different internal representation of the topology.

However, if as discussed yesterday we use a new cpu flag we would not need any special migration structure in the current series.

So it only stays the first reason to do the re-ordering preparation during the plugging of a vCPU, to optimize the STSI instruction.

If we think the optimization is not worth it or do not bring enough to be consider, we can do everything during the STSI interception.

Regards,
Pierre

Quoted 78 lines
Quoted 77 lines
target/s390x/cpu_topology.c new file +40 -0
+static int setup_stsi(S390CPU *cpu, SysIB_151x *sysib, int level)
+{
+    S390Topology *topo = (S390Topology *)cpu->topology;
+    char *p = sysib->tle;
+
+    sysib->mnest = level;
+    switch (level) {
+    case 2:
+        sysib->mag[S390_TOPOLOGY_MAG2] = topo->nr_sockets;
+        sysib->mag[S390_TOPOLOGY_MAG1] = topo->nr_cpus;
+        p = s390_top_set_level2(topo, p);
+        break;
+    }
+
+    return p - (char *)sysib;
+}
+
+#define S390_TOPOLOGY_MAX_MNEST 2
+
+void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar)
+{
+    union {
+        char place_holder[S390_TOPOLOGY_SYSIB_SIZE];
+        SysIB_151x sysib;
+    } buffer QEMU_ALIGNED(8);
+    int len;
+
+    if (s390_is_pv() || !s390_has_topology() ||
+        sel2 < 2 || sel2 > S390_TOPOLOGY_MAX_MNEST) {
+        setcc(cpu, 3);
+        return;
+    }
+
+    len = setup_stsi(cpu, &buffer.sysib, sel2);
+
+    buffer.sysib.length = cpu_to_be16(len);
+    s390_cpu_virt_mem_write(cpu, addr, ar, &buffer.sysib, len);
+    setcc(cpu, 0);
+}
+
target/s390x/kvm/kvm.c +5 -1
index 3ac7ec9acf..7dc96f3663 100644
@@ -51,6 +51,7 @@
   #include "hw/s390x/s390-virtio-ccw.h"
   #include "hw/s390x/s390-virtio-hcall.h"
   #include "hw/s390x/pv.h"
+#include "hw/s390x/cpu-topology.h"
   #ifndef DEBUG_KVM
   #define DEBUG_KVM  0
@@ -1919,9 +1920,12 @@ static int handle_stsi(S390CPU *cpu)
           if (run->s390_stsi.sel1 != 2 || run->s390_stsi.sel2 != 2) {
               return 0;
           }
-        /* Only sysib 3.2.2 needs post-handling for now. */
           insert_stsi_3_2_2(cpu, run->s390_stsi.addr, run->s390_stsi.ar);
           return 0;
+    case 15:
+        insert_stsi_15_1_x(cpu, run->s390_stsi.sel2, 
 run->s390_stsi.addr,
+                           run->s390_stsi.ar);
+        return 0;

      default:
          return 0;
      }

target/s390x/meson.build +1 -0
index 84c1402a6a..890ccfa789 100644
@@ -29,6 +29,7 @@ s390x_softmmu_ss.add(files(
     'sigp.c',
     'cpu-sysemu.c',
     'cpu_models_sysemu.c',
+  'cpu_topology.c',
   ))
   s390x_user_ss = ss.source_set()
Signature · 2 lines
Pierre Morel
IBM Lab Boeblingen
#19 lore raw

Re: [PATCH v11 04/11] s390x/cpu topology: reporting the CPU topology to the guest

Quoted 34 lines
Quoted 26 lines
Quoted 10 lines
target/s390x/cpu_topology.c new file +10 -0
+static char *s390_top_set_level2(S390Topology *topo, char *p)
+{
+    int i, origin;
+
+    for (i = 0; i < topo->nr_sockets; i++) {
+        if (!topo->socket[i].active_count) {
+            continue;
+        }
+        p = fill_container(p, 1, i);
+        for (origin = 0; origin < S390_TOPOLOGY_MAX_ORIGIN; origin++) {

target/s390x/cpu_topology.c line 59

Quoted 10 lines
target/s390x/cpu_topology.c new file +10 -0
+            uint64_t mask = 0L;
+
+            mask = topo->socket[i].mask[origin];
+            if (mask) {
+                p = fill_tle_cpu(p, mask, origin);
+            }
+        }
+    }
+    return p;
+}

Why is it not possible to compute this topo information at "runtime", when stsi is called, without maintaining state in an extra S390Topology object ? Couldn't we loop on the CPU list to gather the topology bits for the same result ?

Quoted 4 lines

It would greatly simplify the feature.

C.

The vCPU are not stored in order of creation in the CPU list and not in a topology order. To be able to build the SYSIB we need an intermediate structure to reorder the CPUs per container.

Quoted 8 lines

We can do this re-ordering during the STSI interception but the idea was to keep this instruction as fast as possible.> The second reason is to have a structure ready for the QEMU migration when we introduce vCPU migration from a socket to another socket, having then a different internal representation of the topology.

However, if as discussed yesterday we use a new cpu flag we would not need any special migration structure in the current series.

So it only stays the first reason to do the re-ordering preparation during the plugging of a vCPU, to optimize the STSI instruction.

If we think the optimization is not worth it or do not bring enough to be consider, we can do everything during the STSI interception.

Is it called on a hot code path ? AFAICT, it is only called once
per cpu when started. insert_stsi_3_2_2 is also a guest exit andit queries the machine definition in a very similar way.

Thanks,

C.

#20 lore raw

Re: [PATCH v11 04/11] s390x/cpu topology: reporting the CPU topology to the guest

Quoted 38 lines · Cédric Le Goater
Quoted 38 lines
Quoted 27 lines
Quoted 12 lines
target/s390x/cpu_topology.c new file +10 -0
+static char *s390_top_set_level2(S390Topology *topo, char *p)
+{
+    int i, origin;
+
+    for (i = 0; i < topo->nr_sockets; i++) {
+        if (!topo->socket[i].active_count) {
+            continue;
+        }
+        p = fill_container(p, 1, i);
+        for (origin = 0; origin < S390_TOPOLOGY_MAX_ORIGIN; 

origin++) {
+            uint64_t mask = 0L;

target/s390x/cpu_topology.c line 59

Quoted 9 lines
target/s390x/cpu_topology.c new file +9 -0
+
+            mask = topo->socket[i].mask[origin];
+            if (mask) {
+                p = fill_tle_cpu(p, mask, origin);
+            }
+        }
+    }
+    return p;
+}

Why is it not possible to compute this topo information at "runtime", when stsi is called, without maintaining state in an extra S390Topology object ? Couldn't we loop on the CPU list to gather the topology bits for the same result ?

Quoted 4 lines

It would greatly simplify the feature.

C.

The vCPU are not stored in order of creation in the CPU list and not in a topology order. To be able to build the SYSIB we need an intermediate structure to reorder the CPUs per container.

We can do this re-ordering during the STSI interception but the idea was to keep this instruction as fast as possible.> The second reason is to have a structure ready for the QEMU migration when we introduce vCPU migration from a socket to another socket, having then a different internal representation of the topology.

Quoted 8 lines
Quoted 5 lines

However, if as discussed yesterday we use a new cpu flag we would not need any special migration structure in the current series.

So it only stays the first reason to do the re-ordering preparation during the plugging of a vCPU, to optimize the STSI instruction.

If we think the optimization is not worth it or do not bring enough to be consider, we can do everything during the STSI interception.

Is it called on a hot code path ? AFAICT, it is only called once
per cpu when started. insert_stsi_3_2_2 is also a guest exit andit queries the machine definition in a very similar way.

It is not fully exact, stsi(15) is called at several moments, not only on CPU creation, but each time the core calls rebuild_sched_domains() that is for s390 on:
- change in the host topology
- changes in CPUSET: for allowed CPU or load balancing

Regards,
Pierre

Quoted 5 lines

Thanks,

C.

Signature · 2 lines
Pierre Morel
IBM Lab Boeblingen
#21 lore raw

Re: [PATCH v11 04/11] s390x/cpu topology: reporting the CPU topology to the guest

Quoted 51 lines · Pierre Morel
Quoted 38 lines · Cédric Le Goater
Quoted 38 lines
Quoted 27 lines
Quoted 12 lines
target/s390x/cpu_topology.c new file +10 -0
+static char *s390_top_set_level2(S390Topology *topo, char *p)
+{
+    int i, origin;
+
+    for (i = 0; i < topo->nr_sockets; i++) {
+        if (!topo->socket[i].active_count) {
+            continue;
+        }
+        p = fill_container(p, 1, i);
+        for (origin = 0; origin < S390_TOPOLOGY_MAX_ORIGIN; 

origin++) {
+            uint64_t mask = 0L;

target/s390x/cpu_topology.c line 59

Quoted 9 lines
target/s390x/cpu_topology.c new file +9 -0
+
+            mask = topo->socket[i].mask[origin];
+            if (mask) {
+                p = fill_tle_cpu(p, mask, origin);
+            }
+        }
+    }
+    return p;
+}

Why is it not possible to compute this topo information at "runtime", when stsi is called, without maintaining state in an extra S390Topology object ? Couldn't we loop on the CPU list to gather the topology bits for the same result ?

Quoted 4 lines

It would greatly simplify the feature.

C.

The vCPU are not stored in order of creation in the CPU list and not in a topology order. To be able to build the SYSIB we need an intermediate structure to reorder the CPUs per container.

We can do this re-ordering during the STSI interception but the idea was to keep this instruction as fast as possible.> The second reason is to have a structure ready for the QEMU migration when we introduce vCPU migration from a socket to another socket, having then a different internal representation of the topology.

Quoted 8 lines
Quoted 5 lines

However, if as discussed yesterday we use a new cpu flag we would not need any special migration structure in the current series.

So it only stays the first reason to do the re-ordering preparation during the plugging of a vCPU, to optimize the STSI instruction.

If we think the optimization is not worth it or do not bring enough to be consider, we can do everything during the STSI interception.

Is it called on a hot code path ? AFAICT, it is only called once
per cpu when started. insert_stsi_3_2_2 is also a guest exit andit queries the machine definition in a very similar way.

Quoted 6 lines

It is not fully exact, stsi(15) is called at several moments, not only on CPU creation, but each time the core calls rebuild_sched_domains() that is for s390 on:
- change in the host topology
- changes in CPUSET: for allowed CPU or load balancing

Regards,
Pierre

These are no good reasons to not make as you propose.
This allows to use the s390_has_feature() and use the cpu feature as proposed Christian. What I can not do with the early topology initialization.

Regards,
Pierre

Quoted 7 lines
Quoted 6 lines

Thanks,

C.

Signature · 2 lines
Pierre Morel
IBM Lab Boeblingen
#22 lore raw

PATCH v11 5/11 s390x/cpu_topology: resetting the Topology-Change-Report

In reply to Pierre Morel

During a subsystem reset the Topology-Change-Report is cleared by the machine.
Let's ask KVM to clear the Modified Topology Change Report (MTCR)
bit of the SCA in the case of a subsystem reset.

---
 target/s390x/cpu.h           |  1 +
 target/s390x/kvm/kvm_s390x.h |  1 +
 hw/s390x/cpu-topology.c      | 12 ++++++++++++
 hw/s390x/s390-virtio-ccw.c   |  1 +
 target/s390x/cpu-sysemu.c    | 13 +++++++++++++
 target/s390x/kvm/kvm.c       | 17 +++++++++++++++++
 6 files changed, 45 insertions(+)
target/s390x/cpu.h +1 -0
index 69a7523146..70de251b07 100644
@@ -856,6 +856,7 @@ void s390_enable_css_support(S390CPU *cpu);
 void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg);
 int s390_assign_subch_ioeventfd(EventNotifier *notifier, uint32_t sch_id,
                                 int vq, bool assign);
+void s390_cpu_topology_reset(void);
 #ifndef CONFIG_USER_ONLY
 unsigned int s390_cpu_set_state(uint8_t cpu_state, S390CPU *cpu);
 #else
target/s390x/kvm/kvm_s390x.h +1 -0
index f9785564d0..649dae5948 100644
@@ -47,5 +47,6 @@ void kvm_s390_crypto_reset(void);
 void kvm_s390_restart_interrupt(S390CPU *cpu);
 void kvm_s390_stop_interrupt(S390CPU *cpu);
 void kvm_s390_set_diag318(CPUState *cs, uint64_t diag318_info);
+int kvm_s390_topology_set_mtcr(uint64_t attr);
 
 #endif /* KVM_S390X_H */
hw/s390x/cpu-topology.c +12 -0
index 9fa8ca1261..21d2785b37 100644
@@ -94,6 +94,17 @@ static Property s390_topology_properties[] = {
     DEFINE_PROP_END_OF_LIST(),
 };
 
+/**
+ * s390_topology_reset:
+ * @dev: the device
+ *
+ * Calls the sysemu topology reset
+ */
+static void s390_topology_reset(DeviceState *dev)
+{
+    s390_cpu_topology_reset();
+}
+
 /**
  * topology_class_init:
  * @oc: Object class
@@ -108,6 +119,7 @@ static void topology_class_init(ObjectClass *oc, void *data)
     dc->realize = s390_topology_realize;
     device_class_set_props(dc, s390_topology_properties);
     set_bit(DEVICE_CATEGORY_MISC, dc->categories);
+    dc->reset = s390_topology_reset;
 }
 
 static const TypeInfo cpu_topology_info = {
hw/s390x/s390-virtio-ccw.c +1 -0
index 5776d3e58f..4de2622f99 100644
@@ -122,6 +122,7 @@ static const char *const reset_dev_types[] = {
     "s390-flic",
     "diag288",
     TYPE_S390_PCI_HOST_BRIDGE,
+    TYPE_S390_CPU_TOPOLOGY,
 };
 
 static void subsystem_reset(void)
target/s390x/cpu-sysemu.c +13 -0
index 948e4bd3e0..e27864c5f5 100644
@@ -306,3 +306,16 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg)
         kvm_s390_set_diag318(cs, arg.host_ulong);
     }
 }
+
+void s390_cpu_topology_reset(void)
+{
+    int ret;
+
+    if (kvm_enabled()) {
+        ret = kvm_s390_topology_set_mtcr(0);
+        if (ret) {
+            error_report("Failed to set Modified Topology Change Report: %s",
+                         strerror(-ret));
+        }
+    }
+}
target/s390x/kvm/kvm.c +17 -0
index 7dc96f3663..5b6383eab0 100644
@@ -2593,6 +2593,23 @@ int kvm_s390_get_zpci_op(void)
     return cap_zpci_op;
 }
 
+int kvm_s390_topology_set_mtcr(uint64_t attr)
+{
+    struct kvm_device_attr attribute = {
+        .group = KVM_S390_VM_CPU_TOPOLOGY,
+        .attr  = attr,
+    };
+
+    if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
+        return 0;
+    }
+    if (!kvm_vm_check_attr(kvm_state, KVM_S390_VM_CPU_TOPOLOGY, attr)) {
+        return -ENOTSUP;
+    }
+
+    return kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, &attribute);
+}
+
 void kvm_arch_accel_class_init(ObjectClass *oc)
 {
 }
Signature · 1 line
2.31.1
#23 lore raw

PATCH v11 6/11 s390x/cpu_topology: CPU topology migration

In reply to Pierre Morel

The migration can only take place if both source and destination of the migration both use or both do not use the CPU topology facility.

We indicate a change in topology during migration postload for the case the topology changed between source and destination.

---
 target/s390x/cpu.h        |  1 +
 hw/s390x/cpu-topology.c   | 79 +++++++++++++++++++++++++++++++++++++++
 target/s390x/cpu-sysemu.c |  8 ++++
 3 files changed, 88 insertions(+)
target/s390x/cpu.h +1 -0
index 70de251b07..53127c249c 100644
@@ -857,6 +857,7 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg);
 int s390_assign_subch_ioeventfd(EventNotifier *notifier, uint32_t sch_id,
                                 int vq, bool assign);
 void s390_cpu_topology_reset(void);
+int s390_cpu_topology_mtcr_set(void);
 #ifndef CONFIG_USER_ONLY
 unsigned int s390_cpu_set_state(uint8_t cpu_state, S390CPU *cpu);
 #else
hw/s390x/cpu-topology.c +79 -0

97 lines not on this page · raw message

target/s390x/cpu-sysemu.c +8 -0
index e27864c5f5..a8e3e6219d 100644
@@ -319,3 +319,11 @@ void s390_cpu_topology_reset(void)
         }
     }
 }
+
+int s390_cpu_topology_mtcr_set(void)
+{
+    if (kvm_enabled()) {
+        return kvm_s390_topology_set_mtcr(1);
+    }
+    return -ENOENT;
+}
Signature · 1 line
2.31.1
#24 lore raw

PATCH v11 7/11 target/s390x: interception of PTF instruction

In reply to Pierre Morel

When the host supports the CPU topology facility, the PTF instruction with function code 2 is interpreted by the SIE, provided that the userland hypervizor activates the interpretation by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension.

The PTF instructions with function code 0 and 1 are intercepted and must be emulated by the userland hypervizor.

---
 include/hw/s390x/s390-virtio-ccw.h |  6 ++++
 hw/s390x/cpu-topology.c            | 52 ++++++++++++++++++++++++++++++
 target/s390x/kvm/kvm.c             | 11 +++++++
 3 files changed, 69 insertions(+)
include/hw/s390x/s390-virtio-ccw.h +6 -0
index 23b472708d..6488279690 100644
@@ -32,6 +32,12 @@ struct S390CcwMachineState {
     void *topology;
 };
 
+#define S390_PTF_REASON_NONE (0x00 << 8)
+#define S390_PTF_REASON_DONE (0x01 << 8)
+#define S390_PTF_REASON_BUSY (0x02 << 8)
+#define S390_TOPO_FC_MASK 0xffUL
+void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra);
+
 struct S390CcwMachineClass {
     /*< private >*/
     MachineClass parent_class;
hw/s390x/cpu-topology.c +52 -0

58 lines not on this page · raw message

target/s390x/kvm/kvm.c +11 -0
index 5b6383eab0..a79fdf1c79 100644
@@ -97,6 +97,7 @@
 
 #define PRIV_B9_EQBS                    0x9c
 #define PRIV_B9_CLP                     0xa0
+#define PRIV_B9_PTF                     0xa2
 #define PRIV_B9_PCISTG                  0xd0
 #define PRIV_B9_PCILG                   0xd2
 #define PRIV_B9_RPCIT                   0xd3
@@ -1465,6 +1466,13 @@ static int kvm_mpcifc_service_call(S390CPU *cpu, struct kvm_run *run)
     }
 }
 
+static void kvm_handle_ptf(S390CPU *cpu, struct kvm_run *run)
+{
+    uint8_t r1 = (run->s390_sieic.ipb >> 20) & 0x0f;
+
+    s390_handle_ptf(cpu, r1, RA_IGNORED);
+}
+
 static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)
 {
     int r = 0;
@@ -1482,6 +1490,9 @@ static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)
     case PRIV_B9_RPCIT:
         r = kvm_rpcit_service_call(cpu, run);
         break;
+    case PRIV_B9_PTF:
+        kvm_handle_ptf(cpu, run);
+        break;
     case PRIV_B9_EQBS:
         /* just inject exception */
         r = -1;
Signature · 1 line
2.31.1
#25 lore raw

PATCH v11 8/11 s390x/cpu topology: add topology_capable QEMU capability

In reply to Pierre Morel

S390 CPU topology is only allowed for s390-virtio-ccw-7.2 and newer S390 machines.

---
 include/hw/s390x/s390-virtio-ccw.h | 1 +
 hw/s390x/s390-virtio-ccw.c         | 2 ++
 2 files changed, 3 insertions(+)
include/hw/s390x/s390-virtio-ccw.h +1 -0
index 6488279690..89fca3f79f 100644
@@ -48,6 +48,7 @@ struct S390CcwMachineClass {
     bool css_migration_enabled;
     bool hpage_1m_allowed;
     int max_threads;
+    bool topology_capable;
 };
 
 /* runtime-instrumentation allowed by the machine */
hw/s390x/s390-virtio-ccw.c +2 -0
index 4de2622f99..f1a9d6e793 100644
@@ -763,6 +763,7 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
     s390mc->css_migration_enabled = true;
     s390mc->hpage_1m_allowed = true;
     s390mc->max_threads = 1;
+    s390mc->topology_capable = true;
     mc->init = ccw_init;
     mc->reset = s390_machine_reset;
     mc->block_default_type = IF_VIRTIO;
@@ -896,6 +897,7 @@ static void ccw_machine_7_1_class_options(MachineClass *mc)
     ccw_machine_7_2_class_options(mc);
     compat_props_add(mc->compat_props, hw_compat_7_1, hw_compat_7_1_len);
     s390mc->max_threads = S390_MAX_CPUS;
+    s390mc->topology_capable = false;
 }
 DEFINE_CCW_MACHINE(7_1, "7.1", false);
 
Signature · 1 line
2.31.1
Reviewed-by on v11 8/11
#26 lore raw

Re: [PATCH v11 08/11] s390x/cpu topology: add topology_capable QEMU capability

Quoted 4 lines · Pierre Morel

S390 CPU topology is only allowed for s390-virtio-ccw-7.2 and newer S390 machines.

Thanks,

C.

Quoted 37 lines
---
  include/hw/s390x/s390-virtio-ccw.h | 1 +
  hw/s390x/s390-virtio-ccw.c         | 2 ++
  2 files changed, 3 insertions(+)
include/hw/s390x/s390-virtio-ccw.h +1 -0
index 6488279690..89fca3f79f 100644
@@ -48,6 +48,7 @@ struct S390CcwMachineClass {
      bool css_migration_enabled;
      bool hpage_1m_allowed;
      int max_threads;
+    bool topology_capable;
  };
 
  /* runtime-instrumentation allowed by the machine */
hw/s390x/s390-virtio-ccw.c +2 -0
index 4de2622f99..f1a9d6e793 100644
@@ -763,6 +763,7 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
      s390mc->css_migration_enabled = true;
      s390mc->hpage_1m_allowed = true;
      s390mc->max_threads = 1;
+    s390mc->topology_capable = true;
      mc->init = ccw_init;
      mc->reset = s390_machine_reset;
      mc->block_default_type = IF_VIRTIO;
@@ -896,6 +897,7 @@ static void ccw_machine_7_1_class_options(MachineClass *mc)
      ccw_machine_7_2_class_options(mc);
      compat_props_add(mc->compat_props, hw_compat_7_1, hw_compat_7_1_len);
      s390mc->max_threads = S390_MAX_CPUS;
+    s390mc->topology_capable = false;
  }
  DEFINE_CCW_MACHINE(7_1, "7.1", false);
 
#28 lore raw

PATCH v11 9/11 s390x/cpu topology: add topology machine property

In reply to Pierre Morel

We keep the possibility to switch on/off the topology on newer machines with the property topology=[on|off].

---
 include/hw/boards.h                |  3 +++
 include/hw/s390x/cpu-topology.h    |  8 +++-----
 include/hw/s390x/s390-virtio-ccw.h |  1 +
 hw/core/machine.c                  |  3 +++
 hw/s390x/cpu-topology.c            | 19 +++++++++++++++++++
 hw/s390x/s390-virtio-ccw.c         | 28 ++++++++++++++++++++++++++++
 util/qemu-config.c                 |  4 ++++
 qemu-options.hx                    |  6 +++++-
 8 files changed, 66 insertions(+), 6 deletions(-)
include/hw/boards.h +3 -0
index 311ed17e18..67147c47bf 100644
@@ -379,6 +379,9 @@ struct MachineState {
     } \
     type_init(machine_initfn##_register_types)
 
+extern GlobalProperty hw_compat_7_2[];
+extern const size_t hw_compat_7_2_len;
+
 extern GlobalProperty hw_compat_7_1[];
 extern const size_t hw_compat_7_1_len;
 
include/hw/s390x/cpu-topology.h +3 -5
index 6fec10e032..f566394302 100644
@@ -12,6 +12,8 @@
 
 #include "hw/qdev-core.h"
 #include "qom/object.h"
+#include "cpu.h"
+#include "hw/s390x/s390-virtio-ccw.h"
 
 #define S390_TOPOLOGY_CPU_IFL 0x03
 #define S390_TOPOLOGY_MAX_ORIGIN ((63 + S390_MAX_CPUS) / 64)
@@ -38,10 +40,6 @@ struct S390Topology {
 OBJECT_DECLARE_SIMPLE_TYPE(S390Topology, S390_CPU_TOPOLOGY)
 
 void s390_topology_new_cpu(S390CPU *cpu);
-
-static inline bool s390_has_topology(void)
-{
-    return false;
-}
+bool s390_has_topology(void);
 
 #endif
include/hw/s390x/s390-virtio-ccw.h +1 -0
index 89fca3f79f..d7602aedda 100644
@@ -28,6 +28,7 @@ struct S390CcwMachineState {
     bool dea_key_wrap;
     bool pv;
     bool zpcii_disable;
+    bool cpu_topology;
     uint8_t loadparm[8];
     void *topology;
 };
hw/core/machine.c +3 -0
index aa520e74a8..4f46d4ef23 100644
@@ -40,6 +40,9 @@
 #include "hw/virtio/virtio-pci.h"
 #include "qom/object_interfaces.h"
 
+GlobalProperty hw_compat_7_2[] = {};
+const size_t hw_compat_7_2_len = G_N_ELEMENTS(hw_compat_7_2);
+
 GlobalProperty hw_compat_7_1[] = {};
 const size_t hw_compat_7_1_len = G_N_ELEMENTS(hw_compat_7_1);
 
hw/s390x/cpu-topology.c +19 -0
index fc220bd8ac..c1550cc1e8 100644
@@ -73,6 +73,25 @@ void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
     }
 }
 
+bool s390_has_topology(void)
+{
+    static S390CcwMachineState *ccw;
+    Object *obj;
+
+    if (ccw) {
+        return ccw->cpu_topology;
+    }
+
+    /* we have to bail out for the "none" machine */
+    obj = object_dynamic_cast(qdev_get_machine(),
+                              TYPE_S390_CCW_MACHINE);
+    if (!obj) {
+        return false;
+    }
+    ccw = S390_CCW_MACHINE(obj);
+    return ccw->cpu_topology;
+}
+
 /*
  * s390_topology_new_cpu:
  * @cpu: a pointer to the new CPU
hw/s390x/s390-virtio-ccw.c +28 -0
index f1a9d6e793..ebb5615337 100644
@@ -710,6 +710,26 @@ bool hpage_1m_allowed(void)
     return get_machine_class()->hpage_1m_allowed;
 }
 
+static inline bool machine_get_topology(Object *obj, Error **errp)
+{
+    S390CcwMachineState *ms = S390_CCW_MACHINE(obj);
+
+    return ms->cpu_topology;
+}
+
+static inline void machine_set_topology(Object *obj, bool value, Error **errp)
+{
+    S390CcwMachineState *ms = S390_CCW_MACHINE(obj);
+
+    if (!get_machine_class()->topology_capable) {
+        error_setg(errp, "Property cpu-topology not available on machine %s",
+                   get_machine_class()->parent_class.name);
+        return;
+    }
+
+    ms->cpu_topology = value;
+}
+
 static void machine_get_loadparm(Object *obj, Visitor *v,
                                  const char *name, void *opaque,
                                  Error **errp)
@@ -809,6 +829,12 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
                                    machine_set_zpcii_disable);
     object_class_property_set_description(oc, "zpcii-disable",
             "disable zPCI interpretation facilties");
+
+    object_class_property_add_bool(oc, "topology",
+                                   machine_get_topology,
+                                   machine_set_topology);
+    object_class_property_set_description(oc, "topology",
+            "enable CPU topology");
 }
 
 static inline void s390_machine_initfn(Object *obj)
@@ -818,6 +844,7 @@ static inline void s390_machine_initfn(Object *obj)
     ms->aes_key_wrap = true;
     ms->dea_key_wrap = true;
     ms->zpcii_disable = false;
+    ms->cpu_topology = true;
 }
 
 static const TypeInfo ccw_machine_info = {
@@ -888,6 +915,7 @@ static void ccw_machine_7_1_instance_options(MachineState *machine)
     s390_cpudef_featoff_greater(16, 1, S390_FEAT_PAIE);
     s390_set_qemu_cpu_model(0x8561, 15, 1, qemu_cpu_feat);
     ms->zpcii_disable = true;
+    ms->cpu_topology = true;
 }
 
 static void ccw_machine_7_1_class_options(MachineClass *mc)
util/qemu-config.c +4 -0
index 5325f6bf80..0a040552bd 100644
@@ -240,6 +240,10 @@ static QemuOptsList machine_opts = {
             .name = "zpcii-disable",
             .type = QEMU_OPT_BOOL,
             .help = "disable zPCI interpretation facilities",
+        },{
+            .name = "topology",
+            .type = QEMU_OPT_BOOL,
+            .help = "disable CPU topology",
         },
         { /* End of list */ }
     }
qemu-options.hx +5 -1
index eb38e5dc40..ef59b28a03 100644
@@ -38,7 +38,8 @@ DEF("machine", HAS_ARG, QEMU_OPTION_machine, \
     "                hmat=on|off controls ACPI HMAT support (default=off)\n"
     "                memory-backend='backend-id' specifies explicitly provided backend for main RAM (default=none)\n"
     "                cxl-fmw.0.targets.0=firsttarget,cxl-fmw.0.targets.1=secondtarget,cxl-fmw.0.size=size[,cxl-fmw.0.interleave-granularity=granularity]\n"
-    "                zpcii-disable=on|off disables zPCI interpretation facilities (default=off)\n",
+    "                zpcii-disable=on|off disables zPCI interpretation facilities (default=off)\n"
+    "                topology=on|off disables CPU topology (default=off)\n",
     QEMU_ARCH_ALL)
 SRST
 ``-machine [type=]name[,prop=value[,...]]``
@@ -163,6 +164,9 @@ SRST
         Disables zPCI interpretation facilties on s390-ccw hosts.
         This feature can be used to disable hardware virtual assists
         related to zPCI devices. The default is off.
+
+    ``topology=on|off``
+        Disables CPU topology on for S390 machines starting with s390-ccw-virtio-7.3.
 ERST
 
 DEF("M", HAS_ARG, QEMU_OPTION_M,
Signature · 1 line
2.31.1
#29 lore raw

Re: [PATCH v11 09/11] s390x/cpu topology: add topology machine property

Quoted 20 lines · Pierre Morel

We keep the possibility to switch on/off the topology on newer machines with the property topology=[on|off].

---
 include/hw/boards.h                |  3 +++
 include/hw/s390x/cpu-topology.h    |  8 +++-----
 include/hw/s390x/s390-virtio-ccw.h |  1 +
 hw/core/machine.c                  |  3 +++
 hw/s390x/cpu-topology.c            | 19 +++++++++++++++++++
 hw/s390x/s390-virtio-ccw.c         | 28 ++++++++++++++++++++++++++++
 util/qemu-config.c                 |  4 ++++
 qemu-options.hx                    |  6 +++++-
 8 files changed, 66 insertions(+), 6 deletions(-)
include/hw/boards.h +0 -0
index 311ed17e18..67147c47bf 100644

include/hw/boards.h struct MachineState { · line 383

Quoted 5 lines
include/hw/boards.h +2 -0
@@ -379,6 +379,9 @@ struct MachineState {
     } \
     type_init(machine_initfn##_register_types)
 
+extern GlobalProperty hw_compat_7_2[];
+extern const size_t hw_compat_7_2_len;

This still needs to go into a separate patch that introduces the 8.0 machine types for the relevant machines... I'll probably write that patch soon (next week or so), you can pick it then into this series.

Quoted 4 lines
include/hw/boards.h +1 -0
+
 extern GlobalProperty hw_compat_7_1[];
 extern const size_t hw_compat_7_1_len;
 
#30 lore raw

Re: [PATCH v11 09/11] s390x/cpu topology: add topology machine property

Quoted 25 lines · Cornelia Huck
Quoted 23 lines · Pierre Morel

We keep the possibility to switch on/off the topology on newer machines with the property topology=[on|off].

---
  include/hw/boards.h                |  3 +++
  include/hw/s390x/cpu-topology.h    |  8 +++-----
  include/hw/s390x/s390-virtio-ccw.h |  1 +
  hw/core/machine.c                  |  3 +++
  hw/s390x/cpu-topology.c            | 19 +++++++++++++++++++
  hw/s390x/s390-virtio-ccw.c         | 28 ++++++++++++++++++++++++++++
  util/qemu-config.c                 |  4 ++++
  qemu-options.hx                    |  6 +++++-
  8 files changed, 66 insertions(+), 6 deletions(-)
include/hw/boards.h +0 -0
index 311ed17e18..67147c47bf 100644
@@ -379,6 +379,9 @@ struct MachineState {
      } \
      type_init(machine_initfn##_register_types)
 
Quoted 7 lines

include/hw/boards.h struct MachineState { · line 383

Quoted 2 lines
include/hw/boards.h +2 -0
+extern GlobalProperty hw_compat_7_2[];
+extern const size_t hw_compat_7_2_len;

This still needs to go into a separate patch that introduces the 8.0 machine types for the relevant machines... I'll probably write that patch soon (next week or so), you can pick it then into this series.

Oh sorry, I forgot to suppress these two definitions for this series. I do not need this for now.
I will probably need your patch introducing the 8.0 for the next spin.

Thanks,
Pierre

Signature · 2 lines
Pierre Morel
IBM Lab Boeblingen
#31 lore raw

Re: [PATCH v11 09/11] s390x/cpu topology: add topology machine property

In reply to Pierre Morel
Quoted 2 lines · Pierre Morel

We keep the possibility to switch on/off the topology on newer machines with the property topology=[on|off].

The code has changed. You will need to rebase. May be after the 8.0 machine is introduced, or include Cornelia's patch in the respin.

https://lore.kernel.org/qemu-devel/[email protected]/

Quoted 83 lines
---
  include/hw/boards.h                |  3 +++
  include/hw/s390x/cpu-topology.h    |  8 +++-----
  include/hw/s390x/s390-virtio-ccw.h |  1 +
  hw/core/machine.c                  |  3 +++
  hw/s390x/cpu-topology.c            | 19 +++++++++++++++++++
  hw/s390x/s390-virtio-ccw.c         | 28 ++++++++++++++++++++++++++++
  util/qemu-config.c                 |  4 ++++
  qemu-options.hx                    |  6 +++++-
  8 files changed, 66 insertions(+), 6 deletions(-)
include/hw/boards.h +3 -0
index 311ed17e18..67147c47bf 100644
@@ -379,6 +379,9 @@ struct MachineState {
      } \
      type_init(machine_initfn##_register_types)
 
+extern GlobalProperty hw_compat_7_2[];
+extern const size_t hw_compat_7_2_len;
+
  extern GlobalProperty hw_compat_7_1[];
  extern const size_t hw_compat_7_1_len;
 
include/hw/s390x/cpu-topology.h +3 -5
index 6fec10e032..f566394302 100644
@@ -12,6 +12,8 @@
 
  #include "hw/qdev-core.h"
  #include "qom/object.h"
+#include "cpu.h"
+#include "hw/s390x/s390-virtio-ccw.h"
 
  #define S390_TOPOLOGY_CPU_IFL 0x03
  #define S390_TOPOLOGY_MAX_ORIGIN ((63 + S390_MAX_CPUS) / 64)
@@ -38,10 +40,6 @@ struct S390Topology {
  OBJECT_DECLARE_SIMPLE_TYPE(S390Topology, S390_CPU_TOPOLOGY)
 
  void s390_topology_new_cpu(S390CPU *cpu);
-
-static inline bool s390_has_topology(void)
-{
-    return false;
-}
+bool s390_has_topology(void);
 
  #endif
include/hw/s390x/s390-virtio-ccw.h +1 -0
index 89fca3f79f..d7602aedda 100644
@@ -28,6 +28,7 @@ struct S390CcwMachineState {
      bool dea_key_wrap;
      bool pv;
      bool zpcii_disable;
+    bool cpu_topology;
      uint8_t loadparm[8];
      void *topology;
  };
hw/core/machine.c +3 -0
index aa520e74a8..4f46d4ef23 100644
@@ -40,6 +40,9 @@
  #include "hw/virtio/virtio-pci.h"
  #include "qom/object_interfaces.h"
 
+GlobalProperty hw_compat_7_2[] = {};
+const size_t hw_compat_7_2_len = G_N_ELEMENTS(hw_compat_7_2);
+
  GlobalProperty hw_compat_7_1[] = {};
  const size_t hw_compat_7_1_len = G_N_ELEMENTS(hw_compat_7_1);
 
hw/s390x/cpu-topology.c +0 -0
index fc220bd8ac..c1550cc1e8 100644

hw/s390x/cpu-topology.c void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra) · line 82

Quoted 10 lines
hw/s390x/cpu-topology.c +7 -0
@@ -73,6 +73,25 @@ void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
      }
  }
 
+bool s390_has_topology(void)
+{
+    static S390CcwMachineState *ccw;
+    Object *obj;
+
+    if (ccw) {
+        return ccw->cpu_topology;

Shouldn't we test the capability also ?

	return s390mc->topology_capable && ccw->cpu_topology;

hw/s390x/cpu-topology.c void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra) · line 90

Quoted 8 lines
hw/s390x/cpu-topology.c +8 -0
+    }
+
+    /* we have to bail out for the "none" machine */
+    obj = object_dynamic_cast(qdev_get_machine(),
+                              TYPE_S390_CCW_MACHINE);
+    if (!obj) {
+        return false;
+    }

Should be an assert I think.

Quoted 15 lines
hw/s390x/cpu-topology.c +4 -0
+    ccw = S390_CCW_MACHINE(obj);
+    return ccw->cpu_topology;
+}
+
  /*
   * s390_topology_new_cpu:
   * @cpu: a pointer to the new CPU
hw/s390x/s390-virtio-ccw.c +0 -0
index f1a9d6e793..ebb5615337 100644
@@ -710,6 +710,26 @@ bool hpage_1m_allowed(void)
      return get_machine_class()->hpage_1m_allowed;
  }
 

hw/s390x/s390-virtio-ccw.c bool hpage_1m_allowed(void) · line 722

Quoted 10 lines
hw/s390x/s390-virtio-ccw.c +10 -0
+static inline bool machine_get_topology(Object *obj, Error **errp)
+{
+    S390CcwMachineState *ms = S390_CCW_MACHINE(obj);
+
+    return ms->cpu_topology;
+}
+
+static inline void machine_set_topology(Object *obj, bool value, Error **errp)
+{
+    S390CcwMachineState *ms = S390_CCW_MACHINE(obj);

You could introduce :

       S390CcwMachineClass *s390mc = S390_CCW_MACHINE_GET_CLASS(ms);

hw/s390x/s390-virtio-ccw.c bool hpage_1m_allowed(void) · line 724

Quoted 2 lines
hw/s390x/s390-virtio-ccw.c +2 -0
+
+    if (!get_machine_class()->topology_capable) {

and

            !s390mc->topology_capable
Quoted 33 lines
hw/s390x/s390-virtio-ccw.c +15 -0
+        error_setg(errp, "Property cpu-topology not available on machine %s",
+                   get_machine_class()->parent_class.name);
+        return;
+    }
+
+    ms->cpu_topology = value;
+}
+
  static void machine_get_loadparm(Object *obj, Visitor *v,
                                   const char *name, void *opaque,
                                   Error **errp)
@@ -809,6 +829,12 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data)
                                     machine_set_zpcii_disable);
      object_class_property_set_description(oc, "zpcii-disable",
              "disable zPCI interpretation facilties");
+
+    object_class_property_add_bool(oc, "topology",
+                                   machine_get_topology,
+                                   machine_set_topology);
+    object_class_property_set_description(oc, "topology",
+            "enable CPU topology");
  }
 
  static inline void s390_machine_initfn(Object *obj)
@@ -818,6 +844,7 @@ static inline void s390_machine_initfn(Object *obj)
      ms->aes_key_wrap = true;
      ms->dea_key_wrap = true;
      ms->zpcii_disable = false;
+    ms->cpu_topology = true;
  }
 
  static const TypeInfo ccw_machine_info = {

hw/s390x/s390-virtio-ccw.c static void ccw_machine_7_1_instance_options(MachineState *machine) · line 918

Quoted 4 lines
hw/s390x/s390-virtio-ccw.c +1 -0
@@ -888,6 +915,7 @@ static void ccw_machine_7_1_instance_options(MachineState *machine)
      s390_cpudef_featoff_greater(16, 1, S390_FEAT_PAIE);
      s390_set_qemu_cpu_model(0x8561, 15, 1, qemu_cpu_feat);
      ms->zpcii_disable = true;
+    ms->cpu_topology = true;

shouldn't this be false ?

Thanks,

C.

Quoted 42 lines
hw/s390x/s390-virtio-ccw.c +0 -0
  }
 
  static void ccw_machine_7_1_class_options(MachineClass *mc)
util/qemu-config.c +4 -0
index 5325f6bf80..0a040552bd 100644
@@ -240,6 +240,10 @@ static QemuOptsList machine_opts = {
              .name = "zpcii-disable",
              .type = QEMU_OPT_BOOL,
              .help = "disable zPCI interpretation facilities",
+        },{
+            .name = "topology",
+            .type = QEMU_OPT_BOOL,
+            .help = "disable CPU topology",
          },
          { /* End of list */ }
      }
qemu-options.hx +5 -1
index eb38e5dc40..ef59b28a03 100644
@@ -38,7 +38,8 @@ DEF("machine", HAS_ARG, QEMU_OPTION_machine, \
      "                hmat=on|off controls ACPI HMAT support (default=off)\n"
      "                memory-backend='backend-id' specifies explicitly provided backend for main RAM (default=none)\n"
      "                cxl-fmw.0.targets.0=firsttarget,cxl-fmw.0.targets.1=secondtarget,cxl-fmw.0.size=size[,cxl-fmw.0.interleave-granularity=granularity]\n"
-    "                zpcii-disable=on|off disables zPCI interpretation facilities (default=off)\n",
+    "                zpcii-disable=on|off disables zPCI interpretation facilities (default=off)\n"
+    "                topology=on|off disables CPU topology (default=off)\n",
      QEMU_ARCH_ALL)
  SRST
  ``-machine [type=]name[,prop=value[,...]]``
@@ -163,6 +164,9 @@ SRST
          Disables zPCI interpretation facilties on s390-ccw hosts.
          This feature can be used to disable hardware virtual assists
          related to zPCI devices. The default is off.
+
+    ``topology=on|off``
+        Disables CPU topology on for S390 machines starting with s390-ccw-virtio-7.3.
  ERST
 
  DEF("M", HAS_ARG, QEMU_OPTION_M,
#32 lore raw

Re: [PATCH v11 09/11] s390x/cpu topology: add topology machine property

Quoted 98 lines · Cédric Le Goater
Quoted 2 lines · Pierre Morel

We keep the possibility to switch on/off the topology on newer machines with the property topology=[on|off].

The code has changed. You will need to rebase. May be after the 8.0 machine is introduced, or include Cornelia's patch in the respin.

https://lore.kernel.org/qemu-devel/[email protected]/

Quoted 75 lines

---
  include/hw/boards.h                |  3 +++ include/hw/s390x/cpu-topology.h    |  8 +++----- include/hw/s390x/s390-virtio-ccw.h |  1 + hw/core/machine.c                  |  3 +++ hw/s390x/cpu-topology.c            | 19 +++++++++++++++++++ hw/s390x/s390-virtio-ccw.c         | 28 ++++++++++++++++++++++++++++ util/qemu-config.c                 |  4 ++++ qemu-options.hx                    |  6 +++++-

  8 files changed, 66 insertions(+), 6 deletions(-)
include/hw/boards.h +3 -0
index 311ed17e18..67147c47bf 100644
@@ -379,6 +379,9 @@ struct MachineState {
       } \
       type_init(machine_initfn##_register_types)
+extern GlobalProperty hw_compat_7_2[];
+extern const size_t hw_compat_7_2_len;
+
   extern GlobalProperty hw_compat_7_1[];
   extern const size_t hw_compat_7_1_len;
include/hw/s390x/cpu-topology.h +3 -5
index 6fec10e032..f566394302 100644
@@ -12,6 +12,8 @@
   #include "hw/qdev-core.h"
   #include "qom/object.h"
+#include "cpu.h"
+#include "hw/s390x/s390-virtio-ccw.h"
   #define S390_TOPOLOGY_CPU_IFL 0x03
   #define S390_TOPOLOGY_MAX_ORIGIN ((63 + S390_MAX_CPUS) / 64)
@@ -38,10 +40,6 @@ struct S390Topology {
   OBJECT_DECLARE_SIMPLE_TYPE(S390Topology, S390_CPU_TOPOLOGY)
   void s390_topology_new_cpu(S390CPU *cpu);
-
-static inline bool s390_has_topology(void)
-{
-    return false;
-}
+bool s390_has_topology(void);
   #endif
include/hw/s390x/s390-virtio-ccw.h +1 -0
index 89fca3f79f..d7602aedda 100644
@@ -28,6 +28,7 @@ struct S390CcwMachineState {
       bool dea_key_wrap;
       bool pv;
       bool zpcii_disable;
+    bool cpu_topology;
       uint8_t loadparm[8];
       void *topology;
   };
hw/core/machine.c +3 -0
index aa520e74a8..4f46d4ef23 100644
@@ -40,6 +40,9 @@
   #include "hw/virtio/virtio-pci.h"
   #include "qom/object_interfaces.h"
+GlobalProperty hw_compat_7_2[] = {};
+const size_t hw_compat_7_2_len = G_N_ELEMENTS(hw_compat_7_2);
+
   GlobalProperty hw_compat_7_1[] = {};
   const size_t hw_compat_7_1_len = G_N_ELEMENTS(hw_compat_7_1);
hw/s390x/cpu-topology.c +0 -0
index fc220bd8ac..c1550cc1e8 100644

hw/s390x/cpu-topology.c void s390_handle_ptf(S390CPU *cpu, uint8_t r1, · line 82

Quoted 10 lines
hw/s390x/cpu-topology.c +7 -0
@@ -73,6 +73,25 @@ void s390_handle_ptf(S390CPU *cpu, uint8_t r1, 
 uintptr_t ra)
       }
   }
+bool s390_has_topology(void)
+{
+    static S390CcwMachineState *ccw;
+    Object *obj;
+
+    if (ccw) {
+        return ccw->cpu_topology;

Shouldn't we test the capability also ?

Quoted 1 line

    return s390mc->topology_capable && ccw->cpu_topology;

yes thanks

Quoted 11 lines

hw/s390x/cpu-topology.c void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra) · line 90

Quoted 9 lines
hw/s390x/cpu-topology.c +8 -0
+    }
+
+    /* we have to bail out for the "none" machine */
+    obj = object_dynamic_cast(qdev_get_machine(),
+                              TYPE_S390_CCW_MACHINE);
+    if (!obj) {
+        return false;
+    }

Should be an assert I think.

OK

Quoted 29 lines
Quoted 16 lines
hw/s390x/cpu-topology.c +4 -0
+    ccw = S390_CCW_MACHINE(obj);
+    return ccw->cpu_topology;
+}
+
   /*
    * s390_topology_new_cpu:
    * @cpu: a pointer to the new CPU
hw/s390x/s390-virtio-ccw.c +1 -0
index f1a9d6e793..ebb5615337 100644
@@ -710,6 +710,26 @@ bool hpage_1m_allowed(void)
       return get_machine_class()->hpage_1m_allowed;
   }
+static inline bool machine_get_topology(Object *obj, Error **errp)

hw/s390x/s390-virtio-ccw.c bool hpage_1m_allowed(void) · line 722

Quoted 10 lines
hw/s390x/s390-virtio-ccw.c +9 -0
+{
+    S390CcwMachineState *ms = S390_CCW_MACHINE(obj);
+
+    return ms->cpu_topology;
+}
+
+static inline void machine_set_topology(Object *obj, bool value, 
 Error **errp)
+{
+    S390CcwMachineState *ms = S390_CCW_MACHINE(obj);

You could introduce :

Quoted 1 line

       S390CcwMachineClass *s390mc = S390_CCW_MACHINE_GET_CLASS(ms);

Yes thanks

Quoted 40 lines

hw/s390x/s390-virtio-ccw.c bool hpage_1m_allowed(void) · line 724

Quoted 4 lines
hw/s390x/s390-virtio-ccw.c +2 -0
+
+    if (!get_machine_class()->topology_capable) {

and
            !s390mc->topology_capable

Quoted 32 lines
hw/s390x/s390-virtio-ccw.c +8 -0
+        error_setg(errp, "Property cpu-topology not available on 
 machine %s",
+                   get_machine_class()->parent_class.name);
+        return;
+    }
+
+    ms->cpu_topology = value;
+}
+
   static void machine_get_loadparm(Object *obj, Visitor *v,
                                    const char *name, void *opaque,

                                   Error **errp)

hw/s390x/s390-virtio-ccw.c +7 -0
@@ -809,6 +829,12 @@ static void ccw_machine_class_init(ObjectClass 
 *oc, void *data)
                                      machine_set_zpcii_disable);
       object_class_property_set_description(oc, "zpcii-disable",
               "disable zPCI interpretation facilties");
+
+    object_class_property_add_bool(oc, "topology",
+                                   machine_get_topology,
+                                   machine_set_topology);
+    object_class_property_set_description(oc, "topology",
+            "enable CPU topology");
   }
   static inline void s390_machine_initfn(Object *obj)
@@ -818,6 +844,7 @@ static inline void s390_machine_initfn(Object *obj)
       ms->aes_key_wrap = true;
       ms->dea_key_wrap = true;
       ms->zpcii_disable = false;
+    ms->cpu_topology = true;
   }
   static const TypeInfo ccw_machine_info = {
Quoted 8 lines

hw/s390x/s390-virtio-ccw.c static void · line 919

Quoted 6 lines
hw/s390x/s390-virtio-ccw.c +1 -0
@@ -888,6 +915,7 @@ static void 
 ccw_machine_7_1_instance_options(MachineState *machine)
       s390_cpudef_featoff_greater(16, 1, S390_FEAT_PAIE);
       s390_set_qemu_cpu_model(0x8561, 15, 1, qemu_cpu_feat);
       ms->zpcii_disable = true;
+    ms->cpu_topology = true;

shouldn't this be false ?

:) yes
I forgot to change this when I change the logic.

Thanks,
Pierre

Signature · 2 lines
Pierre Morel
IBM Lab Boeblingen

A message that is not in the archive (sent off-list or not yet received): PH0PR22MB3210864C22AD57E5B32F626991079@PH0PR22MB3210.namprd22.prod.outlook.com

#34 lore raw

Thank you! s390x/cpu topology

Friends,

I am but a small bystander as I watch in awe at the incredible work each of you do to advance the Kernel and Linux.
What you are working on is so important. Your ideas, and how you express them into the code you write, become the foundation for a better world. I can tell you that your support for s390x architecture is quite meaningful to my company as we work to improve access, cost, and positive outcomes in healthcare. We use many s390x Linux servers and VMs for critical systems.

Thank you for everything you do. Many of us at many companies deeply appreciate you.

With sincere respect,

-Jadon

Jadon McDowell | [optum%20logo%20gif%20200px]
Director & Senior Principal Engineer, TLCP
Enterprise Hosting Services: HPC Infrastructure Delivery
Mobile: 612-940-9268 | [email protected]<mailto:[email protected]>

This e-mail, including attachments, may include confidential and/or proprietary information, and may be used only by the person or entity to which it is addressed. If the reader of this e-mail is not the intended recipient or intended recipient’s authorized agent, the reader is hereby notified that any dissemination, distribution or copying of this e-mail is prohibited. If you have received this e-mail in error, please notify the sender by replying to this message and delete this e-mail immediately.

#35 lore raw

PATCH v11 10/11 s390x/cpu_topology: activating CPU topology

In reply to Pierre Morel

The KVM capability, KVM_CAP_S390_CPU_TOPOLOGY is used to activate the S390_FEAT_CONFIGURATION_TOPOLOGY feature and the topology facility for the guest in the case the topology is available in QEMU and in KVM.

---
 target/s390x/kvm/kvm.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
target/s390x/kvm/kvm.c +16 -0
index a79fdf1c79..724758680b 100644
@@ -2471,6 +2471,22 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp)
         set_bit(S390_FEAT_UNPACK, model->features);
     }
 
+    /*
+     * If we have the CPU Topology allowed in QEMU and
+     * implemented in KVM, activate the CPU TOPOLOGY feature.
+     */
+    if (s390_has_topology()) {
+        if (!kvm_check_extension(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY)) {
+            error_setg(errp, "KVM: S390 topology not present in kernel");
+            return;
+        }
+        if (kvm_vm_enable_cap(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY, 0) < 0) {
+            error_setg(errp, "KVM: Error enabling KVM_CAP_S390_CPU_TOPOLOGY");
+            return;
+        }
+        set_bit(S390_FEAT_CONFIGURATION_TOPOLOGY, model->features);
+    }
+
     /* We emulate a zPCI bus and AEN, therefore we don't need HW support */
     set_bit(S390_FEAT_ZPCI, model->features);
     set_bit(S390_FEAT_ADAPTER_EVENT_NOTIFICATION, model->features);
Signature · 1 line
2.31.1
#36 lore raw

PATCH v11 11/11 docs/s390x: document s390x cpu topology

In reply to Pierre Morel

Add some basic examples for the definition of cpu topology in s390x.

---
 docs/system/s390x/cpu-topology.rst | 80 ++++++++++++++++++++++++++++++
 1 file changed, 80 insertions(+)
 create mode 100644 docs/system/s390x/cpu-topology.rst
docs/system/s390x/cpu-topology.rst new file +80 -0
new file mode 100644
index 0000000000..a3bb0efadf
@@ -0,0 +1,80 @@
+CPU Topology on s390x
+=====================
+
+CPU Topology on S390x provides up to 5 levels of topology containers:
+nodes, drawers, books, sockets and CPUs.
+While the higher level containers, Containers Topology List Entries,
+(Containers TLE) define a tree hierarchy, the lowest level of topology
+definition, the CPU Topology List Entry (CPU TLE), provides the placement
+of the CPUs inside the parent container.
+
+Currently QEMU CPU topology uses a single level of container: the sockets.
+
+For backward compatibility, threads can be declared on the ``-smp`` command
+line. They will be seen as CPUs by the guest as long as multithreading
+is not really supported by QEMU for S390.
+
+Prerequisites
+-------------
+
+To use CPU Topology a Linux QEMU/KVM machine providing the CPU Topology facility
+(STFLE bit 11) is required.
+
+However, since this facility has been enabled by default in an early version
+of QEMU, we use a capability, ``KVM_CAP_S390_CPU_TOPOLOGY``, to notify KVM
+QEMU use of the CPU Topology.
+
+Indicating the CPU topology to the Virtual Machine
+--------------------------------------------------
+
+The CPU Topology, can be specified on the QEMU command line
+with the ``-smp`` or the ``-device`` QEMU command arguments.
+
+Like in :
+
+.. code-block:: sh
+    -smp cpus=5,sockets=8,cores=2,threads=2,maxcpus=32
+    -device host-s390x-cpu,core-id=14
+
+New CPUs can be plugged using the device_add hmp command like in:
+
+.. code-block:: sh
+   (qemu) device_add host-s390x-cpu,core-id=9
+
+The core-id defines the placement of the core in the topology by
+starting with core 0 in socket 0 up to maxcpus.
+
+In the example above:
+
+* There are 5 CPUs provided to the guest with the ``-smp`` command line
+  They will take the core-ids 0,1,2,3,4
+  As we have 2 threads in 2 cores in a socket, we have 4 CPUs provided
+  to the guest in socket 0, with core-ids 0,1,2,3.
+  The last cpu, with core-id 4, will be on socket 1.
+
+* the core with ID 14 provided by the ``-device`` command line will
+  be placed in socket 3, with core-id 14
+
+* the core with ID 9 provided by the ``device_add`` qmp command will
+  be placed in socket 2, with core-id 9
+
+Note that the core ID is machine wide and the CPU TLE masks provided
+by the STSI instruction will be:
+
+* in socket 0: 0xf0000000 (core id 0,1,2,3)
+* in socket 1: 0x00400000 (core id 9)
+* in socket 1: 0x00020000 (core id 14)
+
+Migration
+---------
+
+For virtio-ccw machines older than s390-virtio-ccw-7.2, CPU Topoogy is
+unavailable.
+
+CPU topology is by default enabled for s390-virtio-ccw-7.2 and newer machines.
+
+Disabling CPU topology can be done by setting the global option
+``topology`` to ``off`` like in:
+
+.. code-block:: sh
+   -machine s390-ccw-virtio-7.2,accel=kvm,topology=off
Signature · 1 line
2.31.1