s390x: CPU Topology
Subsystems (2)
v9 cover s390x: CPU Topology
Hi,
The implementation of the CPU Topology in QEMU has been drastically modified since the last patch series and the number of LOCs has been greatly reduced.
Unnecessary objects have been removed, only a single S390Topology object is created to support migration and reset.
Also a documentation has been added to the series.
To use these patches, you will need Linux V6-rc1 or newer.
Mainline patches needed are:
f5ecfee94493 2022-07-20 KVM: s390: resetting the Topology-Change-Report 24fe0195bc19 2022-07-20 KVM: s390: guest support for topology function 0130337ec45b 2022-07-20 KVM: s390: Cleanup ipte lock access and SIIF fac..
Currently this code is for KVM only, I have no idea if it is interesting to provide a TCG patch. If ever it will be done in another series.
To have a better understanding of the S390x CPU Topology and its implementation in QEMU you can have a look at the documentation in the last patch.
New in this series
==================
s390x/cpus: Make absence of multithreading clear
This patch makes clear that CPU-multithreading is not supported in the guest.
s390x/cpu topology: core_id sets s390x CPU topology
This patch uses the core_id to build the container topology and the placement of the CPU inside the container.
s390x/cpu topology: reporting the CPU topology to the guest
This patch is based on the fact that the CPU type for guests is always IFL, CPUs are always dedicated and the polarity is always horizontal.
This may change in the future.
hw/core: introducing drawer and books for s390x s390x/cpu: reporting drawers and books topology to the guest
These two patches extend the topology handling to add two new containers levels above sockets: books and drawers.
The subject of the last patches is clear enough (I hope).
Regards,
Pierre
Pierre Morel (10):
s390x/cpus: Make absence of multithreading clear s390x/cpu topology: core_id sets s390x CPU topology s390x/cpu topology: reporting the CPU topology to the guest hw/core: introducing drawer and books for s390x s390x/cpu: reporting drawers and books topology to the guest s390x/cpu_topology: resetting the Topology-Change-Report s390x/cpu_topology: CPU topology migration target/s390x: interception of PTF instruction s390x/cpu_topology: activating CPU topology docs/s390x: document s390x cpu topology
docs/system/s390x/cpu_topology.rst | 88 +++++++++ hw/core/machine-smp.c | 48 ++++- hw/core/machine.c | 9 + hw/s390x/cpu-topology.c | 293 +++++++++++++++++++++++++++++ hw/s390x/meson.build | 1 + hw/s390x/s390-virtio-ccw.c | 61 +++++- include/hw/boards.h | 11 ++ include/hw/s390x/cpu-topology.h | 53 ++++++ include/hw/s390x/s390-virtio-ccw.h | 7 + qapi/machine.json | 14 +- qemu-options.hx | 6 +- softmmu/vl.c | 6 + target/s390x/cpu-sysemu.c | 15 ++ target/s390x/cpu.h | 51 +++++ target/s390x/cpu_topology.c | 150 +++++++++++++++ target/s390x/kvm/kvm.c | 56 +++++- target/s390x/kvm/kvm_s390x.h | 1 + target/s390x/meson.build | 1 + 18 files changed, 858 insertions(+), 13 deletions(-) create mode 100644 docs/system/s390x/cpu_topology.rst create mode 100644 hw/s390x/cpu-topology.c create mode 100644 include/hw/s390x/cpu-topology.h create mode 100644 target/s390x/cpu_topology.c
Signature · 68 lines
2.31.1 Changelog: - since v8 - Linux patches are now mainline - simplification of the implementation (Janis) - Migration, new machine definition (Thomas) - Documentation - since v7 - Coherence with the Linux patch series changes for MTCR get (Pierre) - check return values during new CPU creation (Thomas) - Improving codding style and argument usages (Thomas) - since v6 - Changes on smp args in qemu-options (Daniel) - changed comments in machine.jason (Daniel) - Added reset (Janosch) - since v5 - rebasing on newer QEMU version - reworked most lines above 80 characters. - since v4 - Added drawer and books to topology - Added numa topology - Added documentation - since v3 - Added migration (Thomas) - Separated STSI instruction from KVM to prepare TCG (Thomas) - Take care of endianess to prepare TCG (Thomas) - Added comments on STSI CPU container and PFT instruction (Thomas) - Moved enabling the instructions as the last patch (Thomas)
PATCH v9 1/10 s390x/cpus: Make absence of multithreading clear
S390x do not support multithreading in the guest. Do not let admin falsely specify multithreading on QEMU smp commandline.
--- hw/s390x/s390-virtio-ccw.c | 3 +++ 1 file changed, 3 insertions(+)
hw/s390x/s390-virtio-ccw.c +3 -0
@@ -86,6 +86,9 @@ static void s390_init_cpus(MachineState *machine)MachineClass *mc =MACHINE_GET_CLASS (machine);int i; +/* Explicitely do not support threads */ +assert (machine->smp.threads ==1 ); +/* initialize possible_cpus */ mc->possible_cpu_arch_ids (machine);
Signature · 1 line
2.31.1
Re: [PATCH v9 01/10] s390x/cpus: Make absence of multithreading clear
Quoting Pierre Morel (2022-09-02 09:55:22)
Quoted 14 lines
S390x do not support multithreading in the guest. Do not let admin falsely specify multithreading on QEMU smp commandline.
Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/s390-virtio-ccw.c | 3 +++ 1 file changed, 3 insertions(+)hw/s390x/s390-virtio-ccw.c +0
-0
hw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 89
Quoted 4 lines
hw/s390x/s390-virtio-ccw.c +1
-0@@ -86,6 +86,9 @@ static void s390_init_cpus(MachineState *machine)MachineClass *mc =MACHINE_GET_CLASS (machine);int i; +/* Explicitely do not support threads */
^
Explicitlyhw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 90
Quoted 1 line
hw/s390x/s390-virtio-ccw.c +1
-0+assert (machine->smp.threads ==1 );
It might be nicer to give a better error message to the user. What do you think about something like (broken whitespace ahead):
if (machine->smp.threads != 1) {
error_setg(&error_fatal, "More than one thread specified, but multithreading unsupported");
return;
}Re: [PATCH v9 01/10] s390x/cpus: Make absence of multithreading clear
Quoted 24 lines · Nico Boehr
Quoting Pierre Morel (2022-09-02 09:55:22)
Quoted 14 lines
S390x do not support multithreading in the guest. Do not let admin falsely specify multithreading on QEMU smp commandline.
Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/s390-virtio-ccw.c | 3 +++ 1 file changed, 3 insertions(+)hw/s390x/s390-virtio-ccw.c +0
-0hw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 89
Quoted 4 lines
hw/s390x/s390-virtio-ccw.c +1
-0@@ -86,6 +86,9 @@ static void s390_init_cpus(MachineState *machine)MachineClass *mc =MACHINE_GET_CLASS (machine);int i; +/* Explicitely do not support threads */ ^ Explicitlyhw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 90
Quoted 1 line
hw/s390x/s390-virtio-ccw.c +1
-0+assert (machine->smp.threads ==1 );
Quoted 8 lines
It might be nicer to give a better error message to the user. What do you think about something like (broken whitespace ahead):
if (machine->smp.threads != 1) {if (machine->smp.threads != 1) { error_setg(&error_fatal, "More than one thread specified, but multithreading unsupported"); return; }
OK, I think I wanted to do this and I changed my mind, obviously, I do not recall why.
I will do almost the same but after a look at error.h I will use error_report()/exit() instead of error_setg()/return as in:
+ /* Explicitly do not support threads */
+ if (machine->smp.threads != 1) {
+ error_report("More than one thread specified, but multithreading unsupported");
+ exit(1);
+ }
Thanks,
Regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v9 01/10] s390x/cpus: Make absence of multithreading clear
Quoted 46 lines · Pierre Morel
Quoted 24 lines · Nico Boehr
Quoting Pierre Morel (2022-09-02 09:55:22)
Quoted 14 lines
S390x do not support multithreading in the guest. Do not let admin falsely specify multithreading on QEMU smp commandline.
Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/s390-virtio-ccw.c | 3 +++ 1 file changed, 3 insertions(+)hw/s390x/s390-virtio-ccw.c +0
-0hw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 89
Quoted 4 lines
hw/s390x/s390-virtio-ccw.c +1
-0@@ -86,6 +86,9 @@ static void s390_init_cpus(MachineState *machine)MachineClass *mc =MACHINE_GET_CLASS (machine);int i; +/* Explicitely do not support threads */ ^ Explicitlyhw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 90
Quoted 1 line
hw/s390x/s390-virtio-ccw.c +1
-0+assert (machine->smp.threads ==1 );Quoted 8 lines
It might be nicer to give a better error message to the user. What do you think about something like (broken whitespace ahead):
if (machine->smp.threads != 1) {if (machine->smp.threads != 1) { error_setg(&error_fatal, "More than one thread specified, but multithreading unsupported"); return; }OK, I think I wanted to do this and I changed my mind, obviously, I do not recall why.
I will do almost the same but after a look aterror.hI will use error_report()/exit() instead of error_setg()/return as in:(quoted diff) +3
-0+ /* Explicitly do not support threads */ + if (machine->smp.threads != 1) { + error_report("More than one thread specified, butmultithreading unsupported");
Review
Quoted 2 lines
(quoted diff) +2
-0+ exit(1); + }
I agree that an assert is not a good solution, and I'm not sure aborting is a good idea either.
I'm assuming that currently if you specify threads > 0 qemu will run with the number of CPUs multiplied by threads (compared to threads=1). If that is true, then a new qemu version will break existing invocations.
An alternative would be to print a warning and do:
cores *= threads
threads = 1
The questions would be what the best place to do that is.
I guess we'd need a new compat variable if that's done in machine-smp.c
Quoted 7 lines
Thanks,
Regards,
Pierre
Re: [PATCH v9 01/10] s390x/cpus: Make absence of multithreading clear
Quoted 57 lines · Janis Schoetterl-Glausch
Quoted 46 lines · Pierre Morel
Quoted 24 lines · Nico Boehr
Quoting Pierre Morel (2022-09-02 09:55:22)
Quoted 14 lines
S390x do not support multithreading in the guest. Do not let admin falsely specify multithreading on QEMU smp commandline.
Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/s390-virtio-ccw.c | 3 +++ 1 file changed, 3 insertions(+)hw/s390x/s390-virtio-ccw.c +0
-0hw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 89
Quoted 4 lines
hw/s390x/s390-virtio-ccw.c +1
-0@@ -86,6 +86,9 @@ static void s390_init_cpus(MachineState *machine)MachineClass *mc =MACHINE_GET_CLASS (machine);int i; +/* Explicitely do not support threads */ ^ Explicitlyhw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 90
Quoted 1 line
hw/s390x/s390-virtio-ccw.c +1
-0+assert (machine->smp.threads ==1 );Quoted 8 lines
It might be nicer to give a better error message to the user. What do you think about something like (broken whitespace ahead):
if (machine->smp.threads != 1) {if (machine->smp.threads != 1) { error_setg(&error_fatal, "More than one thread specified, but multithreading unsupported"); return; }OK, I think I wanted to do this and I changed my mind, obviously, I do not recall why.
I will do almost the same but after a look aterror.hI will use error_report()/exit() instead of error_setg()/return as in:(quoted diff) +3
-0+ /* Explicitly do not support threads */ + if (machine->smp.threads != 1) { + error_report("More than one thread specified, butmultithreading unsupported");
Review
Quoted 2 lines
(quoted diff) +2
-0+ exit(1); + }I agree that an assert is not a good solution, and I'm not sure aborting is a good idea either.
I'm assuming that currently if you specify threads > 0 qemu will run with the number of CPUs multiplied by threads (compared to threads=1). If that is true, then a new qemu version will break existing invocations.
Quoted 6 lines
An alternative would be to print a warning and do:
cores *= threads
threads = 1The questions would be what the best place to do that is.
I guess we'd need a new compat variable if that's done inmachine-smp.c
Right, I think we can use the new "topology_disable" machine property.
Quoted 8 lines
Quoted 7 lines
Thanks,
Regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v9 01/10] s390x/cpus: Make absence of multithreading clear
In reply to Pierre MorelQuoted 40 lines · Pierre Morel
Quoted 23 lines · Nico Boehr
Quoting Pierre Morel (2022-09-02 09:55:22)
Quoted 14 lines
S390x do not support multithreading in the guest. Do not let admin falsely specify multithreading on QEMU smp commandline.
Signed-off-by Pierre Morel <[email protected]>---
hw/s390x/s390-virtio-ccw.c| 3 +++1 file changed, 3 insertions(+)hw/s390x/s390-virtio-ccw.c +0
-0hw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 88
Quoted 3 lines
hw/s390x/s390-virtio-ccw.c +1
-0@@ -86,6 +86,9 @@ static void s390_init_cpus(MachineState *machine) MachineClass *mc = MACHINE_GET_CLASS(machine); int i; + /* Explicitely do not support threads */^
Explicitlyhw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 90
Quoted 1 line
hw/s390x/s390-virtio-ccw.c +1
-0+ assert(machine->smp.threads == 1);Quoted 8 lines
It might be nicer to give a better error message to the user. What do you think about something like (broken whitespace ahead):
if (
machine->smp.threads!= 1) {if (machine->smp.threads!= 1) {error_setg(&error_fatal, "More than one thread specified, but multithreading unsupported"); return;
}OK, I think I wanted to do this and I changed my mind, obviously, I do not recall why.
I will do almost the same but after a look aterror.hI will use error_report()/exit() instead of error_setg()/return as in:
Quoted 5 lines
+ /* Explicitly do not support threads */
+ if (machine->smp.threads!= 1) {
+error_report("More than one thread specified, but multithreading unsupported"); + exit(1);
+ }
or add an 'Error **errp' parameter to s390_init_cpus() and use error_setg() as initially proposed. s390x_new_cpu() would benefit from it also.
Thanks,
C.
Quoted 7 lines
Thanks,
Regards,
Pierre
Re: [PATCH v9 01/10] s390x/cpus: Make absence of multithreading clear
Quoted 49 lines · Cédric Le Goater
Quoted 40 lines · Pierre Morel
Quoted 23 lines · Nico Boehr
Quoting Pierre Morel (2022-09-02 09:55:22)
Quoted 14 lines
S390x do not support multithreading in the guest. Do not let admin falsely specify multithreading on QEMU smp commandline.
Signed-off-by Pierre Morel <[email protected]>---
hw/s390x/s390-virtio-ccw.c| 3 +++1 file changed, 3 insertions(+)hw/s390x/s390-virtio-ccw.c +0
-0hw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 88
Quoted 3 lines
hw/s390x/s390-virtio-ccw.c +1
-0@@ -86,6 +86,9 @@ static void s390_init_cpus(MachineState *machine) MachineClass *mc = MACHINE_GET_CLASS(machine); int i; + /* Explicitely do not support threads */^
Explicitlyhw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 90
Quoted 1 line
hw/s390x/s390-virtio-ccw.c +1
-0+ assert(machine->smp.threads == 1);Quoted 8 lines
It might be nicer to give a better error message to the user. What do you think about something like (broken whitespace ahead):
if (
machine->smp.threads!= 1) {if (machine->smp.threads!= 1) {error_setg(&error_fatal, "More than one thread specified, but multithreading unsupported"); return;
}OK, I think I wanted to do this and I changed my mind, obviously, I do not recall why.
I will do almost the same but after a look aterror.hI will use error_report()/exit() instead of error_setg()/return as in:Quoted 6 lines
+ /* Explicitly do not support threads */
+ if (machine->smp.threads!= 1) {
+error_report("More than one thread specified, but multithreading unsupported");
+ exit(1);
+ }
Quoted 3 lines
or add an 'Error **errp' parameter to
s390_init_cpus()and useerror_setg()as initially proposed.s390x_new_cpu()would benefit from it also.
OK, Thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v9 01/10] s390x/cpus: Make absence of multithreading clear
More thinking about this I will drop this patch for backward compatibility and in topology masks treat CPUs as being cores*threads
Quoted 59 lines · Pierre Morel
Quoted 49 lines · Cédric Le Goater
Quoted 40 lines · Pierre Morel
Quoted 23 lines · Nico Boehr
Quoting Pierre Morel (2022-09-02 09:55:22)
Quoted 14 lines
S390x do not support multithreading in the guest. Do not let admin falsely specify multithreading on QEMU smp commandline.
Signed-off-by Pierre Morel <[email protected]>---
hw/s390x/s390-virtio-ccw.c| 3 +++1 file changed, 3 insertions(+)hw/s390x/s390-virtio-ccw.c +0
-0hw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 88
Quoted 3 lines
hw/s390x/s390-virtio-ccw.c +1
-0@@ -86,6 +86,9 @@ static void s390_init_cpus(MachineState *machine) MachineClass *mc = MACHINE_GET_CLASS(machine); int i; + /* Explicitely do not support threads */^
Explicitlyhw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 90
Quoted 1 line
hw/s390x/s390-virtio-ccw.c +1
-0+ assert(machine->smp.threads == 1);Quoted 8 lines
It might be nicer to give a better error message to the user. What do you think about something like (broken whitespace ahead):
if (
machine->smp.threads!= 1) {if (machine->smp.threads!= 1) {error_setg(&error_fatal, "More than one thread specified, but multithreading unsupported"); return;
}OK, I think I wanted to do this and I changed my mind, obviously, I do not recall why.
I will do almost the same but after a look aterror.hI will use error_report()/exit() instead of error_setg()/return as in:Quoted 6 lines
+ /* Explicitly do not support threads */
+ if (machine->smp.threads!= 1) {
+error_report("More than one thread specified, but multithreading unsupported");
+ exit(1);
+ }Quoted 3 lines
or add an 'Error **errp' parameter to
s390_init_cpus()and useerror_setg()as initially proposed.s390x_new_cpu()would benefit from it also.OK, Thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v9 01/10] s390x/cpus: Make absence of multithreading clear
Quoted 1 line · Pierre Morel
More thinking about this I will drop this patch for backward compatibility and in topology masks treat CPUs as being cores*threads
yes. You never know, people might have set threads=2 in their domain file (like me). You could give the user a warning though, with warn_report().
Thanks,
C.
Quoted 63 lines · Pierre Morel
Quoted 58 lines · Cédric Le Goater
Quoted 48 lines · Pierre Morel
Quoted 40 lines · Nico Boehr
Quoted 23 lines
Quoting Pierre Morel (2022-09-02 09:55:22)
Quoted 14 lines
S390x do not support multithreading in the guest. Do not let admin falsely specify multithreading on QEMU smp commandline.
Signed-off-by Pierre Morel <[email protected]>---
hw/s390x/s390-virtio-ccw.c| 3 +++1 file changed, 3 insertions(+)hw/s390x/s390-virtio-ccw.c +0
-0hw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 88
Quoted 3 lines
hw/s390x/s390-virtio-ccw.c +1
-0@@ -86,6 +86,9 @@ static void s390_init_cpus(MachineState *machine) MachineClass *mc = MACHINE_GET_CLASS(machine); int i; + /* Explicitely do not support threads */^
Explicitlyhw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 90
Quoted 1 line
hw/s390x/s390-virtio-ccw.c +1
-0+ assert(machine->smp.threads == 1);Quoted 8 lines
It might be nicer to give a better error message to the user. What do you think about something like (broken whitespace ahead):
if (
machine->smp.threads!= 1) {if (machine->smp.threads!= 1) {error_setg(&error_fatal, "More than one thread specified, but multithreading unsupported"); return;
}OK, I think I wanted to do this and I changed my mind, obviously, I do not recall why.
I will do almost the same but after a look aterror.hI will use error_report()/exit() instead of error_setg()/return as in:Quoted 5 lines
+ /* Explicitly do not support threads */
+ if (machine->smp.threads!= 1) {
+error_report("More than one thread specified, but multithreading unsupported"); + exit(1);
+ }Quoted 3 lines
or add an 'Error **errp' parameter to
s390_init_cpus()and useerror_setg()as initially proposed.s390x_new_cpu()would benefit from it also.OK, Thanks,
Pierre
Re: [PATCH v9 01/10] s390x/cpus: Make absence of multithreading clear
Quoted 6 lines · Cédric Le Goater
Quoted 1 line · Pierre Morel
More thinking about this I will drop this patch for backward compatibility and in topology masks treat CPUs as being cores*threads
yes. You never know, people might have set threads=2 in their domain file (like me). You could give the user a warning though, with
warn_report().
More thinking, I come back to the first idea after Daniel comment and protect the change with a new machine type version.
Quoted 7 lines
Thanks,
C.
Quoted 1 line
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v9 01/10] s390x/cpus: Make absence of multithreading clear
Quoted 11 lines · Pierre Morel
Quoted 6 lines · Cédric Le Goater
Quoted 1 line · Pierre Morel
More thinking about this I will drop this patch for backward compatibility and in topology masks treat CPUs as being cores*threads
yes. You never know, people might have set threads=2 in their domain file (like me). You could give the user a warning though, with
warn_report().More thinking, I come back to the first idea after Daniel comment and protect the change with a new machine type version.
yes. That would be another machine class attribute to set in the new machine, may be 'max_threads' to compare with the user provided value.
C.
Quoted 10 lines
Quoted 9 lines
Thanks,
C.
Quoted 1 line
Re: [PATCH v9 01/10] s390x/cpus: Make absence of multithreading clear
In reply to Pierre MorelQuoted 14 lines · Pierre Morel
S390x do not support multithreading in the guest. Do not let admin falsely specify multithreading on QEMU smp commandline.
Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/s390-virtio-ccw.c | 3 +++ 1 file changed, 3 insertions(+)hw/s390x/s390-virtio-ccw.c +0
-0
hw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 90
Quoted 5 lines
hw/s390x/s390-virtio-ccw.c +2
-0@@ -86,6 +86,9 @@ static void s390_init_cpus(MachineState *machine)MachineClass *mc =MACHINE_GET_CLASS (machine);int i; +/* Explicitely do not support threads */ +assert (machine->smp.threads ==1 );
What is the functional effect for currently released QEMU versions if a user has set threads == 2 for an s390 machine ? Is the threads setting simply ignored ?
If we want to eliminate this mistake, then there's two possible options
* If it had no effect, treat this like a deprecation process where we print a warning for 2 releases, and then turn the warning into an error. Gives a little grace to fix the config mistakes some users might have made, at a time convenient to them.
Or
* If it had effect and we need migration compatibility then forbid threads > 1 only for new machine type versions, so existing deployed guests are not changed.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH v9 01/10] s390x/cpus: Make absence of multithreading clear
Quoted 18 lines · Daniel P. Berrangé
Quoted 17 lines · Pierre Morel
S390x do not support multithreading in the guest. Do not let admin falsely specify multithreading on QEMU smp commandline.
Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/s390-virtio-ccw.c | 3 +++ 1 file changed, 3 insertions(+)hw/s390x/s390-virtio-ccw.c +0
-0@@ -86,6 +86,9 @@ static void s390_init_cpus(MachineState *machine)MachineClass *mc =MACHINE_GET_CLASS (machine);int i;
Quoted 6 lines
hw/s390x/s390-virtio-ccw.c static void s390_init_cpus(MachineState *machine) · line 90
Quoted 2 lines
hw/s390x/s390-virtio-ccw.c +2
-0+/* Explicitely do not support threads */ +assert (machine->smp.threads ==1 );What is the functional effect for currently released QEMU versions if a user has set threads == 2 for an s390 machine ? Is the threads setting simply ignored ?
It is not ignored, the number of CPUs per sockets seen by the guest is cores*threads
Quoted 10 lines
If we want to eliminate this mistake, then there's two possible options
* If it had no effect, treat this like a deprecation process where we print a warning for 2 releases, and then turn the warning into an error. Gives a little grace to fix the config mistakes some users might have made, at a time convenient to them.
Quoted 8 lines
Or
* If it had effect and we need migration compatibility then forbid threads > 1 only for new machine type versions, so existing deployed guests are not changed.
With regards,
Daniel
Thanks for your comments Daniel.
I will need to forbid threads > 1 for new machine.
regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
PATCH v9 2/10 s390x/cpu topology: core_id sets s390x CPU topology
In reply to Pierre MorelIn the S390x CPU topology the core_id specifies the CPU address and the position of the core withing the topology.
Let's build the topology based on the core_id.
--- hw/s390x/cpu-topology.c | 135 ++++++++++++++++++++++++++++++++ hw/s390x/meson.build | 1 + hw/s390x/s390-virtio-ccw.c | 10 +++ include/hw/s390x/cpu-topology.h | 42 ++++++++++ 4 files changed, 188 insertions(+) create mode 100644 hw/s390x/cpu-topology.c create mode 100644 include/hw/s390x/cpu-topology.h
hw/s390x/cpu-topology.c new file +135 -0
@@ -0,0 +1,135 @@ +/* +* CPU Topology +* +* Copyright IBM Corp. 2022 +* Author(s): Pierre Morel <[email protected]> + +* This work is licensed under the terms of the GNU GPL, version 2 or (at +* your option) any later version. See the COPYING file in the top-level +* directory. +*/ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "hw/sysbus.h" +#include "hw/qdev-properties.h" +#include "hw/boards.h" +#include "qemu/typedefs.h" +#include "target/s390x/cpu.h" +#include "hw/s390x/s390-virtio-ccw.h" +#include "hw/s390x/cpu-topology.h" + +S390Topology *s390_get_topology (void ) +{ +static S390Topology *s390Topology; + +if (!s390Topology) { + s390Topology =S390_CPU_TOPOLOGY ( +object_resolve_path (TYPE_S390_CPU_TOPOLOGY ,NULL )); + } + +return s390Topology; +} + +/* +* s390_topology_new_cpu: +* @core_id: the core ID is machine wide +* +* The topology returned by s390_get_topology(), gives us the CPU +* topology established by the -smp QEMU aruments. +* The core-id gives: +* - the Container TLE (Topology List Entry) containing the CPU TLE. +* - in the CPU TLE the origin, or offset of the first bit in the core mask +* - the bit in the CPU TLE core mask +*/ +void s390_topology_new_cpu (int core_id) +{ +S390Topology *topo =s390_get_topology (); +int socket_id; +int bit, origin; + +/* In the case no Topology is used nothing is to be done here */ +if (!topo) { +return ; + } + + socket_id = core_id / topo->cores; + + bit = core_id; + origin = bit /64 ; + bit %=64 ; + bit =63 - bit; + +/* +* At the core level, each CPU is represented by a bit in a 64bit +* unsigned long. Set on plug and clear on unplug of a CPU. +* The firmware assume that all CPU in a CPU TLE have the same +* type, polarization and are all dedicated or shared. +* In the case a socket contains CPU with different type, polarization +* or entitlement then they will be defined in different CPU containers. +* Currently we assume all CPU are identical IFL CPUs and that they are +* all dedicated CPUs. +* The only reason to have several S390TopologyCores inside a socket is +* to have more than 64 CPUs. +* In that case the origin field, representing the offset of the first CPU +* in the CPU container allows to represent up to the maximal number of +* CPU inside several CPU containers inside the socket container. +*/ + topo->socket[socket_id].active_count++; + topo->tle[socket_id].active_count++; +set_bit (bit, &topo->tle[socket_id].mask[origin]); +} + +/** +* s390_topology_realize: +* @dev: the device state +* @errp: the error pointer (not used) +* +* During realize the machine CPU topology is initialized with the +* QEMU -smp parameters. +* The maximum count of CPU TLE in the all Topology can not be greater +* than the maximum CPUs. +*/ +static void s390_topology_realize (DeviceState *dev,Error **errp) +{ +MachineState *ms =MACHINE (qdev_get_machine ()); +S390Topology *topo =S390_CPU_TOPOLOGY (dev); +int n; + + topo->sockets = ms->smp.sockets; + topo->cores = ms->smp.cores; + topo->tles = ms->smp.max_cpus; + + n = topo->sockets; + topo->socket =g_malloc0 (n *sizeof (S390TopoContainer)); + topo->tle =g_malloc0 (topo->tles *sizeof (S390TopoTLE)); +} + +/** +* topology_class_init: +* @oc: Object class +* @data: (not used) +* +* A very simple object we will need for reset and migration. +*/ +static void topology_class_init (ObjectClass *oc,void *data) +{ +DeviceClass *dc =DEVICE_CLASS (oc); + + dc->realize = s390_topology_realize; +set_bit (DEVICE_CATEGORY_MISC , dc->categories); +} + +static const TypeInfo cpu_topology_info = { + .name =TYPE_S390_CPU_TOPOLOGY , + .parent =TYPE_SYS_BUS_DEVICE , + .instance_size =sizeof (S390Topology), + .class_init = topology_class_init, +}; + +static void topology_register (void ) +{ +type_register_static (&cpu_topology_info); +} +type_init (topology_register);
hw/s390x/meson.build +1 -0
@@ -2,6 +2,7 @@ s390x_ss = ss.source_set() s390x_ss.add(files( 'ap-bridge.c', 'ap-device.c', + 'cpu-topology.c', 'ccw-device.c', 'css-bridge.c', 'css.c',
hw/s390x/s390-virtio-ccw.c +10 -0
@@ -43,6 +43,7 @@#include "sysemu/sysemu.h" #include "hw/s390x/pv.h" #include "migration/blocker.h" +#include "hw/s390x/cpu-topology.h" static Error *pv_mig_blocker;
@@ -247,6 +248,12 @@ static void ccw_init(MachineState *machine)/* init memory + setup max page size. Required for the CPU model */ s390_memory_init (machine->ram); +/* Adding the topology must be done before CPU intialization*/ + dev =qdev_new (TYPE_S390_CPU_TOPOLOGY ); +object_property_add_child (qdev_get_machine (),TYPE_S390_CPU_TOPOLOGY , +OBJECT (dev)); +sysbus_realize_and_unref (SYS_BUS_DEVICE (dev), &error_fatal); +/* init CPUs (incl. CPU model) early so s390_has_feature() works */ s390_init_cpus (machine);
@@ -309,6 +316,9 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev,g_assert (!ms->possible_cpus->cpus[cpu->env.core_id].cpu); ms->possible_cpus->cpus[cpu->env.core_id].cpu =OBJECT (dev); +/* Inserting the CPU in the Topology can not fail */ +s390_topology_new_cpu (cpu->env.core_id); +if (dev->hotplugged) {raise_irq_cpu_hotplug (); }
include/hw/s390x/cpu-topology.h new file +42 -0
42 lines not on this page · raw message
Signature · 1 line
2.31.1
Re: [PATCH v9 02/10] s390x/cpu topology: core_id sets s390x CPU topology
Quoted 17 lines · Pierre Morel
In the S390x CPU topology the
core_idspecifies the CPU address and the position of the core withing the topology.Let's build the topology based on the
core_id.Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/cpu-topology.c | 135 ++++++++++++++++++++++++++++++++ hw/s390x/meson.build | 1 + hw/s390x/s390-virtio-ccw.c | 10 +++ include/hw/s390x/cpu-topology.h | 42 ++++++++++ 4 files changed, 188 insertions(+) create mode 100644 hw/s390x/cpu-topology.c create mode 100644 include/hw/s390x/cpu-topology.hhw/s390x/cpu-topology.c +0
-0
[...]
Quoted 13 lines
hw/s390x/cpu-topology.c +13
-0+/** +* s390_topology_realize: +* @dev: the device state +* @errp: the error pointer (not used) +* +* During realize the machine CPU topology is initialized with the +* QEMU -smp parameters. +* The maximum count of CPU TLE in the all Topology can not be greater +* than the maximum CPUs. +*/ +static void s390_topology_realize (DeviceState *dev,Error **errp) +{ +MachineState *ms =MACHINE (qdev_get_machine ());
hw/s390x/cpu-topology.c line 106
Quoted 10 lines
hw/s390x/cpu-topology.c +10
-0+S390Topology *topo =S390_CPU_TOPOLOGY (dev); +int n; + + topo->sockets = ms->smp.sockets; + topo->cores = ms->smp.cores; + topo->tles = ms->smp.max_cpus; + + n = topo->sockets; + topo->socket =g_malloc0 (n *sizeof (S390TopoContainer)); + topo->tle =g_malloc0 (topo->tles *sizeof (S390TopoTLE));
Seems like a good use case for g_new0.
[...]
Quoted 8 lines
include/hw/s390x/cpu-topology.h new file +1
-0@@ -0,0 +1,42 @@ +/*
include/hw/s390x/cpu-topology.h line 11
Quoted 10 lines
include/hw/s390x/cpu-topology.h new file +10
-0+* CPU Topology +* +* Copyright 2022 IBM Corp. +* +* This work is licensed under the terms of the GNU GPL, version 2 or (at +* your option) any later version. See the COPYING file in the top-level +* directory. +*/ +#ifndef HW_S390X_CPU_TOPOLOGY_H +#define HW_S390X_CPU_TOPOLOGY_H
Is there a reason this is before the includes?
include/hw/s390x/cpu-topology.h line 17
Quoted 6 lines
include/hw/s390x/cpu-topology.h new file +6
-0+ +typedef struct S390TopoContainer { +int active_count; +}S390TopoContainer ; + +#define S390_TOPOLOGY_MAX_ORIGIN (1 + S390_MAX_CPUS / 64)
This is correct because cpu_id == core_id for s390, right? So the cpu limit also applies to the core id.
You could do ((S390_MAX_CPUS + 63) / 64) instead. But if you chose this for simplicity's sake, I'm fine with it.
include/hw/s390x/cpu-topology.h line 19
Quoted 2 lines
include/hw/s390x/cpu-topology.h new file +2
-0+typedef struct S390TopoTLE { + int active_count;
Do you use (read) this field somewhere?
Is this in anticipation of there being multiple TLE arrays, for different polarizations, etc? If so I would defer this for later.
include/hw/s390x/cpu-topology.h line 29
Quoted 10 lines
include/hw/s390x/cpu-topology.h new file +10
-0+uint64_t mask[S390_TOPOLOGY_MAX_ORIGIN ]; +} S390TopoTLE; + +#include "hw/qdev-core.h" +#include "qom/object.h" + +struct S390Topology { +SysBusDevice parent_obj; +int sockets; +int cores;
These are just cached values from machine_state.smp, right? Not sure if I like the redundancy, it doesn't aid in comprehension.
include/hw/s390x/cpu-topology.h line 34
Quoted 5 lines
include/hw/s390x/cpu-topology.h new file +5
-0+int tles; +S390TopoContainer *socket; +S390TopoTLE *tle; +}; +typedef struct S390Topology S390Topology ;
The DECLARE macro takes care of this typedef.
Quoted 8 lines
include/hw/s390x/cpu-topology.h new file +8
-0+ +#define TYPE_S390_CPU_TOPOLOGY "s390-topology" +OBJECT_DECLARE_SIMPLE_TYPE(S390Topology, S390_CPU_TOPOLOGY) + +S390Topology *s390_get_topology (void ); +void s390_topology_new_cpu (int core_id); + +#endif
Re: [PATCH v9 02/10] s390x/cpu topology: core_id sets s390x CPU topology
Quoted 37 lines · Janis Schoetterl-Glausch
Quoted 17 lines · Pierre Morel
In the S390x CPU topology the
core_idspecifies the CPU address and the position of the core withing the topology.Let's build the topology based on the
core_id.Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/cpu-topology.c | 135 ++++++++++++++++++++++++++++++++ hw/s390x/meson.build | 1 + hw/s390x/s390-virtio-ccw.c | 10 +++ include/hw/s390x/cpu-topology.h | 42 ++++++++++ 4 files changed, 188 insertions(+)create mode 100644
hw/s390x/cpu-topology.ccreate mode 100644include/hw/s390x/cpu-topology.hhw/s390x/cpu-topology.c +0
-0[...]
Quoted 17 lines
hw/s390x/cpu-topology.c +17
-0+/** +* s390_topology_realize: +* @dev: the device state +* @errp: the error pointer (not used) +* +* During realize the machine CPU topology is initialized with the +* QEMU -smp parameters. +* The maximum count of CPU TLE in the all Topology can not be greater +* than the maximum CPUs. +*/ +static void s390_topology_realize (DeviceState *dev,Error **errp) +{ +MachineState *ms =MACHINE (qdev_get_machine ()); +S390Topology *topo =S390_CPU_TOPOLOGY (dev); +int n; + + topo->sockets = ms->smp.sockets;
Quoted 8 lines
hw/s390x/cpu-topology.c line 106
Quoted 6 lines
hw/s390x/cpu-topology.c +6
-0+ topo->cores = ms->smp.cores; + topo->tles = ms->smp.max_cpus; + + n = topo->sockets; + topo->socket =g_malloc0 (n *sizeof (S390TopoContainer)); + topo->tle =g_malloc0 (topo->tles *sizeof (S390TopoTLE));Seems like a good use case for
g_new0.
Yes
Quoted 22 lines
[...]
Quoted 8 lines
include/hw/s390x/cpu-topology.h new file +1
-0@@ -0,0 +1,42 @@ +/* include/hw/s390x/cpu-topology.h line 11
Quoted 10 lines
include/hw/s390x/cpu-topology.h new file +10
-0+* CPU Topology +* +* Copyright 2022 IBM Corp. +* +* This work is licensed under the terms of the GNU GPL, version 2 or (at +* your option) any later version. See the COPYING file in the top-level +* directory. +*/ +#ifndef HW_S390X_CPU_TOPOLOGY_H +#define HW_S390X_CPU_TOPOLOGY_H Is there a reason this is before the includes?
Quoted 11 lines
include/hw/s390x/cpu-topology.h line 17
Quoted 6 lines
include/hw/s390x/cpu-topology.h new file +6
-0+ +typedef struct S390TopoContainer { +int active_count; +}S390TopoContainer ; + +#define S390_TOPOLOGY_MAX_ORIGIN (1 + S390_MAX_CPUS / 64)This is correct because
cpu_id==core_idfor s390, right? So the cpu limit also applies to the core id.
You could do ((S390_MAX_CPUS+ 63) / 64) instead. But if you chose this for simplicity's sake, I'm fine with it.
hum, I think your proposition is right and mine is false. If S390_MAX_CPUS is 64 we have only 1 origin.
I count 2 but you count 1.
So I change this.
Quoted 7 lines
include/hw/s390x/cpu-topology.h line 19
Quoted 3 lines
include/hw/s390x/cpu-topology.h new file +2
-0+typedef struct S390TopoTLE { + int active_count;Do you use (read) this field somewhere?
Is this in anticipation of there being multiple TLE arrays, for different polarizations, etc? If so I would defer this for later.
OK
Quoted 15 lines
include/hw/s390x/cpu-topology.h line 29
Quoted 11 lines
include/hw/s390x/cpu-topology.h new file +10
-0+uint64_t mask[S390_TOPOLOGY_MAX_ORIGIN ]; +} S390TopoTLE; + +#include "hw/qdev-core.h" +#include "qom/object.h" + +struct S390Topology { +SysBusDevice parent_obj; +int sockets; +int cores;These are just cached values from machine_state.smp, right? Not sure if I like the redundancy, it doesn't aid in comprehension.
Quoted 7 lines
include/hw/s390x/cpu-topology.h line 34
Quoted 5 lines
include/hw/s390x/cpu-topology.h new file +5
-0+int tles; +S390TopoContainer *socket; +S390TopoTLE *tle; +}; +typedef struct S390Topology S390Topology ;The DECLARE macro takes care of this typedef.
right
Quoted 10 lines
Quoted 9 lines
include/hw/s390x/cpu-topology.h new file +8
-0+ +#define TYPE_S390_CPU_TOPOLOGY "s390-topology" +OBJECT_DECLARE_SIMPLE_TYPE(S390Topology, S390_CPU_TOPOLOGY) + +S390Topology *s390_get_topology (void ); +void s390_topology_new_cpu (int core_id); + +#endif
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v9 02/10] s390x/cpu topology: core_id sets s390x CPU topology
In reply to Pierre MorelQuoting Pierre Morel (2022-09-02 09:55:23)
[...]
Quoted 5 lines
hw/s390x/cpu-topology.c new file +0
-0
[...]
Quoted 2 lines
hw/s390x/cpu-topology.c new file +2
-0+void s390_topology_new_cpu (int core_id) +{
[...]
Quoted 1 line
+
socket_id=core_id/topo->cores;
The comment below is essential for understanding all of this. Move it before this line.
hw/s390x/cpu-topology.c line 66
Quoted 9 lines
hw/s390x/cpu-topology.c new file +9
-0+ + bit = core_id; + origin = bit /64 ; + bit %=64 ; + bit =63 - bit; + +/* +* At the core level, each CPU is represented by a bit in a 64bit +* unsigned long. Set on plug and clear on unplug of a CPU.
cleared ^
[...]
hw/s390x/cpu-topology.c line 77
Quoted 3 lines
hw/s390x/cpu-topology.c new file +3
-0+* In that case the origin field, representing the offset of the first CPU +* in the CPU container allows to represent up to the maximal number of +* CPU inside several CPU containers inside the socket container.
How about:
"In that case the origin variable represents the offset of the first CPU in the CPU container. More than 64 CPUs per socket are represented in several CPU containers inside the socket container."
Quoted 2 lines
hw/s390x/s390-virtio-ccw.c +0
-0
[...]
hw/s390x/s390-virtio-ccw.c static void ccw_init(MachineState *machine) · line 251
Quoted 5 lines
hw/s390x/s390-virtio-ccw.c +1
-0@@ -247,6 +248,12 @@ static void ccw_init(MachineState *machine)/* init memory + setup max page size. Required for the CPU model */ s390_memory_init (machine->ram); +/* Adding the topology must be done before CPU intialization*/
space ^
Re: [PATCH v9 02/10] s390x/cpu topology: core_id sets s390x CPU topology
Quoted 8 lines · Nico Boehr
Quoting Pierre Morel (2022-09-02 09:55:23)
[...]Quoted 5 lines
hw/s390x/cpu-topology.c new file +0
-0[...]
Quoted 7 lines
Quoted 2 lines
hw/s390x/cpu-topology.c new file +2
-0+void s390_topology_new_cpu (int core_id) +{[...]
Quoted 1 line
+
socket_id=core_id/topo->cores;The comment below is essential for understanding all of this. Move it before this line.
OK
Quoted 13 lines
hw/s390x/cpu-topology.c line 66
Quoted 9 lines
hw/s390x/cpu-topology.c new file +9
-0+ + bit = core_id; + origin = bit /64 ; + bit %=64 ; + bit =63 - bit; + +/* +* At the core level, each CPU is represented by a bit in a 64bit +* unsigned long. Set on plug and clear on unplug of a CPU. cleared ^
[...]
Quoted 8 lines
hw/s390x/cpu-topology.c line 77
Quoted 3 lines
hw/s390x/cpu-topology.c new file +3
-0+* In that case the origin field, representing the offset of the first CPU +* in the CPU container allows to represent up to the maximal number of +* CPU inside several CPU containers inside the socket container. How about:
"In that case the origin variable represents the offset of the first CPU in the CPU container. More than 64 CPUs per socket are represented in several CPU containers inside the socket container."
Yes, better, thanks I take it
Quoted 4 lines
Quoted 3 lines
hw/s390x/s390-virtio-ccw.c +0
-0[...]
Quoted 8 lines
hw/s390x/s390-virtio-ccw.c static void ccw_init(MachineState *machine) · line 251
Quoted 5 lines
hw/s390x/s390-virtio-ccw.c +1
-0@@ -247,6 +248,12 @@ static void ccw_init(MachineState *machine)/* init memory + setup max page size. Required for the CPU model */ s390_memory_init (machine->ram); +/* Adding the topology must be done before CPU intialization*/ space ^
Yes thanks
regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v9 02/10] s390x/cpu topology: core_id sets s390x CPU topology
In reply to Pierre MorelQuoted 77 lines · Pierre Morel
In the S390x CPU topology the
core_idspecifies the CPU address and the position of the core withing the topology.Let's build the topology based on the
core_id.Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/cpu-topology.c | 135 ++++++++++++++++++++++++++++++++ hw/s390x/meson.build | 1 + hw/s390x/s390-virtio-ccw.c | 10 +++ include/hw/s390x/cpu-topology.h | 42 ++++++++++ 4 files changed, 188 insertions(+)create mode 100644
hw/s390x/cpu-topology.ccreate mode 100644include/hw/s390x/cpu-topology.hhw/s390x/cpu-topology.c new file +56
-0@@ -0,0 +1,135 @@ +/* +* CPU Topology +* +* Copyright IBM Corp. 2022 +* Author(s): Pierre Morel <[email protected]> + +* This work is licensed under the terms of the GNU GPL, version 2 or (at +* your option) any later version. See the COPYING file in the top-level +* directory. +*/ + +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "hw/sysbus.h" +#include "hw/qdev-properties.h" +#include "hw/boards.h" +#include "qemu/typedefs.h" +#include "target/s390x/cpu.h" +#include "hw/s390x/s390-virtio-ccw.h" +#include "hw/s390x/cpu-topology.h" + +S390Topology *s390_get_topology (void ) +{ +static S390Topology *s390Topology; + +if (!s390Topology) { + s390Topology =S390_CPU_TOPOLOGY ( +object_resolve_path (TYPE_S390_CPU_TOPOLOGY ,NULL )); + } + +return s390Topology; +} + +/* +* s390_topology_new_cpu: +* @core_id: the core ID is machine wide +* +* The topology returned by s390_get_topology(), gives us the CPU +* topology established by the -smp QEMU aruments. +* The core-id gives: +* - the Container TLE (Topology List Entry) containing the CPU TLE. +* - in the CPU TLE the origin, or offset of the first bit in the core mask +* - the bit in the CPU TLE core mask +*/ +void s390_topology_new_cpu (int core_id) +{ +S390Topology *topo =s390_get_topology (); +int socket_id; +int bit, origin; + +/* In the case no Topology is used nothing is to be done here */ +if (!topo) { +return ; + } +
hw/s390x/cpu-topology.c line 66
Quoted 10 lines
hw/s390x/cpu-topology.c new file +10
-0+ socket_id = core_id / topo->cores; + + bit = core_id; + origin = bit /64 ; + bit %=64 ; + bit =63 - bit; + +/* +* At the core level, each CPU is represented by a bit in a 64bit +* unsigned long. Set on plug and clear on unplug of a CPU.
Do we have CPU unplug on s390x ?
Quoted 20 lines
hw/s390x/cpu-topology.c new file +20
-0+* The firmware assume that all CPU in a CPU TLE have the same +* type, polarization and are all dedicated or shared. +* In the case a socket contains CPU with different type, polarization +* or entitlement then they will be defined in different CPU containers. +* Currently we assume all CPU are identical IFL CPUs and that they are +* all dedicated CPUs. +* The only reason to have several S390TopologyCores inside a socket is +* to have more than 64 CPUs. +* In that case the origin field, representing the offset of the first CPU +* in the CPU container allows to represent up to the maximal number of +* CPU inside several CPU containers inside the socket container. +*/ + topo->socket[socket_id].active_count++; + topo->tle[socket_id].active_count++; +set_bit (bit, &topo->tle[socket_id].mask[origin]); +} + +/** +* s390_topology_realize: +* @dev: the device state
hw/s390x/cpu-topology.c line 96
Quoted 10 lines
hw/s390x/cpu-topology.c new file +10
-0+* @errp: the error pointer (not used) +* +* During realize the machine CPU topology is initialized with the +* QEMU -smp parameters. +* The maximum count of CPU TLE in the all Topology can not be greater +* than the maximum CPUs. +*/ +static void s390_topology_realize (DeviceState *dev,Error **errp) +{ +MachineState *ms =MACHINE (qdev_get_machine ());
Using qdev_get_machine() is suspicious :)
hw/s390x/cpu-topology.c line 102
Quoted 6 lines
hw/s390x/cpu-topology.c new file +6
-0+S390Topology *topo =S390_CPU_TOPOLOGY (dev); +int n; + + topo->sockets = ms->smp.sockets; + topo->cores = ms->smp.cores; + topo->tles = ms->smp.max_cpus;
These look like object properties to me.
Quoted 10 lines
hw/s390x/cpu-topology.c new file +10
-0+ + n = topo->sockets; + topo->socket =g_malloc0 (n *sizeof (S390TopoContainer)); + topo->tle =g_malloc0 (topo->tles *sizeof (S390TopoTLE)); +} + +/** +* topology_class_init: +* @oc: Object class +* @data: (not used)
hw/s390x/cpu-topology.c line 122
Quoted 10 lines
hw/s390x/cpu-topology.c new file +10
-0+* +* A very simple object we will need for reset and migration. +*/ +static void topology_class_init (ObjectClass *oc,void *data) +{ +DeviceClass *dc =DEVICE_CLASS (oc); + + dc->realize = s390_topology_realize; +set_bit (DEVICE_CATEGORY_MISC , dc->categories); +}
no vmstate ?
Quoted 37 lines
hw/s390x/cpu-topology.c new file +12
-0+static const TypeInfo cpu_topology_info = { + .name =TYPE_S390_CPU_TOPOLOGY , + .parent =TYPE_SYS_BUS_DEVICE , + .instance_size =sizeof (S390Topology), + .class_init = topology_class_init, +}; + +static void topology_register (void ) +{ +type_register_static (&cpu_topology_info); +} +type_init (topology_register);hw/s390x/meson.build +1
-0@@ -2,6 +2,7 @@ s390x_ss = ss.source_set() s390x_ss.add(files( 'ap-bridge.c', 'ap-device.c', + 'cpu-topology.c', 'ccw-device.c', 'css-bridge.c', 'css.c',hw/s390x/s390-virtio-ccw.c +1
-0@@ -43,6 +43,7 @@#include "sysemu/sysemu.h" #include "hw/s390x/pv.h" #include "migration/blocker.h" +#include "hw/s390x/cpu-topology.h" static Error *pv_mig_blocker;
hw/s390x/s390-virtio-ccw.c static void ccw_init(MachineState *machine) · line 253
Quoted 6 lines
hw/s390x/s390-virtio-ccw.c +3
-0@@ -247,6 +248,12 @@ static void ccw_init(MachineState *machine)/* init memory + setup max page size. Required for the CPU model */ s390_memory_init (machine->ram); +/* Adding the topology must be done before CPU intialization*/ + dev =qdev_new (TYPE_S390_CPU_TOPOLOGY ); + object_property_add_child(qdev_get_machine(), TYPE_S390_CPU_TOPOLOGY,
No need to use qdev_get_machine(), you have 'machine' above.
hw/s390x/s390-virtio-ccw.c static void ccw_init(MachineState *machine) · line 255
Quoted 2 lines
hw/s390x/s390-virtio-ccw.c +2
-0+ OBJECT(dev)); +sysbus_realize_and_unref (SYS_BUS_DEVICE (dev), &error_fatal);
why not store a TYPE_S390_CPU_TOPOLOGY object pointer under the machine state for later use ?
hw/s390x/s390-virtio-ccw.c static void s390_cpu_plug(HotplugHandler *hotplug_dev, · line 321
Quoted 11 lines
hw/s390x/s390-virtio-ccw.c +4
-0+/* init CPUs (incl. CPU model) early so s390_has_feature() works */ s390_init_cpus (machine);@@ -309,6 +316,9 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev,g_assert (!ms->possible_cpus->cpus[cpu->env.core_id].cpu); ms->possible_cpus->cpus[cpu->env.core_id].cpu =OBJECT (dev); +/* Inserting the CPU in the Topology can not fail */ +s390_topology_new_cpu (cpu->env.core_id); +
in which case, we could use the topo object pointer to insert a new CPU id and drop s390_get_topology() which looks overkill.
I would add the test :
if (!S390_CCW_MACHINE(machine)->topology_disable) {before inserting to be consistent. But I am anticipating some other patch.
C.
Quoted 51 lines
hw/s390x/s390-virtio-ccw.c +0
-0if (dev->hotplugged) {raise_irq_cpu_hotplug (); }include/hw/s390x/cpu-topology.h new file +42
-0@@ -0,0 +1,42 @@ +/* +* CPU Topology +* +* Copyright 2022 IBM Corp. +* +* This work is licensed under the terms of the GNU GPL, version 2 or (at +* your option) any later version. See the COPYING file in the top-level +* directory. +*/ +#ifndef HW_S390X_CPU_TOPOLOGY_H +#define HW_S390X_CPU_TOPOLOGY_H + +typedef struct S390TopoContainer { +int active_count; +}S390TopoContainer ; + +#define S390_TOPOLOGY_MAX_ORIGIN (1 + S390_MAX_CPUS / 64) +typedef struct S390TopoTLE { +int active_count; +uint64_t mask[S390_TOPOLOGY_MAX_ORIGIN ]; +}S390TopoTLE ; + +#include "hw/qdev-core.h" +#include "qom/object.h" + +struct S390Topology { +SysBusDevice parent_obj; +int sockets; +int cores; +int tles; +S390TopoContainer *socket; +S390TopoTLE *tle; +}; +typedef struct S390Topology S390Topology ; + +#define TYPE_S390_CPU_TOPOLOGY "s390-topology" +OBJECT_DECLARE_SIMPLE_TYPE(S390Topology, S390_CPU_TOPOLOGY) + +S390Topology *s390_get_topology (void ); +void s390_topology_new_cpu (int core_id); + +#endif
Re: [PATCH v9 02/10] s390x/cpu topology: core_id sets s390x CPU topology
Quoted 84 lines · Cédric Le Goater
Quoted 83 lines · Pierre Morel
In the S390x CPU topology the
core_idspecifies the CPU address and the position of the core withing the topology.Let's build the topology based on the
core_id.Signed-off-by Pierre Morel <[email protected]>---
hw/s390x/cpu-topology.c | 135 ++++++++++++++++++++++++++++++++ hw/s390x/meson.build | 1 + hw/s390x/s390-virtio-ccw.c | 10 +++include/hw/s390x/cpu-topology.h| 42 ++++++++++4 files changed, 188 insertions(+)create mode 100644
hw/s390x/cpu-topology.ccreate mode 100644include/hw/s390x/cpu-topology.hhw/s390x/cpu-topology.c new file +7
-0@@ -0,0 +1,135 @@ +/* +* CPU Topology +* +* Copyright IBM Corp. 2022 +* Author(s): Pierre Morel <[email protected]> + +* This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */hw/s390x/cpu-topology.c new file +33
-0+ +#include "qemu/osdep.h" +#include "qapi/error.h" +#include "qemu/error-report.h" +#include "hw/sysbus.h" +#include "hw/qdev-properties.h" +#include "hw/boards.h" +#include "qemu/typedefs.h" +#include "target/s390x/cpu.h" +#include "hw/s390x/s390-virtio-ccw.h" +#include "hw/s390x/cpu-topology.h" + +S390Topology *s390_get_topology (void ) +{ + static S390Topology *s390Topology; + + if (!s390Topology) { + s390Topology = S390_CPU_TOPOLOGY( + object_resolve_path(TYPE_S390_CPU_TOPOLOGY, NULL)); + } + + return s390Topology; +} + +/* +* s390_topology_new_cpu: +* @core_id: the core ID is machine wide +* +* The topology returned by s390_get_topology(), gives us the CPU +* topology established by the -smp QEMU aruments. +* The core-id gives: +* - the Container TLE (Topology List Entry) containing the CPU TLE. +* - in the CPU TLE the origin, or offset of the first bit in the core mask
+ * - the bit in the CPU TLE core mask
+ */hw/s390x/cpu-topology.c new file +15
-0+void s390_topology_new_cpu (int core_id) +{ + S390Topology *topo = s390_get_topology(); + int socket_id; + int bit, origin; + + /* In the case no Topology is used nothing is to be done here */ + if (!topo) { + return; + } + + socket_id = core_id / topo->cores; + + bit = core_id; + origin = bit / 64;
Quoted 8 lines
Quoted 6 lines
+ bit %= 64;
+ bit = 63 - bit;
+
+ /*
+ * At the core level, each CPU is represented by a bit in a 64bit + * unsigned long. Set on plug and clear on unplug of a CPU.Do we have CPU unplug on s390x ?
not in QEMU.
I will change this sentence.
Quoted 31 lines
Quoted 31 lines
+ * The firmware assume that all CPU in a CPU TLE have the same + * type, polarization and are all dedicated or shared. + * In the case a socket contains CPU with different type, polarization
+ * or entitlement then they will be defined in different CPU containers.
+ * Currently we assume all CPU are identical IFL CPUs and that they are
+ * all dedicated CPUs.
+ * The only reason to have several S390TopologyCores inside a socket is
+ * to have more than 64 CPUs.
+ * In that case the origin field, representing the offset of the first CPU
+ * in the CPU container allows to represent up to the maximal number of
+ * CPU inside several CPU containers inside the socket container. + */
+ topo->socket[socket_id].active_count++;
+ topo->tle[socket_id].active_count++;
+set_bit(bit, &topo->tle[socket_id].mask[origin]);hw/s390x/cpu-topology.c new file +9
-0+} + +/** +* s390_topology_realize: +* @dev: the device state +* @errp: the error pointer (not used) +* +* During realize the machine CPU topology is initialized with the +* QEMU -smp parameters.
Quoted 8 lines
hw/s390x/cpu-topology.c line 96
Quoted 6 lines
+ * The maximum count of CPU TLE in the all Topology can not be greater
+ * than the maximum CPUs.
+ */hw/s390x/cpu-topology.c new file +3
-0+static void s390_topology_realize (DeviceState *dev,Error **errp) +{ + MachineState *ms = MACHINE(qdev_get_machine());Using
qdev_get_machine()is suspicious :)
OK
Quoted 9 lines
Quoted 7 lines
+ S390Topology *topo =
S390_CPU_TOPOLOGY(dev);
+ int n;
+
+topo->sockets=ms->smp.sockets;
+topo->cores=ms->smp.cores;
+topo->tles=ms->smp.max_cpus;These look like object properties to me.
It is a temporary store to keep them at hand.
I will see if I keep them afterall.
If I keep them I will make them property.
Quoted 15 lines
Quoted 15 lines
hw/s390x/cpu-topology.c new file +14
-0+ + n = topo->sockets; + topo->socket = g_malloc0(n * sizeof(S390TopoContainer)); + topo->tle = g_malloc0(topo->tles * sizeof(S390TopoTLE)); +} + +/** +* topology_class_init: +* @oc: Object class +* @data: (not used) +* +* A very simple object we will need for reset and migration. +*/ +static void topology_class_init (ObjectClass *oc,void *data)
Quoted 8 lines
hw/s390x/cpu-topology.c line 122
Quoted 6 lines
hw/s390x/cpu-topology.c new file +6
-0+{ + DeviceClass *dc = DEVICE_CLASS(oc); + + dc->realize = s390_topology_realize; + set_bit(DEVICE_CATEGORY_MISC, dc->categories); +}no vmstate ?
it is added in a later patch
Quoted 36 lines
Quoted 36 lines
hw/s390x/cpu-topology.c new file +12
-0+static const TypeInfo cpu_topology_info = { + .name = TYPE_S390_CPU_TOPOLOGY, + .parent = TYPE_SYS_BUS_DEVICE, + .instance_size = sizeof(S390Topology), + .class_init = topology_class_init, +}; + +static void topology_register (void ) +{ + type_register_static(&cpu_topology_info); +} +type_init (topology_register);hw/s390x/meson.build +0
-0@@ -2,6 +2,7 @@ s390x_ss = ss.source_set()s390x_ss.add(files(
'ap-bridge.c',
'ap-device.c',
+ 'cpu-topology.c',
'ccw-device.c',
'css-bridge.c',
'css.c',hw/s390x/s390-virtio-ccw.c +1
-0@@ -43,6 +43,7 @@ #include "sysemu/sysemu.h" #include "hw/s390x/pv.h" #include "migration/blocker.h" +#include "hw/s390x/cpu-topology.h" static Error *pv_mig_blocker;@@ -247,6 +248,12 @@ static void ccw_init(MachineState *machine)
Quoted 8 lines
hw/s390x/s390-virtio-ccw.c static void ccw_init(MachineState *machine)
Quoted 6 lines
hw/s390x/s390-virtio-ccw.c +3
-0/* init memory + setup max page size. Required for the CPU model */ s390_memory_init(machine->ram); + /* Adding the topology must be done before CPU intialization*/ + dev = qdev_new(TYPE_S390_CPU_TOPOLOGY); + object_property_add_child(qdev_get_machine(), TYPE_S390_CPU_TOPOLOGY,No need to use
qdev_get_machine(), you have 'machine' above.
OK
Quoted 7 lines
hw/s390x/s390-virtio-ccw.c static void ccw_init(MachineState *machine) · line 255
Quoted 3 lines
hw/s390x/s390-virtio-ccw.c +2
-0+ OBJECT(dev)); + sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal);why not store a
TYPE_S390_CPU_TOPOLOGYobject pointer under the machine state for later use ?
May be, I will think about it.
Quoted 15 lines
hw/s390x/s390-virtio-ccw.c static void s390_cpu_plug(HotplugHandler · line 321
Quoted 11 lines
hw/s390x/s390-virtio-ccw.c +4
-0+ /* init CPUs (incl. CPU model) early so s390_has_feature() works */ s390_init_cpus(machine);@@ -309,6 +316,9 @@ static void s390_cpu_plug(HotplugHandler *hotplug_dev, g_assert(!ms->possible_cpus->cpus[cpu->env.core_id].cpu); ms->possible_cpus->cpus[cpu->env.core_id].cpu = OBJECT(dev); + /* Inserting the CPU in the Topology can not fail */ + s390_topology_new_cpu(cpu->env.core_id); +in which case, we could use the topo object pointer to insert a new CPU id and drop
s390_get_topology()which looks overkill.
Quoted 6 lines
I would add the test :
if (!S390_CCW_MACHINE(machine)->topology_disable) {
before inserting to be consistent. But I am anticipating some other patch.
It belongs here for bisect so I will add it here.
Thanks.
Quoted 4 lines
C.
Thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
PATCH v9 3/10 s390x/cpu topology: reporting the CPU topology to the guest
In reply to Pierre MorelThe guest can use the STSI instruction to get a buffer filled with the CPU topology description.
Let us implement the STSI instruction for the basis CPU topology level, level 2.
--- hw/s390x/cpu-topology.c | 4 ++ include/hw/s390x/cpu-topology.h | 5 ++ target/s390x/cpu.h | 49 +++++++++++++++ target/s390x/cpu_topology.c | 108 ++++++++++++++++++++++++++++++++ target/s390x/kvm/kvm.c | 6 +- target/s390x/meson.build | 1 + 6 files changed, 172 insertions(+), 1 deletion(-) create mode 100644 target/s390x/cpu_topology.c
hw/s390x/cpu-topology.c +4 -0
@@ -76,9 +76,11 @@ void s390_topology_new_cpu(int core_id)* in the CPU container allows to represent up to the maximal number of * CPU inside several CPU containers inside the socket container. */ +qemu_mutex_lock (&topo->topo_mutex); topo->socket[socket_id].active_count++; topo->tle[socket_id].active_count++;set_bit (bit, &topo->tle[socket_id].mask[origin]); +qemu_mutex_unlock (&topo->topo_mutex); }/**
@@ -104,6 +106,8 @@ static void s390_topology_realize(DeviceState *dev, Error **errp) n = topo->sockets; topo->socket =g_malloc0 (n *sizeof (S390TopoContainer)); topo->tle =g_malloc0 (topo->tles *sizeof (S390TopoTLE)); + +qemu_mutex_init (&topo->topo_mutex); }/**
include/hw/s390x/cpu-topology.h +5 -0
@@ -10,6 +10,10 @@#ifndef HW_S390X_CPU_TOPOLOGY_H #define HW_S390X_CPU_TOPOLOGY_H +#define S390_TOPOLOGY_CPU_TYPE 0x03 + +#define S390_TOPOLOGY_POLARITY_H 0x00 +typedef struct S390TopoContainer {int active_count; }S390TopoContainer ;
@@ -30,6 +34,7 @@ struct S390Topology {int tles;S390TopoContainer *socket;S390TopoTLE *tle; +QemuMutex topo_mutex; };typedef struct S390Topology S390Topology ;
target/s390x/cpu.h +49 -0
59 lines not on this page · raw message
target/s390x/cpu_topology.c new file +108 -0
108 lines not on this page · raw message
target/s390x/kvm/kvm.c +5 -1
@@ -51,6 +51,7 @@#include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/s390-virtio-hcall.h" #include "hw/s390x/pv.h" +#include "hw/s390x/cpu-topology.h" #ifndef DEBUG_KVM #define DEBUG_KVM 0
@@ -1917,9 +1918,12 @@ static int handle_stsi(S390CPU *cpu)if (run->s390_stsi.sel1 !=2 || run->s390_stsi.sel2 !=2 ) {return 0 ; } -/* Only sysib 3.2.2 needs post-handling for now. */ insert_stsi_3_2_2 (cpu, run->s390_stsi.addr, run->s390_stsi.ar);return 0 ; +case 15 : +insert_stsi_15_1_x (cpu, run->s390_stsi.sel2, run->s390_stsi.addr, + run->s390_stsi.ar); +return 0 ;default :return 0 ; }
target/s390x/meson.build +1 -0
@@ -29,6 +29,7 @@ s390x_softmmu_ss.add(files( 'sigp.c', 'cpu-sysemu.c', 'cpu_models_sysemu.c', + 'cpu_topology.c', )) s390x_user_ss = ss.source_set()
Signature · 1 line
2.31.1
Re: [PATCH v9 03/10] s390x/cpu topology: reporting the CPU topology to the guest
Quoting Pierre Morel (2022-09-02 09:55:24)
Quoted 5 lines
The guest can use the STSI instruction to get a buffer filled with the CPU topology description.
Let us implement the STSI instruction for the basis CPU topology level, level 2.
I like this. It is so much simpler. Thanks.
[...]
hw/s390x/cpu-topology.c void s390_topology_new_cpu(int core_id) · line 79
Quoted 9 lines
hw/s390x/cpu-topology.c +1
-0@@ -76,9 +76,11 @@ void s390_topology_new_cpu(int core_id)* in the CPU container allows to represent up to the maximal number of * CPU inside several CPU containers inside the socket container. */ +qemu_mutex_lock (&topo->topo_mutex);
You access topo->cores above. Do you need the mutex for that? I guess not since it can't change at runtime (right?), so maybe it is worth documenting what the topo_mutex actually protects or you just take the mutex at the start of the function.
[...]
Quoted 5 lines
target/s390x/cpu_topology.c new file +0
-0
[...]
target/s390x/cpu_topology.c line 35
Quoted 9 lines
target/s390x/cpu_topology.c new file +9
-0+static char *fill_tle_cpu (char *p,uint64_t mask,int origin) +{ +SysIBTl_cpu *tle = (SysIBTl_cpu *)p; + + tle->nl =0 ; + tle->dedicated =1 ; + tle->polarity =S390_TOPOLOGY_POLARITY_H ; + tle->type =S390_TOPOLOGY_CPU_TYPE ; + tle->origin = origin *64 ;
origin would also need a byte order conversion.
Quoted 1 line
+
tle->mask=be64_to_cpu(mask);
cpu_to_be64()
[...]
target/s390x/cpu_topology.c line 52
Quoted 13 lines
target/s390x/cpu_topology.c new file +13
-0+static char *s390_top_set_level2 (S390Topology *topo,char *p) +{ +int i, origin; + +for (i =0 ; i < topo->sockets; i++) { +if (!topo->socket[i].active_count) { +continue ; + } + p =fill_container (p,1 , i); +for (origin =0 ; origin <S390_TOPOLOGY_MAX_ORIGIN ; origin++) { +uint64_t mask =0L ; + + mask =be64_to_cpu (topo->tle[i].mask[origin]);
Don't you already do the endianness conversion in fill_tle_cpu()?
[...]
Quoted 7 lines
target/s390x/cpu_topology.c new file +7
-0+void insert_stsi_15_1_x (S390CPU *cpu,int sel2,__u64 addr,uint8_t ar) +{ +SysIB_151x *sysib; +int len =sizeof (*sysib); + +if (s390_is_pv () || sel2 <2 || sel2 >S390_TOPOLOGY_MAX_MNEST ) { +setcc (cpu,3 );
target/s390x/cpu_topology.c line 99
Quoted 10 lines
target/s390x/cpu_topology.c new file +10
-0+return ; + } + + sysib =g_malloc0 (TARGET_PAGE_SIZE ); + + len +=setup_stsi (sysib, sel2); +if (len >TARGET_PAGE_SIZE ) { +setcc (cpu,3 ); + goto out_free; + }
Maybe I don't get it, but isn't it kind of late for this check? You would already have written beyond the end of the buffer at this point in time...
target/s390x/cpu_topology.c
Quoted 2 lines
target/s390x/cpu_topology.c new file +2
-0+ + sysib->length =be16_to_cpu (len);
cpu_to_be16()
Re: [PATCH v9 03/10] s390x/cpu topology: reporting the CPU topology to the guest
Quoted 10 lines · Nico Boehr
Quoting Pierre Morel (2022-09-02 09:55:24)
Quoted 5 lines
The guest can use the STSI instruction to get a buffer filled with the CPU topology description.
Let us implement the STSI instruction for the basis CPU topology level, level 2.
I like this. It is so much simpler. Thanks.
[...]
Quoted 14 lines
hw/s390x/cpu-topology.c void s390_topology_new_cpu(int core_id) · line 79
Quoted 9 lines
hw/s390x/cpu-topology.c +1
-0@@ -76,9 +76,11 @@ void s390_topology_new_cpu(int core_id)* in the CPU container allows to represent up to the maximal number of * CPU inside several CPU containers inside the socket container. */ +qemu_mutex_lock (&topo->topo_mutex);You access
topo->coresabove. Do you need the mutex for that? I guess not since it can't change at runtime (right?), so maybe it is worth documenting what thetopo_mutexactually protects or you just take the mutex at the start of the function.
You are right one should always do that.
I will add this.
Quoted 8 lines
[...]
Quoted 5 lines
target/s390x/cpu_topology.c new file +0
-0[...]
Quoted 11 lines
target/s390x/cpu_topology.c line 35
Quoted 9 lines
target/s390x/cpu_topology.c new file +9
-0+static char *fill_tle_cpu (char *p,uint64_t mask,int origin) +{ +SysIBTl_cpu *tle = (SysIBTl_cpu *)p; + + tle->nl =0 ; + tle->dedicated =1 ; + tle->polarity =S390_TOPOLOGY_POLARITY_H ; + tle->type =S390_TOPOLOGY_CPU_TYPE ; + tle->origin = origin *64 ;origin would also need a byte order conversion.
yes
Quoted 4 lines
Quoted 2 lines
+
tle->mask=be64_to_cpu(mask);
cpu_to_be64()
yes
Quoted 17 lines
[...]
target/s390x/cpu_topology.c line 52
Quoted 13 lines
target/s390x/cpu_topology.c new file +13
-0+static char *s390_top_set_level2 (S390Topology *topo,char *p) +{ +int i, origin; + +for (i =0 ; i < topo->sockets; i++) { +if (!topo->socket[i].active_count) { +continue ; + } + p =fill_container (p,1 , i); +for (origin =0 ; origin <S390_TOPOLOGY_MAX_ORIGIN ; origin++) { +uint64_t mask =0L ; + + mask =be64_to_cpu (topo->tle[i].mask[origin]);Don't you already do the endianness conversion in
fill_tle_cpu()?
yes
Quoted 14 lines
[...]
Quoted 12 lines
target/s390x/cpu_topology.c new file +12
-0+void insert_stsi_15_1_x (S390CPU *cpu,int sel2,__u64 addr,uint8_t ar) +{ +SysIB_151x *sysib; +int len =sizeof (*sysib); + +if (s390_is_pv () || sel2 <2 || sel2 >S390_TOPOLOGY_MAX_MNEST ) { +setcc (cpu,3 ); +return ; + } + + sysib =g_malloc0 (TARGET_PAGE_SIZE ); +
Quoted 8 lines
target/s390x/cpu_topology.c line 99
Quoted 5 lines
target/s390x/cpu_topology.c new file +5
-0+ len +=setup_stsi (sysib, sel2); +if (len >TARGET_PAGE_SIZE ) { +setcc (cpu,3 ); + goto out_free; + }Maybe I don't get it, but isn't it kind of late for this check? You would already have written beyond the end of the buffer at this point in time...
it is
Thanks for your comments.
regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v9 03/10] s390x/cpu topology: reporting the CPU topology to the guest
In reply to Pierre MorelQuoted 22 lines · Pierre Morel
The guest can use the STSI instruction to get a buffer filled with the CPU topology description.
Let us implement the STSI instruction for the basis CPU topology level, level 2.
Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/cpu-topology.c | 4 ++ include/hw/s390x/cpu-topology.h | 5 ++ target/s390x/cpu.h | 49 +++++++++++++++ target/s390x/cpu_topology.c | 108 ++++++++++++++++++++++++++++++++ target/s390x/kvm/kvm.c | 6 +- target/s390x/meson.build | 1 + 6 files changed, 172 insertions(+), 1 deletion(-) create mode 100644 target/s390x/cpu_topology.chw/s390x/cpu-topology.c +0
-0
hw/s390x/cpu-topology.c void s390_topology_new_cpu(int core_id) · line 79
Quoted 4 lines
hw/s390x/cpu-topology.c +1
-0@@ -76,9 +76,11 @@ void s390_topology_new_cpu(int core_id)* in the CPU container allows to represent up to the maximal number of * CPU inside several CPU containers inside the socket container. */ +qemu_mutex_lock (&topo->topo_mutex);
You could use a reader writer lock for this, if qemu has that (I didn't find any tho).
Quoted 21 lines
hw/s390x/cpu-topology.c +3
-0topo->socket[socket_id].active_count++; topo->tle[socket_id].active_count++;set_bit (bit, &topo->tle[socket_id].mask[origin]); +qemu_mutex_unlock (&topo->topo_mutex); }/** @@ -104,6 +106,8 @@ static void s390_topology_realize(DeviceState *dev, Error **errp) n = topo->sockets; topo->socket =g_malloc0 (n *sizeof (S390TopoContainer)); topo->tle =g_malloc0 (topo->tles *sizeof (S390TopoTLE)); + +qemu_mutex_init (&topo->topo_mutex); }/** include/hw/s390x/cpu-topology.h +0
-0
include/hw/s390x/cpu-topology.h line 13
Quoted 4 lines
include/hw/s390x/cpu-topology.h +1
-0@@ -10,6 +10,10 @@#ifndef HW_S390X_CPU_TOPOLOGY_H #define HW_S390X_CPU_TOPOLOGY_H +#define S390_TOPOLOGY_CPU_TYPE 0x03
IMO you should add the name of cpu type 0x03 to the name of the constant, even if there is only one right now.
You did the same for the polarity after all.
Quoted 39 lines
include/hw/s390x/cpu-topology.h +4
-0+ +#define S390_TOPOLOGY_POLARITY_H 0x00 +typedef struct S390TopoContainer {int active_count; }S390TopoContainer ;@@ -30,6 +34,7 @@ struct S390Topology {int tles;S390TopoContainer *socket;S390TopoTLE *tle; +QemuMutex topo_mutex; };typedef struct S390Topology S390Topology ;target/s390x/cpu.h +17
-0@@ -565,6 +565,53 @@ typedef union SysIB { } SysIB;QEMU_BUILD_BUG_ON (sizeof (SysIB) !=4096 ); +/* CPU type Topology List Entry */ +typedef struct SysIBTl_cpu { +uint8_t nl; +uint8_t reserved0[3 ]; +uint8_t reserved1:5 ; +uint8_t dedicated:1 ; +uint8_t polarity:2 ; +uint8_t type; +uint16_t origin; +uint64_t mask; +}SysIBTl_cpu ; +QEMU_BUILD_BUG_ON (sizeof (SysIBTl_cpu) !=16 ); + +/* Container type Topology List Entry */ +typedef struct SysIBTl_container { + uint8_t nl; + uint8_t reserved[6];
target/s390x/cpu.h typedef union SysIB { · line 594
Quoted 10 lines
target/s390x/cpu.h +10
-0+uint8_t id; +}QEMU_PACKED SysIBTl_container; +QEMU_BUILD_BUG_ON (sizeof (SysIBTl_container) !=8 ); + +/* Generic Topology List Entry */ +typedef union SysIBTl_entry { +uint8_t nl; +SysIBTl_container container; +SysIBTl_cpu cpu; +}SysIBTl_entry ;
This isn't used for anything but the declaration in SysIB_151x, is it?
Quoted 7 lines
target/s390x/cpu.h +7
-0+ +#define TOPOLOGY_NR_MAG 6 +#define TOPOLOGY_NR_MAG6 0 +#define TOPOLOGY_NR_MAG5 1 +#define TOPOLOGY_NR_MAG4 2 +#define TOPOLOGY_NR_MAG3 3 +#define TOPOLOGY_NR_MAG2 4
target/s390x/cpu.h typedef union SysIB { · line 611
Quoted 10 lines
target/s390x/cpu.h +10
-0+#define TOPOLOGY_NR_MAG1 5 +/* Configuration topology */ +typedef struct SysIB_151x { + uint8_t reserved0[2]; + uint16_t length; + uint8_t mag[TOPOLOGY_NR_MAG]; + uint8_t reserved1; + uint8_t mnest; + uint32_t reserved2; + SysIBTl_entry tle[0];
I would just use uint64_t[0] as type or uint64_t[] whichever is qemu style.
Quoted 45 lines
target/s390x/cpu.h +5
-0+} SysIB_151x; +QEMU_BUILD_BUG_ON (sizeof (SysIB_151x) !=16 ); +/* MMU defines */ #define ASCE_ORIGIN (~0xfffULL)/* segment table origin */ #define ASCE_SUBSPACE 0x200/* subspace group control */ @@ -843,4 +890,6 @@ S390CPU *s390_cpu_addr2state(uint16_t cpu_addr);#include "exec/cpu-all.h" +void insert_stsi_15_1_x (S390CPU *cpu,int sel2,__u64 addr,uint8_t ar); +#endif target/s390x/cpu_topology.c new file +26
-0@@ -0,0 +1,108 @@ +/* +* QEMU S390x CPU Topology +* +* Copyright IBM Corp. 2022 +* Author(s): Pierre Morel <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or (at +* your option) any later version. See the COPYING file in the top-level +* directory. +*/ +#include "qemu/osdep.h" +#include "cpu.h" +#include "hw/s390x/pv.h" +#include "hw/sysbus.h" +#include "hw/s390x/cpu-topology.h" +#include "hw/s390x/sclp.h" + +static char *fill_container (char *p,int level,int id) +{ +SysIBTl_container *tle = (SysIBTl_container *)p; + + tle->nl = level; + tle->id = id; +return p +sizeof (*tle); +} +
target/s390x/cpu_topology.c line 36
Quoted 10 lines
target/s390x/cpu_topology.c new file +10
-0+static char *fill_tle_cpu (char *p,uint64_t mask,int origin) +{ +SysIBTl_cpu *tle = (SysIBTl_cpu *)p; + + tle->nl =0 ; + tle->dedicated =1 ; + tle->polarity =S390_TOPOLOGY_POLARITY_H ; + tle->type =S390_TOPOLOGY_CPU_TYPE ; + tle->origin = origin *64 ; + tle->mask =be64_to_cpu (mask);
You convert endianess for mask here...
Quoted 6 lines
target/s390x/cpu_topology.c new file +6
-0+return p +sizeof (*tle); +} + +static char *s390_top_set_level2 (S390Topology *topo,char *p) +{ +int i, origin;
target/s390x/cpu_topology.c line 52
Quoted 10 lines
target/s390x/cpu_topology.c new file +10
-0+ +for (i =0 ; i < topo->sockets; i++) { +if (!topo->socket[i].active_count) { +continue ; + } + p =fill_container (p,1 , i); +for (origin =0 ; origin <S390_TOPOLOGY_MAX_ORIGIN ; origin++) { +uint64_t mask =0L ; + + mask =be64_to_cpu (topo->tle[i].mask[origin]);
...and here. So one has to go, I guess this one. Also using cpu_to_be64 seems more intuitive to me.
Quoted 31 lines
target/s390x/cpu_topology.c new file +31
-0+if (mask) { + p =fill_tle_cpu (p, mask, origin); + } + } + } +return p; +} + +static int setup_stsi (SysIB_151x *sysib,int level) +{ +S390Topology *topo =s390_get_topology (); +char *p = (char *)sysib->tle; + +qemu_mutex_lock (&topo->topo_mutex); + + sysib->mnest = level; +switch (level) { +case 2 : + sysib->mag[TOPOLOGY_NR_MAG2 ] = topo->sockets; + sysib->mag[TOPOLOGY_NR_MAG1 ] = topo->cores; + p =s390_top_set_level2 (topo, p); +break ; + } + +qemu_mutex_unlock (&topo->topo_mutex); + +return p - (char *)sysib->tle; +} + +#define S390_TOPOLOGY_MAX_MNEST 2 +void insert_stsi_15_1_x (S390CPU *cpu,int sel2,__u64 addr,uint8_t ar)
target/s390x/cpu_topology.c line 93
Quoted 10 lines
target/s390x/cpu_topology.c new file +10
-0+{ +SysIB_151x *sysib; +int len =sizeof (*sysib); + +if (s390_is_pv () || sel2 <2 || sel2 >S390_TOPOLOGY_MAX_MNEST ) { +setcc (cpu,3 ); +return ; + } + + sysib =g_malloc0 (TARGET_PAGE_SIZE );
What made you decide against stack allocating this?
target/s390x/cpu_topology.c line 96
Quoted 3 lines
target/s390x/cpu_topology.c new file +3
-0+ + len +=setup_stsi (sysib, sel2); +if (len >TARGET_PAGE_SIZE ) {
If you do the check here it's too late.
target/s390x/cpu_topology.c line 102
Quoted 6 lines
target/s390x/cpu_topology.c new file +6
-0+setcc (cpu,3 ); + goto out_free; + } + + sysib->length =be16_to_cpu (len); +s390_cpu_virt_mem_write (cpu, addr, ar, sysib, len);
If the return value of this is <0 it's an error condition. If you ignore the value we'll keep running.
target/s390x/cpu_topology.c line 103
Quoted 1 line
target/s390x/cpu_topology.c new file +1
-0+setcc (cpu,0 );
Is it correct to set the cc value even if s390_cpu_virt_mem_write causes an exception?
Quoted 43 lines
target/s390x/cpu_topology.c new file +5
-0+ +out_free: +g_free (sysib); +} +target/s390x/kvm/kvm.c +5
-1@@ -51,6 +51,7 @@#include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/s390-virtio-hcall.h" #include "hw/s390x/pv.h" +#include "hw/s390x/cpu-topology.h" #ifndef DEBUG_KVM #define DEBUG_KVM 0@@ -1917,9 +1918,12 @@ static int handle_stsi(S390CPU *cpu)if (run->s390_stsi.sel1 !=2 || run->s390_stsi.sel2 !=2 ) {return 0 ; } -/* Only sysib 3.2.2 needs post-handling for now. */ insert_stsi_3_2_2 (cpu, run->s390_stsi.addr, run->s390_stsi.ar);return 0 ; +case 15 : +insert_stsi_15_1_x (cpu, run->s390_stsi.sel2, run->s390_stsi.addr, + run->s390_stsi.ar); +return 0 ;default :return 0 ; }target/s390x/meson.build +1
-0@@ -29,6 +29,7 @@ s390x_softmmu_ss.add(files( 'sigp.c', 'cpu-sysemu.c', 'cpu_models_sysemu.c', + 'cpu_topology.c', )) s390x_user_ss = ss.source_set()
Re: [PATCH v9 03/10] s390x/cpu topology: reporting the CPU topology to the guest
Quoted 18 lines · Janis Schoetterl-Glausch
Quoted 17 lines · Pierre Morel
The guest can use the STSI instruction to get a buffer filled with the CPU topology description.
Let us implement the STSI instruction for the basis CPU topology level, level 2.
Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/cpu-topology.c | 4 ++ include/hw/s390x/cpu-topology.h | 5 ++ target/s390x/cpu.h | 49 +++++++++++++++ target/s390x/cpu_topology.c | 108 ++++++++++++++++++++++++++++++++ target/s390x/kvm/kvm.c | 6 +- target/s390x/meson.build | 1 + 6 files changed, 172 insertions(+), 1 deletion(-)create mode 100644
target/s390x/cpu_topology.c
Quoted 12 lines
hw/s390x/cpu-topology.c void s390_topology_new_cpu(int core_id) · line 79
Quoted 9 lines
hw/s390x/cpu-topology.c +1
-0@@ -76,9 +76,11 @@ void s390_topology_new_cpu(int core_id)* in the CPU container allows to represent up to the maximal number of * CPU inside several CPU containers inside the socket container. */ +qemu_mutex_lock (&topo->topo_mutex);You could use a reader writer lock for this, if qemu has that (I didn't find any tho).
I can use RCU but we could also consider that the write path is very short.
Quoted 25 lines
Quoted 25 lines
hw/s390x/cpu-topology.c +3
-0topo->socket[socket_id].active_count++; topo->tle[socket_id].active_count++;set_bit (bit, &topo->tle[socket_id].mask[origin]); +qemu_mutex_unlock (&topo->topo_mutex); }/** @@ -104,6 +106,8 @@ static void s390_topology_realize(DeviceState *dev, Error **errp) n = topo->sockets; topo->socket =g_malloc0 (n *sizeof (S390TopoContainer)); topo->tle =g_malloc0 (topo->tles *sizeof (S390TopoTLE)); + +qemu_mutex_init (&topo->topo_mutex); }/** include/hw/s390x/cpu-topology.h +0
-0@@ -10,6 +10,10 @@#ifndef HW_S390X_CPU_TOPOLOGY_H #define HW_S390X_CPU_TOPOLOGY_H
Quoted 5 lines
include/hw/s390x/cpu-topology.h line 13
Quoted 1 line
include/hw/s390x/cpu-topology.h +1
-0+#define S390_TOPOLOGY_CPU_TYPE 0x03IMO you should add the name of cpu type
0x03to the name of the constant, even if there is only one right now.
You did the same for the polarity after all.
OK right
#define S390_TOPOLOGY_CPU_IFL 0x03
Quoted 44 lines
Quoted 44 lines
include/hw/s390x/cpu-topology.h +4
-0+ +#define S390_TOPOLOGY_POLARITY_H 0x00 +typedef struct S390TopoContainer {int active_count; }S390TopoContainer ;@@ -30,6 +34,7 @@ struct S390Topology {int tles;S390TopoContainer *socket;S390TopoTLE *tle; +QemuMutex topo_mutex; };typedef struct S390Topology S390Topology ;target/s390x/cpu.h +21
-0@@ -565,6 +565,53 @@ typedef union SysIB { } SysIB;QEMU_BUILD_BUG_ON (sizeof (SysIB) !=4096 ); +/* CPU type Topology List Entry */ +typedef struct SysIBTl_cpu { +uint8_t nl; +uint8_t reserved0[3 ]; +uint8_t reserved1:5 ; +uint8_t dedicated:1 ; +uint8_t polarity:2 ; +uint8_t type; +uint16_t origin; +uint64_t mask; +}SysIBTl_cpu ; +QEMU_BUILD_BUG_ON (sizeof (SysIBTl_cpu) !=16 ); + +/* Container type Topology List Entry */ +typedef struct SysIBTl_container { +uint8_t nl; +uint8_t reserved[6 ]; +uint8_t id; +} QEMU_PACKED SysIBTl_container; +QEMU_BUILD_BUG_ON (sizeof (SysIBTl_container) !=8 ); +
Quoted 8 lines
target/s390x/cpu.h typedef union SysIB { · line 594
Quoted 6 lines
target/s390x/cpu.h +6
-0+/* Generic Topology List Entry */ +typedef union SysIBTl_entry { +uint8_t nl; +SysIBTl_container container; +SysIBTl_cpu cpu; +}SysIBTl_entry ;This isn't used for anything but the declaration in
SysIB_151x, is it?
right.
Quoted 12 lines
Quoted 12 lines
target/s390x/cpu.h +12
-0+ +#define TOPOLOGY_NR_MAG 6 +#define TOPOLOGY_NR_MAG6 0 +#define TOPOLOGY_NR_MAG5 1 +#define TOPOLOGY_NR_MAG4 2 +#define TOPOLOGY_NR_MAG3 3 +#define TOPOLOGY_NR_MAG2 4 +#define TOPOLOGY_NR_MAG1 5 +/* Configuration topology */ +typedef struct SysIB_151x { + uint8_t reserved0[2]; + uint16_t length;
Quoted 8 lines
target/s390x/cpu.h typedef union SysIB { · line 611
Quoted 5 lines
target/s390x/cpu.h +5
-0+uint8_t mag[TOPOLOGY_NR_MAG ]; +uint8_t reserved1; +uint8_t mnest; +uint32_t reserved2; +SysIBTl_entry tle[0 ];I would just use uint64_t[0] as type or uint64_t[] whichever is qemu style.
OK
Quoted 70 lines
Quoted 46 lines
target/s390x/cpu.h +5
-0+} SysIB_151x; +QEMU_BUILD_BUG_ON (sizeof (SysIB_151x) !=16 ); +/* MMU defines */ #define ASCE_ORIGIN (~0xfffULL)/* segment table origin */ #define ASCE_SUBSPACE 0x200/* subspace group control */ @@ -843,4 +890,6 @@ S390CPU *s390_cpu_addr2state(uint16_t cpu_addr);#include "exec/cpu-all.h" +void insert_stsi_15_1_x (S390CPU *cpu,int sel2,__u64 addr,uint8_t ar); +#endif target/s390x/cpu_topology.c new file +26
-0@@ -0,0 +1,108 @@ +/* +* QEMU S390x CPU Topology +* +* Copyright IBM Corp. 2022 +* Author(s): Pierre Morel <[email protected]> +* +* This work is licensed under the terms of the GNU GPL, version 2 or (at +* your option) any later version. See the COPYING file in the top-level +* directory. +*/ +#include "qemu/osdep.h" +#include "cpu.h" +#include "hw/s390x/pv.h" +#include "hw/sysbus.h" +#include "hw/s390x/cpu-topology.h" +#include "hw/s390x/sclp.h" + +static char *fill_container (char *p,int level,int id) +{ +SysIBTl_container *tle = (SysIBTl_container *)p; + + tle->nl = level; + tle->id = id; +return p +sizeof (*tle); +} +target/s390x/cpu_topology.c line 36
Quoted 10 lines
target/s390x/cpu_topology.c new file +10
-0+static char *fill_tle_cpu (char *p,uint64_t mask,int origin) +{ +SysIBTl_cpu *tle = (SysIBTl_cpu *)p; + + tle->nl =0 ; + tle->dedicated =1 ; + tle->polarity =S390_TOPOLOGY_POLARITY_H ; + tle->type =S390_TOPOLOGY_CPU_TYPE ; + tle->origin = origin *64 ; + tle->mask =be64_to_cpu (mask);You convert endianess for mask here...
Quoted 11 lines
target/s390x/cpu_topology.c new file +11
-0+return p +sizeof (*tle); +} + +static char *s390_top_set_level2 (S390Topology *topo,char *p) +{ +int i, origin; + +for (i =0 ; i < topo->sockets; i++) { +if (!topo->socket[i].active_count) { +continue ; + }
Quoted 8 lines
target/s390x/cpu_topology.c line 52
Quoted 5 lines
target/s390x/cpu_topology.c new file +5
-0+ p =fill_container (p,1 , i); +for (origin =0 ; origin <S390_TOPOLOGY_MAX_ORIGIN ; origin++) { +uint64_t mask =0L ; + + mask =be64_to_cpu (topo->tle[i].mask[origin]);...and here. So one has to go, I guess this one. Also using
cpu_to_be64seems more intuitive to me.
cpu_to_be64 is the right thing to do.
Also yes, I suppress this one.
Quoted 36 lines
Quoted 36 lines
target/s390x/cpu_topology.c new file +35
-0+if (mask) { + p =fill_tle_cpu (p, mask, origin); + } + } + } +return p; +} + +static int setup_stsi (SysIB_151x *sysib,int level) +{ +S390Topology *topo =s390_get_topology (); +char *p = (char *)sysib->tle; + +qemu_mutex_lock (&topo->topo_mutex); + + sysib->mnest = level; +switch (level) { +case 2 : + sysib->mag[TOPOLOGY_NR_MAG2 ] = topo->sockets; + sysib->mag[TOPOLOGY_NR_MAG1 ] = topo->cores; + p =s390_top_set_level2 (topo, p); +break ; + } + +qemu_mutex_unlock (&topo->topo_mutex); + +return p - (char *)sysib->tle; +} + +#define S390_TOPOLOGY_MAX_MNEST 2 +void insert_stsi_15_1_x (S390CPU *cpu,int sel2,__u64 addr,uint8_t ar) +{ +SysIB_151x *sysib; +int len =sizeof (*sysib); +
Quoted 8 lines
target/s390x/cpu_topology.c line 93
Quoted 6 lines
target/s390x/cpu_topology.c new file +6
-0+if (s390_is_pv () || sel2 <2 || sel2 >S390_TOPOLOGY_MAX_MNEST ) { +setcc (cpu,3 ); +return ; + } + + sysib =g_malloc0 (TARGET_PAGE_SIZE );What made you decide against stack allocating this?
I just did not think about it.
I will change it.
Quoted 6 lines
target/s390x/cpu_topology.c line 96
Quoted 4 lines
target/s390x/cpu_topology.c new file +3
-0+ + len +=setup_stsi (sysib, sel2); +if (len >TARGET_PAGE_SIZE ) {If you do the check here it's too late.
yes.
Quoted 10 lines
target/s390x/cpu_topology.c line 102
Quoted 7 lines
target/s390x/cpu_topology.c new file +6
-0+setcc (cpu,3 ); + goto out_free; + } + + sysib->length =be16_to_cpu (len); +s390_cpu_virt_mem_write (cpu, addr, ar, sysib, len);If the return value of this is <0 it's an error condition. If you ignore the value we'll keep running.
I understood that exceptions are handled inside s390_cpu_virt_mem_write() and we have no CC to report.
Quoted 5 lines
target/s390x/cpu_topology.c line 103
Quoted 2 lines
target/s390x/cpu_topology.c new file +1
-0+setcc (cpu,0 );Is it correct to set the cc value even if
s390_cpu_virt_mem_writecauses an exception?
I guess that if it caused an exception we do not get so far. Am I wrong?
Thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v9 03/10] s390x/cpu topology: reporting the CPU topology to the guest
In reply to Pierre MorelQuoted 17 lines · Pierre Morel
The guest can use the STSI instruction to get a buffer filled with the CPU topology description.
Let us implement the STSI instruction for the basis CPU topology level, level 2.
Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/cpu-topology.c | 4 ++ include/hw/s390x/cpu-topology.h | 5 ++ target/s390x/cpu.h | 49 +++++++++++++++ target/s390x/cpu_topology.c | 108 ++++++++++++++++++++++++++++++++ target/s390x/kvm/kvm.c | 6 +- target/s390x/meson.build | 1 + 6 files changed, 172 insertions(+), 1 deletion(-) create mode 100644 target/s390x/cpu_topology.c
[...]
Quoted 1 line
target/s390x/cpu_topology.c +0
-0
[...]
target/s390x/cpu_topology.c line 35
Quoted 9 lines
target/s390x/cpu_topology.c +9
-0+static char *fill_tle_cpu (char *p,uint64_t mask,int origin) +{ +SysIBTl_cpu *tle = (SysIBTl_cpu *)p; + + tle->nl =0 ; + tle->dedicated =1 ; + tle->polarity =S390_TOPOLOGY_POLARITY_H ; + tle->type =S390_TOPOLOGY_CPU_TYPE ; + tle->origin = origin *64 ;
origin is a multibyte field too, so needs a conversion too.
Quoted 4 lines
target/s390x/cpu_topology.c +4
-0+ tle->mask =be64_to_cpu (mask); +return p +sizeof (*tle); +} +
[...]
Re: [PATCH v9 03/10] s390x/cpu topology: reporting the CPU topology to the guest
Quoted 24 lines · Janis Schoetterl-Glausch
Quoted 17 lines · Pierre Morel
The guest can use the STSI instruction to get a buffer filled with the CPU topology description.
Let us implement the STSI instruction for the basis CPU topology level, level 2.
Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/cpu-topology.c | 4 ++ include/hw/s390x/cpu-topology.h | 5 ++ target/s390x/cpu.h | 49 +++++++++++++++ target/s390x/cpu_topology.c | 108 ++++++++++++++++++++++++++++++++ target/s390x/kvm/kvm.c | 6 +- target/s390x/meson.build | 1 + 6 files changed, 172 insertions(+), 1 deletion(-)create mode 100644
target/s390x/cpu_topology.c[...]
Quoted 1 line
target/s390x/cpu_topology.c +0
-0[...]
Quoted 11 lines
target/s390x/cpu_topology.c line 35
Quoted 9 lines
target/s390x/cpu_topology.c +9
-0+static char *fill_tle_cpu (char *p,uint64_t mask,int origin) +{ +SysIBTl_cpu *tle = (SysIBTl_cpu *)p; + + tle->nl =0 ; + tle->dedicated =1 ; + tle->polarity =S390_TOPOLOGY_POLARITY_H ; + tle->type =S390_TOPOLOGY_CPU_TYPE ; + tle->origin = origin *64 ;origin is a multibyte field too, so needs a conversion too.
right,
Thanks,
Pierre
Quoted 6 lines
Quoted 5 lines
target/s390x/cpu_topology.c +4
-0+ tle->mask =be64_to_cpu (mask); +return p +sizeof (*tle); +} +[...]
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
PATCH v9 4/10 hw/core: introducing drawer and books for s390x
In reply to Pierre MorelS390x defines two topology levels above sockets: nbooks and drawers.
Let's add these two levels inside the CPU topology implementation.
--- hw/core/machine-smp.c | 48 ++++++++++++++++++++++++++++++++++++------- hw/core/machine.c | 4 ++++ include/hw/boards.h | 8 ++++++++ qapi/machine.json | 14 +++++++++++-- qemu-options.hx | 6 ++++-- softmmu/vl.c | 6 ++++++ 6 files changed, 75 insertions(+), 11 deletions(-)
hw/core/machine-smp.c +41 -7
@@ -31,6 +31,14 @@ static char *cpu_hierarchy_to_string(MachineState *ms)MachineClass *mc =MACHINE_GET_CLASS (ms);GString *s =g_string_new (NULL ); +if (mc->smp_props.drawers_supported) { +g_string_append_printf (s," * drawers (%u)" , ms->smp.drawers); + } + +if (mc->smp_props.books_supported) { +g_string_append_printf (s," * books (%u)" , ms->smp.books); + } +g_string_append_printf (s,"sockets (%u)" , ms->smp.sockets);if (mc->smp_props.dies_supported) {
@@ -73,6 +81,8 @@ void machine_parse_smp_config(MachineState *ms, {MachineClass *mc =MACHINE_GET_CLASS (ms);unsigned cpus = config->has_cpus ? config->cpus :0 ; +unsigned drawers = config->has_drawers ? config->drawers :0 ; +unsigned books = config->has_books ? config->books :0 ;unsigned sockets = config->has_sockets ? config->sockets :0 ;unsigned dies = config->has_dies ? config->dies :0 ;unsigned clusters = config->has_clusters ? config->clusters :0 ;
@@ -85,6 +95,8 @@ void machine_parse_smp_config(MachineState *ms,* explicit configuration like "cpus=0" is not allowed. */ if ((config->has_cpus && config->cpus == 0) || + (config->has_drawers && config->drawers == 0) || + (config->has_books && config->books == 0) || (config->has_sockets && config->sockets == 0) || (config->has_dies && config->dies == 0) || (config->has_clusters && config->clusters == 0) ||
@@ -111,6 +123,20 @@ void machine_parse_smp_config(MachineState *ms, dies = dies >0 ? dies :1 ; clusters = clusters >0 ? clusters :1 ; +if (!mc->smp_props.books_supported && books >1 ) { +error_setg (errp,"books not supported by this machine's CPU topology" ); +return ; + } + + books = books >0 ? books :1 ; + +if (!mc->smp_props.drawers_supported && drawers >1 ) { +error_setg (errp,"drawers not supported by this machine's CPU topology" ); +return ; + } + + drawers = drawers >0 ? drawers :1 ; +/* compute missing values based on the provided ones */ if (cpus ==0 && maxcpus ==0 ) { sockets = sockets >0 ? sockets :1 ;
@@ -124,33 +150,41 @@ void machine_parse_smp_config(MachineState *ms,if (sockets ==0 ) { cores = cores >0 ? cores :1 ; threads = threads >0 ? threads :1 ; - sockets = maxcpus / (dies * clusters * cores * threads); + sockets = maxcpus / + (drawers * books * dies * clusters * cores * threads); }else if (cores ==0 ) { threads = threads >0 ? threads :1 ; - cores = maxcpus / (sockets * dies * clusters * threads); + cores = maxcpus / + (drawers * books * sockets * dies * clusters * threads); } }else {/* prefer cores over sockets since 6.2 */ if (cores ==0 ) { sockets = sockets >0 ? sockets :1 ; threads = threads >0 ? threads :1 ; - cores = maxcpus / (sockets * dies * clusters * threads); + cores = maxcpus / + (drawers * books * sockets * dies * clusters * threads); }else if (sockets ==0 ) { threads = threads >0 ? threads :1 ; - sockets = maxcpus / (dies * clusters * cores * threads); + sockets = maxcpus / + (drawers * books * dies * clusters * cores * threads); } }/* try to calculate omitted threads at last */ if (threads ==0 ) { - threads = maxcpus / (sockets * dies * clusters * cores); + threads = maxcpus / + (drawers * books * sockets * dies * clusters * cores); } } - maxcpus = maxcpus >0 ? maxcpus : sockets * dies * clusters * cores * threads; + maxcpus = maxcpus >0 ? maxcpus : drawers * books * sockets * dies * + clusters * cores * threads; cpus = cpus >0 ? cpus : maxcpus; ms->smp.cpus = cpus; + ms->smp.drawers = drawers; + ms->smp.books = books; ms->smp.sockets = sockets; ms->smp.dies = dies; ms->smp.clusters = clusters;
@@ -159,7 +193,7 @@ void machine_parse_smp_config(MachineState *ms, ms->smp.max_cpus = maxcpus;/* sanity-check of the computed topology */ -if (sockets * dies * clusters * cores * threads != maxcpus) { +if (drawers * books * sockets * dies * clusters * cores * threads != maxcpus) { g_autofree char *topo_msg = cpu_hierarchy_to_string(ms);error_setg (errp,"Invalid CPU topology: " "product of the hierarchy must match maxcpus: "
hw/core/machine.c +4 -0
@@ -821,6 +821,8 @@ static void machine_get_smp(Object *obj, Visitor *v, const char *name,MachineState *ms =MACHINE (obj);SMPConfiguration *config = &(SMPConfiguration ){ .has_cpus = true, .cpus = ms->smp.cpus, + .has_drawers = true, .drawers = ms->smp.drawers, + .has_books = true, .books = ms->smp.books, .has_sockets = true, .sockets = ms->smp.sockets, .has_dies = true, .dies = ms->smp.dies, .has_clusters = true, .clusters = ms->smp.clusters,
@@ -1087,6 +1089,8 @@ static void machine_initfn(Object *obj)/* default to mc->default_cpus */ ms->smp.cpus = mc->default_cpus; ms->smp.max_cpus = mc->default_cpus; + ms->smp.drawers =1 ; + ms->smp.books =1 ; ms->smp.sockets =1 ; ms->smp.dies =1 ; ms->smp.clusters =1 ;
include/hw/boards.h +8 -0
@@ -130,11 +130,15 @@ typedef struct {* @prefer_sockets - whether sockets are preferred over cores in smp parsing * @dies_supported - whether dies are supported by the machine * @clusters_supported - whether clusters are supported by the machine +* @books_supported - whether books are supported by the machine +* @drawers_supported - whether drawers are supported by the machine */ typedef struct {bool prefer_sockets;bool dies_supported;bool clusters_supported; +bool books_supported; +bool drawers_supported; }SMPCompatProps ;/**
@@ -299,6 +303,8 @@ typedef struct DeviceMemoryState {/** * CpuTopology: * @cpus: the number of present logical processors on the machine +* @drawers: the number of drawers on the machine +* @books: the number of books on the machine * @sockets: the number of sockets on the machine * @dies: the number of dies in one socket * @clusters: the number of clusters in one die
@@ -308,6 +314,8 @@ typedef struct DeviceMemoryState {*/ typedef struct CpuTopology { unsigned int cpus; + unsigned int drawers; + unsigned int books; unsigned int sockets; unsigned int dies; unsigned int clusters;
qapi/machine.json +12 -2
@@ -900,13 +900,15 @@ # a CPU is being hotplugged. # # @node-id: NUMA node ID the CPU belongs to -# @socket-id: socket number within node/board the CPU belongs to +# @drawer-id: drawer number within node/board the CPU belongs to +# @book-id: book number within drawer/node/board the CPU belongs to +# @socket-id: socket number within book/node/board the CPU belongs to # @die-id: die number within socket the CPU belongs to (since 4.1) # @cluster-id: cluster number within die the CPU belongs to (since 7.1) # @core-id: core number within cluster the CPU belongs to # @thread-id: thread number within core the CPU belongs to # -# Note: currently there are 6 properties that could be present +# Note: currently there are 7 properties that could be present # but management should be prepared to pass through other # properties with device_add command to allow for future # interface extension. This also requires the filed names to be kept in
@@ -916,6 +918,8 @@ ## { 'struct': 'CpuInstanceProperties', 'data': { '*node-id': 'int', + '*drawer-id': 'int', + '*book-id': 'int', '*socket-id': 'int', '*die-id': 'int', '*cluster-id': 'int',
@@ -1465,6 +1469,10 @@ # # @cpus: number of virtual CPUs in the virtual machine # +# @drawers: number of drawers in the CPU topology +# +# @books: number of books in the CPU topology +# # @sockets: number of sockets in the CPU topology # # @dies: number of dies per socket in the CPU topology
@@ -1481,6 +1489,8 @@ ## { 'struct': 'SMPConfiguration', 'data': { '*cpus': 'int', + '*drawers': 'int', + '*books': 'int', '*sockets': 'int', '*dies': 'int', '*clusters': 'int',
qemu-options.hx +4 -2
@@ -245,11 +245,13 @@ SRST ERST DEF("smp", HAS_ARG, QEMU_OPTION_smp, - "-smp [[cpus=]n][,maxcpus=maxcpus][,sockets=sockets][,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n" + "-smp [[cpus=]n][,maxcpus=maxcpus][,drawers=drawers][,books=books][,sockets=sockets][,dies=dies][,clusters=clusters][,cores=cores][,threads=threads]\n" " set the number of initial CPUs to 'n' [default=1]\n" " maxcpus= maximum number of total CPUs, including\n" " offline CPUs for hotplug, etc\n" - " sockets= number of sockets on the machine board\n" + " drawers= number of drawers on the machine board\n" + " books= number of books in one drawer\n" + " sockets= number of sockets in one book\n" " dies= number of dies in one socket\n" " clusters= number of clusters in one die\n" " cores= number of cores in one cluster\n"
softmmu/vl.c +6 -0
@@ -726,6 +726,12 @@ static QemuOptsList qemu_smp_opts = { { .name = "cpus", .type = QEMU_OPT_NUMBER, + }, { + .name = "drawers", + .type = QEMU_OPT_NUMBER, + }, { + .name = "books", + .type = QEMU_OPT_NUMBER, }, { .name = "sockets", .type = QEMU_OPT_NUMBER,
Signature · 1 line
2.31.1
Re: [PATCH v9 04/10] hw/core: introducing drawer and books for s390x
Quoted 1 line · Pierre Morel
S390x defines two topology levels above sockets: nbooks and drawers.
nbooks or books?
Quoted 4 lines
Let's add these two levels inside the CPU topology implementation.
Signed-off-by Pierre Morel <[email protected]>---
[...]
Quoted 9 lines
qapi/machine.json +0
-1@@ -900,13 +900,15 @@ # a CPU is being hotplugged. # # @node-id: NUMA node ID the CPU belongs to -# @socket-id: socket number within node/board the CPU belongs to
qapi/machine.json line 911
Quoted 10 lines
qapi/machine.json +4
-1+# @drawer-id: drawer number within node/board the CPU belongs to +# @book-id: book number within drawer/node/board the CPU belongs to +# @socket-id: socket number within book/node/board the CPU belongs to # @die-id: die number within socket the CPU belongs to (since 4.1) # @cluster-id: cluster number within die the CPU belongs to (since 7.1) # @core-id: core number within cluster the CPU belongs to # @thread-id: thread number within core the CPU belongs to # -# Note: currently there are 6 properties that could be present +# Note: currently there are 7 properties that could be present
Should this be 8?
qapi/machine.json line 914
Quoted 3 lines
qapi/machine.json +0
-0# but management should be prepared to pass through other # properties with device_add command to allow for future # interface extension. This also requires the filed names to be kept in
# sync with the properties passed to -device/device_add.
Not your patch's fault, but the second sentence is less than clear. What are "the filed names"? A typo perhaps?
Quoted 29 lines
qapi/machine.json +8
-0@@ -916,6 +918,8 @@ ## { 'struct': 'CpuInstanceProperties', 'data': { '*node-id': 'int', + '*drawer-id': 'int', + '*book-id': 'int', '*socket-id': 'int', '*die-id': 'int', '*cluster-id': 'int',@@ -1465,6 +1469,10 @@ # # @cpus: number of virtual CPUs in the virtual machine # +# @drawers: number of drawers in the CPU topology +# +# @books: number of books in the CPU topology +# # @sockets: number of sockets in the CPU topology # # @dies: number of dies per socket in the CPU topology@@ -1481,6 +1489,8 @@ ## { 'struct': 'SMPConfiguration', 'data': { '*cpus': 'int', + '*drawers': 'int', + '*books': 'int', '*sockets': 'int', '*dies': 'int', '*clusters': 'int',
[...]
Re: [PATCH v9 04/10] hw/core: introducing drawer and books for s390x
Quoted 26 lines · Markus Armbruster
Quoted 1 line · Pierre Morel
S390x defines two topology levels above sockets: nbooks and drawers.
nbooks or books?
Quoted 4 lines
Let's add these two levels inside the CPU topology implementation.
Signed-off-by Pierre Morel <[email protected]>---
[...]
Quoted 13 lines
qapi/machine.json +3
-1@@ -900,13 +900,15 @@ # a CPU is being hotplugged. # # @node-id: NUMA node ID the CPU belongs to -# @socket-id: socket number within node/board the CPU belongs to +# @drawer-id: drawer number within node/board the CPU belongs to +# @book-id: book number within drawer/node/board the CPU belongs to +# @socket-id: socket number within book/node/board the CPU belongs to # @die-id: die number within socket the CPU belongs to (since 4.1)
Quoted 8 lines
qapi/machine.json line 911
Quoted 6 lines
qapi/machine.json +1
-1# @cluster-id: cluster number within die the CPU belongs to (since 7.1) # @core-id: core number within cluster the CPU belongs to # @thread-id: thread number within core the CPU belongs to # -# Note: currently there are 6 properties that could be present +# Note: currently there are 7 properties that could be presentShould this be 8?
when one can count .. yes :)
Thanks, I will update in next spin
Quoted 8 lines
qapi/machine.json line 914
Quoted 4 lines
qapi/machine.json +0
-0# but management should be prepared to pass through other # properties with device_add command to allow for future # interface extension. This also requires the filed names to be kept in# sync with the properties passed to -device/device_add.
Not your patch's fault, but the second sentence is less than clear. What are "the filed names"? A typo perhaps?
I understood that it means the names in the structure here under which are filed inside the single quotes.
In this case, may be "filed names" should be replaced by "names in the CpuInstanceProperties structure"
Quoted 12 lines
qapi/machine.json line 1469
Quoted 12 lines
qapi/machine.json +2
-0@@ -916,6 +918,8 @@ ## { 'struct': 'CpuInstanceProperties', 'data': { '*node-id': 'int', + '*drawer-id': 'int', + '*book-id': 'int', '*socket-id': 'int', '*die-id': 'int', '*cluster-id': 'int',@@ -1465,6 +1469,10 @@ #
Regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v9 04/10] hw/core: introducing drawer and books for s390x
In reply to Markus ArmbrusterQuoted 5 lines · Markus Armbruster
Quoted 1 line · Pierre Morel
S390x defines two topology levels above sockets: nbooks and drawers.
nbooks or books?
Sorry, forgot this.
Yes typo, I mean "books"
Thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
PATCH v9 5/10 s390x/cpu: reporting drawers and books topology to the guest
In reply to Pierre MorelThe guest can ask for a topology report on drawer's or book's level.
Let's implement the STSI instruction's handling for the corresponding selector values.
--- hw/s390x/cpu-topology.c | 19 +++++++--- hw/s390x/s390-virtio-ccw.c | 2 ++ include/hw/s390x/cpu-topology.h | 7 +++- target/s390x/cpu_topology.c | 64 +++++++++++++++++++++++++++------ 4 files changed, 76 insertions(+), 16 deletions(-)
hw/s390x/cpu-topology.c +15 -4
@@ -46,7 +46,7 @@ S390Topology *s390_get_topology(void)void s390_topology_new_cpu (int core_id) {S390Topology *topo =s390_get_topology (); -int socket_id; +int socket_id, book_id, drawer_id;int bit, origin;/* In the case no Topology is used nothing is to be done here */
@@ -55,6 +55,8 @@ void s390_topology_new_cpu(int core_id) } socket_id = core_id / topo->cores; + book_id = socket_id / topo->sockets; + drawer_id = book_id / topo->books; bit = core_id; origin = bit /64 ;
@@ -77,6 +79,8 @@ void s390_topology_new_cpu(int core_id)* CPU inside several CPU containers inside the socket container. */ qemu_mutex_lock (&topo->topo_mutex); + topo->drawer[drawer_id].active_count++; + topo->book[book_id].active_count++; topo->socket[socket_id].active_count++; topo->tle[socket_id].active_count++;set_bit (bit, &topo->tle[socket_id].mask[origin]);
@@ -99,13 +103,20 @@ static void s390_topology_realize(DeviceState *dev, Error **errp)S390Topology *topo =S390_CPU_TOPOLOGY (dev);int n; + topo->drawers = ms->smp.drawers; + topo->books = ms->smp.books; + topo->total_books = topo->books * topo->drawers; topo->sockets = ms->smp.sockets; + topo->total_sockets = topo->sockets * topo->books * topo->drawers; topo->cores = ms->smp.cores; - topo->tles = ms->smp.max_cpus; - n = topo->sockets; + n = topo->drawers; + topo->drawer =g_malloc0 (n *sizeof (S390TopoContainer)); + n *= topo->books; + topo->book =g_malloc0 (n *sizeof (S390TopoContainer)); + n *= topo->sockets; topo->socket =g_malloc0 (n *sizeof (S390TopoContainer)); - topo->tle =g_malloc0 (topo->tles *sizeof (S390TopoTLE)); + topo->tle =g_malloc0 (n *sizeof (S390TopoTLE));qemu_mutex_init (&topo->topo_mutex); }
hw/s390x/s390-virtio-ccw.c +2 -0
@@ -626,6 +626,8 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data) hc->unplug_request = s390_machine_device_unplug_request; nc->nmi_monitor_handler = s390_nmi; mc->default_ram_id ="s390.ram" ; + mc->smp_props.books_supported = true; + mc->smp_props.drawers_supported = true; }static inline bool machine_get_aes_key_wrap (Object *obj,Error **errp)
include/hw/s390x/cpu-topology.h +6 -1
@@ -29,9 +29,14 @@ typedef struct S390TopoTLE {struct S390Topology {SysBusDevice parent_obj; +int total_books; +int total_sockets; +int drawers; +int books;int sockets;int cores; -int tles; +S390TopoContainer *drawer; +S390TopoContainer *book;S390TopoContainer *socket;S390TopoTLE *tle;QemuMutex topo_mutex;
target/s390x/cpu_topology.c +53 -11
102 lines not on this page · raw message
Signature · 1 line
2.31.1
Re: [PATCH v9 05/10] s390x/cpu: reporting drawers and books topology to the guest
Quoted 18 lines · Pierre Morel
The guest can ask for a topology report on drawer's or book's level.
Let's implement the STSI instruction's handling for the corresponding selector values.Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/cpu-topology.c | 19 +++++++--- hw/s390x/s390-virtio-ccw.c | 2 ++ include/hw/s390x/cpu-topology.h | 7 +++- target/s390x/cpu_topology.c | 64 +++++++++++++++++++++++++++------ 4 files changed, 76 insertions(+), 16 deletions(-)hw/s390x/cpu-topology.c +0
-0
[...]
Quoted 11 lines
hw/s390x/cpu-topology.c +4
-1@@ -99,13 +103,20 @@ static void s390_topology_realize(DeviceState *dev, Error **errp)S390Topology *topo =S390_CPU_TOPOLOGY (dev);int n; + topo->drawers = ms->smp.drawers; + topo->books = ms->smp.books; + topo->total_books = topo->books * topo->drawers; topo->sockets = ms->smp.sockets; + topo->total_sockets = topo->sockets * topo->books * topo->drawers; topo->cores = ms->smp.cores; - topo->tles = ms->smp.max_cpus;
hw/s390x/cpu-topology.c static void s390_topology_realize(DeviceState *dev, Error **errp) · line 119
Quoted 10 lines
hw/s390x/cpu-topology.c +6
-2- n = topo->sockets; + n = topo->drawers; + topo->drawer =g_malloc0 (n *sizeof (S390TopoContainer)); + n *= topo->books; + topo->book =g_malloc0 (n *sizeof (S390TopoContainer)); + n *= topo->sockets; topo->socket =g_malloc0 (n *sizeof (S390TopoContainer)); - topo->tle =g_malloc0 (topo->tles *sizeof (S390TopoTLE)); + topo->tle =g_malloc0 (n *sizeof (S390TopoTLE));
Same question here about using g_new0.
Quoted 21 lines
hw/s390x/cpu-topology.c +0
-0qemu_mutex_init (&topo->topo_mutex); }hw/s390x/s390-virtio-ccw.c +2
-0@@ -626,6 +626,8 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data) hc->unplug_request = s390_machine_device_unplug_request; nc->nmi_monitor_handler = s390_nmi; mc->default_ram_id ="s390.ram" ; + mc->smp_props.books_supported = true; + mc->smp_props.drawers_supported = true; }static inline bool machine_get_aes_key_wrap (Object *obj,Error **errp)include/hw/s390x/cpu-topology.h +0
-0
include/hw/s390x/cpu-topology.h typedef struct S390TopoTLE { · line 33
Quoted 5 lines
include/hw/s390x/cpu-topology.h +2
-0@@ -29,9 +29,14 @@ typedef struct S390TopoTLE {struct S390Topology {SysBusDevice parent_obj; +int total_books; +int total_sockets;
What are these used for? I'm not seeing anything.
include/hw/s390x/cpu-topology.h typedef struct S390TopoTLE { · line 34
Quoted 5 lines
include/hw/s390x/cpu-topology.h +2
-1+int drawers; +int books;int sockets;int cores; -int tles;
You remove this in this patch and you didn't really need it before. As far as I can tell it was just used for calculating the number of tles to allocate and you could use a local variable instead. So I would get rid of it in the patch that introduced it.
Quoted 10 lines
include/hw/s390x/cpu-topology.h +2
-0+S390TopoContainer *drawer; +S390TopoContainer *book;S390TopoContainer *socket;S390TopoTLE *tle;QemuMutex topo_mutex;target/s390x/cpu_topology.c +0
-0
target/s390x/cpu_topology.c static char *fill_tle_cpu(char *p, uint64_t mask, int origin) · line 41
Quoted 6 lines
target/s390x/cpu_topology.c +1
-1@@ -37,19 +37,18 @@ static char *fill_tle_cpu(char *p, uint64_t mask, int origin)return p +sizeof (*tle); } -static char *s390_top_set_level2 (S390Topology *topo,char *p) +static char *s390_top_set_level2 (S390Topology *topo,char *p,int fs,int ns) {
I wouldn't hate more verbose names for fs and ns. start_socket, num_socket maybe? Same for fb, nb, but it is your call, it's not really hard to understand the code.
target/s390x/cpu_topology.c static char *fill_tle_cpu(char *p, uint64_t mask, int origin) · line 49
Quoted 12 lines
target/s390x/cpu_topology.c +5
-4-int i, origin; +int socket, origin; +uint64_t mask; -for (i =0 ; i < topo->sockets; i++) { -if (!topo->socket[i].active_count) { +for (socket = fs; socket < fs + ns; socket++) { +if (!topo->socket[socket].active_count) {continue ; } - p =fill_container (p,1 , i); + p =fill_container (p,1 , socket);
Have you considered using an enum for the level constants?
Quoted 12 lines
target/s390x/cpu_topology.c +1
-3for (origin =0 ; origin <S390_TOPOLOGY_MAX_ORIGIN ; origin++) { -uint64_t mask =0L ; - - mask =be64_to_cpu (topo->tle[i].mask[origin]); + mask =be64_to_cpu (topo->tle[socket].mask[origin]);if (mask) { p =fill_tle_cpu (p, mask, origin); }@@ -58,19 +57,63 @@ static char *s390_top_set_level2(S390Topology *topo, char *p)return p; }
target/s390x/cpu_topology.c static char *s390_top_set_level2(S390Topology *topo, char *p) · line 69
Quoted 10 lines
target/s390x/cpu_topology.c +10
-0+static char *s390_top_set_level3 (S390Topology *topo,char *p,int fb,int nb) +{ +int book, fs =0 ; + +for (book = fb; book < fb + nb; book++, fs += topo->sockets) { +if (!topo->book[book].active_count) { +continue ; + } + p =fill_container (p,2 , book); + p =s390_top_set_level2 (topo, p, fs, topo->sockets);
Indent is off.
Quoted 23 lines
target/s390x/cpu_topology.c +19
-0+ } +return p; +} + +static char *s390_top_set_level4 (S390Topology *topo,char *p) +{ +int drawer, fb =0 ; + +for (drawer =0 ; drawer < topo->drawers; drawer++, fb += topo->books) { +if (!topo->drawer[drawer].active_count) { +continue ; + } + p =fill_container (p,3 , drawer); + p =s390_top_set_level3 (topo, p, fb, topo->books); + } +return p; +} +static int setup_stsi (SysIB_151x *sysib,int level) {S390Topology *topo =s390_get_topology ();char *p = (char *)sysib->tle; +int max_containers;
target/s390x/cpu_topology.c static char *s390_top_set_level2(S390Topology *topo, char *p) · line 102
Quoted 10 lines
target/s390x/cpu_topology.c +4
-0qemu_mutex_lock (&topo->topo_mutex); sysib->mnest = level;switch (level) {case 2 : + max_containers = topo->sockets * topo->books * topo->drawers; + sysib->mag[TOPOLOGY_NR_MAG2 ] = max_containers; + sysib->mag[TOPOLOGY_NR_MAG1 ] = topo->cores; + p =s390_top_set_level2 (topo, p,0 , max_containers);
Isn't this logic change already required for the patch that introduced stsi 15.1.2 handling?
Quoted 19 lines
target/s390x/cpu_topology.c +12
-1+break ; +case 3 : + max_containers = topo->books * topo->drawers; + sysib->mag[TOPOLOGY_NR_MAG3 ] = max_containers; sysib->mag[TOPOLOGY_NR_MAG2 ] = topo->sockets; sysib->mag[TOPOLOGY_NR_MAG1 ] = topo->cores; - p =s390_top_set_level2 (topo, p); + p =s390_top_set_level3 (topo, p,0 , max_containers); +break ; +case 4 : + sysib->mag[TOPOLOGY_NR_MAG4 ] = topo->drawers; + sysib->mag[TOPOLOGY_NR_MAG3 ] = topo->books; + sysib->mag[TOPOLOGY_NR_MAG2 ] = topo->sockets; + sysib->mag[TOPOLOGY_NR_MAG1 ] = topo->cores; + p =s390_top_set_level4 (topo, p);break ; }
target/s390x/cpu_topology.c static int setup_stsi(SysIB_151x *sysib, int level) · line 125
Quoted 5 lines
target/s390x/cpu_topology.c +1
-1@@ -79,7 +122,7 @@ static int setup_stsi(SysIB_151x *sysib, int level)return p - (char *)sysib->tle; } -#define S390_TOPOLOGY_MAX_MNEST 2 +#define S390_TOPOLOGY_MAX_MNEST 4
AFAIK you're only allowed to increase this if the maximum mnest facility is installed. If it isn't, only level 2 is supported. Which would mean that this patch doesn't do anything.
Quoted 8 lines
target/s390x/cpu_topology.c +0
-1void insert_stsi_15_1_x (S390CPU *cpu,int sel2,__u64 addr,uint8_t ar) {SysIB_151x *sysib;@@ -105,4 +148,3 @@ void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar) out_free:g_free (sysib); } -
Re: [PATCH v9 05/10] s390x/cpu: reporting drawers and books topology to the guest
Quoted 33 lines · Janis Schoetterl-Glausch
Quoted 18 lines · Pierre Morel
The guest can ask for a topology report on drawer's or book's level.
Let's implement the STSI instruction's handling for the corresponding selector values.Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/cpu-topology.c | 19 +++++++--- hw/s390x/s390-virtio-ccw.c | 2 ++ include/hw/s390x/cpu-topology.h | 7 +++- target/s390x/cpu_topology.c | 64 +++++++++++++++++++++++++++------ 4 files changed, 76 insertions(+), 16 deletions(-)hw/s390x/cpu-topology.c +0
-0[...]
Quoted 12 lines
hw/s390x/cpu-topology.c +4
-1@@ -99,13 +103,20 @@ static void s390_topology_realize(DeviceState *dev, Error **errp)S390Topology *topo =S390_CPU_TOPOLOGY (dev);int n; + topo->drawers = ms->smp.drawers; + topo->books = ms->smp.books; + topo->total_books = topo->books * topo->drawers; topo->sockets = ms->smp.sockets; + topo->total_sockets = topo->sockets * topo->books * topo->drawers; topo->cores = ms->smp.cores; - topo->tles = ms->smp.max_cpus;
Quoted 11 lines
hw/s390x/cpu-topology.c static void s390_topology_realize(DeviceState *dev, Error **errp) · line 119
Quoted 9 lines
hw/s390x/cpu-topology.c +6
-2- n = topo->sockets; + n = topo->drawers; + topo->drawer =g_malloc0 (n *sizeof (S390TopoContainer)); + n *= topo->books; + topo->book =g_malloc0 (n *sizeof (S390TopoContainer)); + n *= topo->sockets; topo->socket =g_malloc0 (n *sizeof (S390TopoContainer)); - topo->tle =g_malloc0 (topo->tles *sizeof (S390TopoTLE)); + topo->tle =g_malloc0 (n *sizeof (S390TopoTLE));Same question here about using
g_new0.
yes, g_new0 is better here
Quoted 35 lines
Quoted 21 lines
hw/s390x/cpu-topology.c +0
-0qemu_mutex_init (&topo->topo_mutex); }hw/s390x/s390-virtio-ccw.c +2
-0@@ -626,6 +626,8 @@ static void ccw_machine_class_init(ObjectClass *oc, void *data) hc->unplug_request = s390_machine_device_unplug_request; nc->nmi_monitor_handler = s390_nmi; mc->default_ram_id ="s390.ram" ; + mc->smp_props.books_supported = true; + mc->smp_props.drawers_supported = true; }static inline bool machine_get_aes_key_wrap (Object *obj,Error **errp)include/hw/s390x/cpu-topology.h +0
-0include/hw/s390x/cpu-topology.h typedef struct S390TopoTLE { · line 33
Quoted 5 lines
include/hw/s390x/cpu-topology.h +2
-0@@ -29,9 +29,14 @@ typedef struct S390TopoTLE {struct S390Topology {SysBusDevice parent_obj; +int total_books; +int total_sockets;What are these used for? I'm not seeing anything.
include/hw/s390x/cpu-topology.h typedef struct S390TopoTLE { · line 34
Quoted 5 lines
include/hw/s390x/cpu-topology.h +2
-1+int drawers; +int books;int sockets;int cores; -int tles;
Quoted 4 lines
You remove this in this patch and you didn't really need it before. As far as I can tell it was just used for calculating the number of tles to allocate and you could use a local variable instead. So I would get rid of it in the patch that introduced it.
yes
Quoted 14 lines
Quoted 14 lines
include/hw/s390x/cpu-topology.h +2
-0+S390TopoContainer *drawer; +S390TopoContainer *book;S390TopoContainer *socket;S390TopoTLE *tle;QemuMutex topo_mutex;target/s390x/cpu_topology.c +0
-0@@ -37,19 +37,18 @@ static char *fill_tle_cpu(char *p, uint64_t mask, int origin)return p +sizeof (*tle); }
Quoted 7 lines
target/s390x/cpu_topology.c static char *fill_tle_cpu(char *p, uint64_t mask, int origin) · line 41
Quoted 3 lines
target/s390x/cpu_topology.c +1
-1-static char *s390_top_set_level2 (S390Topology *topo,char *p) +static char *s390_top_set_level2 (S390Topology *topo,char *p,int fs,int ns) {I wouldn't hate more verbose names for fs and ns.
start_socket,num_socketmaybe? Same for fb, nb, but it is your call, it's not really hard to understand the code.
I prefer to keep the names short but I will have a second thought.
Quoted 14 lines
target/s390x/cpu_topology.c static char *fill_tle_cpu(char *p, uint64_t mask, int origin) · line 49
Quoted 13 lines
target/s390x/cpu_topology.c +5
-4-int i, origin; +int socket, origin; +uint64_t mask; -for (i =0 ; i < topo->sockets; i++) { -if (!topo->socket[i].active_count) { +for (socket = fs; socket < fs + ns; socket++) { +if (!topo->socket[socket].active_count) {continue ; } - p =fill_container (p,1 , i); + p =fill_container (p,1 , socket);
Quoted 1 line
Have you considered using an enum for the level constants?
did not but yes it would be better.
Quoted 13 lines
Quoted 13 lines
target/s390x/cpu_topology.c +1
-3for (origin =0 ; origin <S390_TOPOLOGY_MAX_ORIGIN ; origin++) { -uint64_t mask =0L ; - - mask =be64_to_cpu (topo->tle[i].mask[origin]); + mask =be64_to_cpu (topo->tle[socket].mask[origin]);if (mask) { p =fill_tle_cpu (p, mask, origin); }@@ -58,19 +57,63 @@ static char *s390_top_set_level2(S390Topology *topo, char *p)return p; }
Quoted 12 lines
target/s390x/cpu_topology.c static char *s390_top_set_level2(S390Topology *topo, char *p) · line 69
Quoted 10 lines
target/s390x/cpu_topology.c +10
-0+static char *s390_top_set_level3 (S390Topology *topo,char *p,int fb,int nb) +{ +int book, fs =0 ; + +for (book = fb; book < fb + nb; book++, fs += topo->sockets) { +if (!topo->book[book].active_count) { +continue ; + } + p =fill_container (p,2 , book); + p =s390_top_set_level2 (topo, p, fs, topo->sockets);Indent is off.
thx
Quoted 27 lines
Quoted 27 lines
target/s390x/cpu_topology.c +19
-0+ } +return p; +} + +static char *s390_top_set_level4 (S390Topology *topo,char *p) +{ +int drawer, fb =0 ; + +for (drawer =0 ; drawer < topo->drawers; drawer++, fb += topo->books) { +if (!topo->drawer[drawer].active_count) { +continue ; + } + p =fill_container (p,3 , drawer); + p =s390_top_set_level3 (topo, p, fb, topo->books); + } +return p; +} +static int setup_stsi (SysIB_151x *sysib,int level) {S390Topology *topo =s390_get_topology ();char *p = (char *)sysib->tle; +int max_containers;qemu_mutex_lock (&topo->topo_mutex);
Quoted 10 lines
target/s390x/cpu_topology.c static char *s390_top_set_level2(S390Topology *topo, char *p) · line 102
Quoted 7 lines
target/s390x/cpu_topology.c +4
-0sysib->mnest = level;switch (level) {case 2 : + max_containers = topo->sockets * topo->books * topo->drawers; + sysib->mag[TOPOLOGY_NR_MAG2 ] = max_containers; + sysib->mag[TOPOLOGY_NR_MAG1 ] = topo->cores; + p =s390_top_set_level2 (topo, p,0 , max_containers);Isn't this logic change already required for the patch that introduced stsi 15.1.2 handling?
yes, thx
Quoted 23 lines
Quoted 23 lines
target/s390x/cpu_topology.c +12
-1+break ; +case 3 : + max_containers = topo->books * topo->drawers; + sysib->mag[TOPOLOGY_NR_MAG3 ] = max_containers; sysib->mag[TOPOLOGY_NR_MAG2 ] = topo->sockets; sysib->mag[TOPOLOGY_NR_MAG1 ] = topo->cores; - p =s390_top_set_level2 (topo, p); + p =s390_top_set_level3 (topo, p,0 , max_containers); +break ; +case 4 : + sysib->mag[TOPOLOGY_NR_MAG4 ] = topo->drawers; + sysib->mag[TOPOLOGY_NR_MAG3 ] = topo->books; + sysib->mag[TOPOLOGY_NR_MAG2 ] = topo->sockets; + sysib->mag[TOPOLOGY_NR_MAG1 ] = topo->cores; + p =s390_top_set_level4 (topo, p);break ; }@@ -79,7 +122,7 @@ static int setup_stsi(SysIB_151x *sysib, int level)return p - (char *)sysib->tle; }
Quoted 6 lines
target/s390x/cpu_topology.c static int setup_stsi(SysIB_151x *sysib, int level) · line 125
Quoted 2 lines
target/s390x/cpu_topology.c +1
-1-#define S390_TOPOLOGY_MAX_MNEST 2 +#define S390_TOPOLOGY_MAX_MNEST 4AFAIK you're only allowed to increase this if the maximum mnest facility is installed. If it isn't, only level 2 is supported. Which would mean that this patch doesn't do anything.
AFAIU this is model dependant but I have to rework the MNEST entry of the SYSIB anyway.
Quoted 10 lines
target/s390x/cpu_topology.c void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar) · line 108
Quoted 9 lines
target/s390x/cpu_topology.c +0
-1void insert_stsi_15_1_x (S390CPU *cpu,int sel2,__u64 addr,uint8_t ar) {SysIB_151x *sysib;@@ -105,4 +148,3 @@ void insert_stsi_15_1_x(S390CPU *cpu, int sel2, __u64 addr, uint8_t ar) out_free:g_free (sysib); } -
Thanks for the comments
Regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
PATCH v9 6/10 s390x/cpu_topology: resetting the Topology-Change-Report
In reply to Pierre MorelDuring a subsystem reset the Topology-Change-Report is cleared by the machine.
Let's ask KVM to clear the Modified Topology Change Report (MTCR)
bit of the SCA in the case of a subsystem reset.
--- hw/s390x/cpu-topology.c | 12 ++++++++++++ hw/s390x/s390-virtio-ccw.c | 1 + target/s390x/cpu-sysemu.c | 7 +++++++ target/s390x/cpu.h | 1 + target/s390x/kvm/kvm.c | 23 +++++++++++++++++++++++ target/s390x/kvm/kvm_s390x.h | 1 + 6 files changed, 45 insertions(+)
hw/s390x/cpu-topology.c +12 -0
@@ -121,6 +121,17 @@ static void s390_topology_realize(DeviceState *dev, Error **errp)qemu_mutex_init (&topo->topo_mutex); } +/** +* s390_topology_reset: +* @dev: the device +* +* Calls the sysemu topology reset +*/ +static void s390_topology_reset (DeviceState *dev) +{ +s390_cpu_topology_reset (); +} +/** * topology_class_init: * @oc: Object class
@@ -134,6 +145,7 @@ static void topology_class_init(ObjectClass *oc, void *data) dc->realize = s390_topology_realize;set_bit (DEVICE_CATEGORY_MISC , dc->categories); + dc->reset = s390_topology_reset; }static const TypeInfo cpu_topology_info = {
hw/s390x/s390-virtio-ccw.c +1 -0
@@ -104,6 +104,7 @@ static const char *const reset_dev_types[] = {"s390-flic" ,"diag288" ,TYPE_S390_PCI_HOST_BRIDGE , + TYPE_S390_CPU_TOPOLOGY, };static void subsystem_reset (void )
target/s390x/cpu-sysemu.c +7 -0
@@ -306,3 +306,10 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg)kvm_s390_set_diag318 (cs, arg.host_ulong); } } + +void s390_cpu_topology_reset (void ) +{ +if (kvm_enabled ()) { +kvm_s390_topology_set_mtcr (0 ); + } +}
target/s390x/cpu.h +1 -0
@@ -826,6 +826,7 @@ void s390_enable_css_support(S390CPU *cpu);void s390_do_cpu_set_diag318 (CPUState *cs,run_on_cpu_data arg);int s390_assign_subch_ioeventfd (EventNotifier *notifier,uint32_t sch_id,int vq,bool assign); +void s390_cpu_topology_reset (void );#ifndef CONFIG_USER_ONLY unsigned int s390_cpu_set_state (uint8_t cpu_state,S390CPU *cpu);#else
target/s390x/kvm/kvm.c +23 -0
@@ -2585,3 +2585,26 @@ int kvm_s390_get_zpci_op(void) {return cap_zpci_op; } + +int kvm_s390_topology_set_mtcr (uint64_t attr) +{ +struct kvm_device_attr attribute = { + .group =KVM_S390_VM_CPU_TOPOLOGY , + .attr = attr, + }; +int ret; + +if (!s390_has_feat (S390_FEAT_CONFIGURATION_TOPOLOGY )) { +return -EFAULT ; + } +if (!kvm_vm_check_attr (kvm_state,KVM_S390_VM_CPU_TOPOLOGY , attr)) { +return -ENOENT ; + } + + ret =kvm_vm_ioctl (kvm_state,KVM_SET_DEVICE_ATTR , &attribute); +if (ret) { +error_report ("Failed to set cpu topology attribute %lu: %s" , + attr,strerror (-ret)); + } +return ret; +}
target/s390x/kvm/kvm_s390x.h +1 -0
@@ -46,5 +46,6 @@ void kvm_s390_crypto_reset(void);void kvm_s390_restart_interrupt (S390CPU *cpu);void kvm_s390_stop_interrupt (S390CPU *cpu);void kvm_s390_set_diag318 (CPUState *cs,uint64_t diag318_info); +int kvm_s390_topology_set_mtcr (uint64_t attr);#endif /* KVM_S390X_H */
Signature · 1 line
2.31.1
Re: [PATCH v9 06/10] s390x/cpu_topology: resetting the Topology-Change-Report
Quoting Pierre Morel (2022-09-02 09:55:27)
Quoted 6 lines
During a subsystem reset the Topology-Change-Report is cleared by the machine.
Let's ask KVM to clear the Modified Topology Change Report (MTCR)
bit of the SCA in the case of a subsystem reset.Signed-off-by Pierre Morel <[email protected]>
Re: [PATCH v9 06/10] s390x/cpu_topology: resetting the Topology-Change-Report
Quoted 9 lines · Nico Boehr
Quoting Pierre Morel (2022-09-02 09:55:27)
Quoted 6 lines
During a subsystem reset the Topology-Change-Report is cleared by the machine.
Let's ask KVM to clear the Modified Topology Change Report (MTCR)
bit of the SCA in the case of a subsystem reset.Signed-off-by Pierre Morel <[email protected]>Reviewed-by Nico Boehr <[email protected]>
Thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v9 06/10] s390x/cpu_topology: resetting the Topology-Change-Report
In reply to Pierre MorelQuoted 14 lines · Pierre Morel
During a subsystem reset the Topology-Change-Report is cleared by the machine.
Let's ask KVM to clear the Modified Topology Change Report (MTCR)
bit of the SCA in the case of a subsystem reset.Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/cpu-topology.c | 12 ++++++++++++ hw/s390x/s390-virtio-ccw.c | 1 + target/s390x/cpu-sysemu.c | 7 +++++++ target/s390x/cpu.h | 1 + target/s390x/kvm/kvm.c | 23 +++++++++++++++++++++++ target/s390x/kvm/kvm_s390x.h | 1 + 6 files changed, 45 insertions(+)
[...]
Quoted 9 lines
target/s390x/kvm/kvm.c +1
-0@@ -2585,3 +2585,26 @@ int kvm_s390_get_zpci_op(void) {return cap_zpci_op; } +
target/s390x/kvm/kvm.c int kvm_s390_get_zpci_op(void) · line 2598
Quoted 10 lines
target/s390x/kvm/kvm.c +10
-0+int kvm_s390_topology_set_mtcr (uint64_t attr) +{ +struct kvm_device_attr attribute = { + .group =KVM_S390_VM_CPU_TOPOLOGY , + .attr = attr, + }; +int ret; + +if (!s390_has_feat (S390_FEAT_CONFIGURATION_TOPOLOGY )) { +return -EFAULT ;
Why EFAULT?
The return value is just ignored when resetting, isn't it? I wonder if it would be better not to.
Is it necessary because you're detecting the feature after you've already created the S390Topology instance?
And you're doing that because that's just the order in which QEMU does things? So the machine class is inited before the cpu model? I wonder if there is a nice way to create the S390Topology only if the feature is selected.
Anyway:
Quoted 13 lines
target/s390x/kvm/kvm.c +12
-0+ } +if (!kvm_vm_check_attr (kvm_state,KVM_S390_VM_CPU_TOPOLOGY , attr)) { +return -ENOENT ; + } + + ret =kvm_vm_ioctl (kvm_state,KVM_SET_DEVICE_ATTR , &attribute); +if (ret) { +error_report ("Failed to set cpu topology attribute %lu: %s" , + attr,strerror (-ret)); + } +return ret; +}
[...]
Re: [PATCH v9 06/10] s390x/cpu_topology: resetting the Topology-Change-Report
Quoted 31 lines · Janis Schoetterl-Glausch
Quoted 14 lines · Pierre Morel
During a subsystem reset the Topology-Change-Report is cleared by the machine.
Let's ask KVM to clear the Modified Topology Change Report (MTCR)
bit of the SCA in the case of a subsystem reset.Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/cpu-topology.c | 12 ++++++++++++ hw/s390x/s390-virtio-ccw.c | 1 + target/s390x/cpu-sysemu.c | 7 +++++++ target/s390x/cpu.h | 1 + target/s390x/kvm/kvm.c | 23 +++++++++++++++++++++++ target/s390x/kvm/kvm_s390x.h | 1 + 6 files changed, 45 insertions(+)[...]
Quoted 13 lines
target/s390x/kvm/kvm.c +5
-0@@ -2585,3 +2585,26 @@ int kvm_s390_get_zpci_op(void) {return cap_zpci_op; } + +int kvm_s390_topology_set_mtcr (uint64_t attr) +{ +struct kvm_device_attr attribute = { + .group =KVM_S390_VM_CPU_TOPOLOGY ,
Quoted 8 lines
target/s390x/kvm/kvm.c int kvm_s390_get_zpci_op(void) · line 2598
Quoted 6 lines
target/s390x/kvm/kvm.c +6
-0+ .attr = attr, + }; +int ret; + +if (!s390_has_feat (S390_FEAT_CONFIGURATION_TOPOLOGY )) { +return -EFAULT ;Why EFAULT?
any proposition?
Quoted 1 line
The return value is just ignored when resetting, isn't it?
In migration the same function is used and we need to return an error there.
But if we would not use it, after the comments you did for migration, we may indeed not need it, then I guess using error_report and a void function may be better.
Quoted 7 lines
I wonder if it would be better not to.
Is it necessary because you're detecting the feature after you've already created the S390Topology instance?
And you're doing that because that's just the order in which QEMU does things? So the machine class is inited before the cpu model? I wonder if there is a nice way to create the S390Topology only if the feature is selected.
I do not know if it is possible. I will look.
Quoted 3 lines
Anyway:
Reviewed-by Janis Schoetterl-Glausch <[email protected]>
Thanks,
Pierre
Quoted 16 lines
Quoted 14 lines
target/s390x/kvm/kvm.c +12
-0+ } +if (!kvm_vm_check_attr (kvm_state,KVM_S390_VM_CPU_TOPOLOGY , attr)) { +return -ENOENT ; + } + + ret =kvm_vm_ioctl (kvm_state,KVM_SET_DEVICE_ATTR , &attribute); +if (ret) { +error_report ("Failed to set cpu topology attribute %lu: %s" , + attr,strerror (-ret)); + } +return ret; +}[...]
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
PATCH v9 7/10 s390x/cpu_topology: CPU topology migration
In reply to Pierre MorelThe migration can only take place if both source and destination of the migration both use or both do not use the CPU topology facility.
We indicate a change in topology during migration postload for the case the topology changed between source and destination.
--- hw/s390x/cpu-topology.c | 79 +++++++++++++++++++++++++++++++++ include/hw/s390x/cpu-topology.h | 1 + target/s390x/cpu-sysemu.c | 8 ++++ target/s390x/cpu.h | 1 + 4 files changed, 89 insertions(+)
hw/s390x/cpu-topology.c +79 -0
@@ -19,6 +19,7 @@#include "target/s390x/cpu.h" #include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/cpu-topology.h" +#include "migration/vmstate.h" S390Topology *s390_get_topology (void ) {
@@ -132,6 +133,83 @@ static void s390_topology_reset(DeviceState *dev)s390_cpu_topology_reset (); } +/** +* cpu_topology_postload +* @opaque: a pointer to the S390Topology +* @version_id: version identifier +* +* We check that the topology is used or is not used +* on both side identically. +* +* If the topology is in use we set the Modified Topology Change Report +* on the destination host. +*/ +static int cpu_topology_postload (void *opaque,int version_id) +{ +S390Topology *topo = opaque; +int ret; + +if (topo->topology_needed !=s390_has_feat (S390_FEAT_CONFIGURATION_TOPOLOGY )) { +if (topo->topology_needed) { +error_report ("Topology facility is needed in destination" ); + }else { +error_report ("Topology facility can not be used in destination" ); + } +return -EINVAL ; + } + +/* We do not support CPU Topology, all is good */ +if (!s390_has_feat (S390_FEAT_CONFIGURATION_TOPOLOGY )) { +return 0 ; + } + +/* We support CPU Topology, set the MTCR */ + ret =s390_cpu_topology_mtcr_set (); +if (ret) { +error_report ("Failed to set MTCR: %s" ,strerror (-ret)); + } +return ret; +} + +/** +* cpu_topology_presave: +* @opaque: The pointer to the S390Topology +* +* Save the usage of the CPU Topology in the VM State. +*/ +static int cpu_topology_presave (void *opaque) +{ +S390Topology *topo = opaque; + + topo->topology_needed =s390_has_feat (S390_FEAT_CONFIGURATION_TOPOLOGY ); +return 0 ; +} + +/** +* cpu_topology_needed: +* @opaque: The pointer to the S390Topology +* +* If we use the CPU Topology on the source it will be needed on the destination. +*/ +static bool cpu_topology_needed (void *opaque) +{ +return s390_has_feat (S390_FEAT_CONFIGURATION_TOPOLOGY ); +} + + +const VMStateDescription vmstate_cpu_topology = { + .name ="cpu_topology" , + .version_id =1 , + .post_load = cpu_topology_postload, + .pre_save = cpu_topology_presave, + .minimum_version_id =1 , + .needed = cpu_topology_needed, + .fields = (VMStateField []) { +VMSTATE_BOOL (topology_needed, S390Topology), +VMSTATE_END_OF_LIST () + } +}; +/** * topology_class_init: * @oc: Object class
@@ -146,6 +224,7 @@ static void topology_class_init(ObjectClass *oc, void *data) dc->realize = s390_topology_realize;set_bit (DEVICE_CATEGORY_MISC , dc->categories); dc->reset = s390_topology_reset; + dc->vmsd = &vmstate_cpu_topology; }static const TypeInfo cpu_topology_info = {
include/hw/s390x/cpu-topology.h +1 -0
@@ -29,6 +29,7 @@ typedef struct S390TopoTLE {struct S390Topology {SysBusDevice parent_obj; +bool topology_needed;int total_books;int total_sockets;int drawers;
target/s390x/cpu-sysemu.c +8 -0
@@ -313,3 +313,11 @@ void s390_cpu_topology_reset(void)kvm_s390_topology_set_mtcr (0 ); } } + +int s390_cpu_topology_mtcr_set (void ) +{ +if (kvm_enabled ()) { +return kvm_s390_topology_set_mtcr (1 ); + } +return -ENOENT ; +}
target/s390x/cpu.h +1 -0
@@ -827,6 +827,7 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data arg);int s390_assign_subch_ioeventfd (EventNotifier *notifier,uint32_t sch_id,int vq,bool assign);void s390_cpu_topology_reset (void ); +int s390_cpu_topology_mtcr_set (void );#ifndef CONFIG_USER_ONLY unsigned int s390_cpu_set_state (uint8_t cpu_state,S390CPU *cpu);#else
Signature · 1 line
2.31.1
Re: [PATCH v9 07/10] s390x/cpu_topology: CPU topology migration
Quoted 6 lines · Pierre Morel
The migration can only take place if both source and destination of the migration both use or both do not use the CPU topology facility.
We indicate a change in topology during migration postload for the case the topology changed between source and destination.
You always set the report bit after migration, right?
In the last series you actually migrated the bit.
Why the change? With the code you have actually migrating the bit isn't hard.
Quoted 32 lines
Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/cpu-topology.c | 79 +++++++++++++++++++++++++++++++++ include/hw/s390x/cpu-topology.h | 1 + target/s390x/cpu-sysemu.c | 8 ++++ target/s390x/cpu.h | 1 + 4 files changed, 89 insertions(+)hw/s390x/cpu-topology.c +8
-0@@ -19,6 +19,7 @@#include "target/s390x/cpu.h" #include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/cpu-topology.h" +#include "migration/vmstate.h" S390Topology *s390_get_topology (void ) {@@ -132,6 +133,83 @@ static void s390_topology_reset(DeviceState *dev)s390_cpu_topology_reset (); } +/** +* cpu_topology_postload +* @opaque: a pointer to the S390Topology +* @version_id: version identifier +* +* We check that the topology is used or is not used +* on both side identically.
hw/s390x/cpu-topology.c static void s390_topology_reset(DeviceState *dev) · line 152
Quoted 10 lines
hw/s390x/cpu-topology.c +10
-0+* +* If the topology is in use we set the Modified Topology Change Report +* on the destination host. +*/ +static int cpu_topology_postload (void *opaque,int version_id) +{ +S390Topology *topo = opaque; +int ret; + +if (topo->topology_needed !=s390_has_feat (S390_FEAT_CONFIGURATION_TOPOLOGY )) {
Does this function even run if topology_needed is false?
In that case there is no data saved, so no reason to load it either. If so you can only check that both the source and the destination have the feature enabled. You would need to always send the topology VMSD in order to check that the feature is disabled.
Does qemu allow you to attempt to migrate to a host with another cpu model?
If it disallowes that you wouldn't need to do any checks, right?
Quoted 60 lines
hw/s390x/cpu-topology.c +60
-0+if (topo->topology_needed) { +error_report ("Topology facility is needed in destination" ); + }else { +error_report ("Topology facility can not be used in destination" ); + } +return -EINVAL ; + } + +/* We do not support CPU Topology, all is good */ +if (!s390_has_feat (S390_FEAT_CONFIGURATION_TOPOLOGY )) { +return 0 ; + } + +/* We support CPU Topology, set the MTCR */ + ret =s390_cpu_topology_mtcr_set (); +if (ret) { +error_report ("Failed to set MTCR: %s" ,strerror (-ret)); + } +return ret; +} + +/** +* cpu_topology_presave: +* @opaque: The pointer to the S390Topology +* +* Save the usage of the CPU Topology in the VM State. +*/ +static int cpu_topology_presave (void *opaque) +{ +S390Topology *topo = opaque; + + topo->topology_needed =s390_has_feat (S390_FEAT_CONFIGURATION_TOPOLOGY ); +return 0 ; +} + +/** +* cpu_topology_needed: +* @opaque: The pointer to the S390Topology +* +* If we use the CPU Topology on the source it will be needed on the destination. +*/ +static bool cpu_topology_needed (void *opaque) +{ +return s390_has_feat (S390_FEAT_CONFIGURATION_TOPOLOGY ); +} + + +const VMStateDescription vmstate_cpu_topology = { + .name ="cpu_topology" , + .version_id =1 , + .post_load = cpu_topology_postload, + .pre_save = cpu_topology_presave, + .minimum_version_id =1 , + .needed = cpu_topology_needed, + .fields = (VMStateField []) { +VMSTATE_BOOL (topology_needed, S390Topology), +VMSTATE_END_OF_LIST () + } +}; +
[...]
Re: [PATCH v9 07/10] s390x/cpu_topology: CPU topology migration
Quoted 5 lines · Janis Schoetterl-Glausch
Quoted 4 lines · Pierre Morel
The migration can only take place if both source and destination of the migration both use or both do not use the CPU topology facility.
Quoted 7 lines
Quoted 2 lines
We indicate a change in topology during migration postload for the case the topology changed between source and destination.
You always set the report bit after migration, right?
In the last series you actually migrated the bit.
Why the change? With the code you have actually migrating the bit isn't hard.
As for the moment the vCPU do not migrate from real CPU I thought based on the remark of Nico that there is no need to set the bit after a migration.
Quoted 49 lines
Quoted 32 lines
Signed-off-by Pierre Morel <[email protected]>--- hw/s390x/cpu-topology.c | 79 +++++++++++++++++++++++++++++++++ include/hw/s390x/cpu-topology.h | 1 + target/s390x/cpu-sysemu.c | 8 ++++ target/s390x/cpu.h | 1 + 4 files changed, 89 insertions(+)hw/s390x/cpu-topology.c +8
-0@@ -19,6 +19,7 @@#include "target/s390x/cpu.h" #include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/cpu-topology.h" +#include "migration/vmstate.h" S390Topology *s390_get_topology (void ) {@@ -132,6 +133,83 @@ static void s390_topology_reset(DeviceState *dev)s390_cpu_topology_reset (); } +/** +* cpu_topology_postload +* @opaque: a pointer to the S390Topology +* @version_id: version identifier +* +* We check that the topology is used or is not used +* on both side identically. hw/s390x/cpu-topology.c static void s390_topology_reset(DeviceState *dev) · line 152
Quoted 10 lines
hw/s390x/cpu-topology.c +10
-0+* +* If the topology is in use we set the Modified Topology Change Report +* on the destination host. +*/ +static int cpu_topology_postload (void *opaque,int version_id) +{ +S390Topology *topo = opaque; +int ret; + +if (topo->topology_needed !=s390_has_feat (S390_FEAT_CONFIGURATION_TOPOLOGY )) {Does this function even run if
topology_neededis false?
In that case there is no data saved, so no reason to load it either. If so you can only check that both the source and the destination have the feature enabled. You would need to always send the topology VMSD in order to check that the feature is disabled.
Quoted 3 lines
Does qemu allow you to attempt to migrate to a host with another cpu model?
If it disallowes that you wouldn't need to do any checks, right?
hum yes I must rework this
Thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v9 07/10] s390x/cpu_topology: CPU topology migration
Quoted 66 lines · Pierre Morel
Quoted 5 lines · Janis Schoetterl-Glausch
Quoted 4 lines · Pierre Morel
The migration can only take place if both source and destination of the migration both use or both do not use the CPU topology facility.
Quoted 7 lines
Quoted 2 lines
We indicate a change in topology during migration postload for the case the topology changed between source and destination.
You always set the report bit after migration, right?
In the last series you actually migrated the bit.
Why the change? With the code you have actually migrating the bit isn't hard.As for the moment the vCPU do not migrate from real CPU I thought based on the remark of Nico that there is no need to set the bit after a migration.
Quoted 48 lines
Quoted 31 lines
Signed-off-by Pierre Morel <[email protected]>---
hw/s390x/cpu-topology.c | 79 +++++++++++++++++++++++++++++++++include/hw/s390x/cpu-topology.h| 1 + target/s390x/cpu-sysemu.c | 8 ++++ target/s390x/cpu.h | 1 +4 files changed, 89 insertions(+)hw/s390x/cpu-topology.c +9
-0@@ -19,6 +19,7 @@ #include "target/s390x/cpu.h" #include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/cpu-topology.h" +#include "migration/vmstate.h" S390Topology *s390_get_topology(void) {@@ -132,6 +133,83 @@ static void s390_topology_reset(DeviceState *dev) s390_cpu_topology_reset(); } +/** +* cpu_topology_postload +* @opaque: a pointer to the S390Topology +* @version_id: version identifier +* +* We check that the topology is used or is not used +* on both side identically. +* hw/s390x/cpu-topology.c static void s390_topology_reset(DeviceState *dev) · line 152
Quoted 10 lines
hw/s390x/cpu-topology.c +9
-0+* If the topology is in use we set the Modified Topology Change Report +* on the destination host. +*/ +static int cpu_topology_postload (void *opaque,int version_id) +{ + S390Topology *topo = opaque; + int ret; + + if (topo->topology_needed !=s390_has_feat (S390_FEAT_CONFIGURATION_TOPOLOGY )) {Does this function even run if
topology_neededis false?
In that case there is no data saved, so no reason to load it either. If so you can only check that both the source and the destination have the feature enabled. You would need to always send the topology VMSD in order to check that the feature is disabled.
Quoted 10 lines
Quoted 3 lines
Does qemu allow you to attempt to migrate to a host with another cpu model?
If it disallowes that you wouldn't need to do any checks, right?hum yes I must rework this
Thanks,
Pierre
I think my answer was wrong but it helped to correct a bug:
- Only machines > 7.x will be able to use topology and it is correct that migration to another machine is not possible.
- However we keep the possibility to not use the topology in 7.x and we are not able to migrate if one end use the topology and the other do not.
So I think cpu_topology_needed() must return true (no test needed) but the implementation of presave and postload are OK.
Regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
PATCH v9 8/10 target/s390x: interception of PTF instruction
In reply to Pierre MorelWhen the host supports the CPU topology facility, the PTF instruction with function code 2 is interpreted by the SIE, provided that the userland hypervizor activates the interpretation by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension.
The PTF instructions with function code 0 and 1 are intercepted and must be emulated by the userland hypervizor.
--- hw/s390x/cpu-topology.c | 52 ++++++++++++++++++++++++++++++ include/hw/s390x/s390-virtio-ccw.h | 6 ++++ target/s390x/kvm/kvm.c | 13 ++++++++ 3 files changed, 71 insertions(+)
hw/s390x/cpu-topology.c +52 -0
@@ -20,6 +20,58 @@#include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/cpu-topology.h" #include "migration/vmstate.h" +#include "target/s390x/cpu.h" +#include "hw/s390x/s390-virtio-ccw.h" + +/* +* s390_handle_ptf: +* +* @register 1: contains the function code +* +* Function codes 0 and 1 handle the CPU polarization. +* We assume an horizontal topology, the only one supported currently +* by Linux, consequently we answer to function code 0, requesting +* horizontal polarization that it is already the current polarization +* and reject vertical polarization request without further explanation. +* +* Function code 2 is handling topology changes and is interpreted +* by the SIE. +*/ +void s390_handle_ptf (S390CPU *cpu,uint8_t r1,uintptr_t ra) +{ +CPUS390XState *env = &cpu->env; +uint64_t reg = env->regs[r1]; +uint8_t fc = reg &S390_TOPO_FC_MASK ; + +if (!s390_has_feat (S390_FEAT_CONFIGURATION_TOPOLOGY )) { +s390_program_interrupt (env,PGM_OPERATION , ra); +return ; + } + +if (env->psw.mask &PSW_MASK_PSTATE ) { +s390_program_interrupt (env,PGM_PRIVILEGED , ra); +return ; + } + +if (reg & ~S390_TOPO_FC_MASK ) { +s390_program_interrupt (env,PGM_SPECIFICATION , ra); +return ; + } + +switch (fc) { +case 0 :/* Horizontal polarization is already set */ + env->regs[r1] |=S390_PTF_REASON_DONE ; +setcc (cpu,2 ); +break ; +case 1 :/* Vertical polarization is not supported */ + env->regs[r1] |=S390_PTF_REASON_NONE ; +setcc (cpu,2 ); +break ; +default : +/* Note that fc == 2 is interpreted by the SIE */ +s390_program_interrupt (env,PGM_SPECIFICATION , ra); + } +}S390Topology *s390_get_topology (void ) {
include/hw/s390x/s390-virtio-ccw.h +6 -0
@@ -31,6 +31,12 @@ struct S390CcwMachineState {uint8_t loadparm[8 ]; }; +#define S390_PTF_REASON_NONE (0x00 << 8) +#define S390_PTF_REASON_DONE (0x01 << 8) +#define S390_PTF_REASON_BUSY (0x02 << 8) +#define S390_TOPO_FC_MASK 0xffUL +void s390_handle_ptf (S390CPU *cpu,uint8_t r1,uintptr_t ra); +struct S390CcwMachineClass {/*< private >*/ MachineClass parent_class;
target/s390x/kvm/kvm.c +13 -0
@@ -97,6 +97,7 @@#define PRIV_B9_EQBS 0x9c#define PRIV_B9_CLP 0xa0 +#define PRIV_B9_PTF 0xa2#define PRIV_B9_PCISTG 0xd0#define PRIV_B9_PCILG 0xd2#define PRIV_B9_RPCIT 0xd3
@@ -1463,6 +1464,15 @@ static int kvm_mpcifc_service_call(S390CPU *cpu, struct kvm_run *run) } } +static int kvm_handle_ptf (S390CPU *cpu,struct kvm_run *run) +{ +uint8_t r1 = (run->s390_sieic.ipb >>20 ) &0x0f ; + +s390_handle_ptf (cpu, r1,RA_IGNORED ); + +return 0 ; +} +static int handle_b9 (S390CPU *cpu,struct kvm_run *run,uint8_t ipa1) {int r =0 ;
@@ -1480,6 +1490,9 @@ static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)case PRIV_B9_RPCIT : r =kvm_rpcit_service_call (cpu, run);break ; +case PRIV_B9_PTF : + r =kvm_handle_ptf (cpu, run); +break ;case PRIV_B9_EQBS :/* just inject exception */ r =-1 ;
Signature · 1 line
2.31.1
Re: [PATCH v9 08/10] target/s390x: interception of PTF instruction
Quoted 9 lines · Pierre Morel
When the host supports the CPU topology facility, the PTF instruction with function code 2 is interpreted by the SIE, provided that the userland hypervizor activates the interpretation by using the
KVM_CAP_S390_CPU_TOPOLOGYKVM extension.The PTF instructions with function code 0 and 1 are intercepted and must be emulated by the userland hypervizor.
Signed-off-by Pierre Morel <[email protected]>
See note below.
Quoted 30 lines
--- hw/s390x/cpu-topology.c | 52 ++++++++++++++++++++++++++++++ include/hw/s390x/s390-virtio-ccw.h | 6 ++++ target/s390x/kvm/kvm.c | 13 ++++++++ 3 files changed, 71 insertions(+)hw/s390x/cpu-topology.c +16
-0@@ -20,6 +20,58 @@#include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/cpu-topology.h" #include "migration/vmstate.h" +#include "target/s390x/cpu.h" +#include "hw/s390x/s390-virtio-ccw.h" + +/* +* s390_handle_ptf: +* +* @register 1: contains the function code +* +* Function codes 0 and 1 handle the CPU polarization. +* We assume an horizontal topology, the only one supported currently +* by Linux, consequently we answer to function code 0, requesting +* horizontal polarization that it is already the current polarization +* and reject vertical polarization request without further explanation. +* +* Function code 2 is handling topology changes and is interpreted +* by the SIE.
hw/s390x/cpu-topology.c line 48
Quoted 10 lines
hw/s390x/cpu-topology.c +10
-0+*/ +void s390_handle_ptf (S390CPU *cpu,uint8_t r1,uintptr_t ra) +{ +CPUS390XState *env = &cpu->env; +uint64_t reg = env->regs[r1]; +uint8_t fc = reg &S390_TOPO_FC_MASK ; + +if (!s390_has_feat (S390_FEAT_CONFIGURATION_TOPOLOGY )) { +s390_program_interrupt (env,PGM_OPERATION , ra); +return ;
I'm either expecting this function to return -1 here...
Quoted 26 lines
hw/s390x/cpu-topology.c +26
-0+ } + +if (env->psw.mask &PSW_MASK_PSTATE ) { +s390_program_interrupt (env,PGM_PRIVILEGED , ra); +return ; + } + +if (reg & ~S390_TOPO_FC_MASK ) { +s390_program_interrupt (env,PGM_SPECIFICATION , ra); +return ; + } + +switch (fc) { +case 0 :/* Horizontal polarization is already set */ + env->regs[r1] |=S390_PTF_REASON_DONE ; +setcc (cpu,2 ); +break ; +case 1 :/* Vertical polarization is not supported */ + env->regs[r1] |=S390_PTF_REASON_NONE ; +setcc (cpu,2 ); +break ; +default : +/* Note that fc == 2 is interpreted by the SIE */ +s390_program_interrupt (env,PGM_SPECIFICATION , ra); + } +}
[...]
hw/s390x/cpu-topology.c
Quoted 6 lines
hw/s390x/cpu-topology.c +5
-0+static int kvm_handle_ptf (S390CPU *cpu,struct kvm_run *run) +{ +uint8_t r1 = (run->s390_sieic.ipb >>20 ) &0x0f ; + +s390_handle_ptf (cpu, r1,RA_IGNORED );
... and this being returned here...
hw/s390x/cpu-topology.c
Quoted 2 lines
hw/s390x/cpu-topology.c +2
-0+ +return 0 ;
... or this function being void.
Quoted 15 lines
hw/s390x/cpu-topology.c +5
-0+} +static int handle_b9 (S390CPU *cpu,struct kvm_run *run,uint8_t ipa1) {int r =0 ;@@ -1480,6 +1490,9 @@ static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)case PRIV_B9_RPCIT : r =kvm_rpcit_service_call (cpu, run);break ; +case PRIV_B9_PTF : + r =kvm_handle_ptf (cpu, run); +break ;case PRIV_B9_EQBS :/* just inject exception */ r =-1 ;
Re: [PATCH v9 08/10] target/s390x: interception of PTF instruction
Quoted 6 lines · Janis Schoetterl-Glausch
Quoted 5 lines · Pierre Morel
When the host supports the CPU topology facility, the PTF instruction with function code 2 is interpreted by the SIE, provided that the userland hypervizor activates the interpretation by using the
KVM_CAP_S390_CPU_TOPOLOGYKVM extension.
Quoted 6 lines
Quoted 4 lines
The PTF instructions with function code 0 and 1 are intercepted and must be emulated by the userland hypervizor.
Signed-off-by Pierre Morel <[email protected]>Reviewed-by Janis Schoetterl-Glausch <[email protected]>
Thanks
Quoted 80 lines
See note below.
Quoted 30 lines
--- hw/s390x/cpu-topology.c | 52 ++++++++++++++++++++++++++++++ include/hw/s390x/s390-virtio-ccw.h | 6 ++++ target/s390x/kvm/kvm.c | 13 ++++++++ 3 files changed, 71 insertions(+)hw/s390x/cpu-topology.c +16
-0@@ -20,6 +20,58 @@#include "hw/s390x/s390-virtio-ccw.h" #include "hw/s390x/cpu-topology.h" #include "migration/vmstate.h" +#include "target/s390x/cpu.h" +#include "hw/s390x/s390-virtio-ccw.h" + +/* +* s390_handle_ptf: +* +* @register 1: contains the function code +* +* Function codes 0 and 1 handle the CPU polarization. +* We assume an horizontal topology, the only one supported currently +* by Linux, consequently we answer to function code 0, requesting +* horizontal polarization that it is already the current polarization +* and reject vertical polarization request without further explanation. +* +* Function code 2 is handling topology changes and is interpreted +* by the SIE. hw/s390x/cpu-topology.c line 48
Quoted 10 lines
hw/s390x/cpu-topology.c +10
-0+*/ +void s390_handle_ptf (S390CPU *cpu,uint8_t r1,uintptr_t ra) +{ +CPUS390XState *env = &cpu->env; +uint64_t reg = env->regs[r1]; +uint8_t fc = reg &S390_TOPO_FC_MASK ; + +if (!s390_has_feat (S390_FEAT_CONFIGURATION_TOPOLOGY )) { +s390_program_interrupt (env,PGM_OPERATION , ra); +return ;I'm either expecting this function to return -1 here...
Quoted 26 lines
hw/s390x/cpu-topology.c +26
-0+ } + +if (env->psw.mask &PSW_MASK_PSTATE ) { +s390_program_interrupt (env,PGM_PRIVILEGED , ra); +return ; + } + +if (reg & ~S390_TOPO_FC_MASK ) { +s390_program_interrupt (env,PGM_SPECIFICATION , ra); +return ; + } + +switch (fc) { +case 0 :/* Horizontal polarization is already set */ + env->regs[r1] |=S390_PTF_REASON_DONE ; +setcc (cpu,2 ); +break ; +case 1 :/* Vertical polarization is not supported */ + env->regs[r1] |=S390_PTF_REASON_NONE ; +setcc (cpu,2 ); +break ; +default : +/* Note that fc == 2 is interpreted by the SIE */ +s390_program_interrupt (env,PGM_SPECIFICATION , ra); + } +}[...]
hw/s390x/cpu-topology.c
Quoted 6 lines
hw/s390x/cpu-topology.c +5
-0+static int kvm_handle_ptf (S390CPU *cpu,struct kvm_run *run) +{ +uint8_t r1 = (run->s390_sieic.ipb >>20 ) &0x0f ; + +s390_handle_ptf (cpu, r1,RA_IGNORED );... and this being returned here...
Quoted 5 lines
hw/s390x/cpu-topology.c
Quoted 2 lines
hw/s390x/cpu-topology.c +2
-0+ +return 0 ;... or this function being void.
Quoted 1 line
+}
Yes right, thanks
Regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
PATCH v9 9/10 s390x/cpu_topology: activating CPU topology
In reply to Pierre MorelStarting with a new machine, s390-virtio-ccw-7.2, the machine property topology-disable is set to false while it is kept to true for older machine.
This allows migrating older machine without disabling the ctop CPU feature for older machine, thus keeping existing start scripts.
The KVM capability, KVM_CAP_S390_CPU_TOPOLOGY is used to activate the S390_FEAT_CONFIGURATION_TOPOLOGY feature and the topology facility for the guest in the case the topology is not disabled.
--- hw/core/machine.c | 5 +++ hw/s390x/s390-virtio-ccw.c | 55 ++++++++++++++++++++++++++---- include/hw/boards.h | 3 ++ include/hw/s390x/s390-virtio-ccw.h | 1 + target/s390x/kvm/kvm.c | 14 ++++++++ 5 files changed, 72 insertions(+), 6 deletions(-)
hw/core/machine.c +5 -0
@@ -40,6 +40,11 @@#include "hw/virtio/virtio-pci.h" #include "qom/object_interfaces.h" +GlobalProperty hw_compat_7_1[] = { + {"s390x-cpu" ,"ctop" ,"off" }, +}; +const size_t hw_compat_7_1_len =G_N_ELEMENTS (hw_compat_7_1); +GlobalProperty hw_compat_7_0[] = { {"arm-gicv3-common" ,"force-8-bit-prio" ,"on" }, {"nvme-ns" ,"eui64-default" ,"on" },
hw/s390x/s390-virtio-ccw.c +49 -6
92 lines not on this page · raw message
include/hw/boards.h +3 -0
@@ -387,6 +387,9 @@ struct MachineState { } \ type_init(machine_initfn##_register_types) +extern GlobalProperty hw_compat_7_1[]; +extern const size_t hw_compat_7_1_len; +extern GlobalProperty hw_compat_7_0[];extern const size_t hw_compat_7_0_len;
include/hw/s390x/s390-virtio-ccw.h +1 -0
@@ -28,6 +28,7 @@ struct S390CcwMachineState {bool dea_key_wrap;bool pv;bool zpcii_disable; +bool topology_disable;uint8_t loadparm[8 ]; };
target/s390x/kvm/kvm.c +14 -0
@@ -2385,6 +2385,7 @@ bool kvm_s390_cpu_models_supported(void)void kvm_s390_get_host_cpu_model (S390CPUModel *model,Error **errp) { +S390CcwMachineState *ms =S390_CCW_MACHINE (qdev_get_machine ());struct kvm_s390_vm_cpu_machine prop = {};struct kvm_device_attr attr = { .group =KVM_S390_VM_CPU_MODEL ,
@@ -2466,6 +2467,19 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp)set_bit (S390_FEAT_UNPACK , model->features); } +/* +* If we have the CPU Topology implemented in KVM activate +* the CPU TOPOLOGY feature. +*/ +if ((!ms->topology_disable) && +kvm_check_extension (kvm_state,KVM_CAP_S390_CPU_TOPOLOGY )) { +if (kvm_vm_enable_cap (kvm_state,KVM_CAP_S390_CPU_TOPOLOGY ,0 ) <0 ) { +error_setg (errp,"KVM: Error enabling KVM_CAP_S390_CPU_TOPOLOGY" ); +return ; + } +set_bit (S390_FEAT_CONFIGURATION_TOPOLOGY , model->features); + } +/* We emulate a zPCI bus and AEN, therefore we don't need HW support */ set_bit (S390_FEAT_ZPCI , model->features);set_bit (S390_FEAT_ADAPTER_EVENT_NOTIFICATION , model->features);
Signature · 1 line
2.31.1
Re: [PATCH v9 09/10] s390x/cpu_topology: activating CPU topology
Quoted 6 lines · Pierre Morel
Starting with a new machine, s390-virtio-ccw-7.2, the machine property topology-disable is set to false while it is kept to true for older machine.
This allows migrating older machine without disabling the ctop CPU feature for older machine, thus keeping existing start scripts.
Quoted 7 lines
The KVM capability,
KVM_CAP_S390_CPU_TOPOLOGYis used to activate theS390_FEAT_CONFIGURATION_TOPOLOGYfeature and the topology facility for the guest in the case the topology is not disabled.Signed-off-by Pierre Morel <[email protected]>---
Sorry, forget this patch I made a stupid rebase error and lost half of the code.
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v9 09/10] s390x/cpu_topology: activating CPU topology
In reply to Pierre MorelQuoted 3 lines · Pierre Morel
Starting with a new machine, s390-virtio-ccw-7.2, the machine property topology-disable is set to false while it is kept to true for older machine.
We probably need a machine class option also because we don't want this to be possible :
-M s390-ccw-virtio-7.1,topology-disable=false
Quoted 173 lines
This allows migrating older machine without disabling the ctop CPU feature for older machine, thus keeping existing start scripts.
The KVM capability,
KVM_CAP_S390_CPU_TOPOLOGYis used to activate theS390_FEAT_CONFIGURATION_TOPOLOGYfeature and the topology facility for the guest in the case the topology is not disabled.Signed-off-by Pierre Morel <[email protected]>--- hw/core/machine.c | 5 +++ hw/s390x/s390-virtio-ccw.c | 55 ++++++++++++++++++++++++++---- include/hw/boards.h | 3 ++ include/hw/s390x/s390-virtio-ccw.h | 1 + target/s390x/kvm/kvm.c | 14 ++++++++ 5 files changed, 72 insertions(+), 6 deletions(-)hw/core/machine.c +5
-0@@ -40,6 +40,11 @@#include "hw/virtio/virtio-pci.h" #include "qom/object_interfaces.h" +GlobalProperty hw_compat_7_1[] = { + {"s390x-cpu" ,"ctop" ,"off" }, +}; +const size_t hw_compat_7_1_len =G_N_ELEMENTS (hw_compat_7_1); +GlobalProperty hw_compat_7_0[] = { {"arm-gicv3-common" ,"force-8-bit-prio" ,"on" }, {"nvme-ns" ,"eui64-default" ,"on" },hw/s390x/s390-virtio-ccw.c +49
-6@@ -249,11 +249,16 @@ static void ccw_init(MachineState *machine)/* init memory + setup max page size. Required for the CPU model */ s390_memory_init (machine->ram); -/* Adding the topology must be done before CPU intialization*/ - dev =qdev_new (TYPE_S390_CPU_TOPOLOGY ); -object_property_add_child (qdev_get_machine (),TYPE_S390_CPU_TOPOLOGY , -OBJECT (dev)); -sysbus_realize_and_unref (SYS_BUS_DEVICE (dev), &error_fatal); +/* +* Adding the topology must be done before CPU intialization but +* only in the case it is not disabled for migration purpose. +*/ +if (!S390_CCW_MACHINE (machine)->topology_disable) { + dev =qdev_new (TYPE_S390_CPU_TOPOLOGY ); +object_property_add_child (qdev_get_machine (),TYPE_S390_CPU_TOPOLOGY , +OBJECT (dev)); +sysbus_realize_and_unref (SYS_BUS_DEVICE (dev), &error_fatal); + }/* init CPUs (incl. CPU model) early so s390_has_feature() works */ s390_init_cpus (machine);@@ -676,6 +681,21 @@ static inline void machine_set_zpcii_disable(Object *obj, bool value, ms->zpcii_disable = value; } +static inline bool machine_get_topology_disable (Object *obj,Error **errp) +{ +S390CcwMachineState *ms =S390_CCW_MACHINE (obj); + +return ms->topology_disable; +} + +static inline void machine_set_topology_disable (Object *obj,bool value, +Error **errp) +{ +S390CcwMachineState *ms =S390_CCW_MACHINE (obj); + + ms->topology_disable = value; +} +static S390CcwMachineClass *current_mc;/* @@ -778,6 +798,13 @@ static inline void s390_machine_initfn(Object *obj)object_property_set_description (obj,"zpcii-disable" ,"disable zPCI interpretation facilties" );object_property_set_bool (obj,"zpcii-disable" , false,NULL ); + +object_property_add_bool (obj,"topology-disable" , + machine_get_topology_disable, + machine_set_topology_disable); +object_property_set_description (obj,"topology-disable" , +"disable zPCI interpretation facilties" ); +object_property_set_bool (obj,"topology-disable" , false,NULL ); }static const TypeInfo ccw_machine_info = {@@ -830,14 +857,29 @@ bool css_migration_enabled(void) } \ type_init(ccw_machine_register_##suffix) +static void ccw_machine_7_2_instance_options(MachineState *machine) +{ +} + +static void ccw_machine_7_2_class_options (MachineClass *mc) +{ +} +DEFINE_CCW_MACHINE(7_2, "7.2", true); +static void ccw_machine_7_1_instance_options (MachineState *machine) { +S390CcwMachineState *ms =S390_CCW_MACHINE (machine); + +ccw_machine_7_2_instance_options (machine); + ms->topology_disable = true; }static void ccw_machine_7_1_class_options (MachineClass *mc) { +ccw_machine_7_2_class_options (mc); +compat_props_add (mc->compat_props, hw_compat_7_1, hw_compat_7_1_len); } -DEFINE_CCW_MACHINE(7_1, "7.1", true); +DEFINE_CCW_MACHINE(7_1, "7.1", false);static void ccw_machine_7_0_instance_options (MachineState *machine) {@@ -847,6 +889,7 @@ static void ccw_machine_7_0_instance_options(MachineState *machine)ccw_machine_7_1_instance_options (machine);s390_set_qemu_cpu_model (0x8561 ,15 ,1 , qemu_cpu_feat); ms->zpcii_disable = true; + }static void ccw_machine_7_0_class_options (MachineClass *mc)include/hw/boards.h +3
-0@@ -387,6 +387,9 @@ struct MachineState { } \ type_init(machine_initfn##_register_types) +extern GlobalProperty hw_compat_7_1[]; +extern const size_t hw_compat_7_1_len; +extern GlobalProperty hw_compat_7_0[];extern const size_t hw_compat_7_0_len;include/hw/s390x/s390-virtio-ccw.h +1
-0@@ -28,6 +28,7 @@ struct S390CcwMachineState {bool dea_key_wrap;bool pv;bool zpcii_disable; +bool topology_disable;uint8_t loadparm[8 ]; };target/s390x/kvm/kvm.c +1
-0@@ -2385,6 +2385,7 @@ bool kvm_s390_cpu_models_supported(void)void kvm_s390_get_host_cpu_model (S390CPUModel *model,Error **errp) { +S390CcwMachineState *ms =S390_CCW_MACHINE (qdev_get_machine ());struct kvm_s390_vm_cpu_machine prop = {};struct kvm_device_attr attr = { .group =KVM_S390_VM_CPU_MODEL ,
target/s390x/kvm/kvm.c void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp) · line 2474
Quoted 8 lines
target/s390x/kvm/kvm.c +5
-0@@ -2466,6 +2467,19 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp)set_bit (S390_FEAT_UNPACK , model->features); } +/* +* If we have the CPU Topology implemented in KVM activate +* the CPU TOPOLOGY feature. +*/ + if ((!ms->topology_disable) &&
'topology_disable' is a platform level configuration. May be instead, the feature could be cleared at the machine level ?
Thanks,
C.
Quoted 11 lines
target/s390x/kvm/kvm.c +8
-0+ kvm_check_extension(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY)) { +if (kvm_vm_enable_cap (kvm_state,KVM_CAP_S390_CPU_TOPOLOGY ,0 ) <0 ) { +error_setg (errp,"KVM: Error enabling KVM_CAP_S390_CPU_TOPOLOGY" ); +return ; + } +set_bit (S390_FEAT_CONFIGURATION_TOPOLOGY , model->features); + } +/* We emulate a zPCI bus and AEN, therefore we don't need HW support */ set_bit (S390_FEAT_ZPCI , model->features);set_bit (S390_FEAT_ADAPTER_EVENT_NOTIFICATION , model->features);
Re: [PATCH v9 09/10] s390x/cpu_topology: activating CPU topology
Hi Cedric,
Quoted 9 lines · Cédric Le Goater
Quoted 3 lines · Pierre Morel
Starting with a new machine, s390-virtio-ccw-7.2, the machine property topology-disable is set to false while it is kept to true for older machine.
We probably need a machine class option also because we don't want this to be possible :
-M s390-ccw-virtio-7.1,topology-disable=false
hum, you are right it has little interest to do this.
Quoted 171 lines
Quoted 171 lines
This allows migrating older machine without disabling the ctop CPU feature for older machine, thus keeping existing start scripts.
The KVM capability,
KVM_CAP_S390_CPU_TOPOLOGYis used to activate theS390_FEAT_CONFIGURATION_TOPOLOGYfeature and the topology facility for the guest in the case the topology is not disabled.Signed-off-by Pierre Morel <[email protected]>---
hw/core/machine.c | 5 +++ hw/s390x/s390-virtio-ccw.c | 55 ++++++++++++++++++++++++++---- include/hw/boards.h | 3 ++include/hw/s390x/s390-virtio-ccw.h| 1 + target/s390x/kvm/kvm.c | 14 ++++++++5 files changed, 72 insertions(+), 6 deletions(-)hw/core/machine.c +5
-0@@ -40,6 +40,11 @@ #include "hw/virtio/virtio-pci.h" #include "qom/object_interfaces.h" +GlobalProperty hw_compat_7_1[] = { + { "s390x-cpu", "ctop", "off"}, +}; +const size_t hw_compat_7_1_len =G_N_ELEMENTS (hw_compat_7_1); + GlobalProperty hw_compat_7_0[] = { { "arm-gicv3-common", "force-8-bit-prio", "on" }, { "nvme-ns", "eui64-default", "on"},hw/s390x/s390-virtio-ccw.c +49
-6@@ -249,11 +249,16 @@ static void ccw_init(MachineState *machine) /* init memory + setup max page size. Required for the CPU model */ s390_memory_init(machine->ram); - /* Adding the topology must be done before CPU intialization*/ - dev = qdev_new(TYPE_S390_CPU_TOPOLOGY); - object_property_add_child(qdev_get_machine(), TYPE_S390_CPU_TOPOLOGY, - OBJECT(dev)); - sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal); + /* + * Adding the topology must be done before CPU intialization but + * only in the case it is not disabled for migration purpose. + */ + if (!S390_CCW_MACHINE(machine)->topology_disable) { + dev = qdev_new(TYPE_S390_CPU_TOPOLOGY); + object_property_add_child(qdev_get_machine(),TYPE_S390_CPU_TOPOLOGY , + OBJECT(dev)); + sysbus_realize_and_unref(SYS_BUS_DEVICE(dev), &error_fatal); + } /* init CPUs (incl. CPU model) early so s390_has_feature() works */ s390_init_cpus(machine);@@ -676,6 +681,21 @@ static inline void machine_set_zpcii_disable(Object *obj, bool value, ms->zpcii_disable = value; } +static inline bool machine_get_topology_disable(Object *obj, Error **errp) +{ + S390CcwMachineState *ms = S390_CCW_MACHINE(obj); + + return ms->topology_disable; +} + +static inline void machine_set_topology_disable (Object *obj,bool value, + Error **errp) +{ + S390CcwMachineState *ms = S390_CCW_MACHINE(obj); + + ms->topology_disable = value; +} + static S390CcwMachineClass *current_mc; /*@@ -778,6 +798,13 @@ static inline void s390_machine_initfn(Object *obj) object_property_set_description(obj, "zpcii-disable", "disable zPCI interpretation facilties"); object_property_set_bool(obj, "zpcii-disable", false, NULL); + + object_property_add_bool(obj, "topology-disable", + machine_get_topology_disable, + machine_set_topology_disable); + object_property_set_description(obj, "topology-disable", + "disable zPCI interpretation facilties"); + object_property_set_bool(obj, "topology-disable", false, NULL); } static const TypeInfo ccw_machine_info = {@@ -830,14 +857,29 @@ bool css_migration_enabled(void) } \ type_init(ccw_machine_register_##suffix) +static void ccw_machine_7_2_instance_options (MachineState *machine) +{ +} + +static void ccw_machine_7_2_class_options (MachineClass *mc) +{ +} +DEFINE_CCW_MACHINE(7_2, "7.2", true); + static void ccw_machine_7_1_instance_options(MachineState *machine) { + S390CcwMachineState *ms = S390_CCW_MACHINE(machine); + + ccw_machine_7_2_instance_options(machine); + ms->topology_disable = true; } static void ccw_machine_7_1_class_options(MachineClass *mc) { + ccw_machine_7_2_class_options(mc); + compat_props_add(mc->compat_props, hw_compat_7_1, hw_compat_7_1_len); } -DEFINE_CCW_MACHINE(7_1, "7.1", true); +DEFINE_CCW_MACHINE(7_1, "7.1", false); static void ccw_machine_7_0_instance_options(MachineState *machine) {@@ -847,6 +889,7 @@ static void ccw_machine_7_0_instance_options(MachineState *machine) ccw_machine_7_1_instance_options(machine); s390_set_qemu_cpu_model(0x8561, 15, 1, qemu_cpu_feat); ms->zpcii_disable = true; + } static void ccw_machine_7_0_class_options(MachineClass *mc)include/hw/boards.h +3
-0@@ -387,6 +387,9 @@ struct MachineState { } \ type_init(machine_initfn##_register_types) +extern GlobalProperty hw_compat_7_1[]; +extern const size_t hw_compat_7_1_len; + extern GlobalProperty hw_compat_7_0[]; extern const size_t hw_compat_7_0_len;include/hw/s390x/s390-virtio-ccw.h +1
-0@@ -28,6 +28,7 @@ struct S390CcwMachineState { bool dea_key_wrap; bool pv; bool zpcii_disable; + bool topology_disable; uint8_t loadparm[8]; };target/s390x/kvm/kvm.c +1
-0@@ -2385,6 +2385,7 @@ bool kvm_s390_cpu_models_supported(void) void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp) { + S390CcwMachineState *ms = S390_CCW_MACHINE(qdev_get_machine()); struct kvm_s390_vm_cpu_machine prop = {}; struct kvm_device_attr attr = { .group = KVM_S390_VM_CPU_MODEL,@@ -2466,6 +2467,19 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp) set_bit(S390_FEAT_UNPACK, model->features); }
Quoted 8 lines
target/s390x/kvm/kvm.c void kvm_s390_get_host_cpu_model(S390CPUModel *model, Error **errp) · line 2474
Quoted 5 lines
target/s390x/kvm/kvm.c +5
-0+ /* + * If we have the CPU Topology implemented in KVM activate + * the CPU TOPOLOGY feature. + */ + if ((!ms->topology_disable) &&'
topology_disable' is a platform level configuration. May be instead, the feature could be cleared at the machine level ?
thanks, I will try this way.
regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
PATCH v9 10/10 docs/s390x: document s390x cpu topology
In reply to Pierre MorelAdd some basic examples for the definition of cpu topology in s390x.
--- docs/system/s390x/cpu_topology.rst | 88 ++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) create mode 100644 docs/system/s390x/cpu_topology.rst
docs/system/s390x/cpu_topology.rst new file +88 -0
@@ -0,0 +1,88 @@ +CPU Topology on s390x +===================== + +CPU Topology on S390x provides up to 4 levels of topology containers: +drawers, books, sockets and CPUs. +While the three higher level containers, Containers Topology List Entries, +(Containers TLE) define a tree hierarchy, the lowest level of topology +definition, the CPU Topology List Entry (CPU TLE), provides the placement +of the CPUs inside the last container. + +Prerequisites +------------- + +To use CPU Topology a Linux QEMU/KVM machine providing the CPU Topology facility +(STFLE bit 11) is required. + +However, since this facility has been enabled by default in an early version, +the capability ``KVM_CAP_S390_CPU_TOPOLOGY`` is needed to indicate to KVM +that QEMU support CPU Topology. + +Indicating the CPU topology to the Virtual Machine +-------------------------------------------------- + +The CPU Topology, number of drawers, number of books per drawers, number of +sockets per book and number of cores per sockets is specified with the +``-smp`` qemu command arguments. + +Like in : + +.. code-block:: sh + -smp cpus=1,drawers=3,books=4,sockets=2,cores=8,maxcpus=192 + +If drawers or books are not specified, their default to 1. + +New CPUs can be plugged using the device_add hmp command like in: + +.. code-block:: sh + (qemu) device_add host-s390x-cpu,core-id=9 + +The core-id defines the placement of the core in the topology by +starting with core 0 in socket 0, book 0 and drawer 0 up to the maximum +core number of the last socket of the last book in the last drawer. + +In the example above: + +* the core with ID 9 will be placed in container (0,0,1), as core 9 + of CPU TLE 0 of socket 1 in book 0 from drawer 0. +* the core ID 0 is defined by the -smp cpus=1 command and will be + placed as core 0 in CPU TLE 0 of container (0,0,0) + +Note that the core ID is machine wide and the CPU TLE masks provided +by the STSI instruction will be: + +* in socket 0: 0x80000000 (core id 0) +* in socket 1: 0x00400000 (core id 9) + +Indicating the CPU topology to the Guest +---------------------------------------- + +The guest can query for topology changes using the PTF instruction. +In case of a topology change it can request the new topology by issuing +STSI instructions specifying the level of detail required, drawer with +STSI(15.1.4) or books STSI(15.1.3). + +The virtual machine will fill the provided buffer with the count of +drawers (MAG4), books per drawer (MAG3), sockets per book (MAG2) and +cores per socket (MAG1). + +Note that the STSI(15.1.2) is special in two ways: + +* When the firmware detect a change in the values calculated for STSI(15.1.2) + it will trigger the report of the topology change for the PTF instruction. + +Migration +--------- + +For virtio-ccw machines older than s390-virtio-ccw-7.2, CPU Topoogy is +by default disabled. + +CPU Topoogy is by default enabled for s390-virtio-ccw-7.2 and newer machines. + +Enabling the CPU topology on older Machine is done by setting the global +option ''topology-disable'' to false before enabling cpu topology with the +cpu feature "ctop" like in: + +.. code-block:: sh + -machine s390-ccw-virtio-3.0,accel=kvm,topology-disable=false + -cpu z14,ctop=on
Signature · 1 line
2.31.1
Re: [PATCH v9 10/10] docs/s390x: document s390x cpu topology
Quoted 15 lines · Pierre Morel
Add some basic examples for the definition of cpu topology in s390x.
Signed-off-by Pierre Morel <[email protected]>--- docs/system/s390x/cpu_topology.rst | 88 ++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) create mode 100644 docs/system/s390x/cpu_topology.rstdocs/system/s390x/cpu_topology.rst new file +0
-0
docs/system/s390x/cpu_topology.rst line 9
Quoted 9 lines
docs/system/s390x/cpu_topology.rst new file +9
-0@@ -0,0 +1,88 @@ +CPU Topology on s390x +===================== + +CPU Topology on S390x provides up to 4 levels of topology containers: +drawers, books, sockets and CPUs. +While the three higher level containers, Containers Topology List Entries, +(Containers TLE) define a tree hierarchy, the lowest level of topology +definition, the CPU Topology List Entry (CPU TLE), provides the placement +of the CPUs inside the last container.
inside the parent container
docs/system/s390x/cpu_topology.rst line 19
Quoted 10 lines
docs/system/s390x/cpu_topology.rst new file +10
-0+ +Prerequisites +------------- + +To use CPU Topology a Linux QEMU/KVM machine providing the CPU Topology facility +(STFLE bit 11) is required. + +However, since this facility has been enabled by default in an early version, +the capability ``KVM_CAP_S390_CPU_TOPOLOGY`` is needed to indicate to KVM +that QEMU support CPU Topology.
I don't understand this paragraph. Early version of what?
Quoted 43 lines
docs/system/s390x/cpu_topology.rst new file +43
-0+ +Indicating the CPU topology to the Virtual Machine +-------------------------------------------------- + +The CPU Topology, number of drawers, number of books per drawers, number of +sockets per book and number of cores per sockets is specified with the +``-smp`` qemu command arguments. + +Like in : + +.. code-block:: sh + -smp cpus=1,drawers=3,books=4,sockets=2,cores=8,maxcpus=192 + +If drawers or books are not specified, their default to 1. + +New CPUs can be plugged using the device_add hmp command like in: + +.. code-block:: sh + (qemu) device_add host-s390x-cpu,core-id=9 + +The core-id defines the placement of the core in the topology by +starting with core 0 in socket 0, book 0 and drawer 0 up to the maximum +core number of the last socket of the last book in the last drawer. + +In the example above: + +* the core with ID 9 will be placed in container (0,0,1), as core 9 + of CPU TLE 0 of socket 1 in book 0 from drawer 0. +* the core ID 0 is defined by the -smp cpus=1 command and will be + placed as core 0 in CPU TLE 0 of container (0,0,0) + +Note that the core ID is machine wide and the CPU TLE masks provided +by the STSI instruction will be: + +* in socket 0: 0x80000000 (core id 0) +* in socket 1: 0x00400000 (core id 9) + +Indicating the CPU topology to the Guest +---------------------------------------- + +The guest can query for topology changes using the PTF instruction. +In case of a topology change it can request the new topology by issuing +STSI instructions specifying the level of detail required, drawer with
docs/system/s390x/cpu_topology.rst line 72
Quoted 10 lines
docs/system/s390x/cpu_topology.rst new file +10
-0+STSI(15.1.4) or books STSI(15.1.3). + +The virtual machine will fill the provided buffer with the count of +drawers (MAG4), books per drawer (MAG3), sockets per book (MAG2) and +cores per socket (MAG1). + +Note that the STSI(15.1.2) is special in two ways: + +* When the firmware detect a change in the values calculated for STSI(15.1.2) + it will trigger the report of the topology change for the PTF instruction.
I don't know if we need this section, after all documenting this is the job of the principles of operation. You could just refer to the relevant sections.
Quoted 16 lines
docs/system/s390x/cpu_topology.rst new file +16
-0+ +Migration +--------- + +For virtio-ccw machines older than s390-virtio-ccw-7.2, CPU Topoogy is +by default disabled. + +CPU Topoogy is by default enabled for s390-virtio-ccw-7.2 and newer machines. + +Enabling the CPU topology on older Machine is done by setting the global +option ''topology-disable'' to false before enabling cpu topology with the +cpu feature "ctop" like in: + +.. code-block:: sh + -machine s390-ccw-virtio-3.0,accel=kvm,topology-disable=false + -cpu z14,ctop=on
Re: [PATCH v9 10/10] docs/s390x: document s390x cpu topology
Quoted 19 lines · Janis Schoetterl-Glausch
Quoted 18 lines · Pierre Morel
Add some basic examples for the definition of cpu topology in s390x.
Signed-off-by Pierre Morel <[email protected]>--- docs/system/s390x/cpu_topology.rst | 88 ++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+)create mode 100644
docs/system/s390x/cpu_topology.rstdocs/system/s390x/cpu_topology.rst new file +3
-0@@ -0,0 +1,88 @@ +CPU Topology on s390x +===================== +
Quoted 8 lines
docs/system/s390x/cpu_topology.rst line 9
Quoted 6 lines
docs/system/s390x/cpu_topology.rst new file +6
-0+CPU Topology on S390x provides up to 4 levels of topology containers: +drawers, books, sockets and CPUs. +While the three higher level containers, Containers Topology List Entries, +(Containers TLE) define a tree hierarchy, the lowest level of topology +definition, the CPU Topology List Entry (CPU TLE), provides the placement +of the CPUs inside the last container.inside the parent container
OK
Quoted 4 lines
Quoted 4 lines
docs/system/s390x/cpu_topology.rst new file +4
-0+ +Prerequisites +------------- +
Quoted 8 lines
docs/system/s390x/cpu_topology.rst line 19
Quoted 6 lines
docs/system/s390x/cpu_topology.rst new file +6
-0+To use CPU Topology a Linux QEMU/KVM machine providing the CPU Topology facility +(STFLE bit 11) is required. + +However, since this facility has been enabled by default in an early version, +the capability ``KVM_CAP_S390_CPU_TOPOLOGY`` is needed to indicate to KVM +that QEMU support CPU Topology.I don't understand this paragraph. Early version of what?
of QEMU, I add this
Quoted 49 lines
Quoted 49 lines
docs/system/s390x/cpu_topology.rst new file +49
-0+ +Indicating the CPU topology to the Virtual Machine +-------------------------------------------------- + +The CPU Topology, number of drawers, number of books per drawers, number of +sockets per book and number of cores per sockets is specified with the +``-smp`` qemu command arguments. + +Like in : + +.. code-block:: sh + -smp cpus=1,drawers=3,books=4,sockets=2,cores=8,maxcpus=192 + +If drawers or books are not specified, their default to 1. + +New CPUs can be plugged using the device_add hmp command like in: + +.. code-block:: sh + (qemu) device_add host-s390x-cpu,core-id=9 + +The core-id defines the placement of the core in the topology by +starting with core 0 in socket 0, book 0 and drawer 0 up to the maximum +core number of the last socket of the last book in the last drawer. + +In the example above: + +* the core with ID 9 will be placed in container (0,0,1), as core 9 + of CPU TLE 0 of socket 1 in book 0 from drawer 0. +* the core ID 0 is defined by the -smp cpus=1 command and will be + placed as core 0 in CPU TLE 0 of container (0,0,0) + +Note that the core ID is machine wide and the CPU TLE masks provided +by the STSI instruction will be: + +* in socket 0: 0x80000000 (core id 0) +* in socket 1: 0x00400000 (core id 9) + +Indicating the CPU topology to the Guest +---------------------------------------- + +The guest can query for topology changes using the PTF instruction. +In case of a topology change it can request the new topology by issuing +STSI instructions specifying the level of detail required, drawer with +STSI(15.1.4) or books STSI(15.1.3). + +The virtual machine will fill the provided buffer with the count of +drawers (MAG4), books per drawer (MAG3), sockets per book (MAG2) and +cores per socket (MAG1). +
Quoted 8 lines
docs/system/s390x/cpu_topology.rst line 72
Quoted 4 lines
docs/system/s390x/cpu_topology.rst new file +4
-0+Note that the STSI(15.1.2) is special in two ways: + +* When the firmware detect a change in the values calculated for STSI(15.1.2) + it will trigger the report of the topology change for the PTF instruction.I don't know if we need this section, after all documenting this is the job of the principles of operation. You could just refer to the relevant sections.
OK, I will make it shorter
Thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Re: [PATCH v9 10/10] docs/s390x: document s390x cpu topology
In reply to Pierre MorelQuoted 15 lines · Pierre Morel
Add some basic examples for the definition of cpu topology in s390x.
Signed-off-by Pierre Morel <[email protected]>--- docs/system/s390x/cpu_topology.rst | 88 ++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) create mode 100644 docs/system/s390x/cpu_topology.rstdocs/system/s390x/cpu_topology.rst new file +0
-0@@ -0,0 +1,88 @@
[...]
docs/system/s390x/cpu_topology.rst line 33
Quoted 13 lines
docs/system/s390x/cpu_topology.rst new file +13
-0+Indicating the CPU topology to the Virtual Machine +-------------------------------------------------- + +The CPU Topology, number of drawers, number of books per drawers, number of +sockets per book and number of cores per sockets is specified with the +``-smp`` qemu command arguments. + +Like in : + +.. code-block:: sh + -smp cpus=1,drawers=3,books=4,sockets=2,cores=8,maxcpus=192 + +If drawers or books are not specified, their default to 1.
Forgot this:
s/their default/they default/
[...]
I found this version much easier to understand than the previous one.
You could consider splitting up the series into two.
One that introduces support for STSI, PTF, migration, etc. And a second one that adds support for the maximum-MNist facility and drawers and books.
This would also make bisecting a bit nicer because it moves the feature enablement closer to the commits adding the support.
Right now, with this series, the topology is static and cannot change. Most of the value of making the topology visible to the guest is for it to mirror reality, and a static topology is a hindrance for that. I'm completely fine with having a static topology as a stepping stone to a dynamic one.
However I think we should have a rough plan or maybe even a prototype for how we turn a static into a dynamic topology before we merge this series in order to avoid designing us into a corner.
What do you think?
Quoted 94 lines · Pierre Morel
Hi,
The implementation of the CPU Topology in QEMU has been drastically modified since the last patch series and the number of LOCs has been greatly reduced.
Unnecessary objects have been removed, only a single S390Topology object is created to support migration and reset.
Also a documentation has been added to the series.
To use these patches, you will need Linux V6-rc1 or newer.
Mainline patches needed are:
f5ecfee944932022-07-20 KVM: s390: resetting the Topology-Change-Report24fe0195bc192022-07-20 KVM: s390: guest support for topology function0130337ec45b2022-07-20 KVM: s390: Cleanup ipte lock access and SIIF fac..Currently this code is for KVM only, I have no idea if it is interesting to provide a TCG patch. If ever it will be done in another series.
To have a better understanding of the S390x CPU Topology and its implementation in QEMU you can have a look at the documentation in the last patch.
New in this series
==================s390x/cpus: Make absence of multithreading clear
This patch makes clear that CPU-multithreading is not supported in the guest.
s390x/cpu topology:
core_idsets s390x CPU topologyThis patch uses the
core_idto build the container topology and the placement of the CPU inside the container.s390x/cpu topology: reporting the CPU topology to the guest
This patch is based on the fact that the CPU type for guests is always IFL, CPUs are always dedicated and the polarity is always horizontal.
This may change in the future.hw/core: introducing drawer and books for s390x s390x/cpu: reporting drawers and books topology to the guest
These two patches extend the topology handling to add two new containers levels above sockets: books and drawers.
The subject of the last patches is clear enough (I hope).
Regards,
PierrePierre Morel (10):
s390x/cpus: Make absence of multithreading clear s390x/cpu topology:core_idsets s390x CPU topology s390x/cpu topology: reporting the CPU topology to the guest hw/core: introducing drawer and books for s390x s390x/cpu: reporting drawers and books topology to the guest s390x/cpu_topology: resetting the Topology-Change-Report s390x/cpu_topology: CPU topology migration target/s390x: interception of PTF instruction s390x/cpu_topology: activating CPU topology docs/s390x: document s390x cpu topologydocs/system/s390x/cpu_topology.rst | 88 +++++++++ hw/core/machine-smp.c | 48 ++++- hw/core/machine.c | 9 + hw/s390x/cpu-topology.c | 293 +++++++++++++++++++++++++++++ hw/s390x/meson.build | 1 + hw/s390x/s390-virtio-ccw.c | 61 +++++- include/hw/boards.h | 11 ++ include/hw/s390x/cpu-topology.h | 53 ++++++ include/hw/s390x/s390-virtio-ccw.h | 7 + qapi/machine.json | 14 +- qemu-options.hx | 6 +- softmmu/vl.c | 6 + target/s390x/cpu-sysemu.c | 15 ++ target/s390x/cpu.h | 51 +++++ target/s390x/cpu_topology.c | 150 +++++++++++++++ target/s390x/kvm/kvm.c | 56 +++++- target/s390x/kvm/kvm_s390x.h | 1 + target/s390x/meson.build | 1 + 18 files changed, 858 insertions(+), 13 deletions(-) create mode 100644 docs/system/s390x/cpu_topology.rst create mode 100644 hw/s390x/cpu-topology.c create mode 100644 include/hw/s390x/cpu-topology.h create mode 100644 target/s390x/cpu_topology.c
Quoted 6 lines · Janis Schoetterl-Glausch
I found this version much easier to understand than the previous one.
You could consider splitting up the series into two.
One that introduces support for STSI, PTF, migration, etc. And a second one that adds support for the maximum-MNist facility and drawers and books.
I agree, sending the first part as supporting only sockets already is an enhancement that could be proposed on its own.
Quoted 5 lines
This would also make bisecting a bit nicer because it moves the feature enablement closer to the commits adding the support.
Right now, with this series, the topology is static and cannot change.
If we ignore changes on hotplug, yes we consider the topology fixed and that the vCPU does not migrate.
Quoted 8 lines
Most of the value of making the topology visible to the guest is for it to mirror reality, and a static topology is a hindrance for that. I'm completely fine with having a static topology as a stepping stone to a dynamic one.
However I think we should have a rough plan or maybe even a prototype for how we turn a static into a dynamic topology before we merge this series in order to avoid designing us into a corner.
What do you think?
Yes, we can discuss this internally before moving there.
Thanks,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Am 02.09.22 um 09:55 schrieb Pierre Morel:
Quoted 59 lines
Hi,
The implementation of the CPU Topology in QEMU has been drastically modified since the last patch series and the number of LOCs has been greatly reduced.
Unnecessary objects have been removed, only a single S390Topology object is created to support migration and reset.
Also a documentation has been added to the series.
To use these patches, you will need Linux V6-rc1 or newer.
Mainline patches needed are:
f5ecfee944932022-07-20 KVM: s390: resetting the Topology-Change-Report24fe0195bc192022-07-20 KVM: s390: guest support for topology function0130337ec45b2022-07-20 KVM: s390: Cleanup ipte lock access and SIIF fac..Currently this code is for KVM only, I have no idea if it is interesting to provide a TCG patch. If ever it will be done in another series.
To have a better understanding of the S390x CPU Topology and its implementation in QEMU you can have a look at the documentation in the last patch.
New in this series
==================s390x/cpus: Make absence of multithreading clear
This patch makes clear that CPU-multithreading is not supported in the guest.
s390x/cpu topology: core_id sets s390x CPU topologyThis patch uses the
core_idto build the container topology and the placement of the CPU inside the container.s390x/cpu topology: reporting the CPU topology to the guest
This patch is based on the fact that the CPU type for guests is always IFL, CPUs are always dedicated and the polarity is always horizontal.
This may change in the future.hw/core: introducing drawer and books for s390x s390x/cpu: reporting drawers and books topology to the guest
These two patches extend the topology handling to add two new containers levels above sockets: books and drawers.
The subject of the last patches is clear enough (I hope).
Regards,
PierrePierre Morel (10):
Quoted 9 lines
s390x/cpus: Make absence of multithreading clear s390x/cpu topology: core_id sets s390x CPU topology s390x/cpu topology: reporting the CPU topology to the guest hw/core: introducing drawer and books for s390x s390x/cpu: reporting drawers and books topology to the guest s390x/cpu_topology: resetting the Topology-Change-Report s390x/cpu_topology: CPU topology migration target/s390x: interception of PTF instruction s390x/cpu_topology: activating CPU topology
Do we really need a machine property? As far as I can see, old QEMU
cannot activate the ctop facility with old and new kernel unless it
enables CAP_S390_CPU_TOPOLOGY. I do get
oldqemu .... -cpu z14,ctop=on
qemu-system-s390x: Some features requested in the CPU model are not available in the configuration: ctop
With the newer QEMU we can. So maybe we can simply have a topology (and then a cpu model feature) in new QEMUs and non in old. the cpu model would then also fence migration from enabled to disabled.
Quoted 77 lines · Christian Borntraeger
Am 02.09.22 um 09:55 schrieb Pierre Morel:
Quoted 58 lines
Hi,
The implementation of the CPU Topology in QEMU has been drastically modified since the last patch series and the number of LOCs has been greatly reduced.
Unnecessary objects have been removed, only a single S390Topology object is created to support migration and reset.
Also a documentation has been added to the series.
To use these patches, you will need Linux V6-rc1 or newer.
Mainline patches needed are:
f5ecfee944932022-07-20 KVM: s390: resetting the Topology-Change-Report24fe0195bc192022-07-20 KVM: s390: guest support for topology function0130337ec45b2022-07-20 KVM: s390: Cleanup ipte lock access and SIIF fac..Currently this code is for KVM only, I have no idea if it is interesting to provide a TCG patch. If ever it will be done in another series.
To have a better understanding of the S390x CPU Topology and its implementation in QEMU you can have a look at the documentation in the last patch.
New in this series
==================s390x/cpus: Make absence of multithreading clear
This patch makes clear that CPU-multithreading is not supported in the guest.
s390x/cpu topology:
core_idsets s390x CPU topologyThis patch uses the
core_idto build the container topology and the placement of the CPU inside the container.s390x/cpu topology: reporting the CPU topology to the guest
This patch is based on the fact that the CPU type for guests is always IFL, CPUs are always dedicated and the polarity is always horizontal.
This may change in the future.hw/core: introducing drawer and books for s390x s390x/cpu: reporting drawers and books topology to the guest
These two patches extend the topology handling to add two new containers levels above sockets: books and drawers.
The subject of the last patches is clear enough (I hope).
Regards,
PierreQuoted 10 lines
Pierre Morel (10):
s390x/cpus: Make absence of multithreading clear s390x/cpu topology:core_idsets s390x CPU topology s390x/cpu topology: reporting the CPU topology to the guest hw/core: introducing drawer and books for s390x s390x/cpu: reporting drawers and books topology to the guest s390x/cpu_topology: resetting the Topology-Change-Report s390x/cpu_topology: CPU topology migration target/s390x: interception of PTF instruction s390x/cpu_topology: activating CPU topologyDo we really need a machine property? As far as I can see, old QEMU
cannot activate the ctop facility with old and new kernel unless it
enablesCAP_S390_CPU_TOPOLOGY. I do get
oldqemu .... -cpu z14,ctop=on
qemu-system-s390x: Some features requested in the CPU model are not available in the configuration: ctop
Quoted 3 lines
With the newer QEMU we can. So maybe we can simply have a topology (and then a cpu model feature) in new QEMUs and non in old. the cpu model would then also fence migration from enabled to disabled.
OK, I can check this.
In this case migration with topology will be if I understand correctly:
NEW_QEMU/old_machine <-> NEW_QEMU/old_machine OK
While
OLD_QEMU/old_machine <-> NEW_QEMU/old_machine KO
NEW_QEMU/old_machine <-> OLD_QEMU/old_machine KO
Is this something we can accept?
regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Quoted 84 lines · Pierre Morel
Quoted 77 lines · Christian Borntraeger
Am 02.09.22 um 09:55 schrieb Pierre Morel:
Quoted 58 lines
Hi,
The implementation of the CPU Topology in QEMU has been drastically modified since the last patch series and the number of LOCs has been greatly reduced.
Unnecessary objects have been removed, only a single S390Topology object is created to support migration and reset.
Also a documentation has been added to the series.
To use these patches, you will need Linux V6-rc1 or newer.
Mainline patches needed are:
f5ecfee944932022-07-20 KVM: s390: resetting the Topology-Change-Report24fe0195bc192022-07-20 KVM: s390: guest support for topology function0130337ec45b2022-07-20 KVM: s390: Cleanup ipte lock access and SIIF fac..Currently this code is for KVM only, I have no idea if it is interesting to provide a TCG patch. If ever it will be done in another series.
To have a better understanding of the S390x CPU Topology and its implementation in QEMU you can have a look at the documentation in the last patch.
New in this series
==================s390x/cpus: Make absence of multithreading clear
This patch makes clear that CPU-multithreading is not supported in the guest.
s390x/cpu topology:
core_idsets s390x CPU topologyThis patch uses the
core_idto build the container topology and the placement of the CPU inside the container.s390x/cpu topology: reporting the CPU topology to the guest
This patch is based on the fact that the CPU type for guests is always IFL, CPUs are always dedicated and the polarity is always horizontal.
This may change in the future.hw/core: introducing drawer and books for s390x s390x/cpu: reporting drawers and books topology to the guest
These two patches extend the topology handling to add two new containers levels above sockets: books and drawers.
The subject of the last patches is clear enough (I hope).
Regards,
PierreQuoted 10 lines
Pierre Morel (10):
s390x/cpus: Make absence of multithreading clear s390x/cpu topology:core_idsets s390x CPU topology s390x/cpu topology: reporting the CPU topology to the guest hw/core: introducing drawer and books for s390x s390x/cpu: reporting drawers and books topology to the guest s390x/cpu_topology: resetting the Topology-Change-Report s390x/cpu_topology: CPU topology migration target/s390x: interception of PTF instruction s390x/cpu_topology: activating CPU topologyDo we really need a machine property? As far as I can see, old QEMU
cannot activate the ctop facility with old and new kernel unless it
enablesCAP_S390_CPU_TOPOLOGY. I do get
oldqemu .... -cpu z14,ctop=on
qemu-system-s390x: Some features requested in the CPU model are not available in the configuration: ctopQuoted 3 lines
With the newer QEMU we can. So maybe we can simply have a topology (and then a cpu model feature) in new QEMUs and non in old. the cpu model would then also fence migration from enabled to disabled.
Quoted 7 lines
OK, I can check this.
In this case migration with topology will be if I understand correctly:NEW_QEMU/old_machine <-> NEW_QEMU/old_machine OK
While
OLD_QEMU/old_machine <-> NEW_QEMU/old_machine KO
NEW_QEMU/old_machine <-> OLD_QEMU/old_machine KO
I forgot to say that I mean in the examples above without using a flag.
Of course using a flag like -ctop=off in NEW_QEMU/new_machine allows to migrate from and to old_machines in an old QEMU.
Also I had the same behavior already in V9 by having a VMState without the creation of a machine property, a new cpu feature and a new cpu flag.
Quoted 6 lines
Is this something we can accept?
regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Gentle ping.
Did I understand the problem or am I wrong?
Quoted 114 lines · Pierre Morel
Quoted 84 lines · Pierre Morel
Quoted 77 lines · Christian Borntraeger
Am 02.09.22 um 09:55 schrieb Pierre Morel:
Quoted 58 lines
Hi,
The implementation of the CPU Topology in QEMU has been drastically modified since the last patch series and the number of LOCs has been greatly reduced.
Unnecessary objects have been removed, only a single S390Topology object is created to support migration and reset.
Also a documentation has been added to the series.
To use these patches, you will need Linux V6-rc1 or newer.
Mainline patches needed are:
f5ecfee944932022-07-20 KVM: s390: resetting the Topology-Change-Report24fe0195bc192022-07-20 KVM: s390: guest support for topology function0130337ec45b2022-07-20 KVM: s390: Cleanup ipte lock access and SIIF fac..Currently this code is for KVM only, I have no idea if it is interesting to provide a TCG patch. If ever it will be done in another series.
To have a better understanding of the S390x CPU Topology and its implementation in QEMU you can have a look at the documentation in the last patch.
New in this series
==================s390x/cpus: Make absence of multithreading clear
This patch makes clear that CPU-multithreading is not supported in the guest.
s390x/cpu topology:
core_idsets s390x CPU topologyThis patch uses the
core_idto build the container topology and the placement of the CPU inside the container.s390x/cpu topology: reporting the CPU topology to the guest
This patch is based on the fact that the CPU type for guests is always IFL, CPUs are always dedicated and the polarity is always horizontal.
This may change in the future.hw/core: introducing drawer and books for s390x s390x/cpu: reporting drawers and books topology to the guest
These two patches extend the topology handling to add two new containers levels above sockets: books and drawers.
The subject of the last patches is clear enough (I hope).
Regards,
PierreQuoted 10 lines
Pierre Morel (10):
s390x/cpus: Make absence of multithreading clear s390x/cpu topology:core_idsets s390x CPU topology s390x/cpu topology: reporting the CPU topology to the guest hw/core: introducing drawer and books for s390x s390x/cpu: reporting drawers and books topology to the guest s390x/cpu_topology: resetting the Topology-Change-Report s390x/cpu_topology: CPU topology migration target/s390x: interception of PTF instruction s390x/cpu_topology: activating CPU topologyDo we really need a machine property? As far as I can see, old QEMU
cannot activate the ctop facility with old and new kernel unless it
enablesCAP_S390_CPU_TOPOLOGY. I do get
oldqemu .... -cpu z14,ctop=on
qemu-system-s390x: Some features requested in the CPU model are not available in the configuration: ctopQuoted 3 lines
With the newer QEMU we can. So maybe we can simply have a topology (and then a cpu model feature) in new QEMUs and non in old. the cpu model would then also fence migration from enabled to disabled.
Quoted 7 lines
OK, I can check this.
In this case migration with topology will be if I understand correctly:NEW_QEMU/old_machine <-> NEW_QEMU/old_machine OK
While
OLD_QEMU/old_machine <-> NEW_QEMU/old_machine KO
NEW_QEMU/old_machine <-> OLD_QEMU/old_machine KOI forgot to say that I mean in the examples above without using a flag.
Of course using a flag like -ctop=off in NEW_QEMU/new_machine allows to migrate from and to
old_machinesin an old QEMU.Also I had the same behavior already in V9 by having a VMState without the creation of a machine property, a new cpu feature and a new cpu flag.
Quoted 6 lines
Is this something we can accept?
regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen
Quoted 5 lines · Pierre Morel
Gentle ping.
Did I understand the problem or am I wrong?
I guess I was wrong, so I send a new series next week.
Regards,
Pierre
Quoted 117 lines · Pierre Morel
Quoted 114 lines · Pierre Morel
Quoted 84 lines · Christian Borntraeger
Quoted 77 lines
Am 02.09.22 um 09:55 schrieb Pierre Morel:
Quoted 58 lines
Hi,
The implementation of the CPU Topology in QEMU has been drastically modified since the last patch series and the number of LOCs has been greatly reduced.
Unnecessary objects have been removed, only a single S390Topology object is created to support migration and reset.
Also a documentation has been added to the series.
To use these patches, you will need Linux V6-rc1 or newer.
Mainline patches needed are:
f5ecfee944932022-07-20 KVM: s390: resetting the Topology-Change-Report24fe0195bc192022-07-20 KVM: s390: guest support for topology function0130337ec45b2022-07-20 KVM: s390: Cleanup ipte lock access and SIIF fac..Currently this code is for KVM only, I have no idea if it is interesting to provide a TCG patch. If ever it will be done in another series.
To have a better understanding of the S390x CPU Topology and its implementation in QEMU you can have a look at the documentation in the last patch.
New in this series
==================s390x/cpus: Make absence of multithreading clear
This patch makes clear that CPU-multithreading is not supported in the guest.
s390x/cpu topology:
core_idsets s390x CPU topologyThis patch uses the
core_idto build the container topology and the placement of the CPU inside the container.s390x/cpu topology: reporting the CPU topology to the guest
This patch is based on the fact that the CPU type for guests is always IFL, CPUs are always dedicated and the polarity is always horizontal.
This may change in the future.hw/core: introducing drawer and books for s390x s390x/cpu: reporting drawers and books topology to the guest
These two patches extend the topology handling to add two new containers levels above sockets: books and drawers.
The subject of the last patches is clear enough (I hope).
Regards,
PierreQuoted 10 lines
Pierre Morel (10):
s390x/cpus: Make absence of multithreading clear s390x/cpu topology:core_idsets s390x CPU topology s390x/cpu topology: reporting the CPU topology to the guest hw/core: introducing drawer and books for s390x s390x/cpu: reporting drawers and books topology to the guest s390x/cpu_topology: resetting the Topology-Change-Report s390x/cpu_topology: CPU topology migration target/s390x: interception of PTF instruction s390x/cpu_topology: activating CPU topologyDo we really need a machine property? As far as I can see, old QEMU
cannot activate the ctop facility with old and new kernel unless it
enablesCAP_S390_CPU_TOPOLOGY. I do get
oldqemu .... -cpu z14,ctop=on
qemu-system-s390x: Some features requested in the CPU model are not available in the configuration: ctopQuoted 3 lines
With the newer QEMU we can. So maybe we can simply have a topology (and then a cpu model feature) in new QEMUs and non in old. the cpu model would then also fence migration from enabled to disabled.
Quoted 7 lines
OK, I can check this.
In this case migration with topology will be if I understand correctly:NEW_QEMU/old_machine <-> NEW_QEMU/old_machine OK
While
OLD_QEMU/old_machine <-> NEW_QEMU/old_machine KO
NEW_QEMU/old_machine <-> OLD_QEMU/old_machine KOI forgot to say that I mean in the examples above without using a flag.
Of course using a flag like -ctop=off in NEW_QEMU/new_machine allows to migrate from and to
old_machinesin an old QEMU.Also I had the same behavior already in V9 by having a VMState without the creation of a machine property, a new cpu feature and a new cpu flag.
Quoted 6 lines
Is this something we can accept?
regards,
Pierre
Signature · 2 lines
Pierre Morel IBM Lab Boeblingen