Skip to contentSkip to conversation

S390x: CPU Topology Information

PATCH Superseded Pierre Morel 3 participants 13 messages Mar 20, 2023 – Mar 28, 2023

Subsystems (2)

v7 of 21 Superseded Latest is v21 →

lore
Share & tools
lore
#0 lore raw

v7 cover S390x: CPU Topology Information

Hi,

new version of the kvm-unit-test s390x CPU topology series.

0. what is new in this new spin
-------------------------------

- Running the test on LPAR has been getestet on a46lp21
- splitting big functions in small ones
- better tests, reset before test
- Diverse rephrasing and functions reordering

1. what is done
---------------

- First part is checking PTF errors, for KVM and LPAR

- Second part is checking PTF polarization change and STSI with the cpu topology including drawers and books. This tests are run for KVM only.

To run these tests under KVM successfully you need Linux 6.0 and the QEMU patches you find at:

https://lists.gnu.org/archive/html/qemu-devel/2023-02/msg00081.html

Note that Fedora-35 already has the CPU Topology backport for Linux.

To start the test in KVM just do:

# ./run_tests.sh topology

or something like:

# ./s390x-run s390x/topology.elf \
	-smp 5,drawers=3,books=3,sockets=4,cores=4,maxcpus=144 \
	-append '-drawers 3 -books 3 -sockets 4 -cores 4'

Of course the declaration of the number of socket and core must be coherent in -smp and -append arguments.

To start the tests in LPAR you need to define the loader entry. For example, under RedHat:

	title topology
	version topology.bin
	linux /boot/topology.bin
	initrd /boot/initramfs-topology.bin.img
	options -drawers 4 -books 4 -sockets 2 -cores 8
	id rhel-0-topology.bin
	grub_users $grub_users
	grub_arg --unrestricted
	grub_class kernel

Output of the test is done on the SCLP console.

Regards,
Pierre

Pierre Morel (2):
s390x: topology: Check the Perform Topology Function
s390x: topology: Checking Configuration Topology Information

 lib/s390x/sclp.c    |   5 +
 lib/s390x/sclp.h    |   4 +-
 lib/s390x/stsi.h    |  36 ++++
 s390x/Makefile      |   1 +
 s390x/topology.c    | 471 ++++++++++++++++++++++++++++++++++++++++++++
 s390x/unittests.cfg |  11 ++
 6 files changed, 527 insertions(+), 1 deletion(-)
 create mode 100644 s390x/topology.c
Signature · 26 lines
2.31.1

new in v7:

- better checks using device attributes on commandline
  (Pierre)
- use builtin to get the number of CPU in the TLE mask
  (Thomas)
- use Elvis (not dead)
  (Thomas)
- reset before tests
  (Nina)
- splitting test_ptf in small functions
  (Thomas)
- check every ptf function code for program check
  (Nina)
- Test made on LPAR
  (Janosch)
- use a single page for SYSIB
  (Thomas)
- abort on wrong parameter
  (Thomas)
- implement SYSIB check with a recursive funtion
  (Nina)
- diverse little changes (naming, clearer checks
  (Nina, Thomas)
#1 lore raw

PATCH v7 1/2 s390x: topology: Check the Perform Topology Function

We check that the PTF instruction is working correctly when the cpu topology facility is available.

For KVM only, we test changing of the polarity between horizontal and vertical and that a reset set the horizontal polarity.

---
 s390x/Makefile      |   1 +
 s390x/topology.c    | 180 ++++++++++++++++++++++++++++++++++++++++++++
 s390x/unittests.cfg |   3 +
 3 files changed, 184 insertions(+)
 create mode 100644 s390x/topology.c
s390x/Makefile +1 -0
index e94b720..05dac04 100644
@@ -40,6 +40,7 @@ tests += $(TEST_DIR)/panic-loop-pgm.elf
 tests += $(TEST_DIR)/migration-sck.elf
 tests += $(TEST_DIR)/exittime.elf
 tests += $(TEST_DIR)/ex.elf
+tests += $(TEST_DIR)/topology.elf
 
 pv-tests += $(TEST_DIR)/pv-diags.elf
 
s390x/topology.c new file +180 -0

180 lines not on this page · raw message

s390x/unittests.cfg +3 -0
index 453ee9c..d0ac683 100644
@@ -233,3 +233,6 @@ extra_params = -append '--parallel'
 
 [execute]
 file = ex.elf
+
+[topology]
+file = topology.elf
Signature · 1 line
2.31.1
#2 lore raw

Re: [kvm-unit-tests PATCH v7 1/2] s390x: topology: Check the Perform Topology Function

Quoted 87 lines · On Mon, 20 Mar 2023 09:56:41 +0100 Pierre Morel <[email protected]> wrote:

We check that the PTF instruction is working correctly when the cpu topology facility is available.

For KVM only, we test changing of the polarity between horizontal and vertical and that a reset set the horizontal polarity.

---
 s390x/Makefile      |   1 +
 s390x/topology.c    | 180 ++++++++++++++++++++++++++++++++++++++++++++
 s390x/unittests.cfg |   3 +
 3 files changed, 184 insertions(+)
 create mode 100644 s390x/topology.c
s390x/Makefile +1 -0
index e94b720..05dac04 100644
@@ -40,6 +40,7 @@ tests += $(TEST_DIR)/panic-loop-pgm.elf
 tests += $(TEST_DIR)/migration-sck.elf
 tests += $(TEST_DIR)/exittime.elf
 tests += $(TEST_DIR)/ex.elf
+tests += $(TEST_DIR)/topology.elf
 
 pv-tests += $(TEST_DIR)/pv-diags.elf
 
s390x/topology.c new file +55 -0
new file mode 100644
index 0000000..ce248f1
@@ -0,0 +1,180 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022
+ *
+ * Authors:
+ *  Pierre Morel <[email protected]>
+ */
+
+#include <libcflat.h>
+#include <asm/page.h>
+#include <asm/asm-offsets.h>
+#include <asm/interrupt.h>
+#include <asm/facility.h>
+#include <smp.h>
+#include <sclp.h>
+#include <s390x/hardware.h>
+
+#define PTF_REQ_HORIZONTAL	0
+#define PTF_REQ_VERTICAL	1
+#define PTF_REQ_CHECK		2
+
+#define PTF_ERR_NO_REASON	0
+#define PTF_ERR_ALRDY_POLARIZED	1
+#define PTF_ERR_IN_PROGRESS	2
+
+extern int diag308_load_reset(u64);
+
+static int ptf(unsigned long fc, unsigned long *rc)
+{
+	int cc;
+
+	asm volatile(
+		"	ptf	%1	\n"
+		"       ipm     %0	\n"
+		"       srl     %0,28	\n"
+		: "=d" (cc), "+d" (fc)
+		:
+		: "cc");
+
+	*rc = fc >> 8;
+	return cc;
+}
+
+static void check_privilege(int fc)
+{
+	unsigned long rc;
+
+	report_prefix_push("Privilege");
+	report_info("function code %d", fc);
+	enter_pstate();
+	expect_pgm_int();
+	ptf(fc, &rc);
+	check_pgm_int_code(PGM_INT_CODE_PRIVILEGED_OPERATION);

s390x/topology.c line 65

Quoted 10 lines
s390x/topology.c new file +10 -0
+	report_prefix_pop();
+}
+
+static void check_function_code(void)
+{
+	unsigned long rc;
+
+	report_prefix_push("Undefined fc");
+	expect_pgm_int();
+	ptf(0xff, &rc);

please don't use magic numbers, add a new macro PTF_INVALID_FUNCTION (or something like that)

s390x/topology.c line 76

Quoted 11 lines
s390x/topology.c new file +11 -0
+	check_pgm_int_code(PGM_INT_CODE_SPECIFICATION);
+	report_prefix_pop();
+}
+
+static void check_reserved_bits(void)
+{
+	unsigned long rc;
+
+	report_prefix_push("Reserved bits");
+	expect_pgm_int();
+	ptf(0xffffffffffffff00UL, &rc);

I would like every single bit to be tested, since all of them are required to be zero.

make a loop and test each, but please report success of failure only once at the end.
use a report_info in case of failure to indicate which bit failed

Quoted 42 lines
s390x/topology.c new file +42 -0
+	check_pgm_int_code(PGM_INT_CODE_SPECIFICATION);
+	report_prefix_pop();
+}
+
+static void check_mtcr_pending(void)
+{
+	unsigned long rc;
+	int cc;
+
+	report_prefix_push("Topology Report pending");
+	/*
+	 * At this moment the topology may already have changed
+	 * since the VM has been started.
+	 * However, we can test if a second PTF instruction
+	 * reports that the topology did not change since the
+	 * preceding PFT instruction.
+	 */
+	ptf(PTF_REQ_CHECK, &rc);
+	cc = ptf(PTF_REQ_CHECK, &rc);
+	report(cc == 0, "PTF check should clear topology report");
+	report_prefix_pop();
+}
+
+static void check_polarization_change(void)
+{
+	unsigned long rc;
+	int cc;
+
+	report_prefix_push("Topology polarization check");
+
+	/* We expect a clean state through reset */
+	report(diag308_load_reset(1), "load normal reset done");
+
+	/*
+	 * Set vertical polarization to verify that RESET sets
+	 * horizontal polarization back.
+	 */
+	cc = ptf(PTF_REQ_VERTICAL, &rc);
+	report(cc == 0, "Set vertical polarization.");
+
+	report(diag308_load_reset(1), "load normal reset done");
+

s390x/topology.c line 128

Quoted 10 lines
s390x/topology.c new file +10 -0
+	cc = ptf(PTF_REQ_CHECK, &rc);
+	report(cc == 0, "Reset should clear topology report");
+
+	cc = ptf(PTF_REQ_HORIZONTAL, &rc);
+	report(cc == 2 && rc == PTF_ERR_ALRDY_POLARIZED,
+	       "After RESET polarization is horizontal");
+
+	/* Flip between vertical and horizontal polarization */
+	cc = ptf(PTF_REQ_VERTICAL, &rc);
+	report(cc == 0, "Change to vertical polarization.");

either here or in a new block, test that setting vertical twice in a row will also result in a cc == 2 && rc == PTF_ERR_ALRDY_POLARIZED

s390x/topology.c line 134

Quoted 6 lines
s390x/topology.c new file +6 -0
+
+	cc = ptf(PTF_REQ_CHECK, &rc);
+	report(cc == 1, "Polarization change should set topology report");
+
+	cc = ptf(PTF_REQ_HORIZONTAL, &rc);
+	report(cc == 0, "Change to horizontal polarization.");

it cannot hurt to add here another check for pending reports

Quoted 57 lines
s390x/topology.c new file +46 -0
+
+	report_prefix_pop();
+}
+
+static void test_ptf(void)
+{
+	check_privilege(PTF_REQ_HORIZONTAL);
+	check_privilege(PTF_REQ_VERTICAL);
+	check_privilege(PTF_REQ_CHECK);
+	check_function_code();
+	check_reserved_bits();
+	check_mtcr_pending();
+	check_polarization_change();
+}
+
+static struct {
+	const char *name;
+	void (*func)(void);
+} tests[] = {
+	{ "PTF", test_ptf},
+	{ NULL, NULL }
+};
+
+int main(int argc, char *argv[])
+{
+	int i;
+
+	report_prefix_push("CPU Topology");
+
+	if (!test_facility(11)) {
+		report_skip("Topology facility not present");
+		goto end;
+	}
+
+	report_info("Virtual machine level %ld", stsi_get_fc());
+
+	for (i = 0; tests[i].name; i++) {
+		report_prefix_push(tests[i].name);
+		tests[i].func();
+		report_prefix_pop();
+	}
+
+end:
+	report_prefix_pop();
+	return report_summary();
+}
s390x/unittests.cfg +3 -0
index 453ee9c..d0ac683 100644
@@ -233,3 +233,6 @@ extra_params = -append '--parallel'
 
 [execute]
 file = ex.elf
+
+[topology]
+file = topology.elf
#3 lore raw

Re: [kvm-unit-tests PATCH v7 1/2] s390x: topology: Check the Perform Topology Function

Quoted 94 lines · Claudio Imbrenda
Quoted 91 lines · On Mon, 20 Mar 2023 09:56:41 +0100 Pierre Morel <[email protected]> wrote:

We check that the PTF instruction is working correctly when the cpu topology facility is available.

For KVM only, we test changing of the polarity between horizontal and vertical and that a reset set the horizontal polarity.

---
  s390x/Makefile      |   1 +
  s390x/topology.c    | 180 ++++++++++++++++++++++++++++++++++++++++++++
  s390x/unittests.cfg |   3 +
  3 files changed, 184 insertions(+)

create mode 100644 s390x/topology.c

s390x/Makefile +1 -0
index e94b720..05dac04 100644
@@ -40,6 +40,7 @@ tests += $(TEST_DIR)/panic-loop-pgm.elf
  tests += $(TEST_DIR)/migration-sck.elf
  tests += $(TEST_DIR)/exittime.elf
  tests += $(TEST_DIR)/ex.elf
+tests += $(TEST_DIR)/topology.elf
 
  pv-tests += $(TEST_DIR)/pv-diags.elf
 
s390x/topology.c new file +59 -0
new file mode 100644
index 0000000..ce248f1
@@ -0,0 +1,180 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022
+ *
+ * Authors:
+ *  Pierre Morel <[email protected]>
+ */
+
+#include <libcflat.h>
+#include <asm/page.h>
+#include <asm/asm-offsets.h>
+#include <asm/interrupt.h>
+#include <asm/facility.h>
+#include <smp.h>
+#include <sclp.h>
+#include <s390x/hardware.h>
+
+#define PTF_REQ_HORIZONTAL	0
+#define PTF_REQ_VERTICAL	1
+#define PTF_REQ_CHECK		2
+
+#define PTF_ERR_NO_REASON	0
+#define PTF_ERR_ALRDY_POLARIZED	1
+#define PTF_ERR_IN_PROGRESS	2
+
+extern int diag308_load_reset(u64);
+
+static int ptf(unsigned long fc, unsigned long *rc)
+{
+	int cc;
+
+	asm volatile(
+		"	ptf	%1	\n"
+		"       ipm     %0	\n"
+		"       srl     %0,28	\n"
+		: "=d" (cc), "+d" (fc)
+		:
+		: "cc");
+
+	*rc = fc >> 8;
+	return cc;
+}
+
+static void check_privilege(int fc)
+{
+	unsigned long rc;
+
+	report_prefix_push("Privilege");
+	report_info("function code %d", fc);
+	enter_pstate();
+	expect_pgm_int();
+	ptf(fc, &rc);
+	check_pgm_int_code(PGM_INT_CODE_PRIVILEGED_OPERATION);
+	report_prefix_pop();
+}
+
+static void check_function_code(void)
Quoted 8 lines

s390x/topology.c line 65

Quoted 6 lines
s390x/topology.c new file +6 -0
+{
+	unsigned long rc;
+
+	report_prefix_push("Undefined fc");
+	expect_pgm_int();
+	ptf(0xff, &rc);

please don't use magic numbers, add a new macro PTF_INVALID_FUNCTION (or something like that)

OK

Quoted 15 lines

s390x/topology.c line 76

Quoted 12 lines
s390x/topology.c new file +11 -0
+	check_pgm_int_code(PGM_INT_CODE_SPECIFICATION);
+	report_prefix_pop();
+}
+
+static void check_reserved_bits(void)
+{
+	unsigned long rc;
+
+	report_prefix_push("Reserved bits");
+	expect_pgm_int();
+	ptf(0xffffffffffffff00UL, &rc);

I would like every single bit to be tested, since all of them are required to be zero.

Quoted 3 lines

make a loop and test each, but please report success of failure only once at the end.
use a report_info in case of failure to indicate which bit failed

OK

Quoted 47 lines
Quoted 47 lines
s390x/topology.c new file +46 -0
+	check_pgm_int_code(PGM_INT_CODE_SPECIFICATION);
+	report_prefix_pop();
+}
+
+static void check_mtcr_pending(void)
+{
+	unsigned long rc;
+	int cc;
+
+	report_prefix_push("Topology Report pending");
+	/*
+	 * At this moment the topology may already have changed
+	 * since the VM has been started.
+	 * However, we can test if a second PTF instruction
+	 * reports that the topology did not change since the
+	 * preceding PFT instruction.
+	 */
+	ptf(PTF_REQ_CHECK, &rc);
+	cc = ptf(PTF_REQ_CHECK, &rc);
+	report(cc == 0, "PTF check should clear topology report");
+	report_prefix_pop();
+}
+
+static void check_polarization_change(void)
+{
+	unsigned long rc;
+	int cc;
+
+	report_prefix_push("Topology polarization check");
+
+	/* We expect a clean state through reset */
+	report(diag308_load_reset(1), "load normal reset done");
+
+	/*
+	 * Set vertical polarization to verify that RESET sets
+	 * horizontal polarization back.
+	 */
+	cc = ptf(PTF_REQ_VERTICAL, &rc);
+	report(cc == 0, "Set vertical polarization.");
+
+	report(diag308_load_reset(1), "load normal reset done");
+
+	cc = ptf(PTF_REQ_CHECK, &rc);
+	report(cc == 0, "Reset should clear topology report");
+
+	cc = ptf(PTF_REQ_HORIZONTAL, &rc);
Quoted 8 lines

s390x/topology.c line 128

Quoted 6 lines
s390x/topology.c new file +6 -0
+	report(cc == 2 && rc == PTF_ERR_ALRDY_POLARIZED,
+	       "After RESET polarization is horizontal");
+
+	/* Flip between vertical and horizontal polarization */
+	cc = ptf(PTF_REQ_VERTICAL, &rc);
+	report(cc == 0, "Change to vertical polarization.");

either here or in a new block, test that setting vertical twice in a row will also result in a cc == 2 && rc == PTF_ERR_ALRDY_POLARIZED

OK

Quoted 8 lines

s390x/topology.c line 134

Quoted 7 lines
s390x/topology.c new file +6 -0
+
+	cc = ptf(PTF_REQ_CHECK, &rc);
+	report(cc == 1, "Polarization change should set topology report");
+
+	cc = ptf(PTF_REQ_HORIZONTAL, &rc);
+	report(cc == 0, "Change to horizontal polarization.");

it cannot hurt to add here another check for pending reports

OK

Thanks for the comments,

Regards,

Pierre

#4 lore raw

Re: [kvm-unit-tests PATCH v7 1/2] s390x: topology: Check the Perform Topology Function

In reply to Pierre Morel

Quoting Pierre Morel (2023-03-20 09:56:41)
[...]

Quoted 18 lines
s390x/topology.c new file +12 -0
new file mode 100644
index 0000000..ce248f1
@@ -0,0 +1,180 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022
+ *
+ * Authors:
+ *  Pierre Morel <[email protected]>
+ */
+
+#include <libcflat.h>
+#include <asm/page.h>

s390x/topology.c line 22

Quoted 10 lines
s390x/topology.c new file +10 -0
+#include <asm/asm-offsets.h>
+#include <asm/interrupt.h>
+#include <asm/facility.h>
+#include <smp.h>
+#include <sclp.h>
+#include <s390x/hardware.h>
+
+#define PTF_REQ_HORIZONTAL     0
+#define PTF_REQ_VERTICAL       1
+#define PTF_REQ_CHECK          2

These are all function codes, so how about we name these defines PTF_FC_...

and since PTF_REQ_CHECK doesn't request anything we should rename to PTF_FC_CHECK

[...]

s390x/topology.c line 154

Quoted 5 lines
s390x/topology.c new file +5 -0
+static struct {
+       const char *name;
+       void (*func)(void);
+} tests[] = {
+       { "PTF", test_ptf},

missing space ^

#5 lore raw

Re: [kvm-unit-tests PATCH v7 1/2] s390x: topology: Check the Perform Topology Function

Quoted 32 lines · Nico Boehr

Quoting Pierre Morel (2023-03-20 09:56:41)
[...]

Quoted 18 lines
s390x/topology.c new file +12 -0
new file mode 100644
index 0000000..ce248f1
@@ -0,0 +1,180 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022
+ *
+ * Authors:
+ *  Pierre Morel <[email protected]>
+ */
+
+#include <libcflat.h>
+#include <asm/page.h>

s390x/topology.c line 22

Quoted 10 lines
s390x/topology.c new file +10 -0
+#include <asm/asm-offsets.h>
+#include <asm/interrupt.h>
+#include <asm/facility.h>
+#include <smp.h>
+#include <sclp.h>
+#include <s390x/hardware.h>
+
+#define PTF_REQ_HORIZONTAL     0
+#define PTF_REQ_VERTICAL       1
+#define PTF_REQ_CHECK          2

These are all function codes, so how about we name these defines PTF_FC_...

Quoted 1 line

and since PTF_REQ_CHECK doesn't request anything we should rename to PTF_FC_CHECK

OK

Quoted 8 lines

[...]

s390x/topology.c line 154

Quoted 5 lines
s390x/topology.c new file +5 -0
+static struct {
+       const char *name;
+       void (*func)(void);
+} tests[] = {
+       { "PTF", test_ptf},

missing space ^

Yes, thanks.

Regards,

Pierre

#6 lore raw

PATCH v7 2/2 s390x: topology: Checking Configuration Topology Information

In reply to Pierre Morel

STSI with function code 15 is used to store the CPU configuration topology.

We retrieve the maximum nested level with SCLP and use the topology tree provided by the drawers, books, sockets, cores arguments.

We check :
- if the topology stored is coherent between the QEMU -smp parameters and kernel parameters.
- the number of CPUs
- the maximum number of CPUs
- the number of containers of each levels for every STSI(15.1.x) instruction allowed by the machine.

---
 lib/s390x/sclp.c    |   5 +
 lib/s390x/sclp.h    |   4 +-
 lib/s390x/stsi.h    |  36 ++++++
 s390x/topology.c    | 291 ++++++++++++++++++++++++++++++++++++++++++++
 s390x/unittests.cfg |   8 ++
 5 files changed, 343 insertions(+), 1 deletion(-)
lib/s390x/sclp.c +5 -0
index 390fde7..9679332 100644
@@ -238,3 +238,8 @@ uint64_t get_max_ram_size(void)
 {
 	return max_ram_size;
 }
+
+uint64_t sclp_get_stsi_mnest(void)
+{
+	return read_info->stsi_parm;
+}
lib/s390x/sclp.h +3 -1
index 853529b..6a611bc 100644
@@ -150,7 +150,8 @@ typedef struct ReadInfo {
 	SCCBHeader h;
 	uint16_t rnmax;
 	uint8_t rnsize;
-	uint8_t  _reserved1[16 - 11];       /* 11-15 */
+	uint8_t  _reserved1[15 - 11];       /* 11-14 */
+	uint8_t stsi_parm;                  /* 15-15 */
 	uint16_t entries_cpu;               /* 16-17 */
 	uint16_t offset_cpu;                /* 18-19 */
 	uint8_t  _reserved2[24 - 20];       /* 20-23 */
@@ -341,5 +342,6 @@ int sclp_service_call(unsigned int command, void *sccb);
 void sclp_memory_setup(void);
 uint64_t get_ram_size(void);
 uint64_t get_max_ram_size(void);
+uint64_t sclp_get_stsi_mnest(void);
 
 #endif /* _S390X_SCLP_H_ */
lib/s390x/stsi.h +36 -0
index bebc492..1351a6f 100644
@@ -29,4 +29,40 @@ struct sysinfo_3_2_2 {
 	uint8_t ext_names[8][256];
 };
 
+#define CPUS_TLE_RES_BITS 0x00fffffff8000000UL
+struct topology_core {
+	uint8_t nl;
+	uint8_t reserved1[3];
+	uint8_t reserved4:5;
+	uint8_t d:1;
+	uint8_t pp:2;
+	uint8_t type;
+	uint16_t origin;
+	uint64_t mask;
+};
+
+#define CONTAINER_TLE_RES_BITS 0x00ffffffffffff00UL
+struct topology_container {
+	uint8_t nl;
+	uint8_t reserved[6];
+	uint8_t id;
+};
+
+union topology_entry {
+	uint8_t nl;
+	struct topology_core cpu;
+	struct topology_container container;
+};
+
+#define CPU_TOPOLOGY_MAX_LEVEL 6
+struct sysinfo_15_1_x {
+	uint8_t reserved0[2];
+	uint16_t length;
+	uint8_t mag[CPU_TOPOLOGY_MAX_LEVEL];
+	uint8_t reserved0a;
+	uint8_t mnest;
+	uint8_t reserved0c[4];
+	union topology_entry tle[];
+};
+
 #endif  /* _S390X_STSI_H_ */
s390x/topology.c +291 -0

314 lines not on this page · raw message

s390x/unittests.cfg +8 -0
index d0ac683..7285f60 100644
@@ -236,3 +236,11 @@ file = ex.elf
 
 [topology]
 file = topology.elf
+# 3 CPUs on socket 0 with different CPU TLE (standard, dedicated, origin)
+# 1 CPU on socket 2
+extra_params = -smp 1,drawers=3,books=3,sockets=4,cores=4,maxcpus=144 -cpu z14,ctop=on -device z14-s390x-cpu,core-id=1 -device z14-s390x-cpu,core-id=2,dedicated=on -device z14-s390x-cpu,core-id=10 -device z14-s390x-cpu,core-id=20 -device z14-s390x-cpu,core-id=130,socket-id=0,book-id=0,drawer-id=0 -append '-drawers 3 -books 3 -sockets 4 -cores 4'
+
+[topology_max]
+# define 240 CPUs
+file = topology.elf
+extra_params = -smp drawers=3,books=4,sockets=5,cores=4,maxcpus=240 -cpu z14,ctop=on -append '-drawers 3 -books 4 -sockets 5 -cores 4'
Signature · 1 line
2.31.1
#7 lore raw

Re: [kvm-unit-tests PATCH v7 2/2] s390x: topology: Checking Configuration Topology Information

Quoting Pierre Morel (2023-03-20 09:56:42)

Quoted 14 lines

STSI with function code 15 is used to store the CPU configuration topology.

We retrieve the maximum nested level with SCLP and use the topology tree provided by the drawers, books, sockets, cores arguments.

We check :
- if the topology stored is coherent between the QEMU -smp parameters and kernel parameters.
- the number of CPUs
- the maximum number of CPUs
- the number of containers of each levels for every STSI(15.1.x) instruction allowed by the machine.

[...]

lib/s390x/sclp.c uint64_t get_max_ram_size(void) · line 243

Quoted 11 lines
lib/s390x/sclp.c +3 -0
index 390fde7..9679332 100644
@@ -238,3 +238,8 @@ uint64_t get_max_ram_size(void)
 {
        return max_ram_size;
 }
+
+uint64_t sclp_get_stsi_mnest(void)
+{

maybe add:
assert(read_info);

[...]

Quoted 4 lines
s390x/topology.c +0 -0
index ce248f1..11ce931 100644

[...]

s390x/topology.c line 32

Quoted 5 lines
s390x/topology.c +5 -0
+/*
+ * Topology level as defined by architecture, all levels exists with
+ * a single container unless overwritten by the QEMU -smp parameter.
+ */
+static int arch_topo_lvl[CPU_TOPOLOGY_MAX_LEVEL]; // = {1, 1, 1, 1, 1, 1};

So that's what is being provided to the test on the command line, right?

How about renaming this to expected_topo_lvl?

What do you mean by 'defined by architecture'?

[...]

Quoted 8 lines
s390x/topology.c +8 -0
+/*
+ * stsi_check_mag
+ * @info: Pointer to the stsi information
+ *
+ * MAG field should match the architecture defined containers
+ * when MNEST as returned by SCLP matches MNEST of the SYSIB.
+ */
+static void stsi_check_mag(struct sysinfo_15_1_x *info)

s390x/topology.c static void test_ptf(void) · line 198

Quoted 10 lines
s390x/topology.c +10 -0
+{
+       int i;
+
+       report_prefix_push("MAG");
+
+       stsi_check_maxcpus(info);
+
+       /* Explicitly skip the test if both mnest do not match */
+       if (max_nested_lvl != info->mnest)
+               goto done;

What does it mean if the two don't match, i.e. is this an error? Or a skip? Or is it just expected?

[...]

Quoted 9 lines
s390x/topology.c +9 -0
+/**
+ * check_tle:
+ * @tc: pointer to first TLE
+ *
+ * Recursively check the containers TLEs until we
+ * find a CPU TLE.
+ */
+static uint8_t *check_tle(void *tc)
+{

[...]

s390x/topology.c static void test_ptf(void) · line 262

Quoted 2 lines
s390x/topology.c +2 -0
+       report(!cpus->d || (cpus->pp == 3 || cpus->pp == 0),
+              "Dedication versus entitlement");

Maybe skip here if the CPU is not dedicated? With shared CPUs we really can't check much here.

[...]

s390x/topology.c static void test_ptf(void) · line 358

Quoted 13 lines
s390x/topology.c +13 -0
+/*
+ * The Maximum Nested level is given by SCLP READ_SCP_INFO if the MNEST facility
+ * is available.
+ * If the MNEST facility is not available, sclp_get_stsi_mnest  returns 0 and the
+ * Maximum Nested level is 2
+ */
+#define S390_DEFAULT_MNEST     2
+static int sclp_get_mnest(void)
+{
+       return sclp_get_stsi_mnest() ?: S390_DEFAULT_MNEST;
+}
+
+static int arch_max_cpus(void)

If arch_topo_lvl is renamed, also rename this function accordingly.

Quoted 6 lines
 static struct {
        const char *name;
        void (*func)(void);
 } tests[] = {
        { "PTF", test_ptf},

+ { "STSI", test_stsi},

missing space ^

#8 lore raw

Re: [kvm-unit-tests PATCH v7 2/2] s390x: topology: Checking Configuration Topology Information

Quoted 16 lines · Nico Boehr

Quoting Pierre Morel (2023-03-20 09:56:42)

Quoted 14 lines

STSI with function code 15 is used to store the CPU configuration topology.

We retrieve the maximum nested level with SCLP and use the topology tree provided by the drawers, books, sockets, cores arguments.

We check :
- if the topology stored is coherent between the QEMU -smp parameters and kernel parameters.
- the number of CPUs
- the maximum number of CPUs
- the number of containers of each levels for every STSI(15.1.x) instruction allowed by the machine.

[...]

Quoted 13 lines

lib/s390x/sclp.c uint64_t get_max_ram_size(void) · line 243

Quoted 11 lines
lib/s390x/sclp.c +3 -0
index 390fde7..9679332 100644
@@ -238,3 +238,8 @@ uint64_t get_max_ram_size(void)
  {
         return max_ram_size;
  }
+
+uint64_t sclp_get_stsi_mnest(void)
+{

maybe add:
assert(read_info);

right

Quoted 14 lines

[...]

Quoted 4 lines
s390x/topology.c +0 -0
index ce248f1..11ce931 100644

[...]

s390x/topology.c line 32

Quoted 5 lines
s390x/topology.c +5 -0
+/*
+ * Topology level as defined by architecture, all levels exists with
+ * a single container unless overwritten by the QEMU -smp parameter.
+ */
+static int arch_topo_lvl[CPU_TOPOLOGY_MAX_LEVEL]; // = {1, 1, 1, 1, 1, 1};

So that's what is being provided to the test on the command line, right?

Quoted 3 lines

How about renaming this to expected_topo_lvl?

What do you mean by 'defined by architecture'?

This is what is provided by the boot arguments and should correspond to the physical topology.

The test checks that this is corresponding to what LPAR or QEMU shows in the SYSIB.

If a topology level always exist physically and if it is not specified on the QEMU command line it is implicitly unique.

OK for expected_topo_lvl if you prefer.

Quoted 13 lines

[...]

Quoted 11 lines
s390x/topology.c +11 -0
+/*
+ * stsi_check_mag
+ * @info: Pointer to the stsi information
+ *
+ * MAG field should match the architecture defined containers
+ * when MNEST as returned by SCLP matches MNEST of the SYSIB.
+ */
+static void stsi_check_mag(struct sysinfo_15_1_x *info)
+{
+       int i;
+
Quoted 8 lines

s390x/topology.c static void test_ptf(void) · line 198

Quoted 7 lines
s390x/topology.c +7 -0
+       report_prefix_push("MAG");
+
+       stsi_check_maxcpus(info);
+
+       /* Explicitly skip the test if both mnest do not match */
+       if (max_nested_lvl != info->mnest)
+               goto done;

What does it mean if the two don't match, i.e. is this an error? Or a skip? Or is it just expected?

I have no information on the representation of the MAG fields for a SYSIB with a nested level different than the maximum nested level.

There are examples in the documentation but I did not find, and did not get a clear answer, on how the MAG field are calculated.

The examples seems clear for info->mnest between MNEST -1 and 3 but the explication I had on info->mnest = 2 is not to be found in any documentation.

Until it is specified in a documentation I skip all these tests.

Quoted 12 lines

[...]

Quoted 9 lines
s390x/topology.c +9 -0
+/**
+ * check_tle:
+ * @tc: pointer to first TLE
+ *
+ * Recursively check the containers TLEs until we
+ * find a CPU TLE.
+ */
+static uint8_t *check_tle(void *tc)
+{

[...]

Quoted 3 lines

s390x/topology.c static void test_ptf(void) · line 262

Quoted 2 lines
s390x/topology.c +2 -0
+       report(!cpus->d || (cpus->pp == 3 || cpus->pp == 0),
+              "Dedication versus entitlement");

Maybe skip here if the CPU is not dedicated? With shared CPUs we really can't check much here.

OK

Quoted 15 lines

[...]

s390x/topology.c static void test_ptf(void) · line 358

Quoted 13 lines
s390x/topology.c +13 -0
+/*
+ * The Maximum Nested level is given by SCLP READ_SCP_INFO if the MNEST facility
+ * is available.
+ * If the MNEST facility is not available, sclp_get_stsi_mnest  returns 0 and the
+ * Maximum Nested level is 2
+ */
+#define S390_DEFAULT_MNEST     2
+static int sclp_get_mnest(void)
+{
+       return sclp_get_stsi_mnest() ?: S390_DEFAULT_MNEST;
+}
+
+static int arch_max_cpus(void)

If arch_topo_lvl is renamed, also rename this function accordingly.

OK

Quoted 8 lines
Quoted 7 lines
  static struct {
         const char *name;
         void (*func)(void);
  } tests[] = {
         { "PTF", test_ptf},

+ { "STSI", test_stsi},

missing space ^

Right, thanks,

regards,

Pierre

#9 lore raw

Re: [kvm-unit-tests PATCH v7 2/2] s390x: topology: Checking Configuration Topology Information

Quoting Pierre Morel (2023-03-27 14:38:35)

Quoted 15 lines
Quoted 15 lines

[...]

Quoted 4 lines
s390x/topology.c +0 -0
index ce248f1..11ce931 100644

[...]

s390x/topology.c line 32

Quoted 5 lines
s390x/topology.c +5 -0
+/*
+ * Topology level as defined by architecture, all levels exists with
+ * a single container unless overwritten by the QEMU -smp parameter.
+ */
+static int arch_topo_lvl[CPU_TOPOLOGY_MAX_LEVEL]; // = {1, 1, 1, 1, 1, 1};

So that's what is being provided to the test on the command line, right?

How about renaming this to expected_topo_lvl?

Quoted 7 lines
Quoted 1 line

What do you mean by 'defined by architecture'?

This is what is provided by the boot arguments and should correspond to the physical topology.

The test checks that this is corresponding to what LPAR or QEMU shows in the SYSIB.

Yep, OK. Makes sense.

Quoted 2 lines

If a topology level always exist physically and if it is not specified on the QEMU command line it is implicitly unique.

What do you mean by 'implicitly unique'?

Quoted 1 line

OK for expected_topo_lvl if you prefer.

Yes, please.

Quoted 24 lines
Quoted 12 lines

[...]

Quoted 11 lines
s390x/topology.c +11 -0
+/*
+ * stsi_check_mag
+ * @info: Pointer to the stsi information
+ *
+ * MAG field should match the architecture defined containers
+ * when MNEST as returned by SCLP matches MNEST of the SYSIB.
+ */
+static void stsi_check_mag(struct sysinfo_15_1_x *info)
+{
+       int i;
+
Quoted 8 lines

s390x/topology.c static void test_ptf(void) · line 198

Quoted 7 lines
s390x/topology.c +7 -0
+       report_prefix_push("MAG");
+
+       stsi_check_maxcpus(info);
+
+       /* Explicitly skip the test if both mnest do not match */
+       if (max_nested_lvl != info->mnest)
+               goto done;

What does it mean if the two don't match, i.e. is this an error? Or a skip? Or is it just expected?

I have no information on the representation of the MAG fields for a SYSIB with a nested level different than the maximum nested level.

Quoted 8 lines

There are examples in the documentation but I did not find, and did not get a clear answer, on how the MAG field are calculated.

The examples seems clear for info->mnest between MNEST -1 and 3 but the explication I had on info->mnest = 2 is not to be found in any documentation.

Until it is specified in a documentation I skip all these tests.

Alright - then please:
- update the comment to say:
"It is not clear how the MAG fields are calculated when mnest in the SYSIB 15.x is different from the maximum nested level in the SCLP info, so we skip here for now."
- when this is the case, do a report_skip() and show info->mnest and max_nested_lvl in the message.

#10 lore raw

Re: [kvm-unit-tests PATCH v7 2/2] s390x: topology: Checking Configuration Topology Information

Quoted 2 lines · Nico Boehr

Quoting Pierre Morel (2023-03-27 14:38:35)

Quoted 1 line
Quoted 1 line

[...]

[...]

Quoted 3 lines
Quoted 2 lines

If a topology level always exist physically and if it is not specified on the QEMU command line it is implicitly unique.

What do you mean by 'implicitly unique'?

I mean that if the topology level is not explicitly specified on the command line, it exists a single entity of this topology level.

Quoted 34 lines
Quoted 2 lines

OK for expected_topo_lvl if you prefer.

Yes, please.

Quoted 30 lines
Quoted 12 lines

[...]

Quoted 11 lines
s390x/topology.c +11 -0
+/*
+ * stsi_check_mag
+ * @info: Pointer to the stsi information
+ *
+ * MAG field should match the architecture defined containers
+ * when MNEST as returned by SCLP matches MNEST of the SYSIB.
+ */
+static void stsi_check_mag(struct sysinfo_15_1_x *info)
+{
+       int i;
+
Quoted 8 lines

s390x/topology.c static void test_ptf(void) · line 198

Quoted 7 lines
s390x/topology.c +7 -0
+       report_prefix_push("MAG");
+
+       stsi_check_maxcpus(info);
+
+       /* Explicitly skip the test if both mnest do not match */
+       if (max_nested_lvl != info->mnest)
+               goto done;

What does it mean if the two don't match, i.e. is this an error? Or a skip? Or is it just expected?

I have no information on the representation of the MAG fields for a SYSIB with a nested level different than the maximum nested level.

There are examples in the documentation but I did not find, and did not get a clear answer, on how the MAG field are calculated.

The examples seems clear for info->mnest between MNEST -1 and 3 but the explication I had on info->mnest = 2 is not to be found in any documentation.

Quoted 5 lines
Quoted 1 line

Until it is specified in a documentation I skip all these tests.

Alright - then please:
- update the comment to say:
"It is not clear how the MAG fields are calculated when mnest in the SYSIB 15.x is different from the maximum nested level in the SCLP info, so we skip here for now."
- when this is the case, do a report_skip() and show info->mnest and max_nested_lvl in the message.

OK, thanks.

Regards,

Pierre

#11 lore raw

Re: [kvm-unit-tests PATCH v7 2/2] s390x: topology: Checking Configuration Topology Information

Quoting Pierre Morel (2023-03-28 13:37:59)

Quoted 9 lines · Nico Boehr
Quoted 2 lines

Quoting Pierre Morel (2023-03-27 14:38:35)

Quoted 1 line
Quoted 1 line

[...]

[...]

Quoted 6 lines
Quoted 3 lines
Quoted 2 lines

If a topology level always exist physically and if it is not specified on the QEMU command line it is implicitly unique.

What do you mean by 'implicitly unique'?

I mean that if the topology level is not explicitly specified on the command line, it exists a single entity of this topology level.

OK, makes sense! Thanks!

#12 lore raw

Re: [kvm-unit-tests PATCH v7 2/2] s390x: topology: Checking Configuration Topology Information

In reply to Nico Boehr
Quoted 1 line · Nico Boehr

Quoting Pierre Morel (2023-03-20 09:56:42)

[...]

Quoted 10 lines
Quoted 9 lines

+ * check_tle:
+ * @tc: pointer to first TLE
+ *
+ * Recursively check the containers TLEs until we
+ * find a CPU TLE.
+ */

(quoted diff) +2 -0
+static uint8_t *check_tle(void *tc)
+{

[...]

Quoted 3 lines

s390x/topology.c static void test_ptf(void) · line 262

Quoted 2 lines
s390x/topology.c +2 -0
+       report(!cpus->d || (cpus->pp == 3 || cpus->pp == 0),
+              "Dedication versus entitlement");

Maybe skip here if the CPU is not dedicated? With shared CPUs we really can't check much here.

Thinking more about this I think I will not change this but I better will make two check one for horizontal and once for vertical polarization.