s390x: uv-host: Fixups and extensions part 1
cover s390x: uv-host: Fixups and extensions part 1
The uv-host test has a lot of historical growth problems which have largely been overlooked since running it is harder than running a KVM (guest 2) based test.
This series fixes up smaler problems but still leaves the test with fails when running create config base and variable storage tests. Those problems will either be fixed up with the second series or with a firmware fix since I'm unsure on which side of the os/fw fence the problem exists.
The series is based on my other series that introduces pv-ipl and pv-icpt. The memory allocation fix will be added to the new version of that series so all G1 tests are fixed.
Janosch Frank (8):
s390x: uv-host: Fix UV init test memory allocation
s390x: uv-host: Check for sufficient amount of memory
s390x: Add PV tests to unittests.cfg
s390x: uv-host: Beautify code
s390x: uv-host: Add cpu number check
s390x: uv-host: Fix create guest variable storage prefix check
s390x: uv-host: Properly handle config creation errors
s390x: uv-host: Fence access checks when UV debug is enabled
lib/s390x/asm/uv.h | 1 + s390x/unittests.cfg | 16 +++++++ s390x/uv-host.c | 101 +++++++++++++++++++++++++++++++++++--------- 3 files changed, 99 insertions(+), 19 deletions(-)
Signature · 1 line
2.34.1
PATCH 1/8 s390x: uv-host: Fix UV init test memory allocation
The init memory has to be above 2G and 1M aligned but we're currently aligning on 2G which means the allocations need a lot of unused memory.
Let's fix that.
--- s390x/uv-host.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
s390x/uv-host.c +1 -1
@@ -555,7 +555,7 @@ static void test_init(void) rc =uv_call (0 , (uint64_t )&uvcb_init);report (rc ==0 && uvcb_init.header.rc ==UVC_RC_EXECUTED ,"successful" ); - mem = (uint64_t )memalign (1UL <<31 , uvcb_qui.uv_base_stor_len); + mem = (uint64_t )memalign_pages_flags (HPAGE_SIZE , uvcb_qui.uv_base_stor_len,AREA_NORMAL ); rc =uv_call (0 , (uint64_t )&uvcb_init);report (rc ==1 && uvcb_init.header.rc ==0x101 ,"double init" );free ((void *)mem);
Signature · 1 line
2.34.1
Re: [kvm-unit-tests PATCH 1/8] s390x: uv-host: Fix UV init test memory allocation
Quoting Janosch Frank (2023-03-23 11:39:06)
Quoted 16 lines
The init memory has to be above 2G and 1M aligned but we're currently aligning on 2G which means the allocations need a lot of unused memory.
Let's fix that.
Signed-off-by Janosch Frank <[email protected]>--- s390x/uv-host.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)s390x/uv-host.c +0
-0
s390x/uv-host.c static void test_init(void) · line 561
Quoted 8 lines
s390x/uv-host.c +1
-1@@ -555,7 +555,7 @@ static void test_init(void) rc =uv_call (0 , (uint64_t )&uvcb_init);report (rc ==0 && uvcb_init.header.rc ==UVC_RC_EXECUTED ,"successful" ); - mem = (uint64_t )memalign (1UL <<31 , uvcb_qui.uv_base_stor_len); + mem = (uint64_t )memalign_pages_flags (HPAGE_SIZE , uvcb_qui.uv_base_stor_len,AREA_NORMAL ); rc =uv_call (0 , (uint64_t )&uvcb_init);report (rc ==1 && uvcb_init.header.rc ==0x101 ,"double init" );free ((void *)mem);
Your fix looks reasonable, but I think this is still broken, no?
We allocate a new mem, initalize the ultravisor again and then free the memory. We do absolutely nothing with the allocated memory, do we?
Are we maybe missing a
uvcb_init.stor_origin = mem;
here?
Also, a comment stating that AREA_NORMAL starts at PFN 524288 (2G) might be nice.
PATCH 2/8 s390x: uv-host: Check for sufficient amount of memory
In reply to Janosch FrankThe UV init storage needs to be above 2G so we need a little over 2G of memory when running the test.
--- s390x/uv-host.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
s390x/uv-host.c +11 -0
@@ -715,6 +715,17 @@ int main(void)test_invalid ();test_uv_uninitialized ();test_query (); + +/* +* Some of the UV memory needs to be allocated with >31 bit +* addresses which means we need a lot more memory than other +* tests. +*/ +if (get_ram_size () < (SZ_1M *2200UL )) { +report_skip ("Not enough memory. This test needs about 2200MB of memory" ); + goto done; + } +test_init ();setup_vmem ();
Signature · 1 line
2.34.1
Re: [kvm-unit-tests PATCH 2/8] s390x: uv-host: Check for sufficient amount of memory
Quoting Janosch Frank (2023-03-23 11:39:07)
Quoted 4 lines
The UV init storage needs to be above 2G so we need a little over 2G of memory when running the test.
Signed-off-by Janosch Frank <[email protected]>
PATCH 3/8 s390x: Add PV tests to unittests.cfg
In reply to Janosch FrankEven if the first thing those tests do is skipping they should still be run to make sure the tests boots and the skipping works.
--- s390x/unittests.cfg | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
s390x/unittests.cfg +16 -0
@@ -222,3 +222,19 @@ file = migration-cmm.elf groups = migration smp = 2 extra_params = -append '--parallel' + +[pv-diags] +file = pv-diags.elf +extra_params = -m 2200 + +[pv-icptcode] +file = pv-icptcode.elf +extra_params = -m 2200 -smp 3 + +[pv-ipl] +file = pv-ipl.elf +extra_params = -m 2200 + +[uv-host] +file = uv-host.elf +extra_params = -m 2200 -smp 2
Signature · 1 line
2.34.1
PATCH 4/8 s390x: uv-host: Beautify code
In reply to Janosch FrankFixup top comment and add missing space.
--- s390x/uv-host.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
s390x/uv-host.c +2 -2
@@ -1,6 +1,6 @@/* SPDX-License-Identifier: GPL-2.0-only */ /* -* Guest Ultravisor Call tests +* Host Ultravisor Call tests * * Copyright (c) 2021 IBM Corp *
@@ -33,7 +33,7 @@ static struct uv_cb_csc uvcb_csc;extern int diag308_load_reset (u64 code); -struct cmd_list { +struct cmd_list {const char *name;uint16_t cmd;uint16_t len;
Signature · 1 line
2.34.1
Re: [kvm-unit-tests PATCH 4/8] s390x: uv-host: Beautify code
Quoting Janosch Frank (2023-03-23 11:39:09)
Quoted 3 lines
Fixup top comment and add missing space.
Signed-off-by Janosch Frank <[email protected]>
PATCH 5/8 s390x: uv-host: Add cpu number check
In reply to Janosch FrankWe should only run a test that needs more than one cpu if a sufficient number of cpus are available.
--- s390x/uv-host.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
s390x/uv-host.c +9 -5
@@ -546,11 +546,15 @@ static void test_init(void) "storage below 2GB"); uvcb_init.stor_origin = mem; -smp_cpu_setup (1 ,PSW_WITH_CUR_MASK (cpu_loop)); - rc =uv_call (0 , (uint64_t )&uvcb_init); -report (rc ==1 && uvcb_init.header.rc ==0x102 , -"too many running cpus" ); -smp_cpu_stop (1 ); +if (smp_query_num_cpus () >1 ) { +smp_cpu_setup (1 ,PSW_WITH_CUR_MASK (cpu_loop)); + rc =uv_call (0 , (uint64_t )&uvcb_init); +report (rc ==1 && uvcb_init.header.rc ==0x102 , +"too many running cpus" ); +smp_cpu_stop (1 ); + }else { +report_skip ("Not enough cpus for 0x102 test" ); + } rc =uv_call (0 , (uint64_t )&uvcb_init);report (rc ==0 && uvcb_init.header.rc ==UVC_RC_EXECUTED ,"successful" );
Signature · 1 line
2.34.1
Re: [kvm-unit-tests PATCH 5/8] s390x: uv-host: Add cpu number check
Quoting Janosch Frank (2023-03-23 11:39:10)
Quoted 4 lines
We should only run a test that needs more than one cpu if a sufficient number of cpus are available.
Signed-off-by Janosch Frank <[email protected]>
PATCH 6/8 s390x: uv-host: Fix create guest variable storage prefix check
In reply to Janosch FrankWe want more than one cpu and the rc is 10B, not 109.
--- s390x/uv-host.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
s390x/uv-host.c +6 -2
@@ -434,11 +434,15 @@ static void test_config_create(void) "base storage origin contains lowcore"); uvcb_cgc.conf_base_stor_origin = tmp; -if (smp_query_num_cpus () ==1 ) { +/* +* Let's not make it too easy and use a second cpu to set a +* non-zero prefix. +*/ +if (smp_query_num_cpus () >1 ) {sigp_retry (1 ,SIGP_SET_PREFIX , uvcb_cgc.conf_var_stor_origin +PAGE_SIZE ,NULL ); rc =uv_call (0 , (uint64_t )&uvcb_cgc); -report (uvcb_cgc.header.rc ==0x10e && rc ==1 && +report (uvcb_cgc.header.rc ==0x10b && rc ==1 && !uvcb_cgc.guest_handle,"variable storage area contains lowcore" );sigp_retry (1 ,SIGP_SET_PREFIX ,0x0 ,NULL ); }
Signature · 1 line
2.34.1
Re: [kvm-unit-tests PATCH 6/8] s390x: uv-host: Fix create guest variable storage prefix check
Quoting Janosch Frank (2023-03-23 11:39:11)
Quoted 1 line
We want more than one cpu and the rc is 10B, not 109.
^
Code says 10e --------------------------------|
Quoted 12 lines
Signed-off-by Janosch Frank <[email protected]>--- s390x/uv-host.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)s390x/uv-host.c +0
-0@@ -434,11 +434,15 @@ static void test_config_create(void) "base storage origin contains lowcore");
s390x/uv-host.c static void test_config_create(void) · line 443
Quoted 10 lines
s390x/uv-host.c +5
-1uvcb_cgc.conf_base_stor_origin = tmp; -if (smp_query_num_cpus () ==1 ) { +/* +* Let's not make it too easy and use a second cpu to set a +* non-zero prefix. +*/ +if (smp_query_num_cpus () >1 ) {sigp_retry (1 ,SIGP_SET_PREFIX , uvcb_cgc.conf_var_stor_origin +PAGE_SIZE ,NULL );
While at it, this should be smp_sigp, no?
With the commit message fixup:
PATCH 7/8 s390x: uv-host: Properly handle config creation errors
In reply to Janosch FrankIf the first bit is set on a error rc, the hypervisor will need to destroy the config before trying again. Let's properly handle those cases so we're not usign stale data.
--- lib/s390x/asm/uv.h | 1 + s390x/uv-host.c | 52 ++++++++++++++++++++++++++++++++++++++-------- 2 files changed, 44 insertions(+), 9 deletions(-)
lib/s390x/asm/uv.h +1 -0
@@ -24,6 +24,7 @@#define UVC_RC_NO_RESUME 0x0007#define UVC_RC_INV_GHANDLE 0x0020#define UVC_RC_INV_CHANDLE 0x0021 +#define UVC_RC_DSTR_NEEDED_FLG 0x8000#define UVC_CMD_QUI 0x0001#define UVC_CMD_INIT_UV 0x000f
s390x/uv-host.c +43 -9
115 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [kvm-unit-tests PATCH 7/8] s390x: uv-host: Properly handle config creation errors
Quoting Janosch Frank (2023-03-23 11:39:12)
[...]
s390x/uv-host.c static void test_cpu_create(void) · line 377
Quoted 13 lines
s390x/uv-host.c +5
-0@@ -370,6 +370,38 @@ static void test_cpu_create(void)report_prefix_pop (); } +/* +* If the first bit of the rc is set we need to destroy the +* configuration before testing other create config errors. +*/ +static void cgc_destroy_if_needed (struct uv_cb_cgc *uvcb)
Is there a reason why we can't make this a cgc_uv_call() function which performs the uv_call and the cleanups if needed?
Mixing reports and cleanup activity feels a bit odd to me.
[...]
s390x/uv-host.c static void test_cpu_create(void) · line 390
Quoted 1 line
s390x/uv-host.c +1
-0+/* This function expects errors, not successes */
I am confused by this comment. What does it mean?
s390x/uv-host.c static void test_cpu_create(void) · line 391
Quoted 1 line
s390x/uv-host.c +1
-0+static bool cgc_check_data (struct uv_cb_cgc *uvcb,uint16_t rc_expected)
Rename to cgc_check_rc_and_handle?
s390x/uv-host.c static void test_cpu_create(void) · line 396
Quoted 5 lines
s390x/uv-host.c +5
-0+{ +cgc_destroy_if_needed (uvcb); +/* +* We should only receive a handle when the rc is 1 or the +* first bit is set.
Where is the code that checks for rc == 1?
Ah OK, so that's what you mean with the comment above, this function only works if the UVC fails, right?
s390x/uv-host.c static void test_cpu_create(void) · line 399
Quoted 3 lines
s390x/uv-host.c +3
-0+*/ +if (!(uvcb->header.rc &UVC_RC_DSTR_NEEDED_FLG ) && uvcb->guest_handle) +return false;
It would be nicer if I got a proper report message that tells me that we got a handle even though we shouldn't destroy.
Re: [kvm-unit-tests PATCH 7/8] s390x: uv-host: Properly handle config creation errors
Quoted 10 lines · Nico Boehr
Quoting Janosch Frank (2023-03-23 11:39:12)
[...]Quoted 8 lines
s390x/uv-host.c +0
-0@@ -370,6 +370,38 @@ static void test_cpu_create(void)report_prefix_pop (); }
Quoted 7 lines
s390x/uv-host.c static void test_cpu_create(void) · line 377
Quoted 5 lines
s390x/uv-host.c +5
-0+/* +* If the first bit of the rc is set we need to destroy the +* configuration before testing other create config errors. +*/ +static void cgc_destroy_if_needed (struct uv_cb_cgc *uvcb)Is there a reason why we can't make this a
cgc_uv_call()function which performs theuv_calland the cleanups if needed?
I'd much rather put the destroy into the cleanup area after the report.
Quoted 20 lines
Mixing reports and cleanup activity feels a bit odd to me.
[...]
s390x/uv-host.c static void test_cpu_create(void) · line 390
Quoted 1 line
s390x/uv-host.c +1
-0+/* This function expects errors, not successes */ I am confused by this comment. What does it mean?
s390x/uv-host.c static void test_cpu_create(void) · line 391
Quoted 1 line
s390x/uv-host.c +1
-0+static bool cgc_check_data (struct uv_cb_cgc *uvcb,uint16_t rc_expected)Rename to
cgc_check_rc_and_handle?s390x/uv-host.c static void test_cpu_create(void) · line 396
Quoted 5 lines
s390x/uv-host.c +5
-0+{ +cgc_destroy_if_needed (uvcb); +/* +* We should only receive a handle when the rc is 1 or the +* first bit is set. Where is the code that checks for rc == 1?
Quoted 7 lines
Ah OK, so that's what you mean with the comment above, this function only works if the UVC fails, right?
s390x/uv-host.c static void test_cpu_create(void) · line 399
Quoted 3 lines
s390x/uv-host.c +3
-0+*/ +if (!(uvcb->header.rc &UVC_RC_DSTR_NEEDED_FLG ) && uvcb->guest_handle) +return false;It would be nicer if I got a proper report message that tells me that we got a handle even though we shouldn't destroy.
We can report_info() or report_abort().
PATCH 8/8 s390x: uv-host: Fence access checks when UV debug is enabled
In reply to Janosch FrankThe debug print directly accesses the UV header which will result in a second accesses exception which will abort the test. Let's fence the access tests instead.
--- s390x/uv-host.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
s390x/uv-host.c +10 -0
@@ -164,6 +164,15 @@ static void test_access(void)report_prefix_push ("access" ); +/* +* If debug is enabled info from the uv header is printed +* which would lead to a second exception and a test abort. +*/ +if (UVC_ERR_DEBUG ) { +report_skip ("Debug doesn't work with access tests" ); + goto out; + } +report_prefix_push ("non-crossing" );protect_page (uvcb,PAGE_ENTRY_I );for (i =0 ; cmds[i].name; i++) {
@@ -196,6 +205,7 @@ static void test_access(void) uvcb +=1 ;unprotect_page (uvcb,PAGE_ENTRY_I ); +out:free_pages (pages);report_prefix_pop (); }
Signature · 1 line
2.34.1
Re: [kvm-unit-tests PATCH 8/8] s390x: uv-host: Fence access checks when UV debug is enabled
Quoting Janosch Frank (2023-03-23 11:39:13)
Quoted 5 lines
The debug print directly accesses the UV header which will result in a second accesses exception which will abort the test. Let's fence the access tests instead.
Signed-off-by Janosch Frank <[email protected]>