s390x: uv-host: Fixups and extensions part 1
v3 cover s390x: uv-host: Fixups and extensions part 1
The uv-host test has a lot of historical growth problems which have largely been overlooked since running it is harder than running a KVM (guest 2) based test.
This series fixes up smaller problems but still leaves the test with fails when running create config base and variable storage tests. Those problems will either be fixed up with the second series or with a firmware fix since I'm unsure on which side of the os/fw fence the problem exists.
The series is based on my other series that introduces pv-ipl and pv-icpt. The memory allocation fix will be added to the new version of that series so all G1 tests are fixed.
v3:
- Re-based on the ipl/icpt series
- Added review-bys
v2:
- Added patch that exchanges sigp_retry with the smp variant
- Re-worked the create config test handling
- Minor fixups
Janosch Frank (9):
s390x: uv-host: Fix UV init test memory allocation
s390x: uv-host: Check for sufficient amount of memory
s390x: uv-host: Beautify code
s390x: uv-host: Add cpu number check
s390x: uv-host: Fix create guest variable storage prefix check
s390x: uv-host: Switch to smp_sigp
s390x: uv-host: Properly handle config creation errors
s390x: uv-host: Fence access checks when UV debug is enabled
s390x: uv-host: Add the test to unittests.conf
lib/s390x/asm/uv.h | 1 + s390x/unittests.cfg | 5 ++ s390x/uv-host.c | 137 ++++++++++++++++++++++++++++++++++---------- 3 files changed, 114 insertions(+), 29 deletions(-)
Signature · 1 line
2.34.1
PATCH v3 1/9 s390x: uv-host: Fix UV init test memory allocation
The init memory has to be above 2G and 1M aligned but we're currently aligning on 2G which means the allocations need a lot of unused memory.
Also the second block of memory was never actually used for the double init test since its address is never put into the uvcb.
Let's fix that.
--- s390x/uv-host.c | 23 ++++++++++++++--------- 1 file changed, 14 insertions(+), 9 deletions(-)
s390x/uv-host.c +14 -9
@@ -500,14 +500,17 @@ static void test_config_create(void)static void test_init (void ) {int rc; -uint64_t mem; +uint64_t tmp; -/* Donated storage needs to be over 2GB */ - mem = (uint64_t )memalign_pages_flags (SZ_1M , uvcb_qui.uv_base_stor_len,AREA_NORMAL ); +/* +* Donated storage needs to be over 2GB, AREA_NORMAL does that +* on s390x. +*/ + tmp = (uint64_t )memalign_pages_flags (SZ_1M , uvcb_qui.uv_base_stor_len,AREA_NORMAL ); uvcb_init.header.len =sizeof (uvcb_init); uvcb_init.header.cmd =UVC_CMD_INIT_UV ; - uvcb_init.stor_origin = mem; + uvcb_init.stor_origin = tmp; uvcb_init.stor_len = uvcb_qui.uv_base_stor_len;report_prefix_push ("init" );
@@ -528,14 +531,14 @@ static void test_init(void) rc =uv_call (0 , (uint64_t )&uvcb_init);report (rc ==1 && (uvcb_init.header.rc ==0x104 || uvcb_init.header.rc ==0x105 ),"storage origin invalid" ); - uvcb_init.stor_origin = mem; + uvcb_init.stor_origin = tmp;if (uvcb_init.stor_len >=HPAGE_SIZE ) { uvcb_init.stor_origin =get_max_ram_size () -HPAGE_SIZE ; rc =uv_call (0 , (uint64_t )&uvcb_init);report (rc ==1 && uvcb_init.header.rc ==0x105 ,"storage + length invalid" ); - uvcb_init.stor_origin = mem; + uvcb_init.stor_origin = tmp; }else {report_skip ("storage + length invalid, stor_len < HPAGE_SIZE" ); }
@@ -544,7 +547,7 @@ static void test_init(void) rc =uv_call (0 , (uint64_t )&uvcb_init);report (rc ==1 && uvcb_init.header.rc ==0x108 ,"storage below 2GB" ); - uvcb_init.stor_origin = mem; + uvcb_init.stor_origin = tmp;smp_cpu_setup (1 ,PSW_WITH_CUR_MASK (cpu_loop)); rc =uv_call (0 , (uint64_t )&uvcb_init);
@@ -555,10 +558,12 @@ static void test_init(void) rc =uv_call (0 , (uint64_t )&uvcb_init);report (rc ==0 && uvcb_init.header.rc ==UVC_RC_EXECUTED ,"successful" ); - mem = (uint64_t )memalign (1UL <<31 , uvcb_qui.uv_base_stor_len); + tmp = uvcb_init.stor_origin; + uvcb_init.stor_origin = (uint64_t )memalign_pages_flags (HPAGE_SIZE , uvcb_qui.uv_base_stor_len,AREA_NORMAL ); rc =uv_call (0 , (uint64_t )&uvcb_init);report (rc ==1 && uvcb_init.header.rc ==0x101 ,"double init" ); -free ((void *)mem); +free ((void *)uvcb_init.stor_origin); + uvcb_init.stor_origin = tmp;report_prefix_pop (); }
Signature · 1 line
2.34.1
Re: [kvm-unit-tests PATCH v3 1/9] s390x: uv-host: Fix UV init test memory allocation
Quoting Janosch Frank (2023-05-02 15:07:24)
Quoted 3 lines
The init memory has to be above 2G and 1M aligned but we're currently aligning on 2G which means the allocations need a lot of unused memory.
I know I already gave my R-b here, but...
s390x/uv-host.c static void test_config_create(void) · line 506
Quoted 13 lines
s390x/uv-host.c +1
-3@@ -500,14 +500,17 @@ static void test_config_create(void)static void test_init (void ) {int rc; -uint64_t mem; +uint64_t tmp; -/* Donated storage needs to be over 2GB */ - mem = (uint64_t )memalign_pages_flags (SZ_1M , uvcb_qui.uv_base_stor_len,AREA_NORMAL );
...maybe out of coffee, but can you point me to the place where we're aligning to 2G here? I only see alignment to 1M and your change only seems to rename mem to tmp:
Quoted 5 lines
s390x/uv-host.c +5
-0+/* +* Donated storage needs to be over 2GB, AREA_NORMAL does that +* on s390x. +*/ + tmp = (uint64_t )memalign_pages_flags (SZ_1M , uvcb_qui.uv_base_stor_len,AREA_NORMAL );
Re: [kvm-unit-tests PATCH v3 1/9] s390x: uv-host: Fix UV init test memory allocation
Quoted 18 lines · Nico Boehr
Quoting Janosch Frank (2023-05-02 15:07:24)
Quoted 3 lines
The init memory has to be above 2G and 1M aligned but we're currently aligning on 2G which means the allocations need a lot of unused memory.
I know I already gave my R-b here, but...
Quoted 11 lines
s390x/uv-host.c +1
-1@@ -500,14 +500,17 @@ static void test_config_create(void)static void test_init (void ) {int rc; -uint64_t mem; +uint64_t tmp;
Quoted 11 lines
s390x/uv-host.c static void test_config_create(void) · line 506
Quoted 2 lines
s390x/uv-host.c +0
-2-/* Donated storage needs to be over 2GB */ - mem = (uint64_t )memalign_pages_flags (SZ_1M , uvcb_qui.uv_base_stor_len,AREA_NORMAL );...maybe out of coffee, but can you point me to the place where we're aligning to 2G here? I only see alignment to 1M and your change only seems to rename mem to tmp:
Quoted 4 lines
s390x/uv-host.c +4
-0+/* +* Donated storage needs to be over 2GB, AREA_NORMAL does that +* on s390x. +*/
This comment explains it :)
Its a re-name of mem to tmp and an extension of this comment so it makes more sense.
Quoted 1 line
Quoted 1 line
s390x/uv-host.c +1
-0+ tmp = (uint64_t )memalign_pages_flags (SZ_1M , uvcb_qui.uv_base_stor_len,AREA_NORMAL );
Nico Boehr
Reviewed-byRe: [kvm-unit-tests PATCH v3 1/9] s390x: uv-host: Fix UV init test memory allocation
Quoting Janosch Frank (2023-05-10 11:03:54)
Quoted 26 lines · Nico Boehr
Quoted 25 lines
Quoting Janosch Frank (2023-05-02 15:07:24)
Quoted 3 lines
The init memory has to be above 2G and 1M aligned but we're currently aligning on 2G which means the allocations need a lot of unused memory.
I know I already gave my R-b here, but...
s390x/uv-host.c static void test_config_create(void) · line 506
Quoted 13 lines
s390x/uv-host.c +1
-3@@ -500,14 +500,17 @@ static void test_config_create(void)static void test_init (void ) {int rc; -uint64_t mem; +uint64_t tmp; -/* Donated storage needs to be over 2GB */ - mem = (uint64_t )memalign_pages_flags (SZ_1M , uvcb_qui.uv_base_stor_len,AREA_NORMAL );...maybe out of coffee, but can you point me to the place where we're aligning to 2G here? I only see alignment to 1M and your change only seems to rename mem to tmp:
Quoted 8 lines
Quoted 4 lines
s390x/uv-host.c static void test_config_create(void) · line 508
Quoted 4 lines
s390x/uv-host.c +4
-0+/* +* Donated storage needs to be over 2GB, AREA_NORMAL does that +* on s390x. +*/ This comment explains it :)
Its a re-name of mem to tmp and an extension of this comment so it makes more sense.
Alright, thanks. I guess coffee level too low.
PATCH v3 2/9 s390x: uv-host: Check for sufficient amount of memory
In reply to Janosch FrankThe UV init storage needs to be above 2G so we need a little over 2G of memory when running the test.
--- s390x/uv-host.c | 8 ++++++++ 1 file changed, 8 insertions(+)
s390x/uv-host.c +8 -0
@@ -15,6 +15,7 @@#include <sclp.h> #include <smp.h> #include <uv.h> +#include <snippet.h> #include <mmu.h> #include <asm/page.h> #include <asm/sigp.h>
@@ -720,6 +721,13 @@ int main(void)test_invalid ();test_uv_uninitialized ();test_query (); + +if (get_ram_size () <SNIPPET_PV_MIN_MEM_SIZE ) { +report_skip ("Not enough memory. This test needs about %ld MB of memory" , +SNIPPET_PV_MIN_MEM_SIZE /1024 /1024 ); + goto done; + } +test_init ();setup_vmem ();
Signature · 1 line
2.34.1
PATCH v3 3/9 s390x: uv-host: Beautify code
In reply to Janosch FrankFixup top comment and add missing space.
--- s390x/uv-host.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
s390x/uv-host.c +2 -2
@@ -1,6 +1,6 @@/* SPDX-License-Identifier: GPL-2.0-only */ /* -* Guest Ultravisor Call tests +* Host Ultravisor Call tests * * Copyright (c) 2021 IBM Corp *
@@ -34,7 +34,7 @@ static struct uv_cb_csc uvcb_csc;extern int diag308_load_reset (u64 code); -struct cmd_list { +struct cmd_list {const char *name;uint16_t cmd;uint16_t len;
Signature · 1 line
2.34.1
PATCH v3 4/9 s390x: uv-host: Add cpu number check
In reply to Janosch FrankWe should only run a test that needs more than one cpu if a sufficient number of cpus are available.
--- s390x/uv-host.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
s390x/uv-host.c +9 -5
@@ -550,11 +550,15 @@ static void test_init(void) "storage below 2GB"); uvcb_init.stor_origin = tmp; -smp_cpu_setup (1 ,PSW_WITH_CUR_MASK (cpu_loop)); - rc =uv_call (0 , (uint64_t )&uvcb_init); -report (rc ==1 && uvcb_init.header.rc ==0x102 , -"too many running cpus" ); -smp_cpu_stop (1 ); +if (smp_query_num_cpus () >1 ) { +smp_cpu_setup (1 ,PSW_WITH_CUR_MASK (cpu_loop)); + rc =uv_call (0 , (uint64_t )&uvcb_init); +report (rc ==1 && uvcb_init.header.rc ==0x102 , +"too many running cpus" ); +smp_cpu_stop (1 ); + }else { +report_skip ("Not enough cpus for 0x102 test" ); + } rc =uv_call (0 , (uint64_t )&uvcb_init);report (rc ==0 && uvcb_init.header.rc ==UVC_RC_EXECUTED ,"successful" );
Signature · 1 line
2.34.1
PATCH v3 5/9 s390x: uv-host: Fix create guest variable storage prefix check
In reply to Janosch FrankWe want more than one cpu and the rc is 10B, not 109.
--- s390x/uv-host.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
s390x/uv-host.c +6 -2
@@ -435,11 +435,15 @@ static void test_config_create(void) "base storage origin contains lowcore"); uvcb_cgc.conf_base_stor_origin = tmp; -if (smp_query_num_cpus () ==1 ) { +/* +* Let's not make it too easy and use a second cpu to set a +* non-zero prefix. +*/ +if (smp_query_num_cpus () >1 ) {sigp_retry (1 ,SIGP_SET_PREFIX , uvcb_cgc.conf_var_stor_origin +PAGE_SIZE ,NULL ); rc =uv_call (0 , (uint64_t )&uvcb_cgc); -report (uvcb_cgc.header.rc ==0x10e && rc ==1 && +report (uvcb_cgc.header.rc ==0x10b && rc ==1 && !uvcb_cgc.guest_handle,"variable storage area contains lowcore" );sigp_retry (1 ,SIGP_SET_PREFIX ,0x0 ,NULL ); }
Signature · 1 line
2.34.1
PATCH v3 6/9 s390x: uv-host: Switch to smp_sigp
In reply to Janosch FrankLet's move to the new smp_sigp() interface which abstracts cpu numbers.
--- s390x/uv-host.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
s390x/uv-host.c +2 -2
@@ -440,12 +440,12 @@ static void test_config_create(void)* non-zero prefix. */ if (smp_query_num_cpus () >1 ) { -sigp_retry (1 ,SIGP_SET_PREFIX , +smp_sigp (1 ,SIGP_SET_PREFIX , uvcb_cgc.conf_var_stor_origin +PAGE_SIZE ,NULL ); rc =uv_call (0 , (uint64_t )&uvcb_cgc);report (uvcb_cgc.header.rc ==0x10b && rc ==1 && !uvcb_cgc.guest_handle,"variable storage area contains lowcore" ); -sigp_retry (1 ,SIGP_SET_PREFIX ,0x0 ,NULL ); +smp_sigp (1 ,SIGP_SET_PREFIX ,0x0 ,NULL ); } tmp = uvcb_cgc.guest_sca;
Signature · 1 line
2.34.1
Re: [kvm-unit-tests PATCH v3 6/9] s390x: uv-host: Switch to smp_sigp
Quoting Janosch Frank (2023-05-02 15:07:29)
Quoted 4 lines
Let's move to the new
smp_sigp()interface which abstracts cpu numbers.Signed-off-by Janosch Frank <[email protected]>
Did you intentionally drop my R-b?
anyways:
PATCH v3 7/9 s390x: uv-host: Properly handle config creation errors
In reply to Janosch FrankIf the first bit is set on a error rc, the hypervisor will need to destroy the config before trying again. Let's properly handle those cases so we're not using stale data.
--- lib/s390x/asm/uv.h | 1 + s390x/uv-host.c | 66 +++++++++++++++++++++++++++++++++++++++------- 2 files changed, 58 insertions(+), 9 deletions(-)
lib/s390x/asm/uv.h +1 -0
@@ -24,6 +24,7 @@#define UVC_RC_NO_RESUME 0x0007#define UVC_RC_INV_GHANDLE 0x0020#define UVC_RC_INV_CHANDLE 0x0021 +#define UVC_RC_DSTR_NEEDED_FLG 0x8000#define UVC_CMD_QUI 0x0001#define UVC_CMD_INIT_UV 0x000f
s390x/uv-host.c +57 -9
140 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [kvm-unit-tests PATCH v3 7/9] s390x: uv-host: Properly handle config creation errors
Quoting Janosch Frank (2023-05-02 15:07:30)
Quoted 5 lines
If the first bit is set on a error rc, the hypervisor will need to destroy the config before trying again. Let's properly handle those cases so we're not using stale data.
Signed-off-by Janosch Frank <[email protected]>
I am not a huge fan of the naming of cgc_check_data(), but since I can't really come up with a better suggestion let's avoid the bike shedding and be pragmatic:
PATCH v3 8/9 s390x: uv-host: Fence access checks when UV debug is enabled
In reply to Janosch FrankThe debug print directly accesses the UV header which will result in a second accesses exception which will abort the test. Let's fence the access tests instead.
--- s390x/uv-host.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
s390x/uv-host.c +10 -0
@@ -165,6 +165,15 @@ static void test_access(void)report_prefix_push ("access" ); +/* +* If debug is enabled info from the uv header is printed +* which would lead to a second exception and a test abort. +*/ +if (UVC_ERR_DEBUG ) { +report_skip ("Debug doesn't work with access tests" ); + goto out; + } +report_prefix_push ("non-crossing" );protect_page (uvcb,PAGE_ENTRY_I );for (i =0 ; cmds[i].name; i++) {
@@ -197,6 +206,7 @@ static void test_access(void) uvcb +=1 ;unprotect_page (uvcb,PAGE_ENTRY_I ); +out:free_pages (pages);report_prefix_pop (); }
Signature · 1 line
2.34.1
PATCH v3 9/9 s390x: uv-host: Add the test to unittests.conf
In reply to Janosch FrankBetter to skip than to not run it at all.
--- s390x/unittests.cfg | 5 +++++ 1 file changed, 5 insertions(+)
s390x/unittests.cfg +5 -0
@@ -231,3 +231,8 @@ extra_params = -m 2200 [pv-diags] file = pv-diags.elf extra_params = -m 2200 + +[uv-host] +file = uv-host.elf +smp = 2 +extra_params = -m 2200
Signature · 1 line
2.34.1
Re: [kvm-unit-tests PATCH v3 9/9] s390x: uv-host: Add the test to unittests.conf
Quoting Janosch Frank (2023-05-02 15:07:32)
Quoted 3 lines
Better to skip than to not run it at all.
Signed-off-by Janosch Frank <[email protected]>
Yes, that's a good idea.
Quoting Janosch Frank (2023-05-02 15:07:23)
Quoted 10 lines
The uv-host test has a lot of historical growth problems which have largely been overlooked since running it is harder than running a KVM (guest 2) based test.
This series fixes up smaller problems but still leaves the test with fails when running create config base and variable storage tests. Those problems will either be fixed up with the second series or with a firmware fix since I'm unsure on which side of the os/fw fence the problem exists.
Quoted 3 lines
The series is based on my other series that introduces pv-ipl and pv-icpt. The memory allocation fix will be added to the new version of that series so all G1 tests are fixed.
I have also pushed this to our CI, thanks.
Also here, I took the liberty of adding
groups = pv-host
in the last patch. If you are OK with it, I can carry that when picking for the PR.
Quoted 15 lines · Nico Boehr
Quoting Janosch Frank (2023-05-02 15:07:23)
Quoted 10 lines
The uv-host test has a lot of historical growth problems which have largely been overlooked since running it is harder than running a KVM (guest 2) based test.
This series fixes up smaller problems but still leaves the test with fails when running create config base and variable storage tests. Those problems will either be fixed up with the second series or with a firmware fix since I'm unsure on which side of the os/fw fence the problem exists.
Quoted 3 lines
The series is based on my other series that introduces pv-ipl and pv-icpt. The memory allocation fix will be added to the new version of that series so all G1 tests are fixed.
Quoted 6 lines
I have also pushed this to our CI, thanks.
Also here, I took the liberty of adding
groups = pv-host
in the last patch. If you are OK with it, I can carry that when picking for the PR.
Sure, since Thomas reviewed the patch introducing the group I'm fine with that.