s390x: Add PV SIE intercepts and ipl tests
v2 cover s390x: Add PV SIE intercepts and ipl tests
Extend the coverage of the UVC interface.
The patches might be a bit dusty, they've been on a branch for a while.
v2:
- Re-worked the cpu timer tests
- Testing both pages for 112 intercept
- Added skip on insufficient memory
- Fixed comments in pv-ipl.c
Janosch Frank (3):
lib: s390x: Introduce UV validity function
s390x: pv: Test sie entry intercepts and validities
s390x: pv: Add IPL reset tests
lib/s390x/uv.h | 7 + s390x/Makefile | 7 + s390x/pv-icptcode.c | 403 ++++++++++++++++++ s390x/pv-ipl.c | 246 +++++++++++ s390x/snippets/asm/snippet-loop.S | 12 + s390x/snippets/asm/snippet-pv-diag-308.S | 67 +++ s390x/snippets/asm/snippet-pv-icpt-112.S | 77 ++++ s390x/snippets/asm/snippet-pv-icpt-loop.S | 15 + .../snippets/asm/snippet-pv-icpt-vir-timing.S | 22 + 9 files changed, 856 insertions(+) create mode 100644 s390x/pv-icptcode.c create mode 100644 s390x/pv-ipl.c create mode 100644 s390x/snippets/asm/snippet-loop.S create mode 100644 s390x/snippets/asm/snippet-pv-diag-308.S create mode 100644 s390x/snippets/asm/snippet-pv-icpt-112.S create mode 100644 s390x/snippets/asm/snippet-pv-icpt-loop.S create mode 100644 s390x/snippets/asm/snippet-pv-icpt-vir-timing.S
Signature · 1 line
2.34.1
PATCH v2 1/3 lib: s390x: Introduce UV validity function
PV related validities are in the 0x20** range but the last byte might be implementation specific, so everytime we check for a UV validity we need to mask the last byte.
Let's add a function that checks for a UV validity and returns a boolean.
--- lib/s390x/uv.h | 7 +++++++ 1 file changed, 7 insertions(+)
lib/s390x/uv.h +7 -0
@@ -35,4 +35,11 @@ static inline void uv_setup_asces(void) lctlg(13, asce); } +static inline bool uv_validity_check (struct vm *vm) +{ +uint16_t vir =sie_get_validity (vm); + +return vm->sblk->icptcode ==ICPT_VALIDITY && (vir &0xff00 ) ==0x2000 ; +} +#endif /* UV_H */
Signature · 1 line
2.34.1
Re: [kvm-unit-tests PATCH v2 1/3] lib: s390x: Introduce UV validity function
Quoting Janosch Frank (2023-03-24 13:04:29)
Quoted 8 lines
PV related validities are in the 0x20** range but the last byte might be implementation specific, so everytime we check for a UV validity we need to mask the last byte.
Let's add a function that checks for a UV validity and returns a boolean.
Signed-off-by Janosch Frank <[email protected]>
Re: [kvm-unit-tests PATCH v2 1/3] lib: s390x: Introduce UV validity function
In reply to Janosch FrankQuoted 8 lines · On Fri, 24 Mar 2023 12:04:29 +0000 Janosch Frank <[email protected]> wrote:
PV related validities are in the 0x20** range but the last byte might be implementation specific, so everytime we check for a UV validity we need to mask the last byte.
Let's add a function that checks for a UV validity and returns a boolean.
Signed-off-by Janosch Frank <[email protected]>
Quoted 20 lines
--- lib/s390x/uv.h | 7 +++++++ 1 file changed, 7 insertions(+)lib/s390x/uv.h +7
-0@@ -35,4 +35,11 @@ static inline void uv_setup_asces(void) lctlg(13, asce); } +static inline bool uv_validity_check (struct vm *vm) +{ +uint16_t vir =sie_get_validity (vm); + +return vm->sblk->icptcode ==ICPT_VALIDITY && (vir &0xff00 ) ==0x2000 ; +} +#endif /* UV_H */
PATCH v2 2/3 s390x: pv: Test sie entry intercepts and validities
In reply to Janosch FrankThe lowcore is an important part of any s390 cpu so we need to make sure it's always available when we virtualize one. For non-PV guests that would mean ensuring that the lowcore page is read and writable by the guest.
For PV guests we additionally need to make sure that the page is owned by the guest as it is only allowed to access them if that's the case. The code 112 SIE intercept tells us if the lowcore pages aren't secure anymore.
Let's check if that intercept is reported by SIE if we export the lowcore pages. Additionally check if that's also the case if the guest shares the lowcore which will make it readable to the host but ownership of the page should not change.
Also we check for validities in these conditions:
* Manipulated cpu timer
* Double SIE for same vcpu
* Re-use of VCPU handle from another secure configuration
* ASCE re-use
--- s390x/Makefile | 5 + s390x/pv-icptcode.c | 403 ++++++++++++++++++ s390x/snippets/asm/snippet-loop.S | 12 + s390x/snippets/asm/snippet-pv-icpt-112.S | 77 ++++ s390x/snippets/asm/snippet-pv-icpt-loop.S | 15 + .../snippets/asm/snippet-pv-icpt-vir-timing.S | 22 + 6 files changed, 534 insertions(+) create mode 100644 s390x/pv-icptcode.c create mode 100644 s390x/snippets/asm/snippet-loop.S create mode 100644 s390x/snippets/asm/snippet-pv-icpt-112.S create mode 100644 s390x/snippets/asm/snippet-pv-icpt-loop.S create mode 100644 s390x/snippets/asm/snippet-pv-icpt-vir-timing.S
s390x/Makefile +5 -0
@@ -41,6 +41,7 @@ tests += $(TEST_DIR)/migration-sck.elf tests += $(TEST_DIR)/exittime.elf pv-tests += $(TEST_DIR)/pv-diags.elf +pv-tests += $(TEST_DIR)/pv-icptcode.elf ifneq ($(HOST_KEY_DOCUMENT),) ifneq ($(GEN_SE_HEADER),)
@@ -119,6 +120,10 @@ $(TEST_DIR)/spec_ex-sie.elf: snippets = $(SNIPPET_DIR)/c/spec_ex.gbin $(TEST_DIR)/pv-diags.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-diag-yield.gbin $(TEST_DIR)/pv-diags.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-diag-288.gbin $(TEST_DIR)/pv-diags.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-diag-500.gbin +$(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-112.gbin +$(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-loop.gbin +$(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-loop.gbin +$(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-vir-timing.gbin ifneq ($(GEN_SE_HEADER),) snippets += $(pv-snippets)
s390x/pv-icptcode.c new file +403 -0
403 lines not on this page · raw message
s390x/snippets/asm/snippet-loop.S new file +12 -0
@@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Infinite loop snippet with no exit + * + * Copyright (c) 2023 IBM Corp + * + * Authors: + * Janosch Frank <[email protected]> + */ +.section .text +retry: +j retry
s390x/snippets/asm/snippet-pv-icpt-112.S new file +77 -0
77 lines not on this page · raw message
s390x/snippets/asm/snippet-pv-icpt-loop.S new file +15 -0
@@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Infinite loop snippet which can be used to test manipulated SIE + * control block intercepts. E.g. when manipulating the PV handles. + * + * Copyright (c) 2023 IBM Corp + * + * Authors: + * Janosch Frank <[email protected]> + */ +.section .text +xgr %r0, %r0 +retry: +diag 0,0,0x44 +j retry
s390x/snippets/asm/snippet-pv-icpt-vir-timing.S new file +22 -0
@@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Sets a cpu timer which the host can manipulate to check if it will + *receive a validity + * + * Copyright (c) 2023 IBM Corp + * + * Authors: + * Janosch Frank <[email protected]> + */ +.section .text +larl %r1, time_val +spt 0(%r1) +diag 0,0,0x44 +xgr %r1, %r1 +lghi %r1, 42 +diag 1,0,0x9c + + +.align 8 +time_val: + .quad 0x280de80000
Signature · 1 line
2.34.1
Re: [kvm-unit-tests PATCH v2 2/3] s390x: pv: Test sie entry intercepts and validities
Quoted 120 lines · On Fri, 24 Mar 2023 12:04:30 +0000 Janosch Frank <[email protected]> wrote:
The lowcore is an important part of any s390 cpu so we need to make sure it's always available when we virtualize one. For non-PV guests that would mean ensuring that the lowcore page is read and writable by the guest.
For PV guests we additionally need to make sure that the page is owned by the guest as it is only allowed to access them if that's the case. The code 112 SIE intercept tells us if the lowcore pages aren't secure anymore.
Let's check if that intercept is reported by SIE if we export the lowcore pages. Additionally check if that's also the case if the guest shares the lowcore which will make it readable to the host but ownership of the page should not change.
Also we check for validities in these conditions:
* Manipulated cpu timer
* Double SIE for same vcpu
* Re-use of VCPU handle from another secure configuration
* ASCE re-useSigned-off-by Janosch Frank <[email protected]>--- s390x/Makefile | 5 + s390x/pv-icptcode.c | 403 ++++++++++++++++++ s390x/snippets/asm/snippet-loop.S | 12 + s390x/snippets/asm/snippet-pv-icpt-112.S | 77 ++++ s390x/snippets/asm/snippet-pv-icpt-loop.S | 15 + .../snippets/asm/snippet-pv-icpt-vir-timing.S | 22 + 6 files changed, 534 insertions(+) create mode 100644 s390x/pv-icptcode.c create mode 100644 s390x/snippets/asm/snippet-loop.S create mode 100644 s390x/snippets/asm/snippet-pv-icpt-112.S create mode 100644 s390x/snippets/asm/snippet-pv-icpt-loop.S create mode 100644 s390x/snippets/asm/snippet-pv-icpt-vir-timing.Ss390x/Makefile +5
-0@@ -41,6 +41,7 @@ tests += $(TEST_DIR)/migration-sck.elf tests += $(TEST_DIR)/exittime.elf pv-tests += $(TEST_DIR)/pv-diags.elf +pv-tests += $(TEST_DIR)/pv-icptcode.elf ifneq ($(HOST_KEY_DOCUMENT),) ifneq ($(GEN_SE_HEADER),)@@ -119,6 +120,10 @@ $(TEST_DIR)/spec_ex-sie.elf: snippets = $(SNIPPET_DIR)/c/spec_ex.gbin $(TEST_DIR)/pv-diags.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-diag-yield.gbin $(TEST_DIR)/pv-diags.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-diag-288.gbin $(TEST_DIR)/pv-diags.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-diag-500.gbin +$(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-112.gbin +$(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-loop.gbin +$(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-loop.gbin +$(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-vir-timing.gbin ifneq ($(GEN_SE_HEADER),) snippets += $(pv-snippets)s390x/pv-icptcode.c new file +55
-0@@ -0,0 +1,403 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* PV virtualization interception tests for intercepts that are not +* caused by an instruction. +* +* Copyright (c) 2023 IBM Corp +* +* Authors: +* Janosch Frank <[email protected]> +*/ +#include <libcflat.h> +#include <sie.h> +#include <smp.h> +#include <sclp.h> +#include <snippet.h> +#include <asm/facility.h> +#include <asm/barrier.h> +#include <asm/sigp.h> +#include <asm/uv.h> +#include <asm/time.h> + +static struct vm vm, vm2; + +/* +* The hypervisor should not be able to decrease the cpu timer by an +* amount that is higher than the amount of time spent outside of +* SIE. +* +* Warning: A lot of things influence time so decreasing the timer by +* a more significant amount than the difference to have a safety +* margin is advised. +*/ +static void test_validity_timing (void ) +{ +extern const char SNIPPET_NAME_START (asm ,snippet_pv_icpt_vir_timing )[]; +extern const char SNIPPET_NAME_END (asm ,snippet_pv_icpt_vir_timing )[]; +extern const char SNIPPET_HDR_START (asm ,snippet_pv_icpt_vir_timing )[]; +extern const char SNIPPET_HDR_END (asm ,snippet_pv_icpt_vir_timing )[]; + int size_hdr = SNIPPET_HDR_LEN(asm, snippet_pv_icpt_vir_timing); + int size_gbin = SNIPPET_LEN(asm, snippet_pv_icpt_vir_timing); +uint64_t time_exit, time_entry; + +report_prefix_push ("manipulated cpu time" ); + snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_icpt_vir_timing), + SNIPPET_HDR_START(asm, snippet_pv_icpt_vir_timing), + size_gbin, size_hdr,SNIPPET_UNPACK_OFF ); + +sie (&vm); +report (vm.sblk->icptcode ==ICPT_PV_NOTIFY && vm.sblk->ipa ==0x8302 && + vm.sblk->ipb ==0x50000000 && vm.save_area.guest.grs[5 ] ==0x44 , +"stp done" ); +stck (&time_exit); +mb (); + +
s390x/pv-icptcode.c line 65
Quoted 10 lines
s390x/pv-icptcode.c new file +10
-0+/* Cpu timer counts down so adding a us should lead to a validity */ + vm.sblk->cputm +=S390_CLOCK_SHIFT_US ; +sie_expect_validity (&vm); +sie (&vm); +report (uv_validity_check (&vm),"validity entry cput > exit cput" ); + vm.sblk->cputm -=S390_CLOCK_SHIFT_US ; + +/* Cpu timer counts down so adding a us should lead to a validity */ +stck (&time_entry); + vm.sblk->cputm -= (time_entry - time_exit) +S390_CLOCK_SHIFT_US ;
this relies on the fact that the overhead between the actual exit and the first stck combined with the overhead between the second stck and the sie entry never exceeds 1µs. I think this is a little too tight, maybe you can add 1ms instead?
Quoted 32 lines
s390x/pv-icptcode.c new file +32
-0+sie_expect_validity (&vm); +sie (&vm); +report (uv_validity_check (&vm),"validity entry cput < time spent outside SIE" ); + +/* Todo: Handle wrapping */ +uv_destroy_guest (&vm); +report_prefix_pop (); +} + +static void run_loop (void ) +{ +sie (&vm); +sigp_retry (stap (),SIGP_STOP ,0 ,NULL ); +} + +static void test_validity_already_running (void ) +{ +extern const char SNIPPET_NAME_START (asm ,snippet_loop )[]; +extern const char SNIPPET_NAME_END (asm ,snippet_loop )[]; +extern const char SNIPPET_HDR_START (asm ,snippet_loop )[]; +extern const char SNIPPET_HDR_END (asm ,snippet_loop )[]; + int size_hdr = SNIPPET_HDR_LEN(asm, snippet_loop); + int size_gbin = SNIPPET_LEN(asm, snippet_loop); +struct psw psw = { + .mask =PSW_MASK_64 , + .addr = (uint64_t )run_loop, + }; + +report_prefix_push ("already running" ); +if (smp_query_num_cpus () <3 ) { +report_skip ("need at least 3 cpus for this test" ); + goto out;
s390x/pv-icptcode.c line 107
Quoted 10 lines
s390x/pv-icptcode.c new file +10
-0+ } + + snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_loop), + SNIPPET_HDR_START(asm, snippet_loop), + size_gbin, size_hdr,SNIPPET_UNPACK_OFF ); + +sie_expect_validity (&vm); +smp_cpu_setup (1 , psw); +smp_cpu_setup (2 , psw); +while (vm.sblk->icptcode !=ICPT_VALIDITY ) {mb (); }
please put it on multiple lines
s390x/pv-icptcode.c line 108
Quoted 1 line
s390x/pv-icptcode.c new file +1
-0+/* Yes I know this is not reliable as one cpu might overwrite it */
please rephrase :)
also, are you sure that cpu1 could overwrite it? it's running in an endless loop and IIRC we have all other interrupts disabled
Quoted 9 lines
s390x/pv-icptcode.c new file +9
-0+report (uv_validity_check (&vm),"validity" ); +smp_cpu_stop (1 ); +smp_cpu_stop (2 ); +uv_destroy_guest (&vm); + +out: +report_prefix_pop (); +} +
[...]
Quoted 19 lines
s390x/pv-icptcode.c new file +19
-0+/* +* Tests if we get a validity intercept if the CR1 asce at SIE entry +* is not the same as the one given at the UV creation of the VM. +*/ +static void test_validity_asce (void ) +{ +extern const char SNIPPET_NAME_START (asm ,snippet_pv_icpt_112 )[]; +extern const char SNIPPET_NAME_END (asm ,snippet_pv_icpt_112 )[]; +extern const char SNIPPET_HDR_START (asm ,snippet_pv_icpt_112 )[]; +extern const char SNIPPET_HDR_END (asm ,snippet_pv_icpt_112 )[]; + int size_hdr = SNIPPET_HDR_LEN(asm, snippet_pv_icpt_112); + int size_gbin = SNIPPET_LEN(asm, snippet_pv_icpt_112); +uint64_t asce_old, asce_new; +void *pgd_new, *pgd_old; + +report_prefix_push ("asce" ); + snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_icpt_112), + SNIPPET_HDR_START(asm, snippet_pv_icpt_112), + size_gbin, size_hdr,SNIPPET_UNPACK_OFF );
s390x/pv-icptcode.c line 230
Quoted 10 lines
s390x/pv-icptcode.c new file +10
-0+ + asce_old = vm.save_area.guest.asce; + pgd_new =memalign_pages_flags (PAGE_SIZE ,PAGE_SIZE *4 ,0 ); + pgd_old = (void *)(asce_old &PAGE_MASK ); + +/* Copy the contents of the top most table */ +memcpy (pgd_new, pgd_old,PAGE_SIZE *4 ); + +/* Create the replacement ASCE */ + asce_new =__pa (pgd_new) |ASCE_DT_REGION1 |REGION_TABLE_LENGTH |ASCE_P ;
why not __pa(pgd_new) | (asce_old & ~ASCE_ORIGIN) ?
that way you don't have to worry if things change in the future
Quoted 68 lines
s390x/pv-icptcode.c new file +68
-0+ vm.save_area.guest.asce = asce_new; + +sie_expect_validity (&vm); +sie (&vm); +report (uv_validity_check (&vm),"wrong CR1 validity" ); + +/* Restore the old ASCE */ + vm.save_area.guest.asce = asce_old; + +/* Try if we can still do an entry with the correct asce */ +sie (&vm); +report (vm.sblk->icptcode ==ICPT_PV_NOTIFY && vm.sblk->ipa ==0x8302 && + vm.sblk->ipb ==0x50000000 && vm.save_area.guest.grs[5 ] ==0x44 , +"re-entry with valid CR1" ); +uv_destroy_guest (&vm); +free_pages (pgd_new); +report_prefix_pop (); +} + +static void test_icpt_112 (void ) +{ +extern const char SNIPPET_NAME_START (asm ,snippet_pv_icpt_112 )[]; +extern const char SNIPPET_NAME_END (asm ,snippet_pv_icpt_112 )[]; +extern const char SNIPPET_HDR_START (asm ,snippet_pv_icpt_112 )[]; +extern const char SNIPPET_HDR_END (asm ,snippet_pv_icpt_112 )[]; + int size_hdr = SNIPPET_HDR_LEN(asm, snippet_pv_icpt_112); + int size_gbin = SNIPPET_LEN(asm, snippet_pv_icpt_112); + +u64 lc_off =0 ; + +report_prefix_push ("prefix" ); + + snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_icpt_112), + SNIPPET_HDR_START(asm, snippet_pv_icpt_112), + size_gbin, size_hdr,SNIPPET_UNPACK_OFF ); + +report_prefix_push ("0x0" ); +sie (&vm); + +/* Guest indicates that it has been setup via the diag 0x44 */ +report (vm.sblk->icptcode ==ICPT_PV_NOTIFY && vm.sblk->ipa ==0x8302 && + vm.sblk->ipb ==0x50000000 && vm.save_area.guest.grs[5 ] ==0x44 , +"guest set up" ); + +/* +* Let's export the standard prefix 0x0 and check for the 112 +* intercept. +*/ +uv_export (vm.sblk->mso + lc_off); +sie (&vm); +report (vm.sblk->icptcode ==ICPT_PV_PREF ,"Intercept 112 for page 0" ); +uv_import (vm.uv.vm_handle, vm.sblk->mso + lc_off); + +uv_export (vm.sblk->mso + lc_off +PAGE_SIZE ); +report (vm.sblk->icptcode ==ICPT_PV_PREF ,"Intercept 112 for page 1" ); + +uv_import (vm.uv.vm_handle, vm.sblk->mso + lc_off +PAGE_SIZE ); + +/* +* Guest will share the lowcore and we need to check if that +* makes a difference (which it should not). +*/ +report_prefix_push ("shared" ); +sie (&vm); +/* Guest indicates that it has been setup via the diag 0x44 */ +report (vm.sblk->icptcode ==ICPT_PV_NOTIFY && vm.sblk->ipa ==0x8302 && + vm.sblk->ipb ==0x50000000 && vm.save_area.guest.grs[5 ] ==0x44 , +"guest shared the first lowcore page" );
s390x/pv-icptcode.c line 308
Quoted 10 lines
s390x/pv-icptcode.c new file +10
-0+ +/* +* Let's export the standard prefix 0x0 and check for the 112 +* intercept. +*/ +uv_export (vm.sblk->mso + lc_off); +sie (&vm); +report (vm.sblk->icptcode ==ICPT_PV_PREF ,"Intercept 112" ); +uv_import (vm.uv.vm_handle, vm.sblk->mso + lc_off); +report_prefix_pop ();
for completeness, you should check the second prefix page as well
s390x/snippets/asm/snippet-pv-icpt-loop.S
Quoted 201 lines
s390x/pv-icptcode.c new file +95
-0+ +report_prefix_pop (); + + +report_prefix_push ("0x8000" ); +/* +* Import the new prefix pages so we don't get a PGM 0x3E on +* the guest's spx instruction. +*/ + lc_off =0x8000 ; +uv_import (vm.uv.vm_handle, vm.sblk->mso + lc_off); +uv_import (vm.uv.vm_handle, vm.sblk->mso + lc_off +PAGE_SIZE ); +sie (&vm); + +/* SPX generates a PV instruction notification */ +report (vm.sblk->icptcode ==ICPT_PV_NOTIFY && vm.sblk->ipa ==0xb210 , +"Received a PV instruction notification intercept for spx" ); +report (*(u32 *)vm.sblk->sidad ==0x8000 ,"New prefix is 0x8000" ); + +/* Let's export the prefix at 0x8000 and check for the 112 intercept */ +uv_export (vm.sblk->mso + lc_off); +sie (&vm); +report (vm.sblk->icptcode ==ICPT_PV_PREF ,"Intercept 112 for page 0" ); +uv_import (vm.uv.vm_handle, vm.sblk->mso + lc_off); + +uv_export (vm.sblk->mso + lc_off +PAGE_SIZE ); +report (vm.sblk->icptcode ==ICPT_PV_PREF ,"Intercept 112 for page 1" ); + +uv_import (vm.uv.vm_handle, vm.sblk->mso + lc_off +PAGE_SIZE ); + +report_prefix_push ("shared" ); +sie (&vm); +/* Guest indicates that it has shared the new lowcore */ +report (vm.sblk->icptcode ==ICPT_PV_NOTIFY && vm.sblk->ipa ==0x8302 && + vm.sblk->ipb ==0x50000000 && vm.save_area.guest.grs[5 ] ==0x44 , +"intercept values" ); + +uv_export (vm.sblk->mso + lc_off); +sie (&vm); +report (vm.sblk->icptcode ==ICPT_PV_PREF ,"Intercept 112 for page 0" ); +uv_import (vm.uv.vm_handle, vm.sblk->mso + lc_off); + +uv_export (vm.sblk->mso + lc_off +PAGE_SIZE ); +sie (&vm); +report (vm.sblk->icptcode ==ICPT_PV_PREF ,"Intercept 112 for page 1" ); +uv_import (vm.uv.vm_handle, vm.sblk->mso + lc_off +PAGE_SIZE ); + +/* Try a re-entry */ +sie (&vm); +report (vm.sblk->icptcode ==ICPT_PV_NOTIFY && vm.sblk->ipa ==0x8302 && + vm.sblk->ipb ==0x50000000 && vm.save_area.guest.grs[5 ] ==0x9c && + vm.save_area.guest.grs[0 ] ==42 , +"re-entry successful" ); +report_prefix_pop (); + +report_prefix_pop (); + +report_prefix_pop (); +uv_destroy_guest (&vm); +} + +int main (void ) +{ +report_prefix_push ("pv-icpts" ); +if (!test_facility (158 )) { +report_skip ("UV Call facility unavailable" ); + goto done; + } +if (!sclp_facilities.has_sief2) { +report_skip ("SIEF2 facility unavailable" ); + goto done; + } +/* +* Some of the UV memory needs to be allocated with >31 bit +* addresses which means we need a lot more memory than other +* tests. +*/ +if (get_ram_size () < (SZ_1M *2200UL )) { +report_skip ("Not enough memory. This test needs about 2200MB of memory" ); + goto done; + } + +snippet_setup_guest (&vm, true); +test_icpt_112 (); +test_validity_asce (); +test_validity_seid (); +test_validity_handle_not_in_config (); +test_validity_already_running (); +test_validity_timing (); +sie_guest_destroy (&vm); + +done: +report_prefix_pop (); +return report_summary (); +}s390x/snippets/asm/snippet-loop.S new file +12
-0@@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Infinite loop snippet with no exit + * + * Copyright (c) 2023 IBM Corp + * + * Authors: + * Janosch Frank <[email protected]> + */ +.section .text +retry: +j retrys390x/snippets/asm/snippet-pv-icpt-112.S new file +77
-0@@ -0,0 +1,77 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Intercept 112 PV snippet + * + * We setup and share a prefix at 0x0 and 0x8000 which the hypervisor + * test will try to export and then execute a SIE entry which + * should result in a 112 SIE intercept. + * + * Copyright (c) 2023 IBM Corp + * + * Authors: + * Janosch Frank <[email protected]> + */ +#include <asm/asm-offsets.h> + +.section .text +xgr %r0, %r0 +xgr %r1, %r1 + +# Let's tell the hypervisor we're ready to start +diag 0,0,0x44 + +/* + * Hypervisor will export the lowcore and try a SIE entry which should + * result in a 112. It will then import the lowcore again and we + * should continue with the code below. + */ + +# Share the lowcore +larl %r1, share +.insn rrf,0xB9A40000,0,1,0,0 +xgr %r1, %r1 + +# Let's tell the hypervisor we're ready to start shared testing +diag 0,0,0x44 +/* Host: icpt: PV instruction diag 0x44 */ +/* Host: icpt: 112 */ + +# Copy the invalid PGM new PSW to the new lowcore +larl %r1, prfx +l %r2, 0(%r1) +mvc GEN_LC_PGM_NEW_PSW(16, %r2), GEN_LC_PGM_NEW_PSW(%r0) + +# Change the prefix to 0x8000 and re-try +xgr %r1, %r1 +xgr %r2, %r2 +larl %r2, prfx +spx 0(%r2) +/* Host: icpt: PV instruction notification SPX*/ +/* Host: icpt: 112 */ + +# Share the new lowcore +larl %r3, share_addr +stg %r2, 0(%r3) +larl %r2, share +.insn rrf,0xB9A40000,0,2,0,0 + +# Let's tell the hypervisor we're ready to start shared testing +diag 0,0,0x44 +/* Host: icpt: PV instruction diag 0x44 */ +/* Host: icpt: 112 */ + +# Test re-entry +lghi %r1, 42 +diag 1,0,0x9c +/* Host: icpt: PV instruction diag 0x9c */ + +.align 8 +share: + .quad 0x0030100000000000 + .quad 0x0, 0x0, 0x0 +share_addr: + .quad 0x0 + .quad 0x0 +.align 4 +prfx: + .long 0x00008000s390x/snippets/asm/snippet-pv-icpt-loop.S new file +0
-0
I can imagine that this and the next snippet might be useful also without PV, maybe remove "pv" from the name
Quoted 44 lines
s390x/snippets/asm/snippet-pv-icpt-loop.S new file +15
-0@@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Infinite loop snippet which can be used to test manipulated SIE + * control block intercepts. E.g. when manipulating the PV handles. + * + * Copyright (c) 2023 IBM Corp + * + * Authors: + * Janosch Frank <[email protected]> + */ +.section .text +xgr %r0, %r0 +retry: +diag 0,0,0x44 +j retrys390x/snippets/asm/snippet-pv-icpt-vir-timing.S new file +22
-0@@ -0,0 +1,22 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Sets a cpu timer which the host can manipulate to check if it will + *receive a validity + * + * Copyright (c) 2023 IBM Corp + * + * Authors: + * Janosch Frank <[email protected]> + */ +.section .text +larl %r1, time_val +spt 0(%r1) +diag 0,0,0x44 +xgr %r1, %r1 +lghi %r1, 42 +diag 1,0,0x9c + + +.align 8 +time_val: + .quad 0x280de80000
PATCH v2 3/3 s390x: pv: Add IPL reset tests
In reply to Janosch FrankThe diag308 requires extensive cooperation between the hypervisor and the Ultravisor so the Ultravisor can make sure all necessary reset steps have been done.
Let's check if we get the correct validity errors.
--- s390x/Makefile | 2 + s390x/pv-ipl.c | 246 +++++++++++++++++++++++ s390x/snippets/asm/snippet-pv-diag-308.S | 67 ++++++ 3 files changed, 315 insertions(+) create mode 100644 s390x/pv-ipl.c create mode 100644 s390x/snippets/asm/snippet-pv-diag-308.S
s390x/Makefile +2 -0
@@ -42,6 +42,7 @@ tests += $(TEST_DIR)/exittime.elf pv-tests += $(TEST_DIR)/pv-diags.elf pv-tests += $(TEST_DIR)/pv-icptcode.elf +pv-tests += $(TEST_DIR)/pv-ipl.elf ifneq ($(HOST_KEY_DOCUMENT),) ifneq ($(GEN_SE_HEADER),)
@@ -124,6 +125,7 @@ $(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-1 $(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-loop.gbin $(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-loop.gbin $(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-vir-timing.gbin +$(TEST_DIR)/pv-ipl.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-diag-308.gbin ifneq ($(GEN_SE_HEADER),) snippets += $(pv-snippets)
s390x/pv-ipl.c new file +246 -0
246 lines not on this page · raw message
s390x/snippets/asm/snippet-pv-diag-308.S new file +67 -0
67 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [kvm-unit-tests PATCH v2 3/3] s390x: pv: Add IPL reset tests
Quoted 99 lines · On Fri, 24 Mar 2023 12:04:31 +0000 Janosch Frank <[email protected]> wrote:
The diag308 requires extensive cooperation between the hypervisor and the Ultravisor so the Ultravisor can make sure all necessary reset steps have been done.
Let's check if we get the correct validity errors.
Signed-off-by Janosch Frank <[email protected]>--- s390x/Makefile | 2 + s390x/pv-ipl.c | 246 +++++++++++++++++++++++ s390x/snippets/asm/snippet-pv-diag-308.S | 67 ++++++ 3 files changed, 315 insertions(+) create mode 100644 s390x/pv-ipl.c create mode 100644 s390x/snippets/asm/snippet-pv-diag-308.Ss390x/Makefile +2
-0@@ -42,6 +42,7 @@ tests += $(TEST_DIR)/exittime.elf pv-tests += $(TEST_DIR)/pv-diags.elf pv-tests += $(TEST_DIR)/pv-icptcode.elf +pv-tests += $(TEST_DIR)/pv-ipl.elf ifneq ($(HOST_KEY_DOCUMENT),) ifneq ($(GEN_SE_HEADER),)@@ -124,6 +125,7 @@ $(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-1 $(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-loop.gbin $(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-loop.gbin $(TEST_DIR)/pv-icptcode.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-icpt-vir-timing.gbin +$(TEST_DIR)/pv-ipl.elf: pv-snippets += $(SNIPPET_DIR)/asm/snippet-pv-diag-308.gbin ifneq ($(GEN_SE_HEADER),) snippets += $(pv-snippets)s390x/pv-ipl.c new file +58
-0@@ -0,0 +1,246 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* PV diagnose 308 (IPL) tests +* +* Copyright (c) 2023 IBM Corp +* +* Authors: +* Janosch Frank <[email protected]> +*/ +#include <libcflat.h> +#include <sie.h> +#include <sclp.h> +#include <snippet.h> +#include <asm/facility.h> +#include <asm/uv.h> + +static struct vm vm; + +static void setup_gbin (void ) +{ +extern const char SNIPPET_NAME_START (asm ,snippet_pv_diag_308 )[]; +extern const char SNIPPET_NAME_END (asm ,snippet_pv_diag_308 )[]; +extern const char SNIPPET_HDR_START (asm ,snippet_pv_diag_308 )[]; +extern const char SNIPPET_HDR_END (asm ,snippet_pv_diag_308 )[]; + int size_hdr = SNIPPET_HDR_LEN(asm, snippet_pv_diag_308); + int size_gbin = SNIPPET_LEN(asm, snippet_pv_diag_308); + + snippet_pv_init(&vm, SNIPPET_NAME_START(asm, snippet_pv_diag_308), + SNIPPET_HDR_START(asm, snippet_pv_diag_308), + size_gbin, size_hdr,SNIPPET_UNPACK_OFF ); +} + +static void test_diag_308_1 (void ) +{ +uint16_t rc, rrc; +int cc; + +report_prefix_push ("subcode 1" ); +setup_gbin (); + +sie (&vm); +report (vm.sblk->icptcode ==ICPT_PV_INSTR && vm.sblk->ipa ==0x8302 && + vm.sblk->ipb ==0x50000000 && vm.save_area.guest.grs[5 ] ==0x500 , +"intercept values diag 500" ); +/* The snippet asked us for the subcode and we answer with 1 in gr2 */ + vm.save_area.guest.grs[2 ] =1 ; + +/* Continue after diag 0x500, next icpt should be the 0x308 */ +sie (&vm); +report (vm.sblk->icptcode ==ICPT_PV_NOTIFY && vm.sblk->ipa ==0x8302 && + vm.sblk->ipb ==0x50000000 && vm.save_area.guest.grs[5 ] ==0x308 , +"intercept values diag 0x308" ); +report (vm.save_area.guest.grs[2 ] ==1 , +"subcode 1" ); + +/* +* We need to perform several UV calls to emulate the subcode +* 1. Failing to do that should result in a validity.
s390x/pv-ipl.c line 68
Quoted 10 lines
s390x/pv-ipl.c new file +10
-0+* +* - Mark all cpus as stopped +* - Unshare all +* - Prepare for reset +* - Reset the cpus, calling one gets an initial reset +* - Load the reset PSW +*/ +sie_expect_validity (&vm); +sie (&vm); +report ((sie_get_validity (&vm) &0xff00 ) ==0x2000 ,"validity no UVCs" );
didn't you introduce a new function in patch 1 to check for PV validity?
Quoted 50 lines
s390x/pv-ipl.c new file +50
-0+ +/* Mark the CPU as stopped so we can unshare and reset */ + cc =uv_set_cpu_state (vm.sblk->pv_handle_cpu,PV_CPU_STATE_STP ); +report (!cc,"Set cpu stopped" ); + +sie_expect_validity (&vm); +sie (&vm); +report ((sie_get_validity (&vm) &0xff00 ) ==0x2000 ,"validity stopped" ); + +/* Unshare all memory */ + cc =uv_cmd_nodata (vm.sblk->pv_handle_config, +UVC_CMD_SET_UNSHARED_ALL , &rc, &rrc); +report (cc ==0 && rc ==1 ,"Unshare all" ); + +sie_expect_validity (&vm); +sie (&vm); +report ((sie_get_validity (&vm) &0xff00 ) ==0x2000 , +"validity stopped, unshared" ); + +/* Prepare the CPU reset */ + cc =uv_cmd_nodata (vm.sblk->pv_handle_config, +UVC_CMD_PREPARE_RESET , &rc, &rrc); +report (cc ==0 && rc ==1 ,"Prepare reset call" ); + +sie_expect_validity (&vm); +sie (&vm); +report ((sie_get_validity (&vm) &0xff00 ) ==0x2000 , +"validity stopped, unshared, prepare" ); + +/* Do the reset */ + cc =uv_cmd_nodata (vm.sblk->pv_handle_cpu, +UVC_CMD_CPU_RESET_INITIAL , &rc, &rrc); +report (cc ==0 && rc ==1 ,"Initial reset cpu" ); + +sie_expect_validity (&vm); +sie (&vm); +report ((sie_get_validity (&vm) &0xff00 ) ==0x2000 , +"validity stopped, unshared, prepare, reset" ); + +/* Load the PSW from 0x0 */ + cc =uv_set_cpu_state (vm.sblk->pv_handle_cpu,PV_CPU_STATE_OPR_LOAD ); +report (!cc,"Set cpu load" ); + +/* +* Check if we executed the iaddr of the reset PSW, we should +* see a diagnose 0x9c PV instruction notification. +*/ +sie (&vm); + report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 && + vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x9c &&
s390x/pv-ipl.c line 128
Quoted 10 lines
s390x/pv-ipl.c new file +10
-0+ vm.save_area.guest.grs[0 ] ==42 , + "intercept values after diag 0x308"); + + +uv_destroy_guest (&vm); +report_prefix_pop (); +} + +static void test_diag_308_0 (void ) +{
this function seems a clone of the previous, with very minimal changes, can't you merge them?
Quoted 72 lines
s390x/pv-ipl.c new file +72
-0+uint16_t rc, rrc; +int cc; + +report_prefix_push ("subcode 0" ); +setup_gbin (); + +sie (&vm); +report (vm.sblk->icptcode ==ICPT_PV_INSTR && vm.sblk->ipa ==0x8302 && + vm.sblk->ipb ==0x50000000 && vm.save_area.guest.grs[5 ] ==0x500 , +"intercept values diag 500" ); +/* The snippet asked us for the subcode and we answer with 0 in gr2 */ + vm.save_area.guest.grs[2 ] =0 ; + +/* Continue after diag 0x500, next icpt should be the 0x308 */ +sie (&vm); +report (vm.sblk->icptcode ==ICPT_PV_NOTIFY && vm.sblk->ipa ==0x8302 && + vm.sblk->ipb ==0x50000000 && vm.save_area.guest.grs[5 ] ==0x308 , +"intercept values" ); +report (vm.save_area.guest.grs[2 ] ==0 , +"subcode 0" ); + +/* +* We need to perform several UV calls to emulate the subcode +* 0. Failing to do that should result in a validity. +* +* - Mark all cpus as stopped +* - Unshare all memory +* - Prepare the reset +* - Reset the cpus +* - Load the reset PSW +*/ +sie_expect_validity (&vm); +sie (&vm); +report ((sie_get_validity (&vm) &0xff00 ) ==0x2000 ,"validity, no action" ); + +/* Mark the CPU as stopped so we can unshare and reset */ + cc =uv_set_cpu_state (vm.sblk->pv_handle_cpu,PV_CPU_STATE_STP ); +report (!cc,"Set cpu stopped" ); + +sie_expect_validity (&vm); +sie (&vm); +report ((sie_get_validity (&vm) &0xff00 ) ==0x2000 ,"validity, stopped" ); + +/* Unshare all memory */ + cc =uv_cmd_nodata (vm.sblk->pv_handle_config, +UVC_CMD_SET_UNSHARED_ALL , &rc, &rrc); +report (cc ==0 && rc ==1 ,"Unshare all" ); + +sie_expect_validity (&vm); +sie (&vm); +report ((sie_get_validity (&vm) &0xff00 ) ==0x2000 ,"validity stopped, unshared" ); + +/* Prepare the CPU reset */ + cc =uv_cmd_nodata (vm.sblk->pv_handle_config, +UVC_CMD_PREPARE_RESET , &rc, &rrc); +report (cc ==0 && rc ==1 ,"Prepare reset call" ); + +sie_expect_validity (&vm); +sie (&vm); +report ((sie_get_validity (&vm) &0xff00 ) ==0x2000 ,"validity stopped, unshared, prep reset" ); + +/* Do the reset */ + cc =uv_cmd_nodata (vm.sblk->pv_handle_cpu, +UVC_CMD_CPU_RESET_CLEAR , &rc, &rrc); +report (cc ==0 && rc ==1 ,"Clear reset cpu" ); + +sie_expect_validity (&vm); +sie (&vm); +report ((sie_get_validity (&vm) &0xff00 ) ==0x2000 ,"validity stopped, unshared, prep reset, cpu reset" ); + +/* Load the PSW from 0x0 */ + cc =uv_set_cpu_state (vm.sblk->pv_handle_cpu,PV_CPU_STATE_OPR_LOAD );
s390x/pv-ipl.c line 210
Quoted 10 lines
s390x/pv-ipl.c new file +10
-0+report (!cc,"Set cpu load" ); + +/* +* Check if we executed the iaddr of the reset PSW, we should +* see a diagnose 0x9c PV instruction notification. +*/ +sie (&vm); + report(vm.sblk->icptcode == ICPT_PV_NOTIFY && vm.sblk->ipa == 0x8302 && + vm.sblk->ipb == 0x50000000 && vm.save_area.guest.grs[5] == 0x9c && + vm.save_area.guest.grs[0] == 42,
you are checking for DIAGs a lot, maybe it's worth adding a helper function like for validity
something like:
report(uv_diag_check(&vm, 0x9c) && ...
Quoted 13 lines
s390x/pv-ipl.c new file +13
-0+ "intercept values"); + +uv_destroy_guest (&vm); +report_prefix_pop (); +} + +int main (void ) +{ +report_prefix_push ("uv-sie" ); +if (!test_facility (158 )) { +report_skip ("UV Call facility unavailable" ); + goto done; + }
s390x/pv-ipl.c line 233
Quoted 10 lines
s390x/pv-ipl.c new file +10
-0+if (!sclp_facilities.has_sief2) { +report_skip ("SIEF2 facility unavailable" ); + goto done; + } +/* +* Some of the UV memory needs to be allocated with >31 bit +* addresses which means we need a lot more memory than other +* tests. +*/ +if (get_ram_size () < (SZ_1M *2200UL )) {
I think it makes sense to put this in a macro. first, so that it isn't a magic value, and second so that you have a single point where you can change it if it ever needs to be changed in the future
s390x/pv-ipl.c line 234
Quoted 1 line
s390x/pv-ipl.c new file +1
-0+report_skip ("Not enough memory. This test needs about 2200MB of memory" );
then you can do "%luMB" and use the macro here ^
Quoted 29 lines
s390x/pv-ipl.c new file +12
-0+ goto done; + } + +snippet_setup_guest (&vm, true); +test_diag_308_0 (); +test_diag_308_1 (); +sie_guest_destroy (&vm); + +done: +report_prefix_pop (); +return report_summary (); +}s390x/snippets/asm/snippet-pv-diag-308.S new file +11
-0@@ -0,0 +1,67 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Diagnose 0x308 snippet used for PV IPL and reset testing + * + * Copyright (c) 2023 IBM Corp + * + * Authors: + * Janosch Frank <[email protected]> + */ +#include <asm/asm-offsets.h> +.section .text
s390x/snippets/asm/snippet-pv-diag-308.S line 21
Quoted 10 lines
s390x/snippets/asm/snippet-pv-diag-308.S new file +10
-0+ +/* Sets a reset PSW with the given PSW address */ +.macro SET_RESET_PSW_ADDR label +lgrl %r5, reset_psw +larl %r6, \label +ogr %r5, %r6 +stg %r5, 0 +.endm + +/* Does a diagnose 308 with the given subcode */
but it seems you are not actually using this macro?
Quoted 46 lines
s390x/snippets/asm/snippet-pv-diag-308.S new file +46
-0+.macro DIAG308 subcode +xgr %r3, %r3 +lghi %r3, \subcode +diag 1, 3, 0x308 +.endm + +sam64 + +/* Execute the diag500 which will set the subcode we execute in gr2 */ +diag 0, 0, 0x500 + +/* + * A valid PGM new PSW can be a real problem since we never fall out + * of SIE and therefore effectively loop forever. 0 is a valid PSW + * therefore we re-use the reset_psw as this has the short PSW + * bit set which is invalid for a long PSW like the exception new + * PSWs. + * + * For subcode 0/1 there are no PGMs to consider. + */ +lgrl %r5, reset_psw +stg %r5, GEN_LC_PGM_NEW_PSW + +/* Clean registers that are used */ +xgr %r0, %r0 +xgr %r1, %r1 +xgr %r3, %r3 +xgr %r4, %r4 +xgr %r5, %r5 +xgr %r6, %r6 + +/* Subcode 0 - Modified Clear */ +SET_RESET_PSW_ADDR done +diag %r0, %r2, 0x308 + +/* Should never be executed because of the reset PSW */ +diag 0, 0, 0x44 + +done: +lghi %r1, 42 +diag %r1, 0, 0x9c + + + .align 8 +reset_psw: + .quad 0x0008000180000000