s390x: add migration test for CMM
cover s390x: add migration test for CMM
Upon migration, we expect the CMM page states to be preserved. Add a test which checks for that.
The new test gets a new file so the existing cmm test can still run when the prerequisites for running migration tests aren't given (netcat). Therefore, move some definitions to a common header to be able to re-use them.
Nico Boehr (2):
lib: s390x: add header for CMM related defines
s390x: add cmm migration test
lib/s390x/asm/cmm.h | 50 +++++++++++++++++++++++++++ s390x/Makefile | 1 + s390x/cmm-migration.c | 78 +++++++++++++++++++++++++++++++++++++++++++ s390x/cmm.c | 25 ++------------ s390x/unittests.cfg | 4 +++ 5 files changed, 136 insertions(+), 22 deletions(-) create mode 100644 lib/s390x/asm/cmm.h create mode 100644 s390x/cmm-migration.c
Signature · 1 line
2.31.1
PATCH 1/2 lib: s390x: add header for CMM related defines
Since we're going to need the definitions in an upcoming migration test for CMM, add a header for CMM related defines. It is based on arch/s390/include/asm/page-states.h from linux.
While at it, use the constants in existing calls to CMM related functions.
Also move essa() and test_availability() there to be able to use it outside cmm.c.
--- lib/s390x/asm/cmm.h | 50 +++++++++++++++++++++++++++++++++++++++++++++ s390x/cmm.c | 25 +++-------------------- 2 files changed, 53 insertions(+), 22 deletions(-) create mode 100644 lib/s390x/asm/cmm.h
lib/s390x/asm/cmm.h new file +50 -0
@@ -0,0 +1,50 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Copyright IBM Corp. 2017, 2022 +* Author(s): Claudio Imbrenda <[email protected]> +* Nico Boehr <[email protected]> +*/ +#include <asm/interrupt.h> + +#ifndef PAGE_STATES_H +#define PAGE_STATES_H + +#define ESSA_GET_STATE 0 +#define ESSA_SET_STABLE 1 +#define ESSA_SET_UNUSED 2 +#define ESSA_SET_VOLATILE 3 +#define ESSA_SET_POT_VOLATILE 4 +#define ESSA_SET_STABLE_RESIDENT 5 +#define ESSA_SET_STABLE_IF_RESIDENT 6 +#define ESSA_SET_STABLE_NODAT 7 + +#define ESSA_MAX ESSA_SET_STABLE_NODAT + +#define ESSA_USAGE_STABLE 0 +#define ESSA_USAGE_UNUSED 1 +#define ESSA_USAGE_POT_VOLATILE 2 +#define ESSA_USAGE_VOLATILE 3 + +static unsigned long essa (uint8_t state,unsigned long paddr) +{ +uint64_t extr_state; + + asmvolatile (".insn rrf,0xb9ab0000,%[extr_state],%[addr],%[new_state],0" \ + : [extr_state]"=d" (extr_state) \ + : [addr]"a" (paddr), [new_state]"i" (state)); + +return (unsigned long )extr_state; +} + +/* +* Unfortunately the availability is not indicated by stfl bits, but +* we have to try to execute it and test for an operation exception. +*/ +static inline bool check_essa_available (void ) +{ +expect_pgm_int (); +essa (ESSA_GET_STATE ,0 ); +return clear_pgm_int () ==0 ; +} + +#endif
s390x/cmm.c +3 -22
@@ -12,19 +12,10 @@#include <asm/asm-offsets.h> #include <asm/interrupt.h> #include <asm/page.h> +#include <asm/cmm.h> static uint8_t pagebuf[PAGE_SIZE] __attribute__((aligned(PAGE_SIZE))); -static unsigned long essa (uint8_t state,unsigned long paddr) -{ -uint64_t extr_state; - - asmvolatile (".insn rrf,0xb9ab0000,%[extr_state],%[addr],%[new_state],0" - : [extr_state]"=d" (extr_state) - : [addr]"a" (paddr), [new_state]"i" (state)); -return (unsigned long )extr_state; -} -static void test_params (void ) {report_prefix_push ("invalid ORC 8" );
@@ -39,24 +30,14 @@ static void test_priv(void)report_prefix_push ("privileged" );expect_pgm_int ();enter_pstate (); -essa (0 , (unsigned long )pagebuf); +essa (ESSA_GET_STATE , (unsigned long )pagebuf);check_pgm_int_code (PGM_INT_CODE_PRIVILEGED_OPERATION );report_prefix_pop (); } -/* Unfortunately the availability is not indicated by stfl bits, but -* we have to try to execute it and test for an operation exception. -*/ -static bool test_availability (void ) -{ -expect_pgm_int (); -essa (0 , (unsigned long )pagebuf); -return clear_pgm_int () ==0 ; -} -int main (void ) { -bool has_essa =test_availability (); +bool has_essa =check_essa_available ();report_prefix_push ("cmm" );if (!has_essa) {
Signature · 1 line
2.31.1
PATCH 2/2 s390x: add cmm migration test
In reply to Nico BoehrWhen a VM is migrated, we expect the page states to be preserved. Add a test which checks for that.
--- s390x/Makefile | 1 + s390x/cmm-migration.c | 78 +++++++++++++++++++++++++++++++++++++++++++ s390x/unittests.cfg | 4 +++ 3 files changed, 83 insertions(+) create mode 100644 s390x/cmm-migration.c
s390x/Makefile +1 -0
@@ -32,6 +32,7 @@ tests += $(TEST_DIR)/epsw.elf tests += $(TEST_DIR)/adtl-status.elf tests += $(TEST_DIR)/migration.elf tests += $(TEST_DIR)/pv-attest.elf +tests += $(TEST_DIR)/cmm-migration.elf pv-tests += $(TEST_DIR)/pv-diags.elf
s390x/cmm-migration.c new file +78 -0
78 lines not on this page · raw message
s390x/unittests.cfg +4 -0
@@ -176,3 +176,7 @@ extra_params = -cpu qemu,gs=off,vx=off file = migration.elf groups = migration smp = 2 + +[cmm-migration] +file = cmm-migration.elf +groups = migration
Signature · 1 line
2.31.1
Re: [kvm-unit-tests PATCH v1 2/2] s390x: add cmm migration test
Quoted 67 lines · On Mon, 9 May 2022 14:08:05 +0200 Nico Boehr <[email protected]> wrote:
When a VM is migrated, we expect the page states to be preserved. Add a test which checks for that.
Signed-off-by Nico Boehr <[email protected]>--- s390x/Makefile | 1 + s390x/cmm-migration.c | 78 +++++++++++++++++++++++++++++++++++++++++++ s390x/unittests.cfg | 4 +++ 3 files changed, 83 insertions(+) create mode 100644 s390x/cmm-migration.cs390x/Makefile +1
-0@@ -32,6 +32,7 @@ tests += $(TEST_DIR)/epsw.elf tests += $(TEST_DIR)/adtl-status.elf tests += $(TEST_DIR)/migration.elf tests += $(TEST_DIR)/pv-attest.elf +tests += $(TEST_DIR)/cmm-migration.elf pv-tests += $(TEST_DIR)/pv-diags.elfs390x/cmm-migration.c new file +38
-0@@ -0,0 +1,78 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* +* CMM migration tests (ESSA) +* +* Copyright IBM Corp. 2022 +* +* Authors: +* Nico Boehr <[email protected]> +*/ + +#include <libcflat.h> +#include <asm/asm-offsets.h> +#include <asm/interrupt.h> +#include <asm/page.h> +#include <asm/cmm.h> +#include <bitops.h> + +#define NUM_PAGES 128 +static uint8_t pagebuf[NUM_PAGES][PAGE_SIZE] __attribute__((aligned(PAGE_SIZE))); + +static void test_migration (void ) +{ +int i, state_mask, actual_state; +/* +* Maps ESSA actions to states the page is allowed to be in after the +* respective action was executed. +*/ +int allowed_essa_state_masks[4 ] = { +BIT (ESSA_USAGE_STABLE ),/* ESSA_SET_STABLE */ +BIT (ESSA_USAGE_UNUSED ),/* ESSA_SET_UNUSED */ +BIT (ESSA_USAGE_VOLATILE ),/* ESSA_SET_VOLATILE */ +BIT (ESSA_USAGE_VOLATILE ) |BIT (ESSA_USAGE_POT_VOLATILE )/* ESSA_SET_POT_VOLATILE */ + }; + +for (i =0 ; i <NUM_PAGES ; i++) { +switch (i %4 ) { +case 0 : +essa (ESSA_SET_STABLE , (unsigned long )pagebuf[i]);
s390x/cmm-migration.c line 48
Quoted 10 lines
s390x/cmm-migration.c new file +10
-0+break ; +case 1 : +essa (ESSA_SET_UNUSED , (unsigned long )pagebuf[i]); +break ; +case 2 : +essa (ESSA_SET_VOLATILE , (unsigned long )pagebuf[i]); +break ; +case 3 : +essa (ESSA_SET_POT_VOLATILE , (unsigned long )pagebuf[i]); +break ;
const int essa_commands[4] = {ESSA_SET_STABLE, ESSA_SET_UNUSED, ...
for (i = 0; i < NUM_PAGES; i++)
essa(essa_commands[i % 4], ...
I think it would look more compact and more readable
Quoted 42 lines
s390x/cmm-migration.c new file +30
-0+ } + } + +puts ("Please migrate me, then press return\n" ); + (void )getchar (); + +for (i =0 ; i <NUM_PAGES ; i++) { + actual_state =essa (ESSA_GET_STATE , (unsigned long )pagebuf[i]); +/* extract the usage state in bits 60 and 61 */ + actual_state = (actual_state >>2 ) &0x3 ; + state_mask = allowed_essa_state_masks[i %ARRAY_SIZE (allowed_essa_state_masks)]; +report (BIT (actual_state) & state_mask,"page %d state: expected_mask=0x%x actual_mask=0x%lx" , i, state_mask,BIT (actual_state)); + } +} + +int main (void ) +{ +bool has_essa =check_essa_available (); + +report_prefix_push ("cmm-migration" ); +if (!has_essa) { +report_skip ("ESSA is not available" ); + goto done; + } + +test_migration (); +done: +report_prefix_pop (); +return report_summary (); +}s390x/unittests.cfg +4
-0@@ -176,3 +176,7 @@ extra_params = -cpu qemu,gs=off,vx=off file = migration.elf groups = migration smp = 2 + +[cmm-migration] +file = cmm-migration.elf +groups = migration
Re: [kvm-unit-tests PATCH v1 2/2] s390x: add cmm migration test
Quoted 19 lines · Claudio Imbrenda
Quoted 10 lines
+ for (i = 0; i <
NUM_PAGES; i++) {
+ switch(i % 4) {
+ case 0:
+ essa(ESSA_SET_STABLE, (unsigned long)pagebuf[i]);
+ break;
+ case 1:
+ essa(ESSA_SET_UNUSED, (unsigned long)pagebuf[i]);
+ break;Quoted 8 lines
+ case 2:
+ essa(ESSA_SET_VOLATILE, (unsigned long)pagebuf[i]);
+ break;
+ case 3:
+ essa(ESSA_SET_POT_VOLATILE, (unsigned long)pagebuf[i]);
+ break;
Quoted 6 lines
const int essa_commands[4] = {
ESSA_SET_STABLE,ESSA_SET_UNUSED, ...for (i = 0; i <
NUM_PAGES; i++)
essa(essa_commands[i % 4], ...I think it would look more compact and more readable
I like your idea a lot, but the compiler doesn't :-(:
/home/nrb/kvm-unit-tests/lib/asm/cmm.h: In function ‘main’:/home/nrb/kvm-unit-tests/lib/asm/cmm.h:32:9: error: ‘asm’ operand 2 probably does not match constraints [-Werror]
32 | asm volatile(".insn
rrf,0xb9ab0000,%[extr_state],%[addr],%[new_state],0" \
| ^~~/home/nrb/kvm-unit-tests/lib/asm/cmm.h:32:9: error: impossible constraint in ‘asm’
To satify the "i" constraint, new_state needs to be a compile time constant, which it won't be any more with your suggestion unfortunately.
We can do crazy stuff like this in cmm.h:
#define __essa(state) \
asm volatile(".insn
rrf,0xb9ab0000,%[extr_state],%[addr],%[new_state],0" \
: [extr_state] "=d" (extr_state) \
: [addr] "a" (paddr), [new_state] "i"
(state));
static unsigned long essa(uint8_t state, unsigned long paddr)
{
uint64_t extr_state = 0;
switch(state) {
case ESSA_SET_STABLE:
__essa(ESSA_SET_STABLE);
break;
case ESSA_SET_UNUSED:
__essa(ESSA_SET_UNUSED);
break;
case ESSA_SET_VOLATILE:
__essa(ESSA_SET_VOLATILE);
break;
case ESSA_SET_POT_VOLATILE:
__essa(ESSA_SET_POT_VOLATILE);
break;
}
return (unsigned long)extr_state;
}But that essentially just shifts the readability problem to a different file. What do you think?
Or we make essa a marco, which doesn't sound like a particularily attractive alternative either.
Re: [kvm-unit-tests PATCH v1 2/2] s390x: add cmm migration test
Quoted 20 lines · On Tue, 10 May 2022 15:25:09 +0200 Nico Boehr <[email protected]> wrote:
Quoted 19 lines · Claudio Imbrenda
Quoted 18 lines
+ for (i = 0; i <
NUM_PAGES; i++) {
+ switch(i % 4) {
+ case 0:
+ essa(ESSA_SET_STABLE, (unsigned long)pagebuf[i]);
+ break;
+ case 1:
+ essa(ESSA_SET_UNUSED, (unsigned long)pagebuf[i]);
+ break;
+ case 2:
+ essa(ESSA_SET_VOLATILE, (unsigned long)pagebuf[i]);
+ break;
+ case 3:
+ essa(ESSA_SET_POT_VOLATILE, (unsigned long)pagebuf[i]);
+ break;
Quoted 8 lines
Quoted 6 lines
const int essa_commands[4] = {
ESSA_SET_STABLE,ESSA_SET_UNUSED, ...for (i = 0; i <
NUM_PAGES; i++)
essa(essa_commands[i % 4], ...I think it would look more compact and more readable
I like your idea a lot, but the compiler doesn't :-(:
ouch, I had completely forgotten about that.
Quoted 44 lines
/home/nrb/kvm-unit-tests/lib/asm/cmm.h: In function ‘main’:/home/nrb/kvm-unit-tests/lib/asm/cmm.h:32:9: error: ‘asm’ operand 2 probably does not match constraints [-Werror]
32 | asm volatile(".insnrrf,0xb9ab0000,%[extr_state],%[addr],%[new_state],0" \ | ^~~
/home/nrb/kvm-unit-tests/lib/asm/cmm.h:32:9: error: impossible constraint in ‘asm’To satify the "i" constraint,
new_stateneeds to be a compile time constant, which it won't be any more with your suggestion unfortunately.We can do crazy stuff like this in
cmm.h:#define
__essa(state)\
asm volatile(".insnrrf,0xb9ab0000,%[extr_state],%[addr],%[new_state],0" \ : [extr_state] "=d" (extr_state) \ : [addr] "a" (paddr), [new_state] "i" (state)); static unsigned long essa(uint8_t state, unsigned long paddr) { uint64_t extr_state = 0; switch(state) { case ESSA_SET_STABLE: __essa(ESSA_SET_STABLE); break; case ESSA_SET_UNUSED: __essa(ESSA_SET_UNUSED); break; case ESSA_SET_VOLATILE: __essa(ESSA_SET_VOLATILE); break; case ESSA_SET_POT_VOLATILE: __essa(ESSA_SET_POT_VOLATILE); break; } return (unsigned long)extr_state; }
Quoted 5 lines
But that essentially just shifts the readability problem to a different file. What do you think?
Or we make essa a marco, which doesn't sound like a particularily attractive alternative either.
ok, next less ugly thing: unroll the loop
for (i = 0; i < NUM_PAGES; i += 4) {
essa(ESSA_SET_STABLE, (unsigned long)pagebuf[i]);
essa(ESSA_SET_UNUSED, (unsigned long)pagebuf[i + 1]);
... etc
}maybe assert(NUM_PAGES % 4 == 0) before that, just for good measure
Re: [kvm-unit-tests PATCH v1 2/2] s390x: add cmm migration test
Quoted 9 lines · Claudio Imbrenda
ok, next less ugly thing: unroll the loop
for (i = 0; i <
NUM_PAGES; i += 4) {essa(ESSA_SET_STABLE, (unsigned long)pagebuf[i]);essa(ESSA_SET_UNUSED, (unsigned long)pagebuf[i + 1]); ... etc
}maybe
assert(NUM_PAGES % 4 == 0)before that, just for good measure
That's nicer, thanks, fixed.
Quoted 5 lines · On Mon, 9 May 2022 14:08:03 +0200 Nico Boehr <[email protected]> wrote:
Upon migration, we expect the CMM page states to be preserved. Add a test which checks for that.
The new test gets a new file so the existing cmm test can still run when the prerequisites for running migration tests aren't given (netcat). Therefore, move
I wonder if we are going to have more of these "split" tests.
is there a way to make sure migration prerequisites are always present? or rewrite things so that we don't need them?
Quoted 15 lines
some definitions to a common header to be able to re-use them.
Nico Boehr (2):
lib: s390x: add header for CMM related defines
s390x: add cmm migration testlib/s390x/asm/cmm.h | 50 +++++++++++++++++++++++++++ s390x/Makefile | 1 + s390x/cmm-migration.c | 78 +++++++++++++++++++++++++++++++++++++++++++ s390x/cmm.c | 25 ++------------ s390x/unittests.cfg | 4 +++ 5 files changed, 136 insertions(+), 22 deletions(-) create mode 100644 lib/s390x/asm/cmm.h create mode 100644 s390x/cmm-migration.c
Quoted 4 lines · Claudio Imbrenda
I wonder if we are going to have more of these "split" tests.
is there a way to make sure migration prerequisites are always present?
We could not run _any_ tests if netcat is not installed, which seems like a bad idea.
Quoted 1 line
or rewrite things so that we don't need them?
We need ncat to communicate with the QEMU QMP over unix socket. I am not aware of a way to use unix sockets in Bash, but no expert either.
We could ship our own version of netcat and build it for the host, which adds additional complexity and maintenance burden.
I honestly can't think of a good way.
Or we just put all cmm tests in a single file and accept the fact that if you don't have all the migration related requirements installed, you don't get all the tests - even some which are not at all related to migration. I did not like that so I went with the extra file.
Quoted 17 lines · Nico Boehr
Quoted 4 lines · Claudio Imbrenda
I wonder if we are going to have more of these "split" tests.
is there a way to make sure migration prerequisites are always present?
We could not run _any_ tests if netcat is not installed, which seems like a bad idea.
Quoted 1 line
or rewrite things so that we don't need them?
We need ncat to communicate with the QEMU QMP over unix socket. I am not aware of a way to use unix sockets in Bash, but no expert either.
We could ship our own version of netcat and build it for the host, which adds additional complexity and maintenance burden.
Quoted 6 lines
I honestly can't think of a good way.
Or we just put all cmm tests in a single file and accept the fact that if you don't have all the migration related requirements installed, you don't get all the tests - even some which are not at all related to migration. I did not like that so I went with the extra file.
Having them separate is fine. I'd change the file name to migration-cmm.elf though. We might also want to change the name of the first migration test in the future to make the name more specific.
Quoted 4 lines · Janosch Frank
Having them separate is fine. I'd change the file name to migration-cmm.elf though. We might also want to change the name of the
first migration test in the future to make the name more specific.
Yes, good thought. I have it on my TODO.