s390x: Add Protected VM support
v2 cover s390x: Add Protected VM support
Add support for Protected Virtual Machine (PVM) tests. For starting a PVM guest we must be able to generate a PVM image by using the genprotimg tool from the s390-tools collection. This requires the ability to pass a machine-specific host-key document, so the option --host-key-document is added to the configure script.
For everybody's convenience there is a branch:
https://gitlab.com/mhartmay/kvm-unit-tests/-/tree/pv_v2
Changelog:
v1 -> v2:
+ rebased
+ patches 1-3:
- add r-b from Conny, Andrew, and David
+ patch 4:
- add r-b from Janosch
- renamed ${testname} to $testname (David)
- fix print_result function calls and the arguments used
RFC v2 -> v1:
+ Rebased
+ patch 1:
- add r-b from Andrew
+ patch 2:
- add explicit dependency on config.mak (Andrew)
- add comment about the order of sourcing (Andrew)
+ patch 3:
- drop dummy function (Andrew)
- add arch_cmd hook function (Andrew)
+ patch 4:
- rephrased the documentation of the configure option (Conny)
- Skip test case if a PVM image wasn't built or the host-key document wasn't set (Conny)
- Run PV tests by default
RFC v1 -> RFC v2:
+ Remove pv_support option (Janosch, David)
+ Add some preliminary patches:
- move "testname guard"
- add support for architecture dependent functions
+ Add support for specifying a parmline file for the PV image
generation. This is necessary for the selftest because the
kernel cmdline set on the QEMU command line is ignored for PV
guests
Marc Hartmayer (4):
common.bash: run cmd only if a test case was found
scripts: add support for architecture dependent functions run_tests/mkstandalone: add arch_cmd hook
s390x: add Protected VM support
README.md | 3 ++- configure | 9 +++++++++ run_tests.sh | 3 --- s390x/Makefile | 17 +++++++++++++++-- s390x/selftest.parmfile | 1 + s390x/unittests.cfg | 1 + scripts/common.bash | 21 +++++++++++++++++++-- scripts/mkstandalone.sh | 4 ---- scripts/s390x/func.bash | 36 ++++++++++++++++++++++++++++++++++++ 9 files changed, 83 insertions(+), 12 deletions(-) create mode 100644 s390x/selftest.parmfile create mode 100644 scripts/s390x/func.bash
Signature · 1 line
2.25.4
PATCH v2 1/4 common.bash: run `cmd` only if a test case was found
It's only useful to run cmd in for_each_unittest if a test case was found. This change allows us to remove the guards from the functions run_task and mkstandalone.
--- run_tests.sh | 3 --- scripts/common.bash | 8 ++++++-- scripts/mkstandalone.sh | 4 ---- 3 files changed, 6 insertions(+), 9 deletions(-)
run_tests.sh +0 -3
@@ -133,9 +133,6 @@ RUNTIME_log_stdout () { function run_task() { local testname="$1" - if [ -z "$testname" ]; then - return - fi while (( $(jobs | wc -l) == $unittest_run_queues )); do # wait for any background test to finish
scripts/common.bash +6 -2
@@ -17,7 +17,9 @@ function for_each_unittest() while read -r -u $fd line; do if [[ "$line" =~ ^\[(.*)\]$ ]]; then - "$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel" "$timeout" + if [ -n "${testname}" ]; then + "$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel" "$timeout" + fi testname=${BASH_REMATCH[1]} smp=1 kernel=""
@@ -45,6 +47,8 @@ function for_each_unittest() timeout=${BASH_REMATCH[1]} fi done - "$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel" "$timeout" + if [ -n "${testname}" ]; then + "$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel" "$timeout" + fi exec {fd}<&- }
scripts/mkstandalone.sh +0 -4
@@ -83,10 +83,6 @@ function mkstandalone() { local testname="$1" - if [ -z "$testname" ]; then - return - fi - if [ -n "$one_testname" ] && [ "$testname" != "$one_testname" ]; then return fi
Signature · 1 line
2.25.4
PATCH v2 2/4 scripts: add support for architecture dependent functions
In reply to Marc HartmayerThis is necessary to keep architecture dependent code separate from common code.
--- README.md | 3 ++- scripts/common.bash | 8 ++++++++ 2 files changed, 10 insertions(+), 1 deletion(-)
README.md +2 -1
@@ -134,7 +134,8 @@ all unit tests. ## Directory structure .: configure script, top-level Makefile, and run_tests.sh - ./scripts: helper scripts for building and running tests + ./scripts: general architecture neutral helper scripts for building and running tests + ./scripts/<ARCH>: architecture dependent helper scripts for building and running tests ./lib: general architecture neutral services for the tests ./lib/<ARCH>: architecture dependent services for the tests ./<ARCH>: the sources of the tests and the created objects/images
scripts/common.bash +8 -0
@@ -1,3 +1,4 @@ +source config.mak function for_each_unittest() {
@@ -52,3 +53,10 @@ function for_each_unittest() fi exec {fd}<&- } + +# The current file has to be the only file sourcing the arch helper +# file +ARCH_FUNC=scripts/${ARCH}/func.bash +if [ -f "${ARCH_FUNC}" ]; then + source "${ARCH_FUNC}" +fi
Signature · 1 line
2.25.4
PATCH v2 3/4 run_tests/mkstandalone: add arch_cmd hook
In reply to Marc HartmayerThis allows us, for example, to auto generate a new test case based on an existing test case.
--- scripts/common.bash | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
scripts/common.bash +7 -2
@@ -19,7 +19,7 @@ function for_each_unittest() while read -r -u $fd line; do if [[ "$line" =~ ^\[(.*)\]$ ]]; then if [ -n "${testname}" ]; then - "$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel" "$timeout" + $(arch_cmd) "$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel" "$timeout" fi testname=${BASH_REMATCH[1]} smp=1
@@ -49,11 +49,16 @@ function for_each_unittest() fi done if [ -n "${testname}" ]; then - "$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel" "$timeout" + $(arch_cmd) "$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel" "$timeout" fi exec {fd}<&- } +function arch_cmd() +{ + [ "${ARCH_CMD}" ] && echo "${ARCH_CMD}" +} + # The current file has to be the only file sourcing the arch helper # file ARCH_FUNC=scripts/${ARCH}/func.bash
Signature · 1 line
2.25.4
PATCH v2 4/4 s390x: add Protected VM support
In reply to Marc HartmayerAdd support for Protected Virtual Machine (PVM) tests. For starting a PVM guest we must be able to generate a PVM image by using the genprotimg tool from the s390-tools collection. This requires the ability to pass a machine-specific host-key document, so the option --host-key-document is added to the configure script.
--- configure | 9 +++++++++ s390x/Makefile | 17 +++++++++++++++-- s390x/selftest.parmfile | 1 + s390x/unittests.cfg | 1 + scripts/s390x/func.bash | 36 ++++++++++++++++++++++++++++++++++++ 5 files changed, 62 insertions(+), 2 deletions(-) create mode 100644 s390x/selftest.parmfile create mode 100644 scripts/s390x/func.bash
configure +9 -0
@@ -19,6 +19,7 @@ wa_divide= vmm="qemu" errata_force=0 erratatxt="$srcdir/errata.txt" +host_key_document= usage() { cat <<-EOF
@@ -41,6 +42,9 @@ usage() { no environ is provided by the user (enabled by default) --erratatxt=FILE specify a file to use instead of errata.txt. Use '--erratatxt=' to ensure no file is used. + --host-key-document=HOST_KEY_DOCUMENT + Specify the machine-specific host-key document for creating + a PVM image with 'genprotimg' (s390x only) EOF exit 1 }
@@ -93,6 +97,9 @@ while [[ "$1" = -* ]]; do erratatxt= [ "$arg" ] && erratatxt=$(eval realpath "$arg") ;; + --host-key-document) + host_key_document="$arg" + ;; --help) usage ;;
@@ -224,6 +231,8 @@ ENVIRON_DEFAULT=$environ_default ERRATATXT=$erratatxt U32_LONG_FMT=$u32_long WA_DIVIDE=$wa_divide +GENPROTIMG=${GENPROTIMG-genprotimg} +HOST_KEY_DOCUMENT=$host_key_document EOF cat <<EOF > lib/config.h
s390x/Makefile +15 -2
@@ -19,12 +19,19 @@ tests += $(TEST_DIR)/smp.elf tests += $(TEST_DIR)/sclp.elf tests += $(TEST_DIR)/css.elf tests += $(TEST_DIR)/uv-guest.elf -tests_binary = $(patsubst %.elf,%.bin,$(tests)) -all: directories test_cases test_cases_binary +tests_binary = $(patsubst %.elf,%.bin,$(tests)) +ifneq ($(HOST_KEY_DOCUMENT),) +tests_pv_binary = $(patsubst %.bin,%.pv.bin,$(tests_binary)) +else +tests_pv_binary = +endif + +all: directories test_cases test_cases_binary test_cases_pv test_cases: $(tests) test_cases_binary: $(tests_binary) +test_cases_pv: $(tests_pv_binary) CFLAGS += -std=gnu99 CFLAGS += -ffreestanding
@@ -73,6 +80,12 @@ FLATLIBS = $(libcflat) %.bin: %.elf $(OBJCOPY) -O binary $< $@ +%selftest.pv.bin: %selftest.bin $(HOST_KEY_DOCUMENT) $(patsubst %.pv.bin,%.parmfile,$@) + $(GENPROTIMG) --host-key-document $(HOST_KEY_DOCUMENT) --parmfile $(patsubst %.pv.bin,%.parmfile,$@) --no-verify --image $< -o $@ + +%.pv.bin: %.bin $(HOST_KEY_DOCUMENT) + $(GENPROTIMG) --host-key-document $(HOST_KEY_DOCUMENT) --no-verify --image $< -o $@ + arch_clean: asm_offsets_clean $(RM) $(TEST_DIR)/*.{o,elf,bin} $(TEST_DIR)/.*.d lib/s390x/.*.d
s390x/selftest.parmfile new file +1 -0
@@ -0,0 +1 @@ +test 123
s390x/unittests.cfg +1 -0
@@ -21,6 +21,7 @@ [selftest-setup] file = selftest.elf groups = selftest +# please keep the kernel cmdline in sync with $(TEST_DIR)/selftest.parmfile extra_params = -append 'test 123' [intercept]
scripts/s390x/func.bash new file +36 -0
@@ -0,0 +1,36 @@ +# The file scripts/common.bash has to be the only file sourcing this +# arch helper file +source config.mak + +ARCH_CMD=arch_cmd_s390x + +function arch_cmd_s390x() +{ + local cmd=$1 + local testname=$2 + local groups=$3 + local smp=$4 + local kernel=$5 + local opts=$6 + local arch=$7 + local check=$8 + local accel=$9 + local timeout=${10} + + # run the normal test case + "$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel" "$timeout" + + # run PV test case + kernel=${kernel%.elf}.pv.bin + testname=${testname}_PV + if [ ! -f "${kernel}" ]; then + if [ -z "${HOST_KEY_DOCUMENT}" ]; then + print_result 'SKIP' $testname '' 'no host-key document specified' + return 2 + fi + + print_result 'SKIP' $testname '' 'PVM image was not created' + return 2 + fi + "$cmd" "$testname" "$groups pv" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel" "$timeout" +}
Signature · 1 line
2.25.4
Re: [PATCH kvm-unit-tests v2 4/4] s390x: add Protected VM support
Quoted 8 lines · On Wed, 23 Sep 2020 15:47:58 +0200 Marc Hartmayer <[email protected]>…
Add support for Protected Virtual Machine (PVM) tests. For starting a PVM guest we must be able to generate a PVM image by using the
genprotimgtool from the s390-tools collection. This requires the ability to pass a machine-specific host-key document, so the option--host-key-documentis added to the configure script.Reviewed-by Janosch Frank <[email protected]>Signed-off-by Marc Hartmayer <[email protected]>
Quoted 9 lines
--- configure | 9 +++++++++ s390x/Makefile | 17 +++++++++++++++-- s390x/selftest.parmfile | 1 + s390x/unittests.cfg | 1 + scripts/s390x/func.bash | 36 ++++++++++++++++++++++++++++++++++++ 5 files changed, 62 insertions(+), 2 deletions(-) create mode 100644 s390x/selftest.parmfile create mode 100644 scripts/s390x/func.bash
Re: [PATCH kvm-unit-tests v2 4/4] s390x: add Protected VM support
In reply to Marc HartmayerQuoted 142 lines · Marc Hartmayer
Add support for Protected Virtual Machine (PVM) tests. For starting a PVM guest we must be able to generate a PVM image by using the
genprotimgtool from the s390-tools collection. This requires the ability to pass a machine-specific host-key document, so the option--host-key-documentis added to the configure script.Reviewed-by Janosch Frank <[email protected]>Signed-off-by Marc Hartmayer <[email protected]>--- configure | 9 +++++++++ s390x/Makefile | 17 +++++++++++++++-- s390x/selftest.parmfile | 1 + s390x/unittests.cfg | 1 + scripts/s390x/func.bash | 36 ++++++++++++++++++++++++++++++++++++ 5 files changed, 62 insertions(+), 2 deletions(-) create mode 100644 s390x/selftest.parmfile create mode 100644 scripts/s390x/func.bashconfigure +9
-0@@ -19,6 +19,7 @@ wa_divide= vmm="qemu" errata_force=0 erratatxt="$srcdir/errata.txt" +host_key_document= usage() { cat <<-EOF@@ -41,6 +42,9 @@ usage() { no environ is provided by the user (enabled by default) --erratatxt=FILE specify a file to use instead of errata.txt. Use '--erratatxt=' to ensure no file is used. + --host-key-document=HOST_KEY_DOCUMENT + Specify the machine-specific host-key document for creating + a PVM image with 'genprotimg' (s390x only) EOF exit 1 }@@ -93,6 +97,9 @@ while [[ "$1" = -* ]]; do erratatxt= [ "$arg" ] && erratatxt=$(eval realpath "$arg") ;; + --host-key-document) + host_key_document="$arg" + ;; --help) usage ;;@@ -224,6 +231,8 @@ ENVIRON_DEFAULT=$environ_default ERRATATXT=$erratatxt U32_LONG_FMT=$u32_long WA_DIVIDE=$wa_divide +GENPROTIMG=${GENPROTIMG-genprotimg} +HOST_KEY_DOCUMENT=$host_key_document EOF cat <<EOF > lib/config.hs390x/Makefile +15
-2@@ -19,12 +19,19 @@ tests += $(TEST_DIR)/smp.elf tests += $(TEST_DIR)/sclp.elf tests += $(TEST_DIR)/css.elf tests += $(TEST_DIR)/uv-guest.elf -tests_binary = $(patsubst %.elf,%.bin,$(tests)) -all: directories test_cases test_cases_binary +tests_binary = $(patsubst %.elf,%.bin,$(tests)) +ifneq ($(HOST_KEY_DOCUMENT),) +tests_pv_binary = $(patsubst %.bin,%.pv.bin,$(tests_binary)) +else +tests_pv_binary = +endif + +all: directories test_cases test_cases_binary test_cases_pv test_cases: $(tests) test_cases_binary: $(tests_binary) +test_cases_pv: $(tests_pv_binary) CFLAGS += -std=gnu99 CFLAGS += -ffreestanding@@ -73,6 +80,12 @@ FLATLIBS = $(libcflat) %.bin: %.elf $(OBJCOPY) -O binary $< $@ +%selftest.pv.bin: %selftest.bin $(HOST_KEY_DOCUMENT) $(patsubst %.pv.bin,%.parmfile,$@) + $(GENPROTIMG) --host-key-document $(HOST_KEY_DOCUMENT) --parmfile $(patsubst %.pv.bin,%.parmfile,$@) --no-verify --image $< -o $@ + +%.pv.bin: %.bin $(HOST_KEY_DOCUMENT) + $(GENPROTIMG) --host-key-document $(HOST_KEY_DOCUMENT) --no-verify --image $< -o $@ + arch_clean: asm_offsets_clean $(RM) $(TEST_DIR)/*.{o,elf,bin} $(TEST_DIR)/.*.d lib/s390x/.*.ds390x/selftest.parmfile new file +1
-0@@ -0,0 +1 @@ +test 123s390x/unittests.cfg +1
-0@@ -21,6 +21,7 @@ [selftest-setup] file = selftest.elf groups = selftest +# please keep the kernel cmdline in sync with $(TEST_DIR)/selftest.parmfile extra_params = -append 'test 123' [intercept]scripts/s390x/func.bash new file +18
-0@@ -0,0 +1,36 @@ +# The file scripts/common.bash has to be the only file sourcing this +# arch helper file +source config.mak + +ARCH_CMD=arch_cmd_s390x + +function arch_cmd_s390x() +{ + local cmd=$1 + local testname=$2 + local groups=$3 + local smp=$4 + local kernel=$5 + local opts=$6 + local arch=$7 + local check=$8 + local accel=$9 + local timeout=${10}
scripts/s390x/func.bash line 28
Quoted 10 lines
scripts/s390x/func.bash new file +10
-0+ + # run the normal test case + "$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel" "$timeout" + + # run PV test case + kernel=${kernel%.elf}.pv.bin + testname=${testname}_PV + if [ ! -f "${kernel}" ]; then + if [ -z "${HOST_KEY_DOCUMENT}" ]; then + print_result 'SKIP' $testname '' 'no host-key document specified'
I wonder whether we should not simply stay silent here ... ? Currently the output gets quite spoiled with a lot of these "no host-key document specified" messages when PV is not in use.
If you agree, I could drop this line when picking up the patch (no need to respin just because of this).
Thomas
Re: [PATCH kvm-unit-tests v2 4/4] s390x: add Protected VM support
Quoted 151 lines · Thomas Huth
Quoted 150 lines · Marc Hartmayer
Add support for Protected Virtual Machine (PVM) tests. For starting a PVM guest we must be able to generate a PVM image by using the
genprotimgtool from the s390-tools collection. This requires the ability to pass a machine-specific host-key document, so the option--host-key-documentis added to the configure script.Reviewed-by Janosch Frank <[email protected]>Signed-off-by Marc Hartmayer <[email protected]>--- configure | 9 +++++++++ s390x/Makefile | 17 +++++++++++++++-- s390x/selftest.parmfile | 1 + s390x/unittests.cfg | 1 + scripts/s390x/func.bash | 36 ++++++++++++++++++++++++++++++++++++ 5 files changed, 62 insertions(+), 2 deletions(-) create mode 100644 s390x/selftest.parmfile create mode 100644 scripts/s390x/func.bashconfigure +9
-0@@ -19,6 +19,7 @@ wa_divide= vmm="qemu" errata_force=0 erratatxt="$srcdir/errata.txt" +host_key_document= usage() { cat <<-EOF@@ -41,6 +42,9 @@ usage() { no environ is provided by the user (enabled by default) --erratatxt=FILE specify a file to use instead of errata.txt. Use '--erratatxt=' to ensure no file is used. + --host-key-document=HOST_KEY_DOCUMENT + Specify the machine-specific host-key document for creating + a PVM image with 'genprotimg' (s390x only) EOF exit 1 }@@ -93,6 +97,9 @@ while [[ "$1" = -* ]]; do erratatxt= [ "$arg" ] && erratatxt=$(eval realpath "$arg") ;; + --host-key-document) + host_key_document="$arg" + ;; --help) usage ;;@@ -224,6 +231,8 @@ ENVIRON_DEFAULT=$environ_default ERRATATXT=$erratatxt U32_LONG_FMT=$u32_long WA_DIVIDE=$wa_divide +GENPROTIMG=${GENPROTIMG-genprotimg} +HOST_KEY_DOCUMENT=$host_key_document EOF cat <<EOF > lib/config.hs390x/Makefile +15
-2@@ -19,12 +19,19 @@ tests += $(TEST_DIR)/smp.elf tests += $(TEST_DIR)/sclp.elf tests += $(TEST_DIR)/css.elf tests += $(TEST_DIR)/uv-guest.elf -tests_binary = $(patsubst %.elf,%.bin,$(tests)) -all: directories test_cases test_cases_binary +tests_binary = $(patsubst %.elf,%.bin,$(tests)) +ifneq ($(HOST_KEY_DOCUMENT),) +tests_pv_binary = $(patsubst %.bin,%.pv.bin,$(tests_binary)) +else +tests_pv_binary = +endif + +all: directories test_cases test_cases_binary test_cases_pv test_cases: $(tests) test_cases_binary: $(tests_binary) +test_cases_pv: $(tests_pv_binary) CFLAGS += -std=gnu99 CFLAGS += -ffreestanding@@ -73,6 +80,12 @@ FLATLIBS = $(libcflat) %.bin: %.elf $(OBJCOPY) -O binary $< $@ +%selftest.pv.bin: %selftest.bin $(HOST_KEY_DOCUMENT) $(patsubst %.pv.bin,%.parmfile,$@) + $(GENPROTIMG) --host-key-document $(HOST_KEY_DOCUMENT) --parmfile $(patsubst %.pv.bin,%.parmfile,$@) --no-verify --image $< -o $@ + +%.pv.bin: %.bin $(HOST_KEY_DOCUMENT) + $(GENPROTIMG) --host-key-document $(HOST_KEY_DOCUMENT) --no-verify --image $< -o $@ + arch_clean: asm_offsets_clean $(RM) $(TEST_DIR)/*.{o,elf,bin} $(TEST_DIR)/.*.d lib/s390x/.*.ds390x/selftest.parmfile new file +1
-0@@ -0,0 +1 @@ +test 123s390x/unittests.cfg +1
-0@@ -21,6 +21,7 @@ [selftest-setup] file = selftest.elf groups = selftest +# please keep the kernel cmdline in sync with $(TEST_DIR)/selftest.parmfile extra_params = -append 'test 123' [intercept]scripts/s390x/func.bash new file +26
-0@@ -0,0 +1,36 @@ +# The file scripts/common.bash has to be the only file sourcing this +# arch helper file +source config.mak + +ARCH_CMD=arch_cmd_s390x + +function arch_cmd_s390x() +{ + local cmd=$1 + local testname=$2 + local groups=$3 + local smp=$4 + local kernel=$5 + local opts=$6 + local arch=$7 + local check=$8 + local accel=$9 + local timeout=${10} + + # run the normal test case + "$cmd" "$testname" "$groups" "$smp" "$kernel" "$opts" "$arch" "$check" "$accel" "$timeout" + + # run PV test case + kernel=${kernel%.elf}.pv.bin + testname=${testname}_PV + if [ ! -f "${kernel}" ]; then
Quoted 8 lines
scripts/s390x/func.bash line 28
Quoted 2 lines
scripts/s390x/func.bash new file +2
-0+ if [ -z "${HOST_KEY_DOCUMENT}" ]; then + print_result 'SKIP' $testname '' 'no host-key document specified'I wonder whether we should not simply stay silent here ... ? Currently the output gets quite spoiled with a lot of these "no host-key document specified" messages when PV is not in use.
If you agree, I could drop this line when picking up the patch (no need to respin just because of this).
Right… yes, please remove it - thanks!
Quoted 3 lines
Thomas
Signature · 8 lines
Kind regards / Beste Grüße Marc Hartmayer IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Gregor Pillen Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft: Böblingen Registergericht: Amtsgericht Stuttgart, HRB 243294