Improvements for the s390x panic-loop tests
cover Improvements for the s390x panic-loop tests
The panic-loop tests sometimes fail in our downstream CI. The first patch fixes this issue. While we're at it, add the tests to the upstream CI, too (second patch). The third patch is a RFC: Is it ok to drop our dependecy on "jq" for these tests by replacing it with "grep"?
Thomas Huth (3):
s390x: Fix unreliable panic-loop tests .gitlab-ci.yml: Add the s390x panic-loop tests to the CI scripts/arch-run.bash: Drop the dependency on "jq"
.gitlab-ci.yml | 2 ++ scripts/arch-run.bash | 27 +++++---------------------- 2 files changed, 7 insertions(+), 22 deletions(-)
Signature · 1 line
2.50.1
PATCH 1/3 s390x: Fix unreliable panic-loop tests
From: Thomas Huth <[email protected]>
In our CI, the s390x panic-loop-extint and panic-loop-pgm tests are sometimes failing. Having a closer look, this seems to be caused by ncat sometimes complaining about "Connection reset by peer" on stderr, likely because QEMU terminated (due to the panic) before ncat could properly tear down the connection. But having some output on stderr is interpreted as test failure in qemu_fixup_return_code(), so the test is marked as failed, even though the panic event occurred as expected.
To fix it, drop the usage of ncat here and simply handle the QMP input and output via normal fifos instead. This has also the advantage that we do not need an additional program for these tests anymore that might not be available in the installation.
--- scripts/arch-run.bash | 21 +++++---------------- 1 file changed, 5 insertions(+), 16 deletions(-)
scripts/arch-run.bash +5 -16
@@ -63,14 +63,6 @@ qmp () echo '{ "execute": "qmp_capabilities" }{ "execute":' "$2" '}' | ncat -U $1 } -qmp_events () -{ - while ! test -S "$1"; do sleep 0.1; done - echo '{ "execute": "qmp_capabilities" }{ "execute": "cont" }' | - ncat --no-shutdown -U $1 | - jq -c 'select(has("event"))' -} - filter_quiet_msgs () { grep -v "Now migrate the VM (quiet)" |
@@ -295,26 +287,23 @@ do_migration () run_panic () { - if ! command -v ncat >/dev/null 2>&1; then - echo "${FUNCNAME[0]} needs ncat (netcat)" >&2 - return 77 - fi - if ! command -v jq >/dev/null 2>&1; then echo "${FUNCNAME[0]} needs jq" >&2 return 77 fi trap 'trap - TERM ; kill 0 ; exit 2' INT TERM - trap 'rm -f ${qmp}' RETURN EXIT + trap 'rm -f ${qmp}.in ${qmp}.out' RETURN EXIT qmp=$(mktemp -u -t panic-qmp.XXXXXXXXXX) + mkfifo ${qmp}.in ${qmp}.out # start VM stopped so we don't miss any events - "$@" -chardev socket,id=mon,path=${qmp},server=on,wait=off \ + "$@" -chardev pipe,id=mon,path=${qmp} \ -mon chardev=mon,mode=control -S & + echo '{ "execute": "qmp_capabilities" }{ "execute": "cont" }' > ${qmp}.in - panic_event_count=$(qmp_events ${qmp} | jq -c 'select(.event == "GUEST_PANICKED")' | wc -l) + panic_event_count=$(jq -c 'select(.event == "GUEST_PANICKED")' < ${qmp}.out | wc -l) if [ "$panic_event_count" -lt 1 ]; then echo "FAIL: guest did not panic" ret=3
Signature · 1 line
2.50.1
Re: [kvm-unit-tests PATCH 1/3] s390x: Fix unreliable panic-loop tests
Quoted 10 lines · On Thu, 24 Jul 2025 15:30:49 +0200 Thomas Huth <[email protected]> wrote:
From: Thomas Huth <[email protected]>
In our CI, the s390x panic-loop-extint and panic-loop-pgm tests are sometimes failing. Having a closer look, this seems to be caused by ncat sometimes complaining about "Connection reset by peer" on stderr, likely because QEMU terminated (due to the panic) before ncat could properly tear down the connection. But having some output on stderr is interpreted as test failure in
qemu_fixup_return_code(), so the test is marked as failed, even though the panic event occurred as expected.
Quoted 6 lines
To fix it, drop the usage of ncat here and simply handle the QMP input and output via normal fifos instead. This has also the advantage that we do not need an additional program for these tests anymore that might not be available in the installation.
Signed-off-by Thomas Huth <[email protected]>
Quoted 55 lines
--- scripts/arch-run.bash | 21 +++++---------------- 1 file changed, 5 insertions(+), 16 deletions(-)scripts/arch-run.bash +5
-16@@ -63,14 +63,6 @@ qmp () echo '{ "execute": "qmp_capabilities" }{ "execute":' "$2" '}' | ncat -U $1 } -qmp_events () -{ - while ! test -S "$1"; do sleep 0.1; done - echo '{ "execute": "qmp_capabilities" }{ "execute": "cont" }' | - ncat --no-shutdown -U $1 | - jq -c 'select(has("event"))' -} - filter_quiet_msgs () { grep -v "Now migrate the VM (quiet)" |@@ -295,26 +287,23 @@ do_migration () run_panic () { - if ! command -v ncat >/dev/null 2>&1; then - echo "${FUNCNAME[0]} needs ncat (netcat)" >&2 - return 77 - fi - if ! command -v jq >/dev/null 2>&1; then echo "${FUNCNAME[0]} needs jq" >&2 return 77 fi trap 'trap - TERM ; kill 0 ; exit 2' INT TERM - trap 'rm -f ${qmp}' RETURN EXIT + trap 'rm -f ${qmp}.in ${qmp}.out' RETURN EXIT qmp=$(mktemp -u -t panic-qmp.XXXXXXXXXX) + mkfifo ${qmp}.in ${qmp}.out # start VM stopped so we don't miss any events - "$@" -chardev socket,id=mon,path=${qmp},server=on,wait=off \ + "$@" -chardev pipe,id=mon,path=${qmp} \ -mon chardev=mon,mode=control -S & + echo '{ "execute": "qmp_capabilities" }{ "execute": "cont" }' > ${qmp}.in - panic_event_count=$(qmp_events ${qmp} | jq -c 'select(.event == "GUEST_PANICKED")' | wc -l) + panic_event_count=$(jq -c 'select(.event == "GUEST_PANICKED")' < ${qmp}.out | wc -l) if [ "$panic_event_count" -lt 1 ]; then echo "FAIL: guest did not panic" ret=3
PATCH 2/3 .gitlab-ci.yml: Add the s390x panic-loop tests to the CI
In reply to Thomas HuthFrom: Thomas Huth <[email protected]>
Now that these tests should work reliable, we should also run them in our CI.
--- .gitlab-ci.yml | 2 ++ 1 file changed, 2 insertions(+)
.gitlab-ci.yml +2 -0
@@ -517,6 +517,8 @@ s390x-kvm: migration-sck migration-skey-parallel migration-skey-sequential + panic-loop-extint + panic-loop-pgm pfmf sclp-1g sclp-3g
Signature · 1 line
2.50.1
Re: [kvm-unit-tests PATCH 2/3] .gitlab-ci.yml: Add the s390x panic-loop tests to the CI
Quoted 6 lines · On Thu, 24 Jul 2025 15:30:50 +0200 Thomas Huth <[email protected]> wrote:
From: Thomas Huth <[email protected]>
Now that these tests should work reliable, we should also run them in our CI.
Signed-off-by Thomas Huth <[email protected]>
Quoted 17 lines
--- .gitlab-ci.yml | 2 ++ 1 file changed, 2 insertions(+).gitlab-ci.yml +2
-0@@ -517,6 +517,8 @@ s390x-kvm: migration-sck migration-skey-parallel migration-skey-sequential + panic-loop-extint + panic-loop-pgm pfmf sclp-1g sclp-3g
RFC PATCH 3/3 scripts/arch-run.bash: Drop the dependency on "jq"
In reply to Thomas HuthFrom: Thomas Huth <[email protected]>
For checking whether a panic event occurred, a simple "grep" for the related text in the output is enough - it's very unlikely that the output of QEMU will change. This way we can drop the dependency on the program "jq" which might not be installed on some systems.
---
Marked as "RFC" since I'm a little bit torn here - on the one side, it's great to get rid of a dependency, on the other side, using grep might be a little bit less robust in case QEMU ever changes the layout of it's QMP output...
scripts/arch-run.bash | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-)
scripts/arch-run.bash +1 -7
@@ -287,11 +287,6 @@ do_migration () run_panic () { - if ! command -v jq >/dev/null 2>&1; then - echo "${FUNCNAME[0]} needs jq" >&2 - return 77 - fi - trap 'trap - TERM ; kill 0 ; exit 2' INT TERM trap 'rm -f ${qmp}.in ${qmp}.out' RETURN EXIT
@@ -303,8 +298,7 @@ run_panic () -mon chardev=mon,mode=control -S & echo '{ "execute": "qmp_capabilities" }{ "execute": "cont" }' > ${qmp}.in - panic_event_count=$(jq -c 'select(.event == "GUEST_PANICKED")' < ${qmp}.out | wc -l) - if [ "$panic_event_count" -lt 1 ]; then + if ! grep -q '"event": "GUEST_PANICKED"' ${qmp}.out ; then echo "FAIL: guest did not panic" ret=3 else
Signature · 1 line
2.50.1
Re: [kvm-unit-tests RFC PATCH 3/3] scripts/arch-run.bash: Drop the dependency on "jq"
Quoted 8 lines · On Thu, 24 Jul 2025 15:30:51 +0200 Thomas Huth <[email protected]> wrote:
From: Thomas Huth <[email protected]>
For checking whether a panic event occurred, a simple "grep" for the related text in the output is enough - it's very unlikely that the output of QEMU will change. This way we can drop the dependency on the program "jq" which might not be installed on some systems.
Quoted 6 lines
Signed-off-by Thomas Huth <[email protected]>---
Marked as "RFC" since I'm a little bit torn here - on the one side, it's great to get rid of a dependency, on the other side, using grep might be a little bit less robust in case QEMU ever changes the layout of it's QMP output...
I share your concerns, but I'm also in favour of removing a relatively uncommon dependency
Quoted 21 lines
scripts/arch-run.bash | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-)scripts/arch-run.bash +0
-5@@ -287,11 +287,6 @@ do_migration () run_panic () { - if ! command -v jq >/dev/null 2>&1; then - echo "${FUNCNAME[0]} needs jq" >&2 - return 77 - fi - trap 'trap - TERM ; kill 0 ; exit 2' INT TERM trap 'rm -f ${qmp}.in ${qmp}.out' RETURN EXIT
scripts/arch-run.bash run_panic () · line 301
Quoted 6 lines
scripts/arch-run.bash +1
-2@@ -303,8 +298,7 @@ run_panic () -mon chardev=mon,mode=control -S & echo '{ "execute": "qmp_capabilities" }{ "execute": "cont" }' > ${qmp}.in - panic_event_count=$(jq -c 'select(.event == "GUEST_PANICKED")' < ${qmp}.out | wc -l) - if [ "$panic_event_count" -lt 1 ]; then + if ! grep -q '"event": "GUEST_PANICKED"' ${qmp}.out ; then
maybe:
grep -E -q '"event"[[:blank:]]*:[[:blank:]]*"GUEST PANICKED"' ...
Quoted 3 lines
scripts/arch-run.bash +0
-0echo "FAIL: guest did not panic" ret=3 else