Fix NBD TLS iotests on RHEL-7
v2 cover Fix NBD TLS iotests on RHEL-7
This fixes a failure of iotest 233 due to certtool problesm in RHEL7 wrt to SIGPIPE
Changed in v2:
- Put certtool.log into the $tls_dir instead of cwd
Daniel P. Berrangé (2):
iotests: ensure we print nbd server log on error
iotests: avoid broken pipe with certtool
tests/qemu-iotests/233 | 3 +++ tests/qemu-iotests/common.tls | 48 +++++++++++++++++++++++------------ 2 files changed, 35 insertions(+), 16 deletions(-)
Signature · 1 line
2.20.1
PATCH v2 1/2 iotests: ensure we print nbd server log on error
If we abort the iotest early the server.log file might contain useful information for diagnosing the problem. Ensure its contents are displayed in this case.
--- tests/qemu-iotests/233 | 3 +++ 1 file changed, 3 insertions(+)
tests/qemu-iotests/233 +3 -0
@@ -30,6 +30,8 @@ _cleanup() { nbd_server_stop _cleanup_test_img + # If we aborted early we want to see this log for diagnosis + test -f "$TEST_DIR/server.log" && cat "$TEST_DIR/server.log" rm -f "$TEST_DIR/server.log" tls_x509_cleanup }
@@ -120,6 +122,7 @@ $QEMU_IO -f $IMGFMT -r -U -c 'r -P 0x22 1m 1m' "$TEST_IMG" | _filter_qemu_io echo echo "== final server log ==" cat "$TEST_DIR/server.log" +rm -f $TEST_DIR/server.log # success, all done echo "*** done"
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH v2 1/2] iotests: ensure we print nbd server log on error
Quoted 24 lines · Daniel P. Berrangé
If we abort the iotest early the server.log file might contain useful information for diagnosing the problem. Ensure its contents are displayed in this case.
Reviewed-by Eric Blake <[email protected]>Signed-off-by Daniel P. Berrangé <[email protected]>--- tests/qemu-iotests/233 | 3 +++ 1 file changed, 3 insertions(+)tests/qemu-iotests/233 +2
-0@@ -30,6 +30,8 @@ _cleanup() { nbd_server_stop _cleanup_test_img + # If we aborted early we want to see this log for diagnosis + test -f "$TEST_DIR/server.log" && cat "$TEST_DIR/server.log" rm -f "$TEST_DIR/server.log" tls_x509_cleanup }
tests/qemu-iotests/233 $QEMU_IO -f $IMGFMT -r -U -c 'r -P 0x22 1m 1m' "$TEST_IMG" | _filter_qemu_io · line 125
Quoted 4 lines
tests/qemu-iotests/233 +1
-0@@ -120,6 +122,7 @@ $QEMU_IO -f $IMGFMT -r -U -c 'r -P 0x22 1m 1m' "$TEST_IMG" | _filter_qemu_io echo echo "== final server log ==" cat "$TEST_DIR/server.log" +rm -f $TEST_DIR/server.log
I'm not sure how well the iotests currently cope with spaced dir names anyway, but it looks weird to not use quotes here right after a line that does.
Max
Quoted 4 lines
tests/qemu-iotests/233 +0
-0# success, all done echo "*** done"
Re: [Qemu-devel] [PATCH v2 1/2] iotests: ensure we print nbd server log on error
Quoted 25 lines · Max Reitz
Quoted 24 lines · Daniel P. Berrangé
If we abort the iotest early the server.log file might contain useful information for diagnosing the problem. Ensure its contents are displayed in this case.
Reviewed-by Eric Blake <[email protected]>Signed-off-by Daniel P. Berrangé <[email protected]>--- tests/qemu-iotests/233 | 3 +++ 1 file changed, 3 insertions(+)tests/qemu-iotests/233 +2
-0@@ -30,6 +30,8 @@ _cleanup() { nbd_server_stop _cleanup_test_img + # If we aborted early we want to see this log for diagnosis + test -f "$TEST_DIR/server.log" && cat "$TEST_DIR/server.log" rm -f "$TEST_DIR/server.log" tls_x509_cleanup }@@ -120,6 +122,7 @@ $QEMU_IO -f $IMGFMT -r -U -c 'r -P 0x22 1m 1m' "$TEST_IMG" | _filter_qemu_io
Quoted 8 lines
tests/qemu-iotests/233 $QEMU_IO -f $IMGFMT -r -U -c 'r -P 0x22 1m 1m' "$TEST_IMG" | _filter_qemu_io · line 125
Quoted 4 lines
tests/qemu-iotests/233 +1
-0echo echo "== final server log ==" cat "$TEST_DIR/server.log" +rm -f $TEST_DIR/server.logI'm not sure how well the iotests currently cope with spaced dir names anyway, but it looks weird to not use quotes here right after a line that does.
Yes, that is a mistake since we tried quoting throughout the file
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [Qemu-devel] [PATCH v2 1/2] iotests: ensure we print nbd server log on error
Quoted 19 lines · Daniel P. Berrangé
Quoted 8 lines · Max Reitz
Quoted 6 lines · Daniel P. Berrangé
If we abort the iotest early the server.log file might contain useful information for diagnosing the problem. Ensure its contents are displayed in this case.
Reviewed-by Eric Blake <[email protected]>Signed-off-by Daniel P. Berrangé <[email protected]>Quoted 9 lines
tests/qemu-iotests/233 $QEMU_IO -f $IMGFMT -r -U -c 'r -P 0x22 1m 1m' "$TEST_IMG" | _filter_qemu_io · line 125
Quoted 5 lines
tests/qemu-iotests/233 +1
-0@@ -120,6 +122,7 @@ $QEMU_IO -f $IMGFMT -r -U -c 'r -P 0x22 1m 1m' "$TEST_IMG" | _filter_qemu_io echo echo "== final server log ==" cat "$TEST_DIR/server.log" +rm -f $TEST_DIR/server.logI'm not sure how well the iotests currently cope with spaced dir names anyway, but it looks weird to not use quotes here right after a line that does.
Quoted 1 line
Yes, that is a mistake since we tried quoting throughout the file
Can fix that while staging through my NBD queue. I'll probably send a PR on Monday.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
Re: [Qemu-devel] [PATCH v2 1/2] iotests: ensure we print nbd server log on error
Quoted 20 lines · Eric Blake
Quoted 19 lines · Daniel P. Berrangé
Quoted 17 lines · Max Reitz
tests/qemu-iotests/233 $QEMU_IO -f $IMGFMT -r -U -c 'r -P 0x22 1m 1m' "$TEST_IMG" | _filter_qemu_io · line 125
Quoted 12 lines · Daniel P. Berrangé
If we abort the iotest early the server.log file might contain useful information for diagnosing the problem. Ensure its contents are displayed in this case.
Reviewed-by Eric Blake <[email protected]>Signed-off-by Daniel P. Berrangé <[email protected]>tests/qemu-iotests/233 +1
-0@@ -120,6 +122,7 @@ $QEMU_IO -f $IMGFMT -r -U -c 'r -P 0x22 1m 1m' "$TEST_IMG" | _filter_qemu_io echo echo "== final server log ==" cat "$TEST_DIR/server.log" +rm -f $TEST_DIR/server.logI'm not sure how well the iotests currently cope with spaced dir names anyway, but it looks weird to not use quotes here right after a line that does.
Quoted 4 lines
Quoted 1 line
Yes, that is a mistake since we tried quoting throughout the file
Can fix that while staging through my NBD queue. I'll probably send a PR on Monday.
That's OK for me.
Max
Re: [Qemu-devel] [PATCH v2 1/2] iotests: ensure we print nbd server log on error
In reply to Eric BlakeQuoted 20 lines · Eric Blake
Quoted 19 lines · Daniel P. Berrangé
Quoted 17 lines · Max Reitz
tests/qemu-iotests/233 $QEMU_IO -f $IMGFMT -r -U -c 'r -P 0x22 1m 1m' "$TEST_IMG" | _filter_qemu_io · line 125
Quoted 12 lines · Daniel P. Berrangé
If we abort the iotest early the server.log file might contain useful information for diagnosing the problem. Ensure its contents are displayed in this case.
Reviewed-by Eric Blake <[email protected]>Signed-off-by Daniel P. Berrangé <[email protected]>tests/qemu-iotests/233 +1
-0@@ -120,6 +122,7 @@ $QEMU_IO -f $IMGFMT -r -U -c 'r -P 0x22 1m 1m' "$TEST_IMG" | _filter_qemu_io echo echo "== final server log ==" cat "$TEST_DIR/server.log" +rm -f $TEST_DIR/server.logI'm not sure how well the iotests currently cope with spaced dir names anyway, but it looks weird to not use quotes here right after a line that does.
Quoted 4 lines
Quoted 1 line
Yes, that is a mistake since we tried quoting throughout the file
Can fix that while staging through my NBD queue. I'll probably send a PR on Monday.
Great, thanks.
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
PATCH v2 2/2 iotests: avoid broken pipe with certtool
In reply to Daniel P. BerrangéWhen we run "certtool | head -1" the latter command is likely to complete and exit before certtool has written everything it wants to stderr. In at least the RHEL-7 gnutls 3.3.29 this causes certtool to quit with broken pipe before it has finished writing the desired output file to disk. This causes non-deterministic failures of the iotest 233 because the certs are sometimes zero length files. If certtool fails the "head -1" means we also loose any useful error message it would have printed.
Thus this patch gets rid of the pipe and post-processes the output in a more flexible & reliable manner.
--- tests/qemu-iotests/common.tls | 48 +++++++++++++++++++++++------------ 1 file changed, 32 insertions(+), 16 deletions(-)
tests/qemu-iotests/common.tls +32 -16
@@ -29,6 +29,17 @@ tls_x509_cleanup() } +tls_certtool() +{ + certtool "$@" 1>"${tls_dir}"/certtool.log 2>&1 + if test "$?" = 0; then + head -1 "${tls_dir}"/certtool.log + else + cat "${tls_dir}"/certtool.log + fi + rm -f "${tls_dir}"/certtool.log +} + tls_x509_init() { (certtool --help) >/dev/null 2>&1 || \
@@ -71,10 +82,11 @@ ca cert_signing_key EOF - certtool --generate-self-signed \ - --load-privkey "${tls_dir}/key.pem" \ - --template "${tls_dir}/ca.info" \ - --outfile "${tls_dir}/$name-cert.pem" 2>&1 | head -1 + tls_certtool \ + --generate-self-signed \ + --load-privkey "${tls_dir}/key.pem" \ + --template "${tls_dir}/ca.info" \ + --outfile "${tls_dir}/$name-cert.pem" rm -f "${tls_dir}/ca.info" }
@@ -98,12 +110,14 @@ encryption_key signing_key EOF - certtool --generate-certificate \ - --load-ca-privkey "${tls_dir}/key.pem" \ - --load-ca-certificate "${tls_dir}/$caname-cert.pem" \ - --load-privkey "${tls_dir}/key.pem" \ - --template "${tls_dir}/cert.info" \ - --outfile "${tls_dir}/$name/server-cert.pem" 2>&1 | head -1 + tls_certtool \ + --generate-certificate \ + --load-ca-privkey "${tls_dir}/key.pem" \ + --load-ca-certificate "${tls_dir}/$caname-cert.pem" \ + --load-privkey "${tls_dir}/key.pem" \ + --template "${tls_dir}/cert.info" \ + --outfile "${tls_dir}/$name/server-cert.pem" + ln -s "${tls_dir}/$caname-cert.pem" "${tls_dir}/$name/ca-cert.pem" ln -s "${tls_dir}/key.pem" "${tls_dir}/$name/server-key.pem"
@@ -127,12 +141,14 @@ encryption_key signing_key EOF - certtool --generate-certificate \ - --load-ca-privkey "${tls_dir}/key.pem" \ - --load-ca-certificate "${tls_dir}/$caname-cert.pem" \ - --load-privkey "${tls_dir}/key.pem" \ - --template "${tls_dir}/cert.info" \ - --outfile "${tls_dir}/$name/client-cert.pem" 2>&1 | head -1 + tls_certtool \ + --generate-certificate \ + --load-ca-privkey "${tls_dir}/key.pem" \ + --load-ca-certificate "${tls_dir}/$caname-cert.pem" \ + --load-privkey "${tls_dir}/key.pem" \ + --template "${tls_dir}/cert.info" \ + --outfile "${tls_dir}/$name/client-cert.pem" + ln -s "${tls_dir}/$caname-cert.pem" "${tls_dir}/$name/ca-cert.pem" ln -s "${tls_dir}/key.pem" "${tls_dir}/$name/client-key.pem"
Signature · 1 line
2.20.1
Re: [Qemu-devel] [PATCH v2 2/2] iotests: avoid broken pipe with certtool
Quoted 3 lines · Daniel P. Berrangé
When we run "certtool | head -1" the latter command is likely to complete and exit before certtool has written everything it wants to stderr. In at least the RHEL-7 gnutls 3.3.29 this causes certtool to
A bit of a mismatch: had we actually used only 'certtool | head -1', it would be early death before writing all it wants to stdout (not stderr).
But since the patch itself is replacing 'certtool 2>&1 | head -1',
where stderr is also in the picture due to the additional fd redirection, I'm not sure if it is better to just s/"certtool
|/"certtool 2>&1 |/ to match the patch, or to s/stderr/stdout/ for brevity, or to just leave things as written. Your call.
Quoted 4 lines
quit with broken pipe before it has finished writing the desired output file to disk. This causes non-deterministic failures of the iotest 233 because the certs are sometimes zero length files. If certtool fails the "head -1" means we also loose any useful error
s/loose/lose/
Quoted 11 lines
message it would have printed.
Thus this patch gets rid of the pipe and post-processes the output in a more flexible & reliable manner.
Reported-by Thomas Huth <[email protected]>Signed-off-by Daniel P. Berrangé <[email protected]>--- tests/qemu-iotests/common.tls | 48 +++++++++++++++++++++++------------ 1 file changed, 32 insertions(+), 16 deletions(-)
As the fix is for an iotest using NBD, I can take this through my tree if no one else picks it up through some other block or iotests tree first.
Signature · 3 lines
Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org