travis-ci: Add a KVM-only s390x job
v2 cover travis-ci: Add a KVM-only s390x job
Add a Travis job to build a KVM-only QEMU on s390x.
This series also contains few fixes for Travis/s390x.
Since v1:
- Do not disable autoconverge on s390x, but reduce the test initial bandwidth (dgilbert)
- Added danpb R-b tags
Philippe Mathieu-Daudé (4):
tests/test-util-filemonitor: Fix Travis-CI $ARCH env variable name tests/test-util-sockets: Skip test on non-x86 Travis containers tests/migration: Reduce autoconverge initial bandwidth .travis.yml: Add a KVM-only s390x job
tests/qtest/migration-test.c | 2 +- tests/test-util-filemonitor.c | 2 +- tests/test-util-sockets.c | 7 ++++++ .travis.yml | 42 +++++++++++++++++++++++++++++++++++ 4 files changed, 51 insertions(+), 2 deletions(-)
Signature · 1 line
2.21.1
PATCH v2 1/4 tests/test-util-filemonitor: Fix Travis-CI $ARCH env variable name
While we can find reference of a 'TRAVIS_ARCH' variable in the environment and source [1], per the Travis-CI multi-arch documentation [2] the variable is named TRAVIS_CPU_ARCH.
[1] https://github.com/travis-ci/travis-build/blob/v10.0.0/lib/travis/build/bash/travis_setup_env.bash#L39
[2] https://docs.travis-ci.com/user/multi-cpu-architectures/#identifying-cpu-architecture-of-build-jobs
--- tests/test-util-filemonitor.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/test-util-filemonitor.c +1 -1
@@ -415,7 +415,7 @@ test_file_monitor_events(void)* This test does not work on Travis LXD containers since some * syscalls are blocked in that environment. */ - travis_arch =getenv ("TRAVIS_ARCH" ); + travis_arch =getenv ("TRAVIS_CPU_ARCH" );if (travis_arch && !g_str_equal (travis_arch,"x86_64" )) {g_test_skip ("Test does not work on non-x86 Travis containers." );return ;
Signature · 1 line
2.21.1
PATCH v2 2/4 tests/test-util-sockets: Skip test on non-x86 Travis containers
In reply to Philippe Mathieu-DaudéSimilarly to commit 4f370b1098, test-util-sockets fails in restricted non-x86 Travis containers since they apparently blacklisted some required system calls there. Let's simply skip the test if we detect such an environment.
--- tests/test-util-sockets.c | 7 +++++++ 1 file changed, 7 insertions(+)
tests/test-util-sockets.c +7 -0
@@ -231,11 +231,18 @@ static void test_socket_fd_pass_num_nocli(void)int main (int argc,char **argv) {bool has_ipv4, has_ipv6; +char *travis_arch;socket_init ();g_test_init (&argc, &argv,NULL ); + travis_arch =getenv ("TRAVIS_CPU_ARCH" ); +if (travis_arch && !g_str_equal (travis_arch,"x86_64" )) { +g_printerr ("Test does not work on non-x86 Travis containers." ); + goto end; + } +/* We're creating actual IPv4/6 sockets, so we should * check if the host running tests actually supports * each protocol to avoid breaking tests on machines
Signature · 1 line
2.21.1
Re: [PATCH-for-5.0 v2 2/4] tests/test-util-sockets: Skip test on non-x86 Travis containers
Quoted 3 lines · On Wed, 18 Mar 2020 23:27:15 +0100 Philippe Mathieu-Daudé <[email protected]>…
Similarly to commit
4f370b1098, test-util-sockets fails in restricted non-x86 Travis containers since they apparently blacklisted some required system calls there.
Is "they" == "Travis admins"? Can we get them to remove those calls from the blacklist?
(I'm wondering why x86 allows those calls. Probably just because it has been around for longer.)
Quoted 31 lines
Let's simply skip the test if we detect such an environment.
Reviewed-by Daniel P. Berrangé <[email protected]>Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- tests/test-util-sockets.c | 7 +++++++ 1 file changed, 7 insertions(+)tests/test-util-sockets.c +7
-0@@ -231,11 +231,18 @@ static void test_socket_fd_pass_num_nocli(void)int main (int argc,char **argv) {bool has_ipv4, has_ipv6; +char *travis_arch;socket_init ();g_test_init (&argc, &argv,NULL ); + travis_arch =getenv ("TRAVIS_CPU_ARCH" ); +if (travis_arch && !g_str_equal (travis_arch,"x86_64" )) { +g_printerr ("Test does not work on non-x86 Travis containers." ); + goto end; + } +/* We're creating actual IPv4/6 sockets, so we should * check if the host running tests actually supports * each protocol to avoid breaking tests on machines
Re: [PATCH-for-5.0 v2 2/4] tests/test-util-sockets: Skip test on non-x86 Travis containers
Quoted 9 lines · Cornelia Huck
Quoted 3 lines · On Wed, 18 Mar 2020 23:27:15 +0100 Philippe Mathieu-Daudé <[email protected]>…
Similarly to commit
4f370b1098, test-util-sockets fails in restricted non-x86 Travis containers since they apparently blacklisted some required system calls there.Is "they" == "Travis admins"? Can we get them to remove those calls from the blacklist?
I suppose, I copy/pasted Thomas's description from commit 4f370b1098.
No clue, but we can try :)
Quoted 36 lines
(I'm wondering why x86 allows those calls. Probably just because it has been around for longer.)
Quoted 31 lines
Let's simply skip the test if we detect such an environment.
Reviewed-by Daniel P. Berrangé <[email protected]>Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- tests/test-util-sockets.c | 7 +++++++ 1 file changed, 7 insertions(+)tests/test-util-sockets.c +7
-0@@ -231,11 +231,18 @@ static void test_socket_fd_pass_num_nocli(void)int main (int argc,char **argv) {bool has_ipv4, has_ipv6; +char *travis_arch;socket_init ();g_test_init (&argc, &argv,NULL ); + travis_arch =getenv ("TRAVIS_CPU_ARCH" ); +if (travis_arch && !g_str_equal (travis_arch,"x86_64" )) { +g_printerr ("Test does not work on non-x86 Travis containers." ); + goto end; + } +/* We're creating actual IPv4/6 sockets, so we should * check if the host running tests actually supports * each protocol to avoid breaking tests on machines
Re: [PATCH-for-5.0 v2 2/4] tests/test-util-sockets: Skip test on non-x86 Travis containers
In reply to Cornelia HuckQuoted 7 lines · Cornelia Huck
Quoted 3 lines · On Wed, 18 Mar 2020 23:27:15 +0100 Philippe Mathieu-Daudé <[email protected]>…
Similarly to commit
4f370b1098, test-util-sockets fails in restricted non-x86 Travis containers since they apparently blacklisted some required system calls there.
Quoted 5 lines
Is "they" == "Travis admins"? Can we get them to remove those calls from the blacklist?
(I'm wondering why x86 allows those calls. Probably just because it has been around for longer.)
I believe it is due to the different build env architecture for the non-x86 arches, which is based on a strict locked down container, compared to x86 which IIUC was a full VM.
Blacklisting like this is a bit hacky, but a prudent fix in the short term.
Regards,
Daniel
Signature · 3 lines
|: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
Re: [PATCH-for-5.0 v2 2/4] tests/test-util-sockets: Skip test on non-x86 Travis containers
Quoted 11 lines · Daniel P. Berrangé
Quoted 10 lines · Cornelia Huck
Quoted 3 lines · On Wed, 18 Mar 2020 23:27:15 +0100 Philippe Mathieu-Daudé <[email protected]>…
Similarly to commit
4f370b1098, test-util-sockets fails in restricted non-x86 Travis containers since they apparently blacklisted some required system calls there.Is "they" == "Travis admins"? Can we get them to remove those calls from the blacklist?
Quoted 6 lines
Quoted 2 lines
(I'm wondering why x86 allows those calls. Probably just because it has been around for longer.)
I believe it is due to the different build env architecture for the non-x86 arches, which is based on a strict locked down container, compared to x86 which IIUC was a full VM.
Indeed, Thomas described:
"Unfortunately, the builds are running in quite restricted
LXD containers there, for example it is not possible to create huge files there (even if they are just sparse), and certain system calls are blocked. So we have to change some tests first to stop them failing in such environments."
https://www.mail-archive.com/[email protected]/msg659909.html
Quoted 7 lines
Blacklisting like this is a bit hacky, but a prudent fix in the short term.
Regards,
Daniel
PATCH v2 3/4 tests/migration: Reduce autoconverge initial bandwidth
In reply to Philippe Mathieu-DaudéWhen using max-bandwidth=~100Mb/s, this test fails on Travis-CI s390x when configured with --disable-tcg:
$ make check-qtest
TEST check-qtest-s390x: tests/qtest/boot-serial-test
qemu-system-s390x: -accel tcg: invalid accelerator tcg
qemu-system-s390x: falling back to KVM
TEST check-qtest-s390x: tests/qtest/pxe-test
TEST check-qtest-s390x: tests/qtest/test-netfilter
TEST check-qtest-s390x: tests/qtest/test-filter-mirror
TEST check-qtest-s390x: tests/qtest/test-filter-redirector
TEST check-qtest-s390x: tests/qtest/drive_del-test
TEST check-qtest-s390x: tests/qtest/device-plug-test
TEST check-qtest-s390x: tests/qtest/virtio-ccw-test
TEST check-qtest-s390x: tests/qtest/cpu-plug-test
TEST check-qtest-s390x: tests/qtest/migration-test
**
ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE
ERROR - Bail out! ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE
make: *** [tests/Makefile.include:633: check-qtest-s390x] Error 1Per David Gilbert, "it could just be the writing is slow on s390 and the migration thread fast; in which case the autocomplete wouldn't be needed. Perhaps we just need to reduce the bandwidth limit."
Tuning the threshold by reducing the initial bandwidth makes the autoconverge test pass.
--- tests/qtest/migration-test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/qtest/migration-test.c +1 -1
@@ -1211,7 +1211,7 @@ static void test_migrate_auto_converge(void)* without throttling. */ migrate_set_parameter_int (from,"downtime-limit" ,1 ); -migrate_set_parameter_int (from,"max-bandwidth" ,100000000 );/* ~100Mb/s */ +migrate_set_parameter_int (from,"max-bandwidth" ,10000000000 );/* ~10Gb/s */ /* To check remaining size after precopy */ migrate_set_capability (from,"pause-before-switchover" , true);
Signature · 1 line
2.21.1
Re: [PATCH-for-5.0 v2 3/4] tests/migration: Reduce autoconverge initial bandwidth
Quoted 40 lines · * Philippe Mathieu-Daudé ([email protected]) wrote:
When using max-bandwidth=~100Mb/s, this test fails on Travis-CI s390x when configured with --disable-tcg:
$ make check-qtest TEST check-qtest-s390x: tests/qtest/boot-serial-test qemu-system-s390x: -accel tcg: invalid accelerator tcg qemu-system-s390x: falling back to KVM TEST check-qtest-s390x: tests/qtest/pxe-test TEST check-qtest-s390x: tests/qtest/test-netfilter TEST check-qtest-s390x: tests/qtest/test-filter-mirror TEST check-qtest-s390x: tests/qtest/test-filter-redirector TEST check-qtest-s390x: tests/qtest/drive_del-test TEST check-qtest-s390x: tests/qtest/device-plug-test TEST check-qtest-s390x: tests/qtest/virtio-ccw-test TEST check-qtest-s390x: tests/qtest/cpu-plug-test TEST check-qtest-s390x: tests/qtest/migration-test ** ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE ERROR - Bail out! ERROR:tests/qtest/migration-test.c:1229:test_migrate_auto_converge: 'got_stop' should be FALSE make: *** [tests/Makefile.include:633: check-qtest-s390x] Error 1Per David Gilbert, "it could just be the writing is slow on s390 and the migration thread fast; in which case the autocomplete wouldn't be needed. Perhaps we just need to reduce the bandwidth limit."
Tuning the threshold by reducing the initial bandwidth makes the autoconverge test pass.
Suggested-by Dr. David Alan Gilbert <[email protected]>Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- tests/qtest/migration-test.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tests/qtest/migration-test.c +0
-0
tests/qtest/migration-test.c static void test_migrate_auto_converge(void) · line 1214
Quoted 5 lines
tests/qtest/migration-test.c +1
-1@@ -1211,7 +1211,7 @@ static void test_migrate_auto_converge(void)* without throttling. */ migrate_set_parameter_int (from,"downtime-limit" ,1 ); -migrate_set_parameter_int (from,"max-bandwidth" ,100000000 );/* ~100Mb/s */ +migrate_set_parameter_int (from,"max-bandwidth" ,10000000000 );/* ~10Gb/s */
Hmm - I was expecting you to need to *reduce* the speed; now I'm very confused.
You should get 'stop' if the migration completed too soon - surely incresing the bandwidth would make that *more* likely to happen.
Dave
Quoted 5 lines
tests/qtest/migration-test.c +0
-0/* To check remaining size after precopy */ migrate_set_capability (from,"pause-before-switchover" , true);Signature · 1 line
2.21.1
--
Dr. David Alan Gilbert / [email protected] / Manchester, UK
PATCH v2 4/4 .travis.yml: Add a KVM-only s390x job
In reply to Philippe Mathieu-DaudéAdd a job to build QEMU on s390x with TCG disabled, so this configuration won't bitrot over time.
This job is quick, running check-unit: Ran for 4 min 48 sec
https://travis-ci.org/github/philmd/qemu/jobs/663659486
--- .travis.yml | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+)
.travis.yml +42 -0
@@ -524,6 +524,48 @@ jobs: $(exit $BUILD_RC); fi + - name: "[s390x] GCC check (KVM)" + arch: s390x + dist: bionic + addons: + apt_packages: + - libaio-dev + - libattr1-dev + - libbrlapi-dev + - libcap-ng-dev + - libgcrypt20-dev + - libgnutls28-dev + - libgtk-3-dev + - libiscsi-dev + - liblttng-ust-dev + - libncurses5-dev + - libnfs-dev + - libnss3-dev + - libpixman-1-dev + - libpng-dev + - librados-dev + - libsdl2-dev + - libseccomp-dev + - liburcu-dev + - libusb-1.0-0-dev + - libvdeplug-dev + - libvte-2.91-dev + # Tests dependencies + - genisoimage + env: + - TEST_CMD="make check-unit" + - CONFIG="--disable-containers --disable-tcg --enable-kvm --disable-tools" + script: + - ( cd ${SRC_DIR} ; git submodule update --init roms/SLOF ) + - BUILD_RC=0 && make -j${JOBS} || BUILD_RC=$? + - | + if [ "$BUILD_RC" -eq 0 ] ; then + mv pc-bios/s390-ccw/*.img pc-bios/ ; + ${TEST_CMD} ; + else + $(exit $BUILD_RC); + fi + # Release builds # The make-release script expect a QEMU version, so our tag must start with a 'v'. # This is the case when release candidate tags are created.
Signature · 1 line
2.21.1
Re: [PATCH v2 4/4] .travis.yml: Add a KVM-only s390x job
Quoted 10 lines · On Wed, 18 Mar 2020 23:27:17 +0100 Philippe Mathieu-Daudé <[email protected]>…
Add a job to build QEMU on s390x with TCG disabled, so this configuration won't bitrot over time.
This job is quick, running check-unit: Ran for 4 min 48 sec
https://travis-ci.org/github/philmd/qemu/jobs/663659486Signed-off-by Philippe Mathieu-Daudé <[email protected]>--- .travis.yml | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+)
That looks like a good idea.
Patchew URL: https://patchew.org/QEMU/[email protected]/
Patchew URL: https://patchew.org/QEMU/[email protected]/
Hi,
This series failed the docker-quick@centos7 build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.
=== TEST SCRIPT BEGIN ===
#!/bin/bash
make docker-image-centos7 V=1 NETWORK=1
time make docker-test-quick@centos7 SHOW_ENV=1 J=14 NETWORK=1 === TEST SCRIPT END ===
TEST check-unit: tests/test-rcu-simpleq
TEST iotest-qcow2: 024
**
ERROR:/tmp/qemu-test/src/tests/qtest/migration-test.c:1249:test_migrate_auto_converge: assertion failed (remaining < (expected_threshold + expected_threshold / 100)): (103809024 < 101000000) ERROR - Bail out! ERROR:/tmp/qemu-test/src/tests/qtest/migration-test.c:1249:test_migrate_auto_converge: assertion failed (remaining < (expected_threshold + expected_threshold / 100)): (103809024 < 101000000)
make: *** [check-qtest-aarch64] Error 1
make: *** Waiting for unfinished jobs....
TEST check-unit: tests/test-rcu-tailq
TEST iotest-qcow2: 025
---
qemu-system-x86_64: -accel kvm: failed to initialize kvm: No such file or directory
qemu-system-x86_64: falling back to tcg
**
ERROR:/tmp/qemu-test/src/tests/qtest/migration-test.c:1249:test_migrate_auto_converge: assertion failed (remaining < (expected_threshold + expected_threshold / 100)): (103809024 < 101000000) ERROR - Bail out! ERROR:/tmp/qemu-test/src/tests/qtest/migration-test.c:1249:test_migrate_auto_converge: assertion failed (remaining < (expected_threshold + expected_threshold / 100)): (103809024 < 101000000)
make: *** [check-qtest-x86_64] Error 1
TEST iotest-qcow2: 041 TEST iotest-qcow2: 042 TEST iotest-qcow2: 043
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--label', 'com.qemu.instance.uuid=ff24ee2953f1454290305113cefa8f7e', '-u', '1001', '--security-opt', 'seccomp=unconfined', '--rm', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=1', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-qj2x29ta/src/docker-src.2020-03-18-20.12.13.29783:/var/tmp/qemu:z,ro', 'qemu:centos7', '/var/tmp/qemu/run', 'test-quick']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=ff24ee2953f1454290305113cefa8f7e
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-qj2x29ta/src'
make: *** [docker-run-test-quick@centos7] Error 2
real 11m32.251s
user 0m8.589s
The full log is available at
http://patchew.org/logs/[email protected]/testing.docker-quick@centos7/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]