Skip to contentSkip to conversation

travis.yml improvements: Update libraries, build with arm64

PATCH In review Thomas Huth 4 participants 22 messages Oct 9, 2019 – Oct 22, 2019

lore
Share & tools
lore
#0 lore raw

cover travis.yml improvements: Update libraries, build with arm64

Update / add some libraries that we use in the Travis builds, and enable compiling on the aarch64 host, too.

Thomas Huth (5):
travis.yml: Add libvdeplug-dev to compile-test net/vde.c travis.yml: Use libsdl2 instead of libsdl1.2, and install
libsdl2-image
travis.yml: Use newer version of libgnutls and libpng travis.yml: Fix the ccache lines
travis.yml: Compile on arm64, too

 .travis.yml | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)
Signature · 1 line
2.18.1
#1 lore raw

PATCH 1/5 travis.yml: Add libvdeplug-dev to compile-test net/vde.c

This library is needed to compile the VDE network backend.

---
 .travis.yml | 1 +
 1 file changed, 1 insertion(+)
.travis.yml +1 -0
index d0b9e099b9..63328387f6 100644
@@ -46,6 +46,7 @@ addons:
       - libssh-dev
       - liburcu-dev
       - libusb-1.0-0-dev
+      - libvdeplug-dev
       - libvte-2.91-dev
       - sparse
       - uuid-dev
Signature · 1 line
2.18.1
#2 lore raw

PATCH 2/5 travis.yml: Use libsdl2 instead of libsdl1.2, and install libsdl2-image

In reply to Thomas Huth

We've removed support for SDL 1.2 quite a while ago already, so let's use SDL 2 now in Travis to get test coverage for SDL again. And while we're at it, also add libsdl2-image-dev which can be used by QEMU nowadays, too.

---
 .travis.yml | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
.travis.yml +4 -2
index 63328387f6..e21b9ad666 100644
@@ -39,7 +39,8 @@ addons:
       - libpixman-1-dev
       - libpng12-dev
       - librados-dev
-      - libsdl1.2-dev
+      - libsdl2-dev
+      - libsdl2-image-dev
       - libseccomp-dev
       - libspice-protocol-dev
       - libspice-server-dev
@@ -308,7 +309,8 @@ matrix:
             - libpixman-1-dev
             - libpng12-dev
             - librados-dev
-            - libsdl1.2-dev
+            - libsdl2-dev
+            - libsdl2-image-dev
             - libseccomp-dev
             - libspice-protocol-dev
             - libspice-server-dev
Signature · 1 line
2.18.1
#3 lore raw

PATCH 3/5 travis.yml: Use newer version of libgnutls and libpng

In reply to Thomas Huth

libgnutls-dev and libpng12-dev are not available in newer versions of Ubuntu anymore, so installing these packages fails e.g. in the new arm64 containers on Travis. Let's use newer versions of these packages by default instead. (The old versions still get tested in the "gcc-9" build).

---
 .travis.yml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
.travis.yml +2 -2
index e21b9ad666..616e59867a 100644
@@ -29,7 +29,7 @@ addons:
       - libcap-dev
       - libcap-ng-dev
       - libgcc-4.8-dev
-      - libgnutls-dev
+      - libgnutls28-dev
       - libgtk-3-dev
       - libiscsi-dev
       - liblttng-ust-dev
@@ -37,7 +37,7 @@ addons:
       - libnfs-dev
       - libnss3-dev
       - libpixman-1-dev
-      - libpng12-dev
+      - libpng-dev
       - librados-dev
       - libsdl2-dev
       - libsdl2-image-dev
Signature · 1 line
2.18.1
#4 lore raw

PATCH 4/5 travis.yml: Fix the ccache lines

In reply to Thomas Huth

The "command -v ccache && ccache ..." likely were supposed to test the availability of ccache before running the program. But this shell construct causes Travis to abort if ccache is not available. Use an if-statement instead to fix this problem.

---
 .travis.yml | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
.travis.yml +2 -2
index 616e59867a..0c88e8757b 100644
@@ -91,13 +91,13 @@ git:
 
 before_script:
   - if [ "$TRAVIS_OS_NAME" == "osx" ] ; then export PATH="/usr/local/opt/ccache/libexec:$PATH" ; fi
-  - command -v ccache && ccache --zero-stats
+  - if command -v ccache ; then ccache --zero-stats ; fi
   - mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR}
   - ${SRC_DIR}/configure ${BASE_CONFIG} ${CONFIG} || { cat config.log && exit 1; }
 script:
   - make -j3 && travis_retry ${TEST_CMD}
 after_script:
-  - command -v ccache && ccache --show-stats
+  - if command -v ccache ; then ccache --show-stats ; fi
 
 
 matrix:
Signature · 1 line
2.18.1
Reviewed-by on v1 4/5
#5 lore raw

Re: [PATCH 4/5] travis.yml: Fix the ccache lines

Quoted 3 lines · Thomas Huth

The "command -v ccache && ccache ..." likely were supposed to test the availability of ccache before running the program. But this shell construct causes Travis to abort if ccache is not available.

Oops.

Why can't you install ccache if these are Ubuntu systems? It is even more wanted if the arm64 machine are slow...

Quoted 18 lines

Use an if-statement instead to fix this problem.

---
  .travis.yml | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
.travis.yml +1 -1
index 616e59867a..0c88e8757b 100644
@@ -91,13 +91,13 @@ git:
 
  before_script:
    - if [ "$TRAVIS_OS_NAME" == "osx" ] ; then export PATH="/usr/local/opt/ccache/libexec:$PATH" ; fi
-  - command -v ccache && ccache --zero-stats
+  - if command -v ccache ; then ccache --zero-stats ; fi

.travis.yml git: · line 103

Quoted 10 lines
.travis.yml +1 -1
    - mkdir -p ${BUILD_DIR} && cd ${BUILD_DIR}
    - ${SRC_DIR}/configure ${BASE_CONFIG} ${CONFIG} || { cat config.log && exit 1; }
  script:
    - make -j3 && travis_retry ${TEST_CMD}
  after_script:
-  - command -v ccache && ccache --show-stats
+  - if command -v ccache ; then ccache --show-stats ; fi
 
 
  matrix:
#6 lore raw

Re: [PATCH 4/5] travis.yml: Fix the ccache lines

Quoted 9 lines · Philippe Mathieu-Daudé
Quoted 3 lines · Thomas Huth

The "command -v ccache && ccache ..." likely were supposed to test the availability of ccache before running the program. But this shell construct causes Travis to abort if ccache is not available.

Oops.

Why can't you install ccache if these are Ubuntu systems? It is even more wanted if the arm64 machine are slow...

I just tried to add "ccache" to the list of packages that should be installed, but I don't see a difference in the runtime.

First run with ccache enabled:

https://travis-ci.com/huth/qemu/jobs/244117945

Second run where I'd expect a speedup:

https://travis-ci.com/huth/qemu/jobs/244124599

The statistics at the end say that it had only 1 cache hit. Any ideas what might be wrong here?

Thomas

#7 lore raw

Re: [PATCH 4/5] travis.yml: Fix the ccache lines

Quoted 16 lines · Thomas Huth
Quoted 9 lines · Philippe Mathieu-Daudé
Quoted 3 lines · Thomas Huth

The "command -v ccache && ccache ..." likely were supposed to test the availability of ccache before running the program. But this shell construct causes Travis to abort if ccache is not available.

Oops.

Why can't you install ccache if these are Ubuntu systems? It is even more wanted if the arm64 machine are slow...

I just tried to add "ccache" to the list of packages that should be installed, but I don't see a difference in the runtime.

First run with ccache enabled:

Quoted 8 lines

https://travis-ci.com/huth/qemu/jobs/244117945

Second run where I'd expect a speedup:

https://travis-ci.com/huth/qemu/jobs/244124599

The statistics at the end say that it had only 1 cache hit. Any ideas what might be wrong here?

Looking there and your following commit (https://github.com/huth/qemu/commit/eaf80e7851) I see you already figured this out :)

#8 lore raw

Re: [PATCH 4/5] travis.yml: Fix the ccache lines

Quoted 21 lines · Philippe Mathieu-Daudé
Quoted 20 lines · Thomas Huth
Quoted 9 lines · Philippe Mathieu-Daudé
Quoted 3 lines · Thomas Huth

The "command -v ccache && ccache ..." likely were supposed to test the availability of ccache before running the program. But this shell construct causes Travis to abort if ccache is not available.

Oops.

Why can't you install ccache if these are Ubuntu systems? It is even more wanted if the arm64 machine are slow...

I just tried to add "ccache" to the list of packages that should be installed, but I don't see a difference in the runtime.

First run with ccache enabled:

  https://travis-ci.com/huth/qemu/jobs/244117945

Second run where I'd expect a speedup:

Quoted 8 lines
Quoted 4 lines

  https://travis-ci.com/huth/qemu/jobs/244124599

The statistics at the end say that it had only 1 cache hit. Any ideas what might be wrong here?

Looking there and your following commit
(https://github.com/huth/qemu/commit/eaf80e7851) I see you already figured this out :)

No, that was just a try, but it did not change anything:

https://travis-ci.com/huth/qemu/jobs/244137697#L5813

But I also noticed that in the arm64 builds, the cache information is missing:

https://travis-ci.com/huth/qemu/jobs/244137697#L1844

... so I assume that ccache support needs to be supported in the image that is provided by Travis, and you can not simply install it afterwards. So this is likely just a quirk that hopefully will be fixed by Travis later (arm64 is still marked as "alpha" there if I've got that right).

Thomas

#9 lore raw

Re: [PATCH 4/5] travis.yml: Fix the ccache lines

Quoted 27 lines · Thomas Huth
Quoted 26 lines · Philippe Mathieu-Daudé
Quoted 24 lines · Thomas Huth
Quoted 9 lines · Philippe Mathieu-Daudé
Quoted 3 lines · Thomas Huth

The "command -v ccache && ccache ..." likely were supposed to test the availability of ccache before running the program. But this shell construct causes Travis to abort if ccache is not available.

Oops.

Why can't you install ccache if these are Ubuntu systems? It is even more wanted if the arm64 machine are slow...

I just tried to add "ccache" to the list of packages that should be installed, but I don't see a difference in the runtime.

First run with ccache enabled:

  https://travis-ci.com/huth/qemu/jobs/244117945

Second run where I'd expect a speedup:

  https://travis-ci.com/huth/qemu/jobs/244124599

The statistics at the end say that it had only 1 cache hit. Any ideas what might be wrong here?

Quoted 7 lines
Quoted 3 lines

Looking there and your following commit
(https://github.com/huth/qemu/commit/eaf80e7851) I see you already figured this out :)

No, that was just a try, but it did not change anything:

https://travis-ci.com/huth/qemu/jobs/244137697#L5813

Oh I checked the x86 build then...

https://travis-ci.com/huth/qemu/jobs/244137696#L11299

Quoted 10 lines

But I also noticed that in the arm64 builds, the cache information is missing:

https://travis-ci.com/huth/qemu/jobs/244137697#L1844

... so I assume that ccache support needs to be supported in the image that is provided by Travis, and you can not simply install it afterwards. So this is likely just a quirk that hopefully will be fixed by Travis later (arm64 is still marked as "alpha" there if I've got that right).

Ah, you might be correct.

If so, your job is now ready to use the feature once they enable it :)

#10 lore raw

PATCH 5/5 travis.yml: Compile on arm64, too

In reply to Thomas Huth

Travis now features an arm64 build host, so let's check compilation there, too.

Unfortunately, there are some quirks:
- block/ssh.c does not compile properly in this environment, so we have to use --disable-libssh until that problem is fixed.
- test-util-filemonitor fails, so we can not run the unit tests there
- The file system size seems to be very limited, so the iotest tests can't be used and the hd-geo-test fails (thus we can't compile x86_64-softmmu here and run "check-qtest" afterwards)
- Compiling seems to be quite a bit slower than on the x86 builders, so we better limit the target list to some few architectures.

---
 .travis.yml | 6 ++++++
 1 file changed, 6 insertions(+)
.travis.yml +6 -0
index 0c88e8757b..357ca02890 100644
@@ -342,3 +342,9 @@ matrix:
         - CONFIG="--target-list=xtensa-softmmu,arm-softmmu,aarch64-softmmu,alpha-softmmu"
         - TEST_CMD="make -j3 check-tcg V=1"
         - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
+
+    # Non-x86 builds:
+    - env:
+        - CONFIG="--disable-libssh --target-list=aarch64-softmmu,ppc64-softmmu,s390x-softmmu,x86_64-linux-user"
+        - TEST_CMD="make check-qtest check-softfloat -j3 V=1"
+      arch: arm64
Signature · 1 line
2.18.1
#11 lore raw

Re: [PATCH 5/5] travis.yml: Compile on arm64, too

Quoted 23 lines · Thomas Huth

Travis now features an arm64 build host, so let's check compilation there, too.

Unfortunately, there are some quirks:
- block/ssh.c does not compile properly in this environment, so we have to use --disable-libssh until that problem is fixed.
- test-util-filemonitor fails, so we can not run the unit tests there
- The file system size seems to be very limited, so the iotest tests can't be used and the hd-geo-test fails (thus we can't compile x86_64-softmmu here and run "check-qtest" afterwards)
- Compiling seems to be quite a bit slower than on the x86 builders, so we better limit the target list to some few architectures.

---
  .travis.yml | 6 ++++++
  1 file changed, 6 insertions(+)
.travis.yml +0 -0
index 0c88e8757b..357ca02890 100644

.travis.yml matrix: · line 348

Quoted 7 lines
.travis.yml +4 -0
@@ -342,3 +342,9 @@ matrix:
          - CONFIG="--target-list=xtensa-softmmu,arm-softmmu,aarch64-softmmu,alpha-softmmu"
          - TEST_CMD="make -j3 check-tcg V=1"
          - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
+
+    # Non-x86 builds:
+    - env:
+        - CONFIG="--disable-libssh --target-list=aarch64-softmmu,ppc64-softmmu,s390x-softmmu,x86_64-linux-user"

Is aarch64 on aarch64 interesting?

Do you know if we can use KVM there?

Quoted 3 lines
.travis.yml +2 -0
+        - TEST_CMD="make check-qtest check-softfloat -j3 V=1"
+      arch: arm64
#12 lore raw

Re: [PATCH 5/5] travis.yml: Compile on arm64, too

Quoted 38 lines · Philippe Mathieu-Daudé
Quoted 26 lines · Thomas Huth

Travis now features an arm64 build host, so let's check compilation there, too.

Unfortunately, there are some quirks:
- block/ssh.c does not compile properly in this environment, so we have to use --disable-libssh until that problem is fixed.
- test-util-filemonitor fails, so we can not run the unit tests there
- The file system size seems to be very limited, so the iotest tests can't be used and the hd-geo-test fails (thus we can't compile x86_64-softmmu here and run "check-qtest" afterwards)
- Compiling seems to be quite a bit slower than on the x86 builders, so we better limit the target list to some few architectures.

---
  .travis.yml | 6 ++++++

  1 file changed, 6 insertions(+)
.travis.yml +0 -0
index 0c88e8757b..357ca02890 100644
@@ -342,3 +342,9 @@ matrix:

          - CONFIG="--target-list=xtensa-softmmu,arm-softmmu,aarch64-softmmu,alpha-softmmu"

Quoted 8 lines

          - TEST_CMD="make -j3 check-tcg V=1"
          - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
+
+    # Non-x86 builds:
+    - env:
+        - CONFIG="--disable-libssh --target-list=aarch64-softmmu,ppc64-softmmu,s390x-softmmu,x86_64-linux-user"

Is aarch64 on aarch64 interesting?

Quoted 1 line

Do you know if we can use KVM there?

I don't think that KVM is usable there, but at least this should give us some compile coverage for target/arm/kvm.c which we don't get otherwise.

Thomas

#13 lore raw

Re: [PATCH 5/5] travis.yml: Compile on arm64, too

Quoted 39 lines · Thomas Huth
Quoted 38 lines · Philippe Mathieu-Daudé
Quoted 29 lines · Thomas Huth

Travis now features an arm64 build host, so let's check compilation there, too.

Unfortunately, there are some quirks:
- block/ssh.c does not compile properly in this environment, so we have to use --disable-libssh until that problem is fixed.
- test-util-filemonitor fails, so we can not run the unit tests there
- The file system size seems to be very limited, so the iotest tests can't be used and the hd-geo-test fails (thus we can't compile x86_64-softmmu here and run "check-qtest" afterwards)
- Compiling seems to be quite a bit slower than on the x86 builders, so we better limit the target list to some few architectures.

---
  .travis.yml | 6 ++++++
  1 file changed, 6 insertions(+)
.travis.yml +0 -0
index 0c88e8757b..357ca02890 100644
@@ -342,3 +342,9 @@ matrix:
          -

CONFIG="--target-list=xtensa-softmmu,arm-softmmu,aarch64-softmmu,alpha-softmmu"

- TEST_CMD="make -j3 check-tcg V=1"
- CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"

.travis.yml matrix:

Quoted 5 lines
.travis.yml +4 -1
+
+    # Non-x86 builds:
+    - env:
+        - CONFIG="--disable-libssh
--target-list=aarch64-softmmu,ppc64-softmmu,s390x-softmmu,x86_64-linux-user"

Is aarch64 on aarch64 interesting?

Quoted 5 lines
Quoted 1 line

Do you know if we can use KVM there?

I don't think that KVM is usable there, but at least this should give us some compile coverage for target/arm/kvm.c which we don't get otherwise.

We get compile coverage for !x86 kvm from the cross compilers on shippable. I'm more interested in running tests on !x86.

Quoted 2 lines

Thomas

--
Alex Bennée

#14 lore raw

Re: [PATCH 5/5] travis.yml: Compile on arm64, too

In reply to Thomas Huth
Quoted 7 lines · Thomas Huth

Travis now features an arm64 build host, so let's check compilation there, too.

Unfortunately, there are some quirks:
- block/ssh.c does not compile properly in this environment, so we have to use --disable-libssh until that problem is fixed.
- test-util-filemonitor fails, so we can not run the unit tests there

Do you have any more info on this failure ?

Running tests with FILEMONITOR_DEBUG=1 env variable set will make it print more info

Skipping unit tests seems to throw away the main benefit of using the travis arm64 hosts, as it is the tests which add value over our cross-compiled arm builds.

Quoted 29 lines

- The file system size seems to be very limited, so the iotest tests can't be used and the hd-geo-test fails (thus we can't compile x86_64-softmmu here and run "check-qtest" afterwards)
- Compiling seems to be quite a bit slower than on the x86 builders, so we better limit the target list to some few architectures.

---
 .travis.yml | 6 ++++++
 1 file changed, 6 insertions(+)
.travis.yml +6 -0
index 0c88e8757b..357ca02890 100644
@@ -342,3 +342,9 @@ matrix:
         - CONFIG="--target-list=xtensa-softmmu,arm-softmmu,aarch64-softmmu,alpha-softmmu"
         - TEST_CMD="make -j3 check-tcg V=1"
         - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
+
+    # Non-x86 builds:
+    - env:
+        - CONFIG="--disable-libssh --target-list=aarch64-softmmu,ppc64-softmmu,s390x-softmmu,x86_64-linux-user"
+        - TEST_CMD="make check-qtest check-softfloat -j3 V=1"
+      arch: arm64
Signature · 1 line
2.18.1

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#15 lore raw

Re: [PATCH 5/5] travis.yml: Compile on arm64, too

Quoted 9 lines · Daniel P. Berrangé
Quoted 7 lines · Thomas Huth

Travis now features an arm64 build host, so let's check compilation there, too.

Unfortunately, there are some quirks:
- block/ssh.c does not compile properly in this environment, so we have to use --disable-libssh until that problem is fixed.
- test-util-filemonitor fails, so we can not run the unit tests there

Quoted 4 lines

Do you have any more info on this failure ?

Running tests with FILEMONITOR_DEBUG=1 env variable set will make it print more info

Here you go:

https://travis-ci.com/huth/qemu/jobs/244168910#L5775

Quoting:

"
Add watch /tmp/test-util-filemonitor-PH0B9Z (null)
Watch ID 100000000
Add watch /tmp/test-util-filemonitor-PH0B9Z one.txt Watch ID 100000001
Add watch /tmp/test-util-filemonitor-PH0B9Z two.txt Watch ID 100000002
Create /tmp/test-util-filemonitor-PH0B9Z/one.txt
Event id=100000000 event=0 file=one.txt
Queue event id 100000000 event 0 file one.txt
Queue event id 100000001 event 0 file one.txt
Event id=100000001 event=0 file=one.txt
Create /tmp/test-util-filemonitor-PH0B9Z/two.txt
Queue event id 100000000 event 0 file two.txt
Queue event id 100000002 event 0 file two.txt
Event id=100000000 event=0 file=two.txt
Event id=100000002 event=0 file=two.txt
Create /tmp/test-util-filemonitor-PH0B9Z/three.txt
Queue event id 100000000 event 0 file three.txt
Event id=100000000 event=0 file=three.txt
Unlink /tmp/test-util-filemonitor-PH0B9Z/three.txt
Queue event id 100000000 event 2 file three.txt
Event id=100000000 event=2 file=three.txt
Rename /tmp/test-util-filemonitor-PH0B9Z/one.txt -> /tmp/test-util-filemonitor-PH0B9Z/two.txt
Queue event id 100000000 event 2 file one.txt
Queue event id 100000001 event 2 file one.txt
Queue event id 100000000 event 0 file two.txt
Queue event id 100000002 event 0 file two.txt
Event id=100000000 event=2 file=one.txt
Event id=100000001 event=2 file=one.txt
Event id=100000000 event=0 file=two.txt
Event id=100000002 event=0 file=two.txt
Append /tmp/test-util-filemonitor-PH0B9Z/two.txt
Queue event id 100000000 event 1 file two.txt
Queue event id 100000002 event 1 file two.txt
Event id=100000000 event=1 file=two.txt
Event id=100000002 event=1 file=two.txt
Touch /tmp/test-util-filemonitor-PH0B9Z/two.txt
Event id=100000000 event=3 file=two.txt
Queue event id 100000000 event 3 file two.txt
Queue event id 100000002 event 3 file two.txt
Event id=100000002 event=3 file=two.txt
Del watch /tmp/test-util-filemonitor-PH0B9Z 100000001 Add watch /tmp/test-util-filemonitor-PH0B9Z one.txt Watch ID 100000003
Create /tmp/test-util-filemonitor-PH0B9Z/one.txt
Queue event id 100000000 event 0 file one.txt
Event id=100000000 event=0 file=one.txt
Queue event id 100000003 event 0 file one.txt
Event id=100000003 event=0 file=one.txt
Del watch /tmp/test-util-filemonitor-PH0B9Z 100000003 Unlink /tmp/test-util-filemonitor-PH0B9Z/one.txt
Queue event id 100000000 event 2 file one.txt
Event id=100000000 event=2 file=one.txt
Mkdir /tmp/test-util-filemonitor-PH0B9Z/fish
Queue event id 100000000 event 0 file fish
Event id=100000000 event=0 file=fish
Add watch /tmp/test-util-filemonitor-PH0B9Z fish/
Watch ID 200000000
Add watch /tmp/test-util-filemonitor-PH0B9Z fish/one.txt Watch ID 200000001
Create /tmp/test-util-filemonitor-PH0B9Z/fish/one.txt Queue event id 200000000 event 0 file one.txt
Queue event id 200000001 event 0 file one.txt
Event id=200000000 event=0 file=one.txt
Event id=200000001 event=0 file=one.txt
Del watch /tmp/test-util-filemonitor-PH0B9Z 200000001 Rename /tmp/test-util-filemonitor-PH0B9Z/fish/one.txt -> /tmp/test-util-filemonitor-PH0B9Z/two.txt
Queue event id 200000000 event 2 file one.txt
Event id=200000000 event=2 file=one.txt
Queue event id 100000000 event 0 file two.txt
Event id=100000000 event=0 file=two.txt
Event id=100000002 event=0 file=two.txt
Queue event id 100000002 event 0 file two.txt
Rmdir /tmp/test-util-filemonitor-PH0B9Z/fish
Queue event id 100000000 event 2 file fish
Event id=200000000 event=4 file=
Missing event watch id 200000000 event 4 file
** ERROR:tests/test-util-filemonitor.c:690:test_file_monitor_events:
assertion failed: (err == 0)
ERROR - Bail out! ERROR:tests/test-util-filemonitor.c:690:test_file_monitor_events:
assertion failed: (err == 0)
Aborted /home/travis/build/huth/qemu/tests/Makefile.include:904: recipe for target 'check-unit' failed
make: *** [check-unit] Error 1
"

Thomas

#16 lore raw

Re: [PATCH 5/5] travis.yml: Compile on arm64, too

Quoted 94 lines · Thomas Huth
Quoted 9 lines · Daniel P. Berrangé
Quoted 7 lines · Thomas Huth

Travis now features an arm64 build host, so let's check compilation there, too.

Unfortunately, there are some quirks:
- block/ssh.c does not compile properly in this environment, so we have to use --disable-libssh until that problem is fixed.
- test-util-filemonitor fails, so we can not run the unit tests there

Quoted 4 lines

Do you have any more info on this failure ?

Running tests with FILEMONITOR_DEBUG=1 env variable set will make it print more info

Here you go:

https://travis-ci.com/huth/qemu/jobs/244168910#L5775

Quoting:

"
Add watch /tmp/test-util-filemonitor-PH0B9Z (null) Watch ID 100000000
Add watch /tmp/test-util-filemonitor-PH0B9Z one.txt Watch ID 100000001
Add watch /tmp/test-util-filemonitor-PH0B9Z two.txt Watch ID 100000002
Create /tmp/test-util-filemonitor-PH0B9Z/one.txt Event id=100000000 event=0 file=one.txt
Queue event id 100000000 event 0 file one.txt Queue event id 100000001 event 0 file one.txt Event id=100000001 event=0 file=one.txt
Create /tmp/test-util-filemonitor-PH0B9Z/two.txt Queue event id 100000000 event 0 file two.txt Queue event id 100000002 event 0 file two.txt Event id=100000000 event=0 file=two.txt
Event id=100000002 event=0 file=two.txt
Create /tmp/test-util-filemonitor-PH0B9Z/three.txt Queue event id 100000000 event 0 file three.txt Event id=100000000 event=0 file=three.txt Unlink /tmp/test-util-filemonitor-PH0B9Z/three.txt Queue event id 100000000 event 2 file three.txt Event id=100000000 event=2 file=three.txt Rename /tmp/test-util-filemonitor-PH0B9Z/one.txt -> /tmp/test-util-filemonitor-PH0B9Z/two.txt
Queue event id 100000000 event 2 file one.txt Queue event id 100000001 event 2 file one.txt Queue event id 100000000 event 0 file two.txt Queue event id 100000002 event 0 file two.txt Event id=100000000 event=2 file=one.txt
Event id=100000001 event=2 file=one.txt
Event id=100000000 event=0 file=two.txt
Event id=100000002 event=0 file=two.txt
Append /tmp/test-util-filemonitor-PH0B9Z/two.txt Queue event id 100000000 event 1 file two.txt Queue event id 100000002 event 1 file two.txt Event id=100000000 event=1 file=two.txt
Event id=100000002 event=1 file=two.txt
Touch /tmp/test-util-filemonitor-PH0B9Z/two.txt Event id=100000000 event=3 file=two.txt
Queue event id 100000000 event 3 file two.txt Queue event id 100000002 event 3 file two.txt Event id=100000002 event=3 file=two.txt
Del watch /tmp/test-util-filemonitor-PH0B9Z 100000001 Add watch /tmp/test-util-filemonitor-PH0B9Z one.txt Watch ID 100000003
Create /tmp/test-util-filemonitor-PH0B9Z/one.txt Queue event id 100000000 event 0 file one.txt Event id=100000000 event=0 file=one.txt
Queue event id 100000003 event 0 file one.txt Event id=100000003 event=0 file=one.txt
Del watch /tmp/test-util-filemonitor-PH0B9Z 100000003 Unlink /tmp/test-util-filemonitor-PH0B9Z/one.txt Queue event id 100000000 event 2 file one.txt Event id=100000000 event=2 file=one.txt
Mkdir /tmp/test-util-filemonitor-PH0B9Z/fish Queue event id 100000000 event 0 file fish Event id=100000000 event=0 file=fish
Add watch /tmp/test-util-filemonitor-PH0B9Z fish/ Watch ID 200000000
Add watch /tmp/test-util-filemonitor-PH0B9Z fish/one.txt Watch ID 200000001
Create /tmp/test-util-filemonitor-PH0B9Z/fish/one.txt Queue event id 200000000 event 0 file one.txt Queue event id 200000001 event 0 file one.txt Event id=200000000 event=0 file=one.txt
Event id=200000001 event=0 file=one.txt
Del watch /tmp/test-util-filemonitor-PH0B9Z 200000001 Rename /tmp/test-util-filemonitor-PH0B9Z/fish/one.txt -> /tmp/test-util-filemonitor-PH0B9Z/two.txt
Queue event id 200000000 event 2 file one.txt Event id=200000000 event=2 file=one.txt
Queue event id 100000000 event 0 file two.txt

Quoted 8 lines

Event id=100000000 event=0 file=two.txt Event id=100000002 event=0 file=two.txt Queue event id 100000002 event 0 file two.txt Rmdir /tmp/test-util-filemonitor-PH0B9Z/fish Queue event id 100000000 event 2 file fish Event id=200000000 event=4 file= Missing event watch id 200000000 event 4 file **

This is really strange behaviour. I've done some further test builds on travis with various extra debug info & AFAICT, inotify is not working correctly on the host - it is never emitting one of the events that it should. Could be some wierd bug related to the filesystem overlay setup used for the container env in travis I guess.

I'm not sure what todo here. I can't see an obvious trait that I can use to identify a broken setup, in order to auto-skip the test. Might just need to build without the inotify support somehow.

Regards,
Daniel

Signature · 3 lines
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
#17 lore raw

Re: [PATCH 5/5] travis.yml: Compile on arm64, too

In reply to Thomas Huth
Quoted 23 lines · Thomas Huth

Travis now features an arm64 build host, so let's check compilation there, too.

Unfortunately, there are some quirks:
- block/ssh.c does not compile properly in this environment, so we have to use --disable-libssh until that problem is fixed.
- test-util-filemonitor fails, so we can not run the unit tests there
- The file system size seems to be very limited, so the iotest tests can't be used and the hd-geo-test fails (thus we can't compile x86_64-softmmu here and run "check-qtest" afterwards)
- Compiling seems to be quite a bit slower than on the x86 builders, so we better limit the target list to some few architectures.

---
 .travis.yml | 6 ++++++
 1 file changed, 6 insertions(+)
.travis.yml +0 -0
index 0c88e8757b..357ca02890 100644

.travis.yml matrix: · line 350

Quoted 9 lines
.travis.yml +6 -0
@@ -342,3 +342,9 @@ matrix:
         - CONFIG="--target-list=xtensa-softmmu,arm-softmmu,aarch64-softmmu,alpha-softmmu"
         - TEST_CMD="make -j3 check-tcg V=1"
         - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
+
+    # Non-x86 builds:
+    - env:
+        - CONFIG="--disable-libssh --target-list=aarch64-softmmu,ppc64-softmmu,s390x-softmmu,x86_64-linux-user"
+        - TEST_CMD="make check-qtest check-softfloat -j3 V=1"
+      arch: arm64

Hmm this keeps fail for me now but it looks like missing deps:

Reading state information...

E: Unable to locate package libspice-server-dev
apt-get.diagnostics

apt-get install failed

The command "sudo -E apt-get -yq --no-install-suggests --no-install-recommends $(travis_apt_get_options) install libaio-dev libattr1-dev libbrlapi-dev libcap-dev libcap-ng-dev libgcc-4.8-dev libgnutls28-dev libgtk-3-dev libiscsi-dev liblttng-ust-dev libncurses5-dev libnfs-dev libnss3-dev libpixman-1-dev libpng-dev librados-dev libsdl2-dev libsdl2-image-dev libseccomp-dev libspice-protocol-dev libspice-server-dev libssh-dev liburcu-dev libusb-1.0-0-dev libvdeplug-dev libvte-2.91-dev sparse uuid-dev gcovr" failed and exited with 100 during .

--
Alex Bennée

#18 lore raw

Re: [PATCH 5/5] travis.yml: Compile on arm64, too

Quoted 39 lines · Alex Bennée
Quoted 23 lines · Thomas Huth

Travis now features an arm64 build host, so let's check compilation there, too.

Unfortunately, there are some quirks:
- block/ssh.c does not compile properly in this environment, so we have to use --disable-libssh until that problem is fixed.
- test-util-filemonitor fails, so we can not run the unit tests there
- The file system size seems to be very limited, so the iotest tests can't be used and the hd-geo-test fails (thus we can't compile x86_64-softmmu here and run "check-qtest" afterwards)
- Compiling seems to be quite a bit slower than on the x86 builders, so we better limit the target list to some few architectures.

---
 .travis.yml | 6 ++++++
 1 file changed, 6 insertions(+)
.travis.yml +0 -0
index 0c88e8757b..357ca02890 100644

.travis.yml matrix: · line 350

Quoted 9 lines
.travis.yml +6 -0
@@ -342,3 +342,9 @@ matrix:
         - CONFIG="--target-list=xtensa-softmmu,arm-softmmu,aarch64-softmmu,alpha-softmmu"
         - TEST_CMD="make -j3 check-tcg V=1"
         - CACHE_NAME="${TRAVIS_BRANCH}-linux-gcc-default"
+
+    # Non-x86 builds:
+    - env:
+        - CONFIG="--disable-libssh --target-list=aarch64-softmmu,ppc64-softmmu,s390x-softmmu,x86_64-linux-user"
+        - TEST_CMD="make check-qtest check-softfloat -j3 V=1"
+      arch: arm64

Hmm this keeps fail for me now but it looks like missing deps:

Quoted 3 lines

Reading state information...

E: Unable to locate package libspice-server-dev

It used to work two weeks before:

https://travis-ci.com/huth/qemu/jobs/244124599#L484

I guess there was an update to a different version and they removed the package from arm64 now...?

Seems like this arm64 builder is still a moving target (it has been announced as alpha software IIRC), so I think we should not include this patch yet.

Thomas

#19 lore raw
In reply to Thomas Huth
Quoted 2 lines · Thomas Huth

Update / add some libraries that we use in the Travis builds, and enable compiling on the aarch64 host, too.

Queued 1-3 to testing/next and I'll have a play with 4/5 but probably won't include them in the final PR unless they seem solid.

Quoted 11 lines

Thomas Huth (5):
travis.yml: Add libvdeplug-dev to compile-test net/vde.c travis.yml: Use libsdl2 instead of libsdl1.2, and install
libsdl2-image
travis.yml: Use newer version of libgnutls and libpng travis.yml: Fix the ccache lines
travis.yml: Compile on arm64, too

 .travis.yml | 21 +++++++++++++++------
 1 file changed, 15 insertions(+), 6 deletions(-)

--
Alex Bennée

#20 lore raw
Quoted 8 lines · Alex Bennée
Quoted 2 lines · Thomas Huth

Update / add some libraries that we use in the Travis builds, and enable compiling on the aarch64 host, too.

Queued 1-3 to testing/next and I'll have a play with 4/5 but probably won't include them in the final PR unless they seem solid.

Thanks! I think you could also pick patch 4 since the current line with "command -v ccache && ccache ..." is definitely nonsense (it currently just works since we always compile with ccache). We should either apply my patch 4, or if we say that we can not run without ccache, we should remove the "command -v ccache &&" part instead.

Thomas

#21 lore raw
Quoted 7 lines · Thomas Huth
Quoted 6 lines · Alex Bennée
Quoted 2 lines · Thomas Huth

Update / add some libraries that we use in the Travis builds, and enable compiling on the aarch64 host, too.

Quoted 8 lines
Quoted 2 lines

Queued 1-3 to testing/next and I'll have a play with 4/5 but probably won't include them in the final PR unless they seem solid.

Thanks! I think you could also pick patch 4 since the current line with "command -v ccache && ccache ..." is definitely nonsense (it currently just works since we always compile with ccache). We should either apply my patch 4, or if we say that we can not run without ccache, we should remove the "command -v ccache &&" part instead.

Yes, pick also #4 please :)