Skip to contentSkip to conversation

Run tcg tests with tci on Travis

PATCH Superseded Thomas Huth 5 participants 16 messages Nov 28, 2019 – Dec 4, 2019

lore
Share & tools
lore
#0 lore raw

v2 cover Run tcg tests with tci on Travis

It's now possible to run some TCG-based tests with our Tiny Code Generator Interpreter (TCI), too. These two patches enable the testing on Travis.

Alex Bennée (1):
configure: allow disable of cross compilation containers

Thomas Huth (1):
travis.yml: Run tcg tests with tci

 .travis.yml            | 7 ++++---
 configure              | 8 +++++++-
 tests/tcg/configure.sh | 6 ++++--
 3 files changed, 15 insertions(+), 6 deletions(-)
Signature · 1 line
2.18.1
#1 lore raw

PATCH v2 1/2 configure: allow disable of cross compilation containers

From: Alex Bennée <[email protected]>

Our docker infrastructure isn't quite as multiarch as we would wish so lets allow the user to disable it if they want. This will allow us to use still run check-tcg on non-x86 CI setups.

---
 configure              | 8 +++++++-
 tests/tcg/configure.sh | 6 ++++--
 2 files changed, 11 insertions(+), 3 deletions(-)
configure +7 -1
index 6099be1d84..fe6d0971f1 100755
@@ -302,6 +302,7 @@ audio_win_int=""
 libs_qga=""
 debug_info="yes"
 stack_protector=""
+use_containers="yes"
 
 if test -e "$source_path/.git"
 then
@@ -1539,6 +1540,10 @@ for opt do
   ;;
   --disable-plugins) plugins="no"
   ;;
+  --enable-containers) use_containers="yes"
+  ;;
+  --disable-containers) use_containers="no"
+  ;;
   *)
       echo "ERROR: unknown option $opt"
       echo "Try '$0 --help' for more information"
@@ -1722,6 +1727,7 @@ Advanced options (experts only):
                            track the maximum stack usage of stacks created by qemu_alloc_stack
   --enable-plugins
                            enable plugins via shared library loading
+  --disable-containers     don't use containers for cross-building
 
 Optional features, enabled with --enable-FEATURE and
 disabled with --disable-FEATURE, default is enabled if available:
@@ -8039,7 +8045,7 @@ done
 (for i in $cross_cc_vars; do
   export $i
 done
-export target_list source_path
+export target_list source_path use_containers
 $source_path/tests/tcg/configure.sh)
 
 # temporary config to build submodules
tests/tcg/configure.sh +4 -2
index 6c4a471aea..210e68396f 100755
@@ -36,8 +36,10 @@ TMPC="${TMPDIR1}/qemu-conf.c"
 TMPE="${TMPDIR1}/qemu-conf.exe"
 
 container="no"
-if has "docker" || has "podman"; then
-  container=$($python $source_path/tests/docker/docker.py probe)
+if test $use_containers = "yes"; then
+    if has "docker" || has "podman"; then
+        container=$($python $source_path/tests/docker/docker.py probe)
+    fi
 fi
 
 # cross compilers defaults, can be overridden with --cross-cc-ARCH
Signature · 1 line
2.18.1
#3 lore raw

PATCH v2 2/2 travis.yml: Run tcg tests with tci

In reply to Thomas Huth

So far we only have compile coverage for tci. But since commit 2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation for INDEX_op_ld16u_i64") has been included now, we can also run the "tcg" and "qtest" tests with tci, so let's enable them in Travis now. Since we don't gain much additional test coverage by compiling all targets, and TCI is broken e.g. with the Sparc targets, we also limit the target list to a reasonable subset now (which should still get us test coverage by tests/boot-serial-test for example).

---
 .travis.yml | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
.travis.yml +4 -3
index c09b6a0014..de7559e777 100644
@@ -215,10 +215,11 @@ matrix:
         - TEST_CMD=""
 
 
-    # We manually include builds which we disable "make check" for
+    # Check the TCG interpreter (TCI)
     - env:
-        - CONFIG="--enable-debug --enable-tcg-interpreter"
-        - TEST_CMD=""
+        - CONFIG="--enable-debug --enable-tcg-interpreter --disable-containers
+            --target-list=alpha-softmmu,arm-softmmu,hppa-softmmu,m68k-softmmu,microblaze-softmmu,moxie-softmmu,ppc-softmmu,s390x-softmmu,x86_64-softmmu"
+        - TEST_CMD="make check-qtest check-tcg V=1"
 
 
     # We don't need to exercise every backend with every front-end
Signature · 1 line
2.18.1
Reviewed-by on v2 1/2
#2 lore raw

Re: [PATCH v2 1/2] configure: allow disable of cross compilation containers

In reply to Thomas Huth

Am 28.11.19 um 16:35 schrieb Thomas Huth:

Quoted 4 lines

From: Alex Bennée <[email protected]>

Our docker infrastructure isn't quite as multiarch as we would wish so lets allow the user to disable it if they want. This will allow us to

s/lets/let's/ ?

Otherwise fine, thank you.

Tested-by on v2 2/2
#4 lore raw

Re: [PATCH v2 2/2] travis.yml: Run tcg tests with tci

In reply to Thomas Huth

Am 28.11.19 um 16:35 schrieb Thomas Huth:

Quoted 6 lines

So far we only have compile coverage for tci. But since commit 2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation for INDEX_op_ld16u_i64") has been included now, we can also run the "tcg" and "qtest" tests with tci, so let's enable them in Travis now. Since we don't gain much additional test coverage by compiling all targets, and TCI is broken e.g. with the Sparc targets, we also limit

As far as I know it is broken with Sparc hosts (not Sparc targets).

I tested without limiting the target list on an x86_64 host, and the tests passed.

Quoted 13 lines

the target list to a reasonable subset now (which should still get us test coverage by tests/boot-serial-test for example).

---
 .travis.yml | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
.travis.yml +0 -0
index c09b6a0014..de7559e777 100644

.travis.yml matrix: · line 220

Quoted 9 lines
.travis.yml +2 -3
@@ -215,10 +215,11 @@ matrix:
         - TEST_CMD=""
 
 
-    # We manually include builds which we disable "make check" for
+    # Check the TCG interpreter (TCI)
     - env:
-        - CONFIG="--enable-debug --enable-tcg-interpreter"
-        - TEST_CMD=""
+        - CONFIG="--enable-debug --enable-tcg-interpreter --disable-containers

You could also --disable-kvm. It should not be needed, and disabling it might need less build resources.

.travis.yml matrix: · line 225

Quoted 5 lines
.travis.yml +2 -0
+            --target-list=alpha-softmmu,arm-softmmu,hppa-softmmu,m68k-softmmu,microblaze-softmmu,moxie-softmmu,ppc-softmmu,s390x-softmmu,x86_64-softmmu"
+        - TEST_CMD="make check-qtest check-tcg V=1"
 
 
     # We don't need to exercise every backend with every front-end

Thank you for adding these tests.

Reviewed-by on the v2 series
#15 lore raw
In reply to Thomas Huth
Quoted 7 lines · Thomas Huth

It's now possible to run some TCG-based tests with our Tiny Code Generator Interpreter (TCI), too. These two patches enable the testing on Travis.

Alex Bennée (1):
configure: allow disable of cross compilation containers

Quoted 7 lines

Thomas Huth (1):
travis.yml: Run tcg tests with tci

  .travis.yml            | 7 ++++---
  configure              | 8 +++++++-
  tests/tcg/configure.sh | 6 ++++--
  3 files changed, 15 insertions(+), 6 deletions(-)

Good idea to add/use '--disable-containers'.

#5 lore raw

Re: [PATCH v2 2/2] travis.yml: Run tcg tests with tci

In reply to Stefan Weil

Am 28.11.19 um 22:06 schrieb Stefan Weil:

Quoted 9 lines

Am 28.11.19 um 16:35 schrieb Thomas Huth:

Quoted 6 lines

So far we only have compile coverage for tci. But since commit 2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation for INDEX_op_ld16u_i64") has been included now, we can also run the "tcg" and "qtest" tests with tci, so let's enable them in Travis now. Since we don't gain much additional test coverage by compiling all targets, and TCI is broken e.g. with the Sparc targets, we also limit

Quoted 4 lines

As far as I know it is broken with Sparc hosts (not Sparc targets).

I tested without limiting the target list on an x86_64 host, and the tests passed.

Sorry, I have to correct myself: check-qtest-sparc64 fails. I'll examine that.

Stefan

#7 lore raw

Re: [PATCH v2 2/2] travis.yml: Run tcg tests with tci

In reply to Stefan Weil
Quoted 11 lines · Stefan Weil

Am 28.11.19 um 16:35 schrieb Thomas Huth:

Quoted 6 lines

So far we only have compile coverage for tci. But since commit 2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation for INDEX_op_ld16u_i64") has been included now, we can also run the "tcg" and "qtest" tests with tci, so let's enable them in Travis now. Since we don't gain much additional test coverage by compiling all targets, and TCI is broken e.g. with the Sparc targets, we also limit

As far as I know it is broken with Sparc hosts (not Sparc targets).

It was definitely hanging with sparc64-softmmu:

https://travis-ci.com/huth/qemu/jobs/261335163

... but since you've mentioned that it should work with the 32-bit sparc-softmmu, I'll give it another try with sparc-softmmu.

Quoted 16 lines
Quoted 16 lines

the target list to a reasonable subset now (which should still get us test coverage by tests/boot-serial-test for example).

---
 .travis.yml | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
.travis.yml +0 -0
index c09b6a0014..de7559e777 100644
@@ -215,10 +215,11 @@ matrix:
         - TEST_CMD=""
 
 
Quoted 10 lines

.travis.yml matrix: · line 220

Quoted 6 lines
.travis.yml +2 -3
-    # We manually include builds which we disable "make check" for
+    # Check the TCG interpreter (TCI)
     - env:
-        - CONFIG="--enable-debug --enable-tcg-interpreter"
-        - TEST_CMD=""
+        - CONFIG="--enable-debug --enable-tcg-interpreter --disable-containers

You could also --disable-kvm. It should not be needed, and disabling it might need less build resources.

Good idea. KVM is not usable by default in Travis, so we should not accidentally use it for the tests that specify "accel=kvm:tcg", but in case that changes with a future version of the environment, we should maybe be prepared.

Quoted 10 lines

.travis.yml matrix: · line 225

Quoted 5 lines
.travis.yml +2 -0
+            --target-list=alpha-softmmu,arm-softmmu,hppa-softmmu,m68k-softmmu,microblaze-softmmu,moxie-softmmu,ppc-softmmu,s390x-softmmu,x86_64-softmmu"
+        - TEST_CMD="make check-qtest check-tcg V=1"
 
 
     # We don't need to exercise every backend with every front-end

Thank you for adding these tests.

Thanks,
Thomas

#8 lore raw

Re: [PATCH v2 2/2] travis.yml: Run tcg tests with tci

Quoted 47 lines · Thomas Huth
Quoted 11 lines · Stefan Weil

Am 28.11.19 um 16:35 schrieb Thomas Huth:

Quoted 6 lines

So far we only have compile coverage for tci. But since commit 2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation for INDEX_op_ld16u_i64") has been included now, we can also run the "tcg" and "qtest" tests with tci, so let's enable them in Travis now. Since we don't gain much additional test coverage by compiling all targets, and TCI is broken e.g. with the Sparc targets, we also limit

As far as I know it is broken with Sparc hosts (not Sparc targets).

It was definitely hanging with sparc64-softmmu:

https://travis-ci.com/huth/qemu/jobs/261335163

... but since you've mentioned that it should work with the 32-bit sparc-softmmu, I'll give it another try with sparc-softmmu.

Quoted 16 lines
Quoted 16 lines

the target list to a reasonable subset now (which should still get us test coverage by tests/boot-serial-test for example).

---
 .travis.yml | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
.travis.yml +0 -0
index c09b6a0014..de7559e777 100644
@@ -215,10 +215,11 @@ matrix:
         - TEST_CMD=""
 
 
Quoted 10 lines

.travis.yml matrix: · line 220

Quoted 6 lines
.travis.yml +2 -3
-    # We manually include builds which we disable "make check" for
+    # Check the TCG interpreter (TCI)
     - env:
-        - CONFIG="--enable-debug --enable-tcg-interpreter"
-        - TEST_CMD=""
+        - CONFIG="--enable-debug --enable-tcg-interpreter --disable-containers

You could also --disable-kvm. It should not be needed, and disabling it might need less build resources.

Quoted 4 lines

Good idea. KVM is not usable by default in Travis, so we should not accidentally use it for the tests that specify "accel=kvm:tcg", but in case that changes with a future version of the environment, we should maybe be prepared.

Makes sense, I'll wait for v3 before applying to me tree.

Quoted 14 lines
Quoted 11 lines

.travis.yml matrix: · line 225

Quoted 6 lines
.travis.yml +2 -0
+            --target-list=alpha-softmmu,arm-softmmu,hppa-softmmu,m68k-softmmu,microblaze-softmmu,moxie-softmmu,ppc-softmmu,s390x-softmmu,x86_64-softmmu"
+        - TEST_CMD="make check-qtest check-tcg V=1"
 
 
     # We don't need to exercise every backend with every front-end

Thank you for adding these tests.

Thanks,
Thomas

Signature · 1 line
Alex Bennée
#6 lore raw

Re: [PATCH v2 2/2] travis.yml: Run tcg tests with tci

In reply to Stefan Weil
Quoted 11 lines · Stefan Weil

Am 28.11.19 um 22:06 schrieb Stefan Weil:

Quoted 9 lines

Am 28.11.19 um 16:35 schrieb Thomas Huth:

Quoted 6 lines

So far we only have compile coverage for tci. But since commit 2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation for INDEX_op_ld16u_i64") has been included now, we can also run the "tcg" and "qtest" tests with tci, so let's enable them in Travis now. Since we don't gain much additional test coverage by compiling all targets, and TCI is broken e.g. with the Sparc targets, we also limit

Quoted 8 lines
Quoted 4 lines

As far as I know it is broken with Sparc hosts (not Sparc targets).

I tested without limiting the target list on an x86_64 host, and the tests passed.

Sorry, I have to correct myself: check-qtest-sparc64 fails. I'll examine that.

The 32-bit sparc target fails, too - when doing the prom-env-test:

https://travis-ci.com/huth/qemu/jobs/262572937#L7690

Thomas

#9 lore raw

PATCH v3 1/1 travis.yml: Run tcg tests with tci

In reply to Alex Bennée

So far we only have compile coverage for tci. But since commit 2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation for INDEX_op_ld16u_i64") has been included now, we can also run the "tcg" and "qtest" tests with tci, so let's enable them in Travis now. Since we don't gain much additional test coverage by compiling all targets, and TCI is broken e.g. with the Sparc targets, we also limit the target list to a reasonable subset now (which should still get us test coverage by tests/boot-serial-test for example).

---
v3:
- Add --disable-kvm option since we're only interested in TCG here

 .travis.yml | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
.travis.yml +4 -3
index 445b0646c1..d73e2fb744 100644
@@ -215,10 +215,11 @@ matrix:
         - TEST_CMD=""
 
 
-    # We manually include builds which we disable "make check" for
+    # Check the TCG interpreter (TCI)
     - env:
-        - CONFIG="--enable-debug --enable-tcg-interpreter"
-        - TEST_CMD=""
+        - CONFIG="--enable-debug --enable-tcg-interpreter --disable-kvm --disable-containers
+            --target-list=alpha-softmmu,arm-softmmu,hppa-softmmu,m68k-softmmu,microblaze-softmmu,moxie-softmmu,ppc-softmmu,s390x-softmmu,x86_64-softmmu"
+        - TEST_CMD="make check-qtest check-tcg V=1"
 
 
     # We don't need to exercise every backend with every front-end
Signature · 1 line
2.18.1
#10 lore raw

Re: [PATCH v3] travis.yml: Run tcg tests with tci

.travis.yml matrix:

Quoted 6 lines · Thomas Huth
.travis.yml +2 -3
-    # We manually include builds which we disable "make check" for
+    # Check the TCG interpreter (TCI)
     - env:
-        - CONFIG="--enable-debug --enable-tcg-interpreter"
-        - TEST_CMD=""
+        - CONFIG="--enable-debug --enable-tcg-interpreter --disable-kvm

While we're changing things, the interpreter will go much faster with optimization enabled. We can change this to --enable-debug-tcg, which leaves the asserts enabled, but compiles with -O2.

r~

#12 lore raw

Re: [PATCH v3] travis.yml: Run tcg tests with tci

In reply to Thomas Huth
Quoted 8 lines · Thomas Huth

So far we only have compile coverage for tci. But since commit 2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation for INDEX_op_ld16u_i64") has been included now, we can also run the "tcg" and "qtest" tests with tci, so let's enable them in Travis now. Since we don't gain much additional test coverage by compiling all targets, and TCI is broken e.g. with the Sparc targets, we also limit the target list to a reasonable subset now (which should still get us test coverage by tests/boot-serial-test for example).

Queued to testing/next, thanks.

Quoted 30 lines

---
v3:
- Add --disable-kvm option since we're only interested in TCG here

 .travis.yml | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
.travis.yml +4 -3
index 445b0646c1..d73e2fb744 100644
@@ -215,10 +215,11 @@ matrix:
         - TEST_CMD=""
 
 
-    # We manually include builds which we disable "make check" for
+    # Check the TCG interpreter (TCI)
     - env:
-        - CONFIG="--enable-debug --enable-tcg-interpreter"
-        - TEST_CMD=""
+        - CONFIG="--enable-debug --enable-tcg-interpreter --disable-kvm --disable-containers
+            --target-list=alpha-softmmu,arm-softmmu,hppa-softmmu,m68k-softmmu,microblaze-softmmu,moxie-softmmu,ppc-softmmu,s390x-softmmu,x86_64-softmmu"
+        - TEST_CMD="make check-qtest check-tcg V=1"
 
 
     # We don't need to exercise every backend with every front-end
Signature · 1 line
Alex Bennée
#13 lore raw

Re: [PATCH v3] travis.yml: Run tcg tests with tci

Quoted 12 lines · Alex Bennée
Quoted 8 lines · Thomas Huth

So far we only have compile coverage for tci. But since commit 2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation for INDEX_op_ld16u_i64") has been included now, we can also run the "tcg" and "qtest" tests with tci, so let's enable them in Travis now. Since we don't gain much additional test coverage by compiling all targets, and TCI is broken e.g. with the Sparc targets, we also limit the target list to a reasonable subset now (which should still get us test coverage by tests/boot-serial-test for example).

Quoted 1 line

Queued to testing/next, thanks.

Thanks! But could you maybe s/--enable-debug/--enable-debug-tcg/ as Richard just suggested in his mail? Or want me to send a v4?

Thomas

#11 lore raw

Re: [PATCH v3] travis.yml: Run tcg tests with tci

In reply to Richard Henderson
Quoted 11 lines · Richard Henderson

.travis.yml matrix:

Quoted 6 lines · Thomas Huth
.travis.yml +2 -3
-    # We manually include builds which we disable "make check" for
+    # Check the TCG interpreter (TCI)
     - env:
-        - CONFIG="--enable-debug --enable-tcg-interpreter"
-        - TEST_CMD=""
+        - CONFIG="--enable-debug --enable-tcg-interpreter --disable-kvm

While we're changing things, the interpreter will go much faster with optimization enabled. We can change this to --enable-debug-tcg, which leaves the asserts enabled, but compiles with -O2.

I've amended the commit in my tree - no need to resend.

Quoted 3 lines

r~

Signature · 1 line
Alex Bennée
#14 lore raw

Re: [PATCH v3] travis.yml: Run tcg tests with tci

In reply to Thomas Huth
Quoted 13 lines · Thomas Huth
Quoted 12 lines · Alex Bennée
Quoted 8 lines · Thomas Huth

So far we only have compile coverage for tci. But since commit 2f160e0f9797c7522bfd0d09218d0c9340a5137c ("tci: Add implementation for INDEX_op_ld16u_i64") has been included now, we can also run the "tcg" and "qtest" tests with tci, so let's enable them in Travis now. Since we don't gain much additional test coverage by compiling all targets, and TCI is broken e.g. with the Sparc targets, we also limit the target list to a reasonable subset now (which should still get us test coverage by tests/boot-serial-test for example).

Quoted 4 lines
Quoted 1 line

Queued to testing/next, thanks.

Thanks! But could you maybe s/--enable-debug/--enable-debug-tcg/ as Richard just suggested in his mail? Or want me to send a v4?

I fixed it in my tree, no need to send v4

Quoted 2 lines

Thomas

Signature · 1 line
Alex Bennée