meson-with-tests: Allows for build time tests of meson

meson build system has the functionality of setting an executable wrapper to run cross-compiled binaries. The setting is called exe_wrapper and can be defined in the meson.cross file. With a valid exe_wrapper meson build system can test if the built binary is sane.

Another advantage of setting exe_wrapper setting is that we can run the tests defined in the meson build system. This way if this class is inherited not only are the default targets built, their tests will also run and have an opportunity to fail the the build at the recipe level if necessary.

The user of this class should not inherit meson directly but leave it to this class. This class may later be merged into meson.bbclass itself.

---
 meta/classes/meson-with-tests.bbclass | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)
 create mode 100644 meta/classes/meson-with-tests.bbclass

new file mode 100644
index 0000000000..ddaeebc726
@@ -0,0 +1,26 @@
+inherit meson
+
+DEPENDS_append += "qemu-native"
+
+do_write_config_append() {
+  #You need to pass qemu with the sysroot path
+  #arguments, but internally meson uses Popen
+  #so we could try lists of arguments
+  #While passing a list directly in the exe_wrapper
+  #parameter passes the sanity test, it later on
+  #fails in the actual test run with:
+  #TypeError: expect bytes or str, not list
+  #So we create an in here wrapper.
+  cat > ${WORKDIR}/meson_exe_wrapper.sh << EOF
+#!/bin/sh
+qemu-${TARGET_ARCH} -L ${RECIPE_SYSROOT} \$@
+EOF
+
+  chmod u+x ${WORKDIR}/meson_exe_wrapper.sh
+
+  sed "/binaries/a\exe_wrapper = '${WORKDIR}/meson_exe_wrapper.sh'" -i ${WORKDIR}/meson.cross
+}
+
+do_compile_append() {
+  ninja test
+}

2.14.1

qemu is notoriously fragile, or non-existent for some MACHINEs. If something fails, the failure should be reported as a warning.

Also, can this be enabled for anything in oe-core that currently utilizes plain meson? That way the class will actually get used and tested.

Alex

2018-06-19 22:20 GMT+02:00 Paulo Neves <[email protected]>:

By suggestion of Alexander I have tried to change recipes known to use meson My results are mixed.

With libinput the test was skipped automatically
With libepoxy there were no tests defined.
With json-glib I can run 13/14 tests successfully. One of the tests exits with a segmentation fault and I cannot know if it is qemu giving the ghost or a real code bug. Given that the 13 of 14 tests pass successfully I find that the risk of misconfiguration on my part is reduced. Because many heads think better than 1 or even 2 I leave here the result logs of the failing test: My next step is to build the text executable and run it on a target, to rule out qemu error.

There is an error that catches my attention: A child process outputs on stderr "/lib/ld-linux-armhf.so.3: No such file or directory\n". This implies that somehow a fork or execve inside qemu is escaping my exe_wrapper and running without -L sysroot option, but a strace revealed a correct qemu execve syscall. Also, if somehow there is an escape of qemu it is odd it only happens in a specific test. This failure is reproducible.

There is also a segmentation fault on qemu but I cannot know if it is qemu or the program. I think the segmentation fault is the test's fault because just before death there is an assertion fail that does not happen when i run the test built with my native toolchain.

Paulo Neves

ERROR: json-glib-1.4.2-r0 do_compile: Function failed: do_compile (log
file is located at /home/pneves/builds/tmp/work/cortexa15hf-neon-poky-linux-gnueabi/json-glib/1.4.2-r0/temp/log.do_compile.17219)
ERROR: Logfile of failure stored in:
/home/pneves/builds/tmp/work/cortexa15hf-neon-poky-linux-gnueabi/json-glib/1.4.2-r0/temp/log.do_compile.17219 Log data follows:

| DEBUG: Executing shell function do_compile
| ninja: no work to do.
| [0/1] Running all tests.
|  1/14 array                                   OK       0.07 s
|  2/14 boxed                                   OK       0.09 s
|  3/14 builder                                 OK       0.08 s
|  4/14 generator                               OK       0.14 s
|  5/14 gvariant                                OK       0.16 s
|  6/14 invalid                                 OK       0.11 s
|  7/14 node                                    FAIL     0.26 s
|  8/14 object                                  OK       0.11 s
|  9/14 parser                                  OK       0.17 s
| 10/14 path                                    OK       0.11 s
| 11/14 reader                                  OK       0.10 s
| 12/14 serialize-simple                        OK       0.08 s
| 13/14 serialize-complex                       OK       0.10 s
| 14/14 serialize-full                          OK       0.08 s
|
| OK:        13
| FAIL:       1
| SKIP:       0
| TIMEOUT:    0
|
|
| The output from the failed tests:
|
|  7/14 node                                    FAIL     0.26 s
|
| --- command ---
| G_TEST_SRCDIR='/home/pneves/builds/tmp/work/cortexa15hf-neon-poky-linux-gnueabi/json-glib/1.4.2-r0/json-glib-1.4.2/json-glib/tests'

G_TEST_BUILDDIR='/home/pneves/builds/tmp/work/cortexa15hf-neon-poky-linux-gnueabi/json-glib/1.4.2-r0/build/json-glib/tests' /home/pneves/builds/tmp/work/cortexa15hf-neon-poky-linux-gnueabi/json-glib/1.4.2-r0/meson_exe_wrapper.sh /home/pneves/builds/tmp/work/cortexa15hf-neon-poky-linux-gnueabi/json-glib/1.4.2-r0/build/json-glib/tests/node --tap -k

| --- stdout ---
| # random seed: R02Sd21f2673b828a5a1afbadb819142cde4
| 1..27
| # Start of nodes tests
| ok 1 /nodes/gvalue
| # Start of init tests
| ok 2 /nodes/init/int
| ok 3 /nodes/init/double
| ok 4 /nodes/init/boolean
| ok 5 /nodes/init/string
| ok 6 /nodes/init/null
| # End of init tests
| # Start of copy tests
| ok 7 /nodes/copy/null
| ok 8 /nodes/copy/value
| ok 9 /nodes/copy/object
| # End of copy tests
| # Start of get tests
| ok 10 /nodes/get/int
| ok 11 /nodes/get/double
| # End of get tests
| # Start of gvalue tests
| ok 12 /nodes/gvalue/autopromotion
| # End of gvalue tests
| # Start of seal tests
| ok 13 /nodes/seal/int
| ok 14 /nodes/seal/double
| ok 15 /nodes/seal/boolean
| ok 16 /nodes/seal/string
| ok 17 /nodes/seal/null
| ok 18 /nodes/seal/object
| ok 19 /nodes/seal/array
| # End of seal tests
| # Start of immutable tests
| # child process (/nodes/immutable/int [17270]) stdout: ""
| # child process (/nodes/immutable/int [17270]) stderr:

"/lib/ld-linux-armhf.so.3: No such file or directory\n"
| # Json:ERROR:../json-glib-1.4.2/json-glib/tests/node.c:376:test_immutable_int:
stderr of child process (/nodes/immutable/int [17270]) failed to
match: *Json-CRITICAL **: json_node_set_int: assertion '!node->immutable' failed*

| --- stderr ---
| **
| Json:ERROR:../json-glib-1.4.2/json-glib/tests/node.c:376:test_immutable_int:
stderr of child process (/nodes/immutable/int [17270]) failed to
match: *Json-CRITICAL **: json_node_set_int: assertion
'!node->immutable' failed*
| qemu: uncaught target signal 6 (Aborted) - core dumped
| Aborted (core dumped)
| -------
|
| Full log written to

/home/pneves/builds/tmp/work/cortexa15hf-neon-poky-linux-gnueabi/json-glib/1.4.2-r0/build/meson-logs/testlog.txt
| FAILED: meson-test
| /home/pneves/builds/tmp/work/cortexa15hf-neon-poky-linux-gnueabi/json-glib/1.4.2-r0/recipe-sysroot-native/usr/bin/python3-native/python3 -u /home/pneves/builds/tmp/work/cortexa15hf-neon-poky-linux-gnueabi/json-glib/1.4.2-r0/recipe-sysroot-native/usr/bin/meson test --no-rebuild --print-errorlogs
| ninja: build stopped: subcommand failed.
| WARNING: exit code 1 from a shell command.