valgrind: integration of regression tests to ptest
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v2 Latest1 patch 0 repliesNew
- v11 patch 1 replySuperseded
Patches in v2 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/1 | valgrind: integration of regression tests to ptestmeta/ meta/ meta/ +3 more | +455 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v2 yet.
Cover letter [PATCH v2 0/1] valgrind: integration of regression tests to ptest
Hello Saul,
V2 patch includes following changes:
1) no PR bump in recipe
2) lower-cased commit header
Regards,
Dave Lerner
--------------
The attached patch adds PTEST integration to valgrind. The patch was built and tested as shown in the "CONFIGURATION", "BUILD TESTS", and "RUNTIME TESTS" sections below. A summary of changes is shown in "COMMIT STAT".
Thanks,
Dave Lerner
COMMIT STAT
===========
commit 2e539fd064c4ef91a1b9593edfadffffe0f628d1
Author: Dave Lerner <[email protected]> Date: Mon Jan 27 11:46:19 2014 -0600
valgrind: integration of regression tests to ptest
Modifies valgrind's regression test framework to be compatible with the yocto PTEST framework as follows:
* existing recipe valgrind*bb adds new methods: do_compile_ptest and do_install_ptest.
* new file run-ptest adds the wrapper interface to the valgrind regression test script vg_regtest.
* existing valgrind regression test script 'vg_regtest' changes to report the status of the valgrind component tests in the format that PTEST expects, instead of the valgrind formats, but only when vg_regtest is invoked with an optional --yocto-ptest argument
* four new patches disable building tests that don't compile with the yocto compiler and default options. See the patches for details.
.../valgrind/valgrind/add-ptest.patch | 205 ++++++++++++++++++++ .../valgrind/valgrind/force-nostabs.patch | 42 ++++ .../valgrind/remove-arm-variant-specific.patch | 67 +++++++ .../valgrind/remove-ppc-tests-failing-build.patch | 75 +++++++ meta/recipes-devtools/valgrind/valgrind/run-ptest | 11 ++ meta/recipes-devtools/valgrind/valgrind_3.9.0.bb | 56 +++++- 6 files changed, 455 insertions(+), 1 deletion(-)
CONFIGURATION
=============
Appended to each conf/local.conf ...
MACHINE = 'some-machine-name'SSTATE_DIR = "/ord-dlerner-l81/dlerner/poky_16/sstate-cache" DISTRO_FEATURES_append = " ptest" EXTRA_IMAGE_FEATURES = "debug-tweaks dbg-pkgs ptest-pkgs" IMAGE_INSTALL_append = " valgrind ptest-runner"
BUILD TESTS
===========
Configuration fails with reports that valgrind is not supported for:
qemuarm, qemumips*, routerstationpro
ERROR: Nothing RPROVIDES 'valgrind' (but ...core-image-basic.bb
RDEPENDS on or otherwise requires it)
ERROR: valgrind was skipped: incompatible with host
mips-poky-linux (not in COMPATIBLE_HOST)
NOTE: Runtime target 'valgrind' is unbuildable, removing...
Configuration and 'bitbake core-image-basic' passes for:
beagleboard, mpc8316-rdb, qemuppc, qemux86, qemux86-64
RUNTIME TESTS
=============
The tests are divided into per tool tests and then a common set of tests across all tools in the directory 'none'.
The output per PTEST documentation looks like the following snippet:
PASS: cachegrind/tests/chdir
PASS: cachegrind/tests/clreq
PASS: cachegrind/tests/dlclose
PASS: cachegrind/tests/notpower2
FAIL: cachegrind/tests/wrap5
PASS: callgrind/tests/clreq
Below is the output summarized for two bsps. Note that this is not the output produced by this commit.
qemux86-64
----------
cachegrind:
PASS count=4 FAIL count=1 SKIP count=0
callgrind:
PASS count=13 FAIL count=0 SKIP count=0
drd:
PASS count=106 FAIL count=10 SKIP count=7
helgrind:
PASS count=20 FAIL count=31 SKIP count=1
memcheck:
PASS count=116 FAIL count=78 SKIP count=4
massif:
PASS count=9 FAIL count=26 SKIP count=0
none:
PASS count=101 FAIL count=4 SKIP count=16
qemux86
------
cachegrind:
PASS count=5 FAIL count=1 SKIP count=0
callgrind:
PASS count=13 FAIL count=0 SKIP count=0
drd:
PASS count=106 FAIL count=11 SKIP count=6
helgrind:
PASS count=20 FAIL count=31 SKIP count=1
memcheck:
PASS count=126 FAIL count=78 SKIP count=2
massif:
PASS count=9 FAIL count=26 SKIP count=0
none:
PASS count=70 FAIL count=3 SKIP count=1
qemuppc
-------
All of the tests fail for the valgrind tools cachegrind, drd, helgrind, massif, memcheck, that are not skipped due to preconditions failing. The failures of any test match the failure message for each valgrind tool run agains the program /bin/ls.