valgrind upgrade
I know it's late,
I know we're weary,
I know our plans don't include [1]
valgrind 3.14 which was released recently after 1.5 years of development. I believe that some of the M3 valgrind QA results we'ren't very good so if we're going to try to fix those problems, perhaps
we should be working on the 3.14 release.
I've reviewed and updated the patches but so far I've only built for qemux86. The other qemus are building overnight.
If someone wants to test basic runtime functions and better still run the ptests, that would be great.
The important bits of the release notes IMO:
* More than 100 bugs fixed.
* Valgrind is now buildable with link-time optimisation (LTO). A new configure option --enable-lto=yes allows building Valgrind with LTO. If the toolchain supports it, this produces a smaller/faster Valgrind (up to 10%). Note that if you are doing Valgrind development, --enable-lto=yes massively slows down the build process.
-- I haven't added support for that option yet. A 10% performance boost
is hard to turn down but we'd need to understand the build impact.
* The new option --keep-debuginfo=no|yes (default no) can be used to retain debug info for unloaded code. This allows saved stack traces (e.g. for memory leaks) to include file/line info for code that has been dlclose'd (or similar). See the user manual for more information and known limitations.
-- sounds like it should be a default but I haven't added it yet.
Full release notes:
http://valgrind.org/docs/manual/dist.news.html
../Randy
[1] With apologies for the first line to Kenny Rogers.
PATCH 1/1 valgrind: update from 3.13.0 to 3.14.0
Removed patches are all upstream.
Adjusted two patches due to rebase.
---
...se-ucontext_t-instead-of-struct-ucon.patch | 30 -------
...ntext-APIs-are-not-available-on-musl.patch | 17 ++--
.../0004-Fix-out-of-tree-builds.patch | 49 ++++++-----
.../valgrind/valgrind/link-gz-tests.patch | 27 ------
...sk-CPUID-support-in-HWCAP-on-aarch64.patch | 36 --------
.../valgrind/valgrind/ppc-headers.patch | 87 -------------------
...{valgrind_3.13.0.bb => valgrind_3.14.0.bb} | 11 +--
7 files changed, 40 insertions(+), 217 deletions(-)
delete mode 100644 meta/recipes-devtools/valgrind/valgrind/0001-memcheck-tests-Use-ucontext_t-instead-of-struct-ucon.patch
delete mode 100644 meta/recipes-devtools/valgrind/valgrind/link-gz-tests.patch
delete mode 100644 meta/recipes-devtools/valgrind/valgrind/mask-CPUID-support-in-HWCAP-on-aarch64.patch
delete mode 100644 meta/recipes-devtools/valgrind/valgrind/ppc-headers.patch
rename meta/recipes-devtools/valgrind/{valgrind_3.13.0.bb => valgrind_3.14.0.bb} (91%)meta/recipes-devtools/valgrind/valgrind/0001-memcheck-tests-Use-ucontext_t-instead-of-struct-ucon.patch deleted +0 -30
@@ -1,30 +0,0 @@ -From 629ac492b1d9bc709d17337eb9b1c28603eca250 Mon Sep 17 00:00:00 2001 -From: Khem Raj <[email protected]> -Date: Wed, 28 Jun 2017 11:01:25 -0700 -Subject: [PATCH] memcheck/tests: Use ucontext_t instead of struct ucontext - -glibc 2.26 does not expose struct ucontext anymore - -Upstream-Status: Submitted [https://bugs.kde.org/show_bug.cgi?id=381769] - -Signed-off-by: Khem Raj <[email protected]> ---- - memcheck/tests/linux/stack_changes.c | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) - -diff --git a/memcheck/tests/linux/stack_changes.c b/memcheck/tests/linux/stack_changes.c -index ffb49c6..acc4109 100644 ---- a/memcheck/tests/linux/stack_changes.c -+++ b/memcheck/tests/linux/stack_changes.c -@@ -11,7 +11,7 @@ - // checks that Valgrind notices their stack changes properly. - - #ifdef __GLIBC__ --typedef struct ucontext mycontext; -+typedef ucontext_t mycontext; - - mycontext ctx1, ctx2, oldc; - int count; --- -2.13.2 -
meta/recipes-devtools/valgrind/valgrind/0002-context-APIs-are-not-available-on-musl.patch +10 -7
@@ -1,17 +1,20 @@ -From 862b807076d57f2f58ed9d572ddac8bb402774a2 Mon Sep 17 00:00:00 2001 -From: Khem Raj <[email protected]> -Date: Sat, 10 Jun 2017 01:01:10 -0700 -Subject: [PATCH 2/6] context APIs are not available on musl +From bd4e926e7e14747b3cd4d7b2a1bd5410b22f3ea2 Mon Sep 17 00:00:00 2001 +From: Randy MacLeod <[email protected]> +Date: Tue, 16 Oct 2018 21:27:46 -0400 +Subject: [PATCH] context APIs are not available on musl Upstream-Status: Pending +Updated patch for valgrind-3.14 + Signed-off-by: Khem Raj <[email protected]> +Signed-off-by: Randy MacLeod <[email protected]> --- memcheck/tests/linux/stack_changes.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/memcheck/tests/linux/stack_changes.c b/memcheck/tests/linux/stack_changes.c -index a978fc2..ffb49c6 100644 +index 7f97b90a5..a26cb4ae6 100644 --- a/memcheck/tests/linux/stack_changes.c +++ b/memcheck/tests/linux/stack_changes.c @@ -10,6 +10,7 @@
@@ -19,7 +22,7 @@ index a978fc2..ffb49c6 100644 // checks that Valgrind notices their stack changes properly. +#ifdef __GLIBC__ - typedef struct ucontext mycontext; + typedef ucontext_t mycontext; mycontext ctx1, ctx2, oldc; @@ -51,9 +52,11 @@ int init_context(mycontext *uc)
@@ -45,5 +48,5 @@ index a978fc2..ffb49c6 100644 return 0; } -- -2.13.1 +2.17.0
meta/recipes-devtools/valgrind/valgrind/0004-Fix-out-of-tree-builds.patch +27 -22
101 lines not on this page · raw message
meta/recipes-devtools/valgrind/valgrind/link-gz-tests.patch deleted +0 -27
@@ -1,27 +0,0 @@ -When checking if the compiler supports compressed debug sections we need to -actually link instead of just compile. Otherwise the compiler thinks that -they are supported, but gold does not support -gz=zlib. - -Upstream-Status: Backport (r16459) -Signed-off-by: Ross Burton <[email protected]> - ---- a/configure.ac~ 2017-07-11 11:53:16.000000000 +0100 -+++ b/configure.ac 2017-07-11 18:16:13.674130483 +0100 -@@ -2119,7 +2119,7 @@ - safe_CFLAGS=$CFLAGS - CFLAGS="-g -gz=zlib" - --AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ ]], [[ -+AC_LINK_IFELSE([AC_LANG_PROGRAM([[ ]], [[ - return 0; - ]])], [ - ac_have_gz_zlib=yes -@@ -2139,7 +2139,7 @@ - safe_CFLAGS=$CFLAGS - CFLAGS="-g -gz=zlib-gnu" - --AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[ ]], [[ -+AC_LINK_IFELSE([AC_LANG_PROGRAM([[ ]], [[ - return 0; - ]])], [ - ac_have_gz_zlib_gnu=yes
meta/recipes-devtools/valgrind/valgrind/mask-CPUID-support-in-HWCAP-on-aarch64.patch deleted +0 -36
@@ -1,36 +0,0 @@ -Fix runtime Valgrind failure - -This patch is derived from -https://bugzilla.redhat.com/show_bug.cgi?id=1464211 - -At runtime it will fails like this: - -ARM64 front end: branch_etc -disInstr(arm64): unhandled instruction 0xD5380001 -disInstr(arm64): 1101'0101 0011'1000 0000'0000 0000'0001 ==2082== -valgrind: Unrecognised instruction at address 0x4014e64. - -This patch is a workaround by masking all HWCAP - -Upstream-Status: Pending - -Signed-off-by: Manjukumar Matha <[email protected]> - -Index: valgrind-3.13.0/coregrind/m_initimg/initimg-linux.c -=================================================================== - ---- valgrind-3.13.0.orig/coregrind/m_initimg/initimg-linux.c 2018-03-04 22:22:17.698572675 -0800 -+++ valgrind-3.13.0/coregrind/m_initimg/initimg-linux.c 2018-03-04 22:23:25.727815624 -0800 -@@ -703,6 +703,12 @@ - (and anything above) are not supported by Valgrind. */ - auxv->u.a_val &= VKI_HWCAP_S390_TE - 1; - } -+# elif defined(VGP_arm64_linux) -+ { -+ /* Linux 4.11 started populating this for arm64, but we -+ currently don't support any. */ -+ auxv->u.a_val = 0; -+ } - # endif - break; - # if defined(VGP_ppc64be_linux) || defined(VGP_ppc64le_linux)
meta/recipes-devtools/valgrind/valgrind/ppc-headers.patch deleted +0 -87
87 lines not on this page · raw message
meta/recipes-devtools/valgrind/valgrind_3.14.0.bb renamed from meta/recipes-devtools/valgrind/valgrind_3.13.0.bb +3 -8
@@ -12,7 +12,7 @@ DEPENDS = "${@bb.utils.contains('DISTRO_FEATURES', 'x11', '${X11DEPENDS}', '', d ${@bb.utils.contains('DISTRO_FEATURES', 'ptest', 'boost', '', d)} \ " -SRC_URI = "ftp://sourceware.org/pub/valgrind/valgrind-${PV}.tar.bz2 \ +SRC_URI = "http://www.valgrind.org/downloads/valgrind-${PV}.tar.bz2 \ file://fixed-perl-path.patch \ file://Added-support-for-PPC-instructions-mfatbu-mfatbl.patch \ file://run-ptest \
@@ -30,18 +30,13 @@ SRC_URI = "ftp://sourceware.org/pub/valgrind/valgrind-${PV}.tar.bz2 \ file://0004-pth_atfork1.c-Define-error-API-for-musl.patch \ file://0005-tc20_verifywrap.c-Fake-__GLIBC_PREREQ-with-musl.patch \ file://0006-pth_detached3.c-Dereference-pthread_t-before-adding-.patch \ - file://0001-memcheck-tests-Use-ucontext_t-instead-of-struct-ucon.patch \ file://0001-memcheck-arm64-Define-__THROW-if-not-already-defined.patch \ file://0002-memcheck-x86-Define-__THROW-if-not-defined.patch \ file://0003-tests-seg_override-Replace-__modify_ldt-with-syscall.patch \ - file://link-gz-tests.patch \ - file://ppc-headers.patch \ - file://mask-CPUID-support-in-HWCAP-on-aarch64.patch \ file://0001-fix-opcode-not-supported-on-mips32-linux.patch \ - file://0002-fix-broken-inline-asm-in-tests-on-mips32-linux.patch \ " -SRC_URI[md5sum] = "817dd08f1e8a66336b9ff206400a5369" -SRC_URI[sha256sum] = "d76680ef03f00cd5e970bbdcd4e57fb1f6df7d2e2c071635ef2be74790190c3b" +SRC_URI[md5sum] = "74175426afa280184b62591b58c671b3" +SRC_URI[sha256sum] = "037c11bfefd477cc6e9ebe8f193bb237fe397f7ce791b4a4ce3fa1c6a520baa5" UPSTREAM_CHECK_REGEX = "valgrind-(?P<pver>\d+(\.\d+)+)\.tar" COMPATIBLE_HOST = '(i.86|x86_64|arm|aarch64|mips|powerpc|powerpc64).*-linux'
Signature · 1 line
2.17.0
Re: [PATCH] valgrind: update from 3.13.0 to 3.14.0
Quoted 4 lines · Randy MacLeod
Removed patches are all upstream.
Adjusted two patches due to rebase.Signed-off-by Randy MacLeod <[email protected]>
I set the autobuilder at this and it fails in musl in do_compile_ptest_base:
https://autobuilder.yoctoproject.org/typhoon/#/builders/27/builds/126/steps/7/logs/step1b
Cheers,
Richard
Re: [PATCH] valgrind: update from 3.13.0 to 3.14.0
Quoted 2 lines · Richard Purdie
On #if
defined(__GLIBC__)&&defined(__GLIBC_PREREQ)
Quoted 10 lines · Randy MacLeod
Quoted 4 lines
Removed patches are all upstream.
Adjusted two patches due to rebase.Signed-off-by Randy MacLeod <[email protected]>I set the autobuilder at this and it fails in musl in
do_compile_ptest_base:https://autobuilder.yoctoproject.org/typhoon/#/builders/27/builds/126/steps/7/logs/step1b
yeah using __GLIBC_PREREQ needs to be wrapped in
#if defined(__GLIBC__) && defined(__GLIBC_PREREQ)
#endif
Quoted 9 lines
Cheers,
Richard
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Re: [PATCH] valgrind: update from 3.13.0 to 3.14.0
Quoted 13 lines · Khem Raj
Quoted 2 lines · Richard Purdie
On #if
defined(__GLIBC__)&&defined(__GLIBC_PREREQ)Quoted 8 lines · Randy MacLeod
Quoted 4 lines
Removed patches are all upstream.
Adjusted two patches due to rebase.Signed-off-by Randy MacLeod <[email protected]>I set the autobuilder at this and it fails in musl in
do_compile_ptest_base:
Quoted 8 lines
Quoted 2 lines
https://autobuilder.yoctoproject.org/typhoon/#/builders/27/builds/126/steps/7/logs/step1b
yeah using
__GLIBC_PREREQneeds to be wrapped in#if
defined(__GLIBC__)&&defined(__GLIBC_PREREQ)#endif
For v2 I'll fix the build for musl and
remove that unused patch that patchworks pointed out.
All the glibc builds succeeded except for arm and mips64 which were not supported by the recipe. I'll check on that with the new version of valgrind at some point, likely on master-wr-> 2.6.1.
Here is the pass rate compared to 2.6-M3 QA run:
Test run: qa-3.13 vg-3.14 vg-3.13
Tests ran 558 547 538 Passed 215 287 201 Failed 343 260 337
Passrate 38.53 52.5 37.4
Last Passrate 49.1
where:
qa-3.13 is from the 2.6-M3 QA report vg-3.14 is this update and
vg-3.13 is with this update removed using poky at:
3b77e7b785 systemtap: Fix issues from 4.0 upgrade
So the ptest results really are better and they are even better than the last pass rate that QA reported. Nonetheless, a 52.5% pass rate isn't good enough so I'll work on that for master-wr with backports to 2.6.1.
../Randy
Some notes and more raw data than presented above.
BTW, I allocated 4G RAM to the qemu machine after running with the default and seeing the OOM killer run many times. I wonder if the QA test doesn't allocate enough RAM to deal with valgrind's high overhead.
The ptests summary using:
qemux86-64 using core-image-full-cmdline and
40GB extra FS in the image and running:
$ runqemu nographic slirp kvm qemuparams="-m 4096"
valgrind-3.14 is:
=== Test Summary ===
TOTAL: 579
PASSED: 287
FAILED: 260
SKIPPED: 32
with valgrind-3.13:
=== Test Summary ===
TOTAL: 570
PASSED: 201
FAILED: 337
SKIPPED: 32
Quoted 10 lines
Quoted 10 lines
Cheers,
Richard
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
Signature · 2 lines
# Randy MacLeod # Wind River Linux
Re: [PATCH] valgrind: update from 3.13.0 to 3.14.0
Quoted 12 lines · Randy MacLeod
For v2 I'll fix the build for musl and
remove that unused patch that patchworks pointed out.All the glibc builds succeeded except for arm and mips64 which were not supported by the recipe. I'll check on that with the new version of valgrind at some point, likely on master-wr-> 2.6.1.
Here is the pass rate compared to 2.6-M3 QA run:
Quoted 7 lines
Test run: qa-3.13 vg-3.14 vg-3.13
Tests ran 558 547 538 Passed 215 287 201 Failed 343 260 337Passrate 38.53 52.5 37.4
Last Passrate 49.1
Thanks, that at least suggests we are probably better off upgrading and we can do so based on real data! :)
Quoted 15 lines
where:
qa-3.13 is from the 2.6-M3 QA report vg-3.14 is this update and
vg-3.13 is with this update removed using poky at:
3b77e7b785systemtap: Fix issues from 4.0 upgradeSo the ptest results really are better and they are even better than the last pass rate that QA reported. Nonetheless, a 52.5% pass rate isn't good enough so I'll work on that for master-wr with backports to 2.6.1.
../Randy
Quoted 6 lines
Some notes and more raw data than presented above.
BTW, I allocated 4G RAM to the qemu machine after running with the default and seeing the OOM killer run many times. I wonder if the QA test doesn't allocate enough RAM to deal with valgrind's high overhead.
Ross' comments based on Maxin's work looking at valgrind ptest were that the tests do end up OOM a lot and aren't particularly stable/predictable. More investigation is clearly needed to understand what is going on...
Cheers,
Richard
✗ patchtest: failure for valgrind: update from 3.13.0 to 3.14.0
In reply to Randy MacLeod== Series Details ==
Series: valgrind: update from 3.13.0 to 3.14.0
Revision: 1
URL : https://patchwork.openembedded.org/series/14563/
State : failure
== Summary ==
Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures:
* Issue Patches not removed from tree [test_src_uri_left_files]
Suggested fix Amend the patch containing the software patch file removal
Patch 0002-fix-broken-inline-asm-in-tests-on-mips32-linux.patch
If you believe any of these test results are incorrect, please reply to the mailing list ([email protected]) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...).
---
Guidelines: https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe
Quoted 7 lines · Randy MacLeod
* Valgrind is now buildable with link-time optimisation (LTO). A new configure option --enable-lto=yes allows building Valgrind with LTO. If the toolchain supports it, this produces a smaller/faster Valgrind (up to 10%). Note that if you are doing Valgrind development, --enable-lto=yes massively slows down the build process.
-- I haven't added support for that option yet. A 10% performance boost
is hard to turn down but we'd need to understand the build impact.
Building the native valgrind 3.14ish
8b689c66d (origin/master, origin/HEAD)
Implement VG_(apply_ExeContext)().
on my 4 core laptop with 16 GB RAM, SSD running ubuntu-18.04 outside of bitbake just for a baseline. /usr/bin/time says:
CPU: ~3.8x more time and :
RAM: 348 MB vs 519 MB Max memory usage
See below for logs.
I tried to use --enable-lto in the recipe
but that configure option was ignored so for now
I'm not going to worry about it. If we get it working, I'd want the faster build as the default with a PACKAGECONFIG for people who want to pay at compile time for a 0-10% improvement in runtime behaviour.
../Randy
Quoted 6 lines
* The new option --keep-debuginfo=no|yes (default no) can be used to retain debug info for unloaded code. This allows saved stack traces (e.g. for memory leaks) to include file/line info for code that has been dlclose'd (or similar). See the user manual for more information and known limitations.
-- sounds like it should be a default but I haven't added it yet.
This feature will have to wait until next week.
I'll send what I have later tonight or perhaps tomorrow morning.
Signature · 24 lines
# Randy MacLeod # Wind River Linux $ cat no-lto.log ; 296.20user 13.31system 5:10.29elapsed 99%CPU (0avgtext+0avgdata 347800maxresident)k 224inputs+1783440outputs (3major+5964298minor)pagefaults 0swaps $ cat with-lto.log 1101.82user 23.79system 18:48.13elapsed 99%CPU (0avgtext+0avgdata 519400maxresident)k 39200inputs+4644952outputs (177major+9033934minor)pagefaults 0swaps and FYI, on a 128 core server running bitbake -c configure valgrind && /usr/bin/time -o foo bitbake -c compile valgrind: $ cat compile-vg.log 0.89user 0.14system 0:49.64elapsed 2%CPU (0avgtext+0avgdata 28124maxresident)k 0inputs+152outputs (0major+9873minor)pagefaults 0swaps