package/heaptrack: new package
PATCH v3 1/1 package/heaptrack: new package
Add heaptrack, a memory allocation tracer toolkit.
This implementation builds all the command line components, not the heaptrack_gui graphical visualization program.
---
Changes in v3:
- Removed 'linux' from _DEPENDENCIES
- Link to v2: https://patch.msgid.link/[email protected]
Changes in v2:
- removed unnecessary depends on BR2_LINUX_KERNEL
- added missing recursive 'depends on'
- sort alphabetically the 'select' lines
- completed 'depends on' list for comment line
- Add "Locally calculated" for the hash
- Add hashes of license files
- Use single tab for indentation
- Add test case
- Update DEVELOPERS file
- Link to v1: https://patch.msgid.link/[email protected]
--- DEVELOPERS | 2 ++ package/Config.in | 1 + package/heaptrack/Config.in | 37 +++++++++++++++++++++++ package/heaptrack/heaptrack.hash | 10 +++++++ package/heaptrack/heaptrack.mk | 26 ++++++++++++++++ support/testing/tests/package/test_heaptrack.py | 40 +++++++++++++++++++++++++ 6 files changed, 116 insertions(+)
DEVELOPERS +2 -0
@@ -2190,6 +2190,7 @@ F: configs/zynq_zc706_defconfig F: configs/zynqmp_zcu106_defconfig F: package/agentpp/ F: package/exim/ +F: package/heaptrack/ F: package/libpjsip/ F: package/linux-tools/linux-tool-usbtools.mk.in F: package/qpid-proton/
@@ -2198,6 +2199,7 @@ F: package/snmppp/ F: package/stm32flash/ F: package/unzip/ F: support/legal-info/ +F: support/testing/tests/package/test_heaptrack.py N: Lucas De Marchi <[email protected]> F: package/fswebcam/
package/Config.in +1 -0
@@ -106,6 +106,7 @@ menu "Debugging, profiling and benchmark" source "package/fwts/Config.in" source "package/gdb/Config.in" source "package/google-breakpad/Config.in" + source "package/heaptrack/Config.in" source "package/hyperfine/Config.in" source "package/iozone/Config.in" source "package/k3conf/Config.in"
package/heaptrack/Config.in new file +37 -0
@@ -0,0 +1,37 @@ +config BR2_PACKAGE_HEAPTRACK + bool "heaptrack" + depends on BR2_INSTALL_LIBSTDCPP + depends on BR2_TOOLCHAIN_HAS_THREADS # boost, libunwind, elfutils + depends on BR2_USE_WCHAR # boost, elfutils + depends on BR2_TOOLCHAIN_SUPPORTS_ALWAYS_LOCKFREE_ATOMIC_INTS # boost-filesystem + depends on BR2_PACKAGE_LIBUNWIND_ARCH_SUPPORTS # libunwind + depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_9 # libunwind + depends on !BR2_STATIC_LIBS # libunwind, elfutils + select BR2_PACKAGE_BOOST + select BR2_PACKAGE_BOOST_IOSTREAMS + select BR2_PACKAGE_BOOST_PROGRAM_OPTIONS + # For heaptrack_print + select BR2_PACKAGE_BOOST_FILESYSTEM + select BR2_PACKAGE_ELFUTILS + select BR2_PACKAGE_LIBUNWIND + select BR2_PACKAGE_ZLIB + help + Heaptrack traces all memory allocations and annotates these + events with stack traces. Dedicated analysis tools then allow + you to interpret the heap memory profile to find hotspots, + memory leaks, allocation hotspots and temporary allocations. + + Zstandard offers better (de)compression performance compared + with gzip/zlib, making heaptrack faster and datafiles smaller, + so enabling BR2_PACKAGE_ZSTD is recommended. + + https://apps.kde.org/heaptrack/ + +comment "heaptrack needs a toolchain w/ C++, threads, wchar, gcc >= 4.9, dynamic library" + depends on BR2_TOOLCHAIN_SUPPORTS_ALWAYS_LOCKFREE_ATOMIC_INTS + depends on BR2_PACKAGE_LIBUNWIND_ARCH_SUPPORTS + depends on !BR2_INSTALL_LIBSTDCPP \ + || !BR2_TOOLCHAIN_HAS_THREADS \ + || !BR2_USE_WCHAR \ + || !BR2_TOOLCHAIN_GCC_AT_LEAST_4_9 \ + || BR2_STATIC_LIBS
package/heaptrack/heaptrack.hash new file +10 -0
@@ -0,0 +1,10 @@ +# Locally calculated +sha256 e69de2209ab0e16c8e26c1b60feeb852b3a88e46a4f16588c08254e1bd81a183 heaptrack-3e6cce3d210a6672fe6f92de0bed567c49b7a9c5-git4.tar.gz + +sha256 fa6f36630bb1e0c571d34b2bbdf188d08495c9dbf58f28cac112f303fc1f58fb LICENSES/LGPL-2.1-only.txt +sha256 aaf135472f81c5b4a0dca9367e5bb5e9750032b5bebe5442b36e4c0a47430df3 LICENSES/GPL-2.0-or-later.txt +sha256 6aee5c3d7c55f3d8ff45b1245b8427efaf0a59e9c52d7139c79bff7056e9a611 LICENSES/LGPL-2.1-or-later.txt +sha256 84c6ef3ea9e3254a54d0acf5d3e0c61ae011b8fef7dd6940591cf060e6380a8f LICENSES/BSL-1.0.txt +sha256 b85dcd3e453d05982552c52b5fc9e0bdd6d23c6f8e844b984a88af32570b0cc0 LICENSES/MIT.txt +sha256 074e6e32c86a4c0ef8b3ed25b721ca23aca83df277cd88106ef7177c354615ff LICENSES/Apache-2.0.txt +sha256 42639b2e5507163d35069be166e37a784f48a5f17e3577c16eb48fab215247cd LICENSES/BSD-3-Clause.txt
package/heaptrack/heaptrack.mk new file +26 -0
@@ -0,0 +1,26 @@ +################################################################################ +# +# heaptrack +# +################################################################################ + +HEAPTRACK_SITE_METHOD = git +HEAPTRACK_SITE = https://invent.kde.org/sdk/heaptrack.git +HEAPTRACK_VERSION = 3e6cce3d210a6672fe6f92de0bed567c49b7a9c5 +HEAPTRACK_LICENSE = LGPL-2.1-or-later, GPL-2.0-or-later (heaptrack_interpret) +HEAPTRACK_LICENSE_FILES = \ + LICENSES/LGPL-2.1-only.txt \ + LICENSES/GPL-2.0-or-later.txt \ + LICENSES/LGPL-2.1-or-later.txt \ + LICENSES/BSL-1.0.txt \ + LICENSES/MIT.txt \ + LICENSES/Apache-2.0.txt \ + LICENSES/BSD-3-Clause.txt +HEAPTRACK_DEPENDENCIES = host-pkgconf boost libunwind zlib elfutils +HEAPTRACK_CONF_OPTS += -DHEAPTRACK_BUILD_GUI=OFF -DHEAPTRACK_BUILD_PRINT=ON + +ifeq ($(BR2_PACKAGE_ZSTD),y) +HEAPTRACK_DEPENDENCIES += zstd +endif + +$(eval $(cmake-package))
support/testing/tests/package/test_heaptrack.py new file +40 -0
@@ -0,0 +1,40 @@ +import os + +import infra.basetest + + +class TestHeaptrack(infra.basetest.BRTest): + # infra.basetest.BASIC_TOOLCHAIN_CONFIG cannot be used as it does + # not include BR2_TOOLCHAIN_SUPPORTS_ALWAYS_LOCKFREE_ATOMIC_INTS + # needed by heaptrack. + config = \ + """ + BR2_aarch64=y + BR2_TOOLCHAIN_EXTERNAL=y + BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0" + BR2_LINUX_KERNEL=y + BR2_LINUX_KERNEL_CUSTOM_VERSION=y + BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="6.18.42" + BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y + BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/aarch64-virt/linux.config" + BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL=y + BR2_PACKAGE_HEAPTRACK=y + # BR2_TARGET_ROOTFS_TAR is not set + BR2_TARGET_ROOTFS_CPIO=y + """ + + def test_run(self): + cpio_file = os.path.join(self.builddir, "images", "rootfs.cpio") + kern = os.path.join(self.builddir, "images", "Image") + self.emulator.boot(arch="aarch64", + kernel=kern, + kernel_cmdline=["console=ttyAMA0"], + options=["-M", "virt", + "-cpu", "cortex-a57", + "-m", "256M", + "-initrd", cpio_file]) + + self.emulator.login() + + self.assertRunOk("/usr/bin/heaptrack --version") + self.assertRunOk("/usr/bin/heaptrack /bin/busybox")
---
base-commit: d4f3bea066947fdee49a12ebd84cba54a872d43f
change-id: 20260731-heaptrack-cd4f6f8fcec8
Best regards,
Signature · 8 lines
Luca Ceresoli, Bootlin Embedded Linux and Kernel engineering https://bootlin.com _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Hi Luca,
Thanks for the patch. I have few comments, see below.
Quoted 7 lines · Luca Ceresoli via buildroot
Add heaptrack, a memory allocation tracer toolkit.
This implementation builds all the command line components, not the
heaptrack_guigraphical visualization program.Signed-off-by Luca Ceresoli <[email protected]>---
[...]
Quoted 12 lines
package/heaptrack/Config.in new file +6
-0@@ -0,0 +1,37 @@ +config BR2_PACKAGE_HEAPTRACK + bool "heaptrack" + depends on BR2_INSTALL_LIBSTDCPP + depends on BR2_TOOLCHAIN_HAS_THREADS # boost, libunwind, elfutils + depends on BR2_USE_WCHAR # boost, elfutils + depends on BR2_TOOLCHAIN_SUPPORTS_ALWAYS_LOCKFREE_ATOMIC_INTS #
Quoted 1 line
boost-filesystem
This package is failing to build with uclibc toolchains. See:
https://gitlab.com/jolivain/buildroot/-/pipelines/2762905573
heaptrack uses LM_ID_BASE which is not supported by uclibc:
https://invent.kde.org/sdk/heaptrack/-/blob/master/src/track/heaptrack_env.cpp#L33
https://github.com/wbx-github/uclibc-ng/blob/v1.0.59/include/dlfcn.h#L46-L54
You could either try to fix that issue, or disable the package for uclibc toolchains by adding:
depends on !BR2_TOOLCHAIN_USES_UCLIBC # no dlmopen() support
with the corresponding comment.
Quoted 32 lines
package/heaptrack/Config.in new file +24
-0+ depends on BR2_PACKAGE_LIBUNWIND_ARCH_SUPPORTS # libunwind + depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_9 # libunwind + depends on !BR2_STATIC_LIBS # libunwind, elfutils + select BR2_PACKAGE_BOOST + select BR2_PACKAGE_BOOST_IOSTREAMS + select BR2_PACKAGE_BOOST_PROGRAM_OPTIONS + # For heaptrack_print + select BR2_PACKAGE_BOOST_FILESYSTEM + select BR2_PACKAGE_ELFUTILS + select BR2_PACKAGE_LIBUNWIND + select BR2_PACKAGE_ZLIB + help + Heaptrack traces all memory allocations and annotates these + events with stack traces. Dedicated analysis tools then allow + you to interpret the heap memory profile to find hotspots, + memory leaks, allocation hotspots and temporary allocations. + + Zstandard offers better (de)compression performance compared + with gzip/zlib, making heaptrack faster and datafiles smaller, + so enabling BR2_PACKAGE_ZSTD is recommended. + + https://apps.kde.org/heaptrack/ + +comment "heaptrack needs a toolchain w/ C++, threads, wchar, gcc >=4.9, dynamic library"
package/heaptrack/Config.in new file +7
-0+ depends on BR2_TOOLCHAIN_SUPPORTS_ALWAYS_LOCKFREE_ATOMIC_INTS + depends on BR2_PACKAGE_LIBUNWIND_ARCH_SUPPORTS + depends on !BR2_INSTALL_LIBSTDCPP \ + || !BR2_TOOLCHAIN_HAS_THREADS \ + || !BR2_USE_WCHAR \ + || !BR2_TOOLCHAIN_GCC_AT_LEAST_4_9 \ + || BR2_STATIC_LIBS
[...]
Quoted 17 lines
package/heaptrack/heaptrack.mk new file +10
-0@@ -0,0 +1,26 @@ +################################################################################ +# +# heaptrack +# +################################################################################ + +HEAPTRACK_SITE_METHOD = git +HEAPTRACK_SITE = https://invent.kde.org/sdk/heaptrack.git +HEAPTRACK_VERSION = 3e6cce3d210a6672fe6f92de0bed567c49b7a9c5 +HEAPTRACK_LICENSE = LGPL-2.1-or-later, GPL-2.0-or-later(
heaptrack_interpret)
package/heaptrack/heaptrack.mk line 19
Quoted 9 lines
package/heaptrack/heaptrack.mk new file +9
-0+HEAPTRACK_LICENSE_FILES = \ + LICENSES/LGPL-2.1-only.txt \ + LICENSES/GPL-2.0-or-later.txt \ + LICENSES/LGPL-2.1-or-later.txt \ + LICENSES/BSL-1.0.txt \ + LICENSES/MIT.txt \ + LICENSES/Apache-2.0.txt \ + LICENSES/BSD-3-Clause.txt +HEAPTRACK_DEPENDENCIES = host-pkgconf boost libunwind zlib elfutils
Could you sort dependencies, please?
HEAPTRACK_DEPENDENCIES = host-pkgconf boost elfutils libunwind zlib
package/heaptrack/heaptrack.mk
Quoted 2 lines
package/heaptrack/heaptrack.mk new file +1
-1+HEAPTRACK_CONF_OPTS += -DHEAPTRACK_BUILD_GUI=OFF -DHEAPTRACK_BUILD_PRINT=ON
Since it's the first _CONF_OPTS assignement, could you remove the '+', please?
HEAPTRACK_CONF_OPTS = -DHEAPTRACK_BUILD_GUI=OFF ...
Quoted 44 lines
package/heaptrack/heaptrack.mk new file +6
-0+ +ifeq ($(BR2_PACKAGE_ZSTD),y) +HEAPTRACK_DEPENDENCIES += zstd +endif + +$(eval $(cmake-package))support/testing/tests/package/test_heaptrack.py new file +19
-0@@ -0,0 +1,40 @@ +import os + +import infra.basetest + + +class TestHeaptrack(infra.basetest.BRTest): + # infra.basetest.BASIC_TOOLCHAIN_CONFIG cannot be used as it does + # not include BR2_TOOLCHAIN_SUPPORTS_ALWAYS_LOCKFREE_ATOMIC_INTS + # needed by heaptrack. + config = \ + """ + BR2_aarch64=y + BR2_TOOLCHAIN_EXTERNAL=y + BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0" + BR2_LINUX_KERNEL=y + BR2_LINUX_KERNEL_CUSTOM_VERSION=y + BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="6.18.42" + BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y +BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/aarch64-virt/linux.config"
support/testing/tests/package/test_heaptrack.py new file +8
-0+ BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL=y + BR2_PACKAGE_HEAPTRACK=y + # BR2_TARGET_ROOTFS_TAR is not set + BR2_TARGET_ROOTFS_CPIO=y + """ + + def test_run(self): + cpio_file = os.path.join(self.builddir, "images","rootfs.cpio")
support/testing/tests/package/test_heaptrack.py new file +3
-0+ kern = os.path.join(self.builddir, "images", "Image") + self.emulator.boot(arch="aarch64", + kernel=kern,
support/testing/tests/package/test_heaptrack.py line 40
Quoted 10 lines
support/testing/tests/package/test_heaptrack.py new file +10
-0+ kernel_cmdline=["console=ttyAMA0"], + options=["-M", "virt", + "-cpu", "cortex-a57", + "-m", "256M", + "-initrd", cpio_file]) + + self.emulator.login() + + self.assertRunOk("/usr/bin/heaptrack --version") + self.assertRunOk("/usr/bin/heaptrack /bin/busybox")
Maybe we could make this test slightly more complex, to have a bit more allocations, and also printing the recorded data.
For example:
self.assertRunOk("heaptrack -o /tmp/ls.heaptrack ls -al /") self.assertRunOk("heaptrack_print --file /tmp/ls.heaptrack.gz")
Quoted 9 lines
---
base-commit:d4f3bea066947fdee49a12ebd84cba54a872d43f
change-id: 20260731-heaptrack-cd4f6f8fcec8Best regards,
--
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
Could you send an updated patch addressing those issues, please?
Best regards,
Julien.
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Hello Julien,
Quoted 17 lines · Julien Olivain
Quoted 9 lines
+++ b/package/heaptrack/Config.in
package/heaptrack/Config.in new file +6
-0@@ -0,0 +1,37 @@ +config BR2_PACKAGE_HEAPTRACK + bool "heaptrack" + depends on BR2_INSTALL_LIBSTDCPP + depends on BR2_TOOLCHAIN_HAS_THREADS # boost, libunwind, elfutils + depends on BR2_USE_WCHAR # boost, elfutils + depends on BR2_TOOLCHAIN_SUPPORTS_ALWAYS_LOCKFREE_ATOMIC_INTS #boost-filesystem
This package is failing to build with uclibc toolchains. See:
https://gitlab.com/jolivain/buildroot/-/pipelines/2762905573heaptrack uses
LM_ID_BASEwhich is not supported by uclibc:
https://invent.kde.org/sdk/heaptrack/-/blob/master/src/track/heaptrack_env.cpp#L33
https://github.com/wbx-github/uclibc-ng/blob/v1.0.59/include/dlfcn.h#L46-L54
Quoted 6 lines
You could either try to fix that issue, or disable the package for uclibc toolchains by adding:
depends on !BR2_TOOLCHAIN_USES_UCLIBC # no
dlmopen()supportwith the corresponding comment.
Thanks for the review and testing!
Quoted 3 lines
Quoted 1 line
+HEAPTRACK_DEPENDENCIES = host-pkgconf boost libunwind zlib elfutils
Could you sort dependencies, please?
Ah, sure! By a quick look into existing packages I had concluded sorting is not required.
I'm sending v4 with all the changes you suggested.
Luca
--
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot