Skip to contentSkip to conversation

package/heaptrack: new package

PATCH Superseded Luca Ceresoli via buildroot 1 participant 3 messages Aug 11 – Aug 21

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 package/heaptrack: new package

Add heaptrack, a memory allocation tracer toolkit.

This implementation builds all the command line components, not the heaptrack_gui graphical visualization program.

---
Changes in v3:
- Removed 'linux' from _DEPENDENCIES
- Link to v2: https://patch.msgid.link/[email protected]

Changes in v2:
- removed unnecessary depends on BR2_LINUX_KERNEL
- added missing recursive 'depends on'
- sort alphabetically the 'select' lines
- completed 'depends on' list for comment line
- Add "Locally calculated" for the hash
- Add hashes of license files
- Use single tab for indentation
- Add test case
- Update DEVELOPERS file
- Link to v1: https://patch.msgid.link/[email protected]

---
 DEVELOPERS                                      |  2 ++
 package/Config.in                               |  1 +
 package/heaptrack/Config.in                     | 37 +++++++++++++++++++++++
 package/heaptrack/heaptrack.hash                | 10 +++++++
 package/heaptrack/heaptrack.mk                  | 26 ++++++++++++++++
 support/testing/tests/package/test_heaptrack.py | 40 +++++++++++++++++++++++++
 6 files changed, 116 insertions(+)
DEVELOPERS +2 -0
index 75607183c614..0712829672e3 100644
@@ -2190,6 +2190,7 @@ F:	configs/zynq_zc706_defconfig
 F:	configs/zynqmp_zcu106_defconfig
 F:	package/agentpp/
 F:	package/exim/
+F:	package/heaptrack/
 F:	package/libpjsip/
 F:	package/linux-tools/linux-tool-usbtools.mk.in
 F:	package/qpid-proton/
@@ -2198,6 +2199,7 @@ F:	package/snmppp/
 F:	package/stm32flash/
 F:	package/unzip/
 F:	support/legal-info/
+F:	support/testing/tests/package/test_heaptrack.py
 
 N:	Lucas De Marchi <[email protected]>
 F:	package/fswebcam/
package/Config.in +1 -0
index 6058800682b4..307f7ce86a3e 100644
@@ -106,6 +106,7 @@ menu "Debugging, profiling and benchmark"
 	source "package/fwts/Config.in"
 	source "package/gdb/Config.in"
 	source "package/google-breakpad/Config.in"
+	source "package/heaptrack/Config.in"
 	source "package/hyperfine/Config.in"
 	source "package/iozone/Config.in"
 	source "package/k3conf/Config.in"
package/heaptrack/Config.in new file +37 -0
new file mode 100644
index 000000000000..12e33ff2c219
@@ -0,0 +1,37 @@
+config BR2_PACKAGE_HEAPTRACK
+	bool "heaptrack"
+	depends on BR2_INSTALL_LIBSTDCPP
+	depends on BR2_TOOLCHAIN_HAS_THREADS # boost, libunwind, elfutils
+	depends on BR2_USE_WCHAR # boost, elfutils
+	depends on BR2_TOOLCHAIN_SUPPORTS_ALWAYS_LOCKFREE_ATOMIC_INTS # boost-filesystem
+	depends on BR2_PACKAGE_LIBUNWIND_ARCH_SUPPORTS # libunwind
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_9 # libunwind
+	depends on !BR2_STATIC_LIBS # libunwind, elfutils
+	select BR2_PACKAGE_BOOST
+	select BR2_PACKAGE_BOOST_IOSTREAMS
+	select BR2_PACKAGE_BOOST_PROGRAM_OPTIONS
+	# For heaptrack_print
+	select BR2_PACKAGE_BOOST_FILESYSTEM
+	select BR2_PACKAGE_ELFUTILS
+	select BR2_PACKAGE_LIBUNWIND
+	select BR2_PACKAGE_ZLIB
+	help
+	  Heaptrack traces all memory allocations and annotates these
+	  events with stack traces. Dedicated analysis tools then allow
+	  you to interpret the heap memory profile to find hotspots,
+	  memory leaks, allocation hotspots and temporary allocations.
+
+	  Zstandard offers better (de)compression performance compared
+	  with gzip/zlib, making heaptrack faster and datafiles smaller,
+	  so enabling BR2_PACKAGE_ZSTD is recommended.
+
+	  https://apps.kde.org/heaptrack/
+
+comment "heaptrack needs a toolchain w/ C++, threads, wchar, gcc >= 4.9, dynamic library"
+	depends on BR2_TOOLCHAIN_SUPPORTS_ALWAYS_LOCKFREE_ATOMIC_INTS
+	depends on BR2_PACKAGE_LIBUNWIND_ARCH_SUPPORTS
+	depends on !BR2_INSTALL_LIBSTDCPP \
+		|| !BR2_TOOLCHAIN_HAS_THREADS \
+		|| !BR2_USE_WCHAR \
+		|| !BR2_TOOLCHAIN_GCC_AT_LEAST_4_9 \
+		|| BR2_STATIC_LIBS
package/heaptrack/heaptrack.hash new file +10 -0
new file mode 100644
index 000000000000..6ad9619f23b9
@@ -0,0 +1,10 @@
+# Locally calculated
+sha256  e69de2209ab0e16c8e26c1b60feeb852b3a88e46a4f16588c08254e1bd81a183  heaptrack-3e6cce3d210a6672fe6f92de0bed567c49b7a9c5-git4.tar.gz
+
+sha256  fa6f36630bb1e0c571d34b2bbdf188d08495c9dbf58f28cac112f303fc1f58fb  LICENSES/LGPL-2.1-only.txt
+sha256  aaf135472f81c5b4a0dca9367e5bb5e9750032b5bebe5442b36e4c0a47430df3  LICENSES/GPL-2.0-or-later.txt
+sha256  6aee5c3d7c55f3d8ff45b1245b8427efaf0a59e9c52d7139c79bff7056e9a611  LICENSES/LGPL-2.1-or-later.txt
+sha256  84c6ef3ea9e3254a54d0acf5d3e0c61ae011b8fef7dd6940591cf060e6380a8f  LICENSES/BSL-1.0.txt
+sha256  b85dcd3e453d05982552c52b5fc9e0bdd6d23c6f8e844b984a88af32570b0cc0  LICENSES/MIT.txt
+sha256  074e6e32c86a4c0ef8b3ed25b721ca23aca83df277cd88106ef7177c354615ff  LICENSES/Apache-2.0.txt
+sha256  42639b2e5507163d35069be166e37a784f48a5f17e3577c16eb48fab215247cd  LICENSES/BSD-3-Clause.txt
package/heaptrack/heaptrack.mk new file +26 -0
new file mode 100644
index 000000000000..ef12f9d189fe
@@ -0,0 +1,26 @@
+################################################################################
+#
+# heaptrack
+#
+################################################################################
+
+HEAPTRACK_SITE_METHOD = git
+HEAPTRACK_SITE = https://invent.kde.org/sdk/heaptrack.git
+HEAPTRACK_VERSION = 3e6cce3d210a6672fe6f92de0bed567c49b7a9c5
+HEAPTRACK_LICENSE = LGPL-2.1-or-later, GPL-2.0-or-later (heaptrack_interpret)
+HEAPTRACK_LICENSE_FILES = \
+	LICENSES/LGPL-2.1-only.txt \
+	LICENSES/GPL-2.0-or-later.txt \
+	LICENSES/LGPL-2.1-or-later.txt \
+	LICENSES/BSL-1.0.txt \
+	LICENSES/MIT.txt \
+	LICENSES/Apache-2.0.txt \
+	LICENSES/BSD-3-Clause.txt
+HEAPTRACK_DEPENDENCIES = host-pkgconf boost libunwind zlib elfutils
+HEAPTRACK_CONF_OPTS += -DHEAPTRACK_BUILD_GUI=OFF -DHEAPTRACK_BUILD_PRINT=ON
+
+ifeq ($(BR2_PACKAGE_ZSTD),y)
+HEAPTRACK_DEPENDENCIES += zstd
+endif
+
+$(eval $(cmake-package))
support/testing/tests/package/test_heaptrack.py new file +40 -0
new file mode 100644
index 000000000000..de3d5441efa4
@@ -0,0 +1,40 @@
+import os
+
+import infra.basetest
+
+
+class TestHeaptrack(infra.basetest.BRTest):
+    # infra.basetest.BASIC_TOOLCHAIN_CONFIG cannot be used as it does
+    # not include BR2_TOOLCHAIN_SUPPORTS_ALWAYS_LOCKFREE_ATOMIC_INTS
+    # needed by heaptrack.
+    config = \
+        """
+        BR2_aarch64=y
+        BR2_TOOLCHAIN_EXTERNAL=y
+        BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0"
+        BR2_LINUX_KERNEL=y
+        BR2_LINUX_KERNEL_CUSTOM_VERSION=y
+        BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="6.18.42"
+        BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
+        BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/aarch64-virt/linux.config"
+        BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL=y
+        BR2_PACKAGE_HEAPTRACK=y
+        # BR2_TARGET_ROOTFS_TAR is not set
+        BR2_TARGET_ROOTFS_CPIO=y
+        """
+
+    def test_run(self):
+        cpio_file = os.path.join(self.builddir, "images", "rootfs.cpio")
+        kern = os.path.join(self.builddir, "images", "Image")
+        self.emulator.boot(arch="aarch64",
+                           kernel=kern,
+                           kernel_cmdline=["console=ttyAMA0"],
+                           options=["-M", "virt",
+                                    "-cpu", "cortex-a57",
+                                    "-m", "256M",
+                                    "-initrd", cpio_file])
+
+        self.emulator.login()
+
+        self.assertRunOk("/usr/bin/heaptrack --version")
+        self.assertRunOk("/usr/bin/heaptrack /bin/busybox")

---
base-commit: d4f3bea066947fdee49a12ebd84cba54a872d43f
change-id: 20260731-heaptrack-cd4f6f8fcec8

Best regards,

Signature · 8 lines
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
#1 lore raw

Hi Luca,

Thanks for the patch. I have few comments, see below.

Quoted 7 lines · Luca Ceresoli via buildroot

Add heaptrack, a memory allocation tracer toolkit.

This implementation builds all the command line components, not the heaptrack_gui graphical visualization program.

---

[...]

Quoted 12 lines
package/heaptrack/Config.in new file +6 -0
new file mode 100644
index 000000000000..12e33ff2c219
@@ -0,0 +1,37 @@
+config BR2_PACKAGE_HEAPTRACK
+	bool "heaptrack"
+	depends on BR2_INSTALL_LIBSTDCPP
+	depends on BR2_TOOLCHAIN_HAS_THREADS # boost, libunwind, elfutils
+	depends on BR2_USE_WCHAR # boost, elfutils
+	depends on BR2_TOOLCHAIN_SUPPORTS_ALWAYS_LOCKFREE_ATOMIC_INTS # 
Quoted 1 line

boost-filesystem

This package is failing to build with uclibc toolchains. See:
https://gitlab.com/jolivain/buildroot/-/pipelines/2762905573

heaptrack uses LM_ID_BASE which is not supported by uclibc:
https://invent.kde.org/sdk/heaptrack/-/blob/master/src/track/heaptrack_env.cpp#L33
https://github.com/wbx-github/uclibc-ng/blob/v1.0.59/include/dlfcn.h#L46-L54

You could either try to fix that issue, or disable the package for uclibc toolchains by adding:

depends on !BR2_TOOLCHAIN_USES_UCLIBC # no dlmopen() support

with the corresponding comment.

Quoted 32 lines
package/heaptrack/Config.in new file +24 -0
+	depends on BR2_PACKAGE_LIBUNWIND_ARCH_SUPPORTS # libunwind
+	depends on BR2_TOOLCHAIN_GCC_AT_LEAST_4_9 # libunwind
+	depends on !BR2_STATIC_LIBS # libunwind, elfutils
+	select BR2_PACKAGE_BOOST
+	select BR2_PACKAGE_BOOST_IOSTREAMS
+	select BR2_PACKAGE_BOOST_PROGRAM_OPTIONS
+	# For heaptrack_print
+	select BR2_PACKAGE_BOOST_FILESYSTEM
+	select BR2_PACKAGE_ELFUTILS
+	select BR2_PACKAGE_LIBUNWIND
+	select BR2_PACKAGE_ZLIB
+	help
+	  Heaptrack traces all memory allocations and annotates these
+	  events with stack traces. Dedicated analysis tools then allow
+	  you to interpret the heap memory profile to find hotspots,
+	  memory leaks, allocation hotspots and temporary allocations.
+
+	  Zstandard offers better (de)compression performance compared
+	  with gzip/zlib, making heaptrack faster and datafiles smaller,
+	  so enabling BR2_PACKAGE_ZSTD is recommended.
+
+	  https://apps.kde.org/heaptrack/
+
+comment "heaptrack needs a toolchain w/ C++, threads, wchar, gcc >= 

4.9, dynamic library"

package/heaptrack/Config.in new file +7 -0
+	depends on BR2_TOOLCHAIN_SUPPORTS_ALWAYS_LOCKFREE_ATOMIC_INTS
+	depends on BR2_PACKAGE_LIBUNWIND_ARCH_SUPPORTS
+	depends on !BR2_INSTALL_LIBSTDCPP \
+		|| !BR2_TOOLCHAIN_HAS_THREADS \
+		|| !BR2_USE_WCHAR \
+		|| !BR2_TOOLCHAIN_GCC_AT_LEAST_4_9 \
+		|| BR2_STATIC_LIBS

[...]

Quoted 17 lines
package/heaptrack/heaptrack.mk new file +10 -0
new file mode 100644
index 000000000000..ef12f9d189fe
@@ -0,0 +1,26 @@
+################################################################################
+#
+# heaptrack
+#
+################################################################################
+
+HEAPTRACK_SITE_METHOD = git
+HEAPTRACK_SITE = https://invent.kde.org/sdk/heaptrack.git
+HEAPTRACK_VERSION = 3e6cce3d210a6672fe6f92de0bed567c49b7a9c5
+HEAPTRACK_LICENSE = LGPL-2.1-or-later, GPL-2.0-or-later 

(heaptrack_interpret)

package/heaptrack/heaptrack.mk line 19

Quoted 9 lines
package/heaptrack/heaptrack.mk new file +9 -0
+HEAPTRACK_LICENSE_FILES = \
+	LICENSES/LGPL-2.1-only.txt \
+	LICENSES/GPL-2.0-or-later.txt \
+	LICENSES/LGPL-2.1-or-later.txt \
+	LICENSES/BSL-1.0.txt \
+	LICENSES/MIT.txt \
+	LICENSES/Apache-2.0.txt \
+	LICENSES/BSD-3-Clause.txt
+HEAPTRACK_DEPENDENCIES = host-pkgconf boost libunwind zlib elfutils

Could you sort dependencies, please?

HEAPTRACK_DEPENDENCIES = host-pkgconf boost elfutils libunwind zlib

package/heaptrack/heaptrack.mk

Quoted 2 lines
package/heaptrack/heaptrack.mk new file +1 -1
+HEAPTRACK_CONF_OPTS += -DHEAPTRACK_BUILD_GUI=OFF 
-DHEAPTRACK_BUILD_PRINT=ON

Since it's the first _CONF_OPTS assignement, could you remove the '+', please?

HEAPTRACK_CONF_OPTS = -DHEAPTRACK_BUILD_GUI=OFF ...

Quoted 44 lines
package/heaptrack/heaptrack.mk new file +6 -0
+
+ifeq ($(BR2_PACKAGE_ZSTD),y)
+HEAPTRACK_DEPENDENCIES += zstd
+endif
+
+$(eval $(cmake-package))
support/testing/tests/package/test_heaptrack.py new file +19 -0
new file mode 100644
index 000000000000..de3d5441efa4
@@ -0,0 +1,40 @@
+import os
+
+import infra.basetest
+
+
+class TestHeaptrack(infra.basetest.BRTest):
+    # infra.basetest.BASIC_TOOLCHAIN_CONFIG cannot be used as it does
+    # not include BR2_TOOLCHAIN_SUPPORTS_ALWAYS_LOCKFREE_ATOMIC_INTS
+    # needed by heaptrack.
+    config = \
+        """
+        BR2_aarch64=y
+        BR2_TOOLCHAIN_EXTERNAL=y
+        BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0"
+        BR2_LINUX_KERNEL=y
+        BR2_LINUX_KERNEL_CUSTOM_VERSION=y
+        BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="6.18.42"
+        BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
+        

BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/qemu/aarch64-virt/linux.config"

support/testing/tests/package/test_heaptrack.py new file +8 -0
+        BR2_LINUX_KERNEL_NEEDS_HOST_OPENSSL=y
+        BR2_PACKAGE_HEAPTRACK=y
+        # BR2_TARGET_ROOTFS_TAR is not set
+        BR2_TARGET_ROOTFS_CPIO=y
+        """
+
+    def test_run(self):
+        cpio_file = os.path.join(self.builddir, "images", 

"rootfs.cpio")

support/testing/tests/package/test_heaptrack.py new file +3 -0
+        kern = os.path.join(self.builddir, "images", "Image")
+        self.emulator.boot(arch="aarch64",
+                           kernel=kern,

support/testing/tests/package/test_heaptrack.py line 40

Quoted 10 lines
support/testing/tests/package/test_heaptrack.py new file +10 -0
+                           kernel_cmdline=["console=ttyAMA0"],
+                           options=["-M", "virt",
+                                    "-cpu", "cortex-a57",
+                                    "-m", "256M",
+                                    "-initrd", cpio_file])
+
+        self.emulator.login()
+
+        self.assertRunOk("/usr/bin/heaptrack --version")
+        self.assertRunOk("/usr/bin/heaptrack /bin/busybox")

Maybe we could make this test slightly more complex, to have a bit more allocations, and also printing the recorded data.

For example:

self.assertRunOk("heaptrack -o /tmp/ls.heaptrack ls -al /") self.assertRunOk("heaptrack_print --file /tmp/ls.heaptrack.gz")

Quoted 9 lines

---
base-commit: d4f3bea066947fdee49a12ebd84cba54a872d43f
change-id: 20260731-heaptrack-cd4f6f8fcec8

Best regards,
--
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Could you send an updated patch addressing those issues, please?

Best regards,

Julien.
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot

#2 lore raw

Hello Julien,

Quoted 17 lines · Julien Olivain
Quoted 9 lines

+++ b/package/heaptrack/Config.in

package/heaptrack/Config.in new file +6 -0
@@ -0,0 +1,37 @@
+config BR2_PACKAGE_HEAPTRACK
+	bool "heaptrack"
+	depends on BR2_INSTALL_LIBSTDCPP
+	depends on BR2_TOOLCHAIN_HAS_THREADS # boost, libunwind, elfutils
+	depends on BR2_USE_WCHAR # boost, elfutils
+	depends on BR2_TOOLCHAIN_SUPPORTS_ALWAYS_LOCKFREE_ATOMIC_INTS #

boost-filesystem

This package is failing to build with uclibc toolchains. See:
https://gitlab.com/jolivain/buildroot/-/pipelines/2762905573

heaptrack uses LM_ID_BASE which is not supported by uclibc:
https://invent.kde.org/sdk/heaptrack/-/blob/master/src/track/heaptrack_env.cpp#L33
https://github.com/wbx-github/uclibc-ng/blob/v1.0.59/include/dlfcn.h#L46-L54

Quoted 6 lines

You could either try to fix that issue, or disable the package for uclibc toolchains by adding:

depends on !BR2_TOOLCHAIN_USES_UCLIBC # no dlmopen() support

with the corresponding comment.

Thanks for the review and testing!

Quoted 3 lines
Quoted 1 line

+HEAPTRACK_DEPENDENCIES = host-pkgconf boost libunwind zlib elfutils

Could you sort dependencies, please?

Ah, sure! By a quick look into existing packages I had concluded sorting is not required.

I'm sending v4 with all the changes you suggested.

Luca

--
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot