support/scripts/check-host-libs: add new check on host binaries/libs
PATCH v2 1/1 support/scripts/check-host-libs: add new check on host binaries/libs
One frequent issue in Buildroot is that when building host libraries or applications, the build system of the package detects some libraries provided by the system, and happily links to them, without Buildroot knowing. Sometimes this doesn't cause any problem, but sometimes this causes issues, and we're regularly eliminating such mis-detection by forcing those packages to not detect the system libraries that have not been built by Buildroot.
The new script check-host-libs added in this commit, which is executed during the host-finalize step at the end of the build is an attempt at detecting at least some of these situations.
What it does is that at the end of the build, it verifies that all binaries and libraries in $(HOST_DIR) only have shared library dependencies on libraries that are in Buildroot $(HOST_DIR), to the exception of the C library, for which we of course use the system C library.
For example, if the binary output/host/bin/plop is linked against libpng, but libpng was not built and installed by Buildroot, the build will now fail with:
ERROR: in /home/thomas/projets/buildroot/output/host/bin/plop, libpng16.so.16 unknown
make: *** [Makefile:715: host-finalize] Error 1
The script includes an allowlist of libraries provided by the C library. It is potentially possible that this list might need to be extended to cover all systems/distributions/C libraries, but only wider testing of this script will help detect such cases.
It is worth mentioning that for now this script is executed only once at the end of the build. This means that if a package A gets built, detects and uses a system library libfoo and uses it, and then by chance later Buildroot package B builds and installs libfoo into HOST_DIR/lib, this script will believe that package A is correct, as it finds libfoo in HOST_DIR/lib, even though while package A was being built, the libfoo being detected was the system one. Detecting this would require running check-host-libs at the end of each package build, but that would imply re-checking over and over again all host binaries/libraries, which could have a noticeable impact on the build time. So for now, we simply check at the end of the build, which should already help to detect a lot of interesting bogus situations.
---
It would be very useful if a few people could apply this patch to their local tree, run their usual build, and see how it behaves. This way, I can get some feedback to address the most obvious issues before it gets merged and starts causing build failures in the autobuilders.
The v1 of this patch was sent on Sep 20, 2022, almost 4 years ago :-)
Changes since v1:
- Replaced the per-file file --mime-type checks with direct ELF magic detection using Bash read -N 4, significantly reducing scan time.
- Switched file traversal to NUL-delimited find -print0 output, safely handling paths containing whitespace.
- Suppressed harmless readelf errors for valid ELF object files without a dynamic section, such as the Go test fixtures mentioned during review.
- Added librt.so*, libutil.so*, and libresolv.so* to the system-library allowlist.
- Quoted file and host-directory paths where appropriate.
- Fixed shellcheck issues
--- Makefile | 1 + support/scripts/check-host-libs | 37 +++++++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+) create mode 100755 support/scripts/check-host-libs
Makefile +1 -0
@@ -748,6 +748,7 @@ host-finalize: $(PACKAGES) $(HOST_DIR) $(HOST_DIR_SYMLINK) PER_PACKAGE_DIR=$(PER_PACKAGE_DIR) \ $(TOPDIR)/support/scripts/fix-rpath staging $(call ppd-fixup-paths,$(BASE_DIR)) + ./support/scripts/check-host-libs $(HOST_DIR) .PHONY: staging-finalize staging-finalize: $(STAGING_DIR_SYMLINK)
support/scripts/check-host-libs new file +37 -0
@@ -0,0 +1,37 @@ +#!/bin/bash + +HOST_DIR=$1 + +if test -z "${HOST_DIR}" ; then + echo "usage: check-host-libs HOST_DIR" + exit 1 +fi + +bailout="no" +ELF=$'\x7fELF' + +while IFS= read -r -d '' f; do + read -r -N 4 magic < "${f}" + if test "${magic}" != "${ELF}" ; then + continue + fi + for lib in $(LC_ALL=C readelf -d "${f}" 2>/dev/null | \ + sed -n 's,.*Shared library: \[\(.*\)\].*,\1,p'); do + case ${lib} in + libc.so*|libm.so*|libstdc++.so*|libpthread.so*|libgcc_s.so*|libdl.so*|ld-*|libgomp.so*|libcrypt.so*|libcrypto.so*|libatomic.so*|librt.so*|libutil.so*|libresolv.so*) + continue + ;; + *) + if test -e "${HOST_DIR}/lib/${lib}" ; then + continue + fi + echo "ERROR: in ${f}, ${lib} unknown" + bailout="yes" + ;; + esac + done +done < <(find "${HOST_DIR}"/*bin "${HOST_DIR}"/lib* -type f -print0) + +if test "${bailout}" = "yes" ; then + exit 1 +fi
Signature · 6 lines
2.55.0 _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Hi Thomas,
Quoted 43 lines · Thomas Petazzoni via buildroot
One frequent issue in Buildroot is that when building host libraries or applications, the build system of the package detects some libraries provided by the system, and happily links to them, without Buildroot knowing. Sometimes this doesn't cause any problem, but sometimes this causes issues, and we're regularly eliminating such mis-detection by forcing those packages to not detect the system libraries that have not been built by Buildroot.
The new script check-host-libs added in this commit, which is executed during the host-finalize step at the end of the build is an attempt at detecting at least some of these situations.
What it does is that at the end of the build, it verifies that all binaries and libraries in $(HOST_DIR) only have shared library dependencies on libraries that are in Buildroot $(HOST_DIR), to the exception of the C library, for which we of course use the system C library.
For example, if the binary output/host/bin/plop is linked against libpng, but libpng was not built and installed by Buildroot, the build will now fail with:
ERROR: in
/home/thomas/projets/buildroot/output/host/bin/plop, libpng16.so.16 unknown
make: *** [Makefile:715: host-finalize] Error 1The script includes an allowlist of libraries provided by the C library. It is potentially possible that this list might need to be extended to cover all systems/distributions/C libraries, but only wider testing of this script will help detect such cases.
It is worth mentioning that for now this script is executed only once at the end of the build. This means that if a package A gets built, detects and uses a system library libfoo and uses it, and then by chance later Buildroot package B builds and installs libfoo into HOST_DIR/lib, this script will believe that package A is correct, as it finds libfoo in HOST_DIR/lib, even though while package A was being built, the libfoo being detected was the system one. Detecting this would require running check-host-libs at the end of each package build, but that would imply re-checking over and over again all host binaries/libraries, which could have a noticeable impact on the build time. So for now, we simply check at the end of the build, which should already help to detect a lot of interesting bogus situations.
Quoted 6 lines
Signed-off-by Thomas Petazzoni <[email protected]>---
It would be very useful if a few people could apply this patch to their local tree, run their usual build, and see how it behaves. This way, I can get some feedback to address the most obvious issues before it gets merged and starts causing build failures in the autobuilders.
I gave a try to your checker on some personal project, and I get plenty of warnings:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Finalizing host directory
./support/scripts/check-host-libs /home/alexis/src/neon-beat/neon-beat-controller/output/host
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-cgtop, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-creds, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-cgls, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-escape, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-stdio-bridge, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-tty-ask-password-agent, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/cpack, libidn2.so.0 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/cpack, libmd.so.0 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-cat, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-detect-virt, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-hwdb, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-mount, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-vpick, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-run, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/varlinkctl, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-socket-activate, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/floppyd, libX11.so.6 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-ac-power, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-path, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemctl, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/cmake, libidn2.so.0 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/cmake, libmd.so.0 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-delta, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/busctl, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-id128, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/udevadm, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-notify, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-machine-id-setup, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-ask-password, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-nspawn, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/journalctl, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/ctest, libidn2.so.0 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/ctest, libmd.so.0 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-tmpfiles, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/sbin/sfdisk, libreadline.so.8 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/sbin/fdisk, libreadline.so.8 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/udev/iocost, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/udev/mtd_probe, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/udev/dmi_memory_id, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/udev/v4l_id, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/udev/cdrom_id, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/udev/ata_id, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/udev/fido_id, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/udev/scsi_id, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/system-generators/systemd-run-generator, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/system-generators/systemd-debug-generator, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/system-generators/systemd-ssh-generator, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/system-generators/systemd-getty-generator, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/system-generators/systemd-fstab-generator, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/system-generators/systemd-system-update-generator, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/system-generators/systemd-tpm2-generator, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-network-generator, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-socket-proxyd, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-ssh-proxy, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-sleep, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-executor, libsystemd-core-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-executor, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-reply-password, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-cgroups-agent, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-fsck, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-battery-check, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd, libsystemd-core-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/libsystemd-core-256.so, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-shutdown, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-sysctl, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-sulogin-shell, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-makefs, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-remount-fs, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-growfs, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-update-done, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-boot-check-no-failures, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/systemd/systemd-journald, libsystemd-shared-256.so unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/erlang/lib/wx-2.4.1/priv/wxe_driver.so, libwx_gtk3u_stc-3.2.so.0 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/erlang/lib/wx-2.4.1/priv/wxe_driver.so, libwx_gtk3u_xrc-3.2.so.0 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/erlang/lib/wx-2.4.1/priv/wxe_driver.so, libwx_gtk3u_html-3.2.so.0 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/erlang/lib/wx-2.4.1/priv/wxe_driver.so, libwx_baseu_xml-3.2.so.0 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/erlang/lib/wx-2.4.1/priv/wxe_driver.so, libwx_gtk3u_core-3.2.so.0 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/erlang/lib/wx-2.4.1/priv/wxe_driver.so, libwx_baseu-3.2.so.0 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/erlang/lib/wx-2.4.1/priv/wxe_driver.so, libwx_gtk3u_gl-3.2.so.0 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/erlang/lib/wx-2.4.1/priv/wxe_driver.so, libwx_gtk3u_aui-3.2.so.0 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/erlang/lib/wx-2.4.1/priv/wxe_driver.so, libwx_gtk3u_webview-3.2.so.0 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/erlang/lib/wx-2.4.1/priv/erl_gl.so, libGL.so.1 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/erlang/lib/wx-2.4.1/priv/erl_gl.so, libGLU.so.1 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/erlang/lib/odbc-2.14.2/priv/bin/odbcserver, libodbc.so.2 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/python3.12/lib-dynload/_gdbm.cpython-312-x86_64-linux-gnu.so, libgdbm.so.6 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/python3.12/lib-dynload/readline.cpython-312-x86_64-linux-gnu.so, libreadline.so.8 unknown
ERROR: in /home/alexis/src/neon-beat/neon-beat-controller/output/host/lib/python3.12/lib-dynload/_dbm.cpython-312-x86_64-linux-gnu.so, libgdbm_compat.so.4 unknown make[2]: *** [Makefile:747: host-finalize] Error 1
make[1]: *** [Makefile:23: _all] Error 2
make: *** [Makefile:24: neon-beat-controller] Error 2
I did not take a look at all of those, but the first one on systemd-cgtop looks wrong.
$ ldd /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/systemd-cgtop
linux-vdso.so.1 (0x00007fe696105000)
libsystemd-shared-256.so => /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/../lib/systemd/libsystemd-shared-256.so (0x00007fe695c00000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007fe695800000)
libcap.so.2 => /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/../lib/systemd/../../lib/libcap.so.2 (0x00007fe69609d000)
libcrypt.so.2 => /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/../lib/systemd/../../lib/libcrypt.so.2 (0x00007fe69605d000)
libmount.so.1 => /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/../lib/systemd/../../lib/libmount.so.1 (0x00007fe695fe6000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007fe695ac9000)
/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007fe696107000)
libblkid.so.1 => /home/alexis/src/neon-beat/neon-beat-controller/output/host/bin/../lib/systemd/../../lib/libblkid.so.1 (0x00007fe695a65000) $ls -l output/host/lib/systemd/libsystemd-shared-256.so
-rwxr-xr-x 1 alexis alexis 3713824 Aug 25 09:11 output/host/lib/systemd/libsystemd-shared-256.so
$readelf -d output/host/bin/systemd-cgtop
Dynamic section at offset 0x8ae0 contains 29 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libsystemd-shared-256.so] 0x0000000000000001 (NEEDED) Shared library: [libc.so.6] 0x000000000000001d (RUNPATH) Library runpath: [$ORIGIN/../lib:$ORIGIN/../lib/systemd] 0x000000000000000c (INIT) 0x3000 0x000000000000000d (FINI) 0x64a0 0x0000000000000019 (INIT_ARRAY) 0x9910 0x000000000000001b (INIT_ARRAYSZ) 8 (bytes) 0x000000000000001a (FINI_ARRAY) 0x9918
[...]
The main issue seems to be due to libsystemd-shared-256.so being in $(HOST_DIR)/lib/systemd and not $(HOST_DIR)/lib. Shouldn't check-host-libs also parse this RUNPATH ?
Alexis
_______________________________________________
buildroot mailing list
[email protected]
https://lists.buildroot.org/mailman/listinfo/buildroot
Thomas, All,
On 2026-08-24 23:34 +0200, Thomas Petazzoni spake thusly:
[--SNIP--]
Quoted 5 lines
What it does is that at the end of the build, it verifies that all binaries and libraries in $(HOST_DIR) only have shared library dependencies on libraries that are in Buildroot $(HOST_DIR), to the exception of the C library, for which we of course use the system C library.
[--SNIP--]
I was wondering why you would not extend check-host-rpath, see my tentative, totally untested patch at the end...
Also, check-host-rpath is run after each package install, so it will catch executables installed without their dependent libraries, should they later be installed.
Otherwise, some comments below...
support/scripts/check-host-libs line 3
Quoted 9 lines
support/scripts/check-host-libs new file +3
-0@@ -0,0 +1,37 @@ +#!/bin/bash + +HOST_DIR=$1
Double-quote variable expansion (shellcheck should have reported that one, I think).
[--SNIP--]
support/scripts/check-host-libs line 10
Quoted 1 line
support/scripts/check-host-libs new file +1
-0+bailout="no"
There is a construct that I tend to use nowadas, which is to use true/false, rather than 0/1 or yes/no, because that can be reused without a test;
success=true
for loop; do
if [ conditiion ]; then
success=false
fi
done
${success}
[--SNIP--]
support/scripts/check-host-libs line 20
Quoted 1 line
support/scripts/check-host-libs new file +1
-0+ case ${lib} in
Ditto, double-quote around variable expansion.
And here's a tentative, totally untested patch to introduce that in check-host-rpath (indented so that patchwork does not see it):
diff --git a/support/scripts/check-host-rpath b/support/scripts/check-host-rpath index 41aa0aa1ed..e1fdf16829 100755
--- a/support/scripts/check-host-rpath
+++ b/support/scripts/check-host-rpath
@@ -21,6 +21,14 @@ main() {
while read file; do
is_elf "${file}" || continue
elf_needs_rpath "${file}" "${hostdir}" || continue
+ missing_libs="$(get_missing_libs "${file}" "${hostdir}")"
+ if [ "${missing_libs}" ]; then
+ ret=1
+ printf "***\n"
+ printf "*** ERROR: package %s uses libs not in HOST_DIR:\n" "${pkg}"
+ # shellcheck disable=SC2086 # we need the word splitting
+ printf ' - %s\n' ${missing_libs}
+ fi
check_elf_has_rpath "${file}" "${hostdir}" "${perpackagedir}" && continue if [ ${ret} -eq 0 ]; then
ret=1
@@ -57,16 +65,44 @@ is_elf() {
elf_needs_rpath() {
local file="${1}"
local hostdir="${2}"
+
+ [ -n "$(get_libs "${file}" "${hostdir}")" ]
+}
+
+# This function returns all non-toolchain libs tha tare not in HOST_DIR +get_missing_libs() {
+ local file="${1}"
+ local hostdir="${2}"
+ local lib
+
+ get_libs "${file}" "${hostdir}" \
+ | while read lib; do
+ if [ -not -e "${hostdir}/lib/${lib}" ]; then
+ printf '%s\n' "${lib}"
+ fi
+ done
+}
+
+# This function returns the list of non-toolchain libraries that an ELF +# file has as DT_NEEDED
+get_libs() {
+ local file="${1}"
+ local hostdir="${2}"
local lib
while read lib; do
- [ -e "${hostdir}/lib/${lib}" ] && return 0
+ case "${lib}" in
+ libc.so*|libm.so*|libstdc++.so*|libpthread.so*|libgcc_s.so*|libdl.so*|ld-*|libgomp.so*|libcrypt.so*|libcrypto.so*|libatomic.so*|librt.so*|libutil.so*|libresolv.so*)
+ continue
+ ;;
+ *)
+ printf '%s\n' "${lib}"
+ ;;
+ esac
done < <( readelf -d "${file}" 2>/dev/null \
|sed -r -e '/^.* \(NEEDED\) .*Shared library: \[(.+)\]$/!d;' \
-e 's//\1/;' \
)
-
- return 1
}
# This function checks whether at least one of the RPATH of the given
Signature · 10 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------' _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot
Hello,
Quoted 6 lines · Yann E. MORIN
I was wondering why you would not extend check-host-rpath, see my tentative, totally untested patch at the end...
Also, check-host-rpath is run after each package install, so it will catch executables installed without their dependent libraries, should they later be installed.
Ah, good idea. Then maybe check-host-rpath should be renamed into check-host-bins or something like that, because it would no longer check just RPATH, but other aspects of host-installed bins and libs?
Quoted 4 lines
Quoted 1 line
+HOST_DIR=$1
Double-quote variable expansion (shellcheck should have reported that one, I think).
I just passed "./utils/docker-run shellcheck" on it, maybe there's some additional shellcheck options to pass to have stricter checks.
Quoted 11 lines
There is a construct that I tend to use nowadas, which is to use true/false, rather than 0/1 or yes/no, because that can be reused without a test;
success=true
for loop; do
if [ conditiion ]; then
success=false
fi
done
${success}
OK, nice.
Quoted 60 lines
And here's a tentative, totally untested patch to introduce that in check-host-rpath (indented so that patchwork does not see it):
diff --git a/support/scripts/check-host-rpath b/support/scripts/check-host-rpath index
41aa0aa1ed..e1fdf16829100755
--- a/support/scripts/check-host-rpath
+++ b/support/scripts/check-host-rpath
@@ -21,6 +21,14 @@main(){
while read file; do
is_elf"${file}" || continue
elf_needs_rpath"${file}" "${hostdir}" || continue
+ missing_libs="$(get_missing_libs "${file}" "${hostdir}")"
+ if [ "${missing_libs}" ]; then
+ ret=1
+ printf "***\n"
+ printf "*** ERROR: package %s uses libs not in HOST_DIR:\n" "${pkg}"
+ # shellcheck disable=SC2086 # we need the word splitting
+ printf ' - %s\n' ${missing_libs}
+ fi
check_elf_has_rpath"${file}" "${hostdir}" "${perpackagedir}" && continue if [ ${ret} -eq 0 ]; then
ret=1
@@ -57,16 +65,44 @@is_elf(){
elf_needs_rpath(){
local file="${1}"
local hostdir="${2}"
+
+ [ -n "$(get_libs "${file}" "${hostdir}")" ]
+}
+
+# This function returns all non-toolchain libs tha tare not inHOST_DIR+get_missing_libs() {
+ local file="${1}"
+ local hostdir="${2}"
+ local lib
+
+get_libs"${file}" "${hostdir}" \
+ | while read lib; do
+ if [ -not -e "${hostdir}/lib/${lib}" ]; then
+ printf '%s\n' "${lib}"
+ fi
+ done
+}
+
+# This function returns the list of non-toolchain libraries that an ELF +# file has asDT_NEEDED
+get_libs() {
+ local file="${1}"
+ local hostdir="${2}"
local libwhile read lib; do
- [ -e "${hostdir}/lib/${lib}" ] && return 0
+ case "${lib}" in
+ libc.so*|libm.so*|libstdc++.so*|libpthread.so*|libgcc_s.so*|libdl.so*|ld-*|libgomp.so*|libcrypt.so*|libcrypto.so*|libatomic.so*|librt.so*|libutil.so*|libresolv.so*)
+ continue
+ ;;
+ *)
+ printf '%s\n' "${lib}"
+ ;;
Quoted 8 lines
+ esac
done < <( readelf -d "${file}" 2>/dev/null \
|sed -r -e '/^.* \(NEEDED\) .*Shared library: \[(.+)\]$/!d;' \
-e 's//\1/;' \
)
-
- return 1
}
OK, I'll review this and try to get back with a v3... hopefully earlier than in 4 years!
Thanks for the prompt review and feedback!
Thomas
Signature · 7 lines
Thomas Petazzoni, co-owner and CEO, Bootlin Embedded Linux and Kernel engineering and training https://bootlin.com _______________________________________________ buildroot mailing list [email protected] https://lists.buildroot.org/mailman/listinfo/buildroot