classes/cmake: don't put RPATHs to the build tree in binaries
RFC PATCH 1/1 classes/cmake: don't put RPATHs to the build tree in binaries
By default, CMake will compile binaries with an RPATH pointing at the build path, and then on install will "relink" the binaries with an RPATH pointing to the install path.
However, when CMake says "relink" it actually means "edit if possible", using an internal copy of chrpath. This introduces non-determinism in several ways:
- The build path is used to generate the build-id. We've worked around this previously by ensuring that RPATHs are relative to the binary[1].
- Not all upstreams respect this option, so some recipes need extra tweaking to stop them injecting absolute build-time RPATHs[2]
- Whilst the final RPATH may be empty, using chrpath is _not_ relinking so different length build paths can result in binaries with different amounts of now-redundant padding. These binaries are non-reproducible.
Instead of using relative build-time RPATHs and then dealing with other issues one by one, we can just tell CMake to not use built-time RPATHs at all. We don't execute binaries directly from the build tree, so they don't serve any purpose and are just a source of non-determinism.
[1] oe-core 44e77d3f97a ("classes/cmake: Use relative RPATHs")
[2] oe-core d96e0458b69 ("lldb: don't build rpaths into binaries")
--- meta/classes-recipe/cmake.bbclass | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
meta/classes-recipe/cmake.bbclass +4 -2
@@ -163,10 +163,12 @@ set( ENV{QT_CONF_PATH} ${WORKDIR}/qt.conf ) # We need to set the rpath to the correct directory as cmake does not provide any # directory as rpath by default + set( CMAKE_INSTALL_RPATH ${OECMAKE_RPATH} ) -# Use RPATHs relative to build directory for reproducibility -set( CMAKE_BUILD_RPATH_USE_ORIGIN ON ) +# Don't build build RPATHS into the binaries, they're a source of non-determinism +# even when stripped out of the installed files +set (CMAKE_SKIP_BUILD_RPATH ON) # Use our cmake modules list(APPEND CMAKE_MODULE_PATH "${STAGING_DATADIR}/cmake/Modules/")
Signature · 1 line
2.43.0
Quoted 3 lines · Ross Burton via lists.openembedded.org
By default, CMake will compile binaries with an RPATH pointing at the build path, and then on install will "relink" the binaries with an RPATH pointing to the install path.
Context here is that the clang recipes love putting absolute build paths into the RPATHs, destroying reproducibility. This is a sledgehammer to stop it happening globally, if people think it’s a bit too heavy for this late in the cycle I can do it in the clang recipes.
Ross
Quoted 20 lines · Ross Burton via lists.openembedded.org
By default, CMake will compile binaries with an RPATH pointing at the build path, and then on install will "relink" the binaries with an RPATH pointing to the install path.
However, when CMake says "relink" it actually means "edit if possible", using an internal copy of chrpath. This introduces non-determinism in several ways:
- The build path is used to generate the build-id. We've worked around this previously by ensuring that RPATHs are relative to the binary[1].
- Not all upstreams respect this option, so some recipes need extra tweaking to stop them injecting absolute build-time RPATHs[2]
- Whilst the final RPATH may be empty, using chrpath is _not_ relinking so different length build paths can result in binaries with different amounts of now-redundant padding. These binaries are non-reproducible.Instead of using relative build-time RPATHs and then dealing with other issues one by one, we can just tell CMake to not use built-time RPATHs at all. We don't execute binaries directly from the build tree, so they don't serve any purpose and are just a source of non-determinism.
Quoted 5 lines
[1] oe-core
44e77d3f97a("classes/cmake: Use relative RPATHs")
[2] oe-cored96e0458b69("lldb: don't build rpaths into binaries")Signed-off-by Ross Burton <[email protected]>---
Hi Ross,
I gave a try to this patch, even if it's still an RFC. We have a few build issues.
One on clang-native:
ERROR: clang-native-21.1.1-r0 do_compile: Execution of '/srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/x86_64-linux/clang-native/21.1.1/temp/run.do_compile.3920753' failed with exit code 127
...
| FAILED: [code=127] tools/clang/tools/extra/clang-tidy/misc/Confusables.inc /srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/x86_64-linux/clang-native/21.1.1/build/tools/clang/tools/extra/clang-tidy/misc/Confusables.inc | cd /srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/x86_64-linux/clang-native/21.1.1/build/tools/clang/tools/extra/clang-tidy/misc && /srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/x86_64-linux/clang-native/21.1.1/build/bin/clang-tidy-confusable-chars-gen /srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work-shared/llvm-project-source-21.1.1-r0/sources/llvm-project-21.1.1.src/clang-tools-extra/clang-tidy/misc/ConfusableTable/confusables.txt /srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/x86_64-linux/clang-native/21.1.1/build/tools/clang/tools/extra/clang-tidy/misc/Confusables.inc | /srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/x86_64-linux/clang-native/21.1.1/build/bin/clang-tidy-confusable-chars-gen: error while loading shared libraries: libLLVM.so.21.1: cannot open shared object file: No such file or directory
...
| WARNING: exit code 127 from a shell command.
NOTE: recipe clang-native-21.1.1-r0: task do_compile: Failed
One on libical:
ERROR: libical-3.0.20-r0 do_compile: Execution of '/srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/core2-32-poky-linux/libical/3.0.20/temp/run.do_compile.2809930' failed with exit code 1
...
| /srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/core2-32-poky-linux/libical/3.0.20/recipe-sysroot-native/usr/bin/i686-poky-linux/../../libexec/i686-poky-linux/gcc/i686-poky-linux/15.2.0/ld: warning: libical.so.3, needed by /srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/core2-32-poky-linux/libical/3.0.20/build/lib/libical-glib.so, not found (try using -rpath or -rpath-link) | /srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/core2-32-poky-linux/libical/3.0.20/recipe-sysroot-native/usr/bin/i686-poky-linux/../../libexec/i686-poky-linux/gcc/i686-poky-linux/15.2.0/ld: /srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/core2-32-poky-linux/libical/3.0.20/build/lib/libical-glib.so: undefined reference to `icalparameter_new_from_string' | /srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/core2-32-poky-linux/libical/3.0.20/recipe-sysroot-native/usr/bin/i686-poky-linux/../../libexec/i686-poky-linux/gcc/i686-poky-linux/15.2.0/ld: /srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/core2-32-poky-linux/libical/3.0.20/build/lib/libical-glib.so: undefined reference to `icaltimezone_get_builtin_timezone' | /srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/core2-32-poky-linux/libical/3.0.20/recipe-sysroot-native/usr/bin/i686-poky-linux/../../libexec/i686-poky-linux/gcc/i686-poky-linux/15.2.0/ld: /srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/core2-32-poky-linux/libical/3.0.20/build/lib/libical-glib.so: undefined reference to `icalparameter_get_x' | /srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/core2-32-poky-linux/libical/3.0.20/recipe-sysroot-native/usr/bin/i686-poky-linux/../../libexec/i686-poky-linux/gcc/i686-poky-linux/15.2.0/ld: /srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/core2-32-poky-linux/libical/3.0.20/build/lib/libical-glib.so: undefined reference to `icalparameter_get_scheduleforcesend' | /srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/core2-32-poky-linux/libical/3.0.20/recipe-sysroot-native/usr/bin/i686-poky-linux/../../libexec/i686-poky-linux/gcc/i686-poky-linux/15.2.0/ld: /srv/pokybuild/yocto-worker/qemux86-alt/build/build/tmp/work/core2-32-poky-linux/libical/3.0.20/build/lib/libical-glib.so: undefined reference to `icalcomponent_new_xstandard'
...
https://autobuilder.yoctoproject.org/valkyrie/#/builders/20/builds/2405
https://autobuilder.yoctoproject.org/valkyrie/#/builders/19/builds/2426
https://autobuilder.yoctoproject.org/valkyrie/#/builders/60/builds/2429
Thanks,
Mathieu
Signature · 3 lines
Mathieu Dubois-Briand, Bootlin Embedded Linux and Kernel engineering https://bootlin.com