Improve native/cross reproducibility
cover Improve native/cross reproducibility
This patch series is not intended for merge. I only send it out to highlight where the problems are and to get some discussion going on how/if we want to improve the sitation.
This is a patch series that tries to improve the reproducibility of the native/cross binaries when building in different directories. This has been tested on a Fedora 35 system which uses gcc 11.2.1 at the time of writing.
The RUNTIME hack is questionable, maybe there is a better way to enforce a fixed RUNTIME entry size during linking. It probably breaks for recipes that do additional rpath manipulations at link-time.
If in the end we do come up with a solution, then it should be tested on the autobuilders, since otherwise this is going to degrade overtime. It would then be important that the build paths are of significantly different lengths. TMPDIR=/tmp/sysrootA/ and TMPDIR=/tmp/sysrootB/ will *not* uncover all rpath problems.
The end result of this patch series is that I can build python3-native in two different build paths, and the resuling sysroot-components/x86_64/ directories are identical, except for the 'fixmepath.cmd' files, which are not included in the hash equiv calculations. Even so, there remains a lot of other native builds that are going to need to be fixed in similar ways as the ones in this patch series.
For my images to build I had to avoid the rpath-hack for icedtea7-native and openjdk-8-native.
/Jacob
Jacob Kroon (9):
bitbake.conf: Pad rpath and remove build ID in native binaries
libtool: Improve native reproducibility
openssl: Improve native reproducibility perl/perlcross: Improve native reproducability
pkgconfig: Improve native reproducibility
ncurses: Improve native reproducibility
util-linux: Improve native reproducibility
python3: Improve native reproducibility bitbake.conf: Avoid rpath hack for Java recipes
meta/classes/chrpath.bbclass | 3 +
meta/conf/bitbake.conf | 8 +-
...sysroot-and-debug-prefix-map-from-co.patch | 78 -------------------
.../openssl/openssl/strip-buildinfo.patch | 13 ++++
.../openssl/openssl_3.0.0.bb | 10 +--
meta/recipes-core/ncurses/ncurses.inc | 4 +
.../util-linux/util-linux_2.37.2.bb | 2 +-
.../libtool/libtool-native_2.4.6.bb | 1 +
...ism.patch => perl-cross-determinism.patch} | 0
.../perl-cross/perlcross_1.3.6.bb | 4 +-
meta/recipes-devtools/perl/perl_5.34.0.bb | 5 ++
.../pkgconfig/pkgconfig_git.bb | 1 +
.../python/python3/determinism.patch | 15 ++++
.../recipes-devtools/python/python3_3.10.0.bb | 8 ++
14 files changed, 65 insertions(+), 87 deletions(-)
delete mode 100644 meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch
create mode 100644 meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch
rename meta/recipes-devtools/perl-cross/files/{determinism.patch => perl-cross-determinism.patch} (100%)
create mode 100644 meta/recipes-devtools/python/python3/determinism.patchRFC PATCH 1/9 bitbake.conf: Pad rpath and remove build ID in native binaries
Try to make sure that the RUNTIME dynamic entry size is the same for all binaries produced with the native compiler. This is necessary in order to produce identical binaries when using differently sized buildpaths. I've tried using only patchelf, and keeping the linker flags as they are, but I am unable to produce identical binaries. Has anyone else managed to do this with patchelf ? If not, maybe we can write a new tool that can handle it ?
The build-id also needs to be removed since it is calculated based on the data present at link time. This includes STAGING_LIBDIR_NATIVE and STAGING_BASE_LIBDIR_NATIVE. Both will differ and they need to be temporarily preserved since some recipes will execute the binaries during do_install() (for example python3-native). Later on these are removed in chrpath.bbclass.
This hack is the first step for producing identical native binaries when using different build paths. 'zstd-native' is a working example.
--- meta/classes/chrpath.bbclass | 3 +++ meta/conf/bitbake.conf | 6 +++++- 2 files changed, 8 insertions(+), 1 deletion(-)
meta/classes/chrpath.bbclass +3 -0
@@ -24,6 +24,9 @@ def process_file_linux(cmd, fpath, rootdir, baseprefix, tmpdir, d, break_hardlin new_rpaths = [] modified = False for rpath in rpaths: + if rpath.startswith('rpath-padding-'): + modified = True + continue # If rpath is already dynamic copy it to new_rpath and continue if rpath.find("$ORIGIN") != -1: new_rpaths.append(rpath)
meta/conf/bitbake.conf +5 -1
@@ -580,6 +580,8 @@ BUILDSDK_CXXFLAGS = "${BUILDSDK_CFLAGS}" export CXXFLAGS = "${TARGET_CXXFLAGS}" TARGET_CXXFLAGS = "${TARGET_CFLAGS}" +RPATH_PADDING ?= "rpath-padding-${@'x' * (512 - len(d.expand('${STAGING_LIBDIR_NATIVE}:${STAGING_BASE_LIBDIR_NATIVE}:rpath-padding-')))}" +RPATH_PADDING_FLAG ?= "-Wl,-rpath,${RPATH_PADDING}" export BUILD_LDFLAGS = "-L${STAGING_LIBDIR_NATIVE} \ -L${STAGING_BASE_LIBDIR_NATIVE} \ -Wl,--enable-new-dtags \
@@ -587,7 +589,9 @@ export BUILD_LDFLAGS = "-L${STAGING_LIBDIR_NATIVE} \ -Wl,-rpath-link,${STAGING_BASE_LIBDIR_NATIVE} \ -Wl,-rpath,${STAGING_LIBDIR_NATIVE} \ -Wl,-rpath,${STAGING_BASE_LIBDIR_NATIVE} \ - -Wl,-O1" + ${RPATH_PADDING_FLAG} \ + -Wl,-O1 \ + -Wl,--build-id=none" BUILDSDK_LDFLAGS = "-Wl,-O1"
RFC PATCH 2/9 libtool: Improve native reproducibility
In reply to Jacob KroonAvoid encoding build-specific paths in the resulting binaries.
--- meta/recipes-devtools/libtool/libtool-native_2.4.6.bb | 1 + 1 file changed, 1 insertion(+)
meta/recipes-devtools/libtool/libtool-native_2.4.6.bb +1 -0
@@ -7,6 +7,7 @@ SRC_URI += "file://prefix.patch" inherit native EXTRA_OECONF = " --with-libtool-sysroot=${STAGING_DIR_NATIVE}" +CACHED_CONFIGUREVARS += "lt_cv_sys_dlsearch_path=/non/existent" do_configure:prepend () { # Remove any existing libtool m4 since old stale versions would break
RFC PATCH 3/9 openssl: Improve native reproducibility
In reply to Jacob KroonThe proposed changes here should probably be fixed to have no impact on target.
--- ...sysroot-and-debug-prefix-map-from-co.patch | 78 ------------------- .../openssl/openssl/strip-buildinfo.patch | 13 ++++ .../openssl/openssl_3.0.0.bb | 10 +-- 3 files changed, 18 insertions(+), 83 deletions(-) delete mode 100644 meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch create mode 100644 meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch
meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch deleted +0 -78
@@ -1,78 +0,0 @@ -From 5985253f2c9025d7c127443a3a9938946f80c2a1 Mon Sep 17 00:00:00 2001 -From: =?UTF-8?q?Martin=20Hundeb=C3=B8ll?= <[email protected]> -Date: Tue, 6 Nov 2018 14:50:47 +0100 -Subject: [PATCH] buildinfo: strip sysroot and debug-prefix-map from compiler - info -MIME-Version: 1.0 -Content-Type: text/plain; charset=UTF-8 -Content-Transfer-Encoding: 8bit - -The openssl build system generates buildinf.h containing the full -compiler command line used to compile objects. This breaks -reproducibility, as the compile command is baked into libcrypto, where -it is used when running `openssl version -f`. - -Add stripped build variables for the compiler and cflags lines, and use -those when generating buildinfo.h. - -This is based on a similar patch for older openssl versions: -https://patchwork.openembedded.org/patch/147229/ - -Upstream-Status: Inappropriate [OE specific] -Signed-off-by: Martin Hundebøll <[email protected]> - -Update to fix buildpaths qa issue for '-fmacro-prefix-map'. - -Signed-off-by: Kai Kang <[email protected]> - -Update to fix buildpaths qa issue for '-ffile-prefix-map'. - -Signed-off-by: Khem Raj <[email protected]> - ---- - Configurations/unix-Makefile.tmpl | 12 +++++++++++- - crypto/build.info | 2 +- - 2 files changed, 12 insertions(+), 2 deletions(-) - -diff --git a/Configurations/unix-Makefile.tmpl b/Configurations/unix-Makefile.tmpl -index f88a70f..528cdef 100644 ---- a/Configurations/unix-Makefile.tmpl -+++ b/Configurations/unix-Makefile.tmpl -@@ -471,13 +471,23 @@ BIN_LDFLAGS={- join(' ', $target{bin_lflags} || (), - '$(CNF_LDFLAGS)', '$(LDFLAGS)') -} - BIN_EX_LIBS=$(CNF_EX_LIBS) $(EX_LIBS) - --# CPPFLAGS_Q is used for one thing only: to build up buildinf.h -+# *_Q variables are used for one thing only: to build up buildinf.h - CPPFLAGS_Q={- $cppflags1 =~ s|([\\"])|\\$1|g; - $cppflags2 =~ s|([\\"])|\\$1|g; - $lib_cppflags =~ s|([\\"])|\\$1|g; - join(' ', $lib_cppflags || (), $cppflags2 || (), - $cppflags1 || ()) -} - -+CFLAGS_Q={- for (@{$config{CFLAGS}}) { -+ s|-fdebug-prefix-map=[^ ]+|-fdebug-prefix-map=|g; -+ s|-fmacro-prefix-map=[^ ]+|-fmacro-prefix-map=|g; -+ s|-ffile-prefix-map=[^ ]+|-ffile-prefix-map=|g; -+ } -+ join(' ', @{$config{CFLAGS}}) -} -+ -+CC_Q={- $config{CC} =~ s|--sysroot=[^ ]+|--sysroot=recipe-sysroot|g; -+ join(' ', $config{CC}) -} -+ - PERLASM_SCHEME= {- $target{perlasm_scheme} -} - - # For x86 assembler: Set PROCESSOR to 386 if you want to support -diff --git a/crypto/build.info b/crypto/build.info -index efca6cc..eda433e 100644 ---- a/crypto/build.info -+++ b/crypto/build.info -@@ -109,7 +109,7 @@ DEFINE[../libcrypto]=$UPLINKDEF - - DEPEND[info.o]=buildinf.h - DEPEND[cversion.o]=buildinf.h --GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC) $(LIB_CFLAGS) $(CPPFLAGS_Q)" "$(PLATFORM)" -+GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC_Q) $(CFLAGS_Q) $(CPPFLAGS_Q)" "$(PLATFORM)" - - GENERATE[uplink-x86.s]=../ms/uplink-x86.pl - GENERATE[uplink-x86_64.s]=../ms/uplink-x86_64.pl
meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch new file +13 -0
@@ -0,0 +1,13 @@ +Index: openssl-3.0.0/crypto/build.info +=================================================================== +--- openssl-3.0.0.orig/crypto/build.info ++++ openssl-3.0.0/crypto/build.info +@@ -109,7 +109,7 @@ DEFINE[../libcrypto]=$UPLINKDEF + + DEPEND[info.o]=buildinf.h + DEPEND[cversion.o]=buildinf.h +-GENERATE[buildinf.h]=../util/mkbuildinf.pl "$(CC) $(LIB_CFLAGS) $(CPPFLAGS_Q)" "$(PLATFORM)" ++GENERATE[buildinf.h]=../util/mkbuildinf.pl "empty" + + GENERATE[uplink-x86.s]=../ms/uplink-x86.pl + GENERATE[uplink-x86_64.s]=../ms/uplink-x86_64.pl
meta/recipes-connectivity/openssl/openssl_3.0.0.bb +5 -5
@@ -9,10 +9,10 @@ LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=c75985e733726beaba57bc5253e96d04" SRC_URI = "http://www.openssl.org/source/openssl-${PV}.tar.gz \ file://run-ptest \ - file://0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch \ file://afalg.patch \ file://0001-Configure-do-not-tweak-mips-cflags.patch \ file://armv8-32bit.patch \ + file://strip-buildinfo.patch \ " SRC_URI:append:class-nativesdk = " \
@@ -46,10 +46,6 @@ EXTRA_OECONF:append:libc-musl:powerpc64 = " no-asm" EXTRA_OECONF:class-native = "--with-rand-seed=os,devrandom" EXTRA_OECONF:class-nativesdk = "--with-rand-seed=os,devrandom" -# Relying on hardcoded built-in paths causes openssl-native to not be relocateable from sstate. -CFLAGS:append:class-native = " -DOPENSSLDIR=/not/builtin -DENGINESDIR=/not/builtin" -CFLAGS:append:class-nativesdk = " -DOPENSSLDIR=/not/builtin -DENGINESDIR=/not/builtin" - # This allows disabling deprecated or undesirable crypto algorithms. # The default is to trust upstream choices. DEPRECATED_CRYPTO_FLAGS ?= ""
@@ -131,6 +127,10 @@ do_configure () { perl ${B}/configdata.pm --dump } +do_compile:class-native () { + oe_runmake OPENSSLDIR=/non/existent ENGINESDIR=/non/existent MODULESDIR=/non/existent +} + do_install () { oe_runmake DESTDIR="${D}" MANDIR="${mandir}" MANSUFFIX=ssl install
RFC PATCH 4/9 perl/perlcross: Improve native reproducability
In reply to Jacob KroonIn order to make perlcross-native independent of build path we need to follow the symlinks when copying the patches, otherwise they will point to whereever oe-core is checked out for that particular build.
Doing this reveals an issue in perl-native, where it copies the patches from perlcross-native's sysroot, but both perlcross and perl have a patch called "determinism.patch", so one of them gets overridden. Rename the patch in perlcross so that this doesn't happen.
---
.../{determinism.patch => perl-cross-determinism.patch} | 0
meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb | 4 ++--
meta/recipes-devtools/perl/perl_5.34.0.bb | 5 +++++
3 files changed, 7 insertions(+), 2 deletions(-)
rename meta/recipes-devtools/perl-cross/files/{determinism.patch => perl-cross-determinism.patch} (100%)meta/recipes-devtools/perl-cross/files/perl-cross-determinism.patch renamed from meta/recipes-devtools/perl-cross/files/determinism.patch +0 -0
meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb +2 -2
@@ -15,7 +15,7 @@ SRC_URI = "https://github.com/arsv/perl-cross/releases/download/${PV}/perl-cross file://0001-configure_tool.sh-do-not-quote-the-argument-to-comma.patch \ file://0001-perl-cross-add-LDFLAGS-when-linking-libperl.patch \ file://0001-configure_path.sh-do-not-hardcode-prefix-lib-as-libr.patch \ - file://determinism.patch \ + file://perl-cross-determinism.patch \ file://0001-cnf-configure_func_sel.sh-disable-thread_safe_nl_lan.patch \ file://0001-Makefile-check-the-file-if-patched-or-not.patch \ "
@@ -33,7 +33,7 @@ do_compile () { do_install:class-native() { mkdir -p ${D}/${datadir}/perl-cross/ - cp -rf ${S}/* ${D}/${datadir}/perl-cross/ + cp -rfL ${S}/* ${D}/${datadir}/perl-cross/ } BBCLASSEXTEND = "native"
meta/recipes-devtools/perl/perl_5.34.0.bb +5 -0
@@ -97,6 +97,9 @@ do_configure:class-native() { -Dvendorprefix=${prefix} \ -Ui_xlocale \ ${PACKAGECONFIG_CONFARGS} + + # See the comment above + sed -i -e "s,${STAGING_DIR_NATIVE},/non/existent,g" config.h } do_configure:append() {
@@ -395,3 +398,5 @@ SSTATE_HASHEQUIV_FILEMAP = " \ populate_sysroot:*/lib*/perl5/config.sh:${TMPDIR} \ populate_sysroot:*/lib*/perl5/config.sh:${COREBASE} \ " + +EXTRA_STAGING_FIXMES:append:class-native = " RPATH_PADDING"
Re: [OE-core] [RFC PATCH 4/9] perl/perlcross: Improve native reproducability
Can you split the determinism patch fix into a separate patch please?
Alex
Quoted 86 lines · Jacob Kroon
In order to make perlcross-native independent of build path we need to follow
the symlinks when copying the patches, otherwise they will point to whereever
oe-core is checked out for that particular build.Doing this reveals an issue in perl-native, where it copies the patches from perlcross-native's sysroot, but both perlcross and perl have a patch called "determinism.patch", so one of them gets overridden. Rename the patch in perlcross so that this doesn't happen.
Signed-off-by Jacob Kroon <[email protected]>--- .../{determinism.patch => perl-cross-determinism.patch} | 0 meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb | 4 ++-- meta/recipes-devtools/perl/perl_5.34.0.bb | 5 +++++ 3 files changed, 7 insertions(+), 2 deletions(-) rename meta/recipes-devtools/perl-cross/files/{determinism.patch =>perl-cross-determinism.patch} (100%)
meta/recipes-devtools/perl-cross/files/perl-cross-determinism.patch renamed from meta/recipes-devtools/perl-cross/files/determinism.patch +0
-0rename to
meta/recipes-devtools/perl-cross/files/perl-cross-determinism.patchmeta/recipes-devtools/perl-cross/perlcross_1.3.6.bb +0
-0@@ -15,7 +15,7 @@ SRC_URI = "https://github.com/arsv/perl-cross/releases/download/${PV}/perl-cross
file://0001-configure_tool.sh-do-not-quote-the-argument-to-comma.patch \ file://0001-perl-cross-add-LDFLAGS-when-linking-libperl.patch \file://0001-configure_path.sh-do-not-hardcode-prefix-lib-as-libr.patch \
meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb +1
-1- file://determinism.patch \ + file://perl-cross-determinism.patch \file://0001-cnf-configure_func_sel.sh-disable-thread_safe_nl_lan.patch \ file://0001-Makefile-check-the-file-if-patched-or-not.patch \ "meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb +1
-1@@ -33,7 +33,7 @@ do_compile () { do_install:class-native() { mkdir -p ${D}/${datadir}/perl-cross/ - cp -rf ${S}/* ${D}/${datadir}/perl-cross/ + cp -rfL ${S}/* ${D}/${datadir}/perl-cross/ } BBCLASSEXTEND = "native"meta/recipes-devtools/perl/perl_5.34.0.bb +5
-1@@ -97,6 +97,9 @@ do_configure:class-native() { -Dvendorprefix=${prefix} \ -Ui_xlocale \ ${PACKAGECONFIG_CONFARGS} + + # See the comment above + sed -i -e "s,${STAGING_DIR_NATIVE},/non/existent,g" config.h } do_configure:append() {@@ -395,3 +398,5 @@ SSTATE_HASHEQUIV_FILEMAP = " \ populate_sysroot:*/lib*/perl5/config.sh:${TMPDIR} \ populate_sysroot:*/lib*/perl5/config.sh:${COREBASE} \ " + +EXTRA_STAGING_FIXMES:append:class-native = " RPATH_PADDING" -=-=-=-=-=-=-=-=-=-=-=-Links: You receive all messages sent to this group. View/Reply Online (#158871):
https://lists.openembedded.org/g/openembedded-core/message/158871
Mute This Topic: https://lists.openembedded.org/mt/87352797/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [RFC PATCH 4/9] perl/perlcross: Improve native reproducability
Quoted 2 lines · Alexander Kanavin
Can you split the determinism patch fix into a separate patch please?
I don't know what exactly is the right fix here. Should both "determinism.patch":es be applied when building perl-native ?
Jacob
Quoted 94 lines
Alex
On Sun, 28 Nov 2021 at 10:46, Jacob Kroon <[email protected] <mailto:[email protected]>> wrote:
In order to make perlcross-native independent of build path we need to follow the symlinks when copying the patches, otherwise they will point to whereever oe-core is checked out for that particular build. Doing this reveals an issue in perl-native, where it copies the patches from perlcross-native's sysroot, but both perlcross and perl have a patch called "determinism.patch", so one of them gets overridden. Rename the patch in perlcross so that this doesn't happen. Signed-off-by: Jacob Kroon <[email protected] <mailto:[email protected]>> --- .../{determinism.patch => perl-cross-determinism.patch} | 0 meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb <http://perlcross_1.3.6.bb> | 4 ++-- meta/recipes-devtools/perl/perl_5.34.0.bb <http://perl_5.34.0.bb> | 5 +++++ 3 files changed, 7 insertions(+), 2 deletions(-) rename meta/recipes-devtools/perl-cross/files/{determinism.patch => perl-cross-determinism.patch} (100%) diff --git a/meta/recipes-devtools/perl-cross/files/determinism.patch b/meta/recipes-devtools/perl-cross/files/perl-cross-determinism.patch similarity index 100% rename from meta/recipes-devtools/perl-cross/files/determinism.patch rename to meta/recipes-devtools/perl-cross/files/perl-cross-determinism.patch diff --git a/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb <http://perlcross_1.3.6.bb> b/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb <http://perlcross_1.3.6.bb> index 2759ef8a53..dab7f4558f 100644 --- a/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb <http://perlcross_1.3.6.bb> +++ b/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb <http://perlcross_1.3.6.bb> @@ -15,7 +15,7 @@ SRC_URI = "https://github.com/arsv/perl-cross/releases/download/${PV}/perl-cross <https://github.com/arsv/perl-cross/releases/download/$%7BPV%7D/perl-cross> file://0001-configure_tool.sh-do-not-quote-the-argument-to-comma.patch \ file://0001-perl-cross-add-LDFLAGS-when-linking-libperl.patch \ file://0001-configure_path.sh-do-not-hardcode-prefix-lib-as-libr.patch \ - file://determinism.patch \ + file://perl-cross-determinism.patch \ file://0001-cnf-configure_func_sel.sh-disable-thread_safe_nl_lan.patch \ file://0001-Makefile-check-the-file-if-patched-or-not.patch \ " @@ -33,7 +33,7 @@ do_compile () { do_install:class-native() { mkdir -p ${D}/${datadir}/perl-cross/ - cp -rf ${S}/* ${D}/${datadir}/perl-cross/ + cp -rfL ${S}/* ${D}/${datadir}/perl-cross/ } BBCLASSEXTEND = "native" diff --git a/meta/recipes-devtools/perl/perl_5.34.0.bb <http://perl_5.34.0.bb> b/meta/recipes-devtools/perl/perl_5.34.0.bb <http://perl_5.34.0.bb> index 16d45ccff3..0b74d5f072 100644 --- a/meta/recipes-devtools/perl/perl_5.34.0.bb <http://perl_5.34.0.bb> +++ b/meta/recipes-devtools/perl/perl_5.34.0.bb <http://perl_5.34.0.bb> @@ -97,6 +97,9 @@ do_configure:class-native() { -Dvendorprefix=${prefix} \ -Ui_xlocale \ ${PACKAGECONFIG_CONFARGS} + + # See the comment above + sed -i -e "s,${STAGING_DIR_NATIVE},/non/existent,g" config.h } do_configure:append() { @@ -395,3 +398,5 @@ SSTATE_HASHEQUIV_FILEMAP = " \ populate_sysroot:*/lib*/perl5/config.sh:${TMPDIR} \ populate_sysroot:*/lib*/perl5/config.sh:${COREBASE} \ " + +EXTRA_STAGING_FIXMES:append:class-native = " RPATH_PADDING"
Re: [OE-core] [RFC PATCH 4/9] perl/perlcross: Improve native reproducability
Quoted 7 lines · Jacob Kroon
Quoted 2 lines · Alexander Kanavin
Can you split the determinism patch fix into a separate patch please?
I don't know what exactly is the right fix here. Should both "determinism.patch":es be applied when building perl-native ?
Right, I'll take a look at this myself then.
Alex
Quoted 120 lines
Jacob
Quoted 5 lines
Alex
On Sun, 28 Nov 2021 at 10:46, Jacob Kroon <[email protected] <mailto:[email protected]>> wrote:
Quoted 7 lines
In order to make perlcross-native independent of build path we need to follow
the symlinks when copying the patches, otherwise they will point to whereever
oe-core is checked out for that particular build.Doing this reveals an issue in perl-native, where it copies the
patches
Quoted 2 lines
from perlcross-native's sysroot, but both perlcross and perl have a patch called "determinism.patch", so one of them gets overridden.
Rename
Quoted 24 lines
the patch in perlcross so that this doesn't happen. Signed-off-by: Jacob Kroon <[email protected] <mailto:[email protected]>> --- .../{determinism.patch => perl-cross-determinism.patch} | 0 meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb <http://perlcross_1.3.6.bb> | 4 ++-- meta/recipes-devtools/perl/perl_5.34.0.bb <http://perl_5.34.0.bb> | 5 +++++ 3 files changed, 7 insertions(+), 2 deletions(-) rename meta/recipes-devtools/perl-cross/files/{determinism.patch => perl-cross-determinism.patch} (100%) diff --git a/meta/recipes-devtools/perl-cross/files/determinism.patch b/meta/recipes-devtools/perl-cross/files/perl-cross-determinism.patch similarity index 100% rename from meta/recipes-devtools/perl-cross/files/determinism.patch rename to meta/recipes-devtools/perl-cross/files/perl-cross-determinism.patch diff --git a/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb <http://perlcross_1.3.6.bb> b/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bbQuoted 8 lines
<http://perlcross_1.3.6.bb> index 2759ef8a53..dab7f4558f 100644 --- a/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb <http://perlcross_1.3.6.bb> +++ b/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb <http://perlcross_1.3.6.bb> @@ -15,7 +15,7 @@ SRC_URI = "https://github.com/arsv/perl-cross/releases/download/${PV}/perl-cross
Quoted 1 line
<https://github.com/arsv/perl-cross/releases/download/$%7BPV%7D/perl-cross>
Quoted 2 lines
file://0001-configure_tool.sh-do-not-quote-the-argument-to-comma.patch \
Quoted 4 lines
file://0001-perl-cross-add-LDFLAGS-when-linking-libperl.patch \file://0001-configure_path.sh-do-not-hardcode-prefix-lib-as-libr.patch \
Quoted 4 lines
- file://determinism.patch \
+ file://perl-cross-determinism.patch \file://0001-cnf-configure_func_sel.sh-disable-thread_safe_nl_lan.patch \
Quoted 17 lines
file://0001-Makefile-check-the-file-if-patched-or-not.patch \ " @@ -33,7 +33,7 @@ do_compile () { do_install:class-native() { mkdir -p ${D}/${datadir}/perl-cross/ - cp -rf ${S}/* ${D}/${datadir}/perl-cross/ + cp -rfL ${S}/* ${D}/${datadir}/perl-cross/ } BBCLASSEXTEND = "native" diff --git a/meta/recipes-devtools/perl/perl_5.34.0.bb <http://perl_5.34.0.bb> b/meta/recipes-devtools/perl/perl_5.34.0.bb <http://perl_5.34.0.bb> index 16d45ccff3..0b74d5f072 100644 --- a/meta/recipes-devtools/perl/perl_5.34.0.bb <Quoted 1 line
+++ b/meta/recipes-devtools/perl/perl_5.34.0.bb <Quoted 32 lines
@@ -97,6 +97,9 @@ do_configure:class-native() { -Dvendorprefix=${prefix} \ -Ui_xlocale \ ${PACKAGECONFIG_CONFARGS} + + # See the comment above + sed -i -e "s,${STAGING_DIR_NATIVE},/non/existent,g" config.h } do_configure:append() { @@ -395,3 +398,5 @@ SSTATE_HASHEQUIV_FILEMAP = " \ populate_sysroot:*/lib*/perl5/config.sh:${TMPDIR} \ populate_sysroot:*/lib*/perl5/config.sh:${COREBASE} \ " + +EXTRA_STAGING_FIXMES:append:class-native = " RPATH_PADDING" -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#158871): https://lists.openembedded.org/g/openembedded-core/message/158871 <https://lists.openembedded.org/g/openembedded-core/message/158871> Mute This Topic: https://lists.openembedded.org/mt/87352797/1686489 <https://lists.openembedded.org/mt/87352797/1686489> Group Owner: [email protected] <mailto:openembedded-core%[email protected]> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub <https://lists.openembedded.org/g/openembedded-core/unsub> [[email protected] <mailto:[email protected]>] -=-=-=-=-=-=-=-=-=-=-=-
A message that is not in the archive (sent off-list or not yet received): [email protected]
Re: [OE-core] [RFC PATCH 4/9] perl/perlcross: Improve native reproducability
I have fixed this differently, and more reliably - you can cherry-pick into your branch:
https://git.yoctoproject.org/poky-contrib/commit/?h=akanavin/package-version-updates&id=40e8c7af7e32045805bab318e66b644c0e8ed3f2
Alex
Quoted 137 lines · Alexander Kanavin via lists.openembedded.org
Quoted 7 lines · Jacob Kroon
Quoted 2 lines · Alexander Kanavin
Can you split the determinism patch fix into a separate patch please?
I don't know what exactly is the right fix here. Should both "determinism.patch":es be applied when building perl-native ?
Right, I'll take a look at this myself then.
Alex
Quoted 86 lines
Jacob
Quoted 5 lines
Alex
On Sun, 28 Nov 2021 at 10:46, Jacob Kroon <[email protected] <mailto:[email protected]>> wrote:
Quoted 7 lines
In order to make perlcross-native independent of build path we need to follow
the symlinks when copying the patches, otherwise they will point to whereever
oe-core is checked out for that particular build.Doing this reveals an issue in perl-native, where it copies the
patches
Quoted 2 lines
from perlcross-native's sysroot, but both perlcross and perl have a patch called "determinism.patch", so one of them gets overridden.
Rename
Quoted 8 lines
the patch in perlcross so that this doesn't happen. Signed-off-by: Jacob Kroon <[email protected] <mailto:[email protected]>> --- .../{determinism.patch => perl-cross-determinism.patch} | 0 meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb <http://perlcross_1.3.6.bb> | 4 ++--Quoted 9 lines
meta/recipes-devtools/perl/perl_5.34.0.bb <http://perl_5.34.0.bb> | 5 +++++ 3 files changed, 7 insertions(+), 2 deletions(-) rename meta/recipes-devtools/perl-cross/files/{determinism.patch => perl-cross-determinism.patch} (100%) diff --git a/meta/recipes-devtools/perl-cross/files/determinism.patch
b/meta/recipes-devtools/perl-cross/files/perl-cross-determinism.patchQuoted 7 lines
similarity index 100% rename from meta/recipes-devtools/perl-cross/files/determinism.patch rename to meta/recipes-devtools/perl-cross/files/perl-cross-determinism.patch diff --git a/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb <http://perlcross_1.3.6.bb> b/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bbQuoted 8 lines
<http://perlcross_1.3.6.bb> index 2759ef8a53..dab7f4558f 100644 --- a/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb <http://perlcross_1.3.6.bb> +++ b/meta/recipes-devtools/perl-cross/perlcross_1.3.6.bb <http://perlcross_1.3.6.bb> @@ -15,7 +15,7 @@ SRC_URI = "https://github.com/arsv/perl-cross/releases/download/${PV}/perl-cross
Quoted 1 line
<https://github.com/arsv/perl-cross/releases/download/$%7BPV%7D/perl-cross
Quoted 3 lines
file://0001-configure_tool.sh-do-not-quote-the-argument-to-comma.patch \
Quoted 4 lines
file://0001-perl-cross-add-LDFLAGS-when-linking-libperl.patch \file://0001-configure_path.sh-do-not-hardcode-prefix-lib-as-libr.patch \
Quoted 4 lines
- file://determinism.patch \
+ file://perl-cross-determinism.patch \file://0001-cnf-configure_func_sel.sh-disable-thread_safe_nl_lan.patch \
Quoted 17 lines
file://0001-Makefile-check-the-file-if-patched-or-not.patch \ " @@ -33,7 +33,7 @@ do_compile () { do_install:class-native() { mkdir -p ${D}/${datadir}/perl-cross/ - cp -rf ${S}/* ${D}/${datadir}/perl-cross/ + cp -rfL ${S}/* ${D}/${datadir}/perl-cross/ } BBCLASSEXTEND = "native" diff --git a/meta/recipes-devtools/perl/perl_5.34.0.bb <http://perl_5.34.0.bb> b/meta/recipes-devtools/perl/perl_5.34.0.bb <http://perl_5.34.0.bb> index 16d45ccff3..0b74d5f072 100644 --- a/meta/recipes-devtools/perl/perl_5.34.0.bb <Quoted 23 lines
Quoted 1 line
+++ b/meta/recipes-devtools/perl/perl_5.34.0.bb <Quoted 19 lines
@@ -97,6 +97,9 @@ do_configure:class-native() { -Dvendorprefix=${prefix} \ -Ui_xlocale \ ${PACKAGECONFIG_CONFARGS} + + # See the comment above + sed -i -e "s,${STAGING_DIR_NATIVE},/non/existent,g" config.h } do_configure:append() { @@ -395,3 +398,5 @@ SSTATE_HASHEQUIV_FILEMAP = " \ populate_sysroot:*/lib*/perl5/config.sh:${TMPDIR} \ populate_sysroot:*/lib*/perl5/config.sh:${COREBASE} \ " + +EXTRA_STAGING_FIXMES:append:class-native = " RPATH_PADDING"-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#158959):
https://lists.openembedded.org/g/openembedded-core/message/158959
Mute This Topic: https://lists.openembedded.org/mt/87352797/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
RFC PATCH 5/9 pkgconfig: Improve native reproducibility
In reply to Jacob KroonAvoid encoding build-specific paths in the resulting binaries.
--- meta/recipes-devtools/pkgconfig/pkgconfig_git.bb | 1 + 1 file changed, 1 insertion(+)
meta/recipes-devtools/pkgconfig/pkgconfig_git.bb +1 -0
@@ -28,6 +28,7 @@ inherit autotools # so just continue that behaviour. # EXTRA_OECONF += "--disable-indirect-deps" +EXTRA_OECONF:append:class-native = " --libdir=/non/existent --with-pc-path=/non/existent" PACKAGECONFIG ??= "glib" PACKAGECONFIG:class-native = ""
RFC PATCH 6/9 ncurses: Improve native reproducibility
In reply to Jacob KroonAvoid encoding build-specific paths in the resulting binaries.
--- meta/recipes-core/ncurses/ncurses.inc | 4 ++++ 1 file changed, 4 insertions(+)
meta/recipes-core/ncurses/ncurses.inc +4 -0
@@ -91,10 +91,14 @@ ncurses_configure() { --with-manpage-format=normal \ --without-manpage-renames \ --disable-stripping \ + ${EXTRA_CLASS_FLAGS} \ "$@" || return 1 cd .. } +EXTRA_CLASS_FLAGS = "" +EXTRA_CLASS_FLAGS:class-native = "--datadir=/non/existent --with-terminfo-dirs=/non/existent" + # Override the function from the autotools class; ncurses requires a # patched autoconf213 to generate the configure script. This autoconf # is not available so that the shipped script will be used.
RFC PATCH 7/9 util-linux: Improve native reproducibility
In reply to Jacob KroonAvoid encoding build-specific paths in the resulting binaries.
--- meta/recipes-core/util-linux/util-linux_2.37.2.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-core/util-linux/util-linux_2.37.2.bb +1 -1
@@ -83,7 +83,7 @@ EXTRA_OECONF = "\ " EXTRA_OECONF:append:class-target = " --enable-setpriv" -EXTRA_OECONF:append:class-native = " --without-cap-ng --disable-setpriv" +EXTRA_OECONF:append:class-native = " --without-cap-ng --disable-setpriv --runstatedir=/non/existent SYSCONFSTATICDIR=/non/existent" EXTRA_OECONF:append:class-nativesdk = " --without-cap-ng --disable-setpriv" EXTRA_OECONF:append = " --disable-hwclock-gplv3"
RFC PATCH 8/9 python3: Improve native reproducibility
In reply to Jacob KroonLots of hacks to get python reproduce in different build paths:
* Avoid encoding build-specific paths in the resulting binaries
* Build without debug information
* Remove __pycache__/ since the cached files seem to depend on the run-paths
--- .../python/python3/determinism.patch | 15 +++++++++++++++ meta/recipes-devtools/python/python3_3.10.0.bb | 8 ++++++++ 2 files changed, 23 insertions(+) create mode 100644 meta/recipes-devtools/python/python3/determinism.patch
meta/recipes-devtools/python/python3/determinism.patch new file +15 -0
@@ -0,0 +1,15 @@ +Index: Python-3.10.0/Makefile.pre.in +=================================================================== +--- Python-3.10.0.orig/Makefile.pre.in ++++ Python-3.10.0/Makefile.pre.in +@@ -791,8 +791,8 @@ Modules/getbuildinfo.o: $(PARSER_OBJS) \ + + Modules/getpath.o: $(srcdir)/Modules/getpath.c Makefile + $(CC) -c $(PY_CORE_CFLAGS) -DPYTHONPATH='"$(PYTHONPATH)"' \ +- -DPREFIX='"$(prefix)"' \ +- -DEXEC_PREFIX='"$(exec_prefix)"' \ ++ -DPREFIX='"/non/existent"' \ ++ -DEXEC_PREFIX='"/non/existent"' \ + -DVERSION='"$(VERSION)"' \ + -DVPATH='"$(VPATH)"' \ + -o $@ $(srcdir)/Modules/getpath.c
meta/recipes-devtools/python/python3_3.10.0.bb +8 -0
@@ -40,6 +40,7 @@ SRC_URI:append:class-native = " \ file://0001-distutils-sysconfig-append-STAGING_LIBDIR-python-sys.patch \ file://12-distutils-prefix-is-inside-staging-area.patch \ file://0001-Don-t-search-system-for-headers-libraries.patch \ + file://determinism.patch \ " SRC_URI[sha256sum] = "5a99f8e7a6a11a7b98b4e75e0d1303d3832cada5534068f69c7b6222a7b1b002"
@@ -79,6 +80,8 @@ DEPENDS:append:class-nativesdk = " python3-native" # force to use the mutex+cond implementation (https://bugs.python.org/issue41710) CFLAGS += "-DHAVE_BROKEN_POSIX_SEMAPHORES" +CFLAGS:append:class-native = " -ffile-prefix-map=${WORKDIR}=/usr/src" + EXTRA_OECONF = " --without-ensurepip --enable-shared --with-platlibdir=${baselib}" EXTRA_OECONF:append:class-native = " --bindir=${bindir}/${PN}"
@@ -94,6 +97,7 @@ CACHED_CONFIGUREVARS = " \ ac_cv_file__dev_ptc=no \ ac_cv_working_tzset=yes \ " +CACHED_CONFIGUREVARS:append:class-native = " ac_cv_prog_cc_g=no" # PGO currently causes builds to not be reproducible so disable by default, see YOCTO #13407 PACKAGECONFIG:class-target ??= "readline gdbm ${@bb.utils.filter('DISTRO_FEATURES', 'lto', d)}"
@@ -180,6 +184,8 @@ do_install:append() { # More info: http://benno.id.au/blog/2013/01/15/python-determinism rm ${D}${libdir}/python${PYTHON_MAJMIN}/test/__pycache__/test_range.cpython* rm ${D}${libdir}/python${PYTHON_MAJMIN}/test/__pycache__/test_xml_etree.cpython* + + find ${D}${libdir}/python${PYTHON_MAJMIN} -name __pycache__ | xargs -n1 rm -r } do_install:append:class-nativesdk () {
@@ -398,3 +404,5 @@ SYSROOT_PREPROCESS_FUNCS += " py3_sysroot_cleanup" py3_sysroot_cleanup () { rm -rf ${SYSROOT_DESTDIR}${libdir}/python${PYTHON_MAJMIN}/test } + +EXTRA_STAGING_FIXMES:append:class-native = " RPATH_PADDING WORKDIR"
RFC PATCH 9/9 bitbake.conf: Avoid rpath hack for Java recipes
In reply to Jacob KroonThese shouldn't go in here, they are just here to hightlight that the padding trick will not work in all recipes.
--- meta/conf/bitbake.conf | 2 ++ 1 file changed, 2 insertions(+)
meta/conf/bitbake.conf +2 -0
@@ -592,6 +592,8 @@ export BUILD_LDFLAGS = "-L${STAGING_LIBDIR_NATIVE} \ ${RPATH_PADDING_FLAG} \ -Wl,-O1 \ -Wl,--build-id=none" +RPATH_PADDING_FLAG:pn-icedtea7-native = "" +RPATH_PADDING_FLAG:pn-openjdk-8-native = "" BUILDSDK_LDFLAGS = "-Wl,-O1"
Thanks Jacob. When looking at this patchset I kept asking myself, why is this or that change necessary for -native but not for -target. I think it would help if you include that information in the commits, particularly, in 1/9 as it is the most invasive change of all. Maybe we can then figure out a better way.
Alex
Quoted 79 lines · Jacob Kroon
This patch series is not intended for merge. I only send it out to highlight where the problems are and to get some discussion going on how/if we want to improve the sitation.
This is a patch series that tries to improve the reproducibility of the native/cross binaries when building in different directories. This has been tested on a Fedora 35 system which uses gcc 11.2.1 at the time of writing.
The RUNTIME hack is questionable, maybe there is a better way to enforce a fixed RUNTIME entry size during linking. It probably breaks for recipes that do additional rpath manipulations at link-time.
If in the end we do come up with a solution, then it should be tested on the autobuilders, since otherwise this is going to degrade overtime. It would then be important that the build paths are of significantly different lengths. TMPDIR=/tmp/sysrootA/ and TMPDIR=/tmp/sysrootB/ will *not* uncover all
rpath problems.The end result of this patch series is that I can build python3-native in two different build paths, and the resuling sysroot-components/x86_64/ directories are identical, except for the 'fixmepath.cmd' files, which are not included in the hash equiv calculations. Even so, there remains a lot of
other native builds that are going to need to be fixed in similar ways as the ones in this patch series.For my images to build I had to avoid the rpath-hack for icedtea7-native and openjdk-8-native.
/Jacob
Jacob Kroon (9):
bitbake.conf: Pad rpath and remove build ID in native binaries
libtool: Improve native reproducibility
openssl: Improve native reproducibility perl/perlcross: Improve native reproducability
pkgconfig: Improve native reproducibility
ncurses: Improve native reproducibility
util-linux: Improve native reproducibility
python3: Improve native reproducibility bitbake.conf: Avoid rpath hack for Java recipesmeta/classes/chrpath.bbclass | 3 + meta/conf/bitbake.conf | 8 +- ...sysroot-and-debug-prefix-map-from-co.patch | 78 ------------------- .../openssl/openssl/strip-buildinfo.patch | 13 ++++ .../openssl/openssl_3.0.0.bb | 10 +-- meta/recipes-core/ncurses/ncurses.inc | 4 + .../util-linux/util-linux_2.37.2.bb | 2 +- .../libtool/libtool-native_2.4.6.bb | 1 + ...ism.patch => perl-cross-determinism.patch} | 0 .../perl-cross/perlcross_1.3.6.bb | 4 +- meta/recipes-devtools/perl/perl_5.34.0.bb | 5 ++ .../pkgconfig/pkgconfig_git.bb | 1 + .../python/python3/determinism.patch | 15 ++++ .../recipes-devtools/python/python3_3.10.0.bb | 8 ++ 14 files changed, 65 insertions(+), 87 deletions(-) delete mode 100644 meta/recipes-connectivity/openssl/openssl/0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch create mode 100644 meta/recipes-connectivity/openssl/openssl/strip-buildinfo.patch rename meta/recipes-devtools/perl-cross/files/{determinism.patch => perl-cross-determinism.patch} (100%) create mode 100644 meta/recipes-devtools/python/python3/determinism.patch-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#158867):
https://lists.openembedded.org/g/openembedded-core/message/158867
Mute This Topic: https://lists.openembedded.org/mt/87352786/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 8 lines · Alexander Kanavin
Thanks Jacob. When looking at this patchset I kept asking myself, why is this or that change necessary for -native but not for -target. I think it would help if you include that information in the commits, particularly, in 1/9 as it is the most invasive change of all. Maybe we can then figure out a better way.
Alex
Yeah, I think its because there is no extra manipulation of rpath done for the target binaries at all, since they wont be running in a location other than what they were configured for. RP, Khem, please correct me if I'm wrong. The native binaries on the other hand will need to run from each recipe specific sysroot.
I can update the commit message in patch 1 if that makes sense.
Jacob