gcc: Improve reproducibility
PATCH 1/1 gcc: Improve reproducibility
Checksum depends on options that could be different from build to build, e.g. absolute patch. Remove checksum-options from
executable checksum calculation.
--- meta/recipes-devtools/gcc/gcc-11.2.inc | 1 + ...emove-checksum-options-from-checksum.patch | 77 +++++++++++++++++++ 2 files changed, 78 insertions(+) create mode 100644 meta/recipes-devtools/gcc/gcc/0050-gcc-Remove-checksum-options-from-checksum.patch
meta/recipes-devtools/gcc/gcc-11.2.inc +1 -0
@@ -64,6 +64,7 @@ SRC_URI = "\ file://0036-mingw32-Enable-operation_not_supported.patch \ file://0037-libatomic-Do-not-enforce-march-on-aarch64.patch \ file://0041-apply-debug-prefix-maps-before-checksumming-DIEs.patch \ + file://0050-gcc-Remove-checksum-options-from-checksum.patch \ " SRC_URI[sha256sum] = "d08edc536b54c372a1010ff6619dd274c0f1603aa49212ba20f7aa2cda36fa8b"
meta/recipes-devtools/gcc/gcc/0050-gcc-Remove-checksum-options-from-checksum.patch new file +77 -0
77 lines not on this page · raw message
Signature · 1 line
2.26.2.Cisco
Quoted 95 lines · Oleksiy Obitotskyy via lists.openembedded.org
Checksum depends on options that could be different from build to build, e.g. absolute patch. Remove checksum-options from
executable checksum calculation.Change-Id Ifa184febecd2d517826beee784ca5ac7a1fb17feSigned-off-by Oleksiy Obitotskyy <[email protected]>--- meta/recipes-devtools/gcc/gcc-11.2.inc | 1 + ...emove-checksum-options-from-checksum.patch | 77 +++++++++++++++++++ 2 files changed, 78 insertions(+) create mode 100644 meta/recipes-devtools/gcc/gcc/0050-gcc-Remove-checksum-options-from-checksum.patchmeta/recipes-devtools/gcc/gcc-11.2.inc +1
-0@@ -64,6 +64,7 @@ SRC_URI = "\ file://0036-mingw32-Enable-operation_not_supported.patch \ file://0037-libatomic-Do-not-enforce-march-on-aarch64.patch \ file://0041-apply-debug-prefix-maps-before-checksumming-DIEs.patch \ + file://0050-gcc-Remove-checksum-options-from-checksum.patch \ " SRC_URI[sha256sum] = "d08edc536b54c372a1010ff6619dd274c0f1603aa49212ba20f7aa2cda36fa8b"meta/recipes-devtools/gcc/gcc/0050-gcc-Remove-checksum-options-from-checksum.patch new file +64
-0@@ -0,0 +1,77 @@ +From 67402bb4013ea886b68a9f4ee1c2cb1472652dbf Mon Sep 17 00:00:00 2001 +From: Oleksiy Obitotskyy <[email protected]> +Date: Tue, 26 Jan 2021 04:32:27 -0800 +Subject: [PATCH] gcc: Remove checksum-options from checksum + +Executable checksum calculated from object +files and options. Options could contain +absolute path and affect reproducibility. +To improve reproducibility remove +checksum-options conditionally from +executable checksum generation. + +Upstream-Status: Pending +Signed-off-by: Oleksiy Obitotskyy <[email protected]> +--- + gcc/c/Make-lang.in | 2 +- + gcc/cp/Make-lang.in | 2 +- + gcc/objc/Make-lang.in | 3 ++- + gcc/objcp/Make-lang.in | 2 +- + 4 files changed, 5 insertions(+), 4 deletions(-) + +diff --git a/gcc/c/Make-lang.in b/gcc/c/Make-lang.in +index a1cdee872..6ee5c103b 100644 +--- a/gcc/c/Make-lang.in ++++ b/gcc/c/Make-lang.in +@@ -78,7 +78,7 @@ cc1-checksum.c : build/genchecksum$(build_exeext) checksum-options \ + cp ../prev-gcc/cc1-checksum.c cc1-checksum.c; \ + else \ + build/genchecksum$(build_exeext) $(C_OBJS) $(BACKEND) $(LIBDEPS) \ +- checksum-options > cc1-checksum.c.tmp && \ ++ $(if $(SOURCE_DATE_EPOCH),,checksum-options) > cc1-checksum.c.tmp && \ + $(srcdir)/../move-if-change cc1-checksum.c.tmp cc1-checksum.c; \ + fi + +diff --git a/gcc/cp/Make-lang.in b/gcc/cp/Make-lang.in +index 155be74ef..0ac6daa53 100644 +--- a/gcc/cp/Make-lang.in ++++ b/gcc/cp/Make-lang.in +@@ -127,7 +127,7 @@ cc1plus-checksum.c : build/genchecksum$(build_exeext) checksum-options \ + cp ../prev-gcc/cc1plus-checksum.c cc1plus-checksum.c; \ + else \ + build/genchecksum$(build_exeext) $(CXX_OBJS) $(BACKEND) $(CODYLIB) $(LIBDEPS) \ +- checksum-options > cc1plus-checksum.c.tmp && \ ++ $(if $(SOURCE_DATE_EPOCH),,checksum-options) > cc1plus-checksum.c.tmp && \ + $(srcdir)/../move-if-change cc1plus-checksum.c.tmp cc1plus-checksum.c; \ + fi + +diff --git a/gcc/objc/Make-lang.in b/gcc/objc/Make-lang.in +index c91148a57..a83938ee7 100644 +--- a/gcc/objc/Make-lang.in ++++ b/gcc/objc/Make-lang.in +@@ -60,7 +60,8 @@ objc_OBJS = $(OBJC_OBJS) cc1obj-checksum.o + cc1obj-checksum.c : build/genchecksum$(build_exeext) checksum-options \ + $(OBJC_OBJS) $(C_AND_OBJC_OBJS) $(BACKEND) $(LIBDEPS) + build/genchecksum$(build_exeext) $(OBJC_OBJS) $(C_AND_OBJC_OBJS) \ +- $(BACKEND) $(LIBDEPS) checksum-options > cc1obj-checksum.c.tmp && \ ++ $(BACKEND) $(LIBDEPS) \ ++ $(if $(SOURCE_DATE_EPOCH),,checksum-options) > cc1obj-checksum.c.tmp && \ + $(srcdir)/../move-if-change cc1obj-checksum.c.tmp cc1obj-checksum.c + + cc1obj$(exeext): $(OBJC_OBJS) $(C_AND_OBJC_OBJS) cc1obj-checksum.o $(BACKEND) \ +diff --git a/gcc/objcp/Make-lang.in b/gcc/objcp/Make-lang.in +index dfa4d230e..ef16001c2 100644 +--- a/gcc/objcp/Make-lang.in
meta/recipes-devtools/gcc/gcc/0050-gcc-Remove-checksum-options-from-checksum.patch line 74
Quoted 10 lines
meta/recipes-devtools/gcc/gcc/0050-gcc-Remove-checksum-options-from-checksum.patch new file +10
-0++++ b/gcc/objcp/Make-lang.in +@@ -63,7 +63,7 @@ obj-c++_OBJS = $(OBJCXX_OBJS) cc1objplus-checksum.o + cc1objplus-checksum.c : build/genchecksum$(build_exeext) checksum-options \ + $(OBJCXX_OBJS) $(BACKEND) $(CODYLIB) $(LIBDEPS) + build/genchecksum$(build_exeext) $(OBJCXX_OBJS) $(BACKEND) $(CODYLIB) \ +- $(LIBDEPS) checksum-options > cc1objplus-checksum.c.tmp && \ ++ $(LIBDEPS) $(if $(SOURCE_DATE_EPOCH),,checksum-options) > cc1objplus-checksum.c.tmp && \ + $(srcdir)/../move-if-change cc1objplus-checksum.c.tmp \ + cc1objplus-checksum.c +
Do you know why we wouldn't see this issue on the autobuilder tests?
Also, do you plan to submit something upstream? Has this been discussed with them? I'm starting to become reluctant to taking patches marked as "Pending" which just sit in our tree forever...
Cheers,
Richard
Quoted 8 lines · Richard Purdie
Do you know why we wouldn't see this issue on the autobuilder tests?
Also, do you plan to submit something upstream? Has this been discussed with
them? I'm starting to become reluctant to taking patches marked as "Pending"
which just sit in our tree forever...
I was also wondering what the issue really is - there needs to be a description of how it can be observed at least.
Alex
Quoted 4 lines · Alexander Kanavin
Quoted 2 lines · Richard Purdie
Do you know why we wouldn't see this issue on the autobuilder tests?
Quoted 8 lines
Quoted 4 lines
Also, do you plan to submit something upstream? Has this been discussed with them? I'm starting to become reluctant to taking patches marked as "Pending" which just sit in our tree forever...
I was also wondering what the issue really is - there needs to be a description of how it can be observed at least.
I agree at the very least the patch needs more explanation. In my local build:
tmp/work/x86_64-linux/gcc-cross-x86_64/11.2.0-r0/gcc-11.2.0/build.x86_64-linux.x86_64-poky-linux/gcc$ cat checksum-options
g++ -no-pie -isystem/host/usr/include -O2 -pipe -DIN_GCC -DCROSS_DIRECTORY_STRUCTURE -fno-exceptions
-fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wmissing-format-attribute
-Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings -DHAVE_CONFIG_H
-static-libstdc++ -static-libgcc -L/host/usr/lib -L/host/lib
-Wl,--enable-new-dtags -Wl,-rpath-link,/host/usr/lib
-Wl,-rpath-link,/host/lib -Wl,-rpath,/host/usr/lib
-Wl,-rpath,/host/lib -Wl,-O1 -Wl,--allow-shlib-undefined
-Wl,--dynamic-linker=/media/build1/poky/build/tmp/sysroots-uninative/x86_64-linux/lib/ld-linux-x86-64.so.2
so at least in my local build, the problem would appear to be the uninative dynamic linker path?
Oleksiy:
Quoted 4 lines
I am not familiar how tests preformed in your case.
How we check it. We build gcc native (sdk A) and use this gcc to build packages from sdk A. The same is for sdk B.
Binaries for gcc from sdk A and B are different.
Are you saying that the two sdks generate different binaries? Or that the gcc binaries in the two SDKs are different and shouldn't be?
I think we don't compare SDK binaries right now so that might be why our tests don't see that ("oe-selftest -r reproducible" or see lib/oeqa/selftest/cases/reproducibile.py).
Cheers,
Richard