reproducible builds: strip embedded file paths
PATCH 1/1 reproducible builds: strip embedded file paths
A recent commit 12753d22563f7d2d01f2c6644c7b66b06eb5c90f introduced a use of __FILE__ to display the file in some debugging output.
Unfortunately, the use of __FILE__ can embed the absolute build path in the resulting binary, resulting in different binaries depending on which path the build was performed in.
All previous versions tested in Debian built reproducibly:
https://tests.reproducible-builds.org/debian/history/amd64/opensbi.html
The attached patch passes -ffile-prefix-map to CFLAGS in the top-level Makefile, which should strip the top-level build path from the resulting build. This should be supported in both gcc 8 and clang 10.
This patch was also submitted as a pull request:
https://github.com/riscv-software-src/opensbi/pull/229
Please CC me with any concerns or questions.
Thanks!
live well,
vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Pass-ffile-prefix-map-to-ensure-reproducible-builds-.patch
Type: text/x-diff
Size: 948 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/opensbi/attachments/20211118/b4f783cc/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/opensbi/attachments/20211118/b4f783cc/attachment.sig>
PATCH 1/1 reproducible builds: strip embedded file paths
? 2021-11-18???? 17:05 -0800?Vagrant Cascadian???
Quoted 6 lines
Unfortunately, the use of
__FILE__can embed the absolute build path in
the resulting binary, resulting in different binaries depending on which
path the build was performed in.
Under what scenarios will absolute paths be embed.
Regards,
Xiang W
PATCH 1/1 reproducible builds: strip embedded file paths
Quoted 10 lines · Xiang W
? 2021-11-18???? 17:05 -0800?Vagrant Cascadian???
Quoted 6 lines
Unfortunately, the use of
__FILE__can embed the absolute build path in
the resulting binary, resulting in different binaries depending on which
path the build was performed in.Under what scenarios will absolute paths be embed.
As Vagrant said:
"A recent commit 12753d22563f7d2d01f2c6644c7b66b06eb5c90f introduced a use of __FILE__ to display the file in some debugging output.?
The lazy BUG macro, added in that commit, uses __FILE__.
Jess
PATCH 1/1 reproducible builds: strip embedded file paths
In reply to Xiang WQuoted 9 lines · Xiang W.
? 2021-11-18???? 17:05 -0800?Vagrant Cascadian???
Quoted 6 lines
Unfortunately, the use of
__FILE__can embed the absolute build path in
the resulting binary, resulting in different binaries depending on which
path the build was performed in.Under what scenarios will absolute paths be embed.
In each of the directories below, I built using the same toolchain from the same git commit...
$ cd /home/vagrant/src/opensbi/opensbi1 $ CROSS_COMPILE=riscv64-linux-gnu- make V=1 FW_PAYLOAD=n PLATFORM=generic ... $ strings build/platform/generic/firmware/fw_dynamic.bin | grep /home/vagrant /home/vagrant/src/opensbi/opensbi1/lib/sbi/riscv_asm.c
$ cd /home/vagrant/src/opensbi/opensbi2 $ CROSS_COMPILE=riscv64-linux-gnu- make V=1 FW_PAYLOAD=n PLATFORM=generic ... $ strings build/platform/generic/firmware/fw_dynamic.bin | grep /home/vagrant /home/vagrant/src/opensbi/opensbi2/lib/sbi/riscv_asm.c
With the patch applied:
$ cd /home/vagrant/src/opensbi/opensbi3 $ CROSS_COMPILE=riscv64-linux-gnu- make V=1 FW_PAYLOAD=n PLATFORM=generic ... $ strings build/platform/generic/firmware/fw_dynamic.bin | grep /home/vagrant $ strings build/platform/generic/firmware/fw_dynamic.bin | grep riscv_asm.c ./lib/sbi/riscv_asm.c
A relative path within the source should be sufficient for the debug message to properly identify the relevent source files...
An alternate approach would be to not use __FILE__ at all, but that would probably take more code than passing -ffile-prefix-map at compile time.
Thanks!
live well,
vagrant
PATCH 1/1 reproducible builds: strip embedded file paths
? 2021-11-21???? 11:20 -0800?Vagrant Cascadian???
Quoted 20 lines
of the directories below, I built using the same toolchain from the same git commit...
$ cd /home/vagrant/src/opensbi/opensbi1 $ CROSS_COMPILE=riscv64-linux-gnu- make V=1 FW_PAYLOAD=n PLATFORM=generic ... $ strings build/platform/generic/firmware/fw_dynamic.bin | grep /home/vagrant /home/vagrant/src/opensbi/opensbi1/lib/sbi/riscv_asm.c$ cd /home/vagrant/src/opensbi/opensbi2 $ CROSS_COMPILE=riscv64-linux-gnu- make V=1 FW_PAYLOAD=n PLATFORM=generic ... $ strings build/platform/generic/firmware/fw_dynamic.bin | grep /home/vagrant /home/vagrant/src/opensbi/opensbi2/lib/sbi/riscv_asm.c
Quoted 3 lines
With the patch applied:
$ cd /home/vagrant/src/opensbi/opensbi3
This is because the source file uses the absolute path when compiling, you can use the relative path by modifying the Makefile
Makefile +0 -0
@@ -361,7 +361,7 @@ compile_cc_dep = $(CMD_PREFIX)mkdir -p `dirname
$(1)`; \
-MM $(2) >> $(1) || rm -f $(1)
compile_cc = $(CMD_PREFIX)mkdir -p dirname $(1); \
echo " CC $(subst $(build_dir)/,,$(1))"; \
- $(CC) $(CFLAGS) $(call dynamic_flags,$(1),$(2)) -c $(2) -
o $(1)
+ $(CC) $(CFLAGS) $(call dynamic_flags,$(1),$(2)) -c
$(subst $(src_dir)/,,$(2)) -o $(1)
compile_as_dep = $(CMD_PREFIX)mkdir -p dirname $(1); \
echo " AS-DEP $(subst $(build_dir)/,,$(1))"; \
printf %s dirname $(1)/ > $(1) && \
Regards,
Xiang W
PATCH 1/1 reproducible builds: strip embedded file paths
Quoted 11 lines · Xiang W.
? 2021-11-21???? 11:20 -0800?Vagrant Cascadian???
Quoted 10 lines
of the directories below, I built using the same toolchain from the same git commit...
$ cd /home/vagrant/src/opensbi/opensbi1 $ CROSS_COMPILE=riscv64-linux-gnu- make V=1 FW_PAYLOAD=n PLATFORM=generic ... $ strings build/platform/generic/firmware/fw_dynamic.bin | grep /home/vagrant /home/vagrant/src/opensbi/opensbi1/lib/sbi/riscv_asm.c
...
Quoted 17 lines
This is because the source file uses the absolute path when compiling, you can use the relative path by modifying the
MakefileMakefile +1
-1@@ -361,7 +361,7 @@ compile_cc_dep = $(CMD_PREFIX)mkdir -p `dirname $(1)`; \ -MM $(2) >> $(1) || rm -f $(1) compile_cc = $(CMD_PREFIX)mkdir -p `dirname $(1)`; \ echo " CC $(subst $(build_dir)/,,$(1))"; \ - $(CC) $(CFLAGS) $(call dynamic_flags,$(1),$(2)) -c $(2) - o $(1) + $(CC) $(CFLAGS) $(call dynamic_flags,$(1),$(2)) -c $(subst $(src_dir)/,,$(2)) -o $(1)
compile_as_dep= $(CMD_PREFIX)mkdir -pdirname $(1); \
Makefile
Quoted 2 lines
Makefile +0
-0echo " AS-DEP $(subst $(build_dir)/,,$(1))"; \ printf %s `dirname $(1)`/ > $(1) && \
That solves the issue and is probably better than passing more compiler arguments, thanks!
live well,
vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/opensbi/attachments/20211121/3ea4d4ac/attachment.sig>
PATCH 1/1 reproducible builds: strip embedded file paths
Quoted 28 lines · Vagrant Cascadian
Quoted 11 lines · Xiang W.
? 2021-11-21???? 11:20 -0800?Vagrant Cascadian???
Quoted 10 lines
of the directories below, I built using the same toolchain from the same git commit...
$ cd /home/vagrant/src/opensbi/opensbi1 $ CROSS_COMPILE=riscv64-linux-gnu- make V=1 FW_PAYLOAD=n PLATFORM=generic ... $ strings build/platform/generic/firmware/fw_dynamic.bin | grep /home/vagrant /home/vagrant/src/opensbi/opensbi1/lib/sbi/riscv_asm.c...
Quoted 14 lines
This is because the source file uses the absolute path when compiling, you can use the relative path by modifying the
MakefileMakefile +0
-1@@ -361,7 +361,7 @@ compile_cc_dep = $(CMD_PREFIX)mkdir -p `dirname $(1)`; \ -MM $(2) >> $(1) || rm -f $(1) compile_cc = $(CMD_PREFIX)mkdir -p `dirname $(1)`; \ echo " CC $(subst $(build_dir)/,,$(1))"; \ - $(CC) $(CFLAGS) $(call dynamic_flags,$(1),$(2)) -c $(2) - o $(1)
Quoted 8 lines
Quoted 5 lines
Makefile +1
-0+ $(CC) $(CFLAGS) $(call dynamic_flags,$(1),$(2)) -c $(subst $(src_dir)/,,$(2)) -o $(1)compile_as_dep = $(CMD_PREFIX)mkdir -p `dirname $(1)`; \ echo " AS-DEP $(subst $(build_dir)/,,$(1))"; \ printf %s `dirname $(1)`/ > $(1) && \That solves the issue and is probably better than passing more compiler arguments, thanks!
This will only work if we are compiling from the OpenSBI source directory itself. What if users compile OpenSBI with a different current directory using "make -C <opensbi_source_path>". I guess passing compiler argument will allow "make -C <opensbi_source_path>".
Regards,
Anup
Quoted 8 lines
live well,
vagrant
--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi
PATCH 1/1 reproducible builds: strip embedded file paths
? 2021-11-22???? 10:13 +0530?Anup Patel???
Quoted 43 lines
Quoted 32 lines · Vagrant Cascadian
Quoted 11 lines · Xiang W.
? 2021-11-21???? 11:20 -0800?Vagrant Cascadian???
Quoted 10 lines
of the directories below, I built using the same toolchain from the same git commit...
$ cd /home/vagrant/src/opensbi/opensbi1 $ CROSS_COMPILE=riscv64-linux-gnu- make V=1 FW_PAYLOAD=n PLATFORM=generic ... $ strings build/platform/generic/firmware/fw_dynamic.bin | grep /home/vagrant /home/vagrant/src/opensbi/opensbi1/lib/sbi/riscv_asm.c...
Quoted 18 lines
This is because the source file uses the absolute path when compiling,
you can use the relative path by modifying theMakefileMakefile +0
-0@@ -361,7 +361,7 @@ compile_cc_dep = $(CMD_PREFIX)mkdir -p`dirname
$(1)`; \
???????????? -MM $(2) >> $(1) || rm -f $(1) ?compile_cc = $(CMD_PREFIX)mkdir -pdirname $(1); \
?????????? echo " CC??????? $(subst $(build_dir)/,,$(1))"; \
-????????? $(CC) $(CFLAGS) $(call dynamic_flags,$(1),$(2)) -c $(2) -
o $(1)
+????????? $(CC) $(CFLAGS) $(call dynamic_flags,$(1),$(2)) -cQuoted 8 lines
Quoted 4 lines
$(subst $(src_dir)/,,$(2)) -o $(1)
?compile_as_dep = $(CMD_PREFIX)mkdir -pdirname $(1); \
?????????? echo " AS-DEP??? $(subst $(build_dir)/,,$(1))"; \
?????????? printf %sdirname $(1)/ > $(1) && \That solves the issue and is probably better than passing more compiler
arguments, thanks!
Quoted 6 lines
This will only work if we are compiling from the OpenSBI source directory
itself. What if users compile OpenSBI with a different current directory
using "make -C <opensbi_source_path>". I guess passing compiler argument will allow "make -C <opensbi_source_path>".
make -C <directory> will switch the path to <directory> first, so the result is the same.
Regards,
Xiang W
Quoted 12 lines
Regards,
AnupQuoted 8 lines
live well,
? vagrant
--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi
PATCH 1/1 reproducible builds: strip embedded file paths
Quoted 51 lines · Xiang W.
? 2021-11-22???? 10:13 +0530?Anup Patel???
Quoted 43 lines
Quoted 32 lines · Vagrant Cascadian
Quoted 11 lines · Xiang W.
? 2021-11-21???? 11:20 -0800?Vagrant Cascadian???
Quoted 10 lines
of the directories below, I built using the same toolchain from the same git commit...
$ cd /home/vagrant/src/opensbi/opensbi1 $ CROSS_COMPILE=riscv64-linux-gnu- make V=1 FW_PAYLOAD=n PLATFORM=generic ... $ strings build/platform/generic/firmware/fw_dynamic.bin | grep /home/vagrant /home/vagrant/src/opensbi/opensbi1/lib/sbi/riscv_asm.c...
Quoted 18 lines
This is because the source file uses the absolute path when compiling,
you can use the relative path by modifying theMakefileMakefile +0
-0@@ -361,7 +361,7 @@ compile_cc_dep = $(CMD_PREFIX)mkdir -p`dirname
$(1)`; \
???????????? -MM $(2) >> $(1) || rm -f $(1) ?compile_cc = $(CMD_PREFIX)mkdir -pdirname $(1); \
?????????? echo " CC??????? $(subst $(build_dir)/,,$(1))"; \
-????????? $(CC) $(CFLAGS) $(call dynamic_flags,$(1),$(2)) -c $(2) -
o $(1)
+????????? $(CC) $(CFLAGS) $(call dynamic_flags,$(1),$(2)) -cQuoted 8 lines
Quoted 4 lines
$(subst $(src_dir)/,,$(2)) -o $(1)
?compile_as_dep = $(CMD_PREFIX)mkdir -pdirname $(1); \
?????????? echo " AS-DEP??? $(subst $(build_dir)/,,$(1))"; \
?????????? printf %sdirname $(1)/ > $(1) && \That solves the issue and is probably better than passing more compiler
arguments, thanks!Quoted 6 lines
This will only work if we are compiling from the OpenSBI source directory
itself. What if users compile OpenSBI with a different current directory
using "make -C <opensbi_source_path>". I guess passing compiler argument will allow "make -C <opensbi_source_path>".
Quoted 2 lines
make -C <directory> will switch the path to <directory> first, so the result is the same.
Any outstanding concerns with proposing the patch by Xiang W to pass relative directories instead of the original patch I submitted?
Should it be submitted by Xiang W or should I merge the commit message from the original patch and submit it myself?
Thanks!
live well,
vagrant
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 227 bytes
Desc: not available
URL: <http://lists.infradead.org/pipermail/opensbi/attachments/20211126/7eb1c7fd/attachment.sig>
PATCH 1/1 reproducible builds: strip embedded file paths
?On 27/11/21, 8:24 AM, "opensbi on behalf of Vagrant Cascadian" <opensbi-bounces at lists.infradead.org on behalf of [email protected]> wrote:
On 2021-11-22, Xiang W. wrote:
> ? 2021-11-22???? 10:13 +0530?Anup Patel???
>> On Mon, Nov 22, 2021 at 10:02 AM Vagrant Cascadian
>> <[email protected]> wrote:
>> >
>> > On 2021-11-22, Xiang W. wrote:
>> > > ? 2021-11-21???? 11:20 -0800?Vagrant Cascadian???
>> > > > of the directories below, I built using the same toolchain from
>> > > > the same git commit...
>> > > >
>> > > > $ cd /home/vagrant/src/opensbi/opensbi1
>> > > > $ CROSS_COMPILE=riscv64-linux-gnu- make V=1 FW_PAYLOAD=n
>> > > > PLATFORM=generic
>> > > > ...
>> > > > $ strings build/platform/generic/firmware/fw_dynamic.bin | grep
>> > > > /home/vagrant
>> > > > /home/vagrant/src/opensbi/opensbi1/lib/sbi/riscv_asm.c
>> > ...
>> > > This is because the source file uses the absolute path when
>> > > compiling,
>> > > you can use the relative path by modifying the Makefile
>> > >
>> > > diff --git a/Makefile b/Makefile
>> > > index 8623c1c..d017534 100644
>> > > --- a/Makefile
>> > > +++ b/Makefile
>> > > @@ -361,7 +361,7 @@ compile_cc_dep = $(CMD_PREFIX)mkdir -p
>> > > `dirname
>> > > $(1)`; \
>> > > -MM $(2) >> $(1) || rm -f $(1)
>> > > compile_cc = $(CMD_PREFIX)mkdir -p `dirname $(1)`; \
>> > > echo " CC $(subst $(build_dir)/,,$(1))"; \
>> > > - $(CC) $(CFLAGS) $(call dynamic_flags,$(1),$(2)) -c
>> > > $(2) -
>> > > o $(1)
>> > > + $(CC) $(CFLAGS) $(call dynamic_flags,$(1),$(2)) -c
>> > > $(subst $(src_dir)/,,$(2)) -o $(1)
>> > > compile_as_dep = $(CMD_PREFIX)mkdir -p `dirname $(1)`; \
>> > > echo " AS-DEP $(subst $(build_dir)/,,$(1))"; \
>> > > printf %s `dirname $(1)`/ > $(1) && \
>> >
>> > That solves the issue and is probably better than passing more
>> > compiler
>> > arguments, thanks!
>>
>> This will only work if we are compiling from the OpenSBI source
>> directory
>> itself. What if users compile OpenSBI with a different current
>> directory
>> using "make -C <opensbi_source_path>". I guess passing compiler
>> argument will allow "make -C <opensbi_source_path>".
>
> make -C <directory> will switch the path to <directory> first, so the
> result is the same.
Any outstanding concerns with proposing the patch by Xiang W to pass
relative directories instead of the original patch I submitted?
Should it be submitted by Xiang W or should I merge the commit message
from the original patch and submit it myself?[Anup] Change the patch subject to "Makefile: strip embedded file paths" and send v2 patch.
[Anup] Please use "git send-email" to send patch to opensbi mailing list. You will have to join the mailing list before posting because it allows posting by mailing list members only.
Regards,
Anup
Thanks!
live well,
vagrant
PATCH 1/1 reproducible builds: strip embedded file paths
In reply to Vagrant Cascadian? 2021-11-26???? 17:48 -0800?Vagrant Cascadian???
Quoted 58 lines · Xiang W.
Quoted 53 lines
? 2021-11-22???? 10:13 +0530?Anup Patel???
Quoted 45 lines
Quoted 34 lines · Vagrant Cascadian
Quoted 13 lines · Xiang W.
? 2021-11-21???? 11:20 -0800?Vagrant Cascadian???
Quoted 12 lines
of the directories below, I built using the same toolchain from
the same git commit...$ cd /home/vagrant/src/opensbi/opensbi1 $ CROSS_COMPILE=riscv64-linux-gnu- make V=1 FW_PAYLOAD=n PLATFORM=generic ... $ strings build/platform/generic/firmware/fw_dynamic.bin | grep /home/vagrant /home/vagrant/src/opensbi/opensbi1/lib/sbi/riscv_asm.c...
Quoted 18 lines
This is because the source file uses the absolute path when compiling,
you can use the relative path by modifying theMakefileMakefile +0
-0@@ -361,7 +361,7 @@ compile_cc_dep = $(CMD_PREFIX)mkdir -p`dirname
$(1)`; \
???????????? -MM $(2) >> $(1) || rm -f $(1) ?compile_cc = $(CMD_PREFIX)mkdir -pdirname $(1); \
?????????? echo " CC??????? $(subst $(build_dir)/,,$(1))"; \
-????????? $(CC) $(CFLAGS) $(call dynamic_flags,$(1),$(2)) -c $(2) -
o $(1)
+????????? $(CC) $(CFLAGS) $(call dynamic_flags,$(1),$(2)) -cQuoted 8 lines
Quoted 4 lines
$(subst $(src_dir)/,,$(2)) -o $(1)
?compile_as_dep = $(CMD_PREFIX)mkdir -pdirname $(1); \
?????????? echo " AS-DEP??? $(subst $(build_dir)/,,$(1))"; \
?????????? printf %sdirname $(1)/ > $(1) && \That solves the issue and is probably better than passing more compiler
arguments, thanks!Quoted 6 lines
This will only work if we are compiling from the OpenSBI source directory
itself. What if users compile OpenSBI with a different current directory
using "make -C <opensbi_source_path>". I guess passing compiler argument will allow "make -C <opensbi_source_path>".Quoted 3 lines
make -C <directory> will switch the path to <directory> first, so the
result is the same.
Quoted 6 lines
Any outstanding concerns with proposing the patch by Xiang W to pass relative directories instead of the original patch I submitted?
Should it be submitted by Xiang W or should I merge the commit message
from the original patch and submit it myself?
I just made a suggestion, you can submit it.
Regards,
Xiang W
Quoted 6 lines
Thanks!
live well,
? vagrant