pahole: enable --reproducible_build when SOURCE_DATE_EPOCH is set
PATCH v2 1/1 pahole: enable --reproducible_build when SOURCE_DATE_EPOCH is set
The SOURCE_DATE_EPOCH environment variable is used in reproducible builds to set build timestamps and is generally synonymous with reproducible builds: instead of making users figure out how to pass pahole flags (e.g. explicitly setting PAHOLE_FLAGS in linux) just assume that the user wants a reproducible build if this variable is set, printing a debug message in verbose mode.
The impact on build time appears to be minimal enough even if we incorrectly make this decision (on a Ryzen 7 7735HS):
$ hyperfine 'sh pahole-test -j --reproducible_build' 'sh pahole-test -j1' 'sh pahole-test -j' -p 'cp .tmp_vmlinux.orig .tmp_vmlinux.btf' -m 4 Benchmark 1: sh pahole-test -j --reproducible_build Time (mean ± σ): 3.991 s ± 0.045 s [User: 7.223 s, System: 3.741 s] Range (min … max): 3.950 s … 4.042 s 4 runs
Benchmark 2: sh pahole-test -j1
Time (mean ± σ): 7.083 s ± 0.095 s [User: 4.805 s, System: 2.242 s]
Range (min … max): 6.964 s … 7.191 s 4 runs
Benchmark 3: sh pahole-test -j
Time (mean ± σ): 3.858 s ± 0.089 s [User: 13.447 s, System: 7.078 s]
Range (min … max): 3.763 s … 3.978 s 4 runs
Summary
sh pahole-test -j ran
1.03 ± 0.03 times faster than sh pahole-test -j --reproducible_build 1.84 ± 0.05 times faster than sh pahole-test -j1
Where pahole-test is the pahole command of the linux build (minus -j):
LLVM_OBJCOPY=objcopy pahole -J --btf_gen_floats --lang_exclude=rust --skip_encoding_btf_inconsistent_proto --btf_gen_optimized .tmp_vmlinux.btf "$@"
And .tmp_vmlinux.orig was the .tmp_vmlinux.btf binary before being
processed:
ld -m elf_x86_64 -z noexecstack --no-warn-rwx-segments --emit-relocs --discard-none -z max-page-size=0x200000 --build-id=sha1 --orphan-handling=warn --script=./arch/x86/kernel/vmlinux.lds -o .tmp_vmlinux.btf --whole-archive vmlinux.a .vmlinux.export.o init/version-timestamp.o --no-whole-archive --start-group --end-group
---
v1 -> v2:
- added message in verbose mode if setting reproducible build through this.
I wasn't quite sure what is least intrusive message-wise -- most verbose messages just go to stdout, but using pahole in a script to extract some c structs would see this mangled in output which is obviously not great. Since --reproducible_build only impacts BTF generation at this time, I'm only checking when generating BTF to avoid such needless messages -- btf generation always goes through a file so there is no risk of mangling the output in this case.
pahole.c | 9 +++++++++ 1 file changed, 9 insertions(+)
pahole.c +9 -0
@@ -3705,6 +3705,15 @@ int main(int argc, char *argv[]) goto out; } +// SOURCE_DATE_EPOCH being set means whoever called us likeely tries +// to do a reproducible build. The setting only impacts btf_encode code +// so only check in this case to avoid needless messages. +if (btf_encode &&getenv ("SOURCE_DATE_EPOCH" )) { +if (!conf_load.reproducible_build && global_verbose) +printf ("pahole: auto-enabling reproducible build (SOURCE_DATE_EPOCH set)\n" ); + conf_load.reproducible_build = true; + } +if (languages.str &&parse_languages ())return rc;
Signature · 1 line
2.45.1
Quoted 27 lines · Dominique Martinet
The
SOURCE_DATE_EPOCHenvironment variable is used in reproducible builds to set build timestamps and is generally synonymous with reproducible builds: instead of making users figure out how to pass pahole flags (e.g. explicitly settingPAHOLE_FLAGSin linux) just assume that the user wants a reproducible build if this variable is set, printing a debug message in verbose mode.The impact on build time appears to be minimal enough even if we incorrectly make this decision (on a Ryzen 7 7735HS):
$ hyperfine 'sh pahole-test -j --reproducible_build' 'sh pahole-test -j1' 'sh pahole-test -j' -p 'cp .tmp_vmlinux.orig .tmp_vmlinux.btf' -m 4 Benchmark 1: sh pahole-test -j --reproducible_build Time (mean ± σ): 3.991 s ± 0.045 s [User: 7.223 s, System: 3.741 s] Range (min … max): 3.950 s … 4.042 s 4 runsBenchmark 2: sh pahole-test -j1
Time (mean ± σ): 7.083 s ± 0.095 s [User: 4.805 s, System: 2.242 s]
Range (min … max): 6.964 s … 7.191 s 4 runsBenchmark 3: sh pahole-test -j
Time (mean ± σ): 3.858 s ± 0.089 s [User: 13.447 s, System: 7.078 s]
Range (min … max): 3.763 s … 3.978 s 4 runsSummary
sh pahole-test -j ran
1.03 ± 0.03 times faster than sh pahole-test -j --reproducible_build 1.84 ± 0.05 times faster than sh pahole-test -j1
Quoted 9 lines
Where pahole-test is the pahole command of the linux build (minus -j):
LLVM_OBJCOPY=objcopy pahole -J --btf_gen_floats --lang_exclude=rust --skip_encoding_btf_inconsistent_proto --btf_gen_optimized .tmp_vmlinux.btf "$@"And .tmp_vmlinux.orig was the .tmp_vmlinux.btf binary before being
processed:
ld -melf_x86_64-z noexecstack --no-warn-rwx-segments --emit-relocs --discard-none -z max-page-size=0x200000 --build-id=sha1 --orphan-handling=warn --script=./arch/x86/kernel/vmlinux.lds -o .tmp_vmlinux.btf --whole-archive vmlinux.a .vmlinux.export.oinit/version-timestamp.o--no-whole-archive --start-group --end-groupSigned-off-by Dominique Martinet <[email protected]>
One small thing below, and asuggestion; it'd be good to mention this behaviour in man-pages/pahole.1 where we discuss the reproducible_build feature.
Something like the commit message would do, i.e.
"The SOURCE_DATE_EPOCH environment variable is used in reproducible builds to set build timestamps and is generally synonymous with reproducible builds so the "reproducible_build" BTF feature is automatically set when this environment variable is set. This ensures that kernel builds will generated reproducible BTF in such cases."
Thanks!
Alan
Quoted 12 lines
---
v1 -> v2:
- added message in verbose mode if setting reproducible build through this.
I wasn't quite sure what is least intrusive message-wise -- most verbose messages just go to stdout, but using pahole in a script to extract some c structs would see this mangled in output which is obviously not great. Since --reproducible_build only impacts BTF generation at this time, I'm only checking when generating BTF to avoid such needless messages -- btf generation always goes through a file so there is no risk of mangling the output in this case.
yep, that seems like a reasonable approach to me.
pahole.c int main(int argc, char *argv[]) · line 3708
Quoted 12 lines
pahole.c | 9 +++++++++ 1 file changed, 9 insertions(+)pahole.c +1
-0@@ -3705,6 +3705,15 @@ int main(int argc, char *argv[]) goto out; } +// SOURCE_DATE_EPOCH being set means whoever called us likeely tries
nit: typo on likely. also (tho it's not always 100% consistent I admit I'd use a multi-line comment style
/*
* SOURCE_DATE_EPOCH being set means whoever called us likely tries
...
*/
Quoted 11 lines
pahole.c +8
-0+// to do a reproducible build. The setting only impacts btf_encode code +// so only check in this case to avoid needless messages. +if (btf_encode &&getenv ("SOURCE_DATE_EPOCH" )) { +if (!conf_load.reproducible_build && global_verbose) +printf ("pahole: auto-enabling reproducible build (SOURCE_DATE_EPOCH set)\n" ); + conf_load.reproducible_build = true; + } +if (languages.str &&parse_languages ())return rc;