Improvements for reproducible builds
cover Improvements for reproducible builds
While working on a reproducible build of xen in Debian I came up with the following three patches which are needed to build xen reproducibly in Debian. Reproducible builds are useful to verify the binary has actually been built from the corresponding source.
The first patch is an extension of [1] which only omits the timestamp if SOURCE_DATE_EPOCH is defined. Patch two fixes an ordering issue in the generated documentation and the last patch uses the date from SOURCE_DATE_EPOCH if available in the documentation.
[1] https://lists.xenproject.org/archives/html/xen-devel/2020-10/msg02161.html
Maximilian Engelhardt (3):
xen/arch/x86: don't insert timestamp when SOURCE_DATE_EPOCH is defined
docs: use predictable ordering in generated documentation
docs: set date to SOURCE_DATE_EPOCH if available
docs/Makefile | 8 +++++++- docs/xen-headers | 2 +- xen/arch/x86/Makefile | 6 ++++++ 3 files changed, 14 insertions(+), 2 deletions(-)
Signature · 1 line
2.20.1
PATCH 1/3 xen/arch/x86: don't insert timestamp when SOURCE_DATE_EPOCH is defined
By default a timestamp gets added to the xen efi binary. Unfortunately ld doesn't seem to provide a way to set a custom date, like from SOURCE_DATE_EPOCH, so set a zero value for the timestamp (option --no-insert-timestamp) if SOURCE_DATE_EPOCH is defined. This makes reproducible builds possible.
This is an alternative to the patch suggested in [1]. This patch only omits the timestamp when SOURCE_DATE_EPOCH is defined.
[1] https://lists.xenproject.org/archives/html/xen-devel/2020-10/msg02161.html
--- xen/arch/x86/Makefile | 6 ++++++ 1 file changed, 6 insertions(+)
xen/arch/x86/Makefile +6 -0
@@ -184,6 +184,12 @@ EFI_LDFLAGS += --major-image-version=$(XEN_VERSION) EFI_LDFLAGS += --minor-image-version=$(XEN_SUBVERSION) EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0 EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0 +# It seems ld unfortunately can't set a custom timestamp, so add a zero value +# for the timestamp (option --no-insert-timestamp) if SOURCE_DATE_EPOCH is +# defined to make reproducible builds possible. +ifdef SOURCE_DATE_EPOCH +EFI_LDFLAGS += --no-insert-timestamp +endif $(TARGET).efi: VIRT_BASE = 0x$(shell $(NM) efi/relocs-dummy.o | sed -n 's, A VIRT_START$$,,p') $(TARGET).efi: ALT_BASE = 0x$(shell $(NM) efi/relocs-dummy.o | sed -n 's, A ALT_START$$,,p')
Signature · 1 line
2.20.1
Re: [XEN PATCH 1/3] xen/arch/x86: don't insert timestamp when SOURCE_DATE_EPOCH is defined
Quoted 6 lines · Maximilian Engelhardt
By default a timestamp gets added to the xen efi binary. Unfortunately ld doesn't seem to provide a way to set a custom date, like from
SOURCE_DATE_EPOCH, so set a zero value for the timestamp (option --no-insert-timestamp) ifSOURCE_DATE_EPOCHis defined. This makes reproducible builds possible.
Quoted 6 lines
This is an alternative to the patch suggested in [1]. This patch only omits the timestamp when
SOURCE_DATE_EPOCHis defined.[1] https://lists.xenproject.org/archives/html/xen-devel/2020-10/msg02161.html
Signed-off-by Maximilian Engelhardt <[email protected]>
PATCH 2/3 docs: use predictable ordering in generated documentation
In reply to Maximilian EngelhardtWhen the seq number is equal, sort by the title to get predictable output ordering. This is useful for reproducible builds.
--- docs/xen-headers | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
docs/xen-headers +1 -1
@@ -331,7 +331,7 @@ sub output_index () { <h2>Starting points</h2> <ul> END - foreach my $ic (sort { $a->{Seq} <=> $b->{Seq} } @incontents) { + foreach my $ic (sort { $a->{Seq} <=> $b->{Seq} or $a->{Title} cmp $b->{Title} } @incontents) { $o .= "<li><a href=\"$ic->{Href}\">$ic->{Title}</a></li>\n"; } $o .= "</ul>\n";
Signature · 1 line
2.20.1
Re: [XEN PATCH 2/3] docs: use predictable ordering in generated documentation
Quoted 4 lines · Maximilian Engelhardt
When the seq number is equal, sort by the title to get predictable output ordering. This is useful for reproducible builds.
Signed-off-by Maximilian Engelhardt <[email protected]>
PATCH 3/3 docs: set date to SOURCE_DATE_EPOCH if available
In reply to Maximilian EngelhardtUse the solution described in [1] to replace the call to the 'date' command with a version that uses SOURCE_DATE_EPOCH if available. This is needed for reproducible builds.
[1] https://reproducible-builds.org/docs/source-date-epoch/
--- docs/Makefile | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
docs/Makefile +7 -1
@@ -3,7 +3,13 @@ include $(XEN_ROOT)/Config.mk -include $(XEN_ROOT)/config/Docs.mk VERSION := $(shell $(MAKE) -C $(XEN_ROOT)/xen --no-print-directory xenversion) -DATE := $(shell date +%Y-%m-%d) + +DATE_FMT := +%Y-%m-%d +ifdef SOURCE_DATE_EPOCH +DATE := $(shell date -u -d "@$(SOURCE_DATE_EPOCH)" "$(DATE_FMT)" 2>/dev/null || date -u -r "$(SOURCE_DATE_EPOCH)" "$(DATE_FMT)" 2>/dev/null || date -u "$(DATE_FMT)") +else +DATE := $(shell date "$(DATE_FMT)") +endif DOC_ARCHES := arm x86_32 x86_64 MAN_SECTIONS := 1 5 7 8
Signature · 1 line
2.20.1
Re: [XEN PATCH 3/3] docs: set date to SOURCE_DATE_EPOCH if available
docs/Makefile include $(XEN_ROOT)/Config.mk · line 9
Quoted 11 lines · Maximilian Engelhardt
docs/Makefile +4
-1@@ -3,7 +3,13 @@ include $(XEN_ROOT)/Config.mk -include $(XEN_ROOT)/config/Docs.mk VERSION := $(shell $(MAKE) -C $(XEN_ROOT)/xen --no-print-directory xenversion) -DATE := $(shell date +%Y-%m-%d) + +DATE_FMT := +%Y-%m-%d +ifdef SOURCE_DATE_EPOCH +DATE := $(shell date -u -d "@$(SOURCE_DATE_EPOCH)" "$(DATE_FMT)" 2>/dev/null || date -u -r "$(SOURCE_DATE_EPOCH)" "$(DATE_FMT)" 2>/dev/null || date -u "$(DATE_FMT)")
Looking at the doc for a (deliberately) old "date", I can't find any mention of the -d "@..." syntax. I take it the command would fail on that system. It would then go on to try the -r variant, which has entirely different meaning on GNU (Linux) systems.
docs/ being subject to configuring, why don't you determine the capabilities of "date" there and invoke just the one command that was found suitable for the system?
Jan
Re: [XEN PATCH 3/3] docs: set date to SOURCE_DATE_EPOCH if available
Quoted 21 lines · On Montag, 21. Dezember 2020 10:01:14 CET Jan Beulich wrote:
Quoted 13 lines · Maximilian Engelhardt
docs/Makefile +4
-1@@ -3,7 +3,13 @@ include $(XEN_ROOT)/Config.mk -include $(XEN_ROOT)/config/Docs.mk VERSION := $(shell $(MAKE) -C $(XEN_ROOT)/xen --no-print-directory xenversion)> -DATE := $(shell date +%Y-%m-%d) + +DATE_FMT := +%Y-%m-%d +ifdef SOURCE_DATE_EPOCH +DATE := $(shell date -u -d "@$(SOURCE_DATE_EPOCH)" "$(DATE_FMT)"Quoted 2 lines
2>/dev/null || date -u -r "$(SOURCE_DATE_EPOCH)" "$(DATE_FMT)" 2>/dev/null || date -u "$(DATE_FMT)")Looking at the doc for a (deliberately) old "date", I can't find any mention of the -d "@..." syntax. I take it the command would fail on that system. It would then go on to try the -r variant, which has entirely different meaning on GNU (Linux) systems.
Quoted 5 lines
docs/ being subject to configuring, why don't you determine the capabilities of "date" there and invoke just the one command that was found suitable for the system?
Jan
Hi Jan,
I did some research. The -d "@..." syntax was introduced about 2005. Testing a live CD from 2006 (KNOPPIX_V5.0.1CD-2006-06-01-EN.iso) it was supported there. The documentation about this syntax has only been added in 2011 to the date command. I'm wondering if anybody running such an old system wants to use SOURCE_DATE_EPOCH.
However, I came up with a patch to determine which suitable date version is available and only call that, as you suggested. I will post the new patch soon.
Maxi