docs: create reproducible html
PATCH 1/1 docs: create reproducible html
The Makefile uses wildcard to collect a list of files. The resulting list of files is in directory order, which is random. As a result the generated html files will differ when build on different hosts.
Use the built-in sort function to get a stable list of files.
--- docs/Makefile | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
docs/Makefile +3 -3
@@ -110,7 +110,7 @@ install-html: html txt figs install: install-man-pages install-html html/index.html: $(DOC_HTML) $(CURDIR)/gen-html-index INDEX - $(PERL) -w -- $(CURDIR)/gen-html-index -i INDEX html $(DOC_HTML) + $(PERL) -w -- $(CURDIR)/gen-html-index -i INDEX html $(sort $(DOC_HTML)) html/%.html: %.markdown $(INSTALL_DIR) $(@D)
@@ -152,8 +152,8 @@ html/hypercall/%/index.html: $(CURDIR)/xen-headers Makefile $(INSTALL_DIR) $(@D) $(PERL) -w $(CURDIR)/xen-headers -O $(@D) \ -T 'arch-$* - Xen public headers' \ - $(patsubst %,-X arch-%,$(filter-out $*,$(DOC_ARCHES))) \ - $(patsubst %,-X xen-%,$(filter-out $*,$(DOC_ARCHES))) \ + $(sort $(patsubst %,-X arch-%,$(filter-out $*,$(DOC_ARCHES)))) \ + $(sort $(patsubst %,-X xen-%,$(filter-out $*,$(DOC_ARCHES)))) \ $(EXTRA_EXCLUDE) \ $(XEN_ROOT)/xen include/public include/xen/errno.h
Olaf Hering writes ("[PATCH] docs: create reproducible html"):
Quoted 3 lines
The
Makefileuses wildcard to collect a list of files. The resulting list of files is in directory order, which is random. As a result the generated html files will differ when build on different hosts.
I'm in favour of reproducible builds, but I think this might be better done by wrapping every instance of $(wildcard ) with $(sort ), rather than postprocessing the output.
What do you think ?
Thanks,
Ian.
Quoted 1 line · Ian Jackson
What do you think ?
Thats overkill. The maintainers of make removed the internal sort with 3.82, likely for performance reasons. Sorted lists are required only in a few places. The Xen source has only two or three places where the sort is actually required. I have patches for all of them.
Olaf
Quoted 6 lines · Olaf Hering
Quoted 1 line · Ian Jackson
What do you think ?
Thats overkill. The maintainers of make removed the internal sort with 3.82, likely for performance reasons.
Aren't you adding uses of it here?
Quoted 5 lines
Sorted lists are required only in
a few places. The Xen source has only two or three places where the sort is actually required. I have patches for all of them.Olaf
Quoted 9 lines · Ian Campbell
Quoted 6 lines · Olaf Hering
Quoted 1 line · Ian Jackson
What do you think ?
Thats overkill. The maintainers of make removed the internal sort with 3.82, likely for performance reasons.
Aren't you adding uses of it here?
NM I misunderstood Ian's suggestion and the nature of the upstream change.
Quoted 13 lines
Quoted 6 lines
Sorted lists are required only in
a few places. The Xen source has only two or three places where the sort is actually required. I have patches for all of them.Olaf
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel
Olaf Hering writes ("Re: [PATCH] docs: create reproducible html"):
Quoted 7 lines · Ian Jackson
Quoted 1 line
What do you think ?
Thats overkill. The maintainers of make removed the internal sort with 3.82, likely for performance reasons. Sorted lists are required only in a few places. The Xen source has only two or three places where the sort is actually required. I have patches for all of them.
There aren't any places in the Xen build system where the performance impact of sorting is relevant.
So we should be going with whatever is easiest to understand and maintain.
A rule that $(wildcard ) in Makefiles should always be accompanied by $(sort ) is simple, clear, easy to understand, and directly addresses the unpredictability at source.
Personally I think that make(1) should have an option or env variable or something to make $(wildcard ) always sort.
Ian.
Quoted 2 lines · Ian Jackson
There aren't any places in the Xen build system where the performance impact of sorting is relevant.
No idea how much the gain was when automated sorting was removed from make(1).
Quoted 3 lines
A rule that $(wildcard ) in Makefiles should always be accompanied by $(sort ) is simple, clear, easy to understand, and directly addresses the unpredictability at source.
Ok, I will adjust the outstanding patches to that rule.
Olaf
Quoted 5 lines · Olaf Hering
Quoted 2 lines · Ian Jackson
There aren't any places in the Xen build system where the performance impact of sorting is relevant.
Quoted 8 lines
No idea how much the gain was when automated sorting was removed from make(1).
Quoted 3 lines
A rule that $(wildcard ) in Makefiles should always be accompanied by $(sort ) is simple, clear, easy to understand, and directly addresses the unpredictability at source.
Ok, I will adjust the outstanding patches to that rule.
Would a helper macro so we can just write
$(call wildcard,arg arg arg) instead of $(sort $(wildcard arg arg arg))
be possible and/or useful?
Ian.
Quoted 5 lines · Ian Campbell
Would a helper macro so we can just write
$(call wildcard,arg arg arg) instead of $(sort $(wildcard arg arg arg))be possible and/or useful?
Just for the or three two places its likely overkill.
Olaf