implement a stable 'Last updated' in Documentation
Several files in Documentation have an unstable 'Last updated' timestamp. The reason is that their mtime changes every time, which prevents reproducible builds.
341 technical/api-index.txt: technical/api-index-skel.txt \
342 technical/api-index.sh $(patsubst %,%.txt,$(API_DOCS))
343 $(QUIET_GEN)cd technical && '$(SHELL_PATH_SQ)' ./api-index.sh
388 howto-index.txt: howto-index.sh $(wildcard howto/*.txt)
389 $(QUIET_GEN)$(RM) $@+ $@ && \ 390 '$(SHELL_PATH_SQ)' ./howto-index.sh $(sort $(wildcard howto/*.txt)) >$@+ && \ 391 mv $@+ $@
399 $(patsubst %.txt,%.html,$(wildcard howto/*.txt)): %.html : %.txt
400 $(QUIET_ASCIIDOC)$(RM) $@+ $@ && \ 401 sed -e '1,/^$$/d' $< | \ 402 $(TXT_TO_HTML) - >$@+ && \ 403 mv $@+ $@
What file timestamp should be used for them? Likely "../version"? The final file, before passing it to asciidoc, should get a fixed timestamp with 'touch -r $reference_file $file'.
Olaf
Quoted 14 lines · Olaf Hering
Several files in Documentation have an unstable 'Last updated' timestamp. The reason is that their mtime changes every time, which prevents reproducible builds.
341
technical/api-index.txt:technical/api-index-skel.txt\
342technical/api-index.sh$(patsubst %,%.txt,$(API_DOCS))
343 $(QUIET_GEN)cd technical && '$(SHELL_PATH_SQ)' ./api-index.sh388 howto-index.txt:
howto-index.sh$(wildcard howto/*.txt)389 $(QUIET_GEN)$(RM) $@+ $@ && \ 390 '$(SHELL_PATH_SQ)' ./howto-index.sh $(sort $(wildcard howto/*.txt)) >$@+ && \ 391 mv $@+ $@399 $(patsubst %.txt,%.html,$(wildcard howto/*.txt)): %.html : %.txt
Quoted 8 lines
400 $(QUIET_ASCIIDOC)$(RM) $@+ $@ && \ 401 sed -e '1,/^$$/d' $< | \ 402 $(TXT_TO_HTML) - >$@+ && \ 403 mv $@+ $@What file timestamp should be used for them? Likely "../version"? The final file, before passing it to asciidoc, should get a fixed timestamp with 'touch -r $reference_file $file'.
I came up with this change. If anything happens to depend on howto-index.txt and technical/api-index.txt then this part needs rework.
Documentation/Makefile +9 -3
@@ -6,6 +6,7 @@ TECH_DOCS = ARTICLES = SP_ARTICLES = OBSOLETE_HTML = +TIMESTAMP_FILE = ../version MAN1_TXT += $(filter-out \ $(addsuffix .txt, $(ARTICLES) $(SP_ARTICLES)), \
@@ -341,6 +342,7 @@ user-manual.xml: user-manual.txt user-manual.conf technical/api-index.txt: technical/api-index-skel.txt \ technical/api-index.sh $(patsubst %,%.txt,$(API_DOCS)) $(QUIET_GEN)cd technical && '$(SHELL_PATH_SQ)' ./api-index.sh + touch -r '$(TIMESTAMP_FILE)' $@ technical/%.html: ASCIIDOC_EXTRA += -a git-relative-html-prefix=../ $(patsubst %,%.html,$(API_DOCS) technical/api-index $(TECH_DOCS)): %.html : %.txt asciidoc.conf
@@ -388,6 +390,7 @@ $(patsubst %.txt,%.texi,$(MAN_TXT)): %.texi : %.xml howto-index.txt: howto-index.sh $(wildcard howto/*.txt) $(QUIET_GEN)$(RM) $@+ $@ && \ '$(SHELL_PATH_SQ)' ./howto-index.sh $(sort $(wildcard howto/*.txt)) >$@+ && \ + touch -r '$(TIMESTAMP_FILE)' $@+ && \ mv $@+ $@ $(patsubst %,%.html,$(ARTICLES)) : %.html : %.txt
@@ -397,9 +400,12 @@ WEBDOC_DEST = /pub/software/scm/git/docs howto/%.html: ASCIIDOC_EXTRA += -a git-relative-html-prefix=../ $(patsubst %.txt,%.html,$(wildcard howto/*.txt)): %.html : %.txt - $(QUIET_ASCIIDOC)$(RM) $@+ $@ && \ - sed -e '1,/^$$/d' $< | \ - $(TXT_TO_HTML) - >$@+ && \ + $(QUIET_ASCIIDOC)$(RM) $@++ $@+ $@ && \ + sed -e '1,/^$$/d' $< >$@++ && \ + touch -r '$(TIMESTAMP_FILE)' $@++ && \ + $(TXT_TO_HTML) -o $@+ $@++ && \ + $(RM) $@++ && \ + touch $@+ && \ mv $@+ $@ install-webdoc : html
Olaf
Quoted 13 lines · Olaf Hering
Several files in Documentation have an unstable 'Last updated' timestamp. The reason is that their mtime changes every time, which prevents reproducible builds.
341
technical/api-index.txt:technical/api-index-skel.txt\
342technical/api-index.sh$(patsubst %,%.txt,$(API_DOCS))
343 $(QUIET_GEN)cd technical && '$(SHELL_PATH_SQ)' ./api-index.sh388 howto-index.txt:
howto-index.sh$(wildcard howto/*.txt)389 $(QUIET_GEN)$(RM) $@+ $@ && \ 390 '$(SHELL_PATH_SQ)' ./howto-index.sh $(sort $(wildcard howto/*.txt)) >$@+ && \ 391 mv $@+ $@
Quoted 5 lines
399 $(patsubst %.txt,%.html,$(wildcard howto/*.txt)): %.html : %.txt
400 $(QUIET_ASCIIDOC)$(RM) $@+ $@ && \ 401 sed -e '1,/^$$/d' $< | \ 402 $(TXT_TO_HTML) - >$@+ && \ 403 mv $@+ $@
All of them seem to have dependencies so it seems to me that two builds back to back without actually changing their input would not re-build anything. What am I missing???
Quoted 1 line
What file timestamp should be used for them? Likely "../version"?
I tend to think the "Last updated" timestamp taken from the filesystem timestamp is a bad practice inherited by these tools from the days back when nobody used any revision control systems.
If I check out v1.8.5 and build documentation now, it does not matter if the generated documentation had the timestamp of the checkout of individual files or that of a single file generated during the build process, i.e. ../version. Neither has much relevance to the time the actual contents of the documentation was prepared or what vintage of the software the documentation is for.
I am fine with branding generated documentation with the version number (i.e. "You are reading the documentation for Git version 2.2.0"). Replacing that statement with "You are reading the documentation for a version of Git that was committed on such and such time" is also fine. But using file timestamp would not help either.
And that practice of using file timestamp is doubly bad by leading misguided people to want to set timestamps of all the files by copying commit timestamp. That is backwards, I would think.
Quoted 21 lines · Junio C Hamano
Quoted 13 lines · Olaf Hering
Several files in Documentation have an unstable 'Last updated' timestamp. The reason is that their mtime changes every time, which prevents reproducible builds.
341
technical/api-index.txt:technical/api-index-skel.txt\
342technical/api-index.sh$(patsubst %,%.txt,$(API_DOCS))
343 $(QUIET_GEN)cd technical && '$(SHELL_PATH_SQ)' ./api-index.sh388 howto-index.txt:
howto-index.sh$(wildcard howto/*.txt)389 $(QUIET_GEN)$(RM) $@+ $@ && \ 390 '$(SHELL_PATH_SQ)' ./howto-index.sh $(sort $(wildcard howto/*.txt)) >$@+ && \ 391 mv $@+ $@Quoted 5 lines
399 $(patsubst %.txt,%.html,$(wildcard howto/*.txt)): %.html : %.txt
400 $(QUIET_ASCIIDOC)$(RM) $@+ $@ && \ 401 sed -e '1,/^$$/d' $< | \ 402 $(TXT_TO_HTML) - >$@+ && \ 403 mv $@+ $@
Quoted 3 lines
All of them seem to have dependencies so it seems to me that two builds back to back without actually changing their input would not re-build anything. What am I missing???
Per default the files referenced do not exist, so a build from releases will generate the files. But I agree, perhaps there should be some refactoring to handle dependencies properly.
In any case, if the 'Last updated' is unavoidable some reference is required, see my patch in the other mail.
Quoted 5 lines
Quoted 1 line
What file timestamp should be used for them? Likely "../version"?
I tend to think the "Last updated" timestamp taken from the filesystem timestamp is a bad practice inherited by these tools from the days back when nobody used any revision control systems.
I'm not sure. The bug is that such 'Last updated' line exists at all in the default output. Noone asked for it, noone really needs it. And it makes it impossible to get reproducible builds.
But so far I found no way to avoid the 'Last updated' output. The docs and google suggest something like :last-updated-time!:, which did not work for me.
Olaf
Quoted 11 lines · Olaf Hering
Quoted 5 lines · Junio C Hamano
Quoted 1 line
What file timestamp should be used for them? Likely "../version"?
I tend to think the "Last updated" timestamp taken from the filesystem timestamp is a bad practice inherited by these tools from the days back when nobody used any revision control systems.
I'm not sure. The bug is that such 'Last updated' line exists at all in the default output. Noone asked for it, noone really needs it. And it makes it impossible to get reproducible builds.
Amen to that ;-)
Junio C Hamano schrieb am 29.01.2015 um 07:18:
Quoted 10 lines
Quoted 8 lines · Olaf Hering
Quoted 5 lines · Junio C Hamano
Quoted 1 line
What file timestamp should be used for them? Likely "../version"?
I tend to think the "Last updated" timestamp taken from the filesystem timestamp is a bad practice inherited by these tools from the days back when nobody used any revision control systems.
Quoted 6 lines
Quoted 3 lines
I'm not sure. The bug is that such 'Last updated' line exists at all in the default output. Noone asked for it, noone really needs it. And it makes it impossible to get reproducible builds.
Amen to that ;-)
It's a shame one can't simply replace the [footer-text] template which asciidoc insists on.
It turns out asciidoc 8.6.9-3 and later will habe a knob to turn:
https://github.com/asciidoc/asciidoc/pull/9
I'll try and get my hands on it to see whether we can simply use that. I'm wondering though which is more useful - the version of the tree the doc is processed from, or the version of the last commit changing the corresponding doc source file. The first one changes even when the doc source is unchanged (but is stable between reruns, of course).
Michael
Quoted 5 lines · Michael J Gruber
It's a shame one can't simply replace the [footer-text] template which asciidoc insists on.
It turns out asciidoc 8.6.9-3 and later will habe a knob to turn:
Quoted 7 lines
https://github.com/asciidoc/asciidoc/pull/9
I'll try and get my hands on it to see whether we can simply use that. I'm wondering though which is more useful - the version of the tree the doc is processed from, or the version of the last commit changing the corresponding doc source file. The first one changes even when the doc source is unchanged (but is stable between reruns, of course).
I have 8.6.9-3 installed (it is part of Debian testing/unstable now), and confirmed that:
Documentation/asciidoc.conf +1 -0
@@ -21,6 +21,7 @@ tilde=~ apostrophe=' backtick=` litdd=-- +footer-style=none ifdef::backend-docbook[] [linkgit-inlinemacro]
drops the "last-updated" footer.
But note that this only affects the generated HTML. The manpages still get the date in their footer. But this isn't an asciidoc-ism at all; it's added by docbook when converting the xml to roff. I'm sure there is a way to tweak that, too, but looking at docbook gives me nightmares.
-Peff
Quoted 16 lines · Jeff King
I have 8.6.9-3 installed (it is part of Debian testing/unstable now), and confirmed that:
Documentation/asciidoc.conf +1
-0@@ -21,6 +21,7 @@ tilde=~ apostrophe=' backtick=` litdd=-- +footer-style=none ifdef::backend-docbook[] [linkgit-inlinemacro]
Quoted 1 line
drops the "last-updated" footer.
This does not remove "Last updated" from files like using-merge-subtree.html for me, using asciidoc-8.6.8.
Olaf
Quoted 15 lines · Olaf Hering
Quoted 13 lines · Jeff King
I have 8.6.9-3 installed (it is part of Debian testing/unstable now), and confirmed that:
Documentation/asciidoc.conf +1
-0@@ -21,6 +21,7 @@ tilde=~ apostrophe=' backtick=` litdd=-- +footer-style=none
Quoted 7 lines
Quoted 4 lines
Documentation/asciidoc.conf +0
-0ifdef::backend-docbook[] [linkgit-inlinemacro]drops the "last-updated" footer.
This does not remove "Last updated" from files like using-merge-subtree.html for me, using asciidoc-8.6.8.
Yes, the feature is part of 8.6.9-3.
-Peff
Hey, asciidoc made a move, so this patch is good to go:
https://github.com/asciidoc/asciidoc/pull/9
Thanks.
Olaf
Quoted 28 lines · Jeff King
Quoted 15 lines · Olaf Hering
Quoted 13 lines · Jeff King
I have 8.6.9-3 installed (it is part of Debian testing/unstable now), and confirmed that:
Documentation/asciidoc.conf +1
-0@@ -21,6 +21,7 @@ tilde=~ apostrophe=' backtick=` litdd=-- +footer-style=noneQuoted 7 lines
Quoted 4 lines
Documentation/asciidoc.conf +0
-0ifdef::backend-docbook[] [linkgit-inlinemacro]drops the "last-updated" footer.
This does not remove "Last updated" from files like using-merge-subtree.html for me, using asciidoc-8.6.8.
Yes, the feature is part of 8.6.9-3.
-Peff
Quoted 2 lines · Olaf Hering
Hey, asciidoc made a move, so this patch is good to go:
https://github.com/asciidoc/asciidoc/pull/9
Sine this thread is 18 months old, I needed some recap to remember what we were talking about. :)
It's here:
http://public-inbox.org/git/[email protected]/T/#u
and the gist of it is that we'd like to drop the "Last updated" footer from the HTML version of the manpages, but older versions of asciidoc did not provide a mechanism.
The patch you quoted adds "footer-style=none", which would do the trick. But I have two open questions:
1. What does this do on older versions of asciidoc? Is it silently ignored (ok), or does it generate an error (bad)?
2. This covers the HTML versions, but not the roff manpages (which are generated by docbook). Do we have a way to tweak the date in the latter?
I don't think that's necessarily a requirement for this patch, but it is worth thinking about at the same time.
Assuming the answer to (1) is "ok" and (2) is "no, but it's hard because docbook is scary, so let's punt", then somebody needs to write up the commit message and send the actual patch to the list. Would you like to try that?
-Peff