Skip to contentSkip to conversation

implement a stable 'Last updated' in Documentation

DISCUSSION Olaf Hering 4 participants 11 messages Jun 15, 2016 – Sep 1, 2016 Documentation

lore
Share & tools
lore
#0 lore raw

Several files in Documentation have an unstable 'Last updated' timestamp. The reason is that their mtime changes every time, which prevents reproducible builds.

341 technical/api-index.txt: technical/api-index-skel.txt \
342 technical/api-index.sh $(patsubst %,%.txt,$(API_DOCS))
343 $(QUIET_GEN)cd technical && '$(SHELL_PATH_SQ)' ./api-index.sh

388 howto-index.txt: howto-index.sh $(wildcard howto/*.txt)

389         $(QUIET_GEN)$(RM) $@+ $@ && \
390         '$(SHELL_PATH_SQ)' ./howto-index.sh $(sort $(wildcard howto/*.txt)) >$@+ && \
391         mv $@+ $@

399 $(patsubst %.txt,%.html,$(wildcard howto/*.txt)): %.html : %.txt

400         $(QUIET_ASCIIDOC)$(RM) $@+ $@ && \
401         sed -e '1,/^$$/d' $< | \
402         $(TXT_TO_HTML) - >$@+ && \
403         mv $@+ $@

What file timestamp should be used for them? Likely "../version"? The final file, before passing it to asciidoc, should get a fixed timestamp with 'touch -r $reference_file $file'.

Olaf

#1 lore raw
Quoted 14 lines · Olaf Hering

Several files in Documentation have an unstable 'Last updated' timestamp. The reason is that their mtime changes every time, which prevents reproducible builds.

341 technical/api-index.txt: technical/api-index-skel.txt \
342 technical/api-index.sh $(patsubst %,%.txt,$(API_DOCS))
343 $(QUIET_GEN)cd technical && '$(SHELL_PATH_SQ)' ./api-index.sh

388 howto-index.txt: howto-index.sh $(wildcard howto/*.txt)

389         $(QUIET_GEN)$(RM) $@+ $@ && \
390         '$(SHELL_PATH_SQ)' ./howto-index.sh $(sort $(wildcard howto/*.txt)) >$@+ && \
391         mv $@+ $@

399 $(patsubst %.txt,%.html,$(wildcard howto/*.txt)): %.html : %.txt

Quoted 8 lines
400         $(QUIET_ASCIIDOC)$(RM) $@+ $@ && \
401         sed -e '1,/^$$/d' $< | \
402         $(TXT_TO_HTML) - >$@+ && \
403         mv $@+ $@

What file timestamp should be used for them? Likely "../version"? The final file, before passing it to asciidoc, should get a fixed timestamp with 'touch -r $reference_file $file'.

I came up with this change. If anything happens to depend on howto-index.txt and technical/api-index.txt then this part needs rework.

Documentation/Makefile +9 -3
index 3e39e28..25ad0bd 100644
@@ -6,6 +6,7 @@ TECH_DOCS =
 ARTICLES =
 SP_ARTICLES =
 OBSOLETE_HTML =
+TIMESTAMP_FILE = ../version
 
 MAN1_TXT += $(filter-out \
 		$(addsuffix .txt, $(ARTICLES) $(SP_ARTICLES)), \
@@ -341,6 +342,7 @@ user-manual.xml: user-manual.txt user-manual.conf
 technical/api-index.txt: technical/api-index-skel.txt \
 	technical/api-index.sh $(patsubst %,%.txt,$(API_DOCS))
 	$(QUIET_GEN)cd technical && '$(SHELL_PATH_SQ)' ./api-index.sh
+	touch -r '$(TIMESTAMP_FILE)' $@
 
 technical/%.html: ASCIIDOC_EXTRA += -a git-relative-html-prefix=../
 $(patsubst %,%.html,$(API_DOCS) technical/api-index $(TECH_DOCS)): %.html : %.txt asciidoc.conf
@@ -388,6 +390,7 @@ $(patsubst %.txt,%.texi,$(MAN_TXT)): %.texi : %.xml
 howto-index.txt: howto-index.sh $(wildcard howto/*.txt)
 	$(QUIET_GEN)$(RM) $@+ $@ && \
 	'$(SHELL_PATH_SQ)' ./howto-index.sh $(sort $(wildcard howto/*.txt)) >$@+ && \
+	touch -r '$(TIMESTAMP_FILE)' $@+ && \
 	mv $@+ $@
 
 $(patsubst %,%.html,$(ARTICLES)) : %.html : %.txt
@@ -397,9 +400,12 @@ WEBDOC_DEST = /pub/software/scm/git/docs
 
 howto/%.html: ASCIIDOC_EXTRA += -a git-relative-html-prefix=../
 $(patsubst %.txt,%.html,$(wildcard howto/*.txt)): %.html : %.txt
-	$(QUIET_ASCIIDOC)$(RM) $@+ $@ && \
-	sed -e '1,/^$$/d' $< | \
-	$(TXT_TO_HTML) - >$@+ && \
+	$(QUIET_ASCIIDOC)$(RM) $@++ $@+ $@ && \
+	sed -e '1,/^$$/d' $< >$@++ && \
+	touch -r '$(TIMESTAMP_FILE)' $@++ && \
+	$(TXT_TO_HTML) -o $@+ $@++ && \
+	$(RM) $@++ && \
+	touch $@+ && \
 	mv $@+ $@
 
 install-webdoc : html

Olaf

#2 lore raw
In reply to Olaf Hering
Quoted 13 lines · Olaf Hering

Several files in Documentation have an unstable 'Last updated' timestamp. The reason is that their mtime changes every time, which prevents reproducible builds.

341 technical/api-index.txt: technical/api-index-skel.txt \
342 technical/api-index.sh $(patsubst %,%.txt,$(API_DOCS))
343 $(QUIET_GEN)cd technical && '$(SHELL_PATH_SQ)' ./api-index.sh

388 howto-index.txt: howto-index.sh $(wildcard howto/*.txt)

389         $(QUIET_GEN)$(RM) $@+ $@ && \
390         '$(SHELL_PATH_SQ)' ./howto-index.sh $(sort $(wildcard howto/*.txt)) >$@+ && \
391         mv $@+ $@
Quoted 5 lines

399 $(patsubst %.txt,%.html,$(wildcard howto/*.txt)): %.html : %.txt

400         $(QUIET_ASCIIDOC)$(RM) $@+ $@ && \
401         sed -e '1,/^$$/d' $< | \
402         $(TXT_TO_HTML) - >$@+ && \
403         mv $@+ $@

All of them seem to have dependencies so it seems to me that two builds back to back without actually changing their input would not re-build anything. What am I missing???

Quoted 1 line

What file timestamp should be used for them? Likely "../version"?

I tend to think the "Last updated" timestamp taken from the filesystem timestamp is a bad practice inherited by these tools from the days back when nobody used any revision control systems.

If I check out v1.8.5 and build documentation now, it does not matter if the generated documentation had the timestamp of the checkout of individual files or that of a single file generated during the build process, i.e. ../version. Neither has much relevance to the time the actual contents of the documentation was prepared or what vintage of the software the documentation is for.

I am fine with branding generated documentation with the version number (i.e. "You are reading the documentation for Git version 2.2.0"). Replacing that statement with "You are reading the documentation for a version of Git that was committed on such and such time" is also fine. But using file timestamp would not help either.

And that practice of using file timestamp is doubly bad by leading misguided people to want to set timestamps of all the files by copying commit timestamp. That is backwards, I would think.

#3 lore raw
Quoted 21 lines · Junio C Hamano
Quoted 13 lines · Olaf Hering

Several files in Documentation have an unstable 'Last updated' timestamp. The reason is that their mtime changes every time, which prevents reproducible builds.

341 technical/api-index.txt: technical/api-index-skel.txt \
342 technical/api-index.sh $(patsubst %,%.txt,$(API_DOCS))
343 $(QUIET_GEN)cd technical && '$(SHELL_PATH_SQ)' ./api-index.sh

388 howto-index.txt: howto-index.sh $(wildcard howto/*.txt)

389         $(QUIET_GEN)$(RM) $@+ $@ && \
390         '$(SHELL_PATH_SQ)' ./howto-index.sh $(sort $(wildcard howto/*.txt)) >$@+ && \
391         mv $@+ $@
Quoted 5 lines

399 $(patsubst %.txt,%.html,$(wildcard howto/*.txt)): %.html : %.txt

400         $(QUIET_ASCIIDOC)$(RM) $@+ $@ && \
401         sed -e '1,/^$$/d' $< | \
402         $(TXT_TO_HTML) - >$@+ && \
403         mv $@+ $@
Quoted 3 lines

All of them seem to have dependencies so it seems to me that two builds back to back without actually changing their input would not re-build anything. What am I missing???

Per default the files referenced do not exist, so a build from releases will generate the files. But I agree, perhaps there should be some refactoring to handle dependencies properly.
In any case, if the 'Last updated' is unavoidable some reference is required, see my patch in the other mail.

Quoted 5 lines
Quoted 1 line

What file timestamp should be used for them? Likely "../version"?

I tend to think the "Last updated" timestamp taken from the filesystem timestamp is a bad practice inherited by these tools from the days back when nobody used any revision control systems.

I'm not sure. The bug is that such 'Last updated' line exists at all in the default output. Noone asked for it, noone really needs it. And it makes it impossible to get reproducible builds.
But so far I found no way to avoid the 'Last updated' output. The docs and google suggest something like :last-updated-time!:, which did not work for me.

Olaf

#4 lore raw
Quoted 11 lines · Olaf Hering
Quoted 5 lines · Junio C Hamano
Quoted 1 line

What file timestamp should be used for them? Likely "../version"?

I tend to think the "Last updated" timestamp taken from the filesystem timestamp is a bad practice inherited by these tools from the days back when nobody used any revision control systems.

I'm not sure. The bug is that such 'Last updated' line exists at all in the default output. Noone asked for it, noone really needs it. And it makes it impossible to get reproducible builds.

Amen to that ;-)

#5 lore raw

Junio C Hamano schrieb am 29.01.2015 um 07:18:

Quoted 10 lines
Quoted 8 lines · Olaf Hering
Quoted 5 lines · Junio C Hamano
Quoted 1 line

What file timestamp should be used for them? Likely "../version"?

I tend to think the "Last updated" timestamp taken from the filesystem timestamp is a bad practice inherited by these tools from the days back when nobody used any revision control systems.

Quoted 6 lines
Quoted 3 lines

I'm not sure. The bug is that such 'Last updated' line exists at all in the default output. Noone asked for it, noone really needs it. And it makes it impossible to get reproducible builds.

Amen to that ;-)

It's a shame one can't simply replace the [footer-text] template which asciidoc insists on.

It turns out asciidoc 8.6.9-3 and later will habe a knob to turn:

https://github.com/asciidoc/asciidoc/pull/9

I'll try and get my hands on it to see whether we can simply use that. I'm wondering though which is more useful - the version of the tree the doc is processed from, or the version of the last commit changing the corresponding doc source file. The first one changes even when the doc source is unchanged (but is stable between reruns, of course).

Michael

#6 lore raw
Quoted 5 lines · Michael J Gruber

It's a shame one can't simply replace the [footer-text] template which asciidoc insists on.

It turns out asciidoc 8.6.9-3 and later will habe a knob to turn:

Quoted 7 lines

https://github.com/asciidoc/asciidoc/pull/9

I'll try and get my hands on it to see whether we can simply use that. I'm wondering though which is more useful - the version of the tree the doc is processed from, or the version of the last commit changing the corresponding doc source file. The first one changes even when the doc source is unchanged (but is stable between reruns, of course).

I have 8.6.9-3 installed (it is part of Debian testing/unstable now), and confirmed that:

Documentation/asciidoc.conf +1 -0
index 2c16c53..10c777e 100644
@@ -21,6 +21,7 @@ tilde=&#126;
 apostrophe=&#39;
 backtick=&#96;
 litdd=&#45;&#45;
+footer-style=none
 
 ifdef::backend-docbook[]
 [linkgit-inlinemacro]

drops the "last-updated" footer.

But note that this only affects the generated HTML. The manpages still get the date in their footer. But this isn't an asciidoc-ism at all; it's added by docbook when converting the xml to roff. I'm sure there is a way to tweak that, too, but looking at docbook gives me nightmares.

-Peff

#7 lore raw
Quoted 16 lines · Jeff King

I have 8.6.9-3 installed (it is part of Debian testing/unstable now), and confirmed that:

Documentation/asciidoc.conf +1 -0
index 2c16c53..10c777e 100644
@@ -21,6 +21,7 @@ tilde=&#126;
 apostrophe=&#39;
 backtick=&#96;
 litdd=&#45;&#45;
+footer-style=none
 
 ifdef::backend-docbook[]
 [linkgit-inlinemacro]
Quoted 1 line

drops the "last-updated" footer.

This does not remove "Last updated" from files like using-merge-subtree.html for me, using asciidoc-8.6.8.

Olaf

#8 lore raw
Quoted 15 lines · Olaf Hering
Quoted 13 lines · Jeff King

I have 8.6.9-3 installed (it is part of Debian testing/unstable now), and confirmed that:

Documentation/asciidoc.conf +1 -0
index 2c16c53..10c777e 100644
@@ -21,6 +21,7 @@ tilde=&#126;
 apostrophe=&#39;
 backtick=&#96;
 litdd=&#45;&#45;
+footer-style=none
 
Quoted 7 lines
Quoted 4 lines
Documentation/asciidoc.conf +0 -0
 ifdef::backend-docbook[]
 [linkgit-inlinemacro]

drops the "last-updated" footer.

This does not remove "Last updated" from files like using-merge-subtree.html for me, using asciidoc-8.6.8.

Yes, the feature is part of 8.6.9-3.

-Peff

#9 lore raw

Hey, asciidoc made a move, so this patch is good to go:
https://github.com/asciidoc/asciidoc/pull/9

Thanks.

Olaf

Quoted 28 lines · Jeff King
Quoted 15 lines · Olaf Hering
Quoted 13 lines · Jeff King

I have 8.6.9-3 installed (it is part of Debian testing/unstable now), and confirmed that:

Documentation/asciidoc.conf +1 -0
index 2c16c53..10c777e 100644
@@ -21,6 +21,7 @@ tilde=&#126;
 apostrophe=&#39;
 backtick=&#96;
 litdd=&#45;&#45;
+footer-style=none
 
Quoted 7 lines
Quoted 4 lines
Documentation/asciidoc.conf +0 -0
 ifdef::backend-docbook[]
 [linkgit-inlinemacro]

drops the "last-updated" footer.

This does not remove "Last updated" from files like using-merge-subtree.html for me, using asciidoc-8.6.8.

Yes, the feature is part of 8.6.9-3.

-Peff

#10 lore raw
Quoted 2 lines · Olaf Hering

Hey, asciidoc made a move, so this patch is good to go:
https://github.com/asciidoc/asciidoc/pull/9

Sine this thread is 18 months old, I needed some recap to remember what we were talking about. :)

It's here:

http://public-inbox.org/git/[email protected]/T/#u

and the gist of it is that we'd like to drop the "Last updated" footer from the HTML version of the manpages, but older versions of asciidoc did not provide a mechanism.

The patch you quoted adds "footer-style=none", which would do the trick. But I have two open questions:

1. What does this do on older versions of asciidoc? Is it silently ignored (ok), or does it generate an error (bad)?

2. This covers the HTML versions, but not the roff manpages (which are generated by docbook). Do we have a way to tweak the date in the latter?

I don't think that's necessarily a requirement for this patch, but it is worth thinking about at the same time.

Assuming the answer to (1) is "ok" and (2) is "no, but it's hard because docbook is scary, so let's punt", then somebody needs to write up the commit message and send the actual patch to the list. Would you like to try that?

-Peff