manual-text generation improvements
PATCH v4 1/1 manual-text generation improvements
This patch series makes some improvements to the generation of the text version of the manual:
- make sure that URLs are visible
- remove the 'logo' (logo.png) reference
To achieve this, three preparatory patches are needed.
Note that this series stands on its own, it can already be applied. A subsequent series will make structural and content-wise changes to the manual itself.
---
v4: fix 'make clean' (detected by Samuel)
v3: fix 'make manual' without .config (extra patch) (detected by Yann)
v2: refine patch 4 (changing URLs)
Thomas De Schampheleire (5)
Makefile: unconditionally include pkg-utils.mk
gendoc infra: use $(pkgname) instead of explicitly passing 'manual' gendoc infra: add support for asciidoc configuration files
manual-text: make sure URLs are displayed if a link text was provided
manual-text: hide image representations
Makefile | 2 ++ docs/manual/asciidoc-text.conf | 23 +++++++++++++++++++++++ docs/manual/manual.mk | 28 +++++++++++++++++----------- package/Makefile.in | 1 - 4 files changed, 42 insertions(+), 12 deletions(-)
PATCH v4 1/1 Makefile: unconditionally include pkg-utils.mk
Currently, pkg-utils.mk (included via package/Makefile.in) is only included when a configuration file already exists. This means that none of the utilities it defines are available without .config.
In particular:
- the MESSAGE macro, causing pretty build output. Since some make targets can be run even without .config, like 'make manual', not having this pretty printing is odd.
- pkgname, pkgdir: in a subsequent patch, these functions will be used for the generation of the manual, and since this should work also without .config, we need these functions to be available.
This patch moves the include of pkg-utils.mk from package/Makefile.in to Makefile, outside of the check for .config.
This is a quick fix. The full solution involves to minimize the amount of Makefile code that is guarded by a check on .config. This approach will be taken in the 2014.11 release cycle.
---
v4: no changes
v3: new patch, fixing the issue detected by Yann that 'make manual' no longer worked without .config
Makefile | 2 ++ package/Makefile.in | 1 - 2 files changed, 2 insertions(+), 1 deletions(-)
Makefile +2 -0
@@ -271,6 +271,8 @@ # Causes breakage with packages that needs host-ruby unexport RUBYOPT +include package/pkg-utils.mk + ifeq ($(BR2_HAVE_DOT_CONFIG),y) ################################################################################
package/Makefile.in +0 -1
@@ -385,7 +385,6 @@ SHARED_STATIC_LIBS_OPTS = --enable-static --enable-shared endif -include package/pkg-utils.mk include package/pkg-download.mk include package/pkg-autotools.mk include package/pkg-cmake.mk
PATCH v4 1/1 gendoc infra: use $(pkgname) instead of explicitly passing 'manual'
In reply to Thomas De SchampheleireIn the gendoc infrastructure, using an assignment of the form
FOO = docs/$(1)/bar
inside GENDOC_INNER does not work as expected: the $(1) value is empty here and the value of FOO becomes 'docs//bar'.
Parameters $(2), $(3), etc. do not have this problem. The specific thing about $(1) is that it is a parameter to GENDOC itself (indicating the document to create) and passed transparently to GENDOC_INNER.
This is different from the package infrastructures, where $(1) is set from $(pkgname). In fact, the same strategy could be used by the gendoc infrastructure as well, as $(pkgname) resolves to 'manual' for file docs/manual/manual.mk. This has the advantage that the described problem does not occur.
Note that this means that if we want to use the same GENDOC infrastructure for another document, it will have to reside in a separate directory than the manual.
---
v4: fix 'make clean' due to missing replacement of $(1) (Samuel)
v3: no changes
v2: no changes
docs/manual/manual.mk | 20 ++++++++++---------- 1 files changed, 10 insertions(+), 10 deletions(-)
docs/manual/manual.mk +10 -10
@@ -78,16 +78,16 @@ # The variable <DOCUMENT_NAME>_SOURCES defines the dependencies. ################################################################################ define GENDOC -$(call GENDOC_INNER,$(1),xhtml,html,html,HTML,--xsltproc-opts "--stringparam toc.section.depth 2") -$(call GENDOC_INNER,$(1),chunked,split-html,chunked,split HTML,--xsltproc-opts "--stringparam toc.section.depth 2") -$(call GENDOC_INNER,$(1),pdf,pdf,pdf,PDF,--dblatex-opts "-P latex.output.revhistory=0") -$(call GENDOC_INNER,$(1),text,text,text,text) -$(call GENDOC_INNER,$(1),epub,epub,epub,ePUB) -clean: $(1)-clean -$(1)-clean: - $$(Q)$$(RM) -rf $$(O)/docs/$(1) -.PHONY: $(1) $(1)-clean manual-update-lists +$(call GENDOC_INNER,$(pkgname),xhtml,html,html,HTML,--xsltproc-opts "--stringparam toc.section.depth 2") +$(call GENDOC_INNER,$(pkgname),chunked,split-html,chunked,split HTML,--xsltproc-opts "--stringparam toc.section.depth 2") +$(call GENDOC_INNER,$(pkgname),pdf,pdf,pdf,PDF,--dblatex-opts "-P latex.output.revhistory=0") +$(call GENDOC_INNER,$(pkgname),text,text,text,text) +$(call GENDOC_INNER,$(pkgname),epub,epub,epub,ePUB) +clean: $(pkgname)-clean +$(pkgname)-clean: + $$(Q)$$(RM) -rf $$(O)/docs/$(pkgname) +.PHONY: $(pkgname) $(pkgname)-clean manual-update-lists endef MANUAL_SOURCES = $(sort $(wildcard docs/manual/*.txt) $(wildcard docs/images/*)) -$(eval $(call GENDOC,manual)) +$(eval $(call GENDOC))
PATCH v4 1/1 gendoc infra: add support for asciidoc configuration files
In reply to Thomas De SchampheleireThis patch introduces support for asciidoc configuration files, specific for each output format (html, text, pdf, ...). This is useful to make certain tweaks in the document generation.
If a file docs/manual/asciidoc-<format>.conf is present, it is passed to asciidoc as configuration file. If no file for the current format is present, the options passed to asciidoc are empty.
---
v4: no changes
v3: no changes
v2: no changes
docs/manual/manual.mk | 8 +++++++- 1 files changed, 7 insertions(+), 1 deletions(-)
docs/manual/manual.mk +7 -1
@@ -56,6 +56,11 @@ manual-check-dependencies-$(3): +MANUAL_$(2)_ASCIIDOC_CONF = docs/$(1)/asciidoc-$(2).conf +ifneq ($$(wildcard $$(MANUAL_$(2)_ASCIIDOC_CONF)),) +MANUAL_$(2)_ASCIIDOC_OPTS += -f $$(MANUAL_$(2)_ASCIIDOC_CONF) +endif + $$(O)/docs/$(1)/$(1).$(4): docs/$(1)/$(1).txt \ $$($$(call UPPERCASE,$(1))_SOURCES) \ manual-check-dependencies \
@@ -65,7 +70,8 @@ $$(Q)mkdir -p $$(@D)/.build $$(Q)rsync -au docs/$(1)/*.txt $$(@D)/.build $$(Q)a2x $(6) -f $(2) -d book -L -r $$(TOPDIR)/docs/images \ - -D $$(@D) $$(@D)/.build/$(1).txt + -D $$(@D) $$(@D)/.build/$(1).txt \ + --asciidoc-opts="$$(MANUAL_$(2)_ASCIIDOC_OPTS)" -$$(Q)rm -rf $$(@D)/.build endef
PATCH v4 1/1 manual-text: make sure URLs are displayed if a link text was provided
In reply to Thomas De SchampheleireWhen the asciidoc source contain URLs of the form:
http://example.com[An example website]
the text representation of the manual would only contain:
An example website
without displaying the actual URL.
This patch adds an asciidoc configuration file that sets the inline macros for several URL types so that the display becomes:
An example website [http://example.com]
For URLs where no link text was provided, the display becomes:
http://example.com
which is the same as before.
---
v4: no changes
v3: no changes
v2:
- link to relevant documentation (which was hard to find)
- refine so that the display is identical to before when no link text is provided.
docs/manual/asciidoc-text.conf | 17 +++++++++++++++++ 1 files changed, 17 insertions(+), 0 deletions(-)
docs/manual/asciidoc-text.conf new file +17 -0
@@ -0,0 +1,17 @@ +# Refer to following asciidoc documentation: +# http://www.methods.co.nz/asciidoc/userguide.html +# In particular sections "Macros" and "Attribute References" +# +# For hyperlinks, show 'link text [URL]' (if link text provided) or 'URL' +[http-inlinemacro] +{0=}{0? [}{name}:{target}{0?]} +[https-inlinemacro] +{0=}{0? [}{name}:{target}{0?]} +[ftp-inlinemacro] +{0=}{0? [}{name}:{target}{0?]} +[file-inlinemacro] +{0=}{0? [}{name}:{target}{0?]} +[irc-inlinemacro] +{0=}{0? [}{name}:{target}{0?]} +[mailto-inlinemacro] +{0=}{0? [}{name}:{target}{0?]}
PATCH v4 1/1 manual-text: hide image representations
In reply to Thomas De SchampheleireImages specified in the asciidoc sources (currently only the logo) are displayed as the file name in the text version of the manual. This causes an odd line to appear:
logo.png
Avoid this by setting the image representation macros to {empty} in manual-text.
---
v4: no changes
v3: no changes
v2: no changes
docs/manual/asciidoc-text.conf | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)
docs/manual/asciidoc-text.conf +6 -0
@@ -15,3 +15,9 @@ {0=}{0? [}{name}:{target}{0?]} [mailto-inlinemacro] {0=}{0? [}{name}:{target}{0?]} + +# Hide image representation from text manual +[image-inlinemacro] +{empty} +[image-blockmacro] +{empty}
PATCH v4 1/1 gendoc infra: use $(pkgname) instead of explicitly passing 'manual'
In reply to Thomas De SchampheleireQuoted 15 lines · Thomas De Schampheleire
In the gendoc infrastructure, using an assignment of the form
FOO = docs/$(1)/barinside
GENDOC_INNERdoes not work as expected: the $(1) value is empty here and the value of FOO becomes 'docs//bar'.Parameters $(2), $(3), etc. do not have this problem. The specific thing about $(1) is that it is a parameter to GENDOC itself (indicating the document to create) and passed transparently to
GENDOC_INNER.This is different from the package infrastructures, where $(1) is set from $(pkgname). In fact, the same strategy could be used by the gendoc infrastructure as well, as $(pkgname) resolves to 'manual' for file
docs/manual/manual.mk. This has the advantage that the described problem does not occur.
Quoted 5 lines
Note that this means that if we want to use the same GENDOC infrastructure for another document, it will have to reside in a separate directory than the manual.
Signed-off-by Thomas De Schampheleire <[email protected]>
Regards,
Signature · 1 line
Samuel
PATCH v4 1/1 manual-text generation improvements
In reply to Thomas De SchampheleireHi all,
Quoted 5 lines · Thomas De Schampheleire
This patch series makes some improvements to the generation of the text version of the manual:
- make sure that URLs are visible
- remove the 'logo' (logo.png) reference
Quoted 7 lines
To achieve this, three preparatory patches are needed.
Note that this series stands on its own, it can already be applied. A subsequent series will make structural and content-wise changes to the manual itself.
Signed-off-by Thomas De Schampheleire <[email protected]>
Signature · 1 line
Samuel
PATCH v4 1/1 Makefile: unconditionally include pkg-utils.mk
In reply to Thomas De SchampheleireThomas, All,
On 2014-08-15 15:40 +0200, Thomas De Schampheleire spake thusly:
Quoted 16 lines
Currently, pkg-utils.mk (included via package/Makefile.in) is only included when a configuration file already exists. This means that none of the utilities it defines are available without .config.
In particular:
- the MESSAGE macro, causing pretty build output. Since some make targets can be run even without .config, like 'make manual', not having this pretty printing is odd.- pkgname, pkgdir: in a subsequent patch, these functions will be used for the generation of the manual, and since this should work also without .config, we need these functions to be available.
This patch moves the include of pkg-utils.mk from package/Makefile.in to
Makefile, outside of the check for .config.
Quoted 6 lines
This is a quick fix. The full solution involves to minimize the amount of
Makefilecode that is guarded by a check on .config. This approach will be taken in the 2014.11 release cycle.Signed-off-by Thomas De Schampheleire <[email protected]>Reviewed-by Samuel Martin <[email protected]>
Tested by generating the manual with and without a .config. Tested that running make with no .config will popup the menuconfig. Tested that a defconfig still builds OK.
Regards,
Yann E. MORIN.
Quoted 32 lines
---
v4: no changes
v3: new patch, fixing the issue detected by Yann that 'make manual' no longer worked without .configMakefile | 2 ++ package/Makefile.in | 1 - 2 files changed, 2 insertions(+), 1 deletions(-)diff -r
3bcf63a46bc1-r232206819ca1MakefileMakefile +2
-0@@ -271,6 +271,8 @@ # Causes breakage with packages that needs host-ruby unexport RUBYOPT +include package/pkg-utils.mk + ifeq ($(BR2_HAVE_DOT_CONFIG),y) ################################################################################diff -r
3bcf63a46bc1-r232206819ca1package/Makefile.inpackage/Makefile.in +0
-1@@ -385,7 +385,6 @@ SHARED_STATIC_LIBS_OPTS = --enable-static --enable-shared endif -include package/pkg-utils.mk include package/pkg-download.mk include package/pkg-autotools.mk include package/pkg-cmake.mk
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH v4 1/1 gendoc infra: use $(pkgname) instead of explicitly passing 'manual'
In reply to Thomas De SchampheleireTHomas, All,
On 2014-08-15 15:40 +0200, Thomas De Schampheleire spake thusly:
Quoted 15 lines
In the gendoc infrastructure, using an assignment of the form
FOO = docs/$(1)/barinside
GENDOC_INNERdoes not work as expected: the $(1) value is empty here and the value of FOO becomes 'docs//bar'.Parameters $(2), $(3), etc. do not have this problem. The specific thing about $(1) is that it is a parameter to GENDOC itself (indicating the document to create) and passed transparently to
GENDOC_INNER.This is different from the package infrastructures, where $(1) is set from $(pkgname). In fact, the same strategy could be used by the gendoc infrastructure as well, as $(pkgname) resolves to 'manual' for file
docs/manual/manual.mk. This has the advantage that the described problem does not occur.
Quoted 5 lines
Note that this means that if we want to use the same GENDOC infrastructure for another document, it will have to reside in a separate directory than the manual.
Signed-off-by Thomas De Schampheleire <[email protected]>
Tested by building the manual with and without a .config.
Regards,
Yann E. MORIN.
Quoted 38 lines
---
v4: fix 'make clean' due to missing replacement of $(1) (Samuel)
v3: no changes
v2: no changesdocs/manual/manual.mk | 20 ++++++++++---------- 1 files changed, 10 insertions(+), 10 deletions(-)diff -r
232206819ca1-r9d9af165df7bdocs/manual/manual.mkdocs/manual/manual.mk +10
-10@@ -78,16 +78,16 @@ # The variable <DOCUMENT_NAME>_SOURCES defines the dependencies. ################################################################################ define GENDOC -$(call GENDOC_INNER,$(1),xhtml,html,html,HTML,--xsltproc-opts "--stringparam toc.section.depth 2") -$(call GENDOC_INNER,$(1),chunked,split-html,chunked,split HTML,--xsltproc-opts "--stringparam toc.section.depth 2") -$(call GENDOC_INNER,$(1),pdf,pdf,pdf,PDF,--dblatex-opts "-P latex.output.revhistory=0") -$(call GENDOC_INNER,$(1),text,text,text,text) -$(call GENDOC_INNER,$(1),epub,epub,epub,ePUB) -clean: $(1)-clean -$(1)-clean: - $$(Q)$$(RM) -rf $$(O)/docs/$(1) -.PHONY: $(1) $(1)-clean manual-update-lists +$(call GENDOC_INNER,$(pkgname),xhtml,html,html,HTML,--xsltproc-opts "--stringparam toc.section.depth 2") +$(call GENDOC_INNER,$(pkgname),chunked,split-html,chunked,split HTML,--xsltproc-opts "--stringparam toc.section.depth 2") +$(call GENDOC_INNER,$(pkgname),pdf,pdf,pdf,PDF,--dblatex-opts "-P latex.output.revhistory=0") +$(call GENDOC_INNER,$(pkgname),text,text,text,text) +$(call GENDOC_INNER,$(pkgname),epub,epub,epub,ePUB) +clean: $(pkgname)-clean +$(pkgname)-clean: + $$(Q)$$(RM) -rf $$(O)/docs/$(pkgname) +.PHONY: $(pkgname) $(pkgname)-clean manual-update-lists endef MANUAL_SOURCES = $(sort $(wildcard docs/manual/*.txt) $(wildcard docs/images/*)) -$(eval $(call GENDOC,manual)) +$(eval $(call GENDOC))
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH v4 1/1 gendoc infra: add support for asciidoc configuration files
In reply to Thomas De SchampheleireThomas, All,
On 2014-08-15 15:40 +0200, Thomas De Schampheleire spake thusly:
Quoted 10 lines
This patch introduces support for asciidoc configuration files, specific for each output format (html, text, pdf, ...). This is useful to make certain tweaks in the document generation.
If a file docs/manual/asciidoc-<format>.conf is present, it is passed to asciidoc as configuration file. If no file for the current format is present, the options passed to asciidoc are empty.
Signed-off-by Thomas De Schampheleire <[email protected]>Acked-by Samuel Martin <[email protected]>
Regards,
Yann E. MORIN.
Quoted 33 lines
---
v4: no changes
v3: no changes
v2: no changesdocs/manual/manual.mk | 8 +++++++- 1 files changed, 7 insertions(+), 1 deletions(-)diff -r
9d9af165df7b-r81d40b841073docs/manual/manual.mkdocs/manual/manual.mk +7
-1@@ -56,6 +56,11 @@ manual-check-dependencies-$(3): +MANUAL_$(2)_ASCIIDOC_CONF = docs/$(1)/asciidoc-$(2).conf +ifneq ($$(wildcard $$(MANUAL_$(2)_ASCIIDOC_CONF)),) +MANUAL_$(2)_ASCIIDOC_OPTS += -f $$(MANUAL_$(2)_ASCIIDOC_CONF) +endif + $$(O)/docs/$(1)/$(1).$(4): docs/$(1)/$(1).txt \ $$($$(call UPPERCASE,$(1))_SOURCES) \ manual-check-dependencies \@@ -65,7 +70,8 @@ $$(Q)mkdir -p $$(@D)/.build $$(Q)rsync -au docs/$(1)/*.txt $$(@D)/.build $$(Q)a2x $(6) -f $(2) -d book -L -r $$(TOPDIR)/docs/images \ - -D $$(@D) $$(@D)/.build/$(1).txt + -D $$(@D) $$(@D)/.build/$(1).txt \ + --asciidoc-opts="$$(MANUAL_$(2)_ASCIIDOC_OPTS)" -$$(Q)rm -rf $$(@D)/.build endef
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH v4 1/1 manual-text: make sure URLs are displayed if a link text was provided
In reply to Thomas De SchampheleireThomas, All,
On 2014-08-15 15:40 +0200, Thomas De Schampheleire spake thusly:
Quoted 13 lines
When the asciidoc source contain URLs of the form:
http://example.com[An example website]
the text representation of the manual would only contain:
An example website
without displaying the actual URL.This patch adds an asciidoc configuration file that sets the inline macros for several URL types so that the display becomes:
An example website [http://example.com]
For URLs where no link text was provided, the display becomes:
http://example.com
which is the same as before.
Quoted 2 lines
Signed-off-by Thomas De Schampheleire <[email protected]>Acked-by Samuel Martin <[email protected]>
Regards,
Yann E. MORIN.
Quoted 32 lines
---
v4: no changes
v3: no changes
v2:
- link to relevant documentation (which was hard to find)
- refine so that the display is identical to before when no link text is provided.docs/manual/asciidoc-text.conf | 17 +++++++++++++++++ 1 files changed, 17 insertions(+), 0 deletions(-)diff -r
81d40b841073-r93b82c1cf8a4docs/manual/asciidoc-text.confdocs/manual/asciidoc-text.conf new file +17
-0@@ -0,0 +1,17 @@ +# Refer to following asciidoc documentation: +# http://www.methods.co.nz/asciidoc/userguide.html +# In particular sections "Macros" and "Attribute References" +# +# For hyperlinks, show 'link text [URL]' (if link text provided) or 'URL' +[http-inlinemacro] +{0=}{0? [}{name}:{target}{0?]} +[https-inlinemacro] +{0=}{0? [}{name}:{target}{0?]} +[ftp-inlinemacro] +{0=}{0? [}{name}:{target}{0?]} +[file-inlinemacro] +{0=}{0? [}{name}:{target}{0?]} +[irc-inlinemacro] +{0=}{0? [}{name}:{target}{0?]} +[mailto-inlinemacro] +{0=}{0? [}{name}:{target}{0?]}
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH v4 1/1 manual-text: hide image representations
In reply to Thomas De SchampheleireThomas, All,
On 2014-08-15 15:40 +0200, Thomas De Schampheleire spake thusly:
Quoted 10 lines
Images specified in the asciidoc sources (currently only the logo) are displayed as the file name in the text version of the manual. This causes an odd line to appear:
logo.pngAvoid this by setting the image representation macros to {empty} in manual-text.
Signed-off-by Thomas De Schampheleire <[email protected]>Acked-by Samuel Martin <[email protected]>
Tested that the filenames no longer appears, and that's the only difference. :-)
Regards,
Yann E. MORIN.
Quoted 21 lines
---
v4: no changes
v3: no changes
v2: no changesdocs/manual/asciidoc-text.conf | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-)diff -r
93b82c1cf8a4-r6b930ce9f5cbdocs/manual/asciidoc-text.confdocs/manual/asciidoc-text.conf +6
-0@@ -15,3 +15,9 @@ {0=}{0? [}{name}:{target}{0?]} [mailto-inlinemacro] {0=}{0? [}{name}:{target}{0?]} + +# Hide image representation from text manual +[image-inlinemacro] +{empty} +[image-blockmacro] +{empty}
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
Thomas Petazzoni
AppliedPATCH v4 1/1 manual-text generation improvements
In reply to Thomas De SchampheleireDear Thomas De Schampheleire,
Quoted 6 lines · Thomas De Schampheleire
Thomas De Schampheleire (5)
Makefile: unconditionally include pkg-utils.mk
gendoc infra: use $(pkgname) instead of explicitly passing 'manual' gendoc infra: add support for asciidoc configuration files
manual-text: make sure URLs are displayed if a link text was provided
manual-text: hide image representations
Nice, you've added the list of patches to the cover letter :-)
I've applied the patch series, thanks to you, and to Yann and Samuel for testing/reviewing.
Related to this patch series, here are a few things I noted:
* When building the PDF and txt versions of the manual, I get one warning:
a2x: WARNING: --destination-dir option is only applicable to HTML based outputs
It seems to be harmless as the result is really located in the right place (output/docs), but I found it worth mentioning.
* Also, here the PDF version of the manual doesn't build:
a2x: ERROR: "dblatex" -t pdf -p "/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -P latex.output.revhistory=0 "/home/thomas/projets/buildroot/output/docs/manual/manual.xml" returned non-zero exit status 1
It's most likely due to something missing or a wrong version in my installation, but it also means we're maybe not checking for all the necessary tools before generating the manual.
Both of those 'issues' are not introduced by your patch series, they existed before it.
Thanks!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
PATCH v4 1/1 manual-text generation improvements
Quoted 4 lines · Thomas Petazzoni
Dear Thomas De Schampheleire,
On Fri, 15 Aug 2014 15:40:33 +0200, Thomas De Schampheleire wrote:
Quoted 8 lines
Quoted 6 lines
Thomas De Schampheleire (5)
Makefile: unconditionally include pkg-utils.mk
gendoc infra: use $(pkgname) instead of explicitly passing 'manual' gendoc infra: add support for asciidoc configuration files
manual-text: make sure URLs are displayed if a link text was provided
manual-text: hide image representationsNice, you've added the list of patches to the cover letter :-)
Yes :-)
Quoted 6 lines
I've applied the patch series, thanks to you, and to Yann and Samuel for testing/reviewing.
Related to this patch series, here are a few things I noted:
Quoted 7 lines
* When building the PDF and txt versions of the manual, I get one warning:
a2x: WARNING: --destination-dir option is only applicable to HTML based outputs
It seems to be harmless as the result is really located in the right place (output/docs), but I found it worth mentioning.
Yes, I see the same. We could pass format specific options to avoid passing -D for non-html formats. Not sure whether it's worth the effort, I will have a look.
Quoted 4 lines
* Also, here the PDF version of the manual doesn't build:
a2x: ERROR: "dblatex" -t pdf -p "
/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -P latex.output.revhistory=0 "/home/thomas/projets/buildroot/output/docs/manual/manual.xml" returned non-zero exit status 1
I will have a look at this too, but I may need your help here. Could you run the same command manually? It should show more output...
Thanks,
Thomas
PATCH v4 1/1 manual-text generation improvements
Quoted 26 lines · Thomas De Schampheleire
Quoted 4 lines · Thomas Petazzoni
Dear Thomas De Schampheleire,
On Fri, 15 Aug 2014 15:40:33 +0200, Thomas De Schampheleire wrote:
Quoted 8 lines
Quoted 6 lines
Thomas De Schampheleire (5)
Makefile: unconditionally include pkg-utils.mk
gendoc infra: use $(pkgname) instead of explicitly passing 'manual' gendoc infra: add support for asciidoc configuration files
manual-text: make sure URLs are displayed if a link text was provided
manual-text: hide image representationsNice, you've added the list of patches to the cover letter :-)
Yes :-)
Quoted 9 lines
I've applied the patch series, thanks to you, and to Yann and Samuel for testing/reviewing.
Related to this patch series, here are a few things I noted:
* When building the PDF and txt versions of the manual, I get one warning:
Quoted 8 lines
Quoted 4 lines
a2x: WARNING: --destination-dir option is only applicable to HTML based outputs
It seems to be harmless as the result is really located in the right place (output/docs), but I found it worth mentioning.
Yes, I see the same. We could pass format specific options to avoid passing -D for non-html formats. Not sure whether it's worth the effort, I will have a look.
I've noticed this too, but have not found a proper way to suppress it.
Quoted 5 lines
Quoted 5 lines
* Also, here the PDF version of the manual doesn't build:
a2x: ERROR: "dblatex" -t pdf -p "
/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -P latex.output.revhistory=0 "/home/thomas/projets/buildroot/output/docs/manual/manual.xml" returned non-zero exit status 1
Few shortcomings:
- the generated appendix tables are only included in the html manual (when running: make manual-clean manual). It seems these files are not regenerated after the 'rm -rf $$(@D)/.build' :-/
- the above error seems related to these generated appendix tables, because when they don't exits, the pdf manual generation succeeds.
This makes me think about refactoring the manual.mk to be closer to the package ones, i.e. having it build directory under $(0)/build, maybe inheriting from generic-package...
Anyway, this is just wild ideas; for the upcoming release there is this pdf issue to fix.
Quoted 10 lines
I will have a look at this too, but I may need your help here. Could you run the same command manually? It should show more output...
Thanks,
Thomas
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot
Regards,
Signature · 1 line
Samuel
PATCH v4 1/1 manual-text generation improvements
Quoted 3 lines · Samuel Martin
Quoted 1 line · Thomas De Schampheleire
On Sat, Aug 16, 2014 at 9:35 AM, Thomas Petazzoni
[...]
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
* When building the PDF and txt versions of the manual, I get one warning:
a2x: WARNING: --destination-dir option is only applicable to HTML based outputs
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
It seems to be harmless as the result is really located in the right place (output/docs), but I found it worth mentioning.
Yes, I see the same. We could pass format specific options to avoid passing -D for non-html formats. Not sure whether it's worth the effort, I will have a look.
I've noticed this too, but have not found a proper way to suppress it.
I have not looked into this.
Quoted 13 lines
Quoted 5 lines
Quoted 5 lines
* Also, here the PDF version of the manual doesn't build:
a2x: ERROR: "dblatex" -t pdf -p "
/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -P latex.output.revhistory=0 "/home/thomas/projets/buildroot/output/docs/manual/manual.xml" returned non-zero exit status 1Few shortcomings:
- the generated appendix tables are only included in the html manual (when running: make manual-clean manual). It seems these files are not regenerated after the 'rm -rf $$(@D)/.build' :-/
- the above error seems related to these generated appendix tables, because when they don't exits, the pdf manual generation succeeds.
Quoted 6 lines
This makes me think about refactoring the manual.mk to be closer to the package ones, i.e. having it build directory under $(0)/build, maybe inheriting from generic-package...
Anyway, this is just wild ideas; for the upcoming release there is this pdf issue to fix.
Well, after diving into asciidoc/dblatex/xslt, I eventually found the problem and the the fix!
Short-long story:
When generating the pdf manual, a2x calls dblatex, which calls xsltproc. Here is a verbose log [1].
During its execution, xsltproc checks for template recursion and reaches its limit set to 15000 when processing the target package-list.txt file (empirically, this limit allows a 1147-entry table).
To raise this limit, a --maxvars option exists in xsltproc program but the latest release of libxslt is buggy and this option is not honored. :-(
Hopefully the fix is already merged upstream [2]. Backporting this patch is enough to get a working xsltproc, but it also means that we now need to build host-libxslt (and a .config) to build the manual.
So, to build the manual we have to run:
$ make defconfig && make host-libxslt manual
The main drawback of this fix is that building the manual could now require building some host-package.
This also makes echo to the include statement of the *.mk files without having a .config file discussion.
Anyway, I will prepare a series fixing this, though it is not a mandatory for the release
Note that we have already released a couple of Buildroot versions without recieving any complaint about this.
[1] http://code.bulix.org/1905fo-86779
[2] https://gitorious.org/libxslt/libxslt/commit/5af7ad745323004984287e48b42712e7305de35c
Regards,
Signature · 1 line
Samuel
PATCH v4 1/1 manual-text generation improvements
Dear Samuel Martin,
Quoted 19 lines · Samuel Martin
When generating the pdf manual, a2x calls dblatex, which calls xsltproc. Here is a verbose log [1].
During its execution, xsltproc checks for template recursion and reaches its limit set to 15000 when processing the target package-list.txt file (empirically, this limit allows a 1147-entry table).
To raise this limit, a --maxvars option exists in xsltproc program but the latest release of libxslt is buggy and this option is not honored. :-(
Hopefully the fix is already merged upstream [2]. Backporting this patch is enough to get a working xsltproc, but it also means that we now need to build host-libxslt (and a .config) to build the manual.
So, to build the manual we have to run:$ make defconfig && make host-libxslt manualThe main drawback of this fix is that building the manual could now require building some host-package.
Quoted 8 lines
This also makes echo to the include statement of the *.mk files without having a .config file discussion.
Anyway, I will prepare a series fixing this, though it is not a mandatory for the release
Note that we have already released a couple of Buildroot versions without recieving any complaint about this.
I'm not sure we should enforce building host-libxslt as a dependency of the manual. Maybe just check the version of xsltproc and bail out with an error message if it's not the appropriate version. I believe most people are just using the pre-generated versions of the manual that are online on Buildroot website. So it's up to us to have the appropriate xsltproc version there.
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com
PATCH v4 1/1 manual-text generation improvements
In reply to Samuel MartinHi Samuel,
Quoted 71 lines · Samuel Martin
Quoted 3 lines · Samuel Martin
Quoted 1 line · Thomas De Schampheleire
On Sat, Aug 16, 2014 at 9:35 AM, Thomas Petazzoni
[...]
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
* When building the PDF and txt versions of the manual, I get one warning:
a2x: WARNING: --destination-dir option is only applicable to HTML based outputs
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
It seems to be harmless as the result is really located in the right place (output/docs), but I found it worth mentioning.
Yes, I see the same. We could pass format specific options to avoid passing -D for non-html formats. Not sure whether it's worth the effort, I will have a look.
I've noticed this too, but have not found a proper way to suppress it.
I have not looked into this.
Quoted 13 lines
Quoted 5 lines
Quoted 5 lines
* Also, here the PDF version of the manual doesn't build:
a2x: ERROR: "dblatex" -t pdf -p "
/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -P latex.output.revhistory=0 "/home/thomas/projets/buildroot/output/docs/manual/manual.xml" returned non-zero exit status 1Few shortcomings:
- the generated appendix tables are only included in the html manual (when running: make manual-clean manual). It seems these files are not regenerated after the 'rm -rf $$(@D)/.build' :-/
- the above error seems related to these generated appendix tables, because when they don't exits, the pdf manual generation succeeds.Quoted 6 lines
This makes me think about refactoring the manual.mk to be closer to the package ones, i.e. having it build directory under $(0)/build, maybe inheriting from generic-package...
Anyway, this is just wild ideas; for the upcoming release there is this pdf issue to fix.Well, after diving into asciidoc/dblatex/xslt, I eventually found the problem and the the fix!
Short-long story:
When generating the pdf manual, a2x calls dblatex, which calls xsltproc. Here is a verbose log [1].
During its execution, xsltproc checks for template recursion and reaches its limit set to 15000 when processing the target package-list.txt file (empirically, this limit allows a 1147-entry table).
To raise this limit, a --maxvars option exists in xsltproc program but the latest release of libxslt is buggy and this option is not honored. :-(
Hopefully the fix is already merged upstream [2]. Backporting this patch is enough to get a working xsltproc, but it also means that we now need to build host-libxslt (and a .config) to build the manual.
So, to build the manual we have to run:$ make defconfig && make host-libxslt manualThe main drawback of this fix is that building the manual could now require building some host-package.
This also makes echo to the include statement of the *.mk files without having a .config file discussion.
Anyway, I will prepare a series fixing this, though it is not a mandatory for the release
Quoted 6 lines
Note that we have already released a couple of Buildroot versions without recieving any complaint about this.
[1] http://code.bulix.org/1905fo-86779
[2] https://gitorious.org/libxslt/libxslt/commit/5af7ad745323004984287e48b42712e7305de35c
I must say I don't fully understand this.
There are two observations:
1. ThomasP and apparantly you too get an error on generating the pdf manual using 'make manual-clean manual'. However, I don't see the same thing, pdf builds fine for me.
Strangely enough, if I remove the '-$$(Q)rm -rf $$(@D)/.build' line (to fix the second point), then I do get this error.
2. None of the manuals except the html one (the first one) contain the package lists. This is due to the removal of the .build directory between each manual generation, which is the directory where the lists are generated.
Removing the directory seems not really needed to me, so why not just avoid that?
Thanks,
Thomas
PATCH v4 1/1 manual-text generation improvements
Hi Thomas,
Quoted 74 lines · Thomas De Schampheleire
Hi Samuel,
Quoted 71 lines · Samuel Martin
Quoted 3 lines · Samuel Martin
Quoted 1 line · Thomas De Schampheleire
On Sat, Aug 16, 2014 at 9:35 AM, Thomas Petazzoni
[...]
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
* When building the PDF and txt versions of the manual, I get one warning:
a2x: WARNING: --destination-dir option is only applicable to HTML based outputs
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
It seems to be harmless as the result is really located in the right place (output/docs), but I found it worth mentioning.
Yes, I see the same. We could pass format specific options to avoid passing -D for non-html formats. Not sure whether it's worth the effort, I will have a look.
I've noticed this too, but have not found a proper way to suppress it.
I have not looked into this.
Quoted 13 lines
Quoted 5 lines
Quoted 5 lines
* Also, here the PDF version of the manual doesn't build:
a2x: ERROR: "dblatex" -t pdf -p "
/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -P latex.output.revhistory=0 "/home/thomas/projets/buildroot/output/docs/manual/manual.xml" returned non-zero exit status 1Few shortcomings:
- the generated appendix tables are only included in the html manual (when running: make manual-clean manual). It seems these files are not regenerated after the 'rm -rf $$(@D)/.build' :-/
- the above error seems related to these generated appendix tables, because when they don't exits, the pdf manual generation succeeds.Quoted 6 lines
This makes me think about refactoring the manual.mk to be closer to the package ones, i.e. having it build directory under $(0)/build, maybe inheriting from generic-package...
Anyway, this is just wild ideas; for the upcoming release there is this pdf issue to fix.Well, after diving into asciidoc/dblatex/xslt, I eventually found the problem and the the fix!
Short-long story:
When generating the pdf manual, a2x calls dblatex, which calls xsltproc. Here is a verbose log [1].
During its execution, xsltproc checks for template recursion and reaches its limit set to 15000 when processing the target package-list.txt file (empirically, this limit allows a 1147-entry table).
To raise this limit, a --maxvars option exists in xsltproc program but the latest release of libxslt is buggy and this option is not honored. :-(
Hopefully the fix is already merged upstream [2]. Backporting this patch is enough to get a working xsltproc, but it also means that we now need to build host-libxslt (and a .config) to build the manual.
So, to build the manual we have to run:$ make defconfig && make host-libxslt manualThe main drawback of this fix is that building the manual could now require building some host-package.
This also makes echo to the include statement of the *.mk files without having a .config file discussion.
Anyway, I will prepare a series fixing this, though it is not a mandatory for the release
Quoted 8 lines
Quoted 6 lines
Note that we have already released a couple of Buildroot versions without recieving any complaint about this.
[1] http://code.bulix.org/1905fo-86779
[2] https://gitorious.org/libxslt/libxslt/commit/5af7ad745323004984287e48b42712e7305de35cI must say I don't fully understand this.
Ok, I'll try to be clearer.
Quoted 9 lines
There are two observations:
1. ThomasP and apparantly you too get an error on generating the pdf manual using 'make manual-clean manual'. However, I don't see the same thing, pdf builds fine for me.
Strangely enough, if I remove the '-$$(Q)rm -rf $$(@D)/.build' line (to fix the second point), then I do get this error.
Quoted 6 lines
2. None of the manuals except the html one (the first one) contain the package lists. This is due to the removal of the .build directory between each manual generation, which is the directory where the lists are generated.
Removing the directory seems not really needed to me, so why not just avoid that?
First, there is this 2nd observation.
I agree with your suggestion; however since the manual generation takes place in the output doc directory, we should clean the temporary $(O)/docs/manual/.build directory at the end of the manual generation. I propose to move this build directory under $(O)/build/manual/. I'm cooking a patch for this
A consequence of this bug (to be confirmed) is that we have released a couple of version with some missing pieces in the pdf manual included in the tarball.
Another consequence of this bug is that it maskes the observation #1 when you build all the manual formats at the same time. That could explain why you did not see the error before removing the "rm" call. If you run 'make distclean && make manual-pdf' from a clean worktree, you will trigger the dblatex error.
The error of the observation #1 is caused by xsltproc, which reaches some template recursion limits when processing the big target package table.
This limit can be raised thanks to the xsltproc option: --maxvars, but the latest release of libxslt (1.1.28) is buggy and does not correctly handle this argument when it is passed through the command line. This xsltproc bug is fixed upstream on the libxslt master [1], but not yet in any of its release.
So:
- we have already released a couple of version with some missing pieces in manual.pdf;
- if someone wants to successfully build the pdf manual, he/she has to make sure the xsltproc correctly support the --maxvars option (i.e. it has been built with the patch [1]).
- if we want to make possible to anyone to build the pdf manual, then the should provide our own xsltproc program. This introduces a host-package dependency on the manual target, (so a .config dependency and so on, and so forth).
As Yann (on IRC) and Thomas P. noticed, we have not received any complaint about the missing part in the pdf manual, probably because people mainly uses the online html doc and very few people needs to build the manual.
So:
- we can certainly live with a broken pdf manual generation;
- the person doing the release (Peter) just has to make sure his xsltproc is not buggy to properly generate the official manual (both the online one and the one included in the release tarball).
Hope this is clearer.
[1] https://gitorious.org/libxslt/libxslt/commit/5af7ad745323004984287e48b42712e7305de35c
Quoted 3 lines
Thanks,
Thomas
Regards,
Signature · 1 line
Samuel
PATCH v4 1/1 manual-text generation improvements
Hi Samuel,
Quoted 19 lines · Samuel Martin
Hi Thomas,
Ok, I'll try to be clearer.Quoted 9 lines
There are two observations:
1. ThomasP and apparantly you too get an error on generating the pdf manual using 'make manual-clean manual'. However, I don't see the same thing, pdf builds fine for me.
Strangely enough, if I remove the '-$$(Q)rm -rf $$(@D)/.build' line (to fix the second point), then I do get this error.Quoted 6 lines
2. None of the manuals except the html one (the first one) contain the package lists. This is due to the removal of the .build directory between each manual generation, which is the directory where the lists are generated.
Removing the directory seems not really needed to me, so why not just avoid that?
Quoted 6 lines
First, there is this 2nd observation.
I agree with your suggestion; however since the manual generation takes place in the output doc directory, we should clean the temporary $(O)/docs/manual/.build directory at the end of the manual generation. I propose to move this build directory under $(O)/build/manual/. I'm cooking a patch for this
If you do this, then another improvement is to extract the 'mkdir + rsync' steps to a common target that is run only once. Currently, the rsync is repeated for every format, while it is always the same content.
Quoted 36 lines
A consequence of this bug (to be confirmed) is that we have released a couple of version with some missing pieces in the pdf manual included in the tarball.
Another consequence of this bug is that it maskes the observation #1 when you build all the manual formats at the same time. That could explain why you did not see the error before removing the "rm" call. If you run 'make distclean && make manual-pdf' from a clean worktree, you will trigger the dblatex error.
The error of the observation #1 is caused by xsltproc, which reaches some template recursion limits when processing the big target package table.
This limit can be raised thanks to the xsltproc option: --maxvars, but the latest release of libxslt (1.1.28) is buggy and does not correctly handle this argument when it is passed through the command line. This xsltproc bug is fixed upstream on the libxslt master [1], but not yet in any of its release.So:
- we have already released a couple of version with some missing pieces in manual.pdf;
- if someone wants to successfully build the pdf manual, he/she has to make sure the xsltproc correctly support the --maxvars option (i.e. it has been built with the patch [1]).
- if we want to make possible to anyone to build the pdf manual, then the should provide our own xsltproc program. This introduces a host-package dependency on the manual target, (so a .config dependency and so on, and so forth).As Yann (on IRC) and Thomas P. noticed, we have not received any complaint about the missing part in the pdf manual, probably because people mainly uses the online html doc and very few people needs to build the manual.
Quoted 8 lines
So:
- we can certainly live with a broken pdf manual generation;
- the person doing the release (Peter) just has to make sure his xsltproc is not buggy to properly generate the official manual (both the online one and the one included in the release tarball).Hope this is clearer.
Yes, this is clearer, thanks.
Best regards,
Thomas