Skip to contentSkip to conversation

manual-text generation improvements

PATCH Superseded Thomas De Schampheleire 4 participants 21 messages Aug 15, 2014 – Aug 17, 2014 Kbuild

lore
Share & tools
lore
#0 lore raw

PATCH v4 1/1 manual-text generation improvements

This patch series makes some improvements to the generation of the text version of the manual:
- make sure that URLs are visible
- remove the 'logo' (logo.png) reference

To achieve this, three preparatory patches are needed.

Note that this series stands on its own, it can already be applied. A subsequent series will make structural and content-wise changes to the manual itself.

---
v4: fix 'make clean' (detected by Samuel)
v3: fix 'make manual' without .config (extra patch) (detected by Yann)
v2: refine patch 4 (changing URLs)

Thomas De Schampheleire (5)
Makefile: unconditionally include pkg-utils.mk
gendoc infra: use $(pkgname) instead of explicitly passing 'manual' gendoc infra: add support for asciidoc configuration files
manual-text: make sure URLs are displayed if a link text was provided
manual-text: hide image representations

 Makefile                       |   2 ++
 docs/manual/asciidoc-text.conf |  23 +++++++++++++++++++++++
 docs/manual/manual.mk          |  28 +++++++++++++++++-----------
 package/Makefile.in            |   1 -
 4 files changed, 42 insertions(+), 12 deletions(-)
#1 lore raw

PATCH v4 1/1 Makefile: unconditionally include pkg-utils.mk

Currently, pkg-utils.mk (included via package/Makefile.in) is only included when a configuration file already exists. This means that none of the utilities it defines are available without .config.

In particular:
- the MESSAGE macro, causing pretty build output. Since some make targets can be run even without .config, like 'make manual', not having this pretty printing is odd.

- pkgname, pkgdir: in a subsequent patch, these functions will be used for the generation of the manual, and since this should work also without .config, we need these functions to be available.

This patch moves the include of pkg-utils.mk from package/Makefile.in to Makefile, outside of the check for .config.

This is a quick fix. The full solution involves to minimize the amount of Makefile code that is guarded by a check on .config. This approach will be taken in the 2014.11 release cycle.

---
v4: no changes
v3: new patch, fixing the issue detected by Yann that 'make manual' no longer worked without .config

 Makefile            |  2 ++
 package/Makefile.in |  1 -
 2 files changed, 2 insertions(+), 1 deletions(-)
Makefile +2 -0
diff -r 3bcf63a46bc1 -r 232206819ca1 Makefile
@@ -271,6 +271,8 @@
 # Causes breakage with packages that needs host-ruby
 unexport RUBYOPT
 
+include package/pkg-utils.mk
+
 ifeq ($(BR2_HAVE_DOT_CONFIG),y)
 
 ################################################################################
package/Makefile.in +0 -1
diff -r 3bcf63a46bc1 -r 232206819ca1 package/Makefile.in
@@ -385,7 +385,6 @@
 SHARED_STATIC_LIBS_OPTS = --enable-static --enable-shared
 endif
 
-include package/pkg-utils.mk
 include package/pkg-download.mk
 include package/pkg-autotools.mk
 include package/pkg-cmake.mk
#2 lore raw

PATCH v4 1/1 Makefile: unconditionally include pkg-utils.mk

Thomas, All,

On 2014-08-15 15:40 +0200, Thomas De Schampheleire spake thusly:

Quoted 16 lines

Currently, pkg-utils.mk (included via package/Makefile.in) is only included when a configuration file already exists. This means that none of the utilities it defines are available without .config.

In particular:
- the MESSAGE macro, causing pretty build output. Since some make targets can be run even without .config, like 'make manual', not having this pretty printing is odd.

- pkgname, pkgdir: in a subsequent patch, these functions will be used for the generation of the manual, and since this should work also without .config, we need these functions to be available.

This patch moves the include of pkg-utils.mk from package/Makefile.in to Makefile, outside of the check for .config.

Quoted 6 lines

This is a quick fix. The full solution involves to minimize the amount of Makefile code that is guarded by a check on .config. This approach will be taken in the 2014.11 release cycle.

Tested by generating the manual with and without a .config. Tested that running make with no .config will popup the menuconfig. Tested that a defconfig still builds OK.

Regards,
Yann E. MORIN.

Quoted 32 lines

---
v4: no changes
v3: new patch, fixing the issue detected by Yann that 'make manual' no longer worked without .config

 Makefile            |  2 ++
 package/Makefile.in |  1 -
 2 files changed, 2 insertions(+), 1 deletions(-)

diff -r 3bcf63a46bc1 -r 232206819ca1 Makefile

Makefile +2 -0
@@ -271,6 +271,8 @@
 # Causes breakage with packages that needs host-ruby
 unexport RUBYOPT
 
+include package/pkg-utils.mk
+
 ifeq ($(BR2_HAVE_DOT_CONFIG),y)
 
 ################################################################################

diff -r 3bcf63a46bc1 -r 232206819ca1 package/Makefile.in

package/Makefile.in +0 -1
@@ -385,7 +385,6 @@
 SHARED_STATIC_LIBS_OPTS = --enable-static --enable-shared
 endif
 
-include package/pkg-utils.mk
 include package/pkg-download.mk
 include package/pkg-autotools.mk
 include package/pkg-cmake.mk
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#3 lore raw

PATCH v4 1/1 gendoc infra: use $(pkgname) instead of explicitly passing 'manual'

In reply to Thomas De Schampheleire

In the gendoc infrastructure, using an assignment of the form

    FOO = docs/$(1)/bar

inside GENDOC_INNER does not work as expected: the $(1) value is empty here and the value of FOO becomes 'docs//bar'.

Parameters $(2), $(3), etc. do not have this problem. The specific thing about $(1) is that it is a parameter to GENDOC itself (indicating the document to create) and passed transparently to GENDOC_INNER.

This is different from the package infrastructures, where $(1) is set from $(pkgname). In fact, the same strategy could be used by the gendoc infrastructure as well, as $(pkgname) resolves to 'manual' for file docs/manual/manual.mk. This has the advantage that the described problem does not occur.

Note that this means that if we want to use the same GENDOC infrastructure for another document, it will have to reside in a separate directory than the manual.

---
v4: fix 'make clean' due to missing replacement of $(1) (Samuel)
v3: no changes
v2: no changes

 docs/manual/manual.mk |  20 ++++++++++----------
 1 files changed, 10 insertions(+), 10 deletions(-)
docs/manual/manual.mk +10 -10
diff -r 232206819ca1 -r 9d9af165df7b docs/manual/manual.mk
@@ -78,16 +78,16 @@
 # The variable <DOCUMENT_NAME>_SOURCES defines the dependencies.
 ################################################################################
 define GENDOC
-$(call GENDOC_INNER,$(1),xhtml,html,html,HTML,--xsltproc-opts "--stringparam toc.section.depth 2")
-$(call GENDOC_INNER,$(1),chunked,split-html,chunked,split HTML,--xsltproc-opts "--stringparam toc.section.depth 2")
-$(call GENDOC_INNER,$(1),pdf,pdf,pdf,PDF,--dblatex-opts "-P latex.output.revhistory=0")
-$(call GENDOC_INNER,$(1),text,text,text,text)
-$(call GENDOC_INNER,$(1),epub,epub,epub,ePUB)
-clean: $(1)-clean
-$(1)-clean:
-	$$(Q)$$(RM) -rf $$(O)/docs/$(1)
-.PHONY: $(1) $(1)-clean manual-update-lists
+$(call GENDOC_INNER,$(pkgname),xhtml,html,html,HTML,--xsltproc-opts "--stringparam toc.section.depth 2")
+$(call GENDOC_INNER,$(pkgname),chunked,split-html,chunked,split HTML,--xsltproc-opts "--stringparam toc.section.depth 2")
+$(call GENDOC_INNER,$(pkgname),pdf,pdf,pdf,PDF,--dblatex-opts "-P latex.output.revhistory=0")
+$(call GENDOC_INNER,$(pkgname),text,text,text,text)
+$(call GENDOC_INNER,$(pkgname),epub,epub,epub,ePUB)
+clean: $(pkgname)-clean
+$(pkgname)-clean:
+	$$(Q)$$(RM) -rf $$(O)/docs/$(pkgname)
+.PHONY: $(pkgname) $(pkgname)-clean manual-update-lists
 endef
 
 MANUAL_SOURCES = $(sort $(wildcard docs/manual/*.txt) $(wildcard docs/images/*))
-$(eval $(call GENDOC,manual))
+$(eval $(call GENDOC))
#4 lore raw

PATCH v4 1/1 gendoc infra: use $(pkgname) instead of explicitly passing 'manual'

Quoted 15 lines · Thomas De Schampheleire

In the gendoc infrastructure, using an assignment of the form

    FOO = docs/$(1)/bar

inside GENDOC_INNER does not work as expected: the $(1) value is empty here and the value of FOO becomes 'docs//bar'.

Parameters $(2), $(3), etc. do not have this problem. The specific thing about $(1) is that it is a parameter to GENDOC itself (indicating the document to create) and passed transparently to GENDOC_INNER.

This is different from the package infrastructures, where $(1) is set from $(pkgname). In fact, the same strategy could be used by the gendoc infrastructure as well, as $(pkgname) resolves to 'manual' for file docs/manual/manual.mk. This has the advantage that the described problem does not occur.

Quoted 5 lines

Note that this means that if we want to use the same GENDOC infrastructure for another document, it will have to reside in a separate directory than the manual.

Regards,

Signature · 1 line
Samuel
#5 lore raw

PATCH v4 1/1 gendoc infra: use $(pkgname) instead of explicitly passing 'manual'

In reply to Thomas De Schampheleire

THomas, All,

On 2014-08-15 15:40 +0200, Thomas De Schampheleire spake thusly:

Quoted 15 lines

In the gendoc infrastructure, using an assignment of the form

    FOO = docs/$(1)/bar

inside GENDOC_INNER does not work as expected: the $(1) value is empty here and the value of FOO becomes 'docs//bar'.

Parameters $(2), $(3), etc. do not have this problem. The specific thing about $(1) is that it is a parameter to GENDOC itself (indicating the document to create) and passed transparently to GENDOC_INNER.

This is different from the package infrastructures, where $(1) is set from $(pkgname). In fact, the same strategy could be used by the gendoc infrastructure as well, as $(pkgname) resolves to 'manual' for file docs/manual/manual.mk. This has the advantage that the described problem does not occur.

Quoted 5 lines

Note that this means that if we want to use the same GENDOC infrastructure for another document, it will have to reside in a separate directory than the manual.

Tested by building the manual with and without a .config.

Regards,
Yann E. MORIN.

Quoted 38 lines

---
v4: fix 'make clean' due to missing replacement of $(1) (Samuel)
v3: no changes
v2: no changes

 docs/manual/manual.mk |  20 ++++++++++----------
 1 files changed, 10 insertions(+), 10 deletions(-)

diff -r 232206819ca1 -r 9d9af165df7b docs/manual/manual.mk

docs/manual/manual.mk +10 -10
@@ -78,16 +78,16 @@
 # The variable <DOCUMENT_NAME>_SOURCES defines the dependencies.
 ################################################################################
 define GENDOC
-$(call GENDOC_INNER,$(1),xhtml,html,html,HTML,--xsltproc-opts "--stringparam toc.section.depth 2")
-$(call GENDOC_INNER,$(1),chunked,split-html,chunked,split HTML,--xsltproc-opts "--stringparam toc.section.depth 2")
-$(call GENDOC_INNER,$(1),pdf,pdf,pdf,PDF,--dblatex-opts "-P latex.output.revhistory=0")
-$(call GENDOC_INNER,$(1),text,text,text,text)
-$(call GENDOC_INNER,$(1),epub,epub,epub,ePUB)
-clean: $(1)-clean
-$(1)-clean:
-	$$(Q)$$(RM) -rf $$(O)/docs/$(1)
-.PHONY: $(1) $(1)-clean manual-update-lists
+$(call GENDOC_INNER,$(pkgname),xhtml,html,html,HTML,--xsltproc-opts "--stringparam toc.section.depth 2")
+$(call GENDOC_INNER,$(pkgname),chunked,split-html,chunked,split HTML,--xsltproc-opts "--stringparam toc.section.depth 2")
+$(call GENDOC_INNER,$(pkgname),pdf,pdf,pdf,PDF,--dblatex-opts "-P latex.output.revhistory=0")
+$(call GENDOC_INNER,$(pkgname),text,text,text,text)
+$(call GENDOC_INNER,$(pkgname),epub,epub,epub,ePUB)
+clean: $(pkgname)-clean
+$(pkgname)-clean:
+	$$(Q)$$(RM) -rf $$(O)/docs/$(pkgname)
+.PHONY: $(pkgname) $(pkgname)-clean manual-update-lists
 endef
 
 MANUAL_SOURCES = $(sort $(wildcard docs/manual/*.txt) $(wildcard docs/images/*))
-$(eval $(call GENDOC,manual))
+$(eval $(call GENDOC))
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#6 lore raw

PATCH v4 1/1 gendoc infra: add support for asciidoc configuration files

In reply to Thomas De Schampheleire

This patch introduces support for asciidoc configuration files, specific for each output format (html, text, pdf, ...). This is useful to make certain tweaks in the document generation.

If a file docs/manual/asciidoc-<format>.conf is present, it is passed to asciidoc as configuration file. If no file for the current format is present, the options passed to asciidoc are empty.

---
v4: no changes
v3: no changes
v2: no changes

 docs/manual/manual.mk |  8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)
docs/manual/manual.mk +7 -1
diff -r 9d9af165df7b -r 81d40b841073 docs/manual/manual.mk
@@ -56,6 +56,11 @@
 
 manual-check-dependencies-$(3):
 
+MANUAL_$(2)_ASCIIDOC_CONF = docs/$(1)/asciidoc-$(2).conf
+ifneq ($$(wildcard $$(MANUAL_$(2)_ASCIIDOC_CONF)),)
+MANUAL_$(2)_ASCIIDOC_OPTS += -f $$(MANUAL_$(2)_ASCIIDOC_CONF)
+endif
+
 $$(O)/docs/$(1)/$(1).$(4): docs/$(1)/$(1).txt \
 			   $$($$(call UPPERCASE,$(1))_SOURCES) \
 			   manual-check-dependencies \
@@ -65,7 +70,8 @@
 	$$(Q)mkdir -p $$(@D)/.build
 	$$(Q)rsync -au docs/$(1)/*.txt $$(@D)/.build
 	$$(Q)a2x $(6) -f $(2) -d book -L -r $$(TOPDIR)/docs/images \
-	        -D $$(@D) $$(@D)/.build/$(1).txt
+	        -D $$(@D) $$(@D)/.build/$(1).txt \
+	        --asciidoc-opts="$$(MANUAL_$(2)_ASCIIDOC_OPTS)"
 	-$$(Q)rm -rf $$(@D)/.build
 endef
 
#7 lore raw

PATCH v4 1/1 gendoc infra: add support for asciidoc configuration files

Thomas, All,

On 2014-08-15 15:40 +0200, Thomas De Schampheleire spake thusly:

Quoted 10 lines

This patch introduces support for asciidoc configuration files, specific for each output format (html, text, pdf, ...). This is useful to make certain tweaks in the document generation.

If a file docs/manual/asciidoc-<format>.conf is present, it is passed to asciidoc as configuration file. If no file for the current format is present, the options passed to asciidoc are empty.

Regards,
Yann E. MORIN.

Quoted 33 lines

---
v4: no changes
v3: no changes
v2: no changes

 docs/manual/manual.mk |  8 +++++++-
 1 files changed, 7 insertions(+), 1 deletions(-)

diff -r 9d9af165df7b -r 81d40b841073 docs/manual/manual.mk

docs/manual/manual.mk +7 -1
@@ -56,6 +56,11 @@
 
 manual-check-dependencies-$(3):
 
+MANUAL_$(2)_ASCIIDOC_CONF = docs/$(1)/asciidoc-$(2).conf
+ifneq ($$(wildcard $$(MANUAL_$(2)_ASCIIDOC_CONF)),)
+MANUAL_$(2)_ASCIIDOC_OPTS += -f $$(MANUAL_$(2)_ASCIIDOC_CONF)
+endif
+
 $$(O)/docs/$(1)/$(1).$(4): docs/$(1)/$(1).txt \
 			   $$($$(call UPPERCASE,$(1))_SOURCES) \
 			   manual-check-dependencies \
@@ -65,7 +70,8 @@
 	$$(Q)mkdir -p $$(@D)/.build
 	$$(Q)rsync -au docs/$(1)/*.txt $$(@D)/.build
 	$$(Q)a2x $(6) -f $(2) -d book -L -r $$(TOPDIR)/docs/images \
-	        -D $$(@D) $$(@D)/.build/$(1).txt
+	        -D $$(@D) $$(@D)/.build/$(1).txt \
+	        --asciidoc-opts="$$(MANUAL_$(2)_ASCIIDOC_OPTS)"
 	-$$(Q)rm -rf $$(@D)/.build
 endef
 
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#8 lore raw

PATCH v4 1/1 manual-text: make sure URLs are displayed if a link text was provided

In reply to Thomas De Schampheleire

When the asciidoc source contain URLs of the form:
http://example.com[An example website]
the text representation of the manual would only contain:
An example website
without displaying the actual URL.

This patch adds an asciidoc configuration file that sets the inline macros for several URL types so that the display becomes:
An example website [http://example.com]
For URLs where no link text was provided, the display becomes:
http://example.com
which is the same as before.

---
v4: no changes
v3: no changes
v2:
- link to relevant documentation (which was hard to find)
- refine so that the display is identical to before when no link text is provided.

 docs/manual/asciidoc-text.conf |  17 +++++++++++++++++
 1 files changed, 17 insertions(+), 0 deletions(-)
docs/manual/asciidoc-text.conf new file +17 -0
diff -r 81d40b841073 -r 93b82c1cf8a4 docs/manual/asciidoc-text.conf
@@ -0,0 +1,17 @@
+# Refer to following asciidoc documentation:
+#     http://www.methods.co.nz/asciidoc/userguide.html
+# In particular sections "Macros" and "Attribute References"
+#
+# For hyperlinks, show 'link text [URL]' (if link text provided) or 'URL'
+[http-inlinemacro]
+{0=}{0? &#91;}{name}:{target}{0?&#93;}
+[https-inlinemacro]
+{0=}{0? &#91;}{name}:{target}{0?&#93;}
+[ftp-inlinemacro]
+{0=}{0? &#91;}{name}:{target}{0?&#93;}
+[file-inlinemacro]
+{0=}{0? &#91;}{name}:{target}{0?&#93;}
+[irc-inlinemacro]
+{0=}{0? &#91;}{name}:{target}{0?&#93;}
+[mailto-inlinemacro]
+{0=}{0? &#91;}{name}:{target}{0?&#93;}
#9 lore raw

PATCH v4 1/1 manual-text: make sure URLs are displayed if a link text was provided

Thomas, All,

On 2014-08-15 15:40 +0200, Thomas De Schampheleire spake thusly:

Quoted 13 lines

When the asciidoc source contain URLs of the form:
http://example.com[An example website]
the text representation of the manual would only contain:
An example website
without displaying the actual URL.

This patch adds an asciidoc configuration file that sets the inline macros for several URL types so that the display becomes:
An example website [http://example.com]
For URLs where no link text was provided, the display becomes:
http://example.com
which is the same as before.

Quoted 2 lines

Regards,
Yann E. MORIN.

Quoted 32 lines

---
v4: no changes
v3: no changes
v2:
- link to relevant documentation (which was hard to find)
- refine so that the display is identical to before when no link text is provided.

 docs/manual/asciidoc-text.conf |  17 +++++++++++++++++
 1 files changed, 17 insertions(+), 0 deletions(-)

diff -r 81d40b841073 -r 93b82c1cf8a4 docs/manual/asciidoc-text.conf

docs/manual/asciidoc-text.conf new file +17 -0
@@ -0,0 +1,17 @@
+# Refer to following asciidoc documentation:
+#     http://www.methods.co.nz/asciidoc/userguide.html
+# In particular sections "Macros" and "Attribute References"
+#
+# For hyperlinks, show 'link text [URL]' (if link text provided) or 'URL'
+[http-inlinemacro]
+{0=}{0? &#91;}{name}:{target}{0?&#93;}
+[https-inlinemacro]
+{0=}{0? &#91;}{name}:{target}{0?&#93;}
+[ftp-inlinemacro]
+{0=}{0? &#91;}{name}:{target}{0?&#93;}
+[file-inlinemacro]
+{0=}{0? &#91;}{name}:{target}{0?&#93;}
+[irc-inlinemacro]
+{0=}{0? &#91;}{name}:{target}{0?&#93;}
+[mailto-inlinemacro]
+{0=}{0? &#91;}{name}:{target}{0?&#93;}
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#10 lore raw

PATCH v4 1/1 manual-text: hide image representations

In reply to Thomas De Schampheleire

Images specified in the asciidoc sources (currently only the logo) are displayed as the file name in the text version of the manual. This causes an odd line to appear:
logo.png

Avoid this by setting the image representation macros to {empty} in manual-text.

---
v4: no changes
v3: no changes
v2: no changes

 docs/manual/asciidoc-text.conf |  6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)
docs/manual/asciidoc-text.conf +6 -0
diff -r 93b82c1cf8a4 -r 6b930ce9f5cb docs/manual/asciidoc-text.conf
@@ -15,3 +15,9 @@
 {0=}{0? &#91;}{name}:{target}{0?&#93;}
 [mailto-inlinemacro]
 {0=}{0? &#91;}{name}:{target}{0?&#93;}
+
+# Hide image representation from text manual
+[image-inlinemacro]
+{empty}
+[image-blockmacro]
+{empty}
#11 lore raw

PATCH v4 1/1 manual-text: hide image representations

Thomas, All,

On 2014-08-15 15:40 +0200, Thomas De Schampheleire spake thusly:

Quoted 10 lines

Images specified in the asciidoc sources (currently only the logo) are displayed as the file name in the text version of the manual. This causes an odd line to appear:
logo.png

Avoid this by setting the image representation macros to {empty} in manual-text.

Tested that the filenames no longer appears, and that's the only difference. :-)

Regards,
Yann E. MORIN.

Quoted 21 lines

---
v4: no changes
v3: no changes
v2: no changes

 docs/manual/asciidoc-text.conf |  6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff -r 93b82c1cf8a4 -r 6b930ce9f5cb docs/manual/asciidoc-text.conf

docs/manual/asciidoc-text.conf +6 -0
@@ -15,3 +15,9 @@
 {0=}{0? &#91;}{name}:{target}{0?&#93;}
 [mailto-inlinemacro]
 {0=}{0? &#91;}{name}:{target}{0?&#93;}
+
+# Hide image representation from text manual
+[image-inlinemacro]
+{empty}
+[image-blockmacro]
+{empty}
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#12 lore raw

PATCH v4 1/1 manual-text generation improvements

In reply to Thomas De Schampheleire

Hi all,

Quoted 5 lines · Thomas De Schampheleire

This patch series makes some improvements to the generation of the text version of the manual:
- make sure that URLs are visible
- remove the 'logo' (logo.png) reference

Quoted 7 lines

To achieve this, three preparatory patches are needed.

Note that this series stands on its own, it can already be applied. A subsequent series will make structural and content-wise changes to the manual itself.

[the whole series; IMO this series is ready to go.]
Signature · 1 line
Samuel
#13 lore raw

PATCH v4 1/1 manual-text generation improvements

In reply to Thomas De Schampheleire

Dear Thomas De Schampheleire,

Quoted 6 lines · Thomas De Schampheleire

Thomas De Schampheleire (5)
Makefile: unconditionally include pkg-utils.mk
gendoc infra: use $(pkgname) instead of explicitly passing 'manual' gendoc infra: add support for asciidoc configuration files
manual-text: make sure URLs are displayed if a link text was provided
manual-text: hide image representations

Nice, you've added the list of patches to the cover letter :-)

I've applied the patch series, thanks to you, and to Yann and Samuel for testing/reviewing.

Related to this patch series, here are a few things I noted:

* When building the PDF and txt versions of the manual, I get one warning:

a2x: WARNING: --destination-dir option is only applicable to HTML based outputs

It seems to be harmless as the result is really located in the right place (output/docs), but I found it worth mentioning.

* Also, here the PDF version of the manual doesn't build:

a2x: ERROR: "dblatex" -t pdf -p "/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -P latex.output.revhistory=0 "/home/thomas/projets/buildroot/output/docs/manual/manual.xml" returned non-zero exit status 1

It's most likely due to something missing or a wrong version in my installation, but it also means we're maybe not checking for all the necessary tools before generating the manual.

Both of those 'issues' are not introduced by your patch series, they existed before it.

Thanks!

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#14 lore raw

PATCH v4 1/1 manual-text generation improvements

Quoted 4 lines · Thomas Petazzoni

Dear Thomas De Schampheleire,

On Fri, 15 Aug 2014 15:40:33 +0200, Thomas De Schampheleire wrote:

Quoted 8 lines
Quoted 6 lines

Thomas De Schampheleire (5)
Makefile: unconditionally include pkg-utils.mk
gendoc infra: use $(pkgname) instead of explicitly passing 'manual' gendoc infra: add support for asciidoc configuration files
manual-text: make sure URLs are displayed if a link text was provided
manual-text: hide image representations

Nice, you've added the list of patches to the cover letter :-)

Yes :-)

Quoted 6 lines

I've applied the patch series, thanks to you, and to Yann and Samuel for testing/reviewing.

Related to this patch series, here are a few things I noted:

Quoted 7 lines

* When building the PDF and txt versions of the manual, I get one warning:

a2x: WARNING: --destination-dir option is only applicable to HTML based outputs

It seems to be harmless as the result is really located in the right place (output/docs), but I found it worth mentioning.

Yes, I see the same. We could pass format specific options to avoid passing -D for non-html formats. Not sure whether it's worth the effort, I will have a look.

Quoted 4 lines

* Also, here the PDF version of the manual doesn't build:

a2x: ERROR: "dblatex" -t pdf -p "/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -P latex.output.revhistory=0 "/home/thomas/projets/buildroot/output/docs/manual/manual.xml" returned non-zero exit status 1

I will have a look at this too, but I may need your help here. Could you run the same command manually? It should show more output...

Thanks,
Thomas

#15 lore raw

PATCH v4 1/1 manual-text generation improvements

Quoted 26 lines · Thomas De Schampheleire
Quoted 4 lines · Thomas Petazzoni

Dear Thomas De Schampheleire,

On Fri, 15 Aug 2014 15:40:33 +0200, Thomas De Schampheleire wrote:

Quoted 8 lines
Quoted 6 lines

Thomas De Schampheleire (5)
Makefile: unconditionally include pkg-utils.mk
gendoc infra: use $(pkgname) instead of explicitly passing 'manual' gendoc infra: add support for asciidoc configuration files
manual-text: make sure URLs are displayed if a link text was provided
manual-text: hide image representations

Nice, you've added the list of patches to the cover letter :-)

Yes :-)

Quoted 9 lines

I've applied the patch series, thanks to you, and to Yann and Samuel for testing/reviewing.

Related to this patch series, here are a few things I noted:

* When building the PDF and txt versions of the manual, I get one warning:

Quoted 8 lines
Quoted 4 lines

a2x: WARNING: --destination-dir option is only applicable to HTML based outputs

It seems to be harmless as the result is really located in the right place (output/docs), but I found it worth mentioning.

Yes, I see the same. We could pass format specific options to avoid passing -D for non-html formats. Not sure whether it's worth the effort, I will have a look.

I've noticed this too, but have not found a proper way to suppress it.

Quoted 5 lines
Quoted 5 lines

* Also, here the PDF version of the manual doesn't build:

a2x: ERROR: "dblatex" -t pdf -p "/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -P latex.output.revhistory=0 "/home/thomas/projets/buildroot/output/docs/manual/manual.xml" returned non-zero exit status 1

Few shortcomings:
- the generated appendix tables are only included in the html manual (when running: make manual-clean manual). It seems these files are not regenerated after the 'rm -rf $$(@D)/.build' :-/
- the above error seems related to these generated appendix tables, because when they don't exits, the pdf manual generation succeeds.

This makes me think about refactoring the manual.mk to be closer to the package ones, i.e. having it build directory under $(0)/build, maybe inheriting from generic-package...
Anyway, this is just wild ideas; for the upcoming release there is this pdf issue to fix.

Quoted 10 lines

I will have a look at this too, but I may need your help here. Could you run the same command manually? It should show more output...

Thanks,
Thomas
_______________________________________________
buildroot mailing list
buildroot at busybox.net
http://lists.busybox.net/mailman/listinfo/buildroot

Regards,

Signature · 1 line
Samuel
#16 lore raw

PATCH v4 1/1 manual-text generation improvements

Quoted 3 lines · Samuel Martin
Quoted 1 line · Thomas De Schampheleire

On Sat, Aug 16, 2014 at 9:35 AM, Thomas Petazzoni

[...]

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

* When building the PDF and txt versions of the manual, I get one warning:

a2x: WARNING: --destination-dir option is only applicable to HTML based outputs

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

It seems to be harmless as the result is really located in the right place (output/docs), but I found it worth mentioning.

Yes, I see the same. We could pass format specific options to avoid passing -D for non-html formats. Not sure whether it's worth the effort, I will have a look.

I've noticed this too, but have not found a proper way to suppress it.

I have not looked into this.

Quoted 13 lines
Quoted 5 lines
Quoted 5 lines

* Also, here the PDF version of the manual doesn't build:

a2x: ERROR: "dblatex" -t pdf -p "/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -P latex.output.revhistory=0 "/home/thomas/projets/buildroot/output/docs/manual/manual.xml" returned non-zero exit status 1

Few shortcomings:
- the generated appendix tables are only included in the html manual (when running: make manual-clean manual). It seems these files are not regenerated after the 'rm -rf $$(@D)/.build' :-/
- the above error seems related to these generated appendix tables, because when they don't exits, the pdf manual generation succeeds.

Quoted 6 lines

This makes me think about refactoring the manual.mk to be closer to the package ones, i.e. having it build directory under $(0)/build, maybe inheriting from generic-package...
Anyway, this is just wild ideas; for the upcoming release there is this pdf issue to fix.

Well, after diving into asciidoc/dblatex/xslt, I eventually found the problem and the the fix!

Short-long story:

When generating the pdf manual, a2x calls dblatex, which calls xsltproc. Here is a verbose log [1].
During its execution, xsltproc checks for template recursion and reaches its limit set to 15000 when processing the target package-list.txt file (empirically, this limit allows a 1147-entry table).
To raise this limit, a --maxvars option exists in xsltproc program but the latest release of libxslt is buggy and this option is not honored. :-(
Hopefully the fix is already merged upstream [2]. Backporting this patch is enough to get a working xsltproc, but it also means that we now need to build host-libxslt (and a .config) to build the manual.
So, to build the manual we have to run:

$ make defconfig && make host-libxslt manual

The main drawback of this fix is that building the manual could now require building some host-package.

This also makes echo to the include statement of the *.mk files without having a .config file discussion.

Anyway, I will prepare a series fixing this, though it is not a mandatory for the release

Note that we have already released a couple of Buildroot versions without recieving any complaint about this.

[1] http://code.bulix.org/1905fo-86779
[2] https://gitorious.org/libxslt/libxslt/commit/5af7ad745323004984287e48b42712e7305de35c

Regards,

Signature · 1 line
Samuel
#17 lore raw

PATCH v4 1/1 manual-text generation improvements

Dear Samuel Martin,

Quoted 19 lines · Samuel Martin

When generating the pdf manual, a2x calls dblatex, which calls xsltproc. Here is a verbose log [1].
During its execution, xsltproc checks for template recursion and reaches its limit set to 15000 when processing the target package-list.txt file (empirically, this limit allows a 1147-entry table).
To raise this limit, a --maxvars option exists in xsltproc program but the latest release of libxslt is buggy and this option is not honored. :-(
Hopefully the fix is already merged upstream [2]. Backporting this patch is enough to get a working xsltproc, but it also means that we now need to build host-libxslt (and a .config) to build the manual.
So, to build the manual we have to run:

$ make defconfig && make host-libxslt manual

The main drawback of this fix is that building the manual could now require building some host-package.

Quoted 8 lines

This also makes echo to the include statement of the *.mk files without having a .config file discussion.

Anyway, I will prepare a series fixing this, though it is not a mandatory for the release

Note that we have already released a couple of Buildroot versions without recieving any complaint about this.

I'm not sure we should enforce building host-libxslt as a dependency of the manual. Maybe just check the version of xsltproc and bail out with an error message if it's not the appropriate version. I believe most people are just using the pre-generated versions of the manual that are online on Buildroot website. So it's up to us to have the appropriate xsltproc version there.

Thomas

Signature · 3 lines
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com
#18 lore raw

PATCH v4 1/1 manual-text generation improvements

In reply to Samuel Martin

Hi Samuel,

Quoted 71 lines · Samuel Martin
Quoted 3 lines · Samuel Martin
Quoted 1 line · Thomas De Schampheleire

On Sat, Aug 16, 2014 at 9:35 AM, Thomas Petazzoni

[...]

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

* When building the PDF and txt versions of the manual, I get one warning:

a2x: WARNING: --destination-dir option is only applicable to HTML based outputs

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

It seems to be harmless as the result is really located in the right place (output/docs), but I found it worth mentioning.

Yes, I see the same. We could pass format specific options to avoid passing -D for non-html formats. Not sure whether it's worth the effort, I will have a look.

I've noticed this too, but have not found a proper way to suppress it.

I have not looked into this.

Quoted 13 lines
Quoted 5 lines
Quoted 5 lines

* Also, here the PDF version of the manual doesn't build:

a2x: ERROR: "dblatex" -t pdf -p "/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -P latex.output.revhistory=0 "/home/thomas/projets/buildroot/output/docs/manual/manual.xml" returned non-zero exit status 1

Few shortcomings:
- the generated appendix tables are only included in the html manual (when running: make manual-clean manual). It seems these files are not regenerated after the 'rm -rf $$(@D)/.build' :-/
- the above error seems related to these generated appendix tables, because when they don't exits, the pdf manual generation succeeds.

Quoted 6 lines

This makes me think about refactoring the manual.mk to be closer to the package ones, i.e. having it build directory under $(0)/build, maybe inheriting from generic-package...
Anyway, this is just wild ideas; for the upcoming release there is this pdf issue to fix.

Well, after diving into asciidoc/dblatex/xslt, I eventually found the problem and the the fix!

Short-long story:

When generating the pdf manual, a2x calls dblatex, which calls xsltproc. Here is a verbose log [1].
During its execution, xsltproc checks for template recursion and reaches its limit set to 15000 when processing the target package-list.txt file (empirically, this limit allows a 1147-entry table).
To raise this limit, a --maxvars option exists in xsltproc program but the latest release of libxslt is buggy and this option is not honored. :-(
Hopefully the fix is already merged upstream [2]. Backporting this patch is enough to get a working xsltproc, but it also means that we now need to build host-libxslt (and a .config) to build the manual.
So, to build the manual we have to run:

$ make defconfig && make host-libxslt manual

The main drawback of this fix is that building the manual could now require building some host-package.

This also makes echo to the include statement of the *.mk files without having a .config file discussion.

Anyway, I will prepare a series fixing this, though it is not a mandatory for the release

Quoted 6 lines

Note that we have already released a couple of Buildroot versions without recieving any complaint about this.

[1] http://code.bulix.org/1905fo-86779
[2] https://gitorious.org/libxslt/libxslt/commit/5af7ad745323004984287e48b42712e7305de35c

I must say I don't fully understand this.

There are two observations:
1. ThomasP and apparantly you too get an error on generating the pdf manual using 'make manual-clean manual'. However, I don't see the same thing, pdf builds fine for me.
Strangely enough, if I remove the '-$$(Q)rm -rf $$(@D)/.build' line (to fix the second point), then I do get this error.

2. None of the manuals except the html one (the first one) contain the package lists. This is due to the removal of the .build directory between each manual generation, which is the directory where the lists are generated.
Removing the directory seems not really needed to me, so why not just avoid that?

Thanks,
Thomas

#19 lore raw

PATCH v4 1/1 manual-text generation improvements

Hi Thomas,

Quoted 74 lines · Thomas De Schampheleire

Hi Samuel,

Quoted 71 lines · Samuel Martin
Quoted 3 lines · Samuel Martin
Quoted 1 line · Thomas De Schampheleire

On Sat, Aug 16, 2014 at 9:35 AM, Thomas Petazzoni

[...]

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

* When building the PDF and txt versions of the manual, I get one warning:

a2x: WARNING: --destination-dir option is only applicable to HTML based outputs

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

It seems to be harmless as the result is really located in the right place (output/docs), but I found it worth mentioning.

Yes, I see the same. We could pass format specific options to avoid passing -D for non-html formats. Not sure whether it's worth the effort, I will have a look.

I've noticed this too, but have not found a proper way to suppress it.

I have not looked into this.

Quoted 13 lines
Quoted 5 lines
Quoted 5 lines

* Also, here the PDF version of the manual doesn't build:

a2x: ERROR: "dblatex" -t pdf -p "/etc/asciidoc/dblatex/asciidoc-dblatex.xsl" -s "/etc/asciidoc/dblatex/asciidoc-dblatex.sty" -P latex.output.revhistory=0 "/home/thomas/projets/buildroot/output/docs/manual/manual.xml" returned non-zero exit status 1

Few shortcomings:
- the generated appendix tables are only included in the html manual (when running: make manual-clean manual). It seems these files are not regenerated after the 'rm -rf $$(@D)/.build' :-/
- the above error seems related to these generated appendix tables, because when they don't exits, the pdf manual generation succeeds.

Quoted 6 lines

This makes me think about refactoring the manual.mk to be closer to the package ones, i.e. having it build directory under $(0)/build, maybe inheriting from generic-package...
Anyway, this is just wild ideas; for the upcoming release there is this pdf issue to fix.

Well, after diving into asciidoc/dblatex/xslt, I eventually found the problem and the the fix!

Short-long story:

When generating the pdf manual, a2x calls dblatex, which calls xsltproc. Here is a verbose log [1].
During its execution, xsltproc checks for template recursion and reaches its limit set to 15000 when processing the target package-list.txt file (empirically, this limit allows a 1147-entry table).
To raise this limit, a --maxvars option exists in xsltproc program but the latest release of libxslt is buggy and this option is not honored. :-(
Hopefully the fix is already merged upstream [2]. Backporting this patch is enough to get a working xsltproc, but it also means that we now need to build host-libxslt (and a .config) to build the manual.
So, to build the manual we have to run:

$ make defconfig && make host-libxslt manual

The main drawback of this fix is that building the manual could now require building some host-package.

This also makes echo to the include statement of the *.mk files without having a .config file discussion.

Anyway, I will prepare a series fixing this, though it is not a mandatory for the release

Quoted 8 lines
Quoted 6 lines

Note that we have already released a couple of Buildroot versions without recieving any complaint about this.

[1] http://code.bulix.org/1905fo-86779
[2] https://gitorious.org/libxslt/libxslt/commit/5af7ad745323004984287e48b42712e7305de35c

I must say I don't fully understand this.

Ok, I'll try to be clearer.

Quoted 9 lines

There are two observations:
1. ThomasP and apparantly you too get an error on generating the pdf manual using 'make manual-clean manual'. However, I don't see the same thing, pdf builds fine for me.
Strangely enough, if I remove the '-$$(Q)rm -rf $$(@D)/.build' line (to fix the second point), then I do get this error.

Quoted 6 lines

2. None of the manuals except the html one (the first one) contain the package lists. This is due to the removal of the .build directory between each manual generation, which is the directory where the lists are generated.
Removing the directory seems not really needed to me, so why not just avoid that?

First, there is this 2nd observation.
I agree with your suggestion; however since the manual generation takes place in the output doc directory, we should clean the temporary $(O)/docs/manual/.build directory at the end of the manual generation. I propose to move this build directory under $(O)/build/manual/. I'm cooking a patch for this

A consequence of this bug (to be confirmed) is that we have released a couple of version with some missing pieces in the pdf manual included in the tarball.

Another consequence of this bug is that it maskes the observation #1 when you build all the manual formats at the same time. That could explain why you did not see the error before removing the "rm" call. If you run 'make distclean && make manual-pdf' from a clean worktree, you will trigger the dblatex error.

The error of the observation #1 is caused by xsltproc, which reaches some template recursion limits when processing the big target package table.
This limit can be raised thanks to the xsltproc option: --maxvars, but the latest release of libxslt (1.1.28) is buggy and does not correctly handle this argument when it is passed through the command line. This xsltproc bug is fixed upstream on the libxslt master [1], but not yet in any of its release.

So:
- we have already released a couple of version with some missing pieces in manual.pdf;
- if someone wants to successfully build the pdf manual, he/she has to make sure the xsltproc correctly support the --maxvars option (i.e. it has been built with the patch [1]).
- if we want to make possible to anyone to build the pdf manual, then the should provide our own xsltproc program. This introduces a host-package dependency on the manual target, (so a .config dependency and so on, and so forth).

As Yann (on IRC) and Thomas P. noticed, we have not received any complaint about the missing part in the pdf manual, probably because people mainly uses the online html doc and very few people needs to build the manual.

So:
- we can certainly live with a broken pdf manual generation;
- the person doing the release (Peter) just has to make sure his xsltproc is not buggy to properly generate the official manual (both the online one and the one included in the release tarball).

Hope this is clearer.

[1] https://gitorious.org/libxslt/libxslt/commit/5af7ad745323004984287e48b42712e7305de35c

Quoted 3 lines

Thanks,
Thomas

Regards,

Signature · 1 line
Samuel
#20 lore raw

PATCH v4 1/1 manual-text generation improvements

Hi Samuel,

Quoted 19 lines · Samuel Martin

Hi Thomas,
Ok, I'll try to be clearer.

Quoted 9 lines

There are two observations:
1. ThomasP and apparantly you too get an error on generating the pdf manual using 'make manual-clean manual'. However, I don't see the same thing, pdf builds fine for me.
Strangely enough, if I remove the '-$$(Q)rm -rf $$(@D)/.build' line (to fix the second point), then I do get this error.

Quoted 6 lines

2. None of the manuals except the html one (the first one) contain the package lists. This is due to the removal of the .build directory between each manual generation, which is the directory where the lists are generated.
Removing the directory seems not really needed to me, so why not just avoid that?

Quoted 6 lines

First, there is this 2nd observation.
I agree with your suggestion; however since the manual generation takes place in the output doc directory, we should clean the temporary $(O)/docs/manual/.build directory at the end of the manual generation. I propose to move this build directory under $(O)/build/manual/. I'm cooking a patch for this

If you do this, then another improvement is to extract the 'mkdir + rsync' steps to a common target that is run only once. Currently, the rsync is repeated for every format, while it is always the same content.

Quoted 36 lines

A consequence of this bug (to be confirmed) is that we have released a couple of version with some missing pieces in the pdf manual included in the tarball.

Another consequence of this bug is that it maskes the observation #1 when you build all the manual formats at the same time. That could explain why you did not see the error before removing the "rm" call. If you run 'make distclean && make manual-pdf' from a clean worktree, you will trigger the dblatex error.

The error of the observation #1 is caused by xsltproc, which reaches some template recursion limits when processing the big target package table.
This limit can be raised thanks to the xsltproc option: --maxvars, but the latest release of libxslt (1.1.28) is buggy and does not correctly handle this argument when it is passed through the command line. This xsltproc bug is fixed upstream on the libxslt master [1], but not yet in any of its release.

So:
- we have already released a couple of version with some missing pieces in manual.pdf;
- if someone wants to successfully build the pdf manual, he/she has to make sure the xsltproc correctly support the --maxvars option (i.e. it has been built with the patch [1]).
- if we want to make possible to anyone to build the pdf manual, then the should provide our own xsltproc program. This introduces a host-package dependency on the manual target, (so a .config dependency and so on, and so forth).

As Yann (on IRC) and Thomas P. noticed, we have not received any complaint about the missing part in the pdf manual, probably because people mainly uses the online html doc and very few people needs to build the manual.

Quoted 8 lines

So:
- we can certainly live with a broken pdf manual generation;
- the person doing the release (Peter) just has to make sure his xsltproc is not buggy to properly generate the official manual (both the online one and the one included in the release tarball).

Hope this is clearer.

Yes, this is clearer, thanks.

Best regards,
Thomas