Making pdfdocs with sphinx - only select rst targets
I'm excited to see the new documentation format, so I'm changing my documentation in the pending patches I have to use it. I however cannot generate anything other than the main
Documentation/output/pdf/Kernel.pdf
How can I see in PDF the other documentation?
I'm using:
make DOCBOOKS="" pdfdocs
The Documentation/output/pdf/Kernel.pdf only has a bit of the documentation on how to write docs, nothing else. Using htmldocs as a target works but I am not a fan of that output.
Luis
Em Tue, 9 Aug 2016 18:19:14 -0700
"Luis R. Rodriguez" <[email protected]> escreveu:
Quoted 8 lines
I'm excited to see the new documentation format, so I'm changing my documentation in the pending patches I have to use it. I however cannot generate anything other than the main
Documentation/output/pdf/Kernel.pdfHow can I see in PDF the other documentation?
Quoted 7 lines
I'm using:
make DOCBOOKS="" pdfdocs
The
Documentation/output/pdf/Kernel.pdfonly has a bit of the documentation on how to write docs, nothing else. Using htmldocs as a target works but I am not a fan of that output.
Hi Luis,
Unfortunately, on my tests with Sphinx, I noticed that pdf output is not actually working fine. It relies on a tool called rst2pdf. On my tests, I noticed that:
- rst2pdf is completely broken with Sphinx 1.3.x;
- rst2pdf doesn't handle complex documents, even with Sphinx 1.4.5;
- rst2pdf always report a successful build, even when it fails;
- I also consulted a developer with years of experience with Sphinx: he recommended removing the media book from the pdf build, as he doesn't know any fixup or replacement for the tool that would work for the media docs.
If you want to play with PDF output, you should take a look on this patch:
https://patchwork.kernel.org/patch/9231687/
(commit 520a247760f750307b53db905a10a17df1700f3b)
Reverting it or adding your document main rst file there would let you play with pdfdocs, if you're using Sphinx version 1.4.x.
If you revert it, you'll also need the enclosed patch.
Regards,
Cheers,
Mauro
---
doc-rst: build the dynamic rst files for non-html doc targets
Right now, the media makefile is called only for html docs. Call it also for the other documentation targets.
NOTICE: Should not be applied upstream!
The pdf target won't actually build a PDF, because rst2pdf can't handle complex documents and will fail with the media one.
Documentation/Makefile.sphinx +3 -0
@@ -49,13 +49,16 @@ ifeq ($(HAVE_RST2PDF),0) $(warning The Python 'rst2pdf' module was not found. Make sure you have the module installed to produce PDF output.) @echo " SKIP Sphinx $@ target." else # HAVE_RST2PDF + $(MAKE) BUILDDIR=$(BUILDDIR) -f $(srctree)/Documentation/media/Makefile $@ $(call cmd,sphinx,pdf) endif # HAVE_RST2PDF epubdocs: + $(MAKE) BUILDDIR=$(BUILDDIR) -f $(srctree)/Documentation/media/Makefile $@ $(call cmd,sphinx,epub) xmldocs: + $(MAKE) BUILDDIR=$(BUILDDIR) -f $(srctree)/Documentation/media/Makefile $@ $(call cmd,sphinx,xml) # no-ops for the Sphinx toolchain
Documentation/media/Makefile +3 -0
@@ -11,6 +11,9 @@ FILES = audio.h.rst ca.h.rst dmx.h.rst frontend.h.rst net.h.rst video.h.rst \ TARGETS := $(addprefix $(BUILDDIR)/, $(FILES)) htmldocs: $(BUILDDIR) ${TARGETS} +epubdocs: $(BUILDDIR) ${TARGETS} +xmldocs: $(BUILDDIR) ${TARGETS} +pdfdocs: $(BUILDDIR) ${TARGETS} $(BUILDDIR): $(Q)mkdir -p $@
A message that is not in the archive (sent off-list or not yet received): CAB=NE6UUiKSn66=F+WUrqk3E1tj+fnzqFnD=dX55md6RnaiOgQ@mail.gmail.com
Em Wed, 10 Aug 2016 07:42:42 -0700
"Luis R. Rodriguez" <[email protected]> escreveu:
Quoted 36 lines
Quoted 26 lines · Mauro Carvalho Chehab
Em Tue, 9 Aug 2016 18:19:14 -0700
"Luis R. Rodriguez" <[email protected]> escreveu:Quoted 8 lines
I'm excited to see the new documentation format, so I'm changing my documentation in the pending patches I have to use it. I however cannot generate anything other than the main
Documentation/output/pdf/Kernel.pdfHow can I see in PDF the other documentation?
Quoted 7 lines
I'm using:
make DOCBOOKS="" pdfdocs
The
Documentation/output/pdf/Kernel.pdfonly has a bit of the documentation on how to write docs, nothing else. Using htmldocs as a target works but I am not a fan of that output.Hi Luis,
Unfortunately, on my tests with Sphinx, I noticed that pdf output is not actually working fine. It relies on a tool called rst2pdf. On my tests, I noticed that:
Quoted 7 lines
- rst2pdf is completely broken with Sphinx 1.3.x;
- rst2pdf doesn't handle complex documents, even with Sphinx 1.4.5;
- rst2pdf always report a successful build, even when it fails;
- I also consulted a developer with years of experience with Sphinx: he recommended removing the media book from the pdf build, as he doesn't know any fixup or replacement for the tool that would work for the media docs.
Quoted 2 lines
Hmm I see. What was the limitting issue ? The fact that media had embedded PDFs?
I guess not... I tried to debug rst2pdf, but it is very hard to identify the problem, as it doesn't point where the error is. I remember I changed some things to fix rst2pdf parser, but it still complains that some things were missing.
Quoted 6 lines
Quoted 6 lines
If you want to play with PDF output, you should take a look on this patch:
https://patchwork.kernel.org/patch/9231687/(commit
520a247760f750307b53db905a10a17df1700f3b)
Quoted 8 lines
Quoted 1 line
Reverting it or adding your document main rst file there would let you
play
Quoted 3 lines
with pdfdocs, if you're using Sphinx version 1.4.x.
If you revert it, you'll also need the enclosed patch.
Ah OK thanks. So if our documentation works with this and PDF output we can add just that single doc (in my case a new folder), is that right ?
Not sure. Better ask Jon how he wants to proceed with that.
Regards,
Mauro
cover Documentation: switch to pdflatex and fix pdf build
Hi Luis, all -
With these you should be able to get started with pdf generation. It's a quick transition to pdflatex, the patches are not very pretty, but the pdf output is. Patch 3/3 works as an example where to add your stuff (latex_documents in conf.py) and how.
Mauro, please look at patch 1/3 for a quick fix for the media build.
Markus, you can now unleash your "I told you so" on the rst2pdf. ;)
BR,
Jani.
Jani Nikula (3):
Documentation/sphinx: build the media intermediate rst files for all
outputs
Documentation: switch to pdflatex for pdf generation
Documentation: exclude media documentation from pdf generation
Documentation/Makefile.sphinx | 18 +++++++++--------- Documentation/conf.py | 4 +++- Documentation/media/Makefile | 3 ++- 3 files changed, 14 insertions(+), 11 deletions(-)
Signature · 1 line
2.1.4
RFC PATCH 1/3 Documentation/sphinx: build the media intermediate rst files for all outputs
This is a stopgap measure to allow building outputs other than html.
--- Documentation/Makefile.sphinx | 3 +-- Documentation/media/Makefile | 3 ++- 2 files changed, 3 insertions(+), 3 deletions(-)
Documentation/Makefile.sphinx +1 -2
@@ -38,10 +38,9 @@ ALLSPHINXOPTS = -D version=$(KERNELVERSION) -D release=$(KERNELRELEASE) -d $(B I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . quiet_cmd_sphinx = SPHINX $@ - cmd_sphinx = BUILDDIR=$(BUILDDIR) $(SPHINXBUILD) -b $2 $(ALLSPHINXOPTS) $(BUILDDIR)/$2 + cmd_sphinx = $(MAKE) BUILDDIR=$(BUILDDIR) $(build)=Documentation/media all; BUILDDIR=$(BUILDDIR) $(SPHINXBUILD) -b $2 $(ALLSPHINXOPTS) $(BUILDDIR)/$2 htmldocs: - $(MAKE) BUILDDIR=$(BUILDDIR) -f $(srctree)/Documentation/media/Makefile $@ $(call cmd,sphinx,html) pdfdocs:
Documentation/media/Makefile +2 -1
@@ -10,7 +10,8 @@ FILES = audio.h.rst ca.h.rst dmx.h.rst frontend.h.rst net.h.rst video.h.rst \ TARGETS := $(addprefix $(BUILDDIR)/, $(FILES)) -htmldocs: $(BUILDDIR) ${TARGETS} +.PHONY: all +all: $(BUILDDIR) ${TARGETS} $(BUILDDIR): $(Q)mkdir -p $@
Signature · 1 line
2.1.4
Re: [RFC PATCH 1/3] Documentation/sphinx: build the media intermediate rst files for all outputs
Em Wed, 10 Aug 2016 18:54:07 +0300
Jani Nikula <[email protected]> escreveu:
Quoted 1 line
This is a stopgap measure to allow building outputs other than html.
Didn't test, but looking at the patch, it looks OK.
So,
Quoted 28 lines
Signed-off-by Jani Nikula <[email protected]>--- Documentation/Makefile.sphinx | 3 +-- Documentation/media/Makefile | 3 ++- 2 files changed, 3 insertions(+), 3 deletions(-)Documentation/Makefile.sphinx +1
-2@@ -38,10 +38,9 @@ ALLSPHINXOPTS = -D version=$(KERNELVERSION) -D release=$(KERNELRELEASE) -d $(B I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . quiet_cmd_sphinx = SPHINX $@ - cmd_sphinx = BUILDDIR=$(BUILDDIR) $(SPHINXBUILD) -b $2 $(ALLSPHINXOPTS) $(BUILDDIR)/$2 + cmd_sphinx = $(MAKE) BUILDDIR=$(BUILDDIR) $(build)=Documentation/media all; BUILDDIR=$(BUILDDIR) $(SPHINXBUILD) -b $2 $(ALLSPHINXOPTS) $(BUILDDIR)/$2 htmldocs: - $(MAKE) BUILDDIR=$(BUILDDIR) -f $(srctree)/Documentation/media/Makefile $@ $(call cmd,sphinx,html) pdfdocs:Documentation/media/Makefile +0
-0
Documentation/media/Makefile FILES = audio.h.rst ca.h.rst dmx.h.rst frontend.h.rst net.h.rst video.h.rst \ · line 17
Quoted 9 lines
Documentation/media/Makefile +2
-1@@ -10,7 +10,8 @@ FILES = audio.h.rst ca.h.rst dmx.h.rst frontend.h.rst net.h.rst video.h.rst \ TARGETS := $(addprefix $(BUILDDIR)/, $(FILES)) -htmldocs: $(BUILDDIR) ${TARGETS} +.PHONY: all +all: $(BUILDDIR) ${TARGETS} $(BUILDDIR): $(Q)mkdir -p $@
Cheers,
Mauro
RFC PATCH 2/3 Documentation: switch to pdflatex for pdf generation
In reply to Jani NikulaLooks like rst2pdf is not robust enough, especially for large documents.
Use recursive make on the Sphinx generated makefile to convert latex to pdf. The ugly detail is that pdf is generated into Documentation/output/latex.
Unfortunately, the pdflatex build generates huge amounts of build log noise, and also fails in the end. We'll fix that next.
--- Documentation/Makefile.sphinx | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-)
Documentation/Makefile.sphinx +8 -7
@@ -25,8 +25,8 @@ else ifneq ($(DOCBOOKS),) else # HAVE_SPHINX -# User-friendly check for rst2pdf -HAVE_RST2PDF := $(shell if python -c "import rst2pdf" >/dev/null 2>&1; then echo 1; else echo 0; fi) +# User-friendly check for pdflatex +HAVE_PDFLATEX := $(shell if which pdflatex >/dev/null 2>&1; then echo 1; else echo 0; fi) # Internal variables. PAPEROPT_a4 = -D latex_paper_size=a4
@@ -44,12 +44,13 @@ htmldocs: $(call cmd,sphinx,html) pdfdocs: -ifeq ($(HAVE_RST2PDF),0) - $(warning The Python 'rst2pdf' module was not found. Make sure you have the module installed to produce PDF output.) +ifeq ($(HAVE_PDFLATEX),0) + $(warning The 'pdflatex' command was not found. Make sure you have it installed and in PATH to produce PDF output.) @echo " SKIP Sphinx $@ target." -else # HAVE_RST2PDF - $(call cmd,sphinx,pdf) -endif # HAVE_RST2PDF +else # HAVE_PDFLATEX + $(call cmd,sphinx,latex) + $(Q)$(MAKE) -C $(BUILDDIR)/latex +endif # HAVE_PDFLATEX epubdocs: $(call cmd,sphinx,epub)
Signature · 1 line
2.1.4
Re: [RFC PATCH 2/3] Documentation: switch to pdflatex for pdf generation
Am 10.08.2016 um 17:54 schrieb Jani Nikula <[email protected]>:
Quoted 8 lines
Looks like rst2pdf is not robust enough, especially for large documents.
Use recursive make on the Sphinx generated makefile to convert latex to pdf. The ugly detail is that pdf is generated into
Documentation/output/latex.Unfortunately, the pdflatex build generates huge amounts of build log noise, and also fails in the end. We'll fix that next.
A good starting point might be to ship our own tex-Makefile:
create a folder e.g. Documentation/sphinx-tex place your Makefile (a copy of Sphinx's Makefile) in. In conf.py set the
latex_additional_files = [
"sphinx-text/Makefile"
]
In the sphinx-tex/Makefile set e.g.
# Additional LaTeX options
LATEXOPTS = -interaction=batchmode
LATEX_ENV = max_print_line=120
LATEX = $(LATEX_ENV) latex PDFLATEX = $(LATEX_ENV) pdflatex MAKEINDEX = $(LATEX_ENV) makeindex
this is what I do in my projects to separate my tex-Makefile settings from the main sphinx-Makefile.
-- Markus --
Quoted 43 lines
Signed-off-by Jani Nikula <[email protected]>---
Documentation/Makefile.sphinx| 15 ++++++++-------1 file changed, 8 insertions(+), 7 deletions(-)Documentation/Makefile.sphinx +0
-0@@ -25,8 +25,8 @@ else ifneq ($(DOCBOOKS),)else #
HAVE_SPHINXDocumentation/Makefile.sphinx +2
-2-# User-friendly check for rst2pdf -HAVE_RST2PDF := $(shell if python -c "import rst2pdf" >/dev/null 2>&1; then echo 1; else echo 0; fi) +# User-friendly check for pdflatex +HAVE_PDFLATEX := $(shell if which pdflatex >/dev/null 2>&1; then echo 1; else echo 0; fi)# Internal variables.
PAPEROPT_a4= -D latex_paper_size=a4Documentation/Makefile.sphinx +0
-0@@ -44,12 +44,13 @@ htmldocs: $(call cmd,sphinx,html)pdfdocs:
Documentation/Makefile.sphinx +2
-2-ifeq ($(HAVE_RST2PDF),0) - $(warning The Python 'rst2pdf' module was not found. Make sure you have the module installed to produce PDF output.) +ifeq ($(HAVE_PDFLATEX),0) + $(warning The 'pdflatex' command was not found. Make sure you have it installed and in PATH to produce PDF output.)@echo " SKIP Sphinx $@ target."
Documentation/Makefile.sphinx +4
-3-else # HAVE_RST2PDF - $(call cmd,sphinx,pdf) -endif # HAVE_RST2PDF +else # HAVE_PDFLATEX + $(call cmd,sphinx,latex) + $(Q)$(MAKE) -C $(BUILDDIR)/latex +endif # HAVE_PDFLATEXepubdocs:
$(call cmd,sphinx,epub)Signature · 1 line
2.1.4
Re: [RFC PATCH 2/3] Documentation: switch to pdflatex for pdf generation
In reply to Jani NikulaQuoted 10 lines · Jani Nikula
Looks like rst2pdf is not robust enough, especially for large documents.
Use recursive make on the Sphinx generated makefile to convert latex to pdf. The ugly detail is that pdf is generated into
Documentation/output/latex.Unfortunately, the pdflatex build generates huge amounts of build log noise, and also fails in the end. We'll fix that next.
Signed-off-by Jani Nikula <[email protected]>
The commit log really does not much to explain to what bad extend rst2pdf is in, mentioning at least Mauro's feedback that he is aware this will not be fixed seems sufficient.
Luis
RFC PATCH 3/3 Documentation: exclude media documentation from pdf generation
In reply to Jani NikulaAlthough pdflatex is more robust than rst2pdf, building media documentation pdf still fails. Exclude media documentation from pdf generation for now.
--- Documentation/conf.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
Documentation/conf.py +3 -1
@@ -268,7 +268,9 @@ latex_elements = { # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ - (master_doc, 'TheLinuxKernel.tex', 'The Linux Kernel Documentation', + ('kernel-documentation', 'kernel-documentation.tex', 'The Linux Kernel Documentation', + 'The kernel development community', 'manual'), + ('gpu/index', 'gpu.tex', 'Linux GPU Driver Developer\'s Guide', 'The kernel development community', 'manual'), ]
Signature · 1 line
2.1.4
Re: [RFC PATCH 3/3] Documentation: exclude media documentation from pdf generation
Hi Jani,
Em Wed, 10 Aug 2016 18:54:09 +0300
Jani Nikula <[email protected]> escreveu:
Quoted 3 lines
Although pdflatex is more robust than rst2pdf, building media documentation pdf still fails. Exclude media documentation from pdf generation for now.
Again, didn't test, but do you think the issues with pdflatex and media are solvable?
Quoted 11 lines
Signed-off-by Jani Nikula <[email protected]>--- Documentation/conf.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)Documentation/conf.py +0
-0
Documentation/conf.py latex_elements = { · line 276
Quoted 10 lines
Documentation/conf.py +3
-1@@ -268,7 +268,9 @@ latex_elements = { # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ - (master_doc, 'TheLinuxKernel.tex', 'The Linux Kernel Documentation', + ('kernel-documentation', 'kernel-documentation.tex', 'The Linux Kernel Documentation', + 'The kernel development community', 'manual'), + ('gpu/index', 'gpu.tex', 'Linux GPU Driver Developer\'s Guide', 'The kernel development community', 'manual'), ]
Cheers,
Mauro
Re: [RFC PATCH 3/3] Documentation: exclude media documentation from pdf generation
Am 10.08.2016 um 18:16 schrieb Mauro Carvalho Chehab <[email protected]>:
Quoted 11 lines
Hi Jani,
Em Wed, 10 Aug 2016 18:54:09 +0300
Jani Nikula <[email protected]> escreveu:Quoted 3 lines
Although pdflatex is more robust than rst2pdf, building media documentation pdf still fails. Exclude media documentation from pdf generation for now.
Again, didn't test, but do you think the issues with pdflatex and media are solvable?
Not all, we have to test book by book and see what fails and what the reasons. Large tables will never format propper on paper sizes.
But first lets ack and merge all this together.
-- Markus --
Quoted 28 lines
Quoted 20 lines
Signed-off-by Jani Nikula <[email protected]>---
Documentation/conf.py| 4 +++-1 file changed, 3 insertions(+), 1 deletion(-)Documentation/conf.py +0
-0@@ -268,7 +268,9 @@ latex_elements = { # (source start file, target name, title, # author, documentclass [howto, manual, or own class]).
latex_documents= [Documentation/conf.py +3
-1- (master_doc, 'TheLinuxKernel.tex', 'The Linux Kernel Documentation', + ('kernel-documentation', 'kernel-documentation.tex', 'The Linux Kernel Documentation', + 'The kernel development community', 'manual'), + ('gpu/index', 'gpu.tex', 'Linux GPU Driver Developer\'s Guide', 'The kernel development community', 'manual'),Quoted 2 lines
]
Cheers,
Mauro
Re: [RFC PATCH 3/3] Documentation: exclude media documentation from pdf generation
In reply to Jani NikulaQuoted 14 lines · Jani Nikula
Although pdflatex is more robust than rst2pdf, building media documentation pdf still fails. Exclude media documentation from pdf generation for now.
Signed-off-by Jani Nikula <[email protected]>--- Documentation/conf.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)Documentation/conf.py +0
-0
Documentation/conf.py latex_elements = { · line 274
Quoted 8 lines
Documentation/conf.py +3
-1@@ -268,7 +268,9 @@ latex_elements = { # (source start file, target name, title, # author, documentclass [howto, manual, or own class]). latex_documents = [ - (master_doc, 'TheLinuxKernel.tex', 'The Linux Kernel Documentation', + ('kernel-documentation', 'kernel-documentation.tex', 'The Linux Kernel Documentation', + 'The kernel development community', 'manual'), + ('gpu/index', 'gpu.tex', 'Linux GPU Driver Developer\'s Guide', 'The kernel development community', 'manual'),
Ah so other than expanding Documentation/index.rst we also have to edit this file. It would be good to expand the new documentation to reflect this as this was not clear.
With these changes I confirm I get my documentation in PDF format for the new sections I am adding!
Thanks!
Luis
Re: [RFC PATCH 0/3] Documentation: switch to pdflatex and fix pdf build
In reply to Jani NikulaAm 10.08.2016 um 17:54 schrieb Jani Nikula <[email protected]>:
Quoted 10 lines
Hi Luis, all -
With these you should be able to get started with pdf generation. It's a quick transition to pdflatex, the patches are not very pretty, but the pdf output is. Patch 3/3 works as an example where to add your stuff (
latex_documentsinconf.py) and how.Mauro, please look at patch 1/3 for a quick fix for the media build.
Markus, you can now unleash your "I told you so" on the rst2pdf. ;)
:)
same here: Didn't test, but looking at the patch, it looks OK.
One suggestion: later when my "generic way to build only sphinx sub-folders" patch ...
http://marc.info/?l=linux-doc&m=147066213515280&w=2
... is acked and merge, it might be better to place the latex_document option into a gpu/conf.py ?
latex_documents = [
('gpu/index', 'gpu.tex', 'Linux GPU Driver Developer\'s Guide',
'The kernel development community', 'manual') ]
I haven't thought this to the end, just an idea.
-- Markus --
Re: [RFC PATCH 0/3] Documentation: switch to pdflatex and fix pdf build
In reply to Jani NikulaQuoted 11 lines · Jani Nikula
Hi Luis, all -
With these you should be able to get started with pdf generation. It's a quick transition to pdflatex, the patches are not very pretty, but the pdf output is. Patch 3/3 works as an example where to add your stuff (
latex_documentsinconf.py) and how.Mauro, please look at patch 1/3 for a quick fix for the media build.
Markus, you can now unleash your "I told you so" on the rst2pdf. ;)
Thanks this works! It would be nice to have a target for tex so one can then just worry about fixing the errors generated to generate the tex file. Otherwise I have to hit CTRL C right beore pdflatex is called.
Luis
Jonathan Corbet
maintainerRe: [RFC PATCH 0/3] Documentation: switch to pdflatex and fix pdf build
In reply to Jani NikulaQuoted 4 lines · On Wed, 10 Aug 2016 18:54:06 +0300 Jani Nikula <[email protected]> wrote:
With these you should be able to get started with pdf generation. It's a quick transition to pdflatex, the patches are not very pretty, but the pdf output is. Patch 3/3 works as an example where to add your stuff (
latex_documentsinconf.py) and how.
OK, now I have a bone to pick with you.
I applied this, then decided to install the needed toolchain on the Tumbleweed system I've been playing with; it wanted to install 1,727 packages to get pdflatex. Pandoc just doesn't seem so bad anymore.
So I switched to the Fedora system, and found myself in a twisty maze of missing font files, missing style files, missing babel crap, etc., each doled out to me one file per run. But I did eventually get PDFs out of it.
The output isn't great; among other things, it seems to be about 1/2 blank pages. But it's something.
I've applied this so we have something to play with, but it doesn't feel like a great solution. This is the sort of installation hell that we wanted to get away from. It makes me wonder how hard it can really be to fix rst2pdf; I wish I could say I'll find some time to figure that out. Sigh.
jon
Re: [RFC PATCH 0/3] Documentation: switch to pdflatex and fix pdf build
Am 13.08.2016 um 00:40 schrieb Jonathan Corbet <[email protected]>:
Quoted 8 lines
Quoted 4 lines · On Wed, 10 Aug 2016 18:54:06 +0300 Jani Nikula <[email protected]> wrote:
With these you should be able to get started with pdf generation. It's a quick transition to pdflatex, the patches are not very pretty, but the pdf output is. Patch 3/3 works as an example where to add your stuff (
latex_documentsinconf.py) and how.
Quoted 5 lines
OK, now I have a bone to pick with you.
I applied this, then decided to install the needed toolchain on the Tumbleweed system I've been playing with; it wanted to install 1,727 packages to get pdflatex. Pandoc just doesn't seem so bad anymore.
I'am complete disenchanted on this topic. My experience is:
1) You wan't get any reasonable typesetting engine which preserves your disk space. I don't know how many files or packages are installed, the only thing I know is, a TeX installation is always >1GB.
2) You wan't get a (pdf, ps,..) book with a perfect layout without any handcraft or at least a *theming*. TeX has many options to influence the layout and Sphinx provides it's own LaTeX-document class (sphinxmanual) which is IMHO awful.
Quoted 4 lines
So I switched to the Fedora system, and found myself in a twisty maze of missing font files, missing style files, missing babel crap, etc., each doled out to me one file per run. But I did eventually get PDFs out of it.
On debian it should be enough to install *base* and *recommended*
sudo apt-get install
texlive-base texlive-latex-recommended
Quoted 2 lines
The output isn't great; among other things, it seems to be about 1/2 blank pages.
1/2 ? .. I have only empty pages at the start of parts or chapters, which is a typical layout setting.
Quoted 1 line
But it's something.
This is the sphinxmanual document class.
Quoted 3 lines
I've applied this so we have something to play with, but it doesn't feel like a great solution. This is the sort of installation hell that we wanted to get away from.
See above, on debian it should be enough to install the two meta packages.
Quoted 3 lines
It makes me wonder how hard it can really be to
fix rst2pdf; I wish I could say I'll find some time to figure that out. Sigh.
I gave it a try, but as I come closer to the sources I realized that it is hair-raising. I looked at the issues, added a comment to a related issue, a few days later the issue was closed without any comment or code change.
https://github.com/rst2pdf/rst2pdf/issues/556#issuecomment-228779542
My advice, if you don't like to waste your time: forget it.
Some thoughts of mine, wrote in an earlier mail:
Quoted 15 lines
The sphinx-doc build-in LaTeX builder
* http://www.sphinx-doc.org/en/stable/config.html#options-for-latex-output
has some drawbacks, e.g. it produce LaTeX for the pdfTeX engine. LaTeX is by default ASCII and it needs some "inputenc" to supporta wider range of characters. This is not very helpful if you have a toolchain in an international community.
The alternative to LaTeX is to use the XeTeX engine, which supports UTF-8 encoded input by default and supports TrueType/OpenType fonts directly. Thats why I started to write a XeLaTeX builder ...
* https://github.com/return42/sphkerneldoc/blob/master/scripts/site-python/xelatex_ext/__init__.py#L15
Quoted 7 lines
... but I can't predict when this will be finished ...
However which tool is used, my experience is, that building PDF (books) with a minimum of quality is not simple. Layout width tables, split table content over pages, switch from landscape to portrait and versus, the flow of objects etc. .. all this will need some manually interventions.
-- Markus --
Re: [RFC PATCH 0/3] Documentation: switch to pdflatex and fix pdf build
In reply to Jonathan CorbetQuoted 8 lines · Jonathan Corbet
Quoted 4 lines · On Wed, 10 Aug 2016 18:54:06 +0300 Jani Nikula <[email protected]> wrote:
With these you should be able to get started with pdf generation. It's a quick transition to pdflatex, the patches are not very pretty, but the pdf output is. Patch 3/3 works as an example where to add your stuff (
latex_documentsinconf.py) and how.
Quoted 5 lines
OK, now I have a bone to pick with you.
I applied this, then decided to install the needed toolchain on the Tumbleweed system I've been playing with; it wanted to install 1,727 packages to get pdflatex. Pandoc just doesn't seem so bad anymore.
Jon, I sent these to unblock Luis, and as a starting point for a discussion about rst2pdf vs. pdflatex. I didn't mean I'd want these merged as-is! I'm sorry if I didn't make myself clear.
I don't mind at all if you want to drop them.
As I played around, it worked for *me* better than rst2pdf. I had pdflatex already installed, so I didn't pay attention to the deps, and the numbers you quote come as a suprise.
Quoted 5 lines
I've applied this so we have something to play with, but it doesn't feel like a great solution. This is the sort of installation hell that we wanted to get away from. It makes me wonder how hard it can really be to fix rst2pdf; I wish I could say I'll find some time to figure that out. Sigh.
I'm afraid pdf generation is not very high on my list of priorities right now. Surely I'll get back to this eventually, but not any time soon. :(
BR,
Jani.
Signature · 1 line
Jani Nikula, Intel Open Source Technology Center
Re: [RFC PATCH 0/3] Documentation: switch to pdflatex and fix pdf build
Em Mon, 15 Aug 2016 12:40:21 +0300
Jani Nikula <[email protected]> escreveu:
Quoted 31 lines · Jonathan Corbet
Quoted 8 lines
Quoted 4 lines · On Wed, 10 Aug 2016 18:54:06 +0300 Jani Nikula <[email protected]> wrote:
With these you should be able to get started with pdf generation. It's a quick transition to pdflatex, the patches are not very pretty, but the pdf output is. Patch 3/3 works as an example where to add your stuff (
latex_documentsinconf.py) and how.Quoted 5 lines
OK, now I have a bone to pick with you.
I applied this, then decided to install the needed toolchain on the Tumbleweed system I've been playing with; it wanted to install 1,727 packages to get pdflatex. Pandoc just doesn't seem so bad anymore.
Jon, I sent these to unblock Luis, and as a starting point for a discussion about rst2pdf vs. pdflatex. I didn't mean I'd want these merged as-is! I'm sorry if I didn't make myself clear.
I don't mind at all if you want to drop them.
As I played around, it worked for *me* better than rst2pdf. I had pdflatex already installed, so I didn't pay attention to the deps, and the numbers you quote come as a suprise.
Quoted 5 lines
I've applied this so we have something to play with, but it doesn't feel like a great solution. This is the sort of installation hell that we wanted to get away from. It makes me wonder how hard it can really be to fix rst2pdf; I wish I could say I'll find some time to figure that out. Sigh.
Quoted 3 lines
I'm afraid pdf generation is not very high on my list of priorities right now. Surely I'll get back to this eventually, but not any time soon. :(
I'm afraid that, no matter if we'll be using rst2pdf or pdflatex, we'll need LaTex anyway.
One of the things that we want on media is to better represent the math formulas there. Just enabling the math extension is enough for it to require not only LaTex, but also this package (on Fedora):
texlive-anyfontsize.noarch
So, unfortunately, I'm afraid that we'll end by having a much worse dependency chain than what we used to have with DocBook.
An (incomplete) patch to let you test it is enclosed (it is actually after patches 1 and 2 of this series).
Ah, it seems that rst2pdf is not compatible with math extension...
If we change the conf.py to:
extensions = ['sphinx.ext.imgmath', 'kernel-doc', 'rstFlatTable', 'kernel_include']
# Gracefully handle missing rst2pdf.
try:
import rst2pdf
extensions += ['rst2pdf.pdfbuilder']
except ImportError:
pass
The logic that tries to import rst2pdf produces the following error:
Extension error:
Config value 'math_number_all' already present
And, if we don't add the imgmath extension, the math expression appears broken at the generated html page:
The transfer function defined for SMPTE 170M is the same as the one defined in Rec. 709.
L' = -1.099(-L)^{0.45} + 0.099 \text{ for } L \le-0.018 L' = 4.5L \text{ for } -0.018 < L < 0.018 L' = 1.099L^{0.45} - 0.099 \text{ for } L \ge 0.018Btw, changing to math is a requirement for pdflatex to work on media, as the font used by Sphinx/pdflatex doesn't have UTF-8 symbols used on math. So, trying to build the documentation with pdfdocs (after patches 1 and 2, before patch 3) would give fatal errors like:
! Package inputenc Error: Unicode char \u8:≥ not set up for use with LaTeX.
See the inputenc package documentation for explanation.
Type H <return> for immediate help.
...
l.7759 ....099L $^{\text{0.45}}$ - 0.099 for L ≥
0.018The problem is that the "grater or equal than" UTF-8 Math symbol is not supported on the LaTex font.
Thanks,
Mauro
Documentation/conf.py +6 -6
@@ -28,14 +28,14 @@ sys.path.insert(0, os.path.abspath('sphinx')) # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. -extensions = ['kernel-doc', 'rstFlatTable', 'kernel_include'] +extensions = ['sphinx.ext.imgmath', 'kernel-doc', 'rstFlatTable', 'kernel_include'] # Gracefully handle missing rst2pdf. -try: - import rst2pdf - extensions += ['rst2pdf.pdfbuilder'] -except ImportError: - pass +#try: +# import rst2pdf +# extensions += ['rst2pdf.pdfbuilder'] +#except ImportError: +# pass # Add any paths that contain templates here, relative to this directory. templates_path = ['_templates']
Documentation/media/uapi/v4l/pixfmt-007.rst +5 -3
@@ -72,11 +72,13 @@ SMPTE C set, so this colorspace is sometimes called SMPTE C as well. The transfer function defined for SMPTE 170M is the same as the one defined in Rec. 709. - L' = -1.099(-L) :sup:`0.45` + 0.099 for L ≤ -0.018 +.. math:: - L' = 4.5L for -0.018 < L < 0.018 + L' = -1.099(-L)^{0.45} + 0.099 \text{ for } L \le-0.018 - L' = 1.099L :sup:`0.45` - 0.099 for L ≥ 0.018 + L' = 4.5L \text{ for } -0.018 < L < 0.018 + + L' = 1.099L^{0.45} - 0.099 \text{ for } L \ge 0.018 Inverse Transfer function:
Re: [RFC PATCH 0/3] Documentation: switch to pdflatex and fix pdf build
Hi Jon,
Em Mon, 15 Aug 2016 09:17:52 -0300
Mauro Carvalho Chehab <[email protected]> escreveu:
Quoted 23 lines
Em Mon, 15 Aug 2016 12:40:21 +0300
Jani Nikula <[email protected]> escreveu:Quoted 20 lines · Jonathan Corbet
Quoted 8 lines
Quoted 4 lines · On Wed, 10 Aug 2016 18:54:06 +0300 Jani Nikula <[email protected]> wrote:
With these you should be able to get started with pdf generation. It's a quick transition to pdflatex, the patches are not very pretty, but the pdf output is. Patch 3/3 works as an example where to add your stuff (
latex_documentsinconf.py) and how.Quoted 5 lines
OK, now I have a bone to pick with you.
I applied this, then decided to install the needed toolchain on the Tumbleweed system I've been playing with; it wanted to install 1,727 packages to get pdflatex. Pandoc just doesn't seem so bad anymore.
Jon, I sent these to unblock Luis, and as a starting point for a discussion about rst2pdf vs. pdflatex. I didn't mean I'd want these merged as-is! I'm sorry if I didn't make myself clear.
I don't mind at all if you want to drop them.
I played for a while with rst2pdf and LaTeX-based tools, plus using Sphinx math extension to improve the media documentation.
Here's my findings:
1) I'm pretty sure there's no way for us to get rid of LaTex.
The problem is that the Sphinx math extension depends on LaTeX amsmath extension. Ok, someone could write some extension in the future to add math support without requiring LaTeX and fix rst2pdf (or rewrite it), but this would take time and efforts.
So, for now, I think we should just assume that anyone wanting to generate pdf docs will need LaTeX (or, more likely, TeTeX).
2) rst2pdf seems to be incompatible with Sphinx math extension, at least on Sphinx 1.4.5;
3) pdflatex doesn't handle UTF-8 chars. This is a problem for media, as we use two UTF-8 symbols that are incompatible with the fonts used by pdflatex:
- ≥ U+2265 (GREATER-THAN OR EQUAL TO) utf-8 character
- ≤ u+2264 (LESS-THAN OR EQUAL TO) utf-8 character
4) pdflatex output is not nice: all black and white; ugly (IMHO) fonts;
5) At media docs, some tables will only print ok in landscape.
After making the media books build, I think that the best way is to use xelatex instead of pdfdocs. Visually, xelatex output is, IMHO, nice - and it has colors :-)
It seems that there's yet another option: lualatex. I didn't try to build with it. So, not sure if its output is better or not, nor if some extra config for it is needed at conf.py.
I sent a patch series addressing most of the issues above.
If you want to see the output:
https://mchehab.fedorapeople.org/TheLinuxKernel.pdf
This was built on Fedora 24.
Issues:
-------
Even with xelatex, there are still some issues to be addressed:
- There are two hacks to build media: one removes the *.h.rst files and the other one comments out two C code blocks inside tables;
- There are lots of noise at PDF output. I didn't even try to look into them;
- Almost all tables at the media books are mangled. Rotating them to landscape can fix several of them. I added support for it, however, that requires to manually add LaTeX tags before and after the tables, like:
.. raw:: latex
\begin{landscape}
<some table(s)>
.. raw:: latex
\end{landscape}As the above tags are LaTeX specific, they should not interfere on html (or e-pub) output.
One such example is this table:
"Table 2.18: Packed RGB Image Formats"
(at page 92 of the PDF file)
I'm seeking for a solution to scale it and rotate (as just rotating it is not enough).
Signature · 2 lines
Thanks, Mauro
Re: [RFC PATCH 0/3] Documentation: switch to pdflatex and fix pdf build
Quoted 11 lines · Mauro Carvalho Chehab
5) At media docs, some tables will only print ok in landscape.
After making the media books build, I think that the best way is to use xelatex instead of pdfdocs. Visually, xelatex output is, IMHO, nice - and it has colors :-)
It seems that there's yet another option: lualatex. I didn't try to build with it. So, not sure if its output is better or not, nor if some extra config for it is needed at
conf.py.I sent a patch series addressing most of the issues above.
I didn't see the series, but I'm surprised we now have HAVE_PDFLATEX that actually checks for 'xelatex'. The commit subject doing the change doesn't even mention xelatex, it's just about UTF-8:
commit a682ec4ba10c88231cdbb8bb9823b2cc749d6364
Author: Mauro Carvalho Chehab <[email protected]> Date: Tue Aug 16 13:25:39 2016 -0300
docs-rst: Don't mangle with UTF-8 chars on LaTeX/PDF output
</bikeshedding after the fact>
BR,
Jani.
Signature · 1 line
Jani Nikula, Intel Open Source Technology Center
Re: [RFC PATCH 0/3] Documentation: switch to pdflatex and fix pdf build
Em Fri, 19 Aug 2016 15:54:37 +0300
Jani Nikula <[email protected]> escreveu:
Quoted 17 lines · Mauro Carvalho Chehab
Quoted 11 lines
5) At media docs, some tables will only print ok in landscape.
After making the media books build, I think that the best way is to use xelatex instead of pdfdocs. Visually, xelatex output is, IMHO, nice - and it has colors :-)
It seems that there's yet another option: lualatex. I didn't try to build with it. So, not sure if its output is better or not, nor if some extra config for it is needed at
conf.py.I sent a patch series addressing most of the issues above.
I didn't see the series, but I'm surprised we now have
HAVE_PDFLATEXthat actually checks for 'xelatex'. The commit subject doing the change doesn't even mention xelatex, it's just about UTF-8:
Quoted 7 lines
commit
a682ec4ba10c88231cdbb8bb9823b2cc749d6364Author: Mauro Carvalho Chehab <[email protected]> Date: Tue Aug 16 13:25:39 2016 -0300docs-rst: Don't mangle with UTF-8 chars on LaTeX/PDF output
</bikeshedding after the fact>
It was mentioned at patch 0/9:
https://www.mail-archive.com/[email protected]/msg101408.html
I opted to not change the HAVE_PDFLATEX, as the macro that sets the latex variant on Sphinx latex Makefile is called "PDFLATEX". Also, we might change it in the future to use some other tool or even add some logic to allow supporting multiple tools, like we have with DocBook.
The main goal of XeTeX is to add unicode font support to LaTex, according to:
https://en.wikipedia.org/wiki/XeTeX
We need that, as media uses a lot of math Unicode symbols inside the documentation.
Signature · 2 lines
Thanks, Mauro