The downside of math::

Hey, Mauro,

So I was a little surprised to find that the htmldocs build now breaks on one of my machines due to a lack of LaTeX. A bit of digging turned up the culprit: commit b7ff94df5628 (pixfmt-007.rst: use Sphinx math::
expressions). The math:: directive uses LaTeX to process the math markup.

What this means is that said commit has made LaTeX a dependency for the htmldocs build. I'm not convinced that this is a good idea; that's a massive dependency to add for web builds that, ostensibly, should not need it. Certainly we shouldn't add it without discussion...:)

So I'll ask: how important to you is the math extension, and is there any way we could get you your fancy math in a way that doesn't force everybody to install the whole LaTeX package set?

Thanks,

jon

Hi Jon,

Em Wed, 19 Oct 2016 17:02:46 -0600
Jonathan Corbet <[email protected]> escreveu:

As you know, image handling comes with complex expressions ;) We have lots of expressions at the media documentation, but right now, the complex ones that require ReST math:: directive are related to colorspace conversions:

	https://linuxtv.org/downloads/v4l-dvb-apis-new/uapi/v4l/pixfmt-007.html

On DocBook, we used to represent the same expressions using several tricks, like superscript and used some UTF-8 math symbols. As expected, they didn't look nice there:

	https://linuxtv.org/downloads/v4l-dvb-apis/ch02s06.html

The initial conversion of those math expressions from DocBook make them look even worse. It also broke PDF generation, because LaTeX fonts were not compatible with UTF-8 math symbols (such issue was latter solved with xelatex - for some other remaining UTF-8 symbols).

That's said, we're currently discussing APIs for codecs on media. Eventually, their documentation could come with a way more complex expressions than what we have so far.

So, I really prefer not removing math support.

http://www.sphinx-doc.org/en/stable/ext/math.html

It sounded to me that only sphinx.ext.imgmath extension is capable of providing math on all outputs, by generating an image like this one, using LaTeX:

	https://linuxtv.org/downloads/v4l-dvb-apis-new/_images/math/ae0a381368b3837b88b1308aaccedcd9a438757a.png

There are two alternative math extensions that handle math::, but both seem to be specific for HTML, as they use JavaScript. Also, Sphinx require using either one of the extension, so we cannot enable both, one for PDF and the other one for HTML.

Perhaps there are some extensions that would allow using something else, like this one:
https://bitbucket.org/coh/sphinx-contrib-mathml/overview

But I haven't test. So, I'm not sure:
- if it would work;
- if it would be portable to both HTML and PDF outputs;
- if it won't require an even weirder toolbox.

Thanks,
Mauro

Em Wed, 19 Oct 2016 22:26:18 -0200
Mauro Carvalho Chehab <[email protected]> escreveu:

Btw, I was also surprised when I discovered such dependency myself... I even commented about that at the media mailing list, as I didn't understood why make htmldocs were not working on my server:

https://www.mail-archive.com/[email protected]&q=subject:%22Re%5C%3A+%5C%5BPATCH+0%5C%2F9%5C%5D+Prepare+Sphinx+to+build+media+PDF+books%22&o=newest&f=1

Regrds,
Mauro

I wonder if we could cook up a nice way to make the math:: usage conditional on actually being able to render it.

I would rather have the math:: directive produce just the preformatted indented block as-is in the html output, instead of failing, when latex is not available.

Adapt to the environment, produce the prettiest thing when you can, but fallback to something uglier if all the dependencies are not there. See what I did with the sphinx_rtd_theme in conf.py - we use it if it's there, but it's not a hard dependency.

BR,
Jani.

Jani Nikula, Intel Open Source Technology Center

Am 20.10.2016 um 16:55 schrieb Jani Nikula <[email protected]>:

Hi Jani,

you are right. I will add this point to my TODO list. Give me just a view days / ATM I'am working on a other project.

--Markus--

I think that's the ideal solution.

I got the docs build working again on my Fedora machine, but it threw me back into this loop for several iterations:

while (htmldocs build fails)
see which goddam LaTex file is missing now dig around to find which Fedora package provides it dnf install YA-texlive-package

This is just the kind of thing that I don't want to impose on anybody wanting to build the docs; if we get back to a place where almost nobody can do it again, we'll not have improved much.

LaTeX is a necessary evil for PDF output, it seems, but I really think that, one way or another, we need to be able to build the other formats without it.

Thanks,

jon

Am 21.10.2016 um 23:38 schrieb Jonathan Corbet <[email protected]>:

Hmm, when I deinstall latex on my Debian box, I only get a warning:

WARNING: LaTeX command 'latex' cannot be run (needed for math display), check the pngmath_latex setting

IMO the warning is OK, if not read below.

I guess, that Jon walks through the latex installation hell on fedora, which mauro has already done in August. Please take a look at the discussion we had about math and PDF generation:

https://www.mail-archive.com/[email protected]/msg05583.html

With this, I think we have a leak of knowledge and we should document latex installation dependencies right now.

If you don't want to see a warning log, we have to consider a solution like Johannes Berg posted here:

https://www.mail-archive.com/[email protected]/msg07071.html

@johannes: since this thread and our "sequence diagrams" thread addressing the same questions (how should we handle dependencies from extensions) I propose to continue the discussion here in this thread with Jon.

I wish, but I can't see any serious alternative. Thats why I think, we should accept latex and have to document it well.

Further I think we should not generate more (and more) external requirements like e.g. plantuml, Java or reportlab discussed here:

https://www.mail-archive.com/[email protected]/msg07035.html

I understand the need of sequences diagrams and I like to see such a feature in the kernel documentation. On the other side, since we have to maintain all dependencies well over a long time period on distributions with different package managers, I really hesitate to include dependencies which might flood out the banks. Right here, in the discussion of the math:: extension (which is part of the standard sphinx installation) we see, what problems can occur on different systems installations.

-- Markus --

Sure.

As I said before, regarding the annoying "while (build fails)" loop mentioned above - I think that's actually somewhat *better* than just (silently?) failing the build because some dependency wasn't there.

Perhaps, with my code suitably adjusted (should be easy), we could provide some kind of command line option like "DISABLE_PLUGINS=xyz" so you could just build as though you had no plugins installed, to avoid all the follow-up errors. (We'd implement this by changing my dummy code to also pass the import statement, like this:

setup = create_setup("sphinxcontrib.plantuml", ["uml"])

and moving the try/except clause into the dummy, and then that can read the environment variable easily and avoid even trying to import when it's set - that way, no dependencies can be needed.

I still disagree, I think we should make it easy to "opt out" for the build, but if we're really serious about writing good documentation we shouldn't (artificially) limit the tools available.

Sure, we probably shouldn't add seqdiag, plantuml, and others that all the do the same thing - if only for the sake of consistency in the generated output! - but having the ability to have such things is very nice and helpful for documentation.

johannes

I'd like to refine: Do not add non-trivial hard dependencies. Do not add dependencies the lack of which make large parts of generated documentation useless.

Graceful degradation on unmet dependencies is the key here. Give a build warn about missing dependencies. Try to do something sensible without the dependency. For extension directives, all else failing, embed the raw directive block contents using in the output, possibly accompanied by a reStructuredText admonition block [1]. For the math and diagram directives, I think this would work just fine.

BR,
Jani.

[1] http://docutils.sourceforge.net/docs/ref/rst/directives.html#specific-admonitions

Jani Nikula, Intel Open Source Technology Center

Agree.

All of that would be easy to achieve with something like the dummy plugin I posted the other day :)

But yes, I also agree that we shouldn't add any dependencies where the input isn't suitable as a fallback, since that also means that the input is going to be difficult to read and write in the source, which should remain a reasonable way to read things as well.

johannes

Agreed, that's a very good point too.

BR,
Jani.

Jani Nikula, Intel Open Source Technology Center