build: Build and install the info manual.
PATCH 1/1 build: Build and install the info manual.
Take advantage of the Sphinx texinfo backend to generate a QEMU info manual. The texinfo format allows for more structure and info readers provide more advanced navigation capabilities compared to manpages readers.
* configure (infodir): Add the --infodir option, which allows configuring the directory under which the info manuals are installed.
* docs/index.rst: Include the top level documents to prevent warnings (treated as errors by sphinx-build).
* docs/meson.build (sphinxinfo): Add new target.
--- configure | 3 +++ docs/index.rst | 2 ++ docs/meson.build | 13 +++++++++++++ meson.build | 4 +++- 4 files changed, 21 insertions(+), 1 deletion(-)
configure +3 -0
@@ -1069,6 +1069,8 @@ for opt do static="yes" QEMU_PKG_CONFIG_FLAGS="--static $QEMU_PKG_CONFIG_FLAGS" ;; + --infodir=*) infodir="$optarg" + ;; --mandir=*) mandir="$optarg" ;; --bindir=*) bindir="$optarg"
@@ -1786,6 +1788,7 @@ Advanced options (experts only): --smbd=SMBD use specified smbd [$smbd] --with-git=GIT use specified git [$git] --static enable static build [$static] + --infodir=PATH install info manual in PATH --mandir=PATH install man pages in PATH --datadir=PATH install firmware in PATH/$qemu_suffix --docdir=PATH install documentation in PATH/$qemu_suffix
docs/index.rst +2 -0
@@ -9,6 +9,7 @@ Welcome to QEMU's documentation! .. toctree:: :maxdepth: 2 :caption: Contents: + :glob: system/index user/index
@@ -16,3 +17,4 @@ Welcome to QEMU's documentation! interop/index specs/index devel/index + *
docs/meson.build +13 -0
@@ -32,10 +32,13 @@ if build_docs sphinxdocs = [] sphinxmans = [] + conf_py_files = [files('conf.py')] + foreach manual : manuals private_dir = meson.current_build_dir() / (manual + '.p') output_dir = meson.current_build_dir() / manual input_dir = meson.current_source_dir() / manual + conf_py_files += files(manual / 'conf.py') this_manual = custom_target(manual + ' manual', build_by_default: build_docs,
@@ -70,4 +73,14 @@ if build_docs endforeach alias_target('sphinxdocs', sphinxdocs) alias_target('man', sphinxmans) + + sphinxinfo = custom_target( + 'QEMU info manual generated source', + output: ['QEMU.texi', 'sphinxinfo.stamp'], + input: conf_py_files, + depfile: 'sphinxinfo.d', + command: [SPHINX_ARGS, '-Ddepfile=@DEPFILE@', '-Ddepfile_stamp=@OUTPUT1@', + '-b', 'texinfo', + meson.current_source_dir(), meson.current_build_dir()]) + alias_target('sphinxinfo', sphinxinfo) endif
meson.build +3 -1
@@ -1218,6 +1218,7 @@ if build_docs 'qemu_confdir': config_host['qemu_confdir']}) texi = { + 'QEMU': sphinxinfo, 'qemu-qmp-ref': ['docs/interop/qemu-qmp-ref.texi', qapi_doc_texi, version_texi], } if 'CONFIG_GUEST_AGENT' in config_host
@@ -1241,7 +1242,8 @@ if build_docs input: input, output: output, install: true, - install_dir: qemu_docdir / 'interop', + install_dir: ext == 'info' ? get_option('infodir') \ + : qemu_docdir / 'interop', command: cmd + args) endforeach alias_target(ext, t)
Signature · 1 line
2.28.0
Patchew URL: https://patchew.org/QEMU/[email protected]/
Patchew URL: https://patchew.org/QEMU/[email protected]/
Hi,
This series seems to have some coding style problems. See output below for more information:
Type: series
Message-id: [email protected]
Subject: [PATCH] build: Build and install the info manual.
=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===
Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
- [tag update] patchew/[email protected] -> patchew/[email protected] - [tag update] patchew/[email protected] -> patchew/[email protected] * [new tag] patchew/[email protected] -> patchew/[email protected]
Switched to a new branch 'test'
76a252c build: Build and install the info manual.
=== OUTPUT BEGIN ===
ERROR: Missing Signed-off-by: line(s)
total: 1 errors, 0 warnings, 69 lines checked
Commit 76a252c26b29 (build: Build and install the info manual.) has style problems, please review. If any of these errors are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===
Test command exited with code: 1
The full log is available at
http://patchew.org/logs/[email protected]/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]
Patchew URL: https://patchew.org/QEMU/[email protected]/
Patchew URL: https://patchew.org/QEMU/[email protected]/
Hi,
This series failed the docker-mingw@fedora build test. Please find the testing commands and their output below. If you have Docker installed, you can probably reproduce it locally.
=== TEST SCRIPT BEGIN ===
#! /bin/bash
export ARCH=x86_64
make docker-image-fedora V=1 NETWORK=1
time make docker-test-mingw@fedora J=14 NETWORK=1
=== TEST SCRIPT END ===
Host machine cpu: x86_64
Target machine cpu family: x86
Target machine cpu: x86_64
../src/meson.build:10: WARNING: Module unstable-keyval has no backwards or forwards compatibility and might not exist in future releases.
Program sh found: YES
Program python3 found: YES (/usr/bin/python3)
Configuring ninjatool using configuration
---
make[1]: Nothing to be done for 'all'.
make[1]: Leaving directory '/tmp/qemu-test/src/slirp'
Generating nsis with a custom command
make[1]: warning: jobserver unavailable: using -j1. Add '+' to parent make rule.
make[1]: Entering directory '/tmp/qemu-test/build'
make[2]: Entering directory '/tmp/qemu-test/src/slirp'
make[2]: Nothing to be done for 'all'.
---
Host machine cpu: i386
Target machine cpu family: x86
Target machine cpu: i386
../src/meson.build:10: WARNING: Module unstable-keyval has no backwards or forwards compatibility and might not exist in future releases.
Program sh found: YES
Program python3 found: YES (/usr/bin/python3)
Configuring ninjatool using configuration
---
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sudo', '-n', 'docker', 'run', '--rm', '--label', 'com.qemu.instance.uuid=a0b2da4b764d490c9a93a880eb229f51', '-u', '1001', '--security-opt', 'seccomp=unconfined', '-e', 'TARGET_LIST=', '-e', 'EXTRA_CONFIGURE_OPTS=', '-e', 'V=', '-e', 'J=14', '-e', 'DEBUG=', '-e', 'SHOW_ENV=', '-e', 'CCACHE_DIR=/var/tmp/ccache', '-v', '/home/patchew/.cache/qemu-docker-ccache:/var/tmp/ccache:z', '-v', '/var/tmp/patchew-tester-tmp-gg_7lmo7/src/docker-src.2020-09-25-02.32.30.24351:/var/tmp/qemu:z,ro', 'qemu/fedora', '/var/tmp/qemu/run', 'test-mingw']' returned non-zero exit status 2. filter=--filter=label=com.qemu.instance.uuid=a0b2da4b764d490c9a93a880eb229f51
make[1]: *** [docker-run] Error 1
make[1]: Leaving directory `/var/tmp/patchew-tester-tmp-gg_7lmo7/src'
make: *** [docker-run-test-mingw@fedora] Error 2
real 11m5.824s
user 0m19.716s
The full log is available at
http://patchew.org/logs/[email protected]/testing.docker-mingw@fedora/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to [email protected]
Quoted 5 lines · Maxim Cournoyer
Take advantage of the Sphinx texinfo backend to generate a QEMU info manual. The texinfo format allows for more structure and info readers provide more advanced navigation capabilities compared to manpages readers.
Not providing an info manual (or indeed any format other than HTML and manpages) was a deliberate design choice. The rationale is that checking that multiple document formats all ended up rendering correctly is more work than people will in practice put in (as demonstrated by various errors in the old HTML rendering, for instance).
That said, maybe other people will disagree with me.
Quoted 4 lines
* configure (infodir): Add the --infodir option, which allows configuring the directory under which the info manuals are installed.
*docs/index.rst: Include the top level documents to prevent warnings (treated as errors by sphinx-build).
This isn't the right thing. You should be pointing sphinx-build at each of the individual manuals (system, interop, etc) and generating one info file for each. This is because we generate manuals for each of these including the 'devel' manual, but we don't want to install 'devel', because it's developer-facing and not needed by end-users of QEMU. If you do that then you won't find you have problems with the orphan top level documents.
(We really need to move those orphan docs into the right places in the manual structure at some point.)
Quoted 1 line
* docs/meson.build (sphinxinfo): Add new target.
You've forgotten your Signed-off-by: line.
thanks
-- PMM
Hello Peter,
Quoted 7 lines · Peter Maydell
Quoted 5 lines · Maxim Cournoyer
Take advantage of the Sphinx texinfo backend to generate a QEMU info manual. The texinfo format allows for more structure and info readers provide more advanced navigation capabilities compared to manpages readers.
Quoted 6 lines
Not providing an info manual (or indeed any format other than HTML and manpages) was a deliberate design choice. The rationale is that checking that multiple document formats all ended up rendering correctly is more work than people will in practice put in (as demonstrated by various errors in the old HTML rendering, for instance).
It seems to me that any problem found in the texinfo rendering produced by Sphinx would be a bug in Sphinx, which could simply be forwarded upstream when encountered. My experience with Sphinx-generated texinfo manuals is rather good. The CMake project allows producing an info manual that way, for example, and the result is at least as usable as their HTML-based doc (as an avid info user, I'd say more usable, given most commands are indexed).
Quoted 11 lines
Quoted 4 lines
* configure (infodir): Add the --infodir option, which allows configuring the directory under which the info manuals are installed.
*docs/index.rst: Include the top level documents to prevent warnings (treated as errors by sphinx-build).This isn't the right thing. You should be pointing sphinx-build at each of the individual manuals (system, interop, etc) and generating one info file for each. This is because we generate manuals for each of these including the 'devel' manual, but we don't want to install 'devel', because it's developer-facing and not needed by end-users of QEMU.
Is this the only reason individual manuals are being generated? It makes sense for the manpages (which have their own macros), but otherwise (for HTML and info) introduces a lot of complexity for not much gain, in my opinion. Users not wanting to know about devel internals can simply skip that section; no harm done.
The individual manuals are all stitched back together using a top index.html page (derived from index.html.in), anyway. That'd be better taken care of by Sphinx automatically upon generating the whole doc tree, in my opinion, as it seems to be the way it was designed to work and simplifies things all around.
I initially went the individual manual route and devised complicated Make macros and got individual info manuals produced, only to find out I wanted to replicate that index.html.in in the texinfo format (Texinfo is as navigable as HTML, so having every QEMU manual under the same node as different sections makes more sense in my opinion). The individually produced info manual were also all named 'QEMU Documentation' rather than with their own more accurate names, because of directives in the included the top-level conf.py.
Quoted 4 lines
won't find you have problems with the orphan top level documents.
(We really need to move those orphan docs into the right places in the manual structure at some point.)
We could simply put them in another section called "Others" as a starting point. For this patch we can also choose to ignore them rather than include them, but if someone went to the length of writing those docs, I'd argue it was to be read rather than forgotten :-).
Quoted 3 lines
Quoted 1 line
* docs/meson.build (sphinxinfo): Add new target.
You've forgotten your Signed-off-by: line.
I've added it in my local tree; will wait for your feedback on my above comments before sending an updated patch.
Thanks for the review,
Maxim
Quoted 7 lines · Maxim Cournoyer
Quoted 6 lines
This isn't the right thing. You should be pointing sphinx-build at each of the individual manuals (system, interop, etc) and generating one info file for each. This is because we generate manuals for each of these including the 'devel' manual, but we don't want to install 'devel', because it's developer-facing and not needed by end-users of QEMU.
Quoted 5 lines
Is this the only reason individual manuals are being generated? It makes sense for the manpages (which have their own macros), but otherwise (for HTML and info) introduces a lot of complexity for not much gain, in my opinion. Users not wanting to know about devel internals can simply skip that section; no harm done.
It is the best way we found for getting Sphinx to do what we wanted. I agree that it would be nicer to have one manual with all the user facing parts in it, but it is apparently not possible to do that without shipping the devel docs to users, which we didn't want to do.
thanks
-- PMM
Hello Peter,
Quoted 14 lines · Peter Maydell
Quoted 7 lines · Maxim Cournoyer
Quoted 6 lines
This isn't the right thing. You should be pointing sphinx-build at each of the individual manuals (system, interop, etc) and generating one info file for each. This is because we generate manuals for each of these including the 'devel' manual, but we don't want to install 'devel', because it's developer-facing and not needed by end-users of QEMU.
Quoted 5 lines
Is this the only reason individual manuals are being generated? It makes sense for the manpages (which have their own macros), but otherwise (for HTML and info) introduces a lot of complexity for not much gain, in my opinion. Users not wanting to know about devel internals can simply skip that section; no harm done.
Quoted 4 lines
It is the best way we found for getting Sphinx to do what we wanted. I agree that it would be nicer to have one manual with all the user facing parts in it, but it is apparently not possible to do that without shipping the devel docs to users, which we didn't want to do.
I personally don't understand the rationale of hiding the devel section from users, especially given the kind of users QEMU is likely to attract (e.g, teksavvy people, perhaps themselves developers that could be curious peeking into that section to deepen their understanding of QEMU's architecture and internals).
But if you feel strongly about it, I'd suggest the following, which excludes the devel section from being built when calling Sphinx from the top level:
--8<---------------cut here---------------start------------->8---
7 files changed, 13 insertions(+), 4 deletions(-) docs/conf.py | 5 ++++- docs/cpu-hotplug.rst | 2 ++ docs/index.rst | 3 --- docs/microvm.rst | 2 ++ docs/pr-manager.rst | 2 ++ docs/virtio-net-failover.rst | 2 ++ docs/virtio-pmem.rst | 1 +
modified docs/conf.py
@@ -116,7 +116,10 @@ language = None
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files. # This patterns also effect to html_static_path and html_extra_path -exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] +exclude_patterns = ['_build', + 'Thumbs.db', + '.DS_Store', + 'devel/*']
# The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx'
modified docs/cpu-hotplug.rst
@@ -1,3 +1,5 @@
+:orphan:
+
===================
Virtual CPU hotplug
===================
modified docs/index.rst
@@ -9,12 +9,9 @@ Welcome to QEMU's documentation!
.. toctree::
:maxdepth: 2
:caption: Contents:
- :glob:
system/index
user/index
tools/index
interop/index
specs/index
- devel/index
- *
modified docs/microvm.rst
@@ -1,3 +1,5 @@
+:orphan:
+
====================
microvm Machine Type
====================
modified docs/pr-manager.rst
@@ -1,3 +1,5 @@
+:orphan:
+
======================================
Persistent reservation managers
======================================
modified docs/virtio-net-failover.rst
@@ -1,3 +1,5 @@
+:orphan:
+
======================================
QEMU virtio-net standby (net_failover)
======================================
modified docs/virtio-pmem.rst
@@ -1,3 +1,4 @@
+:orphan:
========================
QEMU virtio pmem
--8<---------------cut here---------------end--------------->8---
With this change, the top level node of "info qemu" reads as:
--8<---------------cut here---------------start------------->8--- QEMU Documentation
******************
QEMU , Sep 26, 2020
The QEMU Project Developers
Copyright (C) 2020, The QEMU Project Developers
* Menu:
* QEMU System Emulation User's Guide::
* QEMU User Mode Emulation User's Guide::
* QEMU Tools Guide::
* QEMU System Emulation Management and Interoperability Guide::
* QEMU System Emulation Guest Hardware Specifications::
* Index::
--8<---------------cut here---------------end--------------->8---
I'd still prefer to include the developer's manual as the last section of the info manual, if you don't mind, for the reason explained above.
The scheme used above to exclude the devel section could also be used to simplify building the HTML manual (e.g., build it by calling sphinx on the top level with a correctly defined index.rst instead of individually for each of its constituting sections before stitching them back together with a custom index.html).
What do you think?
Thanks again,
Maxim
Quoted 11 lines · Maxim Cournoyer
Quoted 4 lines · Peter Maydell
It is the best way we found for getting Sphinx to do what we wanted. I agree that it would be nicer to have one manual with all the user facing parts in it, but it is apparently not possible to do that without shipping the devel docs to users, which we didn't want to do.
I personally don't understand the rationale of hiding the devel section from users, especially given the kind of users QEMU is likely to attract (e.g, teksavvy people, perhaps themselves developers that could be curious peeking into that section to deepen their understanding of QEMU's architecture and internals).
Mostly I think we came to this opinion because
(a) it was how we handled developer docs before -- they tended to be standalone files in docs/ somewhere, not part of the old shipped-to-user Texinfo docs
(b) internals docs are much more likely to become quickly outdated:
you almost always want to be looking at the docs for current-git, not for some older distro-installed QEMU version
(c) sure, some users might want to look at QEMU internals docs, but they are definitely going to be the minority
(d) the developer docs are rougher in quality overall
(e) you need the source tree anyway if you're interested in the internals, because so much is not documented, or not in the rST manuals
That said, we are kind of working against the grain of how Sphinx wants to be used here, which is usually not a great idea, and it does result in some awkwardnesses (it would be nice to have the devel docs on the qemu.org website, for instance). I asked around on IRC and nobody seemed to be very strongly against moving to the just-one-manual setup. So maybe we should do that.
thanks
-- PMM
Hello Peter,
Quoted 1 line · Peter Maydell
On Sun, 27 Sep 2020 at 03:21, Maxim Cournoyer <[email protected]> wrote:
[...]
Quoted 20 lines
Quoted 5 lines
I personally don't understand the rationale of hiding the devel section from users, especially given the kind of users QEMU is likely to attract (e.g, teksavvy people, perhaps themselves developers that could be curious peeking into that section to deepen their understanding of QEMU's architecture and internals).
Mostly I think we came to this opinion because
(a) it was how we handled developer docs before -- they tended to be standalone files in docs/ somewhere, not part of the old shipped-to-user Texinfo docs
(b) internals docs are much more likely to become quickly outdated:
you almost always want to be looking at the docs for current-git, not for some older distro-installed QEMU version
(c) sure, some users might want to look at QEMU internals docs, but they are definitely going to be the minority
(d) the developer docs are rougher in quality overall
(e) you need the source tree anyway if you're interested in the internals, because so much is not documented, or not in the rST manuals
Quoted 7 lines
That said, we are kind of working against the grain of how Sphinx wants to be used here, which is usually not a great idea, and it does result in some awkwardnesses (it would be nice to have the devel docs on the qemu.org website, for instance). I asked around on IRC and nobody seemed to be very strongly against moving to the just-one-manual setup. So maybe we should do that.
OK. Thanks for asking others if they had an opinion about it. Given they don't seem to feel strongly about it, I suggest:
1) Keep the developer section as the last section of the overall doc table of content (as was done in my original patch).
2) Mark the top level couple orphaned .rst as "orphaned" as proposed in one of my previous reply, since this is what they are. They will get built, but won't appear anywhere in the QEMU documentation.
If that sounds reasonable, I'll send an updated patch soon.
Another thing; there was some CI failures with the origin patches I sent [0]; are they worth investigating or are they unrelated, "known" failures?
Thanks,
Maxim