docs/python: bump minimum Sphinx version
cover docs/python: bump minimum Sphinx version
With recent deprecations, we can advance our minimum sphinx version safely. This is heavily motivated by new qapidoc work which is much easier to maintain cross-version compatibility for - see difficulties in our dbus documentation which only works on sphinx >= 4.
We can only guarantee >= 3.4.3 now, but that's still vastly easier than maintaining compatibility all the way back to 1.x.
GitLab: https://gitlab.com/jsnow/qemu/-/pipelines/1357902509
(failures appear to be unrelated to the series.)
John Snow (2):
Python: bump minimum sphinx version to 3.4.3
docs: remove Sphinx 1.x compatibility code
docs/conf.py | 7 +++---- docs/sphinx/hxtool.py | 21 ++++----------------- docs/sphinx/kerneldoc.py | 38 ++++++++++++-------------------------- docs/sphinx/kernellog.py | 28 ---------------------------- docs/sphinx/qapidoc.py | 29 +++-------------------------- pythondeps.toml | 2 +- 6 files changed, 23 insertions(+), 102 deletions(-) delete mode 100644 docs/sphinx/kernellog.py
Signature · 1 line
2.45.0
PATCH 1/2 Python: bump minimum sphinx version to 3.4.3
With RHEL 8 support retired (It's been two years today since RHEL 9 came out), our very oldest build platform version of Sphinx is now 3.4.3; and keeping backwards compatibility for versions as old as v1.6 when using domain extensions is a lot of work we don't need to do.
This patch is motivated by my work creating a new QAPI domain, which unlike the dbus documentation, cannot be allowed to regress by creating a "dummy" doc. Easier is to improve our minimum version as far as we can push it forwards, reducing my burden in creating cross-compatibility hacks and patches.
A sampling of sphinx versions from various distributions, courtesy
https://repology.org/project/python:sphinx/versions
Alpine 3.16: v4.3.0 (QEMU support ended 2024-05-23) Alpine 3.17: v5.3.0
Alpine 3.18: v6.1.3
Alpine 3.19: v6.2.1
Ubuntu 20.04 LTS: EOL
Ubuntu 22.04 LTS: v4.3.2
Ubuntu 22.10: EOL
Ubuntu 23.04: EOL
Ubuntu 23.10: v5.3.0
Ubuntu 24.04 LTS: v7.2.6
Debian 11: v3.4.3 (QEMU support ends 2024-07-xx) Debian 12: v5.3.0
Fedora 38: EOL
Fedora 39: v6.2.1
Fedora 40: v7.2.6
CentOS Stream 8: v1.7.6 (QEMU support ended 2024-05-17) CentOS Stream 9: v3.4.3
OpenSUSE Leap 15.4: EOL
OpenSUSE Leap 15.5: 2.3.1, 4.2.0 and 7.2.6
--- docs/conf.py | 7 +++---- docs/sphinx/qapidoc.py | 29 +++-------------------------- pythondeps.toml | 2 +- 3 files changed, 7 insertions(+), 31 deletions(-)
docs/conf.py +3 -4
@@ -53,10 +53,9 @@ # If your documentation needs a minimal Sphinx version, state it here. # -# Sphinx 1.5 and earlier can't build our docs because they are too -# picky about the syntax of the argument to the option:: directive -# (see Sphinx bugs #646, #3366). -needs_sphinx = '1.6' +# 3.4.3 is the oldest version of Sphinx that ships on a platform we +# pledge build support for. +needs_sphinx = '3.4.3' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
docs/sphinx/qapidoc.py +3 -26
@@ -31,6 +31,7 @@ from docutils.statemachine import ViewList from docutils.parsers.rst import directives, Directive from sphinx.errors import ExtensionError +from sphinx.util.docutils import switch_source_input from sphinx.util.nodes import nested_parse_with_titles import sphinx from qapi.gen import QAPISchemaVisitor
@@ -38,15 +39,6 @@ from qapi.schema import QAPISchema -# Sphinx up to 1.6 uses AutodocReporter; 1.7 and later -# use switch_source_input. Check borrowed from kerneldoc.py. -Use_SSI = sphinx.__version__[:3] >= '1.7' -if Use_SSI: - from sphinx.util.docutils import switch_source_input -else: - from sphinx.ext.autodoc import AutodocReporter - - __version__ = '1.0'
@@ -517,23 +509,8 @@ def do_parse(self, rstlist, node): subheadings (titles) without confusing the rendering of anything else. """ - # This is from kerneldoc.py -- it works around an API change in - # Sphinx between 1.6 and 1.7. Unlike kerneldoc.py, we use - # sphinx.util.nodes.nested_parse_with_titles() rather than the - # plain self.state.nested_parse(), and so we can drop the saving - # of title_styles and section_level that kerneldoc.py does, - # because nested_parse_with_titles() does that for us. - if Use_SSI: - with switch_source_input(self.state, rstlist): - nested_parse_with_titles(self.state, rstlist, node) - else: - save = self.state.memo.reporter - self.state.memo.reporter = AutodocReporter( - rstlist, self.state.memo.reporter) - try: - nested_parse_with_titles(self.state, rstlist, node) - finally: - self.state.memo.reporter = save + with switch_source_input(self.state, rstlist): + nested_parse_with_titles(self.state, rstlist, node) def setup(app):
pythondeps.toml +1 -1
@@ -23,7 +23,7 @@ meson = { accepted = ">=0.63.0", installed = "1.2.3", canary = "meson" } [docs] # Please keep the installed versions in sync with docs/requirements.txt -sphinx = { accepted = ">=1.6", installed = "5.3.0", canary = "sphinx-build" } +sphinx = { accepted = ">=3.4.3", installed = "5.3.0", canary = "sphinx-build" } sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.1.1" } [avocado]
Signature · 1 line
2.45.0
Re: [PATCH 1/2] Python: bump minimum sphinx version to 3.4.3
Quoted 4 lines · John Snow
With RHEL 8 support retired (It's been two years today since RHEL 9 came out), our very oldest build platform version of Sphinx is now 3.4.3; and keeping backwards compatibility for versions as old as v1.6 when using domain extensions is a lot of work we don't need to do.
Technically that's unrelated: thanks to your venv work, :) builds on RHEL 8 / CentOS Stream 8 do not pick the platform Sphinx, because it runs under Python 3.6. Therefore the version included in RHEL 8 does not matter for picking the minimum supported Sphinx version.
Quoted 1 line
Debian 11: v3.4.3 (QEMU support ends 2024-07-xx)
Nice. :)
pythondeps.toml meson = { accepted = ">=0.63.0", installed = "1.2.3", canary = "meson" } · line 29
Quoted 13 lines
pythondeps.toml +1
-1@@ -23,7 +23,7 @@ meson = { accepted = ">=0.63.0", installed = "1.2.3", canary = "meson" } [docs] # Please keep the installed versions in sync with docs/requirements.txt -sphinx = { accepted = ">=1.6", installed = "5.3.0", canary = "sphinx-build" } +sphinx = { accepted = ">=3.4.3", installed = "5.3.0", canary = "sphinx-build" } sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.1.1" } [avocado]
Paolo
Re: [PATCH 1/2] Python: bump minimum sphinx version to 3.4.3
Quoted 11 lines · Paolo Bonzini
Quoted 4 lines · John Snow
With RHEL 8 support retired (It's been two years today since RHEL 9 came out), our very oldest build platform version of Sphinx is now 3.4.3; and keeping backwards compatibility for versions as old as v1.6 when using domain extensions is a lot of work we don't need to do.
Technically that's unrelated: thanks to your venv work, :) builds on RHEL 8 / CentOS Stream 8 do not pick the platform Sphinx, because it runs under Python 3.6. Therefore the version included in RHEL 8 does not matter for picking the minimum supported Sphinx version.
...!
I think I can't mandate 4.x because of RHEL 9 builds though, and offline requirements.
Quoted 26 lines
Quoted 1 line
Debian 11: v3.4.3 (QEMU support ends 2024-07-xx)
Nice. :)
pythondeps.toml meson = { accepted = ">=0.63.0", installed = "1.2.3",
Quoted 5 lines
pythondeps.toml +0
-0@@ -23,7 +23,7 @@ meson = { accepted = ">=0.63.0", installed = "1.2.3",canary = "meson" }
pythondeps.toml meson = { accepted = ">=0.63.0", installed = "1.2.3", canary = "meson" }
Quoted 4 lines
pythondeps.toml +0
-1[docs] # Please keep the installed versions in sync with docs/requirements.txt -sphinx = { accepted = ">=1.6", installed = "5.3.0", canary ="sphinx-build" }
pythondeps.toml meson = { accepted = ">=0.63.0", installed = "1.2.3", · line 25
Quoted 1 line
pythondeps.toml +1
-0+sphinx = { accepted = ">=3.4.3", installed = "5.3.0", canary ="sphinx-build" }
pythondeps.toml meson = { accepted = ">=0.63.0", installed = "1.2.3", canary = "meson" } · line 29
Quoted 3 lines
pythondeps.toml +0
-0sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.1.1" } [avocado]Acked-by Paolo Bonzini <[email protected]>Paolo
Re: [PATCH 1/2] Python: bump minimum sphinx version to 3.4.3
Quoted 7 lines · John Snow
Quoted 6 lines · Paolo Bonzini
Quoted 4 lines · John Snow
With RHEL 8 support retired (It's been two years today since RHEL 9 came out), our very oldest build platform version of Sphinx is now 3.4.3; and keeping backwards compatibility for versions as old as v1.6 when using domain extensions is a lot of work we don't need to do.
Quoted 6 lines
Quoted 4 lines
Technically that's unrelated: thanks to your venv work, :) builds on RHEL 8 / CentOS Stream 8 do not pick the platform Sphinx, because it runs under Python 3.6. Therefore the version included in RHEL 8 does not matter for picking the minimum supported Sphinx version.
I think I can't mandate 4.x because of RHEL 9 builds though, and offline requirements.
Offline requirements are not a problem; on RHEL 8 you just have to install with pip in order to build docs offline. But yes, RHEL 9 is still using platform Python and therefore 3.4.3 remains the limit even after we stop supporting bullseye.
Paolo
Re: [PATCH 1/2] Python: bump minimum sphinx version to 3.4.3
Quoted 13 lines · Paolo Bonzini
Quoted 12 lines · John Snow
Quoted 10 lines · Paolo Bonzini
Quoted 4 lines · John Snow
With RHEL 8 support retired (It's been two years today since RHEL 9 came out), our very oldest build platform version of Sphinx is now 3.4.3; and keeping backwards compatibility for versions as old as v1.6 when using domain extensions is a lot of work we don't need to do.
Technically that's unrelated: thanks to your venv work, :) builds on RHEL 8 / CentOS Stream 8 do not pick the platform Sphinx, because it runs under Python 3.6. Therefore the version included in RHEL 8 does not matter for picking the minimum supported Sphinx version.
Quoted 8 lines
Quoted 1 line
I think I can't mandate 4.x because of RHEL 9 builds though, and offline
requirements.
Offline requirements are not a problem; on RHEL 8 you just have to install with pip in order to build docs offline. But yes, RHEL 9 is still using platform Python and therefore 3.4.3 remains the limit even after we stop supporting bullseye.
To be clear I mean offline, isolated RPM builds under RHEL9 where I don't think we can utilize PyPI at all; and vendoring Sphinx is I think not a practical option due to the number of dependencies and non-pure Python deps.
It's not a problem for developer workflow, just downstream packaging.
Luckily OpenSUSE offers newer Sphinx, but RHEL doesn't yet. Maybe that can be rectified eventually - possibly after 3.8 is EOL and there is increased demand for newer Python packages to be made available in RHEL... but not yet, today.
Quoted 4 lines
Paolo
Thanks for the ack O:-)
Re: [PATCH 1/2] Python: bump minimum sphinx version to 3.4.3
Quoted 3 lines · John Snow
To be clear I mean offline, isolated RPM builds under RHEL9 where I don't think we can utilize PyPI at all; and vendoring Sphinx is I think not a practical option due to the number of dependencies and non-pure Python deps.
It's not a problem for developer workflow, just downstream packaging.
Agreed, switching to PyPI packages is not something that should be done lightly---only after the platform Python is EOL, and only if there are benefits for QEMU to require 3.9 (like there were for 3.7).
In that case, downstream could still package QEMU but they'd have to build python39-sphinx or something like that.
Paolo
Quoted 8 lines
Luckily OpenSUSE offers newer Sphinx, but RHEL doesn't yet. Maybe that can be rectified eventually - possibly after 3.8 is EOL and there is increased demand for newer Python packages to be made available in RHEL... but not yet, today.
Quoted 4 lines
Paolo
Thanks for the ack O:-)
PATCH 2/2 docs: remove Sphinx 1.x compatibility code
In reply to John SnowIn general, the Use_SSI workaround is no longer needed, and neither is the pre-1.6 logging shim for kerneldoc.
--- docs/sphinx/hxtool.py | 21 ++++----------------- docs/sphinx/kerneldoc.py | 38 ++++++++++++-------------------------- docs/sphinx/kernellog.py | 28 ---------------------------- 3 files changed, 16 insertions(+), 71 deletions(-) delete mode 100644 docs/sphinx/kernellog.py
docs/sphinx/hxtool.py +4 -17
@@ -24,16 +24,10 @@ from docutils.statemachine import ViewList from docutils.parsers.rst import directives, Directive from sphinx.errors import ExtensionError +from sphinx.util.docutils import switch_source_input from sphinx.util.nodes import nested_parse_with_titles import sphinx -# Sphinx up to 1.6 uses AutodocReporter; 1.7 and later -# use switch_source_input. Check borrowed from kerneldoc.py. -Use_SSI = sphinx.__version__[:3] >= '1.7' -if Use_SSI: - from sphinx.util.docutils import switch_source_input -else: - from sphinx.ext.autodoc import AutodocReporter __version__ = '1.0'
@@ -185,16 +179,9 @@ def run(self): # of title_styles and section_level that kerneldoc.py does, # because nested_parse_with_titles() does that for us. def do_parse(self, result, node): - if Use_SSI: - with switch_source_input(self.state, result): - nested_parse_with_titles(self.state, result, node) - else: - save = self.state.memo.reporter - self.state.memo.reporter = AutodocReporter(result, self.state.memo.reporter) - try: - nested_parse_with_titles(self.state, result, node) - finally: - self.state.memo.reporter = save + with switch_source_input(self.state, result): + nested_parse_with_titles(self.state, result, node) + def setup(app): """ Register hxtool-doc directive with Sphinx"""
docs/sphinx/kerneldoc.py +12 -26
@@ -38,20 +38,14 @@ from docutils.statemachine import ViewList from docutils.parsers.rst import directives, Directive -# -# AutodocReporter is only good up to Sphinx 1.7 -# import sphinx +from sphinx.util import logging +from sphinx.util.docutils import switch_source_input -Use_SSI = sphinx.__version__[:3] >= '1.7' -if Use_SSI: - from sphinx.util.docutils import switch_source_input -else: - from sphinx.ext.autodoc import AutodocReporter - -import kernellog __version__ = '1.0' +logger = logging.getLogger('kerneldoc') + class KernelDocDirective(Directive): """Extract kernel-doc comments from the specified file"""
@@ -111,8 +105,7 @@ def run(self): cmd += [filename] try: - kernellog.verbose(env.app, - 'calling kernel-doc \'%s\'' % (" ".join(cmd))) + logger.verbose('calling kernel-doc \'%s\'' % (" ".join(cmd))) p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) out, err = p.communicate()
@@ -122,8 +115,10 @@ def run(self): if p.returncode != 0: sys.stderr.write(err) - kernellog.warn(env.app, - 'kernel-doc \'%s\' failed with return code %d' % (" ".join(cmd), p.returncode)) + logger.warning( + 'kernel-doc \'%s\' failed with return code %d' % + (" ".join(cmd), p.returncode) + ) return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))] elif env.config.kerneldoc_verbosity > 0: sys.stderr.write(err)
@@ -149,22 +144,13 @@ def run(self): return node.children except Exception as e: # pylint: disable=W0703 - kernellog.warn(env.app, 'kernel-doc \'%s\' processing failed with: %s' % + logger.warning('kernel-doc \'%s\' processing failed with: %s' % (" ".join(cmd), str(e))) return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))] def do_parse(self, result, node): - if Use_SSI: - with switch_source_input(self.state, result): - self.state.nested_parse(result, 0, node, match_titles=1) - else: - save = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter - self.state.memo.reporter = AutodocReporter(result, self.state.memo.reporter) - self.state.memo.title_styles, self.state.memo.section_level = [], 0 - try: - self.state.nested_parse(result, 0, node, match_titles=1) - finally: - self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = save + with switch_source_input(self.state, result): + self.state.nested_parse(result, 0, node, match_titles=1) def setup(app):
docs/sphinx/kernellog.py deleted +0 -28
@@ -1,28 +0,0 @@ -# SPDX-License-Identifier: GPL-2.0 -# -# Sphinx has deprecated its older logging interface, but the replacement -# only goes back to 1.6. So here's a wrapper layer to keep around for -# as long as we support 1.4. -# -import sphinx - -if sphinx.__version__[:3] >= '1.6': - UseLogging = True - from sphinx.util import logging - logger = logging.getLogger('kerneldoc') -else: - UseLogging = False - -def warn(app, message): - if UseLogging: - logger.warning(message) - else: - app.warn(message) - -def verbose(app, message): - if UseLogging: - logger.verbose(message) - else: - app.verbose(message) - -
Signature · 1 line
2.45.0
Quoted 25 lines · John Snow
With recent deprecations, we can advance our minimum sphinx version safely. This is heavily motivated by new qapidoc work which is much easier to maintain cross-version compatibility for - see difficulties in our dbus documentation which only works on sphinx >= 4.
We can only guarantee >= 3.4.3 now, but that's still vastly easier than maintaining compatibility all the way back to 1.x.
GitLab: https://gitlab.com/jsnow/qemu/-/pipelines/1357902509
(failures appear to be unrelated to the series.)
John Snow (2):
Python: bump minimum sphinx version to 3.4.3
docs: remove Sphinx 1.x compatibility codedocs/conf.py | 7 +++---- docs/sphinx/hxtool.py | 21 ++++----------------- docs/sphinx/kerneldoc.py | 38 ++++++++++++-------------------------- docs/sphinx/kernellog.py | 28 ---------------------------- docs/sphinx/qapidoc.py | 29 +++-------------------------- pythondeps.toml | 2 +- 6 files changed, 23 insertions(+), 102 deletions(-) delete mode 100644 docs/sphinx/kernellog.py
Quoted 3 lines
--
2.45.0
Bleurgh. I meant to shuffle some of the changes in patch 1 into patch 2, I'll fix that on re-spin. If you want to review it anyway, just presume I'll fix that next go-around.
--js
Markus Armbruster
Acked-byQuoted 27 lines · John Snow
Quoted 25 lines · John Snow
With recent deprecations, we can advance our minimum sphinx version safely. This is heavily motivated by new qapidoc work which is much easier to maintain cross-version compatibility for - see difficulties in our dbus documentation which only works on sphinx >= 4.
We can only guarantee >= 3.4.3 now, but that's still vastly easier than maintaining compatibility all the way back to 1.x.
GitLab: https://gitlab.com/jsnow/qemu/-/pipelines/1357902509
(failures appear to be unrelated to the series.)
John Snow (2):
Python: bump minimum sphinx version to 3.4.3
docs: remove Sphinx 1.x compatibility codedocs/conf.py | 7 +++---- docs/sphinx/hxtool.py | 21 ++++----------------- docs/sphinx/kerneldoc.py | 38 ++++++++++++-------------------------- docs/sphinx/kernellog.py | 28 ---------------------------- docs/sphinx/qapidoc.py | 29 +++-------------------------- pythondeps.toml | 2 +- 6 files changed, 23 insertions(+), 102 deletions(-) delete mode 100644 docs/sphinx/kernellog.py
Quoted 5 lines
Quoted 3 lines
--
2.45.0Bleurgh. I meant to shuffle some of the changes in patch 1 into patch 2,
I was wondering about the split :)
Quoted 2 lines
I'll fix that on re-spin. If you want to review it anyway, just presume I'll fix that next go-around.