Skip to contentSkip to conversation

docs/python: bump minimum Sphinx version

PATCH In review John Snow 2 participants 6 messages Jul 3, 2024 – Jul 5, 2024

lore
Share & tools
lore

John Snow

author
#0 lore raw

v2 cover docs/python: bump minimum Sphinx version

With recent deprecations, we can advance our minimum sphinx version safely. This is heavily motivated by new qapidoc work which is much easier to maintain cross-version compatibility for - see difficulties in our dbus documentation which only works on sphinx >= 4.

We can only guarantee >= 3.4.3 now, but that's still vastly easier than maintaining compatibility all the way back to 1.x.

https://gitlab.com/jsnow/qemu/-/pipelines/1359549621

(failures appear to be unrelated to the series.)

V2: refactor all code deletions into patch 2

John Snow (2):
Python: bump minimum sphinx version to 3.4.3
docs: remove Sphinx 1.x compatibility code

 docs/conf.py             |  7 +++----
 docs/sphinx/hxtool.py    | 21 ++++-----------------
 docs/sphinx/kerneldoc.py | 38 ++++++++++++--------------------------
 docs/sphinx/kernellog.py | 28 ----------------------------
 docs/sphinx/qapidoc.py   | 29 +++--------------------------
 pythondeps.toml          |  2 +-
 6 files changed, 23 insertions(+), 102 deletions(-)
 delete mode 100644 docs/sphinx/kernellog.py
Signature · 1 line
2.45.0
author
#1 lore raw

PATCH v2 1/2 Python: bump minimum sphinx version to 3.4.3

With RHEL 8 support retired (It's been two years since RHEL9 released), our very oldest build platform version of Sphinx is now 3.4.3; and keeping backwards compatibility for versions as old as v1.6 when using domain extensions is a lot of work we don't need to do.

This patch is motivated by my work creating a new QAPI domain, which unlike the dbus documentation, cannot be allowed to regress by creating a "dummy" doc when operating under older sphinx versions. Easier is to raise our minimum version as far as we can push it forwards, reducing my burden in creating cross-compatibility hacks and patches.

A sampling of sphinx versions from various distributions, courtesy
https://repology.org/project/python:sphinx/versions

Alpine 3.16: v4.3.0 (QEMU support ended 2024-05-23) Alpine 3.17: v5.3.0
Alpine 3.18: v6.1.3
Alpine 3.19: v6.2.1
Ubuntu 20.04 LTS: EOL
Ubuntu 22.04 LTS: v4.3.2
Ubuntu 22.10: EOL
Ubuntu 23.04: EOL
Ubuntu 23.10: v5.3.0
Ubuntu 24.04 LTS: v7.2.6
Debian 11: v3.4.3 (QEMU support ends 2024-07-xx) Debian 12: v5.3.0
Fedora 38: EOL
Fedora 39: v6.2.1
Fedora 40: v7.2.6
CentOS Stream 8: v1.7.6 (QEMU support ended 2024-05-17) CentOS Stream 9: v3.4.3
OpenSUSE Leap 15.4: EOL
OpenSUSE Leap 15.5: 2.3.1, 4.2.0 and 7.2.6

RHEL9 / CentOS Stream 9 becomes the new defining factor in staying at Sphinx 3.4.3 due to downstream offline build requirements that force us to use platform Sphinx instead of newer packages from PyPI.

---
 docs/conf.py    | 7 +++----
 pythondeps.toml | 2 +-
 2 files changed, 4 insertions(+), 5 deletions(-)
docs/conf.py +3 -4
index aae0304ac6e..876f6768815 100644
@@ -53,10 +53,9 @@
 
 # If your documentation needs a minimal Sphinx version, state it here.
 #
-# Sphinx 1.5 and earlier can't build our docs because they are too
-# picky about the syntax of the argument to the option:: directive
-# (see Sphinx bugs #646, #3366).
-needs_sphinx = '1.6'
+# 3.4.3 is the oldest version of Sphinx that ships on a platform we
+# pledge build support for.
+needs_sphinx = '3.4.3'
 
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
pythondeps.toml +1 -1
index 9c16602d303..bc656376caa 100644
@@ -23,7 +23,7 @@ meson = { accepted = ">=0.63.0", installed = "1.2.3", canary = "meson" }
 
 [docs]
 # Please keep the installed versions in sync with docs/requirements.txt
-sphinx = { accepted = ">=1.6", installed = "5.3.0", canary = "sphinx-build" }
+sphinx = { accepted = ">=3.4.3", installed = "5.3.0", canary = "sphinx-build" }
 sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.1.1" }
 
 [avocado]
Signature · 1 line
2.45.0
Reviewed-by on v2 1/2
#2 lore raw

Re: [PATCH v2 1/2] Python: bump minimum sphinx version to 3.4.3

Quoted 38 lines · John Snow

With RHEL 8 support retired (It's been two years since RHEL9 released), our very oldest build platform version of Sphinx is now 3.4.3; and keeping backwards compatibility for versions as old as v1.6 when using domain extensions is a lot of work we don't need to do.

This patch is motivated by my work creating a new QAPI domain, which unlike the dbus documentation, cannot be allowed to regress by creating a "dummy" doc when operating under older sphinx versions. Easier is to raise our minimum version as far as we can push it forwards, reducing my burden in creating cross-compatibility hacks and patches.

A sampling of sphinx versions from various distributions, courtesy
https://repology.org/project/python:sphinx/versions

Alpine 3.16: v4.3.0 (QEMU support ended 2024-05-23) Alpine 3.17: v5.3.0
Alpine 3.18: v6.1.3
Alpine 3.19: v6.2.1
Ubuntu 20.04 LTS: EOL
Ubuntu 22.04 LTS: v4.3.2
Ubuntu 22.10: EOL
Ubuntu 23.04: EOL
Ubuntu 23.10: v5.3.0
Ubuntu 24.04 LTS: v7.2.6
Debian 11: v3.4.3 (QEMU support ends 2024-07-xx) Debian 12: v5.3.0
Fedora 38: EOL
Fedora 39: v6.2.1
Fedora 40: v7.2.6
CentOS Stream 8: v1.7.6 (QEMU support ended 2024-05-17) CentOS Stream 9: v3.4.3
OpenSUSE Leap 15.4: EOL
OpenSUSE Leap 15.5: 2.3.1, 4.2.0 and 7.2.6

RHEL9 / CentOS Stream 9 becomes the new defining factor in staying at Sphinx 3.4.3 due to downstream offline build requirements that force us to use platform Sphinx instead of newer packages from PyPI.

Quoted 7 lines
---
  docs/conf.py    | 7 +++----
  pythondeps.toml | 2 +-
  2 files changed, 4 insertions(+), 5 deletions(-)
author
#3 lore raw

PATCH v2 2/2 docs: remove Sphinx 1.x compatibility code

In reply to John Snow

In general, the Use_SSI workaround is no longer needed, and neither is the pre-1.6 logging shim for kerneldoc.

---
 docs/sphinx/hxtool.py    | 21 ++++-----------------
 docs/sphinx/kerneldoc.py | 38 ++++++++++++--------------------------
 docs/sphinx/kernellog.py | 28 ----------------------------
 docs/sphinx/qapidoc.py   | 29 +++--------------------------
 4 files changed, 19 insertions(+), 97 deletions(-)
 delete mode 100644 docs/sphinx/kernellog.py
docs/sphinx/hxtool.py +4 -17
index 3729084a36c..a84723be19e 100644
@@ -24,16 +24,10 @@
 from docutils.statemachine import ViewList
 from docutils.parsers.rst import directives, Directive
 from sphinx.errors import ExtensionError
+from sphinx.util.docutils import switch_source_input
 from sphinx.util.nodes import nested_parse_with_titles
 import sphinx
 
-# Sphinx up to 1.6 uses AutodocReporter; 1.7 and later
-# use switch_source_input. Check borrowed from kerneldoc.py.
-Use_SSI = sphinx.__version__[:3] >= '1.7'
-if Use_SSI:
-    from sphinx.util.docutils import switch_source_input
-else:
-    from sphinx.ext.autodoc import AutodocReporter
 
 __version__ = '1.0'
 
@@ -185,16 +179,9 @@ def run(self):
     # of title_styles and section_level that kerneldoc.py does,
     # because nested_parse_with_titles() does that for us.
     def do_parse(self, result, node):
-        if Use_SSI:
-            with switch_source_input(self.state, result):
-                nested_parse_with_titles(self.state, result, node)
-        else:
-            save = self.state.memo.reporter
-            self.state.memo.reporter = AutodocReporter(result, self.state.memo.reporter)
-            try:
-                nested_parse_with_titles(self.state, result, node)
-            finally:
-                self.state.memo.reporter = save
+        with switch_source_input(self.state, result):
+            nested_parse_with_titles(self.state, result, node)
+
 
 def setup(app):
     """ Register hxtool-doc directive with Sphinx"""
docs/sphinx/kerneldoc.py +12 -26
index 72c403a7379..3aa972f2e89 100644
@@ -38,20 +38,14 @@
 from docutils.statemachine import ViewList
 from docutils.parsers.rst import directives, Directive
 
-#
-# AutodocReporter is only good up to Sphinx 1.7
-#
 import sphinx
+from sphinx.util import logging
+from sphinx.util.docutils import switch_source_input
 
-Use_SSI = sphinx.__version__[:3] >= '1.7'
-if Use_SSI:
-    from sphinx.util.docutils import switch_source_input
-else:
-    from sphinx.ext.autodoc import AutodocReporter
-
-import kernellog
 
 __version__  = '1.0'
+logger = logging.getLogger('kerneldoc')
+
 
 class KernelDocDirective(Directive):
     """Extract kernel-doc comments from the specified file"""
@@ -111,8 +105,7 @@ def run(self):
         cmd += [filename]
 
         try:
-            kernellog.verbose(env.app,
-                              'calling kernel-doc \'%s\'' % (" ".join(cmd)))
+            logger.verbose('calling kernel-doc \'%s\'' % (" ".join(cmd)))
 
             p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
             out, err = p.communicate()
@@ -122,8 +115,10 @@ def run(self):
             if p.returncode != 0:
                 sys.stderr.write(err)
 
-                kernellog.warn(env.app,
-                               'kernel-doc \'%s\' failed with return code %d' % (" ".join(cmd), p.returncode))
+                logger.warning(
+                    'kernel-doc \'%s\' failed with return code %d' %
+                    (" ".join(cmd), p.returncode)
+                )
                 return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
             elif env.config.kerneldoc_verbosity > 0:
                 sys.stderr.write(err)
@@ -149,22 +144,13 @@ def run(self):
             return node.children
 
         except Exception as e:  # pylint: disable=W0703
-            kernellog.warn(env.app, 'kernel-doc \'%s\' processing failed with: %s' %
+            logger.warning('kernel-doc \'%s\' processing failed with: %s' %
                            (" ".join(cmd), str(e)))
             return [nodes.error(None, nodes.paragraph(text = "kernel-doc missing"))]
 
     def do_parse(self, result, node):
-        if Use_SSI:
-            with switch_source_input(self.state, result):
-                self.state.nested_parse(result, 0, node, match_titles=1)
-        else:
-            save = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter
-            self.state.memo.reporter = AutodocReporter(result, self.state.memo.reporter)
-            self.state.memo.title_styles, self.state.memo.section_level = [], 0
-            try:
-                self.state.nested_parse(result, 0, node, match_titles=1)
-            finally:
-                self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = save
+        with switch_source_input(self.state, result):
+            self.state.nested_parse(result, 0, node, match_titles=1)
 
 
 def setup(app):
docs/sphinx/kernellog.py deleted +0 -28
deleted file mode 100644
index af924f51a7d..00000000000
@@ -1,28 +0,0 @@
-# SPDX-License-Identifier: GPL-2.0
-#
-# Sphinx has deprecated its older logging interface, but the replacement
-# only goes back to 1.6.  So here's a wrapper layer to keep around for
-# as long as we support 1.4.
-#
-import sphinx
-
-if sphinx.__version__[:3] >= '1.6':
-    UseLogging = True
-    from sphinx.util import logging
-    logger = logging.getLogger('kerneldoc')
-else:
-    UseLogging = False
-
-def warn(app, message):
-    if UseLogging:
-        logger.warning(message)
-    else:
-        app.warn(message)
-
-def verbose(app, message):
-    if UseLogging:
-        logger.verbose(message)
-    else:
-        app.verbose(message)
-
-
docs/sphinx/qapidoc.py +3 -26
index f270b494f01..9a0cfcbce75 100644
@@ -31,6 +31,7 @@
 from docutils.statemachine import ViewList
 from docutils.parsers.rst import directives, Directive
 from sphinx.errors import ExtensionError
+from sphinx.util.docutils import switch_source_input
 from sphinx.util.nodes import nested_parse_with_titles
 import sphinx
 from qapi.gen import QAPISchemaVisitor
@@ -38,15 +39,6 @@
 from qapi.schema import QAPISchema
 
 
-# Sphinx up to 1.6 uses AutodocReporter; 1.7 and later
-# use switch_source_input. Check borrowed from kerneldoc.py.
-Use_SSI = sphinx.__version__[:3] >= '1.7'
-if Use_SSI:
-    from sphinx.util.docutils import switch_source_input
-else:
-    from sphinx.ext.autodoc import AutodocReporter
-
-
 __version__ = '1.0'
 
 
@@ -517,23 +509,8 @@ def do_parse(self, rstlist, node):
         subheadings (titles) without confusing the rendering of
         anything else.
         """
-        # This is from kerneldoc.py -- it works around an API change in
-        # Sphinx between 1.6 and 1.7. Unlike kerneldoc.py, we use
-        # sphinx.util.nodes.nested_parse_with_titles() rather than the
-        # plain self.state.nested_parse(), and so we can drop the saving
-        # of title_styles and section_level that kerneldoc.py does,
-        # because nested_parse_with_titles() does that for us.
-        if Use_SSI:
-            with switch_source_input(self.state, rstlist):
-                nested_parse_with_titles(self.state, rstlist, node)
-        else:
-            save = self.state.memo.reporter
-            self.state.memo.reporter = AutodocReporter(
-                rstlist, self.state.memo.reporter)
-            try:
-                nested_parse_with_titles(self.state, rstlist, node)
-            finally:
-                self.state.memo.reporter = save
+        with switch_source_input(self.state, rstlist):
+            nested_parse_with_titles(self.state, rstlist, node)
 
 
 def setup(app):
Signature · 1 line
2.45.0
Reviewed-by on v2 2/2
#4 lore raw

Re: [PATCH v2 2/2] docs: remove Sphinx 1.x compatibility code

Quoted 5 lines · John Snow

In general, the Use_SSI workaround is no longer needed, and neither is the pre-1.6 logging shim for kerneldoc.

Quoted 7 lines
---
  docs/sphinx/hxtool.py    | 21 ++++-----------------
  docs/sphinx/kerneldoc.py | 38 ++++++++++++--------------------------
  docs/sphinx/kernellog.py | 28 ----------------------------
  docs/sphinx/qapidoc.py   | 29 +++--------------------------
  4 files changed, 19 insertions(+), 97 deletions(-)

delete mode 100644 docs/sphinx/kernellog.py

author
#5 lore raw
In reply to John Snow
Quoted 27 lines · John Snow

With recent deprecations, we can advance our minimum sphinx version safely. This is heavily motivated by new qapidoc work which is much easier to maintain cross-version compatibility for - see difficulties in our dbus documentation which only works on sphinx >= 4.

We can only guarantee >= 3.4.3 now, but that's still vastly easier than maintaining compatibility all the way back to 1.x.

https://gitlab.com/jsnow/qemu/-/pipelines/1359549621

(failures appear to be unrelated to the series.)

V2: refactor all code deletions into patch 2

John Snow (2):
Python: bump minimum sphinx version to 3.4.3
docs: remove Sphinx 1.x compatibility code

 docs/conf.py             |  7 +++----
 docs/sphinx/hxtool.py    | 21 ++++-----------------
 docs/sphinx/kerneldoc.py | 38 ++++++++++++--------------------------
 docs/sphinx/kernellog.py | 28 ----------------------------
 docs/sphinx/qapidoc.py   | 29 +++--------------------------
 pythondeps.toml          |  2 +-
 6 files changed, 23 insertions(+), 102 deletions(-)
 delete mode 100644 docs/sphinx/kernellog.py
Quoted 3 lines

--
2.45.0

Going to assume It's okay for me to stage this one under my Python tree.

Markus: This will create a minor conflict with the notes patches (the delint patch), but the resolution is straightforward.

--js