python: require python3.12+
cover python: require python3.12+
The stars have aligned, and we can jump all the way from Python3.9 to Python3.12.
GitLab CI: https://gitlab.com/jsnow/qemu/-/pipelines/2781408192
Python 3.9 is EOL, and Python 3.10 will be EOL in about two months (October 2026). Python 3.11 is, to be blunt, perfectly serviceable - but when investigating our actual minimum dependency requirements, I realized none of our supported platforms used anything older than Python 3.12, so we may as well build and test with the version that all of our build platforms are actually using.
This reduces the Python compat test matrix by quite a lot and allows us a few nice things. From memory, the highlights are:
- Pattern matching syntax (fancy case statements). I am not dying to use these anywhere, but they're nice!
- Vastly improved error reporting from CPython
- Ability to annotate exceptions with "notes", which allows us augment the default reporting behavior for unhandled exceptions - this can be used in e.g. iotests to help give additional information on tool crashes that is sometimes hard to discover otherwise
- The ability to use standard library types for typing syntax instead of importing from Typing (Technically, valid since 3.9, but...)
- The ability to use "x | y" instead of Union[x, y] in type hint syntax. Combined with the above, it's a good time to clean up the typing in qapi to make it read much more nicely.
- ExceptionGroups - the ability to capture and report multiple exceptions simultaneously. This is extremely useful for async QMP and managing multiple VMs from a single python context. Async QMP will be able to get a few nice QOL improvements with this.
My personal favorite:
- We can move to a much newer Sphinx, which gets rid of an awful lot of very ugly compat code. This is the *actual* motivation of this series.
Anyway, here's the version chart for our supported build platforms and why this is safe to do:
distro EOS python3 pip setuptools sphinx wheel
-------------------------------------------------------------------------
centos_stream_9 2026-12-12 3.9.25 21.3.1 53.0.0 3.4.3 0.36.2 centos_stream_9+ 2026-12-12 3.12.13 23.2.1 68.2.2 --- 0.41.2 freebsd -- 3.12.14 23.3.2 63.1.0 9.0.4 0.48.0 ubuntu_24_04 2028-04-23 3.12.3 24.0 68.1.2 7.2.6 0.42.0 centos_stream_10 2030-01-01 3.12.13 23.3.2 69.0.3 7.2.6 0.41.2 alpine_3_21 2026-11-01 3.12.14 24.3.1 70.3.0 8.1.3 0.43.0 debian_13 2028-08-09 3.13.5 25.1.1 78.1.1 8.1.3 0.46.1 fedora_43 2026-12-09 3.14.7 25.1.1 78.1.1 8.2.3 0.45.1 ubuntu_26_04 2031-05-29 3.14.4 25.1.1 78.1.1 8.2.3 0.46.3 fedora_44 2027-06-02 3.14.7 26.0.1 80.10.2 8.2.3 0.45.1 macports -- 3.14.7 26.0.1 80.10.2 9.1.0 0.48.0 alpine_3_22 2027-05-01 3.12.14 25.1.1 80.9.0 8.2.3 0.46.3 alpine_3_23 2027-11-01 3.12.14 25.1.1 80.9.0 8.2.3 0.46.3 openbsd -- 3.14.6 26.2.1 80.9.0 9.1.0 0.48.0 alpine_3_24 2028-06-01 3.14.7 26.1.2 82.0.1 9.1.0 0.47.0 pkgsrc_current -- 3.13.15 26.2.1 84.0.0 9.1.0 0.48.0
CentOS stream 9 note: python3.12, python3.12-pip, python3.12-setuptools and python3.12-wheel are available as optional packages in the standard repository. Sphinx is not available, but can be side-loaded from PyPI using pip3.12.
More cleanups to follow: we may have the chance to upgrade to pyproject.toml packaging once and for all now that FreeBSD has moved to Python3.12, which will get us off the legacy packaging we've been using. This isn't a problem *currently*, but it has been the cause of a lot of pain in the past, and the sooner we can ditch it the better.
toodles,
--js
John Snow (12):
tests: re-add opencv2 and numpy dependencies
tests: add Python3.12 to centos9.docker
tests: update emsdk-wasm64-cross
qapi: update type hint for types_seen
python: upgrade to Python3.12+
python: fix license configuration
python: allow mypy to run as the current python version
python: remove distutils workarounds
python: remove stale setup.cfg options
sphinx: require sphinx 7.2.6
sphinx: drop fakedbusdoc
sphinx: drop qapidoc back-compat overboiled spaghetti dinner
docs/conf.py | 11 +- docs/interop/dbus-display.rst | 14 +- docs/interop/dbus-vmstate.rst | 14 +- docs/interop/dbus-vnc.rst | 14 +- docs/sphinx/compat.py | 230 ------------------ docs/sphinx/fakedbusdoc.py | 30 --- docs/sphinx/qapi_domain.py | 47 ++-- configure | 11 +- tests/docker/dockerfiles/alpine.docker | 1 + tests/docker/dockerfiles/centos9.docker | 14 +- .../dockerfiles/debian-amd64-cross.docker | 2 + .../dockerfiles/debian-arm64-cross.docker | 2 + .../dockerfiles/debian-armhf-cross.docker | 2 + .../dockerfiles/debian-i686-cross.docker | 2 + .../dockerfiles/debian-ppc64el-cross.docker | 2 + .../dockerfiles/debian-riscv64-cross.docker | 2 + .../dockerfiles/debian-s390x-cross.docker | 2 + tests/docker/dockerfiles/debian.docker | 2 + .../dockerfiles/emsdk-wasm64-cross.docker | 4 +- .../dockerfiles/fedora-rust-nightly.docker | 2 + .../dockerfiles/fedora-win64-cross.docker | 2 + tests/docker/dockerfiles/fedora.docker | 2 + tests/docker/dockerfiles/opensuse-leap.docker | 2 + tests/docker/dockerfiles/ubuntu2404.docker | 2 + .gitlab-ci.d/macos-14.vars | 2 +- python/Makefile | 10 +- python/scripts/mkvenv.py | 58 +---- python/setup.cfg | 33 +-- python/tests/linters.py | 11 - python/tests/minreqs.txt | 66 ++--- pythondeps.toml | 13 +- .../ci/setup/debian/debian-13-ppc64le.yaml | 2 + .../ci/setup/ubuntu/ubuntu-2404-aarch64.yaml | 2 + .../ci/setup/ubuntu/ubuntu-2404-s390x.yaml | 2 + scripts/qapi/types.py | 2 +- tests/lcitool/mappings.yml | 24 +- tests/lcitool/projects/qemu.yml | 2 + tests/lcitool/targets/centos-stream-9.yml | 4 +- tests/vm/generated/freebsd.json | 2 + 39 files changed, 156 insertions(+), 493 deletions(-) delete mode 100644 docs/sphinx/compat.py delete mode 100644 docs/sphinx/fakedbusdoc.py
Signature · 1 line
2.55.0
PATCH 1/12 tests: re-add opencv2 and numpy dependencies
Some of the dependencies identified as avocado dependencies were in fact still being used, and should be re-added to our lcitool configuration to ensure that these tests are being executed in the CI environment.
--- tests/docker/dockerfiles/alpine.docker | 1 + tests/docker/dockerfiles/centos9.docker | 2 ++ tests/docker/dockerfiles/debian-amd64-cross.docker | 2 ++ tests/docker/dockerfiles/debian-arm64-cross.docker | 2 ++ tests/docker/dockerfiles/debian-armhf-cross.docker | 2 ++ tests/docker/dockerfiles/debian-i686-cross.docker | 2 ++ tests/docker/dockerfiles/debian-ppc64el-cross.docker | 2 ++ tests/docker/dockerfiles/debian-riscv64-cross.docker | 2 ++ tests/docker/dockerfiles/debian-s390x-cross.docker | 2 ++ tests/docker/dockerfiles/debian.docker | 2 ++ tests/docker/dockerfiles/fedora-rust-nightly.docker | 2 ++ tests/docker/dockerfiles/fedora-win64-cross.docker | 2 ++ tests/docker/dockerfiles/fedora.docker | 2 ++ tests/docker/dockerfiles/opensuse-leap.docker | 2 ++ tests/docker/dockerfiles/ubuntu2404.docker | 2 ++ .gitlab-ci.d/macos-14.vars | 2 +- scripts/ci/setup/debian/debian-13-ppc64le.yaml | 2 ++ scripts/ci/setup/ubuntu/ubuntu-2404-aarch64.yaml | 2 ++ scripts/ci/setup/ubuntu/ubuntu-2404-s390x.yaml | 2 ++ tests/lcitool/mappings.yml | 12 ++++++++++++ tests/lcitool/projects/qemu.yml | 2 ++ tests/vm/generated/freebsd.json | 2 ++ 22 files changed, 52 insertions(+), 1 deletion(-)
tests/docker/dockerfiles/alpine.docker +1 -0
@@ -83,6 +83,7 @@ RUN apk update && \ pixman-dev \ pkgconf \ pulseaudio-dev \ + py3-numpy \ py3-pip \ py3-setuptools \ py3-sphinx \
tests/docker/dockerfiles/centos9.docker +2 -0
@@ -130,6 +130,8 @@ RUN dnf --quiet distro-sync -y && \ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc RUN /usr/bin/pip3.11 install \ + numpy \ + opencv-python \ sphinx \ sphinx-rtd-theme
tests/docker/dockerfiles/debian-amd64-cross.docker +2 -0
@@ -41,6 +41,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ openssh-client \ pkgconf \ python3 \ + python3-numpy \ + python3-opencv \ python3-pip \ python3-setuptools \ python3-sphinx \
tests/docker/dockerfiles/debian-arm64-cross.docker +2 -0
@@ -41,6 +41,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ openssh-client \ pkgconf \ python3 \ + python3-numpy \ + python3-opencv \ python3-pip \ python3-setuptools \ python3-sphinx \
tests/docker/dockerfiles/debian-armhf-cross.docker +2 -0
@@ -41,6 +41,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ openssh-client \ pkgconf \ python3 \ + python3-numpy \ + python3-opencv \ python3-pip \ python3-setuptools \ python3-sphinx \
tests/docker/dockerfiles/debian-i686-cross.docker +2 -0
@@ -41,6 +41,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ openssh-client \ pkgconf \ python3 \ + python3-numpy \ + python3-opencv \ python3-pip \ python3-setuptools \ python3-sphinx \
tests/docker/dockerfiles/debian-ppc64el-cross.docker +2 -0
@@ -41,6 +41,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ openssh-client \ pkgconf \ python3 \ + python3-numpy \ + python3-opencv \ python3-pip \ python3-setuptools \ python3-sphinx \
tests/docker/dockerfiles/debian-riscv64-cross.docker +2 -0
@@ -41,6 +41,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ openssh-client \ pkgconf \ python3 \ + python3-numpy \ + python3-opencv \ python3-pip \ python3-setuptools \ python3-sphinx \
tests/docker/dockerfiles/debian-s390x-cross.docker +2 -0
@@ -41,6 +41,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ openssh-client \ pkgconf \ python3 \ + python3-numpy \ + python3-opencv \ python3-pip \ python3-setuptools \ python3-sphinx \
tests/docker/dockerfiles/debian.docker +2 -0
@@ -113,6 +113,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ openssh-client \ pkgconf \ python3 \ + python3-numpy \ + python3-opencv \ python3-pip \ python3-setuptools \ python3-sphinx \
tests/docker/dockerfiles/fedora-rust-nightly.docker +2 -0
@@ -104,6 +104,8 @@ exec "$@"\n' > /usr/bin/nosync && \ pkgconfig \ pulseaudio-libs-devel \ python3 \ + python3-numpy \ + python3-opencv \ python3-pip \ python3-setuptools \ python3-sphinx \
tests/docker/dockerfiles/fedora-win64-cross.docker +2 -0
@@ -44,6 +44,8 @@ exec "$@"\n' > /usr/bin/nosync && \ ninja-build \ openssh-clients \ python3 \ + python3-numpy \ + python3-opencv \ python3-pip \ python3-setuptools \ python3-sphinx \
tests/docker/dockerfiles/fedora.docker +2 -0
@@ -104,6 +104,8 @@ exec "$@"\n' > /usr/bin/nosync && \ pkgconfig \ pulseaudio-libs-devel \ python3 \ + python3-numpy \ + python3-opencv \ python3-pip \ python3-setuptools \ python3-sphinx \
tests/docker/dockerfiles/opensuse-leap.docker +2 -0
@@ -95,6 +95,8 @@ RUN zypper update -y && \ pkgconfig \ python3-Sphinx \ python3-base \ + python3-numpy \ + python3-opencv \ python3-pip \ python3-setuptools \ python3-sphinx_rtd_theme \
tests/docker/dockerfiles/ubuntu2404.docker +2 -0
@@ -113,6 +113,8 @@ RUN export DEBIAN_FRONTEND=noninteractive && \ openssh-client \ pkgconf \ python3 \ + python3-numpy \ + python3-opencv \ python3-pip \ python3-setuptools \ python3-sphinx \
.gitlab-ci.d/macos-14.vars +1 -1
@@ -12,5 +12,5 @@ NINJA='/opt/homebrew/bin/ninja' PACKAGING_COMMAND='brew' PIP3='/opt/homebrew/bin/pip3' PKGS='bash bc bindgen bison bzip2 capstone ccache cmocka coreutils ctags curl dbus diffutils dtc flex gcovr gettext git glib gnu-sed gnutls gtk+3 gtk-vnc jemalloc jpeg-turbo json-c libcbor libepoxy libffi libgcrypt libiscsi libnfs libpng libslirp libssh libtasn1 libusb llvm lzo make meson mtools ncurses nettle ninja pixman pkg-config python-setuptools python3 rpm2cpio rust sdl2 sdl2_image snappy socat sparse spice-protocol swtpm tesseract usbredir vde vte3 vulkan-tools xorriso zlib zstd' -PYPI_PKGS='sphinx sphinx-rtd-theme tomli' +PYPI_PKGS='numpy sphinx sphinx-rtd-theme tomli' PYTHON='/opt/homebrew/bin/python3'
scripts/ci/setup/debian/debian-13-ppc64le.yaml +2 -0
@@ -105,6 +105,8 @@ packages: - openssh-client - pkgconf - python3 + - python3-numpy + - python3-opencv - python3-pip - python3-setuptools - python3-sphinx
scripts/ci/setup/ubuntu/ubuntu-2404-aarch64.yaml +2 -0
@@ -106,6 +106,8 @@ packages: - openssh-client - pkgconf - python3 + - python3-numpy + - python3-opencv - python3-pip - python3-setuptools - python3-sphinx
scripts/ci/setup/ubuntu/ubuntu-2404-s390x.yaml +2 -0
@@ -104,6 +104,8 @@ packages: - openssh-client - pkgconf - python3 + - python3-numpy + - python3-opencv - python3-pip - python3-setuptools - python3-sphinx
tests/lcitool/mappings.yml +12 -0
@@ -21,6 +21,12 @@ mappings: python3-sphinx-rtd-theme: CentOSStream9: + python3-numpy: + CentOSStream9: + + python3-opencv: + CentOSStream9: + python3-venv: CentOSStream9: python3.11
@@ -54,3 +60,9 @@ pypi_mappings: apk: Fedora: Debian12: + + python3-numpy: + CentOSStream9: numpy + + python3-opencv: + CentOSStream9: opencv-python
tests/lcitool/projects/qemu.yml +2 -0
@@ -91,6 +91,8 @@ packages: - pkg-config - pulseaudio - python3 + - python3-numpy + - python3-opencv - python3-pip - python3-setuptools - python3-sphinx
tests/vm/generated/freebsd.json +2 -0
@@ -49,9 +49,11 @@ "ncurses", "nettle", "ninja", + "opencv", "pixman", "pkgconf", "png", + "py311-numpy", "py311-pip", "py311-setuptools", "py311-sphinx",
Signature · 1 line
2.55.0
Re: [PATCH 01/12] tests: re-add opencv2 and numpy dependencies
Quoted 24 lines · John Snow
Fixes 2a428c8541ec5e9be54aad5f0a446ec8172226c9Some of the dependencies identified as avocado dependencies were in fact still being used, and should be re-added to our lcitool configuration to ensure that these tests are being executed in the CI environment.
Signed-off-by John Snow <[email protected]>--- tests/docker/dockerfiles/alpine.docker | 1 + tests/docker/dockerfiles/centos9.docker | 2 ++ tests/docker/dockerfiles/debian-amd64-cross.docker | 2 ++ tests/docker/dockerfiles/debian-arm64-cross.docker | 2 ++ tests/docker/dockerfiles/debian-armhf-cross.docker | 2 ++ tests/docker/dockerfiles/debian-i686-cross.docker | 2 ++ tests/docker/dockerfiles/debian-ppc64el-cross.docker | 2 ++ tests/docker/dockerfiles/debian-riscv64-cross.docker | 2 ++ tests/docker/dockerfiles/debian-s390x-cross.docker | 2 ++ tests/docker/dockerfiles/debian.docker | 2 ++ tests/docker/dockerfiles/fedora-rust-nightly.docker | 2 ++ tests/docker/dockerfiles/fedora-win64-cross.docker | 2 ++ tests/docker/dockerfiles/fedora.docker | 2 ++ tests/docker/dockerfiles/opensuse-leap.docker | 2 ++ tests/docker/dockerfiles/ubuntu2404.docker | 2 ++
Quoted 8 lines
.gitlab-ci.d/macos-14.vars | 2 +- scripts/ci/setup/debian/debian-13-ppc64le.yaml | 2 ++ scripts/ci/setup/ubuntu/ubuntu-2404-aarch64.yaml | 2 ++ scripts/ci/setup/ubuntu/ubuntu-2404-s390x.yaml | 2 ++ tests/lcitool/mappings.yml | 12 ++++++++++++ tests/lcitool/projects/qemu.yml | 2 ++ tests/vm/generated/freebsd.json | 2 ++ 22 files changed, 52 insertions(+), 1 deletion(-)
Re: [PATCH 01/12] tests: re-add opencv2 and numpy dependencies
In reply to John SnowQuoted 8 lines · John Snow
Fixes 2a428c8541ec5e9be54aad5f0a446ec8172226c9Some of the dependencies identified as avocado dependencies were in fact still being used, and should be re-added to our lcitool configuration to ensure that these tests are being executed in the CI environment.
[...]
Signature · 1 line
Marc-André Lureau <[email protected]>
PATCH 2/12 tests: add Python3.12 to centos9.docker
In reply to John SnowWhile we recently upgraded to Python 3.11 for the CentOS 9 container, upgrade it even further to Python 3.12.
--- tests/docker/dockerfiles/centos9.docker | 12 ++++++------ tests/lcitool/mappings.yml | 12 ++++++------ tests/lcitool/targets/centos-stream-9.yml | 4 ++-- 3 files changed, 14 insertions(+), 14 deletions(-)
tests/docker/dockerfiles/centos9.docker +6 -6
@@ -96,10 +96,10 @@ RUN dnf --quiet distro-sync -y && \ pkgconfig \ pulseaudio-libs-devel \ python3-tomli \ - python3.11 \ - python3.11-pip \ - python3.11-setuptools \ - python3.11-wheel \ + python3.12 \ + python3.12-pip \ + python3.12-setuptools \ + python3.12-wheel \ rdma-core-devel \ rust \ rust-std-static \
@@ -129,7 +129,7 @@ RUN dnf --quiet distro-sync -y && \ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/clang && \ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc -RUN /usr/bin/pip3.11 install \ +RUN /usr/bin/pip3.12 install \ numpy \ opencv-python \ sphinx \
@@ -139,7 +139,7 @@ ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers" ENV LANG="en_US.UTF-8" ENV MAKE="/usr/bin/make" ENV NINJA="/usr/bin/ninja" -ENV PYTHON="/usr/bin/python3.11" +ENV PYTHON="/usr/bin/python3.12" # As a final step configure the user (if env is defined) ARG USER ARG UID
tests/lcitool/mappings.yml +6 -6
@@ -4,16 +4,16 @@ mappings: Ubuntu2204: python3: - CentOSStream9: python3.11 + CentOSStream9: python3.12 python3-devel: - CentOSStream9: python3.11-devel + CentOSStream9: python3.21-devel python3-pip: - CentOSStream9: python3.11-pip + CentOSStream9: python3.12-pip python3-setuptools: - CentOSStream9: python3.11-setuptools + CentOSStream9: python3.12-setuptools python3-sphinx: CentOSStream9:
@@ -28,10 +28,10 @@ mappings: CentOSStream9: python3-venv: - CentOSStream9: python3.11 + CentOSStream9: python3.12 python3-wheel: - CentOSStream9: python3.11-wheel + CentOSStream9: python3.12-wheel meson: # Use Meson from PyPI wherever Rust is enabled
tests/lcitool/targets/centos-stream-9.yml +2 -2
@@ -1,3 +1,3 @@ paths: - pip3: /usr/bin/pip3.11 - python: /usr/bin/python3.11 + pip3: /usr/bin/pip3.12 + python: /usr/bin/python3.12
Signature · 1 line
2.55.0
Re: [PATCH 02/12] tests: add Python3.12 to centos9.docker
Quoted 48 lines
While we recently upgraded to Python 3.11 for the CentOS 9 container, upgrade it even further to Python 3.12.
Signed-off-by John Snow <[email protected]>Message-ID: <[email protected]>
tests/docker/dockerfiles/centos9.docker +6
-6@@ -96,10 +96,10 @@ RUN dnf --quiet distro-sync -y && \ pkgconfig \ pulseaudio-libs-devel \ python3-tomli \ - python3.11 \ - python3.11-pip \ - python3.11-setuptools \ - python3.11-wheel \ + python3.12 \ + python3.12-pip \ + python3.12-setuptools \ + python3.12-wheel \ rdma-core-devel \ rust \ rust-std-static \@@ -129,7 +129,7 @@ RUN dnf --quiet distro-sync -y && \ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/clang && \ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc -RUN /usr/bin/pip3.11 install \ +RUN /usr/bin/pip3.12 install \ numpy \ opencv-python \ sphinx \@@ -139,7 +139,7 @@ ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers" ENV LANG="en_US.UTF-8" ENV MAKE="/usr/bin/make" ENV NINJA="/usr/bin/ninja" -ENV PYTHON="/usr/bin/python3.11" +ENV PYTHON="/usr/bin/python3.12" # As a final step configure the user (if env is defined) ARG USER ARG UIDtests/lcitool/mappings.yml +0
-0
tests/lcitool/mappings.yml mappings: · line 10
Quoted 9 lines
tests/lcitool/mappings.yml +2
-2@@ -4,16 +4,16 @@ mappings: Ubuntu2204: python3: - CentOSStream9: python3.11 + CentOSStream9: python3.12 python3-devel: - CentOSStream9: python3.11-devel + CentOSStream9: python3.21-devel
3.21 -> 3.12
Signature · 1 line
Marc-André Lureau <[email protected]>
Re: [PATCH 02/12] tests: add Python3.12 to centos9.docker
Quoted 50 lines · On 8/24/26 09:38, [email protected] wrote:
Quoted 50 lines
While we recently upgraded to Python 3.11 for the CentOS 9 container, upgrade it even further to Python 3.12.
Signed-off-by John Snow <[email protected]>Message-ID: <[email protected]>
tests/docker/dockerfiles/centos9.docker +6
-6@@ -96,10 +96,10 @@ RUN dnf --quiet distro-sync -y && \ pkgconfig \ pulseaudio-libs-devel \ python3-tomli \ - python3.11 \ - python3.11-pip \ - python3.11-setuptools \ - python3.11-wheel \ + python3.12 \ + python3.12-pip \ + python3.12-setuptools \ + python3.12-wheel \ rdma-core-devel \ rust \ rust-std-static \@@ -129,7 +129,7 @@ RUN dnf --quiet distro-sync -y && \ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/clang && \ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gcc -RUN /usr/bin/pip3.11 install \ +RUN /usr/bin/pip3.12 install \ numpy \ opencv-python \ sphinx \@@ -139,7 +139,7 @@ ENV CCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers" ENV LANG="en_US.UTF-8" ENV MAKE="/usr/bin/make" ENV NINJA="/usr/bin/ninja" -ENV PYTHON="/usr/bin/python3.11" +ENV PYTHON="/usr/bin/python3.12" # As a final step configure the user (if env is defined) ARG USER ARG UIDtests/lcitool/mappings.yml +0
-0@@ -4,16 +4,16 @@ mappings: Ubuntu2204:
Quoted 7 lines
tests/lcitool/mappings.yml mappings: · line 10
Quoted 7 lines
tests/lcitool/mappings.yml +2
-2python3: - CentOSStream9: python3.11 + CentOSStream9: python3.12 python3-devel: - CentOSStream9: python3.11-devel + CentOSStream9: python3.21-devel
Or drop it altogether, it seems not to be needed?
Paolo
Re: [PATCH 02/12] tests: add Python3.12 to centos9.docker
Quoted 57 lines · Paolo Bonzini
Quoted 7 lines · On 8/24/26 09:38, [email protected] wrote:
Review
Quoted 7 lines
While we recently upgraded to Python 3.11 for the CentOS 9 container, upgrade it even further to Python 3.12.
Signed-off-by John Snow <[email protected]>Message-ID: <[email protected]>
(quoted diff) +0
-0b/tests/docker/dockerfiles/centos9.docker
Quoted 22 lines
Quoted 22 lines
index
6f5c400c2db1..8bfce395f9d7100644tests/docker/dockerfiles/centos9.docker +4
-4@@ -96,10 +96,10 @@ RUN dnf --quiet distro-sync -y && \ pkgconfig \ pulseaudio-libs-devel \ python3-tomli \ - python3.11 \ - python3.11-pip \ - python3.11-setuptools \ - python3.11-wheel \ + python3.12 \ + python3.12-pip \ + python3.12-setuptools \ + python3.12-wheel \ rdma-core-devel \ rust \ rust-std-static \@@ -129,7 +129,7 @@ RUN dnf --quiet distro-sync -y && \ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/clang && \ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gccQuoted 6 lines
tests/docker/dockerfiles/centos9.docker ENV
Quoted 6 lines
tests/docker/dockerfiles/centos9.docker +1
-1-RUN /usr/bin/pip3.11 install \ +RUN /usr/bin/pip3.12 install \ numpy \ opencv-python \ sphinx \@@ -139,7 +139,7 @@ ENVCCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
Quoted 19 lines
Quoted 19 lines
tests/docker/dockerfiles/centos9.docker +1
-1ENV LANG="en_US.UTF-8" ENV MAKE="/usr/bin/make" ENV NINJA="/usr/bin/ninja" -ENV PYTHON="/usr/bin/python3.11" +ENV PYTHON="/usr/bin/python3.12" # As a final step configure the user (if env is defined) ARG USER ARG UIDtests/lcitool/mappings.yml +1
-1@@ -4,16 +4,16 @@ mappings: Ubuntu2204: python3: - CentOSStream9: python3.11 + CentOSStream9: python3.12
Quoted 7 lines
Quoted 3 lines
tests/lcitool/mappings.yml mappings: · line 10
Quoted 3 lines
tests/lcitool/mappings.yml +1
-1python3-devel: - CentOSStream9: python3.11-devel + CentOSStream9: python3.21-develOr drop it altogether, it seems not to be needed?
Paolo
Hah! Oops. Surprised that worked, but it does tell us something, eh?
Re: [PATCH 02/12] tests: add Python3.12 to centos9.docker
Quoted 64 lines · John Snow
Quoted 62 lines · Paolo Bonzini
Quoted 7 lines · On 8/24/26 09:38, [email protected] wrote:
Review
Quoted 7 lines
While we recently upgraded to Python 3.11 for the CentOS 9 container, upgrade it even further to Python 3.12.
Signed-off-by John Snow <[email protected]>Message-ID: <[email protected]>
(quoted diff) +0
-0b/tests/docker/dockerfiles/centos9.docker
Quoted 22 lines
Quoted 22 lines
index
6f5c400c2db1..8bfce395f9d7100644tests/docker/dockerfiles/centos9.docker +4
-4@@ -96,10 +96,10 @@ RUN dnf --quiet distro-sync -y && \ pkgconfig \ pulseaudio-libs-devel \ python3-tomli \ - python3.11 \ - python3.11-pip \ - python3.11-setuptools \ - python3.11-wheel \ + python3.12 \ + python3.12-pip \ + python3.12-setuptools \ + python3.12-wheel \ rdma-core-devel \ rust \ rust-std-static \@@ -129,7 +129,7 @@ RUN dnf --quiet distro-sync -y && \ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/clang && \ ln -s /usr/bin/ccache /usr/libexec/ccache-wrappers/gccQuoted 6 lines
tests/docker/dockerfiles/centos9.docker ENV
Quoted 6 lines
tests/docker/dockerfiles/centos9.docker +1
-1-RUN /usr/bin/pip3.11 install \ +RUN /usr/bin/pip3.12 install \ numpy \ opencv-python \ sphinx \@@ -139,7 +139,7 @@ ENVCCACHE_WRAPPERSDIR="/usr/libexec/ccache-wrappers"
Quoted 15 lines
Quoted 15 lines
tests/docker/dockerfiles/centos9.docker +1
-1ENV LANG="en_US.UTF-8" ENV MAKE="/usr/bin/make" ENV NINJA="/usr/bin/ninja" -ENV PYTHON="/usr/bin/python3.11" +ENV PYTHON="/usr/bin/python3.12" # As a final step configure the user (if env is defined) ARG USER ARG UIDtests/lcitool/mappings.yml +0
-0@@ -4,16 +4,16 @@ mappings: Ubuntu2204:Quoted 7 lines
tests/lcitool/mappings.yml mappings: · line 10
Quoted 7 lines
tests/lcitool/mappings.yml +2
-2python3: - CentOSStream9: python3.11 + CentOSStream9: python3.12 python3-devel: - CentOSStream9: python3.11-devel + CentOSStream9: python3.21-develOr drop it altogether, it seems not to be needed?
Quoted 4 lines
Quoted 2 lines
Paolo
Hah! Oops. Surprised that worked, but it does tell us something, eh?
Yes:
$ git grep python3-devel tests/lcitool/projects/
tells us that 'python3-devel' is not referenced in our definition for QEMU as a build-dep, so in turn we don't need a mapping for it.
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com ~~ https://hachyderm.io/@berrange :| |: https://libvirt.org ~~ https://entangle-photo.org :| |: https://pixelfed.art/berrange ~~ https://fstop138.berrange.com :|
PATCH 3/12 tests: update emsdk-wasm64-cross
In reply to John SnowThis container currently uses Ubuntu 22.04 as a base, which is no longer supported by QEMU as a build platform. Upgrade to a newer version of the image which will provide Python3.12+.
--- tests/docker/dockerfiles/emsdk-wasm64-cross.docker | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tests/docker/dockerfiles/emsdk-wasm64-cross.docker +2 -2
@@ -1,6 +1,6 @@ # syntax = docker/dockerfile:1.5 -ARG EMSDK_VERSION_QEMU=4.0.10 +ARG EMSDK_VERSION_QEMU=5.0.2 ARG ZLIB_VERSION=1.3.2 ARG GLIB_MINOR_VERSION=2.84 ARG GLIB_VERSION=${GLIB_MINOR_VERSION}.0
@@ -27,7 +27,7 @@ RUN apt-get update && apt-get install -y \ pkgconf \ ninja-build \ python3-pip -RUN pip3 install meson==${MESON_VERSION} tomli +RUN pip3 install --break-system-packages meson==${MESON_VERSION} tomli RUN mkdir /build WORKDIR /build RUN mkdir -p $TARGET
Signature · 1 line
2.55.0
Re: [PATCH 03/12] tests: update emsdk-wasm64-cross
Quoted 3 lines · John Snow
This container currently uses Ubuntu 22.04 as a base, which is no longer supported by QEMU as a build platform. Upgrade to a newer version of the image which will provide Python3.12+.
Signature · 1 line
Marc-André Lureau <[email protected]>
Kohei Tokunaga
Reviewed-byRe: [PATCH 03/12] tests: update emsdk-wasm64-cross
Review
Quoted 10 lines
This container currently uses Ubuntu 22.04 as a base, which is no longer supported by QEMU as a build platform. Upgrade to a newer version of the image which will provide Python3.12+.
Signed-off-by John Snow <[email protected]>--- tests/docker/dockerfiles/emsdk-wasm64-cross.docker | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/tests/docker/dockerfiles/emsdk-wasm64-cross.docker
Quoted 12 lines
index
c04a741b0e5..f11a9f34244100644tests/docker/dockerfiles/emsdk-wasm64-cross.docker +1
-1@@ -1,6 +1,6 @@ # syntax = docker/dockerfile:1.5 -ARG EMSDK_VERSION_QEMU=4.0.10 +ARG EMSDK_VERSION_QEMU=5.0.2 ARG ZLIB_VERSION=1.3.2 ARG GLIB_MINOR_VERSION=2.84 ARG GLIB_VERSION=${GLIB_MINOR_VERSION}.0
tests/docker/dockerfiles/emsdk-wasm64-cross.docker RUN apt-get update && apt-get install -y \ · line 33
Quoted 8 lines
tests/docker/dockerfiles/emsdk-wasm64-cross.docker +1
-1@@ -27,7 +27,7 @@ RUN apt-get update && apt-get install -y \ pkgconf \ ninja-build \ python3-pip -RUN pip3 install meson==${MESON_VERSION} tomli +RUN pip3 install --break-system-packages meson==${MESON_VERSION} tomli RUN mkdir /build WORKDIR /build RUN mkdir -p $TARGET
PATCH 4/12 qapi: update type hint for types_seen
In reply to John SnowIn preparation for upgrading Python to 3.12+, adjust the type hint on types_seen to prevent mypy from chirping about an unknown type for this variable.
Why didn't this trigger earlier? A mystery for the ages.
--- scripts/qapi/types.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/qapi/types.py +1 -1
@@ -34,7 +34,7 @@ # variants must be emitted before their container; track what has already # been output -objects_seen = set() +objects_seen: set[str] = set() def gen_enum_lookup(name: str,
Signature · 1 line
2.55.0
Re: [PATCH 04/12] qapi: update type hint for types_seen
Quoted 10 lines · John Snow
In preparation for upgrading Python to 3.12+, adjust the type hint on
types_seento prevent mypy from chirping about an unknown type for this variable.Why didn't this trigger earlier? A mystery for the ages.
Signed-off-by John Snow <[email protected]>--- scripts/qapi/types.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Re: [PATCH 04/12] qapi: update type hint for types_seen
In reply to John SnowQuoted 2 lines
In preparation for upgrading Python to 3.12+, adjust the type hint on
types_seento prevent mypy from chirping about an unknown type for
^^^
objects_seen
Quoted 12 lines
this variable.
Why didn't this trigger earlier? A mystery for the ages.
Signed-off-by John Snow <[email protected]>Message-ID: <[email protected]>
scripts/qapi/types.py +0
-0
scripts/qapi/types.py line 40
Quoted 8 lines
scripts/qapi/types.py +1
-1@@ -34,7 +34,7 @@ # variants must be emitted before their container; track what has already # been output -objects_seen = set() +objects_seen: set[str] = set() def gen_enum_lookup(name: str,
Signature · 1 line
Marc-André Lureau <[email protected]>
Re: [PATCH 04/12] qapi: update type hint for types_seen
In reply to John SnowQuoted 16 lines · John Snow
In preparation for upgrading Python to 3.12+, adjust the type hint on
types_seento prevent mypy from chirping about an unknown type for this variable.Why didn't this trigger earlier? A mystery for the ages.
Signed-off-by John Snow <[email protected]>--- scripts/qapi/types.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)scripts/qapi/types.py +0
-0
scripts/qapi/types.py line 40
Quoted 8 lines
scripts/qapi/types.py +1
-1@@ -34,7 +34,7 @@ # variants must be emitted before their container; track what has already # been output -objects_seen = set() +objects_seen: set[str] = set() def gen_enum_lookup(name: str,
PATCH 5/12 python: upgrade to Python3.12+
In reply to John SnowWith Python3.9 EOL and Python3.10 soon to be EOL (October 2026), and with ... no other supported build platform actually using Python 3.11, require the use of Python3.12 or later.
Additionally, as part of this upgrade, remove the use of tomllib from mkvenv as it is now standard in the Python stdlib: leaving this transitional code in causes a regression with mypy which becomes confused about the overloaded use of 'tomllib'.
Our current minimum requirements as determined by standard repository packages on supported distributions is as follows:
distro EOS python3 pip setuptools sphinx wheel
-------------------------------------------------------------------------
centos_stream_9 2026-12-12 3.9.25 21.3.1 53.0.0 3.4.3 0.36.2 centos_stream_9+ 2026-12-12 3.12.13 23.2.1 68.2.2 --- 0.41.2 freebsd -- 3.12.14 23.3.2 63.1.0 9.0.4 0.48.0 ubuntu_24_04 2028-04-23 3.12.3 24.0 68.1.2 7.2.6 0.42.0 centos_stream_10 2030-01-01 3.12.13 23.3.2 69.0.3 7.2.6 0.41.2 alpine_3_21 2026-11-01 3.12.14 24.3.1 70.3.0 8.1.3 0.43.0 debian_13 2028-08-09 3.13.5 25.1.1 78.1.1 8.1.3 0.46.1 fedora_43 2026-12-09 3.14.7 25.1.1 78.1.1 8.2.3 0.45.1 ubuntu_26_04 2031-05-29 3.14.4 25.1.1 78.1.1 8.2.3 0.46.3 fedora_44 2027-06-02 3.14.7 26.0.1 80.10.2 8.2.3 0.45.1 macports -- 3.14.7 26.0.1 80.10.2 9.1.0 0.48.0 alpine_3_22 2027-05-01 3.12.14 25.1.1 80.9.0 8.2.3 0.46.3 alpine_3_23 2027-11-01 3.12.14 25.1.1 80.9.0 8.2.3 0.46.3 openbsd -- 3.14.6 26.2.1 80.9.0 9.1.0 0.48.0 alpine_3_24 2028-06-01 3.14.7 26.1.2 82.0.1 9.1.0 0.47.0 pkgsrc_current -- 3.13.15 26.2.1 84.0.0 9.1.0 0.48.0
CentOS stream 9 note: python3.12, python3.12-pip, python3.12-setuptools and python3.12-wheel are available as optional packages in the standard repository. Sphinx is not available, but can be side-loaded from PyPI using pip3.12.
--- configure | 11 ++++---- python/Makefile | 10 +++---- python/scripts/mkvenv.py | 58 +++------------------------------------- python/setup.cfg | 17 +++++------- python/tests/minreqs.txt | 41 +++++++++++++--------------- pythondeps.toml | 9 +++---- 6 files changed, 42 insertions(+), 104 deletions(-)
configure +5 -6
@@ -503,17 +503,16 @@ if test -n "$linux_arch" && ! test -d "$source_path/linux-headers/asm-$linux_arc fi check_py_version() { - # We require python >= 3.9. + # We require python >= 3.12. # NB: a True python conditional creates a non-zero return code (Failure) - "$1" -c 'import sys; sys.exit(sys.version_info < (3,9))' + "$1" -c 'import sys; sys.exit(sys.version_info < (3,12))' } first_python= if test -z "${PYTHON}"; then # A bare 'python' is traditionally python 2.x, but some distros # have it as python 3.x, so check in both places. - for binary in python3 python python3.14 python3.13 python3.12 \ - python3.11 python3.10 python3.9 ; do + for binary in python3 python python3.14 python3.13 python3.12; do if has "$binary"; then python=$(command -v "$binary") if check_py_version "$python"; then
@@ -902,7 +901,7 @@ then # If first_python is set, there was a binary somewhere even though # it was not suitable. Use it for the error message. if test -n "$first_python"; then - error_exit "Cannot use '$first_python', Python >= 3.9 is required." \ + error_exit "Cannot use '$first_python', Python >= 3.12 is required." \ "Use --python=/path/to/python to specify a supported Python." else error_exit "Python not found. Use --python=/path/to/python"
@@ -910,7 +909,7 @@ then fi if ! check_py_version "$python"; then - error_exit "Cannot use '$python', Python >= 3.9 is required." \ + error_exit "Cannot use '$python', Python >= 3.12 is required." \ "Use --python=/path/to/python to specify a supported Python." \ "Maybe try:" \ " CentOS: dnf install python3.12"
python/Makefile +5 -5
@@ -9,14 +9,14 @@ help: @echo "make check-minreqs:" @echo " Run tests in the minreqs virtual environment." @echo " These tests use the oldest dependencies." - @echo " Requires: Python 3.9" - @echo " Hint (Fedora): 'sudo dnf install python3.9'" + @echo " Requires: Python 3.12" + @echo " Hint (Fedora): 'sudo dnf install python3.12'" @echo "" @echo "make check-tox:" @echo " Run tests against multiple python versions." @echo " These tests use the newest dependencies." - @echo " Requires: Python 3.9 - 3.11, and tox." - @echo " Hint (Fedora): 'sudo dnf install python3-tox python3.11'" + @echo " Requires: Python 3.12 - 3.14, and tox." + @echo " Hint (Fedora): 'sudo dnf install python3-tox python3.14'" @echo " The variable QEMU_TOX_EXTRA_ARGS can be use to pass extra" @echo " arguments to tox". @echo ""
@@ -59,7 +59,7 @@ PIP_INSTALL = pip install --disable-pip-version-check min-venv: $(QEMU_MINVENV_DIR) $(QEMU_MINVENV_DIR)/bin/activate $(QEMU_MINVENV_DIR) $(QEMU_MINVENV_DIR)/bin/activate: setup.cfg tests/minreqs.txt @echo "VENV $(QEMU_MINVENV_DIR)" - @python3.9 -m venv $(QEMU_MINVENV_DIR) + @python3.12 -m venv $(QEMU_MINVENV_DIR) @( \ echo "ACTIVATE $(QEMU_MINVENV_DIR)"; \ . $(QEMU_MINVENV_DIR)/bin/activate; \
python/scripts/mkvenv.py +4 -54
92 lines not on this page · raw message
python/setup.cfg +6 -11
@@ -14,16 +14,13 @@ classifiers = Natural Language :: English Operating System :: OS Independent Programming Language :: Python :: 3 :: Only - Programming Language :: Python :: 3.9 - Programming Language :: Python :: 3.10 - Programming Language :: Python :: 3.11 Programming Language :: Python :: 3.12 Programming Language :: Python :: 3.13 Programming Language :: Python :: 3.14 Typing :: Typed [options] -python_requires = >= 3.9 +python_requires = >= 3.12 packages = qemu.machine qemu.utils
@@ -39,12 +36,10 @@ devel = distlib >= 0.3.6 flake8 >= 5.0.4 fusepy >= 2.0.4 - isort >= 5.6.0 - # Max ver is temporary until we drop 3.9 - mypy >= 1.4.0, <2.0.0 + isort >= 5.11.3 + mypy >= 1.4.0 pylint >= 2.17.3 - pylint != 3.2.4; python_version<"3.9" - pytest >= 6.0.2 + pytest >= 6.2.4 tox >= 3.18.0 sphinx >= 3.4.3
@@ -69,7 +64,7 @@ exclude = __pycache__, [mypy] strict = True -python_version = 3.9 +python_version = 3.12 namespace_packages = True warn_unused_ignores = False
@@ -164,7 +159,7 @@ multi_line_output=3 # of python available on your system to run this test. [tox:tox] -envlist = py39, py310, py311, py312, py313, py314 +envlist = py312, py313, py314 skip_missing_interpreters = true [testenv]
python/tests/minreqs.txt +19 -22
84 lines not on this page · raw message
pythondeps.toml +3 -6
@@ -38,14 +38,11 @@ sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.2.2" } [tooling] "qemu.qmp" = { accepted = ">=0.0.5", installed = "0.0.6" } "qemu" = { path = "python/" } -# NB: The following dependencies should be a little bit more modern than -# the versions listed here, but we are still using Debian 11 for several -# GitLab CI tests, so we are further restricted. -"setuptools" = { accepted = ">=44.1.1" } -"wheel" = { accepted = ">=0.34.2" } +"setuptools" = { accepted = ">=63.1.0" } +"wheel" = { accepted = ">=0.41.2" } # pip should be guaranteed by mkvenv, this is merely a sanity check for # which version we are counting on being present. -"pip" = { accepted = ">=20.3.4" } +"pip" = { accepted = ">=23.2.1" } # This test group is for functional tests, and can include dependencies # fetched from PyPI.
Signature · 1 line
2.55.0
Re: [PATCH 05/12] python: upgrade to Python3.12+
Quoted 10 lines · John Snow
With Python3.9 EOL and Python3.10 soon to be EOL (October 2026), and with ... no other supported build platform actually using Python 3.11, require the use of Python3.12 or later.
Additionally, as part of this upgrade, remove the use of tomllib from mkvenv as it is now standard in the Python stdlib: leaving this transitional code in causes a regression with mypy which becomes confused about the overloaded use of 'tomllib'.
[...]
Signature · 1 line
Marc-André Lureau <[email protected]>
Re: [PATCH 05/12] python: upgrade to Python3.12+
In reply to John SnowQuoted 209 lines · John Snow
With Python3.9 EOL and Python3.10 soon to be EOL (October 2026), and with ... no other supported build platform actually using Python 3.11, require the use of Python3.12 or later.
Additionally, as part of this upgrade, remove the use of tomllib from mkvenv as it is now standard in the Python stdlib: leaving this transitional code in causes a regression with mypy which becomes confused about the overloaded use of 'tomllib'.
Our current minimum requirements as determined by standard repository packages on supported distributions is as follows:
distro EOS python3 pip setuptools sphinx wheel
-------------------------------------------------------------------------centos_stream_9 2026-12-12 3.9.25 21.3.1 53.0.0 3.4.3 0.36.2 centos_stream_9+ 2026-12-12 3.12.13 23.2.1 68.2.2 --- 0.41.2 freebsd -- 3.12.14 23.3.2 63.1.0 9.0.4 0.48.0 ubuntu_24_04 2028-04-23 3.12.3 24.0 68.1.2 7.2.6 0.42.0 centos_stream_10 2030-01-01 3.12.13 23.3.2 69.0.3 7.2.6 0.41.2 alpine_3_21 2026-11-01 3.12.14 24.3.1 70.3.0 8.1.3 0.43.0 debian_13 2028-08-09 3.13.5 25.1.1 78.1.1 8.1.3 0.46.1 fedora_43 2026-12-09 3.14.7 25.1.1 78.1.1 8.2.3 0.45.1 ubuntu_26_04 2031-05-29 3.14.4 25.1.1 78.1.1 8.2.3 0.46.3 fedora_44 2027-06-02 3.14.7 26.0.1 80.10.2 8.2.3 0.45.1 macports -- 3.14.7 26.0.1 80.10.2 9.1.0 0.48.0 alpine_3_22 2027-05-01 3.12.14 25.1.1 80.9.0 8.2.3 0.46.3 alpine_3_23 2027-11-01 3.12.14 25.1.1 80.9.0 8.2.3 0.46.3 openbsd -- 3.14.6 26.2.1 80.9.0 9.1.0 0.48.0 alpine_3_24 2028-06-01 3.14.7 26.1.2 82.0.1 9.1.0 0.47.0 pkgsrc_current -- 3.13.15 26.2.1 84.0.0 9.1.0 0.48.0CentOS stream 9 note: python3.12, python3.12-pip, python3.12-setuptools and python3.12-wheel are available as optional packages in the standard repository. Sphinx is not available, but can be side-loaded from PyPI using pip3.12.
Signed-off-by John Snow <[email protected]>--- configure | 11 ++++---- python/Makefile | 10 +++---- python/scripts/mkvenv.py | 58 +++------------------------------------- python/setup.cfg | 17 +++++------- python/tests/minreqs.txt | 41 +++++++++++++--------------- pythondeps.toml | 9 +++---- 6 files changed, 42 insertions(+), 104 deletions(-)configure +5
-6@@ -503,17 +503,16 @@ if test -n "$linux_arch" && ! test -d "$source_path/linux-headers/asm-$linux_arc fi check_py_version() { - # We require python >= 3.9. + # We require python >= 3.12. # NB: a True python conditional creates a non-zero return code (Failure) - "$1" -c 'import sys; sys.exit(sys.version_info < (3,9))' + "$1" -c 'import sys; sys.exit(sys.version_info < (3,12))' } first_python= if test -z "${PYTHON}"; then # A bare 'python' is traditionally python 2.x, but some distros # have it as python 3.x, so check in both places. - for binary in python3 python python3.14 python3.13 python3.12 \ - python3.11 python3.10 python3.9 ; do + for binary in python3 python python3.14 python3.13 python3.12; do if has "$binary"; then python=$(command -v "$binary") if check_py_version "$python"; then@@ -902,7 +901,7 @@ then # If first_python is set, there was a binary somewhere even though # it was not suitable. Use it for the error message. if test -n "$first_python"; then - error_exit "Cannot use '$first_python', Python >= 3.9 is required." \ + error_exit "Cannot use '$first_python', Python >= 3.12 is required." \ "Use --python=/path/to/python to specify a supported Python." else error_exit "Python not found. Use --python=/path/to/python"@@ -910,7 +909,7 @@ then fi if ! check_py_version "$python"; then - error_exit "Cannot use '$python', Python >= 3.9 is required." \ + error_exit "Cannot use '$python', Python >= 3.12 is required." \ "Use --python=/path/to/python to specify a supported Python." \ "Maybe try:" \ " CentOS: dnf install python3.12"python/Makefile +5
-5@@ -9,14 +9,14 @@ help: @echo "make check-minreqs:" @echo " Run tests in the minreqs virtual environment." @echo " These tests use the oldest dependencies." - @echo " Requires: Python 3.9" - @echo " Hint (Fedora): 'sudo dnf install python3.9'" + @echo " Requires: Python 3.12" + @echo " Hint (Fedora): 'sudo dnf install python3.12'" @echo "" @echo "make check-tox:" @echo " Run tests against multiple python versions." @echo " These tests use the newest dependencies." - @echo " Requires: Python 3.9 - 3.11, and tox." - @echo " Hint (Fedora): 'sudo dnf install python3-tox python3.11'" + @echo " Requires: Python 3.12 - 3.14, and tox." + @echo " Hint (Fedora): 'sudo dnf install python3-tox python3.14'" @echo " The variable QEMU_TOX_EXTRA_ARGS can be use to pass extra" @echo " arguments to tox". @echo ""@@ -59,7 +59,7 @@ PIP_INSTALL = pip install --disable-pip-version-check min-venv: $(QEMU_MINVENV_DIR) $(QEMU_MINVENV_DIR)/bin/activate $(QEMU_MINVENV_DIR) $(QEMU_MINVENV_DIR)/bin/activate: setup.cfg tests/minreqs.txt @echo "VENV $(QEMU_MINVENV_DIR)" - @python3.9 -m venv $(QEMU_MINVENV_DIR) + @python3.12 -m venv $(QEMU_MINVENV_DIR) @( \ echo "ACTIVATE $(QEMU_MINVENV_DIR)"; \ . $(QEMU_MINVENV_DIR)/bin/activate; \python/scripts/mkvenv.py +3
-49@@ -75,6 +75,7 @@ import subprocess import sys import sysconfig +import tomllib from types import SimpleNamespace from typing import ( Any,@@ -134,20 +135,6 @@ HAVE_PACKAGING_VERSION = _import_ok -# Try to load tomllib, with a fallback to tomli. -# HAVE_TOMLLIB is checked below, just-in-time, so that mkvenv does not fail -# outside the venv or before a potential call to ensurepip in checkpip(). -_import_ok = True -try: - import tomllib -except ImportError: - try: - import tomli as tomllib - except ImportError: - _import_ok = False - -HAVE_TOMLLIB = _import_ok - # Do not add any mandatory dependencies from outside the stdlib: # This script *must* be usable standalone!@@ -261,36 +248,6 @@ def get_parent_libpath(self) -> Optional[str]: return sysconfig.get_path("purelib") return None - @staticmethod - def compute_venv_libpath(context: SimpleNamespace) -> str: - """ - Compatibility wrapper for context.lib_path for Python < 3.12 - """ - # Python 3.12+, not strictly necessary because it's documented - # to be the same as 3.10 code below: - if sys.version_info >= (3, 12): - return context.lib_path - - # Python 3.10+ - if "venv" in sysconfig.get_scheme_names(): - lib_path = sysconfig.get_path( - "purelib", scheme="venv", vars={"base": context.env_dir} - ) - assert lib_path is not None - return lib_path - - # For Python <= 3.9 we need to hardcode this. Fortunately the - # code below was the same in Python 3.6-3.10, so there is only - # one case. - if sys.platform == "win32": - return os.path.join(context.env_dir, "Lib", "site-packages") - return os.path.join( - context.env_dir, - "lib", - "python%d.%d" % sys.version_info[:2], - "site-packages", - ) - def ensure_directories(self, env_dir: DirType) -> SimpleNamespace: logger.debug("ensure_directories(env_dir=%s)", env_dir) self._context = super().ensure_directories(env_dir)@@ -313,7 +270,8 @@ def post_post_setup(self, context: SimpleNamespace) -> None: assert parent_libpath is not None logger.debug("parent_libpath: %s", parent_libpath) - our_libpath = self.compute_venv_libpath(context) + assert isinstance(context.lib_path, str) + our_libpath = context.lib_path logger.debug("our_libpath: %s", our_libpath) pth_file = os.path.join(our_libpath, "nested.pth")@@ -810,19 +768,11 @@ def _do_ensure( def _parse_groups(file: str) -> Dict[str, Dict[str, Any]]: - if not HAVE_TOMLLIB: - if sys.version_info < (3, 11): - raise Ouch("found no usable tomli, please install it") -
python/scripts/mkvenv.py def _do_ensure( · line 775
Quoted 10 lines
python/scripts/mkvenv.py +1
-5- raise Ouch( - "Python >=3.11 does not have tomllib... what have you done!?" - ) - # Use loads() to support both tomli v1.2.x (Ubuntu 22.04, # Debian bullseye-backports) and v2.0.x with open(file, "r", encoding="ascii") as depfile: contents = depfile.read() - return tomllib.loads(contents) # type: ignore + return tomllib.loads(contents)
You can now use
with open(file, "r", encoding="ascii") as depfile:
return tomllib.load(depfile)
since the comment about loads() does not apply anymore.
Thanks,
Paolo
PATCH 6/12 python: fix license configuration
In reply to John SnowLicense trove identifiers were deprecated in favor of SPDX identifiers, correct this to stop some whining from various python tooling.
--- python/setup.cfg | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
python/setup.cfg +1 -1
@@ -8,9 +8,9 @@ download_url = https://www.qemu.org/download/ description = QEMU Python Build, Debug and SDK tooling. long_description = file:PACKAGE.rst long_description_content_type = text/x-rst +license = GPL-2.0-or-later classifiers = Development Status :: 3 - Alpha - License :: OSI Approved :: GNU General Public License v2 (GPLv2) Natural Language :: English Operating System :: OS Independent Programming Language :: Python :: 3 :: Only
Signature · 1 line
2.55.0
Re: [PATCH 06/12] python: fix license configuration
Quoted 3 lines · John Snow
License trove identifiers were deprecated in favor of SPDX identifiers, correct this to stop some whining from various python tooling.
Signature · 1 line
Marc-André Lureau <[email protected]>
PATCH 7/12 python: allow mypy to run as the current python version
In reply to John SnowCurrently, we instruct mypy to run assuming a specific version of python. Lift this restriction so that under tox tests, it will always run assuming the current running version.
--- python/setup.cfg | 1 - 1 file changed, 1 deletion(-)
python/setup.cfg +0 -1
@@ -64,7 +64,6 @@ exclude = __pycache__, [mypy] strict = True -python_version = 3.12 namespace_packages = True warn_unused_ignores = False
Signature · 1 line
2.55.0
Re: [PATCH 07/12] python: allow mypy to run as the current python version
Quoted 8 lines · John Snow
Currently, we instruct mypy to run assuming a specific version of python. Lift this restriction so that under tox tests, it will always run assuming the current running version.
Signed-off-by John Snow <[email protected]>--- python/setup.cfg | 1 - 1 file changed, 1 deletion(-)
Re: [PATCH 07/12] python: allow mypy to run as the current python version
In reply to John SnowQuoted 3 lines · John Snow
Currently, we instruct mypy to run assuming a specific version of python. Lift this restriction so that under tox tests, it will always run assuming the current running version.
Signature · 1 line
Marc-André Lureau <[email protected]>
PATCH 8/12 python: remove distutils workarounds
In reply to John SnowThese are no longer necessary now that we are using Python3.12+.
--- python/tests/linters.py | 11 ----------- 1 file changed, 11 deletions(-)
python/tests/linters.py +0 -11
@@ -83,23 +83,13 @@ def test_mypy_iotests(self): cwd="../tests/qemu-iotests/", ) - # Setuptools v60 introduced the SETUPTOOLS_USE_DISTUTILS=stdlib - # workaround; stdlib distutils was fully removed in Python - # 3.12+. Once we are on >=3.12+ exclusively, this workaround can be - # dropped safely. Until then, it is needed for some versions on - # Fedora/Debian distributions which relied upon distro-patched - # setuptools present in CPython, but not within setuptools itself. - def test_pylint_pkg(self): - os.environ["SETUPTOOLS_USE_DISTUTILS"] = "stdlib" check_call([sys.executable, "-m", "pylint", "qemu/"]) def test_pylint_scripts(self): - os.environ["SETUPTOOLS_USE_DISTUTILS"] = "stdlib" check_call([sys.executable, "-m", "pylint", "scripts/"]) def test_pylint_qapi(self): - os.environ["SETUPTOOLS_USE_DISTUTILS"] = "stdlib" check_call( [ sys.executable,
@@ -113,7 +103,6 @@ def test_pylint_qapi(self): ) def test_pylint_iotests(self): - os.environ["SETUPTOOLS_USE_DISTUTILS"] = "stdlib" check_call( [sys.executable, "-m", "linters", "--pylint"], cwd="../tests/qemu-iotests/",
Signature · 1 line
2.55.0
Re: [PATCH 08/12] python: remove distutils workarounds
Quoted 1 line · John Snow
These are no longer necessary now that we are using Python3.12+.
Signature · 1 line
Marc-André Lureau <[email protected]>
PATCH 9/12 python: remove stale setup.cfg options
In reply to John SnowWith the recent version bumps, we no longer need these directives.
--- python/setup.cfg | 13 ------------- 1 file changed, 13 deletions(-)
python/setup.cfg +0 -13
@@ -65,7 +65,6 @@ exclude = __pycache__, [mypy] strict = True namespace_packages = True -warn_unused_ignores = False [mypy-qemu.utils.qom_fuse] # fusepy has no type stubs:
@@ -76,12 +75,6 @@ allow_subclassing_any = True [mypy-fuse] ignore_missing_imports = True -[mypy-tomli] -ignore_missing_imports = True - -[mypy-tomllib] -ignore_missing_imports = True - [mypy-distlib] ignore_missing_imports = True
@@ -91,12 +84,6 @@ ignore_missing_imports = True [mypy-distlib.version] ignore_missing_imports = True -[mypy-pip._vendor.distlib] -ignore_missing_imports = True - -[mypy-pip._vendor.distlib.scripts] -ignore_missing_imports = True - [mypy-pip._vendor.distlib.version] ignore_missing_imports = True
Signature · 1 line
2.55.0
Re: [PATCH 09/12] python: remove stale setup.cfg options
Quoted 1 line · John Snow
With the recent version bumps, we no longer need these directives.
Signature · 1 line
Marc-André Lureau <[email protected]>
PATCH 10/12 sphinx: require sphinx 7.2.6
In reply to John SnowMake the extraordinary leap from Sphinx 3.4.3 to Sphinx 7.2.6. Simplifications and cleanup will follow in subsequent patches.
--- docs/conf.py | 4 ++-- python/setup.cfg | 2 +- python/tests/minreqs.txt | 29 ++++++++++++++++------------- pythondeps.toml | 4 ++-- 4 files changed, 21 insertions(+), 18 deletions(-)
docs/conf.py +2 -2
@@ -51,9 +51,9 @@ # If your documentation needs a minimal Sphinx version, state it here. # -# 3.4.3 is the oldest version of Sphinx that ships on a platform we +# 7.2.6 is the oldest version of Sphinx that ships on a platform we # pledge build support for. -needs_sphinx = '3.4.3' +needs_sphinx = '7.2.6' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
python/setup.cfg +1 -1
@@ -41,7 +41,7 @@ devel = pylint >= 2.17.3 pytest >= 6.2.4 tox >= 3.18.0 - sphinx >= 3.4.3 + sphinx >= 7.2.6 # Provides qom-fuse functionality fuse =
python/tests/minreqs.txt +16 -13
@@ -12,7 +12,7 @@ # on PyPI that allows the test suite to pass. # Dependencies for qapidoc/qapi_domain et al -sphinx==3.4.3 +sphinx==7.2.6 # Dependencies for qemu.machine qemu.qmp==0.0.6
@@ -48,23 +48,26 @@ platformdirs==2.2.0 tomlkit==0.10.1 # Transitive sphinx dependencies -Jinja2==2.7 -MarkupSafe==1.1.0 alabaster==0.7.1 -babel==1.3 -docutils==0.12 -imagesize==0.5.0 -packaging==14.0 -Pygments==2.0 -pytz==2014.9 -requests==2.5.0 -snowballstemmer==1.1 +babel==2.9 +certifi==2023.5.7 +charset-normalizer==2.0.0 +docutils==0.18.1 +idna==2.5 +imagesize==1.3 +Jinja2==3.0.0 +MarkupSafe==2.0 +packaging==21.0 +Pygments==2.14 +requests==2.25.0 +snowballstemmer==2.0 sphinxcontrib-applehelp==1.0.0 sphinxcontrib-devhelp==1.0.0 -sphinxcontrib-htmlhelp==1.0.0 +sphinxcontrib-htmlhelp==2.0.0 sphinxcontrib-jsmath==1.0.0 sphinxcontrib-qthelp==1.0.0 -sphinxcontrib-serializinghtml==1.0.0 +sphinxcontrib-serializinghtml==1.1.9 +urllib3==1.26 # Transitive pytest dependencies attrs==19.2.0
pythondeps.toml +2 -2
@@ -28,8 +28,8 @@ meson = { accepted = ">=1.12.0", installed = "1.12.0", canary = "meson" } [docs] # Please keep the installed versions in sync with docs/requirements.txt -sphinx = { accepted = ">=3.4.3", installed = "6.2.1", canary = "sphinx-build" } -sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.2.2" } +sphinx = { accepted = ">=7.2.6", installed = "7.2.6", canary = "sphinx-build" } +sphinx_rtd_theme = { accepted = ">=1.2.2", installed = "1.2.2" } # This test group is for dependencies required to run various tooling # and tests that should always be installed at configure time. It should
Signature · 1 line
2.55.0
Re: [PATCH 10/12] sphinx: require sphinx 7.2.6
Quoted 2 lines · John Snow
Make the extraordinary leap from Sphinx 3.4.3 to Sphinx 7.2.6. Simplifications and cleanup will follow in subsequent patches.
Signature · 1 line
Marc-André Lureau <[email protected]>
Re: [PATCH 10/12] sphinx: require sphinx 7.2.6
In reply to John SnowQuoted 2 lines · John Snow
Make the extraordinary leap from Sphinx 3.4.3 to Sphinx 7.2.6. Simplifications and cleanup will follow in subsequent patches.
Does this have any implications on what platforms we build docs on ? If so, mention any notable impacted platforms in the commit message.
Quoted 109 lines
Signed-off-by John Snow <[email protected]>--- docs/conf.py | 4 ++-- python/setup.cfg | 2 +- python/tests/minreqs.txt | 29 ++++++++++++++++------------- pythondeps.toml | 4 ++-- 4 files changed, 21 insertions(+), 18 deletions(-)docs/conf.py +2
-2@@ -51,9 +51,9 @@ # If your documentation needs a minimal Sphinx version, state it here. # -# 3.4.3 is the oldest version of Sphinx that ships on a platform we +# 7.2.6 is the oldest version of Sphinx that ships on a platform we # pledge build support for. -needs_sphinx = '3.4.3' +needs_sphinx = '7.2.6' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custompython/setup.cfg +1
-1@@ -41,7 +41,7 @@ devel = pylint >= 2.17.3 pytest >= 6.2.4 tox >= 3.18.0 - sphinx >= 3.4.3 + sphinx >= 7.2.6 # Provides qom-fuse functionality fuse =python/tests/minreqs.txt +16
-13@@ -12,7 +12,7 @@ # on PyPI that allows the test suite to pass. # Dependencies for qapidoc/qapi_domain et al -sphinx==3.4.3 +sphinx==7.2.6 # Dependencies for qemu.machine qemu.qmp==0.0.6@@ -48,23 +48,26 @@ platformdirs==2.2.0 tomlkit==0.10.1 # Transitive sphinx dependencies -Jinja2==2.7 -MarkupSafe==1.1.0 alabaster==0.7.1 -babel==1.3 -docutils==0.12 -imagesize==0.5.0 -packaging==14.0 -Pygments==2.0 -pytz==2014.9 -requests==2.5.0 -snowballstemmer==1.1 +babel==2.9 +certifi==2023.5.7 +charset-normalizer==2.0.0 +docutils==0.18.1 +idna==2.5 +imagesize==1.3 +Jinja2==3.0.0 +MarkupSafe==2.0 +packaging==21.0 +Pygments==2.14 +requests==2.25.0 +snowballstemmer==2.0 sphinxcontrib-applehelp==1.0.0 sphinxcontrib-devhelp==1.0.0 -sphinxcontrib-htmlhelp==1.0.0 +sphinxcontrib-htmlhelp==2.0.0 sphinxcontrib-jsmath==1.0.0 sphinxcontrib-qthelp==1.0.0 -sphinxcontrib-serializinghtml==1.0.0 +sphinxcontrib-serializinghtml==1.1.9 +urllib3==1.26 # Transitive pytest dependencies attrs==19.2.0pythondeps.toml +2
-2@@ -28,8 +28,8 @@ meson = { accepted = ">=1.12.0", installed = "1.12.0", canary = "meson" } [docs] # Please keep the installed versions in sync with docs/requirements.txt -sphinx = { accepted = ">=3.4.3", installed = "6.2.1", canary = "sphinx-build" } -sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.2.2" } +sphinx = { accepted = ">=7.2.6", installed = "7.2.6", canary = "sphinx-build" } +sphinx_rtd_theme = { accepted = ">=1.2.2", installed = "1.2.2" } # This test group is for dependencies required to run various tooling # and tests that should always be installed at configure time. It shouldSignature · 1 line
2.55.0
With regards,
Daniel
Signature · 3 lines
|: https://berrange.com ~~ https://hachyderm.io/@berrange :| |: https://libvirt.org ~~ https://entangle-photo.org :| |: https://pixelfed.art/berrange ~~ https://fstop138.berrange.com :|
Re: [PATCH 10/12] sphinx: require sphinx 7.2.6
Quoted 8 lines · Daniel P. Berrangé
Quoted 2 lines · John Snow
Make the extraordinary leap from Sphinx 3.4.3 to Sphinx 7.2.6. Simplifications and cleanup will follow in subsequent patches.
Does this have any implications on what platforms we build docs on ? If so, mention any notable impacted platforms in the commit message.
Technically not, since Paolo already dropped 3.9 from the CentOS container.
But, yes, CentOS9 will require sourcing Sphinx from PyPI and I'll update the python upgrade commit to mention this.
This commit itself doesn't actually restrict any of our supported platforms, see the cover letter version chart.
Quoted 121 lines
pythondeps.toml meson = { accepted = ">=1.12.0", installed = "1.12.0",
Quoted 95 lines
Signed-off-by John Snow <[email protected]>--- docs/conf.py | 4 ++-- python/setup.cfg | 2 +- python/tests/minreqs.txt | 29 ++++++++++++++++------------- pythondeps.toml | 4 ++-- 4 files changed, 21 insertions(+), 18 deletions(-)docs/conf.py +2
-2@@ -51,9 +51,9 @@ # If your documentation needs a minimal Sphinx version, state it here. # -# 3.4.3 is the oldest version of Sphinx that ships on a platform we +# 7.2.6 is the oldest version of Sphinx that ships on a platform we # pledge build support for. -needs_sphinx = '3.4.3' +needs_sphinx = '7.2.6' # Add any Sphinx extension module names here, as strings. They can be # extensions coming with Sphinx (named 'sphinx.ext.*') or your custompython/setup.cfg +1
-1@@ -41,7 +41,7 @@ devel = pylint >= 2.17.3 pytest >= 6.2.4 tox >= 3.18.0 - sphinx >= 3.4.3 + sphinx >= 7.2.6 # Provides qom-fuse functionality fuse =python/tests/minreqs.txt +16
-13@@ -12,7 +12,7 @@ # on PyPI that allows the test suite to pass. # Dependencies for qapidoc/qapi_domain et al -sphinx==3.4.3 +sphinx==7.2.6 # Dependencies for qemu.machine qemu.qmp==0.0.6@@ -48,23 +48,26 @@ platformdirs==2.2.0 tomlkit==0.10.1 # Transitive sphinx dependencies -Jinja2==2.7 -MarkupSafe==1.1.0 alabaster==0.7.1 -babel==1.3 -docutils==0.12 -imagesize==0.5.0 -packaging==14.0 -Pygments==2.0 -pytz==2014.9 -requests==2.5.0 -snowballstemmer==1.1 +babel==2.9 +certifi==2023.5.7 +charset-normalizer==2.0.0 +docutils==0.18.1 +idna==2.5 +imagesize==1.3 +Jinja2==3.0.0 +MarkupSafe==2.0 +packaging==21.0 +Pygments==2.14 +requests==2.25.0 +snowballstemmer==2.0 sphinxcontrib-applehelp==1.0.0 sphinxcontrib-devhelp==1.0.0 -sphinxcontrib-htmlhelp==1.0.0 +sphinxcontrib-htmlhelp==2.0.0 sphinxcontrib-jsmath==1.0.0 sphinxcontrib-qthelp==1.0.0 -sphinxcontrib-serializinghtml==1.0.0 +sphinxcontrib-serializinghtml==1.1.9 +urllib3==1.26 # Transitive pytest dependencies attrs==19.2.0pythondeps.toml +0
-0@@ -28,8 +28,8 @@ meson = { accepted = ">=1.12.0", installed = "1.12.0",canary = "meson" }
pythondeps.toml meson = { accepted = ">=1.12.0", installed = "1.12.0", canary = "meson" }
Quoted 4 lines
pythondeps.toml +0
-1[docs] # Please keep the installed versions in sync with docs/requirements.txt -sphinx = { accepted = ">=3.4.3", installed = "6.2.1", canary ="sphinx-build" }
pythondeps.toml meson = { accepted = ">=1.12.0", installed = "1.12.0", · line 30
Quoted 2 lines
pythondeps.toml +1
-1-sphinx_rtd_theme = { accepted = ">=0.5", installed = "1.2.2" } +sphinx = { accepted = ">=7.2.6", installed = "7.2.6", canary ="sphinx-build" }
Quoted 8 lines
pythondeps.toml +1
-1+sphinx_rtd_theme = { accepted = ">=1.2.2", installed = "1.2.2" } # This test group is for dependencies required to run various tooling # and tests that should always be installed at configure time. It should --2.55.0
With regards,
Daniel
--|: https://berrange.com ~~ https://hachyderm.io/@berrange :| |: https://libvirt.org ~~ https://entangle-photo.org :| |: https://pixelfed.art/berrange ~~ https://fstop138.berrange.com :|
PATCH 11/12 sphinx: drop fakedbusdoc
In reply to John SnowNow that we rely on a sufficiently modern Sphinx, we no longer need a stub doc for dbus.
--- docs/conf.py | 7 +------ docs/interop/dbus-display.rst | 14 +------------- docs/interop/dbus-vmstate.rst | 14 +------------- docs/interop/dbus-vnc.rst | 14 +------------- docs/sphinx/fakedbusdoc.py | 30 ------------------------------ 5 files changed, 4 insertions(+), 75 deletions(-) delete mode 100644 docs/sphinx/fakedbusdoc.py
docs/conf.py +1 -6
@@ -59,6 +59,7 @@ # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom # ones. extensions = [ + 'dbusdoc', 'depfile', 'hxtool', 'kerneldoc',
@@ -67,12 +68,6 @@ 'qmp_lexer', ] -if sphinx.version_info[:3] > (4, 0, 0): - tags.add('sphinx4') - extensions += ['dbusdoc'] -else: - extensions += ['fakedbusdoc'] - # Add any paths that contain templates here, relative to this directory. templates_path = [os.path.join(qemu_docdir, '_templates')]
docs/interop/dbus-display.rst +1 -13
@@ -18,16 +18,4 @@ QEMU also implements the standard interfaces, such as :local: :depth: 1 -.. only:: sphinx4 - - .. dbus-doc:: ui/dbus-display1.xml - -.. only:: not sphinx4 - - .. warning:: - Sphinx 4 is required to build D-Bus documentation. - - This is the content of ``ui/dbus-display1.xml``: - - .. literalinclude:: ../../ui/dbus-display1.xml - :language: xml +.. dbus-doc:: ui/dbus-display1.xml
docs/interop/dbus-vmstate.rst +1 -13
@@ -23,16 +23,4 @@ dbus-vmstate object can be configured with the expected list of helpers by setting its ``id-list`` property, with a comma-separated ``Id`` list. -.. only:: sphinx4 - - .. dbus-doc:: backends/dbus-vmstate1.xml - -.. only:: not sphinx4 - - .. warning:: - Sphinx 4 is required to build D-Bus documentation. - - This is the content of ``backends/dbus-vmstate1.xml``: - - .. literalinclude:: ../../backends/dbus-vmstate1.xml - :language: xml +.. dbus-doc:: backends/dbus-vmstate1.xml
docs/interop/dbus-vnc.rst +1 -13
@@ -11,16 +11,4 @@ Objects are exported under ``/org/qemu/Vnc1/``. :local: :depth: 1 -.. only:: sphinx4 - - .. dbus-doc:: tools/qemu-vnc/qemu-vnc1.xml - -.. only:: not sphinx4 - - .. warning:: - Sphinx 4 is required to build D-Bus documentation. - - This is the content of ``tools/qemu-vnc/qemu-vnc1.xml``: - - .. literalinclude:: ../../tools/qemu-vnc/qemu-vnc1.xml - :language: xml +.. dbus-doc:: tools/qemu-vnc/qemu-vnc1.xml
docs/sphinx/fakedbusdoc.py deleted +0 -30
@@ -1,30 +0,0 @@ -# D-Bus XML documentation extension, compatibility gunk for <sphinx4 -# -# Copyright (C) 2021, Red Hat Inc. -# -# SPDX-License-Identifier: LGPL-2.1-or-later -# -# Author: Marc-André Lureau <[email protected]> -"""dbus-doc is a Sphinx extension that provides documentation from D-Bus XML.""" - -from docutils.parsers.rst import Directive -from sphinx.application import Sphinx -from typing import Any, Dict - - -class FakeDBusDocDirective(Directive): - has_content = True - required_arguments = 1 - - def run(self): - return [] - - -def setup(app: Sphinx) -> Dict[str, Any]: - """Register a fake dbus-doc directive with Sphinx""" - app.add_directive("dbus-doc", FakeDBusDocDirective) - - return dict( - parallel_read_safe = True, - parallel_write_safe = True - )
Signature · 1 line
2.55.0
Re: [PATCH 11/12] sphinx: drop fakedbusdoc
Quoted 2 lines · John Snow
Now that we rely on a sufficiently modern Sphinx, we no longer need a stub doc for dbus.
Signature · 1 line
Marc-André Lureau <[email protected]>
PATCH 12/12 sphinx: drop qapidoc back-compat overboiled spaghetti dinner
In reply to John SnowDrop the disgusting amount of hacks required to keep qapidoc working on older Sphinx versions, enjoying the cutting edge technology of 2023.
This felt like an achievement to have written, and equally feels an achievement to have deleted.
--- docs/sphinx/compat.py | 230 ------------------------------------- docs/sphinx/qapi_domain.py | 47 ++++---- 2 files changed, 22 insertions(+), 255 deletions(-) delete mode 100644 docs/sphinx/compat.py
docs/sphinx/compat.py deleted +0 -230
@@ -1,230 +0,0 @@ -""" -Sphinx cross-version compatibility goop -""" - -import re -from typing import ( - TYPE_CHECKING, - Any, - Callable, - Optional, - Type, -) - -from docutils import nodes -from docutils.nodes import Element, Node, Text -from docutils.statemachine import StringList - -import sphinx -from sphinx import addnodes, util -from sphinx.directives import ObjectDescription -from sphinx.environment import BuildEnvironment -from sphinx.roles import XRefRole -from sphinx.util import docfields -from sphinx.util.docutils import ( - ReferenceRole, - SphinxDirective, - switch_source_input, -) -from sphinx.util.typing import TextlikeNode - - -MAKE_XREF_WORKAROUND = sphinx.version_info[:3] < (4, 1, 0) - - -SpaceNode: Callable[[str], Node] -KeywordNode: Callable[[str, str], Node] - -if sphinx.version_info[:3] >= (4, 0, 0): - SpaceNode = addnodes.desc_sig_space - KeywordNode = addnodes.desc_sig_keyword -else: - SpaceNode = Text - KeywordNode = addnodes.desc_annotation - - -def nested_parse_with_titles( - directive: SphinxDirective, content_node: Element -) -> None: - """ - This helper preserves error parsing context across sphinx versions. - """ - - # necessary so that the child nodes get the right source/line set - content_node.document = directive.state.document - - try: - # Modern sphinx (6.2.0+) supports proper offsetting for - # nested parse error context management - util.nodes.nested_parse_with_titles( - directive.state, - directive.content, - content_node, - content_offset=directive.content_offset, - ) - except TypeError: - # No content_offset argument. Fall back to SSI method. - with switch_source_input(directive.state, directive.content): - util.nodes.nested_parse_with_titles( - directive.state, directive.content, content_node - ) - - -# ########################################### -# xref compatibility hacks for Sphinx < 4.1 # -# ########################################### - -# When we require >= Sphinx 4.1, the following function and the -# subsequent 3 compatibility classes can be removed. Anywhere in -# qapi_domain that uses one of these Compat* types can be switched to -# using the garden-variety lib-provided classes with no trickery. - - -def _compat_make_xref( # pylint: disable=unused-argument - self: sphinx.util.docfields.Field, - rolename: str, - domain: str, - target: str, - innernode: Type[TextlikeNode] = addnodes.literal_emphasis, - contnode: Optional[Node] = None, - env: Optional[BuildEnvironment] = None, - inliner: Any = None, - location: Any = None, -) -> Node: - """ - Compatibility workaround for Sphinx versions prior to 4.1.0. - - Older sphinx versions do not use the domain's XRefRole for parsing - and formatting cross-references, so we need to perform this magick - ourselves to avoid needing to write the parser/formatter in two - separate places. - - This workaround isn't brick-for-brick compatible with modern Sphinx - versions, because we do not have access to the parent directive's - state during this parsing like we do in more modern versions. - - It's no worse than what pre-Sphinx 4.1.0 does, so... oh well! - """ - - # Yes, this function is gross. Pre-4.1 support is a miracle. - # pylint: disable=too-many-locals - - assert env - # Note: Sphinx's own code ignores the type warning here, too. - if not rolename: - return contnode or innernode(target, target) # type: ignore[call-arg] - - # Get the role instance, but don't *execute it* - we lack the - # correct state to do so. Instead, we'll just use its public - # methods to do our reference formatting, and emulate the rest. - role = env.get_domain(domain).roles[rolename] - assert isinstance(role, XRefRole) - - # XRefRole features not supported by this compatibility shim; - # these were not supported in Sphinx 3.x either, so nothing of - # value is really lost. - assert not target.startswith("!") - assert not re.match(ReferenceRole.explicit_title_re, target) - assert not role.lowercase - assert not role.fix_parens - - # Code below based mostly on sphinx.roles.XRefRole; run() and - # create_xref_node() - options = { - "refdoc": env.docname, - "refdomain": domain, - "reftype": rolename, - "refexplicit": False, - "refwarn": role.warn_dangling, - } - refnode = role.nodeclass(target, **options) - title, target = role.process_link(env, refnode, False, target, target) - refnode["reftarget"] = target - classes = ["xref", domain, f"{domain}-{rolename}"] - refnode += role.innernodeclass(target, title, classes=classes) - - # This is the very gross part of the hack. Normally, - # result_nodes takes a document object to which we would pass - # self.inliner.document. Prior to Sphinx 4.1, we don't *have* an - # inliner to pass, so we have nothing to pass here. However, the - # actual implementation of role.result_nodes in this case - # doesn't actually use that argument, so this winds up being - # ... fine. Rest easy at night knowing this code only runs under - # old versions of Sphinx, so at least it won't change in the - # future on us and lead to surprising new failures. - # Gross, I know. - result_nodes, _messages = role.result_nodes( - None, # type: ignore - env, - refnode, - is_ref=True, - ) - return nodes.inline(target, "", *result_nodes) - - -class CompatField(docfields.Field): - if MAKE_XREF_WORKAROUND: - make_xref = _compat_make_xref - - -class CompatGroupedField(docfields.GroupedField): - if MAKE_XREF_WORKAROUND: - make_xref = _compat_make_xref - - -class CompatTypedField(docfields.TypedField): - if MAKE_XREF_WORKAROUND: - make_xref = _compat_make_xref - - -# ################################################################ -# Nested parsing error location fix for Sphinx 5.3.0 < x < 6.2.0 # -# ################################################################ - -# When we require Sphinx 4.x, the TYPE_CHECKING hack where we avoid -# subscripting ObjectDescription at runtime can be removed in favor of -# just always subscripting the class. - -# When we require Sphinx > 6.2.0, the rest of this compatibility hack -# can be dropped and QAPIObject can just inherit directly from -# ObjectDescription[Signature]. - -SOURCE_LOCATION_FIX = (5, 3, 0) <= sphinx.version_info[:3] < (6, 2, 0) - -Signature = str - - -if TYPE_CHECKING: - _BaseClass = ObjectDescription[Signature] -else: - _BaseClass = ObjectDescription - - -class ParserFix(_BaseClass): - - _temp_content: StringList - _temp_offset: int - _temp_node: Optional[addnodes.desc_content] - - def before_content(self) -> None: - # Work around a sphinx bug and parse the content ourselves. - self._temp_content = self.content - self._temp_offset = self.content_offset - self._temp_node = None - - if SOURCE_LOCATION_FIX: - self._temp_node = addnodes.desc_content() - self.state.nested_parse( - self.content, self.content_offset, self._temp_node - ) - # Sphinx will try to parse the content block itself, - # Give it nothingness to parse instead. - self.content = StringList() - self.content_offset = 0 - - def transform_content(self, content_node: addnodes.desc_content) -> None: - # Sphinx workaround: Inject our parsed content and restore state. - if self._temp_node: - content_node += self._temp_node.children - self.content = self._temp_content - self.content_offset = self._temp_offset
docs/sphinx/qapi_domain.py +22 -25
134 lines not on this page · raw message
Signature · 1 line
2.55.0
Re: [PATCH 12/12] sphinx: drop qapidoc back-compat overboiled spaghetti dinner
Quoted 8 lines · John Snow
Drop the disgusting amount of hacks required to keep qapidoc working on older Sphinx versions, enjoying the cutting edge technology of 2023.
This felt like an achievement to have written, and equally feels an achievement to have deleted.
[...]
Signature · 1 line
Marc-André Lureau <[email protected]>
Re: [PATCH 12/12] sphinx: drop qapidoc back-compat overboiled spaghetti dinner
In reply to John SnowQuoted 6 lines · John Snow
Drop the disgusting amount of hacks required to keep qapidoc working on older Sphinx versions, enjoying the cutting edge technology of 2023.
This felt like an achievement to have written, and equally feels an achievement to have deleted.
It was, and it is.
docs/sphinx/compat.py is black magic. It was difficult and expensive to write, and it would be difficult and expensive to modify. More so (much more so) for anybody not John Snow. Since it can go now, it must.
Quoted 312 lines
Signed-off-by John Snow <[email protected]>--- docs/sphinx/compat.py | 230 ------------------------------------- docs/sphinx/qapi_domain.py | 47 ++++---- 2 files changed, 22 insertions(+), 255 deletions(-) delete mode 100644 docs/sphinx/compat.pydocs/sphinx/compat.py deleted +0
-230@@ -1,230 +0,0 @@ -""" -Sphinx cross-version compatibility goop -""" - -import re -from typing import ( - TYPE_CHECKING, - Any, - Callable, - Optional, - Type, -) - -from docutils import nodes -from docutils.nodes import Element, Node, Text -from docutils.statemachine import StringList - -import sphinx -from sphinx import addnodes, util -from sphinx.directives import ObjectDescription -from sphinx.environment import BuildEnvironment -from sphinx.roles import XRefRole -from sphinx.util import docfields -from sphinx.util.docutils import ( - ReferenceRole, - SphinxDirective, - switch_source_input, -) -from sphinx.util.typing import TextlikeNode - - -MAKE_XREF_WORKAROUND = sphinx.version_info[:3] < (4, 1, 0) - - -SpaceNode: Callable[[str], Node] -KeywordNode: Callable[[str, str], Node] - -if sphinx.version_info[:3] >= (4, 0, 0): - SpaceNode = addnodes.desc_sig_space - KeywordNode = addnodes.desc_sig_keyword -else: - SpaceNode = Text - KeywordNode = addnodes.desc_annotation - - -def nested_parse_with_titles( - directive: SphinxDirective, content_node: Element -) -> None: - """ - This helper preserves error parsing context across sphinx versions. - """ - - # necessary so that the child nodes get the right source/line set - content_node.document = directive.state.document - - try: - # Modern sphinx (6.2.0+) supports proper offsetting for - # nested parse error context management - util.nodes.nested_parse_with_titles( - directive.state, - directive.content, - content_node, - content_offset=directive.content_offset, - ) - except TypeError: - # No content_offset argument. Fall back to SSI method. - with switch_source_input(directive.state, directive.content): - util.nodes.nested_parse_with_titles( - directive.state, directive.content, content_node - ) - - -# ########################################### -# xref compatibility hacks for Sphinx < 4.1 # -# ########################################### - -# When we require >= Sphinx 4.1, the following function and the -# subsequent 3 compatibility classes can be removed. Anywhere in -# qapi_domain that uses one of these Compat* types can be switched to -# using the garden-variety lib-provided classes with no trickery. - - -def _compat_make_xref( # pylint: disable=unused-argument - self: sphinx.util.docfields.Field, - rolename: str, - domain: str, - target: str, - innernode: Type[TextlikeNode] = addnodes.literal_emphasis, - contnode: Optional[Node] = None, - env: Optional[BuildEnvironment] = None, - inliner: Any = None, - location: Any = None, -) -> Node: - """ - Compatibility workaround for Sphinx versions prior to 4.1.0. - - Older sphinx versions do not use the domain's XRefRole for parsing - and formatting cross-references, so we need to perform this magick - ourselves to avoid needing to write the parser/formatter in two - separate places. - - This workaround isn't brick-for-brick compatible with modern Sphinx - versions, because we do not have access to the parent directive's - state during this parsing like we do in more modern versions. - - It's no worse than what pre-Sphinx 4.1.0 does, so... oh well! - """ - - # Yes, this function is gross. Pre-4.1 support is a miracle. - # pylint: disable=too-many-locals - - assert env - # Note: Sphinx's own code ignores the type warning here, too. - if not rolename: - return contnode or innernode(target, target) # type: ignore[call-arg] - - # Get the role instance, but don't *execute it* - we lack the - # correct state to do so. Instead, we'll just use its public - # methods to do our reference formatting, and emulate the rest. - role = env.get_domain(domain).roles[rolename] - assert isinstance(role, XRefRole) - - # XRefRole features not supported by this compatibility shim; - # these were not supported in Sphinx 3.x either, so nothing of - # value is really lost. - assert not target.startswith("!") - assert not re.match(ReferenceRole.explicit_title_re, target) - assert not role.lowercase - assert not role.fix_parens - - # Code below based mostly on sphinx.roles.XRefRole; run() and - # create_xref_node() - options = { - "refdoc": env.docname, - "refdomain": domain, - "reftype": rolename, - "refexplicit": False, - "refwarn": role.warn_dangling, - } - refnode = role.nodeclass(target, **options) - title, target = role.process_link(env, refnode, False, target, target) - refnode["reftarget"] = target - classes = ["xref", domain, f"{domain}-{rolename}"] - refnode += role.innernodeclass(target, title, classes=classes) - - # This is the very gross part of the hack. Normally, - # result_nodes takes a document object to which we would pass - # self.inliner.document. Prior to Sphinx 4.1, we don't *have* an - # inliner to pass, so we have nothing to pass here. However, the - # actual implementation of role.result_nodes in this case - # doesn't actually use that argument, so this winds up being - # ... fine. Rest easy at night knowing this code only runs under - # old versions of Sphinx, so at least it won't change in the - # future on us and lead to surprising new failures. - # Gross, I know. - result_nodes, _messages = role.result_nodes( - None, # type: ignore - env, - refnode, - is_ref=True, - ) - return nodes.inline(target, "", *result_nodes) - - -class CompatField(docfields.Field): - if MAKE_XREF_WORKAROUND: - make_xref = _compat_make_xref - - -class CompatGroupedField(docfields.GroupedField): - if MAKE_XREF_WORKAROUND: - make_xref = _compat_make_xref - - -class CompatTypedField(docfields.TypedField): - if MAKE_XREF_WORKAROUND: - make_xref = _compat_make_xref - - -# ################################################################ -# Nested parsing error location fix for Sphinx 5.3.0 < x < 6.2.0 # -# ################################################################ - -# When we require Sphinx 4.x, the TYPE_CHECKING hack where we avoid -# subscripting ObjectDescription at runtime can be removed in favor of -# just always subscripting the class. - -# When we require Sphinx > 6.2.0, the rest of this compatibility hack -# can be dropped and QAPIObject can just inherit directly from -# ObjectDescription[Signature]. - -SOURCE_LOCATION_FIX = (5, 3, 0) <= sphinx.version_info[:3] < (6, 2, 0) - -Signature = str - - -if TYPE_CHECKING: - _BaseClass = ObjectDescription[Signature] -else: - _BaseClass = ObjectDescription - - -class ParserFix(_BaseClass): - - _temp_content: StringList - _temp_offset: int - _temp_node: Optional[addnodes.desc_content] - - def before_content(self) -> None: - # Work around a sphinx bug and parse the content ourselves. - self._temp_content = self.content - self._temp_offset = self.content_offset - self._temp_node = None - - if SOURCE_LOCATION_FIX: - self._temp_node = addnodes.desc_content() - self.state.nested_parse( - self.content, self.content_offset, self._temp_node - ) - # Sphinx will try to parse the content block itself, - # Give it nothingness to parse instead. - self.content = StringList() - self.content_offset = 0 - - def transform_content(self, content_node: addnodes.desc_content) -> None: - # Sphinx workaround: Inject our parsed content and restore state. - if self._temp_node: - content_node += self._temp_node.children - self.content = self._temp_content - self.content_offset = self._temp_offsetdocs/sphinx/qapi_domain.py +9
-16@@ -30,20 +30,10 @@ ) from sphinx.locale import _, __ from sphinx.roles import XRefRole -from sphinx.util import logging +from sphinx.util import docfields, logging from sphinx.util.docutils import SphinxDirective from sphinx.util.nodes import make_id, make_refnode -from compat import ( - CompatField, - CompatGroupedField, - CompatTypedField, - KeywordNode, - ParserFix, - Signature, - SpaceNode, -) - if TYPE_CHECKING: from typing import (@@ -157,7 +147,10 @@ def result_nodes( return results, [] -class QAPIDescription(ParserFix): +Signature = str + + +class QAPIDescription(ObjectDescription[Signature]): """ Generic QAPI description.@@ -315,7 +308,7 @@ class QAPIObject(QAPIDescription): doc_field_types = [ # :feat name: descr - CompatGroupedField( + docfields.GroupedField( "feature", label=_("Features"), names=("feat",),@@ -327,8 +320,8 @@ def get_signature_prefix(self) -> List[nodes.Node]: """Return a prefix to put before the object name in the signature.""" assert self.objtype return [ - KeywordNode("", self.objtype.title()), - SpaceNode(" "), + addnodes.desc_sig_keyword("", self.objtype.title()), + addnodes.desc_sig_space(" "), ] def get_signature_suffix(self) -> List[nodes.Node]:@@ -337,7 +330,7 @@ def get_signature_suffix(self) -> List[nodes.Node]: if "since" in self.options: ret += [ - SpaceNode(" "), + addnodes.desc_sig_space(" "), addnodes.desc_sig_element( "", f"(Since: {self.options['since']})" ),
docs/sphinx/qapi_domain.py def handle_signature(self, sig: str, signode: desc_signature) -> Signature: · line 375
Quoted 8 lines
docs/sphinx/qapi_domain.py +1
-1@@ -376,7 +369,7 @@ def handle_signature(self, sig: str, signode: desc_signature) -> Signature: return sig - def _add_infopips(self, contentnode: addnodes.desc_content) -> None: + def _add_infopips(self, content_node: addnodes.desc_content) -> None: # Add various eye-catches and things that go below the signature # bar, but precede the user-defined content. infopips = nodes.container()
Are this hunk and the next one necessary parts of the patch, or have they crept in?
Quoted 12 lines
docs/sphinx/qapi_domain.py +1
-1@@ -420,7 +413,7 @@ def _add_pip( ) if infopips.children: - contentnode.insert(0, infopips) + content_node.insert(0, infopips) def _validate_field(self, field: nodes.field) -> None: """Validate field lists in this QAPI Object Description."""@@ -470,7 +463,11 @@ def _validate_field(self, field: nodes.field) -> None: ) logger.warning(msg, location=field)
docs/sphinx/qapi_domain.py def _validate_field(self, field: nodes.field) -> None: · line 473
Quoted 10 lines
docs/sphinx/qapi_domain.py +5
-1- def transform_content(self, content_node: addnodes.desc_content) -> None: + def transform_content( + self, + # pylint: disable=arguments-renamed + content_node: addnodes.desc_content + ) -> None: # This hook runs after before_content and the nested parse, but # before the DocFieldTransformer is executed. super().transform_content(content_node)
Likewise.
Quoted 53 lines
docs/sphinx/qapi_domain.py +6
-6@@ -485,7 +482,7 @@ def transform_content(self, content_node: addnodes.desc_content) -> None: self._validate_field(field) -class SpecialTypedField(CompatTypedField): +class SpecialTypedField(docfields.TypedField): def make_field(self, *args: Any, **kwargs: Any) -> nodes.field: ret = super().make_field(*args, **kwargs)@@ -518,14 +515,14 @@ class QAPICommand(QAPIObject): can_collapse=False, ), # :error: descr - CompatField( + docfields.Field( "error", label=_("Errors"), names=("error", "errors"), has_arg=False, ), # :return TypeName: descr - CompatGroupedField( + docfields.GroupedField( "returnvalue", label=_("Return"), rolename="type",@@ -533,7 +530,7 @@ class QAPICommand(QAPIObject): can_collapse=True, ), # :return-nodesc: TypeName - CompatField( + docfields.Field( "returnvalue", label=_("Return"), names=("return-nodesc",),@@ -551,7 +548,7 @@ class QAPIEnum(QAPIObject): doc_field_types.extend( [ # :value name: descr - CompatGroupedField( + docfields.GroupedField( "value", label=_("Values"), names=("value",),@@ -568,7 +565,7 @@ class QAPIAlternate(QAPIObject): doc_field_types.extend( [ # :alt type name: descr - CompatTypedField( + docfields.TypedField( "alternative", label=_("Alternatives"), names=("alt",),
Re: [PATCH 12/12] sphinx: drop qapidoc back-compat overboiled spaghetti dinner
Quoted 331 lines · Markus Armbruster
Quoted 6 lines · John Snow
Drop the disgusting amount of hacks required to keep qapidoc working on older Sphinx versions, enjoying the cutting edge technology of 2023.
This felt like an achievement to have written, and equally feels an achievement to have deleted.
It was, and it is.
docs/sphinx/compat.pyis black magic. It was difficult and expensive to write, and it would be difficult and expensive to modify. More so (much more so) for anybody not John Snow. Since it can go now, it must.Quoted 315 lines
Signed-off-by John Snow <[email protected]>--- docs/sphinx/compat.py | 230 ------------------------------------- docs/sphinx/qapi_domain.py | 47 ++++---- 2 files changed, 22 insertions(+), 255 deletions(-) delete mode 100644 docs/sphinx/compat.pydocs/sphinx/compat.py deleted +0
-230@@ -1,230 +0,0 @@ -""" -Sphinx cross-version compatibility goop -""" - -import re -from typing import ( - TYPE_CHECKING, - Any, - Callable, - Optional, - Type, -) - -from docutils import nodes -from docutils.nodes import Element, Node, Text -from docutils.statemachine import StringList - -import sphinx -from sphinx import addnodes, util -from sphinx.directives import ObjectDescription -from sphinx.environment import BuildEnvironment -from sphinx.roles import XRefRole -from sphinx.util import docfields -from sphinx.util.docutils import ( - ReferenceRole, - SphinxDirective, - switch_source_input, -) -from sphinx.util.typing import TextlikeNode - - -MAKE_XREF_WORKAROUND = sphinx.version_info[:3] < (4, 1, 0) - - -SpaceNode: Callable[[str], Node] -KeywordNode: Callable[[str, str], Node] - -if sphinx.version_info[:3] >= (4, 0, 0): - SpaceNode = addnodes.desc_sig_space - KeywordNode = addnodes.desc_sig_keyword -else: - SpaceNode = Text - KeywordNode = addnodes.desc_annotation - - -def nested_parse_with_titles( - directive: SphinxDirective, content_node: Element -) -> None: - """ - This helper preserves error parsing context across sphinx versions. - """ - - # necessary so that the child nodes get the right source/line set - content_node.document = directive.state.document - - try: - # Modern sphinx (6.2.0+) supports proper offsetting for - # nested parse error context management - util.nodes.nested_parse_with_titles( - directive.state, - directive.content, - content_node, - content_offset=directive.content_offset, - ) - except TypeError: - # No content_offset argument. Fall back to SSI method. - with switch_source_input(directive.state, directive.content): - util.nodes.nested_parse_with_titles( - directive.state, directive.content, content_node - ) - - -# ########################################### -# xref compatibility hacks for Sphinx < 4.1 # -# ########################################### - -# When we require >= Sphinx 4.1, the following function and the -# subsequent 3 compatibility classes can be removed. Anywhere in -# qapi_domain that uses one of these Compat* types can be switched to -# using the garden-variety lib-provided classes with no trickery. - - -def _compat_make_xref( # pylint: disable=unused-argument - self: sphinx.util.docfields.Field, - rolename: str, - domain: str, - target: str, - innernode: Type[TextlikeNode] = addnodes.literal_emphasis, - contnode: Optional[Node] = None, - env: Optional[BuildEnvironment] = None, - inliner: Any = None, - location: Any = None, -) -> Node: - """ - Compatibility workaround for Sphinx versions prior to 4.1.0. - - Older sphinx versions do not use the domain's XRefRole for parsing - and formatting cross-references, so we need to perform this magick - ourselves to avoid needing to write the parser/formatter in two - separate places. - - This workaround isn't brick-for-brick compatible with modern Sphinx - versions, because we do not have access to the parent directive's - state during this parsing like we do in more modern versions. - - It's no worse than what pre-Sphinx 4.1.0 does, so... oh well! - """ - - # Yes, this function is gross. Pre-4.1 support is a miracle. - # pylint: disable=too-many-locals - - assert env - # Note: Sphinx's own code ignores the type warning here, too. - if not rolename: - return contnode or innernode(target, target) # type: ignore[call-arg] - - # Get the role instance, but don't *execute it* - we lack the - # correct state to do so. Instead, we'll just use its public - # methods to do our reference formatting, and emulate the rest. - role = env.get_domain(domain).roles[rolename] - assert isinstance(role, XRefRole) - - # XRefRole features not supported by this compatibility shim; - # these were not supported in Sphinx 3.x either, so nothing of - # value is really lost. - assert not target.startswith("!") - assert not re.match(ReferenceRole.explicit_title_re, target) - assert not role.lowercase - assert not role.fix_parens - - # Code below based mostly on sphinx.roles.XRefRole; run() and - # create_xref_node() - options = { - "refdoc": env.docname, - "refdomain": domain, - "reftype": rolename, - "refexplicit": False, - "refwarn": role.warn_dangling, - } - refnode = role.nodeclass(target, **options) - title, target = role.process_link(env, refnode, False, target, target) - refnode["reftarget"] = target - classes = ["xref", domain, f"{domain}-{rolename}"] - refnode += role.innernodeclass(target, title, classes=classes) - - # This is the very gross part of the hack. Normally, - # result_nodes takes a document object to which we would pass - # self.inliner.document. Prior to Sphinx 4.1, we don't *have* an - # inliner to pass, so we have nothing to pass here. However, the - # actual implementation of role.result_nodes in this case - # doesn't actually use that argument, so this winds up being - # ... fine. Rest easy at night knowing this code only runs under - # old versions of Sphinx, so at least it won't change in the - # future on us and lead to surprising new failures. - # Gross, I know. - result_nodes, _messages = role.result_nodes( - None, # type: ignore - env, - refnode, - is_ref=True, - ) - return nodes.inline(target, "", *result_nodes) - - -class CompatField(docfields.Field): - if MAKE_XREF_WORKAROUND: - make_xref = _compat_make_xref - - -class CompatGroupedField(docfields.GroupedField): - if MAKE_XREF_WORKAROUND: - make_xref = _compat_make_xref - - -class CompatTypedField(docfields.TypedField): - if MAKE_XREF_WORKAROUND: - make_xref = _compat_make_xref - - -# ################################################################ -# Nested parsing error location fix for Sphinx 5.3.0 < x < 6.2.0 # -# ################################################################ - -# When we require Sphinx 4.x, the TYPE_CHECKING hack where we avoid -# subscripting ObjectDescription at runtime can be removed in favor of -# just always subscripting the class. - -# When we require Sphinx > 6.2.0, the rest of this compatibility hack -# can be dropped and QAPIObject can just inherit directly from -# ObjectDescription[Signature]. - -SOURCE_LOCATION_FIX = (5, 3, 0) <= sphinx.version_info[:3] < (6, 2, 0) - -Signature = str - - -if TYPE_CHECKING: - _BaseClass = ObjectDescription[Signature] -else: - _BaseClass = ObjectDescription - - -class ParserFix(_BaseClass): - - _temp_content: StringList - _temp_offset: int - _temp_node: Optional[addnodes.desc_content] - - def before_content(self) -> None: - # Work around a sphinx bug and parse the content ourselves. - self._temp_content = self.content - self._temp_offset = self.content_offset - self._temp_node = None - - if SOURCE_LOCATION_FIX: - self._temp_node = addnodes.desc_content() - self.state.nested_parse( - self.content, self.content_offset, self._temp_node - ) - # Sphinx will try to parse the content block itself, - # Give it nothingness to parse instead. - self.content = StringList() - self.content_offset = 0 - - def transform_content(self, content_node: addnodes.desc_content) -> None: - # Sphinx workaround: Inject our parsed content and restore state. - if self._temp_node: - content_node += self._temp_node.children - self.content = self._temp_content - self.content_offset = self._temp_offsetdocs/sphinx/qapi_domain.py +9
-16@@ -30,20 +30,10 @@ ) from sphinx.locale import _, __ from sphinx.roles import XRefRole -from sphinx.util import logging +from sphinx.util import docfields, logging from sphinx.util.docutils import SphinxDirective from sphinx.util.nodes import make_id, make_refnode -from compat import ( - CompatField, - CompatGroupedField, - CompatTypedField, - KeywordNode, - ParserFix, - Signature, - SpaceNode, -) - if TYPE_CHECKING: from typing import (@@ -157,7 +147,10 @@ def result_nodes( return results, [] -class QAPIDescription(ParserFix): +Signature = str + + +class QAPIDescription(ObjectDescription[Signature]): """ Generic QAPI description.@@ -315,7 +308,7 @@ class QAPIObject(QAPIDescription): doc_field_types = [ # :feat name: descr - CompatGroupedField( + docfields.GroupedField( "feature", label=_("Features"), names=("feat",),@@ -327,8 +320,8 @@ def get_signature_prefix(self) -> List[nodes.Node]: """Return a prefix to put before the object name in the signature.""" assert self.objtype return [ - KeywordNode("", self.objtype.title()), - SpaceNode(" "), + addnodes.desc_sig_keyword("", self.objtype.title()), + addnodes.desc_sig_space(" "), ] def get_signature_suffix(self) -> List[nodes.Node]:@@ -337,7 +330,7 @@ def get_signature_suffix(self) -> List[nodes.Node]: if "since" in self.options: ret += [ - SpaceNode(" "), + addnodes.desc_sig_space(" "), addnodes.desc_sig_element( "", f"(Since: {self.options['since']})" ),@@ -376,7 +369,7 @@ def handle_signature(self, sig: str, signode: desc_signature) -> Signature: return sig
Quoted 8 lines
docs/sphinx/qapi_domain.py def handle_signature(self, sig: str, signode: desc_signature) -> Signature: · line 375
Quoted 5 lines
docs/sphinx/qapi_domain.py +1
-1- def _add_infopips(self, contentnode: addnodes.desc_content) -> None: + def _add_infopips(self, content_node: addnodes.desc_content) -> None: # Add various eye-catches and things that go below the signature # bar, but precede the user-defined content. infopips = nodes.container()Are this hunk and the next one necessary parts of the patch, or have they crept in?
Necessary. The newer Sphinx version changes these parameter names, so type checking and linters begin to complain about us changing the parameter name. This block is fairly straightforward, ...
Quoted 14 lines
Quoted 14 lines
docs/sphinx/qapi_domain.py +1
-1@@ -420,7 +413,7 @@ def _add_pip( ) if infopips.children: - contentnode.insert(0, infopips) + content_node.insert(0, infopips) def _validate_field(self, field: nodes.field) -> None: """Validate field lists in this QAPI Object Description."""@@ -470,7 +463,11 @@ def _validate_field(self, field: nodes.field) -> None: ) logger.warning(msg, location=field)
Quoted 11 lines
docs/sphinx/qapi_domain.py def _validate_field(self, field: nodes.field) -> None: · line 473
Quoted 9 lines
docs/sphinx/qapi_domain.py +5
-1- def transform_content(self, content_node: addnodes.desc_content) -> None: + def transform_content( + self, + # pylint: disable=arguments-renamed + content_node: addnodes.desc_content + ) -> None: # This hook runs after before_content and the nested parse, but # before the DocFieldTransformer is executed. super().transform_content(content_node)Likewise.
This block is less straightforward: the argument/parameter name here was changed in the versions we still support, so either name I choose for it causes errors on either the older or newer end of the tests. Adopt the eventual name, and silence the error on the minreqs test.
Quoted 55 lines
Quoted 54 lines
docs/sphinx/qapi_domain.py +6
-6@@ -485,7 +482,7 @@ def transform_content(self, content_node: addnodes.desc_content) -> None: self._validate_field(field) -class SpecialTypedField(CompatTypedField): +class SpecialTypedField(docfields.TypedField): def make_field(self, *args: Any, **kwargs: Any) -> nodes.field: ret = super().make_field(*args, **kwargs)@@ -518,14 +515,14 @@ class QAPICommand(QAPIObject): can_collapse=False, ), # :error: descr - CompatField( + docfields.Field( "error", label=_("Errors"), names=("error", "errors"), has_arg=False, ), # :return TypeName: descr - CompatGroupedField( + docfields.GroupedField( "returnvalue", label=_("Return"), rolename="type",@@ -533,7 +530,7 @@ class QAPICommand(QAPIObject): can_collapse=True, ), # :return-nodesc: TypeName - CompatField( + docfields.Field( "returnvalue", label=_("Return"), names=("return-nodesc",),@@ -551,7 +548,7 @@ class QAPIEnum(QAPIObject): doc_field_types.extend( [ # :value name: descr - CompatGroupedField( + docfields.GroupedField( "value", label=_("Values"), names=("value",),@@ -568,7 +565,7 @@ class QAPIAlternate(QAPIObject): doc_field_types.extend( [ # :alt type name: descr - CompatTypedField( + docfields.TypedField( "alternative", label=_("Alternatives"), names=("alt",),