Skip to content

python: require python3.12+

John Snow qemu-devel 3 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest16 patches 1 replyIn review
  2. v216 patches 13 repliesSuperseded
  3. v112 patches 25 repliesSuperseded

Changes in v3 from the cover letter

- 1/16: s/vm-run-xxx/vm-build-xxx in commit msg
- 1/16: Add "Fixes: 05e4982" to commit msg
- Drop "tests: update FreeBSD dependencies" patch and replace with "tests: update libvirt-ci, lcitool" (thanks, dan)
- Add r-bs, rebase, etc.

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/16tests: update vm tests to prefer locally-built QEMUtests/vm/Makefile.include+1 -10 replies
2/16tests: update libvirt-ci, lcitoolscripts/ci/setup/debian/debian-13-ppc64le.yaml scripts/ci/setup/ubuntu/ubuntu-2404-aarch64.yaml scripts/ci/setup/ubuntu/ubuntu-2404-s390x.yaml +12 more+20 -201 reply
3/16tests: update netbsd dependenciestests/vm/netbsd+1 -00 replies
4/16tests: drop python3-devel mappingtests/lcitool/mappings.yml+0 -30 replies
5/16tests: re-add opencv2 and numpy dependencies.gitlab-ci.d/macos-14.vars scripts/ci/setup/debian/debian-13-ppc64le.yaml scripts/ci/setup/ubuntu/ubuntu-2404-aarch64.yaml +19 more+52 -10 replies
6/16tests: add Python3.12 to centos9.dockertests/docker/dockerfiles/centos9.docker tests/lcitool/mappings.yml tests/lcitool/targets/centos-stream-9.yml+13 -130 replies
7/16tests: update emsdk-wasm64-crosstests/docker/dockerfiles/emsdk-wasm64-cross.docker+2 -20 replies
8/16qapi: update type hint for objects_seenscripts/qapi/types.py+1 -10 replies
9/16python: upgrade to Python3.12+configure python/Makefile python/scripts/mkvenv.py +3 more+43 -1080 replies
10/16python: fix license configurationpython/setup.cfg+1 -10 replies
11/16python: allow mypy to run as the current python versionpython/setup.cfg+0 -10 replies
12/16python: remove distutils workaroundspython/tests/linters.py+0 -110 replies
13/16python: remove stale setup.cfg optionspython/setup.cfg+0 -130 replies
14/16sphinx: require sphinx 7.2.6docs/conf.py docs/requirements.txt python/setup.cfg +2 more+22 -190 replies
15/16sphinx: drop fakedbusdocdocs/conf.py docs/interop/dbus-display.rst docs/interop/dbus-vmstate.rst +2 more+4 -750 replies
16/16sphinx: drop qapidoc back-compat overboiled spaghetti dinnerdocs/sphinx/compat.py docs/sphinx/qapi_domain.py+28 -2550 replies

Reviews trailers per patch

All 16 patches have a Reviewed-by or Acked-by.

PatchDaniel P. BerrangéMarc-André LureauKohei TokunagaPhilippe Mathieu-DaudéMarkus Armbruster
1/16R-b
2/16R-b
3/16R-b R-b
4/16R-b
5/16R-b R-b
6/16R-b
7/16R-b R-b
8/16R-b R-b R-b
9/16R-b
10/16R-b
11/16R-b R-b
12/16R-b
13/16R-b
14/16R-b
15/16R-b
16/16R-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 5 reviewers.

Read the v3 thread

Cover letter [PATCH v3 00/16] python: require python3.12+

The stars have aligned, and we can jump all the way from Python3.9 to Python3.12.

GitLab CI: https://gitlab.com/jsnow/qemu/-/pipelines/2789954132

v3:
- 1/16: s/vm-run-xxx/vm-build-xxx in commit msg
- 1/16: Add "Fixes: 05e4982" to commit msg
- Drop "tests: update FreeBSD dependencies" patch and replace with "tests: update libvirt-ci, lcitool" (thanks, dan)
- Add r-bs, rebase, etc.

v2:
- s/types_seen/objects_seen in patch 8
- drop python3-devel from mappings.yml
- add comments explaining contentnode vs content_node in #16
- fix VM tests to use local QEMU build to support -object qmp-monitor in order to run tests (Thanks danpb)
- fix FreeBSD vm test (Pre-existing breakage)
- fix netBSD vm test (Pre-existing breakage)
- update docs/requirements.txt for sphinx on readthedocs
- Known issue: Haiku VM test is broken, but this series makes it worse. Existing problem is that tomli has gone missing; new problem is that our Haiku image still only offers Python 3.9. This VM test has not been meaningfully updated since its introduction in 2020, and the 2023 r4 build on vagrant's registry is still the most recent version, so... I think if we want Haiku to work, it's up to someone else to show it some love.

Python 3.9 is EOL, and Python 3.10 will be EOL in about two months (October 2026). Python 3.11 is, to be blunt, perfectly serviceable - but when investigating our actual minimum dependency requirements, I realized none of our supported platforms used anything older than Python 3.12, so we may as well build and test with the version that all of our build platforms are actually using.

This reduces the Python compat test matrix by quite a lot and allows us a few nice things. From memory, the highlights are:

- Pattern matching syntax (fancy case statements). I am not dying to use these anywhere, but they're nice!
- Vastly improved error reporting from CPython
- Ability to annotate exceptions with "notes", which allows us augment the default reporting behavior for unhandled exceptions - this can be used in e.g. iotests to help give additional information on tool crashes that is sometimes hard to discover otherwise
- The ability to use standard library types for typing syntax instead of importing from Typing (Technically, valid since 3.9, but...)
- The ability to use "x | y" instead of Union[x, y] in type hint syntax. Combined with the above, it's a good time to clean up the typing in qapi to make it read much more nicely.
- ExceptionGroups - the ability to capture and report multiple exceptions simultaneously. This is extremely useful for async QMP and managing multiple VMs from a single python context. Async QMP will be able to get a few nice QOL improvements with this.

My personal favorite:

- We can move to a much newer Sphinx, which gets rid of an awful lot of very ugly compat code. This is the *actual* motivation of this series.

Anyway, here's the version chart for our supported build platforms and why this is safe to do:

distro EOS python3 pip setuptools sphinx wheel
-------------------------------------------------------------------------

centos_stream_9   2026-12-12  3.9.25   21.3.1  53.0.0      3.4.3   0.36.2
centos_stream_9+  2026-12-12  3.12.13  23.2.1  68.2.2      ---     0.41.2
freebsd           --          3.12.14  23.3.2  63.1.0      9.0.4   0.48.0
ubuntu_24_04      2028-04-23  3.12.3   24.0    68.1.2      7.2.6   0.42.0
centos_stream_10  2030-01-01  3.12.13  23.3.2  69.0.3      7.2.6   0.41.2
alpine_3_21       2026-11-01  3.12.14  24.3.1  70.3.0      8.1.3   0.43.0
debian_13         2028-08-09  3.13.5   25.1.1  78.1.1      8.1.3   0.46.1
fedora_43         2026-12-09  3.14.7   25.1.1  78.1.1      8.2.3   0.45.1
ubuntu_26_04      2031-05-29  3.14.4   25.1.1  78.1.1      8.2.3   0.46.3
fedora_44         2027-06-02  3.14.7   26.0.1  80.10.2     8.2.3   0.45.1
macports          --          3.14.7   26.0.1  80.10.2     9.1.0   0.48.0
alpine_3_22       2027-05-01  3.12.14  25.1.1  80.9.0      8.2.3   0.46.3
alpine_3_23       2027-11-01  3.12.14  25.1.1  80.9.0      8.2.3   0.46.3
openbsd           --          3.14.6   26.2.1  80.9.0      9.1.0   0.48.0
alpine_3_24       2028-06-01  3.14.7   26.1.2  82.0.1      9.1.0   0.47.0
pkgsrc_current    --          3.13.15  26.2.1  84.0.0      9.1.0   0.48.0

CentOS stream 9 note: python3.12, python3.12-pip, python3.12-setuptools and python3.12-wheel are available as optional packages in the standard repository. Sphinx is not available, but can be side-loaded from PyPI using pip3.12.

More cleanups to follow: we may have the chance to upgrade to pyproject.toml packaging once and for all now that FreeBSD has moved to Python3.12, which will get us off the legacy packaging we've been using. This isn't a problem *currently*, but it has been the cause of a lot of pain in the past, and the sooner we can ditch it the better.

toodles,
--js

John Snow (16):
tests: update vm tests to prefer locally-built QEMU
tests: update libvirt-ci, lcitool
tests: update netbsd dependencies
tests: drop python3-devel mapping
tests: re-add opencv2 and numpy dependencies
tests: add Python3.12 to centos9.docker
tests: update emsdk-wasm64-cross
qapi: update type hint for objects_seen
python: upgrade to Python3.12+
python: fix license configuration
python: allow mypy to run as the current python version
python: remove distutils workarounds
python: remove stale setup.cfg options
sphinx: require sphinx 7.2.6
sphinx: drop fakedbusdoc
sphinx: drop qapidoc back-compat overboiled spaghetti dinner

 docs/conf.py                                  |  11 +-
 docs/interop/dbus-display.rst                 |  14 +-
 docs/interop/dbus-vmstate.rst                 |  14 +-
 docs/interop/dbus-vnc.rst                     |  14 +-
 docs/requirements.txt                         |   2 +-
 docs/sphinx/compat.py                         | 230 ------------------
 docs/sphinx/fakedbusdoc.py                    |  30 ---
 docs/sphinx/qapi_domain.py                    |  53 ++--
 configure                                     |  11 +-
 tests/docker/dockerfiles/alpine.docker        |   1 +
 tests/docker/dockerfiles/centos9.docker       |  14 +-
 .../dockerfiles/debian-amd64-cross.docker     |   4 +-
 .../dockerfiles/debian-arm64-cross.docker     |   4 +-
 .../dockerfiles/debian-armhf-cross.docker     |   4 +-
 .../dockerfiles/debian-i686-cross.docker      |   4 +-
 .../dockerfiles/debian-ppc64el-cross.docker   |   4 +-
 .../dockerfiles/debian-riscv64-cross.docker   |   4 +-
 .../dockerfiles/debian-s390x-cross.docker     |   4 +-
 tests/docker/dockerfiles/debian.docker        |   4 +-
 .../dockerfiles/emsdk-wasm64-cross.docker     |   4 +-
 .../dockerfiles/fedora-rust-nightly.docker    |   2 +
 .../dockerfiles/fedora-win64-cross.docker     |   2 +
 tests/docker/dockerfiles/fedora.docker        |   2 +
 tests/docker/dockerfiles/opensuse-leap.docker |   4 +-
 tests/docker/dockerfiles/ubuntu2404.docker    |   4 +-
 .gitlab-ci.d/macos-14.vars                    |   2 +-
 python/Makefile                               |  10 +-
 python/scripts/mkvenv.py                      |  63 +----
 python/setup.cfg                              |  33 +--
 python/tests/linters.py                       |  11 -
 python/tests/minreqs.txt                      |  66 ++---
 pythondeps.toml                               |  13 +-
 .../ci/setup/debian/debian-13-ppc64le.yaml    |   4 +-
 .../ci/setup/ubuntu/ubuntu-2404-aarch64.yaml  |   4 +-
 .../ci/setup/ubuntu/ubuntu-2404-s390x.yaml    |   4 +-
 scripts/qapi/types.py                         |   2 +-
 tests/lcitool/libvirt-ci                      |   2 +-
 tests/lcitool/mappings.yml                    |  25 +-
 tests/lcitool/projects/qemu.yml               |   2 +
 tests/lcitool/targets/centos-stream-9.yml     |   4 +-
 tests/vm/Makefile.include                     |   2 +-
 tests/vm/generated/freebsd.json               |  14 +-
 tests/vm/netbsd                               |   1 +
 43 files changed, 185 insertions(+), 521 deletions(-)
 delete mode 100644 docs/sphinx/compat.py
 delete mode 100644 docs/sphinx/fakedbusdoc.py
Signature · 1 line
2.55.0