Python: Fix 'make check-dev' and modernize to 3.9+
cover Python: Fix 'make check-dev' and modernize to 3.9+
This series does a couple things that I'll probably end up splitting out into smaller series if history is any guide, but either way, here it goes:
A) Convert qemu.git/python/ to a PEP517/pyproject.toml package. Ultimately this means deleting setup.py and fully migrating to newer python infrastructure. I think this should be safe to do by now, but admittedly I am not *confident* as it relies on setuptools versions in the wild, not python versions. My motivation for trying it is to fix "make check-dev", which has been broken for the last two Fedora releases under newer setuptools which have started removing support for the pre-PEP517 packaging formats, which will only continue to get worse from here on out.
B) Sync changes from the qemu.qmp package back over to qemu.git. I know I need to decouple this badly, but in order to do so, I need to make sure they're synchronized to be assured that the switch to the standalone version won't break anything, so this is a necessary step. It's happening here because of the 3.6+ compat crud we are still carrying in qemu.git that has since been removed from the standalone library.
C) Move us to 3.9+ style type hints. I know it feels like we just got the type hints settled and we're already changing them. Sorry about that. They are deprecated in 3.9, and *could* be removed at any time. I figured now was a good time as any to get rid of them before they become a problem randomly some day in the future.
D) Update the mypy configuration to check under multiple Python versions more effectively and thoroughly.
Whew.
--js
John Snow (8):
python: convert packages to PEP517/pyproject.toml
python: update pylint ignores
python: sync changes from external qemu.qmp package
python: use 3.9+ builtin type hints
python: convert remaining deprecated type hints for 3.9+
python: clean up requirements for 3.9+
python: update mkvenv to type-check under different python versions
python: remove version restriction for mypy
docs/sphinx/compat.py | 11 +- docs/sphinx/dbusdoc.py | 13 +- docs/sphinx/dbusdomain.py | 32 ++-- docs/sphinx/fakedbusdoc.py | 4 +- docs/sphinx/qapi_domain.py | 77 ++++---- docs/sphinx/qapidoc.py | 16 +- python/README.rst | 33 ++-- python/Makefile | 18 +- python/pyproject.toml | 8 + python/qemu/machine/console_socket.py | 4 +- python/qemu/machine/machine.py | 32 ++-- python/qemu/machine/qtest.py | 13 +- python/qemu/qmp/error.py | 7 +- python/qemu/qmp/events.py | 72 +++++-- python/qemu/qmp/legacy.py | 31 +-- python/qemu/qmp/message.py | 38 ++-- python/qemu/qmp/models.py | 26 ++- python/qemu/qmp/protocol.py | 179 ++++++++++------- python/qemu/qmp/qmp_client.py | 147 +++++++++----- python/qemu/qmp/qmp_shell.py | 180 ++++++++++++------ python/qemu/qmp/qmp_tui.py | 55 +++--- python/qemu/qmp/util.py | 116 +---------- python/qemu/utils/accel.py | 4 +- python/qemu/utils/qemu_ga_client.py | 11 +- python/qemu/utils/qom_common.py | 17 +- python/qemu/utils/qom_fuse.py | 12 +- python/scripts/mkvenv.py | 38 ++-- python/setup.cfg | 7 +- python/setup.py | 40 ---- python/tests/minreqs.txt | 4 +- python/tests/protocol.py | 10 +- scripts/block-coroutine-wrapper.py | 2 +- .../codeconverter/codeconverter/patching.py | 50 ++--- .../codeconverter/codeconverter/qom_macros.py | 36 ++-- .../codeconverter/qom_type_info.py | 16 +- scripts/compare-machine-types.py | 41 ++-- scripts/qapi/commands.py | 13 +- scripts/qapi/common.py | 20 +- scripts/qapi/events.py | 8 +- scripts/qapi/expr.py | 28 ++- scripts/qapi/features.py | 2 +- scripts/qapi/gen.py | 13 +- scripts/qapi/introspect.py | 44 ++--- scripts/qapi/parser.py | 42 ++-- scripts/qapi/schema.py | 117 ++++++------ scripts/qapi/source.py | 10 +- scripts/qapi/types.py | 20 +- scripts/qapi/visit.py | 14 +- scripts/rust/rustc_args.py | 7 +- scripts/u2f-setup-gen.py | 3 +- tests/functional/qemu_test/ports.py | 3 +- tests/functional/test_acpi_bits.py | 9 +- tests/qemu-iotests/151 | 4 +- tests/qemu-iotests/297 | 5 +- tests/qemu-iotests/300 | 4 +- tests/qemu-iotests/fat16.py | 11 +- tests/qemu-iotests/findtests.py | 13 +- tests/qemu-iotests/iotests.py | 38 ++-- tests/qemu-iotests/linters.py | 7 +- tests/qemu-iotests/testenv.py | 11 +- tests/qemu-iotests/testrunner.py | 19 +- .../qemu-iotests/tests/stream-under-throttle | 3 +- 62 files changed, 925 insertions(+), 943 deletions(-) create mode 100644 python/pyproject.toml delete mode 100755 python/setup.py
Signature · 1 line
2.48.1
PATCH 1/8 python: convert packages to PEP517/pyproject.toml
Newer versions of setuptools increasingly expect that packages are defined using the pyproject.toml/PEP517 packaging layout format. With 3.9 as our minimum, I believe it's finally appropriate to make the shift away from the legacy packaging format.
Update documentation and dependencies that change as a result of the different build/packaging/installation pathways.
This change has the effect of fixing "make check-dev", which has been broken on newer versions of Fedora for a while, now.
--- python/README.rst | 33 ++++++++++++++++----------------- python/Makefile | 18 +++++++++--------- python/pyproject.toml | 8 ++++++++ python/setup.py | 40 ---------------------------------------- python/tests/minreqs.txt | 4 +++- 5 files changed, 36 insertions(+), 67 deletions(-) create mode 100644 python/pyproject.toml delete mode 100755 python/setup.py
python/README.rst +16 -17
@@ -5,24 +5,23 @@ This directory houses Python tooling used by the QEMU project to build, configure, and test QEMU. It is organized by namespace (``qemu``), and then by package (e.g. ``qemu/machine``, ``qemu/qmp``, etc). -``setup.py`` is used by ``pip`` to install this tooling to the current -environment. ``setup.cfg`` provides the packaging configuration used by -``setup.py``. You will generally invoke it by doing one of the following: +``pyproject.toml`` and ``setup.cfg`` are used by ``pip`` to install this +tooling to the current environment. ``setup.cfg`` provides the packaging +configuration, while ``pyproject.toml`` describes the package build +system requirements. -1. ``pip3 install .`` will install these packages to your current - environment. If you are inside a virtual environment, they will - install there. If you are not, it will attempt to install to the - global environment, which is **not recommended**. +You will generally install these packages by invoking ``pip3 install +.``; which will install these packages to your current environment. If +you are inside a virtual environment, they will install there. If you +are not, modern versions of pip will attempt instead to install to your +local user environment. Older versions of pip will attempt to install to +the global environment, which is **not recommended**. -2. ``pip3 install --user .`` will install these packages to your user's - local python packages. If you are inside of a virtual environment, - this will fail; you want the first invocation above. - -If you append the ``--editable`` or ``-e`` argument to either invocation -above, pip will install in "editable" mode. This installs the package as -a forwarder ("qemu.egg-link") that points to the source tree. In so -doing, the installed package always reflects the latest version in your -source tree. +If you append the ``--editable`` or ``-e`` argument to the above +invocation, pip will install in "editable" mode. This installs the +package as a "forwarder" that points to the source tree, so that the +installed package always reflects the latest version in your source +tree. Installing ".[devel]" instead of "." will additionally pull in required packages for testing this package. They are not runtime requirements,
@@ -81,4 +80,4 @@ Files in this directory - ``VERSION`` contains the PEP-440 compliant version used to describe this package; it is referenced by ``setup.cfg``. - ``setup.cfg`` houses setuptools package configuration. -- ``setup.py`` is the setuptools installer used by pip; See above. +- ``pyproject.toml`` lists build system requirements for the Python packages.
python/Makefile +9 -9
@@ -15,8 +15,8 @@ help: @echo "make check-tox:" @echo " Run tests against multiple python versions." @echo " These tests use the newest dependencies." - @echo " Requires: Python 3.9 - 3.11, and tox." - @echo " Hint (Fedora): 'sudo dnf install python3-tox python3.11'" + @echo " Requires: Python 3.9 - 3.13, and tox." + @echo " Hint (Fedora): 'sudo dnf install python3-tox python3.13'" @echo " The variable QEMU_TOX_EXTRA_ARGS can be use to pass extra" @echo " arguments to tox". @echo ""
@@ -63,12 +63,12 @@ $(QEMU_MINVENV_DIR) $(QEMU_MINVENV_DIR)/bin/activate: setup.cfg tests/minreqs.tx @( \ echo "ACTIVATE $(QEMU_MINVENV_DIR)"; \ . $(QEMU_MINVENV_DIR)/bin/activate; \ - echo "INSTALL wheel $(QEMU_MINVENV_DIR)"; \ - $(PIP_INSTALL) wheel 1>/dev/null; \ + echo "INSTALL wheel $(QEMU_MINVENV_DIR)"; \ + $(PIP_INSTALL) wheel 1>/dev/null; \ echo "INSTALL -r tests/minreqs.txt $(QEMU_MINVENV_DIR)";\ $(PIP_INSTALL) -r tests/minreqs.txt 1>/dev/null; \ echo "INSTALL -e qemu $(QEMU_MINVENV_DIR)"; \ - $(PIP_INSTALL) -e . 1>/dev/null; \ + $(PIP_INSTALL) -e . --config-settings=editable_mode=compat 1>/dev/null; \ ) @touch $(QEMU_MINVENV_DIR)
@@ -103,7 +103,7 @@ check-dev: dev-venv .PHONY: develop develop: - $(PIP_INSTALL) -e .[devel] + $(PIP_INSTALL) -e .[devel] --config-settings=editable_mode=compat .PHONY: check check:
@@ -122,12 +122,12 @@ check-coverage: .PHONY: clean clean: - python3 setup.py clean --all - rm -f pyproject.toml + rm -rf build/ .PHONY: distclean distclean: clean - rm -rf qemu.egg-info/ .eggs/ dist/ + rm -rf qemu.egg-info/ rm -rf $(QEMU_VENV_DIR) $(QEMU_MINVENV_DIR) .tox/ + rm -rf .mypy_cache/ rm -f .coverage .coverage.* rm -rf htmlcov/
python/pyproject.toml new file +8 -0
@@ -0,0 +1,8 @@ +[build-system] +requires = [ + "setuptools>=39.2", + "wheel", +] +build-backend = "setuptools.build_meta" + +[tool.setuptools_scm]
python/setup.py deleted +0 -40
@@ -1,40 +0,0 @@ -#!/usr/bin/env python3 -""" -QEMU tooling installer script -Copyright (c) 2020-2021 John Snow for Red Hat, Inc. -""" - -import setuptools -from setuptools.command import bdist_egg -import sys -import pkg_resources - - -class bdist_egg_guard(bdist_egg.bdist_egg): - """ - Protect against bdist_egg from being executed - - This prevents calling 'setup.py install' directly, as the 'install' - CLI option will invoke the deprecated bdist_egg hook. "pip install" - calls the more modern bdist_wheel hook, which is what we want. - """ - def run(self): - sys.exit( - 'Installation directly via setup.py is not supported.\n' - 'Please use `pip install .` instead.' - ) - - -def main(): - """ - QEMU tooling installer - """ - - # https://medium.com/@daveshawley/safely-using-setup-cfg-for-metadata-1babbe54c108 - pkg_resources.require('setuptools>=39.2') - - setuptools.setup(cmdclass={'bdist_egg': bdist_egg_guard}) - - -if __name__ == '__main__': - main()
python/tests/minreqs.txt +3 -1
@@ -11,6 +11,9 @@ # When adding new dependencies, pin the very oldest non-yanked version # on PyPI that allows the test suite to pass. +# Avocado requires setuptools at runtime, and it requires an older one. +setuptools<71 + # Dependencies for qapidoc/qapi_domain et al sphinx==3.4.3
@@ -49,7 +52,6 @@ astroid==2.15.4 dill==0.2 lazy-object-proxy==1.4.0 platformdirs==2.2.0 -toml==0.10.0 tomlkit==0.10.1 wrapt==1.14.0
Signature · 1 line
2.48.1
PATCH 2/8 python: update pylint ignores
In reply to John SnowThe next patch will synchronize the qemu.qmp library with the external, standalone version. That synchronization will require a few extra ignores for pylint, so do that now.
--- python/setup.cfg | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
python/setup.cfg +3 -2
@@ -140,11 +140,12 @@ ignore_missing_imports = True # --disable=W". disable=consider-using-f-string, consider-using-with, + no-member, # mypy also handles this better. too-many-arguments, - too-many-positional-arguments, too-many-function-args, # mypy handles this with less false positives. too-many-instance-attributes, - no-member, # mypy also handles this better. + too-many-positional-arguments, + unknown-option-value, [pylint.basic] # Good variable names which should always be accepted, separated by a comma.
Signature · 1 line
2.48.1
Re: [PATCH 2/8] python: update pylint ignores
Quoted 3 lines · John Snow
The next patch will synchronize the qemu.qmp library with the external, standalone version. That synchronization will require a few extra ignores for pylint, so do that now.
Just one, unless I'm cross-eyed again.
Quoted 10 lines
Signed-off-by John Snow <[email protected]>--- python/setup.cfg | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)python/setup.cfg +0
-0
python/setup.cfg ignore_missing_imports = True · line 147
Quoted 10 lines
python/setup.cfg +2
-2@@ -140,11 +140,12 @@ ignore_missing_imports = True # --disable=W". disable=consider-using-f-string, consider-using-with, + no-member, # mypy also handles this better. too-many-arguments, - too-many-positional-arguments, too-many-function-args, # mypy handles this with less false positives. too-many-instance-attributes, - no-member, # mypy also handles this better. + too-many-positional-arguments,
Up to here, just a reorder, which I appreciate.
python/setup.cfg ignore_missing_imports = True · line 148
Quoted 1 line
python/setup.cfg +1
-0+ unknown-option-value,
This is the new one.
Quoted 3 lines
python/setup.cfg +0
-0[pylint.basic] # Good variable names which should always be accepted, separated by a comma.
Re: [PATCH 2/8] python: update pylint ignores
Quoted 8 lines · Markus Armbruster
Quoted 3 lines · John Snow
The next patch will synchronize the qemu.qmp library with the external, standalone version. That synchronization will require a few extra ignores for pylint, so do that now.
Just one, unless I'm cross-eyed again.
Nope. changed the commit message. Just doing things in weird orders as is my usual...
Quoted 35 lines
Quoted 11 lines
Signed-off-by John Snow <[email protected]>--- python/setup.cfg | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)python/setup.cfg +0
-0python/setup.cfg ignore_missing_imports = True · line 145
Quoted 7 lines
python/setup.cfg +1
-1@@ -140,11 +140,12 @@ ignore_missing_imports = True # --disable=W". disable=consider-using-f-string, consider-using-with, + no-member, # mypy also handles this better. too-many-arguments, - too-many-positional-arguments, too-many-function-args, # mypy handles this with less falsepositives.
python/setup.cfg ignore_missing_imports = True · line 147
Quoted 3 lines
python/setup.cfg +1
-1too-many-instance-attributes, - no-member, # mypy also handles this better. + too-many-positional-arguments,Up to here, just a reorder, which I appreciate.
python/setup.cfg ignore_missing_imports = True · line 148
Quoted 1 line
python/setup.cfg +1
-0+ unknown-option-value,This is the new one.
python/setup.cfg ignore_missing_imports = True · line 150
Quoted 3 lines
python/setup.cfg +0
-0[pylint.basic] # Good variable names which should always be accepted, separated by acomma.
PATCH 3/8 python: sync changes from external qemu.qmp package
In reply to John SnowSynchronize changes from the standalone python-qemu-qmp repository for the primary purpose of removing some pre-3.9 compatibility crud that we no longer need or want.
My plan for this release cycle is to finally eliminate this duplication so this type of patch will no longer be necessary -- but for now, it is.
NB: A few changes are not synchronized; mostly license and documentation strings. Everything functional is fully copied verbatim. The license strings are not sync'd only because they point to different LICENSE filenames for the different repositories.
--- python/qemu/qmp/error.py | 7 +- python/qemu/qmp/events.py | 50 ++++++++-- python/qemu/qmp/legacy.py | 16 +++- python/qemu/qmp/message.py | 22 +++-- python/qemu/qmp/models.py | 6 +- python/qemu/qmp/protocol.py | 169 +++++++++++++++++++++------------- python/qemu/qmp/qmp_client.py | 133 +++++++++++++++++++------- python/qemu/qmp/qmp_shell.py | 167 +++++++++++++++++++++++---------- python/qemu/qmp/qmp_tui.py | 34 +++++-- python/qemu/qmp/util.py | 116 ++--------------------- python/tests/protocol.py | 10 +- 11 files changed, 443 insertions(+), 287 deletions(-)
python/qemu/qmp/error.py +5 -2
@@ -44,7 +44,10 @@ class ProtocolError(QMPError): :param error_message: Human-readable string describing the error. """ - def __init__(self, error_message: str): - super().__init__(error_message) + def __init__(self, error_message: str, *args: object): + super().__init__(error_message, *args) #: Human-readable error message, without any prefix. self.error_message: str = error_message + + def __str__(self) -> str: + return self.error_message
python/qemu/qmp/events.py +42 -8
104 lines not on this page · raw message
python/qemu/qmp/legacy.py +13 -3
@@ -86,7 +86,14 @@ def __init__(self, "server argument should be False when passing a socket") self._qmp = QMPClient(nickname) - self._aloop = asyncio.get_event_loop() + + try: + self._aloop = asyncio.get_running_loop() + except RuntimeError: + # No running loop; since this is a sync shim likely to be + # used in fully sync programs, create one if neccessary. + self._aloop = asyncio.get_event_loop_policy().get_event_loop() + self._address = address self._timeout: Optional[float] = None
@@ -231,6 +238,9 @@ def pull_event(self, :return: The first available QMP event, or None. """ + # Kick the event loop to allow events to accumulate + self._sync(asyncio.sleep(0)) + if not wait: # wait is False/0: "do not wait, do not except." if self._qmp.events.empty():
@@ -286,8 +296,8 @@ def settimeout(self, timeout: Optional[float]) -> None: """ Set the timeout for QMP RPC execution. - This timeout affects the `cmd`, `cmd_obj`, and `command` methods. - The `accept`, `pull_event` and `get_event` methods have their + This timeout affects the `cmd`, `cmd_obj`, and `cmd_raw` methods. + The `accept`, `pull_event` and `get_events` methods have their own configurable timeouts. :param timeout:
python/qemu/qmp/message.py +15 -7
@@ -28,7 +28,8 @@ class Message(MutableMapping[str, object]): be instantiated from either another mapping (like a `dict`), or from raw `bytes` that still need to be deserialized. - Once instantiated, it may be treated like any other MutableMapping:: + Once instantiated, it may be treated like any other + :py:obj:`~collections.abc.MutableMapping`:: >>> msg = Message(b'{"hello": "world"}') >>> assert msg['hello'] == 'world'
@@ -50,12 +51,19 @@ class Message(MutableMapping[str, object]): >>> dict(msg) {'hello': 'world'} + Or pretty-printed:: + + >>> print(str(msg)) + { + "hello": "world" + } :param value: Initial value, if any. :param eager: When `True`, attempt to serialize or deserialize the initial value immediately, so that conversion exceptions are raised during the call to ``__init__()``. + """ # pylint: disable=too-many-ancestors
@@ -178,15 +186,15 @@ class DeserializationError(ProtocolError): :param raw: The raw `bytes` that prompted the failure. """ def __init__(self, error_message: str, raw: bytes): - super().__init__(error_message) + super().__init__(error_message, raw) #: The raw `bytes` that were not understood as JSON. self.raw: bytes = raw def __str__(self) -> str: - return "\n".join([ + return "\n".join(( super().__str__(), f" raw bytes were: {str(self.raw)}", - ]) + )) class UnexpectedTypeError(ProtocolError):
@@ -197,13 +205,13 @@ class UnexpectedTypeError(ProtocolError): :param value: The deserialized JSON value that wasn't an object. """ def __init__(self, error_message: str, value: object): - super().__init__(error_message) + super().__init__(error_message, value) #: The JSON value that was expected to be an object. self.value: object = value def __str__(self) -> str: strval = json.dumps(self.value, indent=2) - return "\n".join([ + return "\n".join(( super().__str__(), f" json value was: {strval}", - ]) + ))
python/qemu/qmp/models.py +5 -1
@@ -54,7 +54,11 @@ def __repr__(self) -> str: class Greeting(Model): """ - Defined in qmp-spec.rst, section "Server Greeting". + Defined in qmp-spec.rst, "Server Greeting". + + See `2.2 Server Greeting + <https://gitlab.com/qemu-project/qemu/-/blob/master/docs/interop/qmp-spec.txt#L61>`_ + for details. :param raw: The raw Greeting object. :raise KeyError: If any required fields are absent.
python/qemu/qmp/protocol.py +103 -66
336 lines not on this page · raw message
python/qemu/qmp/qmp_client.py +100 -33
247 lines not on this page · raw message
python/qemu/qmp/qmp_shell.py +120 -47
248 lines not on this page · raw message
python/qemu/qmp/qmp_tui.py +25 -9
@@ -21,6 +21,7 @@ import logging from logging import Handler, LogRecord import signal +import sys from typing import ( List, Optional,
@@ -30,17 +31,27 @@ cast, ) -from pygments import lexers -from pygments import token as Token -import urwid -import urwid_readline + +try: + from pygments import lexers + from pygments import token as Token + import urwid + import urwid_readline +except ModuleNotFoundError as exc: + print( + f"Module '{exc.name}' not found.", + "You need the optional 'tui' group: pip install qemu.qmp[tui]", + sep='\n', + file=sys.stderr, + ) + sys.exit(1) from .error import ProtocolError from .legacy import QEMUMonitorProtocol, QMPBadPortError from .message import DeserializationError, Message, UnexpectedTypeError from .protocol import ConnectError, Runstate from .qmp_client import ExecInterruptedError, QMPClient -from .util import create_task, pretty_traceback +from .util import pretty_traceback # The name of the signal that is used to update the history list
@@ -225,7 +236,7 @@ def cb_send_to_server(self, raw_msg: str) -> None: """ try: msg = Message(bytes(raw_msg, encoding='utf-8')) - create_task(self._send_to_server(msg)) + asyncio.create_task(self._send_to_server(msg)) except (DeserializationError, UnexpectedTypeError) as err: raw_msg = format_json(raw_msg) logging.info('Invalid message: %s', err.error_message)
@@ -246,7 +257,7 @@ def kill_app(self) -> None: Initiates killing of app. A bridge between asynchronous and synchronous code. """ - create_task(self._kill_app()) + asyncio.create_task(self._kill_app()) async def _kill_app(self) -> None: """
@@ -377,7 +388,12 @@ def run(self, debug: bool = False) -> None: screen = urwid.raw_display.Screen() screen.set_terminal_properties(256) - self.aloop = asyncio.get_event_loop() + try: + self.aloop = asyncio.get_running_loop() + except RuntimeError: + # No running asyncio event loop. Create one if necessary. + self.aloop = asyncio.get_event_loop_policy().get_event_loop() + self.aloop.set_debug(debug) # Gracefully handle SIGTERM and SIGINT signals
@@ -393,7 +409,7 @@ def run(self, debug: bool = False) -> None: handle_mouse=True, event_loop=event_loop) - create_task(self.manage_connection(), self.aloop) + self.aloop.create_task(self.manage_connection()) try: main_loop.run() except Exception as err:
python/qemu/qmp/util.py +10 -106
152 lines not on this page · raw message
python/tests/protocol.py +5 -5
@@ -8,7 +8,6 @@ from qemu.qmp import ConnectError, Runstate from qemu.qmp.protocol import AsyncProtocol, StateError -from qemu.qmp.util import asyncio_run, create_task class NullProtocol(AsyncProtocol[None]):
@@ -124,7 +123,7 @@ async def _runner(): if allow_cancellation: return raise - return create_task(_runner()) + return asyncio.create_task(_runner()) @contextmanager
@@ -228,7 +227,7 @@ def async_test(async_test_method): Decorator; adds SetUp and TearDown to async tests. """ async def _wrapper(self, *args, **kwargs): - loop = asyncio.get_event_loop() + loop = asyncio.get_running_loop() loop.set_debug(True) await self._asyncSetUp()
@@ -271,7 +270,7 @@ async def _watcher(): msg=f"Expected state '{state.name}'", ) - self.runstate_watcher = create_task(_watcher()) + self.runstate_watcher = asyncio.create_task(_watcher()) # Kick the loop and force the task to block on the event. await asyncio.sleep(0)
@@ -589,7 +588,8 @@ async def _asyncTearDown(self): async def testSmoke(self): with TemporaryDirectory(suffix='.qmp') as tmpdir: sock = os.path.join(tmpdir, type(self.proto).__name__ + ".sock") - server_task = create_task(self.server.start_server_and_accept(sock)) + server_task = asyncio.create_task( + self.server.start_server_and_accept(sock)) # give the server a chance to start listening [...] await asyncio.sleep(0)
Signature · 1 line
2.48.1
PATCH 4/8 python: use 3.9+ builtin type hints
In reply to John SnowThis patch changes type hints that have built-in equivalents as of Python 3.9. The versions we currently use, imported from the typing module, have been deprecated since 3.9 and may be removed at any time - though to my knowledge, they are not scheduled for removal in 3.14.
The type hints to update are:
typing.Dict => dict
typing.List => list
typing.Tuple => tuple
typing.Set => set
typing.Type => type
I performed this change the dumb way (I couldn't work out for the life of me how to beg ls-files to exclude subprojects and empty directories, desipte it having an option which claims to do precisely that. Suggest a nicer way to do this if you'd like.)
git ls-files | grep -v 'roms/' | grep -v 'subprojects/' | grep -v 'tests/lcitool/libvirt-ci' | xargs sed -i -e 's/Dict\[/dict\[/g' git ls-files | grep -v 'roms/' | grep -v 'subprojects/' | grep -v 'tests/lcitool/libvirt-ci' | xargs sed -i -e 's/List\[/list\[/g' git ls-files | grep -v 'roms/' | grep -v 'subprojects/' | grep -v 'tests/lcitool/libvirt-ci' | xargs sed -i -e 's/Tuple\[/tuple\[/g' git ls-files | grep -v 'roms/' | grep -v 'subprojects/' | grep -v 'tests/lcitool/libvirt-ci' | xargs sed -i -e 's/Set\[/set\[/g' git ls-files | grep -v 'roms/' | grep -v 'subprojects/' | grep -v 'tests/lcitool/libvirt-ci' | xargs sed -i -e 's/Type\[/type\[/g'
I then used "git add -p" and excluded changes by hand that weren't appropriate. I then reviewed all of the changed files by hand to review the imports and update them accordingly.
--- docs/sphinx/compat.py | 3 +- docs/sphinx/dbusdoc.py | 9 +- docs/sphinx/dbusdomain.py | 29 ++--- docs/sphinx/fakedbusdoc.py | 4 +- docs/sphinx/qapi_domain.py | 68 +++++------ docs/sphinx/qapidoc.py | 7 +- python/qemu/machine/machine.py | 30 ++--- python/qemu/machine/qtest.py | 12 +- python/qemu/qmp/events.py | 15 +-- python/qemu/qmp/legacy.py | 13 +- python/qemu/qmp/message.py | 7 +- python/qemu/qmp/models.py | 3 +- python/qemu/qmp/protocol.py | 8 +- python/qemu/qmp/qmp_client.py | 6 +- python/qemu/qmp/qmp_shell.py | 10 +- python/qemu/qmp/qmp_tui.py | 21 ++-- python/qemu/utils/accel.py | 4 +- python/qemu/utils/qemu_ga_client.py | 3 +- python/qemu/utils/qom_common.py | 17 +-- python/qemu/utils/qom_fuse.py | 3 +- python/scripts/mkvenv.py | 14 +-- .../codeconverter/codeconverter/patching.py | 44 +++---- .../codeconverter/codeconverter/qom_macros.py | 36 +++--- .../codeconverter/qom_type_info.py | 16 +-- scripts/compare-machine-types.py | 40 +++--- scripts/qapi/commands.py | 13 +- scripts/qapi/common.py | 9 +- scripts/qapi/events.py | 8 +- scripts/qapi/expr.py | 20 ++- scripts/qapi/gen.py | 12 +- scripts/qapi/introspect.py | 42 +++---- scripts/qapi/parser.py | 37 +++--- scripts/qapi/schema.py | 114 +++++++++--------- scripts/qapi/source.py | 10 +- scripts/qapi/types.py | 20 +-- scripts/qapi/visit.py | 14 +-- scripts/rust/rustc_args.py | 6 +- scripts/u2f-setup-gen.py | 3 +- tests/functional/qemu_test/ports.py | 3 +- tests/functional/test_acpi_bits.py | 3 +- tests/qemu-iotests/151 | 4 +- tests/qemu-iotests/297 | 5 +- tests/qemu-iotests/300 | 4 +- tests/qemu-iotests/fat16.py | 10 +- tests/qemu-iotests/findtests.py | 12 +- tests/qemu-iotests/iotests.py | 38 +++--- tests/qemu-iotests/linters.py | 6 +- tests/qemu-iotests/testenv.py | 6 +- tests/qemu-iotests/testrunner.py | 12 +- .../qemu-iotests/tests/stream-under-throttle | 3 +- 50 files changed, 375 insertions(+), 461 deletions(-)
docs/sphinx/compat.py +1 -2
@@ -8,7 +8,6 @@ Any, Callable, Optional, - Type, ) from docutils import nodes
@@ -85,7 +84,7 @@ def _compat_make_xref( # pylint: disable=unused-argument rolename: str, domain: str, target: str, - innernode: Type[TextlikeNode] = addnodes.literal_emphasis, + innernode: type[TextlikeNode] = addnodes.literal_emphasis, contnode: Optional[Node] = None, env: Optional[BuildEnvironment] = None, inliner: Any = None,
docs/sphinx/dbusdoc.py +2 -7
@@ -13,14 +13,9 @@ TYPE_CHECKING, Any, Callable, - Dict, Iterator, - List, Optional, Sequence, - Set, - Tuple, - Type, TypeVar, Union, )
@@ -116,7 +111,7 @@ def add_interface(self, iface): self.indent = self.indent[:-3] -def parse_generated_content(state: RSTState, content: StringList) -> List[Node]: +def parse_generated_content(state: RSTState, content: StringList) -> list[Node]: """Parse a generated content by Documenter.""" with switch_source_input(state, content): node = nodes.paragraph()
@@ -157,7 +152,7 @@ def run(self): return result -def setup(app: Sphinx) -> Dict[str, Any]: +def setup(app: Sphinx) -> dict[str, Any]: """Register dbus-doc directive with Sphinx""" app.add_config_value("dbusdoc_srctree", None, "env") app.add_directive("dbus-doc", DBusDocDirective)
docs/sphinx/dbusdomain.py +13 -16
@@ -8,13 +8,10 @@ from typing import ( Any, - Dict, Iterable, Iterator, - List, NamedTuple, Optional, - Tuple, cast, )
@@ -87,7 +84,7 @@ def handle_signature(self, sig: str, signode: desc_signature) -> str: signode += addnodes.desc_name(sig, sig) return sig - def run(self) -> List[Node]: + def run(self) -> list[Node]: _, node = super().run() name = self.arguments[0] section = nodes.section(ids=[name + "-section"])
@@ -113,7 +110,7 @@ class DBusMethod(DBusMember): } ) - doc_field_types: List[Field] = [ + doc_field_types: list[Field] = [ TypedField( "arg", label=_("Arguments"),
@@ -171,7 +168,7 @@ class DBusSignal(DBusMethod): Implementation of ``dbus:signal``. """ - doc_field_types: List[Field] = [ + doc_field_types: list[Field] = [ TypedField( "arg", label=_("Arguments"),
@@ -203,7 +200,7 @@ class DBusProperty(DBusMember): } ) - doc_field_types: List[Field] = [] + doc_field_types: list[Field] = [] def get_index_text(self, ifacename: str, name: str) -> str: return _("%s (%s property)") % (name, ifacename)
@@ -244,7 +241,7 @@ def handle_signature(self, sig: str, signode: desc_signature) -> str: signode += addnodes.desc_sig_keyword_type(ty, ty) return sig - def run(self) -> List[Node]: + def run(self) -> list[Node]: self.name = "dbus:member" return super().run()
@@ -281,10 +278,10 @@ class DBusIndex(Index): def generate( self, docnames: Iterable[str] = None - ) -> Tuple[List[Tuple[str, List[IndexEntry]]], bool]: - content: Dict[str, List[IndexEntry]] = {} + ) -> tuple[list[tuple[str, list[IndexEntry]]], bool]: + content: dict[str, list[IndexEntry]] = {} # list of prefixes to ignore - ignores: List[str] = self.domain.env.config["dbus_index_common_prefix"] + ignores: list[str] = self.domain.env.config["dbus_index_common_prefix"] ignores = sorted(ignores, key=len, reverse=True) ifaces = sorted(
@@ -329,7 +326,7 @@ class DBusDomain(Domain): name = "dbus" label = "D-Bus" - object_types: Dict[str, ObjType] = { + object_types: dict[str, ObjType] = { "interface": ObjType(_("interface"), "iface", "obj"), "method": ObjType(_("method"), "meth", "obj"), "signal": ObjType(_("signal"), "sig", "obj"),
@@ -347,7 +344,7 @@ class DBusDomain(Domain): "sig": DBusXRef(), "prop": DBusXRef(), } - initial_data: Dict[str, Dict[str, Tuple[Any]]] = { + initial_data: dict[str, dict[str, tuple[Any]]] = { "objects": {}, # fullname -> ObjectEntry } indices = [
@@ -355,7 +352,7 @@ class DBusDomain(Domain): ] @property - def objects(self) -> Dict[str, ObjectEntry]: + def objects(self) -> dict[str, ObjectEntry]: return self.data.setdefault("objects", {}) # fullname -> ObjectEntry def note_object(
@@ -368,7 +365,7 @@ def clear_doc(self, docname: str) -> None: if obj.docname == docname: del self.objects[fullname] - def find_obj(self, typ: str, name: str) -> Optional[Tuple[str, ObjectEntry]]: + def find_obj(self, typ: str, name: str) -> Optional[tuple[str, ObjectEntry]]: # skip parens if name[-2:] == "()": name = name[:-2]
@@ -396,7 +393,7 @@ def resolve_xref( builder, fromdocname, objdef.docname, objdef.node_id, contnode ) - def get_objects(self) -> Iterator[Tuple[str, str, str, str, str, int]]: + def get_objects(self) -> Iterator[tuple[str, str, str, str, str, int]]: for refname, obj in self.objects.items(): yield (refname, refname, obj.objtype, obj.docname, obj.node_id, 1)
docs/sphinx/fakedbusdoc.py +2 -2
@@ -9,7 +9,7 @@ from docutils.parsers.rst import Directive from sphinx.application import Sphinx -from typing import Any, Dict +from typing import Any class FakeDBusDocDirective(Directive):
@@ -20,7 +20,7 @@ def run(self): return [] -def setup(app: Sphinx) -> Dict[str, Any]: +def setup(app: Sphinx) -> dict[str, Any]: """Register a fake dbus-doc directive with Sphinx""" app.add_directive("dbus-doc", FakeDBusDocDirective)
docs/sphinx/qapi_domain.py +30 -38
231 lines not on this page · raw message
docs/sphinx/qapidoc.py +3 -4
@@ -71,7 +71,6 @@ from typing import ( Any, Generator, - List, Optional, Sequence, Union,
@@ -674,7 +673,7 @@ def _highlightlang(self) -> addnodes.highlightlang: ) return node - def admonition_wrap(self, *content: nodes.Node) -> List[nodes.Node]: + def admonition_wrap(self, *content: nodes.Node) -> list[nodes.Node]: title = "Example:" if "title" in self.options: title = f"{title} {self.options['title']}"
@@ -687,7 +686,7 @@ def admonition_wrap(self, *content: nodes.Node) -> List[nodes.Node]: ) return [admon] - def run_annotated(self) -> List[nodes.Node]: + def run_annotated(self) -> list[nodes.Node]: lang_node = self._highlightlang() content_node: nodes.Element = nodes.section()
@@ -708,7 +707,7 @@ def run_annotated(self) -> List[nodes.Node]: return content_node.children - def run(self) -> List[nodes.Node]: + def run(self) -> list[nodes.Node]: annotated = "annotated" in self.options if annotated:
python/qemu/machine/machine.py +13 -17
@@ -31,12 +31,8 @@ from typing import ( Any, BinaryIO, - Dict, - List, Optional, Sequence, - Tuple, - Type, TypeVar, )
@@ -156,9 +152,9 @@ def __init__(self, self._qmp_timer = qmp_timer self._name = name or f"{id(self):x}" - self._sock_pair: Optional[Tuple[socket.socket, socket.socket]] = None + self._sock_pair: Optional[tuple[socket.socket, socket.socket]] = None self._cons_sock_pair: Optional[ - Tuple[socket.socket, socket.socket]] = None + tuple[socket.socket, socket.socket]] = None self._temp_dir: Optional[str] = None self._base_temp_dir = base_temp_dir self._log_dir = log_dir
@@ -176,11 +172,11 @@ def __init__(self, self._qemu_log_path: Optional[str] = None self._qemu_log_file: Optional[BinaryIO] = None self._popen: Optional['subprocess.Popen[bytes]'] = None - self._events: List[QMPMessage] = [] + self._events: list[QMPMessage] = [] self._iolog: Optional[str] = None self._qmp_set = True # Enable QMP monitor by default. self._qmp_connection: Optional[QEMUMonitorProtocol] = None - self._qemu_full_args: Tuple[str, ...] = () + self._qemu_full_args: tuple[str, ...] = () self._launched = False self._machine: Optional[str] = None self._console_index = 0
@@ -188,7 +184,7 @@ def __init__(self, self._console_device_type: Optional[str] = None self._console_socket: Optional[socket.socket] = None self._console_file: Optional[socket.SocketIO] = None - self._remove_files: List[str] = [] + self._remove_files: list[str] = [] self._user_killed = False self._quit_issued = False
@@ -196,7 +192,7 @@ def __enter__(self: _T) -> _T: return self def __exit__(self, - exc_type: Optional[Type[BaseException]], + exc_type: Optional[type[BaseException]], exc_val: Optional[BaseException], exc_tb: Optional[TracebackType]) -> None: self.shutdown()
@@ -292,7 +288,7 @@ def _load_io_log(self) -> None: self._iolog = iolog.read() @property - def _base_args(self) -> List[str]: + def _base_args(self) -> list[str]: args = ['-display', 'none', '-vga', 'none'] if self._qmp_set:
@@ -324,7 +320,7 @@ def _base_args(self) -> List[str]: return args @property - def args(self) -> List[str]: + def args(self) -> list[str]: """Returns the list of arguments given to the QEMU binary.""" return self._args
@@ -685,14 +681,14 @@ def _qmp(self) -> QEMUMonitorProtocol: @classmethod def _qmp_args(cls, conv_keys: bool, - args: Dict[str, Any]) -> Dict[str, object]: + args: dict[str, Any]) -> dict[str, object]: if conv_keys: return {k.replace('_', '-'): v for k, v in args.items()} return args def qmp(self, cmd: str, - args_dict: Optional[Dict[str, object]] = None, + args_dict: Optional[dict[str, object]] = None, conv_keys: Optional[bool] = None, **args: Any) -> QMPMessage: """
@@ -714,7 +710,7 @@ def qmp(self, cmd: str, return ret def cmd(self, cmd: str, - args_dict: Optional[Dict[str, object]] = None, + args_dict: Optional[dict[str, object]] = None, conv_keys: Optional[bool] = None, **args: Any) -> QMPReturnValue: """
@@ -745,7 +741,7 @@ def get_qmp_event(self, wait: bool = False) -> Optional[QMPMessage]: return self._events.pop(0) return self._qmp.pull_event(wait=wait) - def get_qmp_events(self, wait: bool = False) -> List[QMPMessage]: + def get_qmp_events(self, wait: bool = False) -> list[QMPMessage]: """ Poll for queued QMP events and return a list of dicts """
@@ -800,7 +796,7 @@ def event_wait(self, name: str, return self.events_wait([(name, match)], timeout) def events_wait(self, - events: Sequence[Tuple[str, Any]], + events: Sequence[tuple[str, Any]], timeout: float = 60.0) -> Optional[QMPMessage]: """ events_wait waits for and returns a single named event from QMP.
python/qemu/machine/qtest.py +3 -9
@@ -19,13 +19,7 @@ import os import socket -from typing import ( - List, - Optional, - Sequence, - TextIO, - Tuple, -) +from typing import Optional, Sequence, TextIO from qemu.qmp import SocketAddrT
@@ -145,10 +139,10 @@ def __init__(self, qmp_timer=qmp_timer) self._qtest: Optional[QEMUQtestProtocol] = None self._qtest_sock_pair: Optional[ - Tuple[socket.socket, socket.socket]] = None + tuple[socket.socket, socket.socket]] = None @property - def _base_args(self) -> List[str]: + def _base_args(self) -> list[str]: args = super()._base_args assert self._qtest_sock_pair is not None fd = self._qtest_sock_pair[0].fileno()
python/qemu/qmp/events.py +6 -9
@@ -455,10 +455,7 @@ def accept(self, event) -> bool: Callable, Iterable, Iterator, - List, Optional, - Set, - Tuple, Union, )
@@ -504,10 +501,10 @@ def __init__( self._queue: 'asyncio.Queue[Message]' = asyncio.Queue() # Intended as a historical record, NOT a processing queue or backlog. - self._history: List[Message] = [] + self._history: list[Message] = [] #: Primary event filter, based on one or more event names. - self.names: Set[str] = set() + self.names: set[str] = set() if isinstance(names, str): self.names.add(names) elif names is not None:
@@ -517,7 +514,7 @@ def __init__( self.event_filter: Optional[EventFilter] = event_filter def __repr__(self) -> str: - args: List[str] = [] + args: list[str] = [] if self.names: args.append(f"names={self.names!r}") if self.event_filter:
@@ -532,7 +529,7 @@ def __repr__(self) -> str: return f"{type(self).__name__}{state}({argstr})" @property - def history(self) -> Tuple[Message, ...]: + def history(self) -> tuple[Message, ...]: """ A read-only history of all events seen so far.
@@ -596,7 +593,7 @@ def empty(self) -> bool: """ return self._queue.empty() - def clear(self) -> List[Message]: + def clear(self) -> list[Message]: """ Clear this listener of all pending events.
@@ -650,7 +647,7 @@ class Events: property. """ def __init__(self) -> None: - self._listeners: List[EventListener] = [] + self._listeners: list[EventListener] = [] #: Default, all-events `EventListener`. See `qmp.events` for more info. self.events: EventListener = EventListener()
python/qemu/qmp/legacy.py +5 -8
@@ -27,10 +27,7 @@ from typing import ( Any, Awaitable, - Dict, - List, Optional, - Type, TypeVar, Union, )
@@ -41,13 +38,13 @@ #: QMPMessage is an entire QMP message of any kind. -QMPMessage = Dict[str, Any] +QMPMessage = dict[str, Any] #: QMPReturnValue is the 'return' value of a command. QMPReturnValue = object #: QMPObject is any object in a QMP message. -QMPObject = Dict[str, object] +QMPObject = dict[str, object] # QMPMessage can be outgoing commands or incoming events/returns. # QMPReturnValue is usually a dict/json object, but due to QAPI's
@@ -121,7 +118,7 @@ def __enter__(self: _T) -> _T: return self def __exit__(self, - exc_type: Optional[Type[BaseException]], + exc_type: Optional[type[BaseException]], exc_val: Optional[BaseException], exc_tb: Optional[TracebackType]) -> None: # Implement context manager exit function.
@@ -202,7 +199,7 @@ def cmd_obj(self, qmp_cmd: QMPMessage) -> QMPMessage: ) def cmd_raw(self, name: str, - args: Optional[Dict[str, object]] = None) -> QMPMessage: + args: Optional[dict[str, object]] = None) -> QMPMessage: """ Build a QMP command and send it to the QMP Monitor.
@@ -260,7 +257,7 @@ def pull_event(self, ) ) - def get_events(self, wait: Union[bool, float] = False) -> List[QMPMessage]: + def get_events(self, wait: Union[bool, float] = False) -> list[QMPMessage]: """ Get a list of QMP events and clear all pending events.
python/qemu/qmp/message.py +3 -4
@@ -8,7 +8,6 @@ import json from json import JSONDecodeError from typing import ( - Dict, Iterator, Mapping, MutableMapping,
@@ -71,7 +70,7 @@ def __init__(self, value: Union[bytes, Mapping[str, object]] = b'{}', *, eager: bool = True): self._data: Optional[bytes] = None - self._obj: Optional[Dict[str, object]] = None + self._obj: Optional[dict[str, object]] = None if isinstance(value, bytes): self._data = value
@@ -125,7 +124,7 @@ def __bytes__(self) -> bytes: # Conversion Methods @property - def _object(self) -> Dict[str, object]: + def _object(self) -> dict[str, object]: """ A `dict` representing this QMP message.
@@ -150,7 +149,7 @@ def _serialize(cls, value: object) -> bytes: return json.dumps(value, separators=(',', ':')).encode('utf-8') @classmethod - def _deserialize(cls, data: bytes) -> Dict[str, object]: + def _deserialize(cls, data: bytes) -> dict[str, object]: """ Deserialize JSON `bytes` into a native Python `dict`.
python/qemu/qmp/models.py +1 -2
@@ -11,7 +11,6 @@ import copy from typing import ( Any, - Dict, Mapping, Optional, Sequence,
@@ -72,7 +71,7 @@ def __init__(self, raw: Mapping[str, Any]): self._check_member('QMP', abc.Mapping, "JSON object") self.QMP = QMPGreeting(self._raw['QMP']) - def _asdict(self) -> Dict[str, object]: + def _asdict(self) -> dict[str, object]: """ For compatibility with the iotests sync QMP wrapper.
python/qemu/qmp/protocol.py +3 -5
@@ -28,9 +28,7 @@ Awaitable, Callable, Generic, - List, Optional, - Tuple, TypeVar, Union, cast,
@@ -50,7 +48,7 @@ _U = TypeVar('_U') _TaskFN = Callable[[], Awaitable[None]] # aka ``async def func() -> None`` -InternetAddrT = Tuple[str, int] +InternetAddrT = tuple[str, int] UnixAddrT = str SocketAddrT = Union[UnixAddrT, InternetAddrT]
@@ -248,7 +246,7 @@ def __init__(self, name: Optional[str] = None) -> None: self._writer_task: Optional[asyncio.Future[None]] = None # Aggregate of the above two tasks, used for Exception management. - self._bh_tasks: Optional[asyncio.Future[Tuple[None, None]]] = None + self._bh_tasks: Optional[asyncio.Future[tuple[None, None]]] = None #: Disconnect task. The disconnect implementation runs in a task #: so that asynchronous disconnects (initiated by the
@@ -744,7 +742,7 @@ async def _wait_disconnect(self) -> None: assert self.runstate == Runstate.DISCONNECTING assert self._dc_task - aws: List[Awaitable[object]] = [self._dc_task] + aws: list[Awaitable[object]] = [self._dc_task] if self._bh_tasks: aws.insert(0, self._bh_tasks) all_defined_tasks = asyncio.gather(*aws)
python/qemu/qmp/qmp_client.py +2 -4
@@ -12,8 +12,6 @@ import socket import struct from typing import ( - Dict, - List, Mapping, Optional, Union,
@@ -262,7 +260,7 @@ def __init__(self, name: Optional[str] = None) -> None: self._execute_id = 0 # Incoming RPC reply messages. - self._pending: Dict[ + self._pending: dict[ Union[str, None], 'asyncio.Queue[QMPClient._PendingT]' ] = {}
@@ -339,7 +337,7 @@ async def _negotiate(self) -> None: """ self.logger.debug("Negotiating capabilities ...") - arguments: Dict[str, List[str]] = {} + arguments: dict[str, list[str]] = {} if self._greeting and 'oob' in self._greeting.QMP.capabilities: arguments.setdefault('enable', []).append('oob') msg = self.make_execute_msg('qmp_capabilities', arguments=arguments)
python/qemu/qmp/qmp_shell.py +4 -6
@@ -134,9 +134,7 @@ import sys from typing import ( IO, - Dict, Iterator, - List, NoReturn, Optional, Sequence,
@@ -165,9 +163,9 @@ class QMPCompleter: QMPCompleter provides a readline library tab-complete behavior. """ # NB: Python 3.9+ will probably allow us to subclass list[str] directly, - # but pylint as of today does not know that List[str] is simply 'list'. + # but pylint as of today does not know that list[str] is simply 'list'. def __init__(self) -> None: - self._matches: List[str] = [] + self._matches: list[str] = [] def append(self, value: str) -> None: """Append a new valid completion to the list of possibilities."""
@@ -229,7 +227,7 @@ def __init__(self, address: SocketAddrT, self._greeting: Optional[QMPMessage] = None self._completer = QMPCompleter() self._transmode = False - self._actions: List[QMPMessage] = [] + self._actions: list[QMPMessage] = [] self._histfile = os.path.join(os.path.expanduser('~'), '.qmp-shell_history') self.pretty = pretty
@@ -246,7 +244,7 @@ def close(self) -> None: def _fill_completion(self) -> None: try: - cmds = cast(List[Dict[str, str]], self.cmd('query-commands')) + cmds = cast(list[dict[str, str]], self.cmd('query-commands')) for cmd in cmds: self._completer.append(cmd['name']) except ExecuteError:
python/qemu/qmp/qmp_tui.py +7 -14
@@ -22,14 +22,7 @@ from logging import Handler, LogRecord import signal import sys -from typing import ( - List, - Optional, - Tuple, - Type, - Union, - cast, -) +from typing import Optional, Union, cast try:
@@ -120,7 +113,7 @@ def format_json(msg: str) -> str: def has_handler_type(logger: logging.Logger, - handler_type: Type[Handler]) -> bool: + handler_type: type[Handler]) -> bool: """ The Logger class has no interface to check if a certain type of handler is installed or not. So we provide an interface to do so.
@@ -151,7 +144,7 @@ class App(QMPClient): :param retry_delay: The delay(sec) before each retry """ - def __init__(self, address: Union[str, Tuple[str, int]], num_retries: int, + def __init__(self, address: Union[str, tuple[str, int]], num_retries: int, retry_delay: Optional[int]) -> None: urwid.register_signal(type(self), UPDATE_MSG) self.window = Window(self)
@@ -448,11 +441,11 @@ class Editor(urwid_readline.ReadlineEdit): def __init__(self, parent: App) -> None: super().__init__(caption='> ', multiline=True) self.parent = parent - self.history: List[str] = [] + self.history: list[str] = [] self.last_index: int = -1 self.show_history: bool = False - def keypress(self, size: Tuple[int, int], key: str) -> Optional[str]: + def keypress(self, size: tuple[int, int], key: str) -> Optional[str]: """ Handles the keypress on this widget.
@@ -530,7 +523,7 @@ def __init__(self, parent: App) -> None: super().__init__(self.history) def add_to_history(self, - history: Union[str, List[Tuple[str, str]]]) -> None: + history: Union[str, list[tuple[str, str]]]) -> None: """ Appends a message to the list and set the focus to the last appended message.
@@ -541,7 +534,7 @@ def add_to_history(self, self.history.append(urwid.Text(history)) self.history.set_focus(len(self.history) - 1) - def mouse_event(self, size: Tuple[int, int], _event: str, button: float, + def mouse_event(self, size: tuple[int, int], _event: str, button: float, _x: int, _y: int, focus: bool) -> None: # Unfortunately there are no urwid constants that represent the mouse # events.
python/qemu/utils/accel.py +2 -2
@@ -17,7 +17,7 @@ import logging import os import subprocess -from typing import List, Optional +from typing import Optional LOG = logging.getLogger(__name__)
@@ -31,7 +31,7 @@ } -def list_accel(qemu_bin: str) -> List[str]: +def list_accel(qemu_bin: str) -> list[str]: """ List accelerators enabled in the QEMU binary.
python/qemu/utils/qemu_ga_client.py +1 -2
@@ -45,7 +45,6 @@ from typing import ( Any, Callable, - Dict, Optional, Sequence, )
@@ -159,7 +158,7 @@ def fsfreeze(self, cmd: str) -> object: # Can be int (freeze, thaw) or GuestFsfreezeStatus (status) return getattr(self.qga, 'fsfreeze' + '_' + cmd)() - def fstrim(self, minimum: int) -> Dict[str, object]: + def fstrim(self, minimum: int) -> dict[str, object]: # returns GuestFilesystemTrimResponse ret = getattr(self.qga, 'fstrim')(minimum=minimum) assert isinstance(ret, dict)
python/qemu/utils/qom_common.py +5 -12
@@ -18,14 +18,7 @@ import argparse import os import sys -from typing import ( - Any, - Dict, - List, - Optional, - Type, - TypeVar, -) +from typing import Any, Optional, TypeVar from qemu.qmp import QMPError from qemu.qmp.legacy import QEMUMonitorProtocol
@@ -44,7 +37,7 @@ def __init__(self, name: str, type_: str, self.default_value = default_value @classmethod - def make(cls, value: Dict[str, Any]) -> 'ObjectPropertyInfo': + def make(cls, value: dict[str, Any]) -> 'ObjectPropertyInfo': """ Build an ObjectPropertyInfo from a Dict with an unknown shape. """
@@ -136,18 +129,18 @@ def run(self) -> int: """ raise NotImplementedError - def qom_list(self, path: str) -> List[ObjectPropertyInfo]: + def qom_list(self, path: str) -> list[ObjectPropertyInfo]: """ :return: a strongly typed list from the 'qom-list' command. """ rsp = self.qmp.cmd('qom-list', path=path) - # qom-list returns List[ObjectPropertyInfo] + # qom-list returns list[ObjectPropertyInfo] assert isinstance(rsp, list) return [ObjectPropertyInfo.make(x) for x in rsp] @classmethod def command_runner( - cls: Type[CommandT], + cls: type[CommandT], args: argparse.Namespace ) -> int: """
python/qemu/utils/qom_fuse.py +1 -2
@@ -38,7 +38,6 @@ import sys from typing import ( IO, - Dict, Iterator, Mapping, Optional,
@@ -79,7 +78,7 @@ def configure_parser(cls, parser: argparse.ArgumentParser) -> None: def __init__(self, args: argparse.Namespace): super().__init__(args) self.mount = args.mount - self.ino_map: Dict[str, int] = {} + self.ino_map: dict[str, int] = {} self.ino_count = 1 def run(self) -> int:
python/scripts/mkvenv.py +6 -8
@@ -78,11 +78,9 @@ from types import SimpleNamespace from typing import ( Any, - Dict, Iterator, Optional, Sequence, - Tuple, Union, ) import venv
@@ -510,7 +508,7 @@ def diagnose( online: bool, wheels_dir: Optional[Union[str, Path]], prog: Optional[str], -) -> Tuple[str, bool]: +) -> tuple[str, bool]: """ Offer a summary to the user as to why a package failed to be installed.
@@ -623,7 +621,7 @@ def pip_install( ) -def _make_version_constraint(info: Dict[str, str], install: bool) -> str: +def _make_version_constraint(info: dict[str, str], install: bool) -> str: """ Construct the version constraint part of a PEP 508 dependency specification (for example '>=0.61.5') from the accepted and
@@ -648,10 +646,10 @@ def _make_version_constraint(info: Dict[str, str], install: bool) -> str: def _do_ensure( - group: Dict[str, Dict[str, str]], + group: dict[str, dict[str, str]], online: bool = False, wheels_dir: Optional[Union[str, Path]] = None, -) -> Optional[Tuple[str, bool]]: +) -> Optional[tuple[str, bool]]: """ Use pip to ensure we have the packages specified in @group.
@@ -716,7 +714,7 @@ def _do_ensure( return None -def _parse_groups(file: str) -> Dict[str, Dict[str, Any]]: +def _parse_groups(file: str) -> dict[str, dict[str, Any]]: if not HAVE_TOMLLIB: if sys.version_info < (3, 11): raise Ouch("found no usable tomli, please install it")
@@ -756,7 +754,7 @@ def ensure_group( parsed_deps = _parse_groups(file) - to_install: Dict[str, Dict[str, str]] = {} + to_install: dict[str, dict[str, str]] = {} for group in groups: try: to_install.update(parsed_deps[group])
scripts/codeconverter/codeconverter/patching.py +22 -22
154 lines not on this page · raw message
scripts/codeconverter/codeconverter/qom_macros.py +18 -18
@@ -68,7 +68,7 @@ def __str__(self) -> str: s = ', '.join('%s=%s' % (f,v) for f,v in values if v is not None) return f'{s}' - def check_consistency(self) -> List[str]: + def check_consistency(self) -> list[str]: """Check if identifiers are consistent with each other, return list of problems (or empty list if everything seems consistent) """
@@ -235,7 +235,7 @@ def find_typedef(f: FileInfo, name: Optional[str]) -> Optional[TypedefMatch]: NAMED('qom_typename', RE_IDENTIFIER), r'\s*\)\n', n='?', name='check_args')) -EXPECTED_CHECKER_SUFFIXES: List[Tuple[CheckerMacroName, str]] = [ +EXPECTED_CHECKER_SUFFIXES: list[tuple[CheckerMacroName, str]] = [ ('OBJECT_GET_CLASS', '_GET_CLASS'), ('OBJECT_CLASS_CHECK', '_CLASS'), ]
@@ -267,7 +267,7 @@ def sanity_check(self) -> None: if self.typedefname and self.find_typedef() is None: self.warn("typedef used by %s not found", self.name) - def find_matching_macros(self) -> List['TypeCheckMacro']: + def find_matching_macros(self) -> list['TypeCheckMacro']: """Find other check macros that generate the same macro names The returned list will always be sorted.
@@ -280,7 +280,7 @@ def find_matching_macros(self) -> List['TypeCheckMacro']: and (my_ids.uppercase == m.type_identifiers.uppercase or my_ids.typename == m.type_identifiers.typename)] - def merge_ids(self, matches: List['TypeCheckMacro']) -> Optional[TypeIdentifiers]: + def merge_ids(self, matches: list['TypeCheckMacro']) -> Optional[TypeIdentifiers]: """Try to merge info about type identifiers from all matches in a list""" if not matches: return None
@@ -580,10 +580,10 @@ def gen_patches(self) -> Iterable[Patch]: return # group checkers by uppercase name: - decl_types: List[Type[TypeDeclaration]] = [DeclareInstanceChecker, DeclareInstanceType, + decl_types: list[type[TypeDeclaration]] = [DeclareInstanceChecker, DeclareInstanceType, DeclareClassCheckers, DeclareClassType, DeclareObjCheckers] - checker_dict: Dict[str, List[TypeDeclaration]] = {} + checker_dict: dict[str, list[TypeDeclaration]] = {} for t in decl_types: for m in self.file.matches_of_type(t): checker_dict.setdefault(m.group('uppercase'), []).append(m)
@@ -602,9 +602,9 @@ def gen_patches(self) -> Iterable[Patch]: field_dict = dict((f, v.pop() if v else None) for f,v in fvalues.items()) yield from self.gen_patches_for_type(uppercase, checkers, field_dict) - def find_conflicts(self, uppercase: str, checkers: List[TypeDeclaration]) -> bool: + def find_conflicts(self, uppercase: str, checkers: list[TypeDeclaration]) -> bool: """Look for conflicting declarations that would make it unsafe to add new ones""" - conflicting: List[FileMatch] = [] + conflicting: list[FileMatch] = [] # conflicts in the same file: conflicting.extend(chain(self.file.find_matches(DefineDirective, uppercase), self.file.find_matches(DeclareInterfaceChecker, uppercase, 'uppercase'),
@@ -633,8 +633,8 @@ def find_conflicts(self, uppercase: str, checkers: List[TypeDeclaration]) -> boo return False def gen_patches_for_type(self, uppercase: str, - checkers: List[TypeDeclaration], - fields: Dict[str, Optional[str]]) -> Iterable[Patch]: + checkers: list[TypeDeclaration], + fields: dict[str, Optional[str]]) -> Iterable[Patch]: """Should be reimplemented by subclasses""" return yield
@@ -643,8 +643,8 @@ class DeclareVoidTypes(TypeDeclarationFixup): """Add DECLARE_*_TYPE(..., void) when there's no declared type""" regexp = RE_FILE_BEGIN def gen_patches_for_type(self, uppercase: str, - checkers: List[TypeDeclaration], - fields: Dict[str, Optional[str]]) -> Iterable[Patch]: + checkers: list[TypeDeclaration], + fields: dict[str, Optional[str]]) -> Iterable[Patch]: if self.find_conflicts(uppercase, checkers): return
@@ -671,8 +671,8 @@ def gen_patches_for_type(self, uppercase: str, class AddDeclareTypeName(TypeDeclarationFixup): """Add DECLARE_TYPE_NAME declarations if necessary""" def gen_patches_for_type(self, uppercase: str, - checkers: List[TypeDeclaration], - fields: Dict[str, Optional[str]]) -> Iterable[Patch]: + checkers: list[TypeDeclaration], + fields: dict[str, Optional[str]]) -> Iterable[Patch]: typename = fields.get('typename') if typename is None: self.warn("typename unavailable")
@@ -753,7 +753,7 @@ def find_typename_uppercase(files: FileList, typename: str) -> Optional[str]: def find_type_checkers(files:FileList, name:str, group:str='uppercase') -> Iterable[TypeCheckerDeclaration]: """Find usage of DECLARE*CHECKER macro""" - c: Type[TypeCheckerDeclaration] + c: type[TypeCheckerDeclaration] for c in (DeclareInstanceChecker, DeclareClassCheckers, DeclareObjCheckers, ObjectDeclareType, ObjectDeclareSimpleType): yield from files.find_matches(c, name=name, group=group)
@@ -774,8 +774,8 @@ class InitialIncludes(FileMatch): n='*', name='includes')) class SymbolUserList(NamedTuple): - definitions: List[FileMatch] - users: List[FileMatch] + definitions: list[FileMatch] + users: list[FileMatch] class MoveSymbols(FileMatch): """Handle missing symbols
@@ -789,7 +789,7 @@ def gen_patches(self) -> Iterator[Patch]: self.debug("skipping object.h") return - index: Dict[RequiredIdentifier, SymbolUserList] = {} + index: dict[RequiredIdentifier, SymbolUserList] = {} definition_classes = [SimpleTypedefMatch, FullStructTypedefMatch, ConstantDefine, Include] user_classes = [TypeCheckMacro, DeclareObjCheckers, DeclareInstanceChecker, DeclareClassCheckers, InterfaceCheckMacro]
scripts/codeconverter/codeconverter/qom_type_info.py +8 -8
@@ -25,7 +25,7 @@ RE_TYPEINFO_START = S(r'^[ \t]*', M(r'(static|const)\s+', name='modifiers'), r'TypeInfo\s+', NAMED('name', RE_IDENTIFIER), r'\s*=\s*{[ \t]*\n') -ParsedArray = List[str] +ParsedArray = list[str] ParsedInitializerValue = Union[str, ParsedArray] class InitializerValue(NamedTuple): raw: str
@@ -59,7 +59,7 @@ def parsed(self) -> ParsedInitializerValue: return array.parsed() return parsed -TypeInfoInitializers = Dict[str, FieldInitializer] +TypeInfoInitializers = dict[str, FieldInitializer] class TypeDefinition(FileMatch): """
@@ -343,7 +343,7 @@ def gen_patches(self) -> Iterable[Patch]: ok = True - #checkers: List[TypeCheckerDeclaration] = list(find_type_checkers(self.allfiles, uppercase)) + #checkers: list[TypeCheckerDeclaration] = list(find_type_checkers(self.allfiles, uppercase)) #for c in checkers: # c.info("instance type checker declaration (%s) is here", c.group('uppercase')) #if not checkers:
@@ -448,7 +448,7 @@ class ObjectDefineType(TypeDefinition): r'\s*\);?\n?') def find_type_definitions(files: FileList, uppercase: str) -> Iterable[TypeDefinition]: - types: List[Type[TypeDefinition]] = [TypeInfoVar, ObjectDefineType, ObjectDefineTypeExtended] + types: list[type[TypeDefinition]] = [TypeInfoVar, ObjectDefineType, ObjectDefineTypeExtended] for t in types: for m in files.matches_of_type(t): m.debug("uppercase: %s", m.uppercase)
@@ -459,8 +459,8 @@ def find_type_definitions(files: FileList, uppercase: str) -> Iterable[TypeDefin class AddDeclareVoidClassType(TypeDeclarationFixup): """Will add DECLARE_CLASS_TYPE(..., void) if possible""" def gen_patches_for_type(self, uppercase: str, - checkers: List[TypeDeclaration], - fields: Dict[str, Optional[str]]) -> Iterable[Patch]: + checkers: list[TypeDeclaration], + fields: dict[str, Optional[str]]) -> Iterable[Patch]: defs = list(find_type_definitions(self.allfiles, uppercase)) if len(defs) > 1: self.warn("multiple definitions for %s", uppercase)
@@ -554,7 +554,7 @@ def gen_patches(self) -> Iterable[Patch]: if not self.file.force: return - decl_types: List[Type[TypeDeclaration]] = [DeclareClassCheckers, DeclareObjCheckers] + decl_types: list[type[TypeDeclaration]] = [DeclareClassCheckers, DeclareObjCheckers] class_decls = [m for t in decl_types for m in self.allfiles.find_matches(t, uppercase, 'uppercase')]
@@ -634,7 +634,7 @@ def gen_patches(self) -> Iterable[Patch]: if not self.file.force: return - decl_types: List[Type[TypeDeclaration]] = [DeclareClassCheckers, DeclareObjCheckers] + decl_types: list[type[TypeDeclaration]] = [DeclareClassCheckers, DeclareObjCheckers] class_decls = [m for t in decl_types for m in self.allfiles.find_matches(t, uppercase, 'uppercase')] if class_decls:
scripts/compare-machine-types.py +20 -20
@@ -31,7 +31,7 @@ from argparse import ArgumentParser, RawTextHelpFormatter, Namespace import pandas as pd from contextlib import ExitStack -from typing import Optional, List, Dict, Generator, Tuple, Union, Any, Set +from typing import Optional, Generator, Union, Any try: qemu_dir = path.abspath(path.dirname(path.dirname(__file__)))
@@ -81,7 +81,7 @@ def is_child_of(self, parent: 'Driver') -> bool: return False - def set_implementations(self, implementations: List['Driver']) -> None: + def set_implementations(self, implementations: list['Driver']) -> None: self.implementations = implementations
@@ -89,7 +89,7 @@ class QEMUObject(Driver): def __init__(self, vm: QEMUMachine, name: str) -> None: super().__init__(vm, name, True) - def set_implementations(self, implementations: List[Driver]) -> None: + def set_implementations(self, implementations: list[Driver]) -> None: self.implementations = implementations # each implementation of the abstract driver has to use property getter
@@ -105,7 +105,7 @@ def set_implementations(self, implementations: List[Driver]) -> None: class QEMUDevice(QEMUObject): def __init__(self, vm: QEMUMachine) -> None: super().__init__(vm, 'device') - self.cached: Dict[str, List[Dict[str, Any]]] = {} + self.cached: dict[str, list[dict[str, Any]]] = {} def get_prop(self, driver: str, prop_name: str) -> str: if driver not in self.cached:
@@ -121,7 +121,7 @@ def get_prop(self, driver: str, prop_name: str) -> str: class QEMUx86CPU(QEMUObject): def __init__(self, vm: QEMUMachine) -> None: super().__init__(vm, 'x86_64-cpu') - self.cached: Dict[str, Dict[str, Any]] = {} + self.cached: dict[str, dict[str, Any]] = {} def get_prop(self, driver: str, prop_name: str) -> str: if not driver.endswith('-x86_64-cpu'):
@@ -141,7 +141,7 @@ def get_prop(self, driver: str, prop_name: str) -> str: class QEMUMemoryBackend(QEMUObject): def __init__(self, vm: QEMUMachine) -> None: super().__init__(vm, 'memory-backend') - self.cached: Dict[str, List[Dict[str, Any]]] = {} + self.cached: dict[str, list[dict[str, Any]]] = {} def get_prop(self, driver: str, prop_name: str) -> str: if driver not in self.cached:
@@ -172,7 +172,7 @@ class VMPropertyGetter: """It implements the relationship between drivers and how to get their properties""" def __init__(self, vm: QEMUMachine) -> None: - self.drivers: Dict[str, Driver] = {} + self.drivers: dict[str, Driver] = {} qom_all_types = vm.cmd('qom-list-types', abstract=True) self.drivers = {t['name']: new_driver(vm, t['name'],
@@ -201,7 +201,7 @@ def get_prop(self, driver: str, prop: str) -> str: return drv.get_prop(driver, prop) - def get_implementations(self, driver: str) -> List[str]: + def get_implementations(self, driver: str) -> list[str]: return [impl.name for impl in self.drivers[driver].implementations]
@@ -211,10 +211,10 @@ class Machine: implementations) """ # raw_mt_dict - dict produced by `query-machines` - def __init__(self, raw_mt_dict: Dict[str, Any], + def __init__(self, raw_mt_dict: dict[str, Any], qemu_drivers: VMPropertyGetter) -> None: self.name = raw_mt_dict['name'] - self.compat_props: Dict[str, Any] = {} + self.compat_props: dict[str, Any] = {} # properties are applied sequentially and can rewrite values like in # QEMU. Also it has to resolve class relationships to apply appropriate # values from abstract class to all implementations
@@ -239,7 +239,7 @@ class Configuration(): """Class contains all necessary components to generate table and is used to compare different binaries""" def __init__(self, vm: QEMUMachine, - req_mt: List[str], all_mt: bool) -> None: + req_mt: list[str], all_mt: bool) -> None: self._vm = vm self._binary = vm.binary self._qemu_args = args.qemu_args.split(' ')
@@ -247,11 +247,11 @@ def __init__(self, vm: QEMUMachine, self._qemu_drivers = VMPropertyGetter(vm) self.req_mt = get_req_mt(self._qemu_drivers, vm, req_mt, all_mt) - def get_implementations(self, driver_name: str) -> List[str]: + def get_implementations(self, driver_name: str) -> list[str]: return self._qemu_drivers.get_implementations(driver_name) - def get_table(self, req_props: List[Tuple[str, str]]) -> pd.DataFrame: - table: List[pd.DataFrame] = [] + def get_table(self, req_props: list[tuple[str, str]]) -> pd.DataFrame: + table: list[pd.DataFrame] = [] for mt in self.req_mt: name = f'{self._binary}\n{mt.name}' column = []
@@ -339,7 +339,7 @@ def parse_args() -> Namespace: return parser.parse_args() -def mt_comp(mt: Machine) -> Tuple[str, int, int, int]: +def mt_comp(mt: Machine) -> tuple[str, int, int, int]: """Function to compare and sort machine by names. It returns socket_name, major version, minor version, revision""" # none, microvm, x-remote and etc.
@@ -353,7 +353,7 @@ def mt_comp(mt: Machine) -> Tuple[str, int, int, int]: def get_mt_definitions(qemu_drivers: VMPropertyGetter, - vm: QEMUMachine) -> List[Machine]: + vm: QEMUMachine) -> list[Machine]: """Constructs list of machine definitions (primarily compat_props) via info from QEMU""" raw_mt_defs = vm.cmd('query-machines', compat_props=True)
@@ -366,7 +366,7 @@ def get_mt_definitions(qemu_drivers: VMPropertyGetter, def get_req_mt(qemu_drivers: VMPropertyGetter, vm: QEMUMachine, - req_mt: Optional[List[str]], all_mt: bool) -> List[Machine]: + req_mt: Optional[list[str]], all_mt: bool) -> list[Machine]: """Returns list of requested by user machines""" mt_defs = get_mt_definitions(qemu_drivers, vm) if all_mt:
@@ -384,12 +384,12 @@ def get_req_mt(qemu_drivers: VMPropertyGetter, vm: QEMUMachine, return matched_mt -def get_affected_props(configs: List[Configuration]) -> Generator[Tuple[str, +def get_affected_props(configs: list[Configuration]) -> Generator[tuple[str, str], None, None]: """Helps to go through all affected in machine definitions drivers and properties""" - driver_props: Dict[str, Set[Any]] = {} + driver_props: dict[str, set[Any]] = {} for config in configs: for mt in config.req_mt: compat_props = mt.compat_props
@@ -437,7 +437,7 @@ def simplify_table(table: pd.DataFrame) -> pd.DataFrame: # driver2 | property3 | value5 | value6 | ... # ... | ... | ... | ... | ... # -def fill_prop_table(configs: List[Configuration], +def fill_prop_table(configs: list[Configuration], is_raw: bool) -> pd.DataFrame: req_props = list(get_affected_props(configs)) if not req_props:
scripts/qapi/commands.py +4 -9
@@ -13,12 +13,7 @@ See the COPYING file in the top-level directory. """ -from typing import ( - Dict, - List, - Optional, - Set, -) +from typing import Optional from .common import c_name, mcgen from .gen import (
@@ -276,7 +271,7 @@ def gen_marshal(name: str, def gen_register_command(name: str, - features: List[QAPISchemaFeature], + features: list[QAPISchemaFeature], success_response: bool, allow_oob: bool, allow_preconfig: bool,
@@ -308,7 +303,7 @@ def __init__(self, prefix: str, gen_tracing: bool): prefix, 'qapi-commands', ' * Schema-defined QAPI/QMP commands', None, __doc__, gen_tracing=gen_tracing) - self._visited_ret_types: Dict[QAPIGenC, Set[QAPISchemaType]] = {} + self._visited_ret_types: dict[QAPIGenC, set[QAPISchemaType]] = {} self._gen_tracing = gen_tracing def _begin_user_module(self, name: str) -> None:
@@ -375,7 +370,7 @@ def visit_command(self, name: str, info: Optional[QAPISourceInfo], ifcond: QAPISchemaIfCond, - features: List[QAPISchemaFeature], + features: list[QAPISchemaFeature], arg_type: Optional[QAPISchemaObjectType], ret_type: Optional[QAPISchemaType], gen: bool,
scripts/qapi/common.py +4 -5
@@ -14,7 +14,6 @@ import re from typing import ( Any, - Dict, Match, Optional, Sequence,
@@ -199,11 +198,11 @@ def guardend(name: str) -> str: name=c_fname(name).upper()) -def gen_ifcond(ifcond: Optional[Union[str, Dict[str, Any]]], +def gen_ifcond(ifcond: Optional[Union[str, dict[str, Any]]], cond_fmt: str, not_fmt: str, all_operator: str, any_operator: str) -> str: - def do_gen(ifcond: Union[str, Dict[str, Any]], + def do_gen(ifcond: Union[str, dict[str, Any]], need_parens: bool) -> str: if isinstance(ifcond, str): return cond_fmt % ifcond
@@ -226,11 +225,11 @@ def gen_infix(operator: str, operands: Sequence[Any]) -> str: return do_gen(ifcond, False) -def cgen_ifcond(ifcond: Optional[Union[str, Dict[str, Any]]]) -> str: +def cgen_ifcond(ifcond: Optional[Union[str, dict[str, Any]]]) -> str: return gen_ifcond(ifcond, 'defined(%s)', '!%s', ' && ', ' || ') -def docgen_ifcond(ifcond: Optional[Union[str, Dict[str, Any]]]) -> str: +def docgen_ifcond(ifcond: Optional[Union[str, dict[str, Any]]]) -> str: # TODO Doc generated for conditions needs polish return gen_ifcond(ifcond, '%s', 'not %s', ' and ', ' or ')
scripts/qapi/events.py +4 -4
@@ -12,7 +12,7 @@ See the COPYING file in the top-level directory. """ -from typing import List, Optional +from typing import Optional from .common import c_enum_const, c_name, mcgen from .gen import QAPISchemaModularCVisitor, build_params, ifcontext
@@ -80,7 +80,7 @@ def gen_param_var(typ: QAPISchemaObjectType) -> str: def gen_event_send(name: str, arg_type: Optional[QAPISchemaObjectType], - features: List[QAPISchemaFeature], + features: list[QAPISchemaFeature], boxed: bool, event_enum_name: str, event_emit: str) -> str:
@@ -180,7 +180,7 @@ def __init__(self, prefix: str): prefix, 'qapi-events', ' * Schema-defined QAPI/QMP events', None, __doc__) self._event_enum_name = c_name(prefix + 'QAPIEvent', protect=False) - self._event_enum_members: List[QAPISchemaEnumMember] = [] + self._event_enum_members: list[QAPISchemaEnumMember] = [] self._event_emit_name = c_name(prefix + 'qapi_event_emit') def _begin_user_module(self, name: str) -> None:
@@ -230,7 +230,7 @@ def visit_event(self, name: str, info: Optional[QAPISourceInfo], ifcond: QAPISchemaIfCond, - features: List[QAPISchemaFeature], + features: list[QAPISchemaFeature], arg_type: Optional[QAPISchemaObjectType], boxed: bool) -> None: with ifcontext(ifcond, self._genh, self._genc):
scripts/qapi/expr.py +9 -11
@@ -33,9 +33,7 @@ import re from typing import ( - Dict, Iterable, - List, Optional, Union, cast,
@@ -183,11 +181,11 @@ def check_defn_name_str(name: str, info: QAPISourceInfo, meta: str) -> None: info, "%s name should not end in 'List'" % meta) -def check_keys(value: Dict[str, object], +def check_keys(value: dict[str, object], info: QAPISourceInfo, source: str, - required: List[str], - optional: List[str]) -> None: + required: list[str], + optional: list[str]) -> None: """ Ensure that a dict has a specific set of keys.
@@ -247,7 +245,7 @@ def check_flags(expr: QAPIExpression) -> None: expr.info, "flags 'allow-oob' and 'coroutine' are incompatible") -def check_if(expr: Dict[str, object], +def check_if(expr: dict[str, object], info: QAPISourceInfo, source: str) -> None: """ Validate the ``if`` member of an object.
@@ -319,8 +317,8 @@ def normalize_members(members: object) -> None: ``{'type': value}``. :forms: - :sugared: ``Dict[str, Union[str, TypeRef]]`` - :canonical: ``Dict[str, TypeRef]`` + :sugared: ``dict[str, Union[str, TypeRef]]`` + :canonical: ``dict[str, TypeRef]`` :param members: The members value to normalize.
@@ -416,8 +414,8 @@ def check_features(features: Optional[object], Any ``str`` element will be normalized to ``{'name': element}``. :forms: - :sugared: ``List[Union[str, Feature]]`` - :canonical: ``List[Feature]`` + :sugared: ``list[Union[str, Feature]]`` + :canonical: ``list[Feature]`` :param features: The features member value to validate. :param info: QAPI schema source file information.
@@ -592,7 +590,7 @@ def check_event(expr: QAPIExpression) -> None: check_type_name_or_implicit(args, expr.info, "'data'", None) -def check_exprs(exprs: List[QAPIExpression]) -> List[QAPIExpression]: +def check_exprs(exprs: list[QAPIExpression]) -> list[QAPIExpression]: """ Validate and normalize a list of parsed QAPI schema expressions.
scripts/qapi/gen.py +3 -9
@@ -15,13 +15,7 @@ import os import re import sys -from typing import ( - Dict, - Iterator, - Optional, - Sequence, - Tuple, -) +from typing import Iterator, Optional, Sequence from .common import ( c_enum_const,
@@ -136,7 +130,7 @@ def build_params(arg_type: Optional[QAPISchemaObjectType], class QAPIGenCCode(QAPIGen): def __init__(self, fname: str): super().__init__(fname) - self._start_if: Optional[Tuple[QAPISchemaIfCond, str, str]] = None + self._start_if: Optional[tuple[QAPISchemaIfCond, str, str]] = None def start_if(self, ifcond: QAPISchemaIfCond) -> None: assert self._start_if is None
@@ -264,7 +258,7 @@ def __init__(self, self._builtin_blurb = builtin_blurb self._pydoc = pydoc self._current_module: Optional[str] = None - self._module: Dict[str, Tuple[QAPIGenC, QAPIGenH, + self._module: dict[str, tuple[QAPIGenC, QAPIGenH, Optional[QAPIGenTrace]]] = {} self._main_module: Optional[str] = None self._gen_tracing = gen_tracing
scripts/qapi/introspect.py +20 -22
111 lines not on this page · raw message
scripts/qapi/parser.py +17 -20
@@ -20,12 +20,9 @@ from typing import ( TYPE_CHECKING, Any, - Dict, - List, Mapping, Match, Optional, - Set, Union, )
@@ -41,10 +38,10 @@ # Return value alias for get_expr(). -_ExprValue = Union[List[object], Dict[str, object], str, bool] +_ExprValue = Union[list[object], dict[str, object], str, bool] -class QAPIExpression(Dict[str, Any]): +class QAPIExpression(dict[str, Any]): # pylint: disable=too-few-public-methods def __init__(self, data: Mapping[str, object],
@@ -91,7 +88,7 @@ class QAPISchemaParser: """ def __init__(self, fname: str, - previously_included: Optional[Set[str]] = None, + previously_included: Optional[set[str]] = None, incl_info: Optional[QAPISourceInfo] = None): self._fname = fname self._included = previously_included or set()
@@ -107,8 +104,8 @@ def __init__(self, self.line_pos = 0 # Parser output: - self.exprs: List[QAPIExpression] = [] - self.docs: List[QAPIDoc] = [] + self.exprs: list[QAPIExpression] = [] + self.docs: list[QAPIDoc] = [] # Showtime! self._parse()
@@ -195,7 +192,7 @@ def reject_expr_doc(doc: Optional['QAPIDoc']) -> None: def _include(include: str, info: QAPISourceInfo, incl_fname: str, - previously_included: Set[str] + previously_included: set[str] ) -> Optional['QAPISchemaParser']: incl_abs_fname = os.path.abspath(incl_fname) # catch inclusion cycle
@@ -220,7 +217,7 @@ def _include(include: str, @staticmethod def _pragma(name: str, value: object, info: QAPISourceInfo) -> None: - def check_list_str(name: str, value: object) -> List[str]: + def check_list_str(name: str, value: object) -> list[str]: if (not isinstance(value, list) or any(not isinstance(elt, str) for elt in value)): raise QAPISemError(
@@ -354,8 +351,8 @@ def accept(self, skip_comment: bool = True) -> None: self.src[self.cursor-1:]) raise QAPIParseError(self, "stray '%s'" % match.group(0)) - def get_members(self) -> Dict[str, object]: - expr: Dict[str, object] = {} + def get_members(self) -> dict[str, object]: + expr: dict[str, object] = {} if self.tok == '}': self.accept() return expr
@@ -381,8 +378,8 @@ def get_members(self) -> Dict[str, object]: if self.tok != "'": raise QAPIParseError(self, "expected string") - def get_values(self) -> List[object]: - expr: List[object] = [] + def get_values(self) -> list[object]: + expr: list[object] = [] if self.tok == ']': self.accept() return expr
@@ -694,21 +691,21 @@ def __init__(self, info: QAPISourceInfo, symbol: Optional[str] = None): # definition doc's symbol, None for free-form doc self.symbol: Optional[str] = symbol # the sections in textual order - self.all_sections: List[QAPIDoc.Section] = [ + self.all_sections: list[QAPIDoc.Section] = [ QAPIDoc.Section(info, QAPIDoc.Kind.PLAIN) ] # the body section self.body: Optional[QAPIDoc.Section] = self.all_sections[0] # dicts mapping parameter/feature names to their description - self.args: Dict[str, QAPIDoc.ArgSection] = {} - self.features: Dict[str, QAPIDoc.ArgSection] = {} + self.args: dict[str, QAPIDoc.ArgSection] = {} + self.features: dict[str, QAPIDoc.ArgSection] = {} # a command's "Returns" and "Errors" section self.returns: Optional[QAPIDoc.Section] = None self.errors: Optional[QAPIDoc.Section] = None # "Since" section self.since: Optional[QAPIDoc.Section] = None # sections other than .body, .args, .features - self.sections: List[QAPIDoc.Section] = [] + self.sections: list[QAPIDoc.Section] = [] def end(self) -> None: for section in self.all_sections:
@@ -763,7 +760,7 @@ def _new_description( info: QAPISourceInfo, name: str, kind: 'QAPIDoc.Kind', - desc: Dict[str, ArgSection] + desc: dict[str, ArgSection] ) -> None: if not name: raise QAPISemError(info, "invalid parameter name")
@@ -834,7 +831,7 @@ def check_expr(self, expr: QAPIExpression) -> None: def check(self) -> None: def check_args_section( - args: Dict[str, QAPIDoc.ArgSection], what: str + args: dict[str, QAPIDoc.ArgSection], what: str ) -> None: bogus = [name for name, section in args.items() if not section.member]
scripts/qapi/schema.py +56 -58
372 lines not on this page · raw message
scripts/qapi/source.py +5 -5
@@ -10,7 +10,7 @@ # See the COPYING file in the top-level directory. import copy -from typing import List, Optional, TypeVar +from typing import Optional, TypeVar class QAPISchemaPragma:
@@ -21,13 +21,13 @@ def __init__(self) -> None: # Are documentation comments required? self.doc_required = False # Commands whose names may use '_' - self.command_name_exceptions: List[str] = [] + self.command_name_exceptions: list[str] = [] # Commands allowed to return a non-dictionary - self.command_returns_exceptions: List[str] = [] + self.command_returns_exceptions: list[str] = [] # Types, commands, and events with undocumented members - self.documentation_exceptions: List[str] = [] + self.documentation_exceptions: list[str] = [] # Types whose member names may violate case conventions - self.member_name_exceptions: List[str] = [] + self.member_name_exceptions: list[str] = [] class QAPISourceInfo:
scripts/qapi/types.py +10 -10
@@ -13,7 +13,7 @@ # See the COPYING file in the top-level directory. """ -from typing import List, Optional +from typing import Optional from .common import c_enum_const, c_name, mcgen from .gen import QAPISchemaModularCVisitor, gen_features, ifcontext
@@ -38,7 +38,7 @@ def gen_enum_lookup(name: str, - members: List[QAPISchemaEnumMember], + members: list[QAPISchemaEnumMember], prefix: Optional[str] = None) -> str: max_index = c_enum_const(name, '_MAX', prefix) feats = ''
@@ -82,7 +82,7 @@ def gen_enum_lookup(name: str, def gen_enum(name: str, - members: List[QAPISchemaEnumMember], + members: list[QAPISchemaEnumMember], prefix: Optional[str] = None) -> str: # append automatically generated _MAX value enum_members = members + [QAPISchemaEnumMember('_MAX', None)]
@@ -136,7 +136,7 @@ def gen_array(name: str, element_type: QAPISchemaType) -> str: c_name=c_name(name), c_type=element_type.c_type()) -def gen_struct_members(members: List[QAPISchemaObjectTypeMember]) -> str: +def gen_struct_members(members: list[QAPISchemaObjectTypeMember]) -> str: ret = '' for memb in members: ret += memb.ifcond.gen_if()
@@ -155,7 +155,7 @@ def gen_struct_members(members: List[QAPISchemaObjectTypeMember]) -> str: def gen_object(name: str, ifcond: QAPISchemaIfCond, base: Optional[QAPISchemaObjectType], - members: List[QAPISchemaObjectTypeMember], + members: list[QAPISchemaObjectTypeMember], variants: Optional[QAPISchemaVariants]) -> str: if name in objects_seen: return ''
@@ -325,8 +325,8 @@ def visit_enum_type(self, name: str, info: Optional[QAPISourceInfo], ifcond: QAPISchemaIfCond, - features: List[QAPISchemaFeature], - members: List[QAPISchemaEnumMember], + features: list[QAPISchemaFeature], + members: list[QAPISchemaEnumMember], prefix: Optional[str]) -> None: with ifcontext(ifcond, self._genh, self._genc): self._genh.preamble_add(gen_enum(name, members, prefix))
@@ -346,9 +346,9 @@ def visit_object_type(self, name: str, info: Optional[QAPISourceInfo], ifcond: QAPISchemaIfCond, - features: List[QAPISchemaFeature], + features: list[QAPISchemaFeature], base: Optional[QAPISchemaObjectType], - members: List[QAPISchemaObjectTypeMember], + members: list[QAPISchemaObjectTypeMember], branches: Optional[QAPISchemaBranches]) -> None: # Nothing to do for the special empty builtin if name == 'q_empty':
@@ -369,7 +369,7 @@ def visit_alternate_type(self, name: str, info: Optional[QAPISourceInfo], ifcond: QAPISchemaIfCond, - features: List[QAPISchemaFeature], + features: list[QAPISchemaFeature], alternatives: QAPISchemaAlternatives) -> None: with ifcontext(ifcond, self._genh): self._genh.preamble_add(gen_fwd_object_or_array(name))
scripts/qapi/visit.py +7 -7
@@ -13,7 +13,7 @@ See the COPYING file in the top-level directory. """ -from typing import List, Optional +from typing import Optional from .common import ( c_enum_const,
@@ -59,7 +59,7 @@ def gen_visit_members_decl(name: str) -> str: def gen_visit_object_members(name: str, base: Optional[QAPISchemaObjectType], - members: List[QAPISchemaObjectTypeMember], + members: list[QAPISchemaObjectTypeMember], branches: Optional[QAPISchemaBranches]) -> str: ret = mcgen('''
@@ -370,8 +370,8 @@ def visit_enum_type(self, name: str, info: Optional[QAPISourceInfo], ifcond: QAPISchemaIfCond, - features: List[QAPISchemaFeature], - members: List[QAPISchemaEnumMember], + features: list[QAPISchemaFeature], + members: list[QAPISchemaEnumMember], prefix: Optional[str]) -> None: with ifcontext(ifcond, self._genh, self._genc): self._genh.add(gen_visit_decl(name, scalar=True))
@@ -390,9 +390,9 @@ def visit_object_type(self, name: str, info: Optional[QAPISourceInfo], ifcond: QAPISchemaIfCond, - features: List[QAPISchemaFeature], + features: list[QAPISchemaFeature], base: Optional[QAPISchemaObjectType], - members: List[QAPISchemaObjectTypeMember], + members: list[QAPISchemaObjectTypeMember], branches: Optional[QAPISchemaBranches]) -> None: # Nothing to do for the special empty builtin if name == 'q_empty':
@@ -412,7 +412,7 @@ def visit_alternate_type(self, name: str, info: Optional[QAPISourceInfo], ifcond: QAPISchemaIfCond, - features: List[QAPISchemaFeature], + features: list[QAPISchemaFeature], alternatives: QAPISchemaAlternatives) -> None: with ifcontext(ifcond, self._genh, self._genc): self._genh.add(gen_visit_decl(name))
scripts/rust/rustc_args.py +3 -3
@@ -28,7 +28,7 @@ from dataclasses import dataclass import logging from pathlib import Path -from typing import Any, Iterable, List, Mapping, Optional, Set +from typing import Any, Iterable, Mapping, Optional try: import tomllib
@@ -41,7 +41,7 @@ class CargoTOML: tomldata: Mapping[Any, Any] workspace_data: Mapping[Any, Any] - check_cfg: Set[str] + check_cfg: set[str] def __init__(self, path: Optional[str], workspace: Optional[str]): if path is not None:
@@ -78,7 +78,7 @@ def get_table(self, key: str, can_be_workspace: bool = False) -> Mapping[Any, An @dataclass class LintFlag: - flags: List[str] + flags: list[str] priority: int
scripts/u2f-setup-gen.py +1 -2
@@ -12,7 +12,6 @@ import sys import os from random import randint -from typing import Tuple from cryptography.hazmat.backends import default_backend from cryptography.hazmat.primitives.asymmetric import ec
@@ -53,7 +52,7 @@ def write_setup_dir(dirpath: str, privkey_pem: bytes, cert_pem: bytes, f.write(f'{str(counter)}\n') -def generate_ec_key_pair() -> Tuple[str, str]: +def generate_ec_key_pair() -> tuple[str, str]: """ Generate an ec key pair.
tests/functional/qemu_test/ports.py +1 -2
@@ -12,7 +12,6 @@ import socket from .config import BUILD_DIR -from typing import List class Ports():
@@ -41,7 +40,7 @@ def check_bind(self, port: int) -> bool: return True - def find_free_ports(self, count: int) -> List[int]: + def find_free_ports(self, count: int) -> list[int]: result = [] for port in range(self.PORTS_START, self.PORTS_END): if self.check_bind(port):
tests/functional/test_acpi_bits.py +1 -2
@@ -37,7 +37,6 @@ import subprocess from typing import ( - List, Optional, Sequence, )
@@ -76,7 +75,7 @@ def __init__(self, self.base_temp_dir = base_temp_dir @property - def _base_args(self) -> List[str]: + def _base_args(self) -> list[str]: args = super()._base_args args.extend([ '-chardev',
tests/qemu-iotests/151 +2 -2
@@ -23,7 +23,7 @@ import math import os import subprocess import time -from typing import List, Optional +from typing import Optional import iotests from iotests import qemu_img
@@ -195,7 +195,7 @@ class TestActiveMirror(iotests.QMPTestCase): class TestThrottledWithNbdExportBase(iotests.QMPTestCase): image_len = 128 * 1024 * 1024 # MB iops: Optional[int] = None - background_processes: List['subprocess.Popen[str]'] = [] + background_processes: list['subprocess.Popen[str]'] = [] def setUp(self): # Must be set by subclasses
tests/qemu-iotests/297 +2 -3
@@ -19,7 +19,6 @@ import os import subprocess import sys -from typing import List import iotests import linters
@@ -41,7 +40,7 @@ def check_linter(linter: str) -> bool: return True -def test_pylint(files: List[str]) -> None: +def test_pylint(files: list[str]) -> None: print('=== pylint ===') sys.stdout.flush()
@@ -51,7 +50,7 @@ def test_pylint(files: List[str]) -> None: linters.run_linter('pylint', files) -def test_mypy(files: List[str]) -> None: +def test_mypy(files: list[str]) -> None: print('=== mypy ===') sys.stdout.flush()
tests/qemu-iotests/300 +2 -2
@@ -22,12 +22,12 @@ import os import random import re -from typing import Dict, List, Optional +from typing import Optional import iotests -BlockBitmapMapping = List[Dict[str, object]] +BlockBitmapMapping = list[dict[str, object]] mig_sock = os.path.join(iotests.sock_dir, 'mig_sock')
tests/qemu-iotests/fat16.py +5 -5
@@ -15,7 +15,7 @@ # You should have received a copy of the GNU General Public License # along with this program. If not, see <http://www.gnu.org/licenses/>. -from typing import Callable, List, Optional, Protocol, Set +from typing import Callable, Optional, Protocol import string SECTOR_SIZE = 512
@@ -227,7 +227,7 @@ def __init__( self.fats = self.read_sectors( self.boot_sector.reserved_sectors, fat_size_in_sectors ) - self.fats_dirty_sectors: Set[int] = set() + self.fats_dirty_sectors: set[int] = set() def read_sectors(self, start_sector: int, num_sectors: int) -> bytes: return self.sector_reader(start_sector + self.start_sector,
@@ -238,7 +238,7 @@ def write_sectors(self, start_sector: int, data: bytes) -> None: def directory_from_bytes( self, data: bytes, start_sector: int - ) -> List[FatDirectoryEntry]: + ) -> list[FatDirectoryEntry]: """ Convert `bytes` into a list of `FatDirectoryEntry` objects. Will ignore long file names.
@@ -268,7 +268,7 @@ def directory_from_bytes( ) return entries - def read_root_directory(self) -> List[FatDirectoryEntry]: + def read_root_directory(self) -> list[FatDirectoryEntry]: root_dir = self.read_sectors( self.boot_sector.root_dir_start(), self.boot_sector.root_dir_size() )
@@ -373,7 +373,7 @@ def write_cluster(self, cluster: int, data: bytes) -> None: def read_directory( self, cluster: Optional[int] - ) -> List[FatDirectoryEntry]: + ) -> list[FatDirectoryEntry]: """ Read the directory at the given cluster. """
tests/qemu-iotests/findtests.py +6 -6
@@ -21,7 +21,7 @@ import re from collections import defaultdict from contextlib import contextmanager -from typing import Optional, List, Iterator, Set +from typing import Optional, Iterator @contextmanager
@@ -89,10 +89,10 @@ def parse_test_name(self, name: str) -> str: return name - def find_tests(self, groups: Optional[List[str]] = None, - exclude_groups: Optional[List[str]] = None, - tests: Optional[List[str]] = None, - start_from: Optional[str] = None) -> List[str]: + def find_tests(self, groups: Optional[list[str]] = None, + exclude_groups: Optional[list[str]] = None, + tests: Optional[list[str]] = None, + start_from: Optional[str] = None) -> list[str]: """Find tests Algorithm:
@@ -123,7 +123,7 @@ def find_tests(self, groups: Optional[List[str]] = None, if tests is None: tests = [] - res: Set[str] = set() + res: set[str] = set() if groups: # Some groups specified. exclude_groups supported, additionally # selecting some individual tests supported as well.
tests/qemu-iotests/iotests.py +19 -19
@@ -31,8 +31,8 @@ import subprocess import sys import time -from typing import (Any, Callable, Dict, Iterable, Iterator, - List, Optional, Sequence, TextIO, Tuple, Type, TypeVar) +from typing import (Any, Callable, Iterable, Iterator, + Optional, Sequence, TextIO, TypeVar) import unittest from contextlib import contextmanager
@@ -154,7 +154,7 @@ def qemu_tool_popen(args: Sequence[str], def qemu_tool_pipe_and_status(tool: str, args: Sequence[str], connect_stderr: bool = True, drop_successful_output: bool = False) \ - -> Tuple[str, int]: + -> tuple[str, int]: """ Run a tool and return both its output and its exit code """
@@ -168,7 +168,7 @@ def qemu_tool_pipe_and_status(tool: str, args: Sequence[str], output = '' return (output, subp.returncode) -def qemu_img_create_prepare_args(args: List[str]) -> List[str]: +def qemu_img_create_prepare_args(args: list[str]) -> list[str]: if not args or args[0] != 'create': return list(args) args = args[1:]
@@ -344,7 +344,7 @@ def img_info_log(filename: str, filter_path: Optional[str] = None, filter_path = filename log(filter_img_info(output, filter_path, drop_child_info)) -def qemu_io_wrap_args(args: Sequence[str]) -> List[str]: +def qemu_io_wrap_args(args: Sequence[str]) -> list[str]: if '-f' in args or '--image-opts' in args: return qemu_io_args_no_fmt + list(args) else:
@@ -457,7 +457,7 @@ def __init__(self, *args: str, instance_id: str = 'a', qmp: bool = False): assert self._pid == self._p.pid - def qmp(self, cmd: str, args: Optional[Dict[str, object]] = None) \ + def qmp(self, cmd: str, args: Optional[dict[str, object]] = None) \ -> QMPMessage: assert self._qmp is not None return self._qmp.cmd_raw(cmd, args)
@@ -466,7 +466,7 @@ def get_qmp(self) -> QEMUMonitorProtocol: assert self._qmp is not None return self._qmp - def cmd(self, cmd: str, args: Optional[Dict[str, object]] = None) \ + def cmd(self, cmd: str, args: Optional[dict[str, object]] = None) \ -> QMPReturnValue: assert self._qmp is not None return self._qmp.cmd(cmd, **(args or {}))
@@ -498,7 +498,7 @@ def qemu_nbd(*args): '''Run qemu-nbd in daemon mode and return the parent's exit code''' return subprocess.call(qemu_nbd_args + ['--fork'] + list(args)) -def qemu_nbd_early_pipe(*args: str) -> Tuple[int, str]: +def qemu_nbd_early_pipe(*args: str) -> tuple[int, str]: '''Run qemu-nbd in daemon mode and return both the parent's exit code and its output in case of an error''' full_args = qemu_nbd_args + ['--fork'] + list(args)
@@ -706,7 +706,7 @@ def filter_qtest(output: str) -> str: output = re.sub(r'\n?\[I \+\d+\.\d+\] CLOSED\n?$', '', output) return output -Msg = TypeVar('Msg', Dict[str, Any], List[Any], str) +Msg = TypeVar('Msg', dict[str, Any], list[Any], str) def log(msg: Msg, filters: Iterable[Callable[[Msg], Msg]] = (),
@@ -919,7 +919,7 @@ def add_paused(self): def hmp(self, command_line: str, use_log: bool = False) -> QMPMessage: cmd = 'human-monitor-command' - kwargs: Dict[str, Any] = {'command-line': command_line} + kwargs: dict[str, Any] = {'command-line': command_line} if use_log: return self.qmp_log(cmd, **kwargs) else:
@@ -1462,7 +1462,7 @@ def verify_quorum(): if not supports_quorum(): notrun('quorum support missing') -def has_working_luks() -> Tuple[bool, str]: +def has_working_luks() -> tuple[bool, str]: """ Check whether our LUKS driver can actually create images (this extends to LUKS encryption for qcow2).
@@ -1550,8 +1550,8 @@ def skip_if_unsupported(required_formats=(), read_only=False): '''Skip Test Decorator Runs the test if all the required formats are whitelisted''' def skip_test_decorator(func): - def func_wrapper(test_case: QMPTestCase, *args: List[Any], - **kwargs: Dict[str, Any]) -> None: + def func_wrapper(test_case: QMPTestCase, *args: list[Any], + **kwargs: dict[str, Any]) -> None: if callable(required_formats): fmts = required_formats(test_case) else:
@@ -1567,13 +1567,13 @@ def func_wrapper(test_case: QMPTestCase, *args: List[Any], return skip_test_decorator def skip_for_formats(formats: Sequence[str] = ()) \ - -> Callable[[Callable[[QMPTestCase, List[Any], Dict[str, Any]], None]], - Callable[[QMPTestCase, List[Any], Dict[str, Any]], None]]: + -> Callable[[Callable[[QMPTestCase, list[Any], dict[str, Any]], None]], + Callable[[QMPTestCase, list[Any], dict[str, Any]], None]]: '''Skip Test Decorator Skips the test for the given formats''' def skip_test_decorator(func): - def func_wrapper(test_case: QMPTestCase, *args: List[Any], - **kwargs: Dict[str, Any]) -> None: + def func_wrapper(test_case: QMPTestCase, *args: list[Any], + **kwargs: dict[str, Any]) -> None: if imgfmt in formats: msg = f'{test_case}: Skipped for format {imgfmt}' test_case.case_skip(msg)
@@ -1625,7 +1625,7 @@ class ReproducibleTestRunner(unittest.TextTestRunner): def __init__( self, stream: Optional[TextIO] = None, - resultclass: Type[unittest.TextTestResult] = + resultclass: type[unittest.TextTestResult] = ReproducibleTestResult, **kwargs: Any ) -> None:
@@ -1635,7 +1635,7 @@ def __init__( resultclass=resultclass, **kwargs) -def execute_unittest(argv: List[str], debug: bool = False) -> None: +def execute_unittest(argv: list[str], debug: bool = False) -> None: """Executes unittests within the calling module.""" # Some tests have warnings, especially ResourceWarnings for unclosed
tests/qemu-iotests/linters.py +3 -3
@@ -17,7 +17,7 @@ import re import subprocess import sys -from typing import List, Mapping, Optional +from typing import Mapping, Optional # TODO: Empty this list!
@@ -49,7 +49,7 @@ def is_python_file(filename): return False -def get_test_files() -> List[str]: +def get_test_files() -> list[str]: named_tests = [f'tests/{entry}' for entry in os.listdir('tests')] check_tests = set(os.listdir('.') + named_tests) - set(SKIP_FILES) return list(filter(is_python_file, check_tests))
@@ -57,7 +57,7 @@ def get_test_files() -> List[str]: def run_linter( tool: str, - args: List[str], + args: list[str], env: Optional[Mapping[str, str]] = None, suppress_output: bool = False, ) -> None:
tests/qemu-iotests/testenv.py +3 -3
@@ -25,7 +25,7 @@ import random import subprocess import glob -from typing import List, Dict, Any, Optional +from typing import Any, Optional if sys.version_info >= (3, 9): from contextlib import AbstractContextManager as ContextManager
@@ -81,7 +81,7 @@ class TestEnv(ContextManager['TestEnv']): 'IMGKEYSECRET', 'QEMU_DEFAULT_MACHINE', 'MALLOC_PERTURB_', 'GDB_OPTIONS', 'PRINT_QEMU'] - def prepare_subprocess(self, args: List[str]) -> Dict[str, str]: + def prepare_subprocess(self, args: list[str]) -> dict[str, str]: if self.debug: args.append('-d')
@@ -96,7 +96,7 @@ def prepare_subprocess(self, args: List[str]) -> Dict[str, str]: os_env.update(self.get_env()) return os_env - def get_env(self) -> Dict[str, str]: + def get_env(self) -> dict[str, str]: env = {} for v in self.env_variables: val = getattr(self, v.lower(), None)
tests/qemu-iotests/testrunner.py +6 -6
@@ -27,7 +27,7 @@ import shutil import sys from multiprocessing import Pool -from typing import List, Optional, Any, Sequence, Dict +from typing import Optional, Any, Sequence from testenv import TestEnv if sys.version_info >= (3, 9):
@@ -43,7 +43,7 @@ def silent_unlink(path: Path) -> None: pass -def file_diff(file1: str, file2: str) -> List[str]: +def file_diff(file1: str, file2: str) -> list[str]: with open(file1, encoding="utf-8") as f1, \ open(file2, encoding="utf-8") as f2: # We want to ignore spaces at line ends. There are a lot of mess about
@@ -66,7 +66,7 @@ class LastElapsedTime(ContextManager['LastElapsedTime']): def __init__(self, cache_file: str, env: TestEnv) -> None: self.env = env self.cache_file = cache_file - self.cache: Dict[str, Dict[str, Dict[str, float]]] + self.cache: dict[str, dict[str, dict[str, float]]] try: with open(cache_file, encoding="utf-8") as f:
@@ -122,8 +122,8 @@ def proc_run_test(test: str, test_field_width: int) -> TestResult: assert runner is not None return runner.run_test(test, test_field_width, mp=True) - def run_tests_pool(self, tests: List[str], - test_field_width: int, jobs: int) -> List[TestResult]: + def run_tests_pool(self, tests: list[str], + test_field_width: int, jobs: int) -> list[TestResult]: # passing self directly to Pool.starmap() just doesn't work, because # it's a context manager.
@@ -369,7 +369,7 @@ def run_test(self, test: str, sys.stdout.flush() return res - def run_tests(self, tests: List[str], jobs: int = 1) -> bool: + def run_tests(self, tests: list[str], jobs: int = 1) -> bool: n_run = 0 failed = [] notrun = []
tests/qemu-iotests/tests/stream-under-throttle +1 -2
@@ -21,7 +21,6 @@ import asyncio import os -from typing import List import iotests from iotests import qemu_img_create, qemu_io
@@ -39,7 +38,7 @@ class TcgVM(iotests.VM): the order they appear. ''' @property - def _base_args(self) -> List[str]: + def _base_args(self) -> list[str]: # Put -accel tcg first so it takes precedence return ['-accel', 'tcg'] + super()._base_args
Signature · 1 line
2.48.1
Re: [PATCH 4/8] python: use 3.9+ builtin type hints
Quoted 24 lines · John Snow
This patch changes type hints that have built-in equivalents as of Python 3.9. The versions we currently use, imported from the typing module, have been deprecated since 3.9 and may be removed at any time - though to my knowledge, they are not scheduled for removal in 3.14.
The type hints to update are:
typing.Dict => dict
typing.List => list
typing.Tuple => tuple
typing.Set => set
typing.Type => typeI performed this change the dumb way (I couldn't work out for the life of me how to beg ls-files to exclude subprojects and empty directories, desipte it having an option which claims to do precisely that. Suggest a nicer way to do this if you'd like.)
git ls-files | grep -v 'roms/' | grep -v 'subprojects/' | grep -v 'tests/lcitool/libvirt-ci' | xargs sed -i -e 's/Dict\[/dict\[/g' git ls-files | grep -v 'roms/' | grep -v 'subprojects/' | grep -v 'tests/lcitool/libvirt-ci' | xargs sed -i -e 's/List\[/list\[/g' git ls-files | grep -v 'roms/' | grep -v 'subprojects/' | grep -v 'tests/lcitool/libvirt-ci' | xargs sed -i -e 's/Tuple\[/tuple\[/g' git ls-files | grep -v 'roms/' | grep -v 'subprojects/' | grep -v 'tests/lcitool/libvirt-ci' | xargs sed -i -e 's/Set\[/set\[/g' git ls-files | grep -v 'roms/' | grep -v 'subprojects/' | grep -v 'tests/lcitool/libvirt-ci' | xargs sed -i -e 's/Type\[/type\[/g'
Quoted 5 lines
I then used "git add -p" and excluded changes by hand that weren't appropriate. I then reviewed all of the changed files by hand to review the imports and update them accordingly.
Signed-off-by John Snow <[email protected]>
I'd expect to catch mistakes in the manual parts. I gave scripts/qapi/ an eye-over anyway, and it looks good to me.
Re: [PATCH 4/8] python: use 3.9+ builtin type hints
Quoted 31 lines · Markus Armbruster
Quoted 13 lines · John Snow
This patch changes type hints that have built-in equivalents as of Python 3.9. The versions we currently use, imported from the typing module, have been deprecated since 3.9 and may be removed at any time - though to my knowledge, they are not scheduled for removal in 3.14.
The type hints to update are:
typing.Dict => dict
typing.List => list
typing.Tuple => tuple
typing.Set => set
typing.Type => typeQuoted 6 lines
I performed this change the dumb way (I couldn't work out for the life of me how to beg ls-files to exclude subprojects and empty directories, desipte it having an option which claims to do precisely that. Suggest a nicer way to do this if you'd like.)
git ls-files | grep -v 'roms/' | grep -v 'subprojects/' | grep -v
'tests/lcitool/libvirt-ci' | xargs sed -i -e 's/Dict\[/dict\[/g'
Quoted 1 line
git ls-files | grep -v 'roms/' | grep -v 'subprojects/' | grep -v
'tests/lcitool/libvirt-ci' | xargs sed -i -e 's/List\[/list\[/g'
Quoted 1 line
git ls-files | grep -v 'roms/' | grep -v 'subprojects/' | grep -v
'tests/lcitool/libvirt-ci' | xargs sed -i -e 's/Tuple\[/tuple\[/g'
Quoted 1 line
git ls-files | grep -v 'roms/' | grep -v 'subprojects/' | grep -v
'tests/lcitool/libvirt-ci' | xargs sed -i -e 's/Set\[/set\[/g'
Quoted 1 line
git ls-files | grep -v 'roms/' | grep -v 'subprojects/' | grep -v
'tests/lcitool/libvirt-ci' | xargs sed -i -e 's/Type\[/type\[/g'
Quoted 1 line
Quoted 9 lines
Quoted 5 lines
I then used "git add -p" and excluded changes by hand that weren't appropriate. I then reviewed all of the changed files by hand to review the imports and update them accordingly.
Signed-off-by John Snow <[email protected]>I'd expect to catch mistakes in the manual parts. I gave
scripts/qapi/an eye-over anyway, and it looks good to me.
I've prepared an alternate series that's vastly more automated, using "pyupgrade", "autoflake", and isort. As a downside it touches a lot more than just the type hints, removing other outdated pythonisms. As an upside it's way, way less manual.
Quoted 4 lines
PATCH 5/8 python: convert remaining deprecated type hints for 3.9+
In reply to John SnowIn this patch, we don't have a builtin type to use for a type hint, but we do have collections.abc and other standard library types. Replace deprecated type hints with their standard library equivalents:
typing.Awaitable => collections.abc.Awaitable typing.AbstractSet => collections.abc.AbstractSet typing.AsyncIterator => collections.abc.AsyncIterator typing.Callable => collections.abc.Callable typing.Coroutine => collections.abc.Coroutine typing.Deque => collections.deque typing.Generator => collections.abc.Generator typing.Iterable => collections.abc.Iterable typing.Iterator => collections.abc.Iterator typing.Mapping => collections.abc.Mapping typing.Match => re.Match
typing.MutableMapping => collections.abc.MutableMapping
typing.Sequence => collections.abc.Sequence
typing.ValuesView => collections.abc.ValuesView
The primary benefit of this is, of course, that the standard type can now be used directly as the type hint. In general, this means far fewer imports from typing. The reason we *have* to do this is because the old type hints have been deprecated and will be dropped from a Python release in the future, so I am just getting ahead of it before it causes a problem.
(Granted, yes, in practice this just means we're usually importing from collections.abc instead of typing, but... ah well. What are you gonna do.)
--- docs/sphinx/compat.py | 8 ++------ docs/sphinx/dbusdoc.py | 4 +--- docs/sphinx/dbusdomain.py | 3 +-- docs/sphinx/qapi_domain.py | 9 ++------- docs/sphinx/qapidoc.py | 9 ++------- python/qemu/machine/console_socket.py | 4 ++-- python/qemu/machine/machine.py | 2 +- python/qemu/machine/qtest.py | 3 ++- python/qemu/qmp/events.py | 9 ++++----- python/qemu/qmp/legacy.py | 2 +- python/qemu/qmp/message.py | 9 ++------- python/qemu/qmp/models.py | 17 ++++++----------- python/qemu/qmp/protocol.py | 4 +--- python/qemu/qmp/qmp_client.py | 8 ++------ python/qemu/qmp/qmp_shell.py | 3 +-- python/qemu/utils/qemu_ga_client.py | 8 ++------ python/qemu/utils/qom_fuse.py | 9 ++------- python/scripts/mkvenv.py | 9 ++------- scripts/block-coroutine-wrapper.py | 2 +- scripts/codeconverter/codeconverter/patching.py | 8 +++++--- scripts/compare-machine-types.py | 3 ++- scripts/qapi/common.py | 11 +++-------- scripts/qapi/expr.py | 8 ++------ scripts/qapi/features.py | 2 +- scripts/qapi/gen.py | 3 ++- scripts/qapi/introspect.py | 2 +- scripts/qapi/parser.py | 5 ++--- scripts/qapi/schema.py | 3 +-- scripts/rust/rustc_args.py | 3 ++- tests/functional/test_acpi_bits.py | 6 ++---- tests/qemu-iotests/fat16.py | 3 ++- tests/qemu-iotests/findtests.py | 3 ++- tests/qemu-iotests/iotests.py | 4 ++-- tests/qemu-iotests/linters.py | 3 ++- tests/qemu-iotests/testenv.py | 5 +---- tests/qemu-iotests/testrunner.py | 9 +++------ 36 files changed, 72 insertions(+), 131 deletions(-)
docs/sphinx/compat.py +2 -6
@@ -2,13 +2,9 @@ Sphinx cross-version compatibility goop """ +from collections.abc import Callable import re -from typing import ( - TYPE_CHECKING, - Any, - Callable, - Optional, -) +from typing import TYPE_CHECKING, Any, Optional from docutils import nodes from docutils.nodes import Element, Node, Text
docs/sphinx/dbusdoc.py +1 -3
@@ -7,15 +7,13 @@ # Author: Marc-André Lureau <[email protected]> """dbus-doc is a Sphinx extension that provides documentation from D-Bus XML.""" +from collections.abc import Callable, Iterator, Sequence import os import re from typing import ( TYPE_CHECKING, Any, - Callable, - Iterator, Optional, - Sequence, TypeVar, Union, )
docs/sphinx/dbusdomain.py +1 -2
@@ -6,10 +6,9 @@ # # Author: Marc-André Lureau <[email protected]> +from collections.abc import Iterable, Iterator from typing import ( Any, - Iterable, - Iterator, NamedTuple, Optional, cast,
docs/sphinx/qapi_domain.py +2 -7
@@ -39,13 +39,8 @@ if TYPE_CHECKING: - from typing import ( - AbstractSet, - Any, - Iterable, - Optional, - Union, - ) + from collections.abc import AbstractSet, Iterable + from typing import Any, Optional, Union from docutils.nodes import Element, Node from sphinx.addnodes import desc_signature, pending_xref
docs/sphinx/qapidoc.py +2 -7
@@ -68,13 +68,8 @@ if TYPE_CHECKING: - from typing import ( - Any, - Generator, - Optional, - Sequence, - Union, - ) + from collections.abc import Generator, Sequence + from typing import Any, Optional, Union from sphinx.application import Sphinx from sphinx.util.typing import ExtensionMetadata
python/qemu/machine/console_socket.py +2 -2
@@ -17,7 +17,7 @@ import socket import threading import time -from typing import Deque, Optional +from typing import Optional class ConsoleSocket(socket.socket):
@@ -43,7 +43,7 @@ def __init__(self, self._recv_timeout_sec = 300.0 self._sleep_time = 0.5 - self._buffer: Deque[int] = deque() + self._buffer: deque[int] = deque() if address is not None: socket.socket.__init__(self, socket.AF_UNIX, socket.SOCK_STREAM) self.connect(address)
python/qemu/machine/machine.py +1 -1
@@ -17,6 +17,7 @@ # Based on qmp.py. # +from collections.abc import Sequence import errno from itertools import chain import locale
@@ -32,7 +33,6 @@ Any, BinaryIO, Optional, - Sequence, TypeVar, )
python/qemu/machine/qtest.py +2 -1
@@ -17,9 +17,10 @@ # Based on qmp.py. # +from collections.abc import Sequence import os import socket -from typing import Optional, Sequence, TextIO +from typing import Optional, TextIO from qemu.qmp import SocketAddrT
python/qemu/qmp/events.py +4 -5
@@ -448,16 +448,15 @@ def accept(self, event) -> bool: """ import asyncio -from contextlib import contextmanager -import logging -from typing import ( +from collections.abc import ( AsyncIterator, Callable, Iterable, Iterator, - Optional, - Union, ) +from contextlib import contextmanager +import logging +from typing import Optional, Union from .error import QMPError from .message import Message
python/qemu/qmp/legacy.py +1 -1
@@ -22,11 +22,11 @@ # import asyncio +from collections.abc import Awaitable import socket from types import TracebackType from typing import ( Any, - Awaitable, Optional, TypeVar, Union,
python/qemu/qmp/message.py +2 -7
@@ -5,15 +5,10 @@ message sent to or from the server. """ +from collections.abc import Iterator, Mapping, MutableMapping import json from json import JSONDecodeError -from typing import ( - Iterator, - Mapping, - MutableMapping, - Optional, - Union, -) +from typing import Optional, Union from .error import ProtocolError
python/qemu/qmp/models.py +6 -11
@@ -7,14 +7,9 @@ """ # pylint: disable=too-few-public-methods -from collections import abc +from collections.abc import Mapping, Sequence import copy -from typing import ( - Any, - Mapping, - Optional, - Sequence, -) +from typing import Any, Optional class Model:
@@ -68,7 +63,7 @@ def __init__(self, raw: Mapping[str, Any]): #: 'QMP' member self.QMP: QMPGreeting # pylint: disable=invalid-name - self._check_member('QMP', abc.Mapping, "JSON object") + self._check_member('QMP', Mapping, "JSON object") self.QMP = QMPGreeting(self._raw['QMP']) def _asdict(self) -> dict[str, object]:
@@ -98,10 +93,10 @@ def __init__(self, raw: Mapping[str, Any]): #: 'capabilities' member self.capabilities: Sequence[object] - self._check_member('version', abc.Mapping, "JSON object") + self._check_member('version', Mapping, "JSON object") self.version = self._raw['version'] - self._check_member('capabilities', abc.Sequence, "JSON array") + self._check_member('capabilities', Sequence, "JSON array") self.capabilities = self._raw['capabilities']
@@ -120,7 +115,7 @@ def __init__(self, raw: Mapping[str, Any]): #: 'id' member self.id: Optional[object] = None # pylint: disable=invalid-name - self._check_member('error', abc.Mapping, "JSON object") + self._check_member('error', Mapping, "JSON object") self.error = ErrorInfo(self._raw['error']) if 'id' in raw:
python/qemu/qmp/protocol.py +1 -3
@@ -15,6 +15,7 @@ import asyncio from asyncio import StreamReader, StreamWriter +from collections.abc import AsyncGenerator, Awaitable, Callable from contextlib import asynccontextmanager from enum import Enum from functools import wraps
@@ -24,9 +25,6 @@ from ssl import SSLContext from typing import ( Any, - AsyncGenerator, - Awaitable, - Callable, Generic, Optional, TypeVar,
python/qemu/qmp/qmp_client.py +2 -6
@@ -8,15 +8,11 @@ """ import asyncio +from collections.abc import Mapping import logging import socket import struct -from typing import ( - Mapping, - Optional, - Union, - cast, -) +from typing import Optional, Union, cast from .error import ProtocolError, QMPError from .events import Events
python/qemu/qmp/qmp_shell.py +1 -2
@@ -125,6 +125,7 @@ import argparse import ast +from collections.abc import Iterator, Sequence import json import logging import os
@@ -134,10 +135,8 @@ import sys from typing import ( IO, - Iterator, NoReturn, Optional, - Sequence, cast, )
python/qemu/utils/qemu_ga_client.py +2 -6
@@ -39,15 +39,11 @@ import argparse import asyncio import base64 +from collections.abc import Callable, Sequence import os import random import sys -from typing import ( - Any, - Callable, - Optional, - Sequence, -) +from typing import Any, Optional from qemu.qmp import ConnectError, SocketAddrT from qemu.qmp.legacy import QEMUMonitorProtocol
python/qemu/utils/qom_fuse.py +2 -7
@@ -33,16 +33,11 @@ ## import argparse +from collections.abc import Iterator, Mapping from errno import ENOENT, EPERM import stat import sys -from typing import ( - IO, - Iterator, - Mapping, - Optional, - Union, -) +from typing import IO, Optional, Union import fuse from fuse import FUSE, FuseOSError, Operations
python/scripts/mkvenv.py +2 -7
@@ -58,6 +58,7 @@ # later. See the COPYING file in the top-level directory. import argparse +from collections.abc import Iterator, Sequence from importlib.metadata import ( Distribution, EntryPoint,
@@ -76,13 +77,7 @@ import sys import sysconfig from types import SimpleNamespace -from typing import ( - Any, - Iterator, - Optional, - Sequence, - Union, -) +from typing import Any, Optional, Union import venv
scripts/block-coroutine-wrapper.py +1 -1
@@ -23,9 +23,9 @@ along with this program. If not, see <http://www.gnu.org/licenses/>. """ +from collections.abc import Iterator import sys import re -from typing import Iterator def gen_header():
scripts/codeconverter/codeconverter/patching.py +5 -3
@@ -5,7 +5,9 @@ # # This work is licensed under the terms of the GNU GPL, version 2. See # the COPYING file in the top-level directory. -from typing import IO, Match, NamedTuple, Optional, Literal, Iterable, Any, TypeVar, NewType, Union + +from collections.abc import Iterable +from typing import IO, NamedTuple, Optional, Literal, Any, TypeVar, NewType, Union from pathlib import Path from itertools import chain from tempfile import NamedTemporaryFile
@@ -45,9 +47,9 @@ class FileMatch: """ regexp: Optional[str] = None - def __init__(self, f: 'FileInfo', m: Match) -> None: + def __init__(self, f: 'FileInfo', m: re.Match) -> None: self.file: 'FileInfo' = f - self.match: Match[str] = m + self.match: re.Match[str] = m @property def name(self) -> str:
scripts/compare-machine-types.py +2 -1
@@ -26,12 +26,13 @@ # You should have received a copy of the GNU General Public License # along with this program; if not, see <http://www.gnu.org/licenses/>. +from collections.abc import Generator import sys from os import path from argparse import ArgumentParser, RawTextHelpFormatter, Namespace import pandas as pd from contextlib import ExitStack -from typing import Optional, Generator, Union, Any +from typing import Optional, Union, Any try: qemu_dir = path.abspath(path.dirname(path.dirname(__file__)))
scripts/qapi/common.py +3 -8
@@ -11,14 +11,9 @@ # This work is licensed under the terms of the GNU GPL, version 2. # See the COPYING file in the top-level directory. +from collections.abc import Sequence import re -from typing import ( - Any, - Match, - Optional, - Sequence, - Union, -) +from typing import Any, Optional, Union #: Magic string that gets removed along with all space to its right.
@@ -250,7 +245,7 @@ def gen_endif(cond: str) -> str: ''', cond=cond) -def must_match(pattern: str, string: str) -> Match[str]: +def must_match(pattern: str, string: str) -> re.Match[str]: match = re.match(pattern, string) assert match is not None return match
scripts/qapi/expr.py +2 -6
@@ -31,13 +31,9 @@ structures and contextual semantic validation. """ +from collections.abc import Iterable import re -from typing import ( - Iterable, - Optional, - Union, - cast, -) +from typing import Optional, Union, cast from .common import c_name from .error import QAPISemError
scripts/qapi/features.py +1 -1
@@ -7,7 +7,7 @@ # See the COPYING file in the top-level directory. """ -from typing import ValuesView +from collections.abc import ValuesView from .common import c_enum_const, c_name from .gen import QAPISchemaMonolithicCVisitor
scripts/qapi/gen.py +2 -1
@@ -11,11 +11,12 @@ # This work is licensed under the terms of the GNU GPL, version 2. # See the COPYING file in the top-level directory. +from collections.abc import Iterator, Sequence from contextlib import contextmanager import os import re import sys -from typing import Iterator, Optional, Sequence +from typing import Optional from .common import ( c_enum_const,
scripts/qapi/introspect.py +1 -1
@@ -11,12 +11,12 @@ See the COPYING file in the top-level directory. """ +from collections.abc import Sequence from dataclasses import dataclass from typing import ( Any, Generic, Optional, - Sequence, TypeVar, Union, )
scripts/qapi/parser.py +2 -3
@@ -14,14 +14,13 @@ # This work is licensed under the terms of the GNU GPL, version 2. # See the COPYING file in the top-level directory. +from collections.abc import Mapping import enum import os import re from typing import ( TYPE_CHECKING, Any, - Mapping, - Match, Optional, Union, )
@@ -430,7 +429,7 @@ def get_doc_line(self) -> Optional[str]: return self.val[2:].rstrip() @staticmethod - def _match_at_name_colon(string: str) -> Optional[Match[str]]: + def _match_at_name_colon(string: str) -> Optional[re.Match[str]]: return re.match(r'@([^:]*): *', string) def get_doc_indented(self, doc: 'QAPIDoc') -> Optional[str]:
scripts/qapi/schema.py +1 -2
@@ -19,14 +19,13 @@ from __future__ import annotations from abc import ABC, abstractmethod +from collections.abc import Callable, ValuesView import os import re from typing import ( Any, - Callable, Optional, Union, - ValuesView, cast, )
scripts/rust/rustc_args.py +2 -1
@@ -25,10 +25,11 @@ """ import argparse +from collections.abc import Iterable, Mapping from dataclasses import dataclass import logging from pathlib import Path -from typing import Any, Iterable, Mapping, Optional +from typing import Any, Optional try: import tomllib
tests/functional/test_acpi_bits.py +2 -4
@@ -31,15 +31,13 @@ https://gitlab.com/qemu-project/biosbits-bits . """ +from collections.abc import Sequence import os import re import shutil import subprocess +from typing import Optional -from typing import ( - Optional, - Sequence, -) from qemu.machine import QEMUMachine from qemu_test import (QemuSystemTest, Asset, skipIfMissingCommands, skipIfNotMachine)
tests/qemu-iotests/fat16.py +2 -1
@@ -15,7 +15,8 @@ # You should have received a copy of the GNU General Public License # along with this program. If not, see <http://www.gnu.org/licenses/>. -from typing import Callable, Optional, Protocol +from collections.abc import Callable +from typing import Optional, Protocol import string SECTOR_SIZE = 512
tests/qemu-iotests/findtests.py +2 -1
@@ -20,8 +20,9 @@ import glob import re from collections import defaultdict +from collections.abc import Iterator from contextlib import contextmanager -from typing import Optional, Iterator +from typing import Optional @contextmanager
tests/qemu-iotests/iotests.py +2 -2
@@ -20,6 +20,7 @@ import atexit import bz2 from collections import OrderedDict +from collections.abc import Callable, Iterable, Iterator, Sequence import faulthandler import json import logging
@@ -31,8 +32,7 @@ import subprocess import sys import time -from typing import (Any, Callable, Iterable, Iterator, - Optional, Sequence, TextIO, TypeVar) +from typing import Any, Optional, TextIO, TypeVar import unittest from contextlib import contextmanager
tests/qemu-iotests/linters.py +2 -1
@@ -13,11 +13,12 @@ # You should have received a copy of the GNU General Public License # along with this program. If not, see <http://www.gnu.org/licenses/>. +from collections.abc import Mapping import os import re import subprocess import sys -from typing import Mapping, Optional +from typing import Optional # TODO: Empty this list!
tests/qemu-iotests/testenv.py +1 -4
@@ -16,6 +16,7 @@ # along with this program. If not, see <http://www.gnu.org/licenses/>. # +from contextlib import AbstractContextManager as ContextManager import os import sys import tempfile
@@ -27,10 +28,6 @@ import glob from typing import Any, Optional -if sys.version_info >= (3, 9): - from contextlib import AbstractContextManager as ContextManager -else: - from typing import ContextManager DEF_GDB_OPTIONS = 'localhost:12345'
tests/qemu-iotests/testrunner.py +3 -6
@@ -16,6 +16,8 @@ # along with this program. If not, see <http://www.gnu.org/licenses/>. # +from collections.abc import Sequence +from contextlib import AbstractContextManager as ContextManager import os from pathlib import Path import datetime
@@ -27,14 +29,9 @@ import shutil import sys from multiprocessing import Pool -from typing import Optional, Any, Sequence +from typing import Optional, Any from testenv import TestEnv -if sys.version_info >= (3, 9): - from contextlib import AbstractContextManager as ContextManager -else: - from typing import ContextManager - def silent_unlink(path: Path) -> None: try:
Signature · 1 line
2.48.1
Re: [PATCH 5/8] python: convert remaining deprecated type hints for 3.9+
Quoted 26 lines · John Snow
In this patch, we don't have a builtin type to use for a type hint, but we do have collections.abc and other standard library types. Replace deprecated type hints with their standard library equivalents:
typing.Awaitable => collections.abc.Awaitable typing.AbstractSet => collections.abc.AbstractSet typing.AsyncIterator => collections.abc.AsyncIterator typing.Callable => collections.abc.Callable typing.Coroutine => collections.abc.Coroutine typing.Deque => collections.deque typing.Generator => collections.abc.Generator typing.Iterable => collections.abc.Iterable typing.Iterator => collections.abc.Iterator typing.Mapping => collections.abc.Mapping typing.Match => re.Matchtyping.MutableMapping => collections.abc.MutableMapping
typing.Sequence => collections.abc.Sequence
typing.ValuesView => collections.abc.ValuesViewThe primary benefit of this is, of course, that the standard type can now be used directly as the type hint. In general, this means far fewer imports from
typing. The reason we *have* to do this is because the old type hints have been deprecated and will be dropped from a Python release in the future, so I am just getting ahead of it before it causes a problem.
Quoted 5 lines
(Granted, yes, in practice this just means we're usually importing from collections.abc instead of typing, but... ah well. What are you gonna do.)
Signed-off-by John Snow <[email protected]>
I'd expect to catch mistakes in the manual parts. I gave scripts/qapi/ an eye-over anyway, and it looks good to me.
Typing imports left at the end of the series:
*
Any
BinaryIO
Generic
IO
Literal
NamedTuple
NewType
NoReturn
Optional
Protocol
TYPE_CHECKING
TextIO
TypeVar
Union
castThe * are in scripts/codeconverter/. You might want to double-check nothing deprecated hides behind the *.
The others are all still good, I presume.
Re: [PATCH 5/8] python: convert remaining deprecated type hints for 3.9+
Quoted 41 lines · Markus Armbruster
Quoted 26 lines · John Snow
In this patch, we don't have a builtin type to use for a type hint, but we do have collections.abc and other standard library types. Replace deprecated type hints with their standard library equivalents:
typing.Awaitable => collections.abc.Awaitable typing.AbstractSet => collections.abc.AbstractSet typing.AsyncIterator => collections.abc.AsyncIterator typing.Callable => collections.abc.Callable typing.Coroutine => collections.abc.Coroutine typing.Deque => collections.deque typing.Generator => collections.abc.Generator typing.Iterable => collections.abc.Iterable typing.Iterator => collections.abc.Iterator typing.Mapping => collections.abc.Mapping typing.Match => re.Matchtyping.MutableMapping => collections.abc.MutableMapping
typing.Sequence => collections.abc.Sequence
typing.ValuesView => collections.abc.ValuesViewThe primary benefit of this is, of course, that the standard type can now be used directly as the type hint. In general, this means far fewer imports from
typing. The reason we *have* to do this is because the old type hints have been deprecated and will be dropped from a Python release in the future, so I am just getting ahead of it before it causes a problem.Quoted 5 lines
(Granted, yes, in practice this just means we're usually importing from collections.abc instead of typing, but... ah well. What are you gonna do.)
Signed-off-by John Snow <[email protected]>I'd expect to catch mistakes in the manual parts. I gave
scripts/qapi/an eye-over anyway, and it looks good to me.Typing imports left at the end of the series:
Quoted 17 lines
*
Any
BinaryIO
Generic
IO
Literal
NamedTuple
NewType
NoReturn
Optional
Protocol
TYPE_CHECKING
TextIO
TypeVar
Union
castYep, all of these are good. Except the *.
Quoted 4 lines
The * are in
scripts/codeconverter/. You might want to double-check nothing deprecated hides behind the *.
Yep. Meant to, forgot. O:-) Thanks for the callout.
Quoted 2 lines
The others are all still good, I presume.
Yep! I double-checked the list and none of the remaining ones here are deprecated.
--js
PATCH 6/8 python: clean up requirements for 3.9+
In reply to John SnowWe don't need these requirements anymore, so remove them.
--- python/setup.cfg | 1 - 1 file changed, 1 deletion(-)
python/setup.cfg +0 -1
@@ -41,7 +41,6 @@ devel = isort >= 5.1.2 mypy >= 1.4.0 pylint >= 2.17.3 - pylint != 3.2.4; python_version<"3.9" tox >= 3.18.0 urwid >= 2.1.2 urwid-readline >= 0.13
Signature · 1 line
2.48.1
PATCH 7/8 python: update mkvenv to type-check under different python versions
In reply to John SnowCurrently, we instruct mypy to pretend it is type checking under a specific version of python. For our case, where our scripts may in fact actually be executing under a number of different environments in the build system, we want mypy to remove this restriction.
This patch adjusts some of the import logic in mkvenv.py to make it type check under a wider range of python versions.
--- python/scripts/mkvenv.py | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
python/scripts/mkvenv.py +9 -6
@@ -99,16 +99,18 @@ HAVE_DISTLIB = False # Try to load tomllib, with a fallback to tomli. -# HAVE_TOMLLIB is checked below, just-in-time, so that mkvenv does not fail +# TOML is checked below, just-in-time, so that mkvenv does not fail # outside the venv or before a potential call to ensurepip in checkpip(). -HAVE_TOMLLIB = True +TOML = None try: import tomllib + TOML = tomllib except ImportError: try: - import tomli as tomllib + import tomli + TOML = tomli except ImportError: - HAVE_TOMLLIB = False + pass # Do not add any mandatory dependencies from outside the stdlib: # This script *must* be usable standalone!
@@ -194,6 +196,7 @@ def compute_venv_libpath(context: SimpleNamespace) -> str: # Python 3.12+, not strictly necessary because it's documented # to be the same as 3.10 code below: if sys.version_info >= (3, 12): + assert isinstance(context.lib_path, str) return context.lib_path # Python 3.10+
@@ -710,7 +713,7 @@ def _do_ensure( def _parse_groups(file: str) -> dict[str, dict[str, Any]]: - if not HAVE_TOMLLIB: + if not TOML: if sys.version_info < (3, 11): raise Ouch("found no usable tomli, please install it")
@@ -722,7 +725,7 @@ def _parse_groups(file: str) -> dict[str, dict[str, Any]]: # Debian bullseye-backports) and v2.0.x with open(file, "r", encoding="ascii") as depfile: contents = depfile.read() - return tomllib.loads(contents) # type: ignore + return TOML.loads(contents) # type: ignore def ensure_group(
Signature · 1 line
2.48.1
PATCH 8/8 python: remove version restriction for mypy
In reply to John SnowRemoving this configuration line allows mypy to type check files from the context of whatever version of Python it happens to be running under, which is actually what we want here, so long as we do actually support running these scripts under multiple versions of Python, which we currently do.
--- python/setup.cfg | 1 - 1 file changed, 1 deletion(-)
python/setup.cfg +0 -1
@@ -77,7 +77,6 @@ exclude = __pycache__, [mypy] strict = True -python_version = 3.9 namespace_packages = True warn_unused_ignores = False
Signature · 1 line
2.48.1
Quoted 28 lines · John Snow
This series does a couple things that I'll probably end up splitting out into smaller series if history is any guide, but either way, here it goes:
A) Convert qemu.git/python/ to a
PEP517/pyproject.tomlpackage. Ultimately this means deletingsetup.pyand fully migrating to newer python infrastructure. I think this should be safe to do by now, but admittedly I am not *confident* as it relies on setuptools versions in the wild, not python versions. My motivation for trying it is to fix "make check-dev", which has been broken for the last two Fedora releases under newer setuptools which have started removing support for the pre-PEP517 packaging formats, which will only continue to get worse from here on out.B) Sync changes from the qemu.qmp package back over to qemu.git. I know I need to decouple this badly, but in order to do so, I need to make sure they're synchronized to be assured that the switch to the standalone version won't break anything, so this is a necessary step. It's happening here because of the 3.6+ compat crud we are still carrying in qemu.git that has since been removed from the standalone library.
C) Move us to 3.9+ style type hints. I know it feels like we just got the type hints settled and we're already changing them. Sorry about that. They are deprecated in 3.9, and *could* be removed at any time. I figured now was a good time as any to get rid of them before they become a problem randomly some day in the future.
Quoted 5 lines
D) Update the mypy configuration to check under multiple Python versions more effectively and thoroughly.
Whew.
--js
Appears to be
Based-on: <[email protected]>