python: add QAPI and qapidoc et al to python linter tests
cover python: add QAPI and qapidoc et al to python linter tests
Hiya, this series turns on automated linting for scripts/qapi, docs/sphinx/qapidoc.py and docs/sphinx/qapi_domain.py.
This includes flake8/isort/pylint/mypy for scripts/qapi, but omits mypy from the Sphinx plugins owing to my inability to strictly type the extensions given the wide versions of Sphinx we actually support.
Though I have been using black in my own development, I have not yet enabled it anywhere automatically. Maybe soon.
John Snow (5):
qapi: Add some pylint ignores
docs/qapidoc: linting fixes
python: update missing dependencies from minreqs
python: add qapi static analysis tests
qapi: delete un-needed python static analysis configs
docs/sphinx/qapi_domain.py | 25 ++++++++++++++----------- docs/sphinx/qapidoc.py | 5 +++-- python/setup.cfg | 1 + python/tests/minreqs.txt | 25 +++++++++++++++++++++++++ python/tests/qapi-flake8.sh | 4 ++++ python/tests/qapi-isort.sh | 6 ++++++ python/tests/qapi-mypy.sh | 2 ++ python/tests/qapi-pylint.sh | 6 ++++++ scripts/qapi/.flake8 | 3 --- scripts/qapi/.isort.cfg | 7 ------- scripts/qapi/backend.py | 2 ++ scripts/qapi/mypy.ini | 4 ---- scripts/qapi/pylintrc | 1 + 13 files changed, 64 insertions(+), 27 deletions(-) create mode 100755 python/tests/qapi-flake8.sh create mode 100755 python/tests/qapi-isort.sh create mode 100755 python/tests/qapi-mypy.sh create mode 100755 python/tests/qapi-pylint.sh delete mode 100644 scripts/qapi/.flake8 delete mode 100644 scripts/qapi/.isort.cfg delete mode 100644 scripts/qapi/mypy.ini
Signature · 1 line
2.48.1
PATCH 1/5 qapi: Add some pylint ignores
This restores the linting baseline in QAPI.
--- scripts/qapi/backend.py | 2 ++ 1 file changed, 2 insertions(+)
scripts/qapi/backend.py +2 -0
@@ -13,6 +13,7 @@ class QAPIBackend(ABC): + # pylint: disable=too-few-public-methods @abstractmethod def generate(self,
@@ -36,6 +37,7 @@ def generate(self, class QAPICBackend(QAPIBackend): + # pylint: disable=too-few-public-methods def generate(self, schema: QAPISchema,
Signature · 1 line
2.48.1
PATCH 2/5 docs/qapidoc: linting fixes
In reply to John SnowThis restores the linting baseline in qapidoc. The order of some imports have changed slightly due to configuring isort a little better: isort was having difficulty understanding that "compat" and "qapidoc_legacy" were local modules because docs/sphinx "isn't a python package".
--- docs/sphinx/qapi_domain.py | 25 ++++++++++++++----------- docs/sphinx/qapidoc.py | 5 +++-- 2 files changed, 17 insertions(+), 13 deletions(-)
docs/sphinx/qapi_domain.py +14 -11
@@ -20,16 +20,6 @@ from docutils import nodes from docutils.parsers.rst import directives - -from compat import ( - CompatField, - CompatGroupedField, - CompatTypedField, - KeywordNode, - ParserFix, - Signature, - SpaceNode, -) from sphinx import addnodes from sphinx.directives import ObjectDescription from sphinx.domains import (
@@ -44,6 +34,16 @@ from sphinx.util.docutils import SphinxDirective from sphinx.util.nodes import make_id, make_refnode +from compat import ( + CompatField, + CompatGroupedField, + CompatTypedField, + KeywordNode, + ParserFix, + Signature, + SpaceNode, +) + if TYPE_CHECKING: from typing import (
@@ -56,7 +56,6 @@ ) from docutils.nodes import Element, Node - from sphinx.addnodes import desc_signature, pending_xref from sphinx.application import Sphinx from sphinx.builders import Builder
@@ -168,6 +167,8 @@ class QAPIDescription(ParserFix): """ def handle_signature(self, sig: str, signode: desc_signature) -> Signature: + # pylint: disable=unused-argument + # Do nothing. The return value here is the "name" of the entity # being documented; for QAPI, this is the same as the # "signature", which is just a name.
@@ -210,6 +211,8 @@ def _get_fqn(self, name: Signature) -> str: def add_target_and_index( self, name: Signature, sig: str, signode: desc_signature ) -> None: + # pylint: disable=unused-argument + # name is the return value of handle_signature. # sig is the original, raw text argument to handle_signature. # For QAPI, these are identical, currently.
docs/sphinx/qapidoc.py +3 -2
@@ -27,6 +27,7 @@ from __future__ import annotations + __version__ = "2.0" from contextlib import contextmanager
@@ -56,8 +57,6 @@ QAPISchemaVisitor, ) from qapi.source import QAPISourceInfo - -from qapidoc_legacy import QAPISchemaGenRSTVisitor # type: ignore from sphinx import addnodes from sphinx.directives.code import CodeBlock from sphinx.errors import ExtensionError
@@ -65,6 +64,8 @@ from sphinx.util.docutils import SphinxDirective, switch_source_input from sphinx.util.nodes import nested_parse_with_titles +from qapidoc_legacy import QAPISchemaGenRSTVisitor # type: ignore + if TYPE_CHECKING: from typing import (
Signature · 1 line
2.48.1
Re: [PATCH 2/5] docs/qapidoc: linting fixes
Quoted 2 lines · John Snow
This restores the linting baseline in qapidoc. The order of some imports have changed slightly due to configuring isort a little better: isort
Changed since when / what?
Quoted 4 lines
was having difficulty understanding that "compat" and "
qapidoc_legacy" were local modules because docs/sphinx "isn't a python package".Signed-off-by John Snow <[email protected]>
Re: [PATCH 2/5] docs/qapidoc: linting fixes
Quoted 7 lines · Markus Armbruster
Quoted 2 lines · John Snow
This restores the linting baseline in qapidoc. The order of some imports have changed slightly due to configuring isort a little better: isort
Changed since when / what?
Changed as of this patch. Before, I was running isort directly from the docs/sphinx folder, but now I'm running it from the Python directory with some improved configuration as I explain in this commit: by teaching isort that compat and qapidoc_legacy are local modules, isort decides to arrange them differently.
Imports should always go in three sections, in order:
1. Standard library imports
2. Third party imports
3. First party (local) imports
Before, it was not, because it does not understand "docs/sphinx" as a local package with local modules - I think it gets confused because of the folder being named "sphinx".
Really, I am just explaining why some imports get shuffled around a little bit - the new order is "correct" and the old order was slightly wrong.
Quoted 7 lines
Quoted 5 lines
was having difficulty understanding that "compat" and "
qapidoc_legacy" were local modules because docs/sphinx "isn't a python package".Signed-off-by John Snow <[email protected]>
Re: [PATCH 2/5] docs/qapidoc: linting fixes
Quoted 22 lines · John Snow
Quoted 7 lines · Markus Armbruster
Quoted 2 lines · John Snow
This restores the linting baseline in qapidoc. The order of some imports have changed slightly due to configuring isort a little better: isort
Changed since when / what?
Changed as of this patch. Before, I was running isort directly from the docs/sphinx folder, but now I'm running it from the Python directory with some improved configuration as I explain in this commit: by teaching isort that compat and
qapidoc_legacyare local modules, isort decides to arrange them differently.Imports should always go in three sections, in order:
1. Standard library imports
2. Third party imports
3. First party (local) imports
Quoted 6 lines
Before, it was not, because it does not understand "docs/sphinx" as a local package with local modules - I think it gets confused because of the folder being named "sphinx".
Really, I am just explaining why some imports get shuffled around a little bit - the new order is "correct" and the old order was slightly wrong.
Please use "The order of imorts change" instead of "have changed". Present tense makes it clear that you're talking about this patch.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
was having difficulty understanding that "compat" and "
qapidoc_legacy" were local modules because docs/sphinx "isn't a python package".Signed-off-by John Snow <[email protected]>
PATCH 3/5 python: update missing dependencies from minreqs
In reply to John SnowA few transitive dependencies were left floating; as a result, pip's dependency solver can pull in newer dependencies, which we don't want. Pin them down.
--- python/tests/minreqs.txt | 4 ++++ 1 file changed, 4 insertions(+)
python/tests/minreqs.txt +4 -0
@@ -38,10 +38,14 @@ pyflakes==2.5.0 # Transitive mypy dependencies mypy-extensions==1.0.0 +tomli==1.1.0 typing-extensions==4.7.1 # Transitive pylint dependencies astroid==2.15.4 +dill==0.2 lazy-object-proxy==1.4.0 +platformdirs==2.2.0 toml==0.10.0 +tomlkit==0.10.1 wrapt==1.14.0
Signature · 1 line
2.48.1
Re: [PATCH 3/5] python: update missing dependencies from minreqs
Quoted 5 lines · John Snow
A few transitive dependencies were left floating; as a result, pip's dependency solver can pull in newer dependencies, which we don't want. Pin them down.
Signed-off-by John Snow <[email protected]>
What problem exactly does this fix? Make target check-minreqs?
Quoted 23 lines
--- python/tests/minreqs.txt | 4 ++++ 1 file changed, 4 insertions(+)python/tests/minreqs.txt +4
-0@@ -38,10 +38,14 @@ pyflakes==2.5.0 # Transitive mypy dependencies mypy-extensions==1.0.0 +tomli==1.1.0 typing-extensions==4.7.1 # Transitive pylint dependencies astroid==2.15.4 +dill==0.2 lazy-object-proxy==1.4.0 +platformdirs==2.2.0 toml==0.10.0 +tomlkit==0.10.1 wrapt==1.14.0
Re: [PATCH 3/5] python: update missing dependencies from minreqs
Quoted 10 lines · Markus Armbruster
Quoted 5 lines · John Snow
A few transitive dependencies were left floating; as a result, pip's dependency solver can pull in newer dependencies, which we don't want. Pin them down.
Signed-off-by John Snow <[email protected]>What problem exactly does this fix? Make target check-minreqs?
I'm not sure it's a "problem" as such, but an inconsistency. Yes, it's with check-minreqs -- without this patch, pip is free to choose newer versions of these dependencies as appropriate. Though unlikely at this point, in theory, new dependency updates could be selected by pip and invalidate the concept of an entirely fixed/pinned virtual environment.
That these transitive dependencies were not frozen initially was an oversight.
check-minreqs is supposed to build the exact same venv every time without fail. Without this change, it's *possible* that it might do something different on release day if someone releases a new package. No good, probably.
Quoted 26 lines
Quoted 24 lines
--- python/tests/minreqs.txt | 4 ++++ 1 file changed, 4 insertions(+)python/tests/minreqs.txt +4
-0@@ -38,10 +38,14 @@ pyflakes==2.5.0 # Transitive mypy dependencies mypy-extensions==1.0.0 +tomli==1.1.0 typing-extensions==4.7.1 # Transitive pylint dependencies astroid==2.15.4 +dill==0.2 lazy-object-proxy==1.4.0 +platformdirs==2.2.0 toml==0.10.0 +tomlkit==0.10.1 wrapt==1.14.0
Re: [PATCH 3/5] python: update missing dependencies from minreqs
Quoted 19 lines · John Snow
Quoted 10 lines · Markus Armbruster
Quoted 5 lines · John Snow
A few transitive dependencies were left floating; as a result, pip's dependency solver can pull in newer dependencies, which we don't want. Pin them down.
Signed-off-by John Snow <[email protected]>What problem exactly does this fix? Make target check-minreqs?
I'm not sure it's a "problem" as such, but an inconsistency. Yes, it's with check-minreqs -- without this patch, pip is free to choose newer versions of these dependencies as appropriate. Though unlikely at this point, in theory, new dependency updates could be selected by pip and invalidate the concept of an entirely fixed/pinned virtual environment.
Quoted 7 lines
That these transitive dependencies were not frozen initially was an oversight.
check-minreqs is supposed to build the exact same venv every time without fail. Without this change, it's *possible* that it might do something different on release day if someone releases a new package. No good, probably.
I see.
You've been spoiling me with really nice commit messages... If you'd like to push this one to that level, I'd suggest to start with a short paragraph explaining why we pin versions for check-minreq, then state the issue being fixed: we missed some pins.
[...]
Re: [PATCH 3/5] python: update missing dependencies from minreqs
Quoted 32 lines · Markus Armbruster
Quoted 1 line · John Snow
On Wed, Mar 26, 2025 at 2:08 AM Markus Armbruster <[email protected]>
Quoted 7 lines · wrote:
Quoted 7 lines
Quoted 4 lines · John Snow
A few transitive dependencies were left floating; as a result, pip's dependency solver can pull in newer dependencies, which we don't want. Pin them down.
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Signed-off-by John Snow <[email protected]>What problem exactly does this fix? Make target check-minreqs?
I'm not sure it's a "problem" as such, but an inconsistency. Yes, it's
with
Quoted 3 lines
check-minreqs -- without this patch, pip is free to choose newer versions of these dependencies as appropriate. Though unlikely at this point, in theory, new dependency updates could be selected by pip and invalidate
the
Quoted 9 lines
concept of an entirely fixed/pinned virtual environment.
That these transitive dependencies were not frozen initially was an oversight.
check-minreqs is supposed to build the exact same venv every time without fail. Without this change, it's *possible* that it might do something different on release day if someone releases a new package. No good, probably.
Quoted 7 lines
I see.
You've been spoiling me with really nice commit messages... If you'd like to push this one to that level, I'd suggest to start with a short paragraph explaining why we pin versions for check-minreq, then state the issue being fixed: we missed some pins.
"If you give a mouse a cookie, ..."
Already typed it all out to you, might as well update the commit message at this point.
--js
PATCH 4/5 python: add qapi static analysis tests
In reply to John SnowUpdate the python tests to also check qapi. No idea why I didn't do this before. I guess I was counting on moving it under python/ and then just forgot after that was NACKed. Oops, this turns out to be really easy.
flake8, isort and mypy use the tool configuration from the existing python directory (in setup.cfg). pylint continues to use the special configuration located in scripts/qapi/ - that configuration is more permissive. If we wish to unify the two configurations, that's a separate series and a discussion for a later date.
As a result of this patch, one would be able to run any of the following tests locally from the qemu.git/python directory and have it cover the scripts/qapi/ module as well. All of the following options run the python tests, static analysis tests, and linter checks; but with different combinations of dependencies and interpreters.
- "make check-minreqs" Run tests specifically under our oldest supported Python and our oldest supported dependencies. This is the test that runs on GitLab as "check-python-minreqs". This helps ensure we do not regress support on older platforms accidentally.
- "make check-tox" Runs the tests under the newest supported dependencies, but under each supported version of Python in turn. At time of writing, this is Python 3.8 to 3.13 inclusive. This test helps catch bleeding-edge problems before they become problems for developer workstations. This is the GitLab test "check-python-tox" and is an optionally run, may-fail test due to the unpredictable nature of new dependencies being released into the ecosystem that may cause regressions.
- "make check-dev" Runs the tests under the newest supported dependencies using whatever version of Python the user happens to have installed. This is a quick convenience check that does not map to any particular GitLab test.
(Note! check-dev may be busted on Fedora 41 and bleeding edge versions of setuptools. That's unrelated to this patch and I'll address it separately and soon. Thank you for your patience, --mgmt)
--- python/setup.cfg | 1 + python/tests/minreqs.txt | 21 +++++++++++++++++++++ python/tests/qapi-flake8.sh | 4 ++++ python/tests/qapi-isort.sh | 6 ++++++ python/tests/qapi-mypy.sh | 2 ++ python/tests/qapi-pylint.sh | 6 ++++++ scripts/qapi/pylintrc | 1 + 7 files changed, 41 insertions(+) create mode 100755 python/tests/qapi-flake8.sh create mode 100755 python/tests/qapi-isort.sh create mode 100755 python/tests/qapi-mypy.sh create mode 100755 python/tests/qapi-pylint.sh
python/setup.cfg +1 -0
@@ -47,6 +47,7 @@ devel = urwid >= 2.1.2 urwid-readline >= 0.13 Pygments >= 2.9.0 + sphinx >= 3.4.3 # Provides qom-fuse functionality fuse =
python/tests/minreqs.txt +21 -0
@@ -11,6 +11,9 @@ # When adding new dependencies, pin the very oldest non-yanked version # on PyPI that allows the test suite to pass. +# Dependencies for qapidoc/qapi_domain et al +sphinx==3.4.3 + # Dependencies for the TUI addon (Required for successful linting) urwid==2.1.2 urwid-readline==0.13
@@ -49,3 +52,21 @@ platformdirs==2.2.0 toml==0.10.0 tomlkit==0.10.1 wrapt==1.14.0 + +# Transitive sphinx dependencies +Jinja2==2.7 +MarkupSafe==1.1.0 +alabaster==0.7.1 +babel==1.3 +docutils==0.12 +imagesize==0.5.0 +packaging==14.0 +pytz==2011b0 +requests==2.5.0 +snowballstemmer==1.1 +sphinxcontrib-applehelp==1.0.0 +sphinxcontrib-devhelp==1.0.0 +sphinxcontrib-htmlhelp==1.0.0 +sphinxcontrib-jsmath==1.0.0 +sphinxcontrib-qthelp==1.0.0 +sphinxcontrib-serializinghtml==1.0.0
python/tests/qapi-flake8.sh new file +4 -0
@@ -0,0 +1,4 @@ +#!/bin/sh -e +python3 -m flake8 ../scripts/qapi/ \ + ../docs/sphinx/qapidoc.py \ + ../docs/sphinx/qapi_domain.py
python/tests/qapi-isort.sh new file +6 -0
@@ -0,0 +1,6 @@ +#!/bin/sh -e +python3 -m isort --sp . -c ../scripts/qapi/ +# Force isort to recognize "compat" as a local module and not third-party +python3 -m isort --sp . -c -p compat -p qapidoc_legacy \ + ../docs/sphinx/qapi_domain.py \ + ../docs/sphinx/qapidoc.py
python/tests/qapi-mypy.sh new file +2 -0
@@ -0,0 +1,2 @@ +#!/bin/sh -e +python3 -m mypy ../scripts/qapi
python/tests/qapi-pylint.sh new file +6 -0
@@ -0,0 +1,6 @@ +#!/bin/sh -e +SETUPTOOLS_USE_DISTUTILS=stdlib python3 -m pylint \ + --rcfile=../scripts/qapi/pylintrc \ + ../scripts/qapi/ \ + ../docs/sphinx/qapidoc.py \ + ../docs/sphinx/qapi_domain.py
scripts/qapi/pylintrc +1 -0
@@ -19,6 +19,7 @@ disable=consider-using-f-string, too-many-instance-attributes, too-many-positional-arguments, too-many-statements, + unknown-option-value, useless-option-value, [REPORTS]
Signature · 1 line
2.48.1
Re: [PATCH 4/5] python: add qapi static analysis tests
Quoted 35 lines · John Snow
Update the python tests to also check qapi. No idea why I didn't do this before. I guess I was counting on moving it under python/ and then just forgot after that was NACKed. Oops, this turns out to be really easy.
flake8, isort and mypy use the tool configuration from the existing python directory (in setup.cfg). pylint continues to use the special configuration located in
scripts/qapi/- that configuration is more permissive. If we wish to unify the two configurations, that's a separate series and a discussion for a later date.As a result of this patch, one would be able to run any of the following tests locally from the qemu.git/python directory and have it cover the
scripts/qapi/module as well. All of the following options run the python tests, static analysis tests, and linter checks; but with different combinations of dependencies and interpreters.- "make check-minreqs" Run tests specifically under our oldest supported Python and our oldest supported dependencies. This is the test that runs on GitLab as "check-python-minreqs". This helps ensure we do not regress support on older platforms accidentally.
- "make check-tox" Runs the tests under the newest supported dependencies, but under each supported version of Python in turn. At time of writing, this is Python 3.8 to 3.13 inclusive. This test helps catch bleeding-edge problems before they become problems for developer workstations. This is the GitLab test "check-python-tox" and is an optionally run, may-fail test due to the unpredictable nature of new dependencies being released into the ecosystem that may cause regressions.
- "make check-dev" Runs the tests under the newest supported dependencies using whatever version of Python the user happens to have installed. This is a quick convenience check that does not map to any particular GitLab test.
Quoted 5 lines
(Note! check-dev may be busted on Fedora 41 and bleeding edge versions of setuptools. That's unrelated to this patch and I'll address it separately and soon. Thank you for your patience, --mgmt)
Signed-off-by John Snow <[email protected]>
Let's mention this is a step towards having "make check" run the static analysis we want developers to run, but we're not there, yet.
Quoted 52 lines
--- python/setup.cfg | 1 + python/tests/minreqs.txt | 21 +++++++++++++++++++++ python/tests/qapi-flake8.sh | 4 ++++ python/tests/qapi-isort.sh | 6 ++++++ python/tests/qapi-mypy.sh | 2 ++ python/tests/qapi-pylint.sh | 6 ++++++ scripts/qapi/pylintrc | 1 + 7 files changed, 41 insertions(+) create mode 100755 python/tests/qapi-flake8.sh create mode 100755 python/tests/qapi-isort.sh create mode 100755 python/tests/qapi-mypy.sh create mode 100755 python/tests/qapi-pylint.shpython/setup.cfg +1
-0@@ -47,6 +47,7 @@ devel = urwid >= 2.1.2 urwid-readline >= 0.13 Pygments >= 2.9.0 + sphinx >= 3.4.3 # Provides qom-fuse functionality fuse =python/tests/minreqs.txt +11
-0@@ -11,6 +11,9 @@ # When adding new dependencies, pin the very oldest non-yanked version # on PyPI that allows the test suite to pass. +# Dependencies for qapidoc/qapi_domain et al +sphinx==3.4.3 + # Dependencies for the TUI addon (Required for successful linting) urwid==2.1.2 urwid-readline==0.13@@ -49,3 +52,21 @@ platformdirs==2.2.0 toml==0.10.0 tomlkit==0.10.1 wrapt==1.14.0 + +# Transitive sphinx dependencies +Jinja2==2.7 +MarkupSafe==1.1.0 +alabaster==0.7.1 +babel==1.3 +docutils==0.12 +imagesize==0.5.0
python/tests/minreqs.txt platformdirs==2.2.0 · line 72
Quoted 10 lines
python/tests/minreqs.txt +10
-0+packaging==14.0 +pytz==2011b0 +requests==2.5.0 +snowballstemmer==1.1 +sphinxcontrib-applehelp==1.0.0 +sphinxcontrib-devhelp==1.0.0 +sphinxcontrib-htmlhelp==1.0.0 +sphinxcontrib-jsmath==1.0.0 +sphinxcontrib-qthelp==1.0.0 +sphinxcontrib-serializinghtml==1.0.0
This wasn't there when I last saw this patch. The previous patch also updates this file. How did you decide which updates go where? Or is this an accident?
python/tests/qapi-flake8.sh line 4
Quoted 10 lines
python/tests/qapi-flake8.sh new file +4
-0@@ -0,0 +1,4 @@ +#!/bin/sh -e +python3 -m flake8 ../scripts/qapi/ \ + ../docs/sphinx/qapidoc.py \ + ../docs/sphinx/qapi_domain.py
Not linting docs/sphinx/qapidoc_legacy.py. This is intentional (see its initial commit message). Since we plan to drop it soon, there's no real need for a comment here, but mentioning it in the commit message wouldn't hurt.
python/tests/qapi-isort.sh line 6
Quoted 12 lines
python/tests/qapi-isort.sh new file +6
-0@@ -0,0 +1,6 @@ +#!/bin/sh -e +python3 -m isort --sp . -c ../scripts/qapi/ +# Force isort to recognize "compat" as a local module and not third-party +python3 -m isort --sp . -c -p compat -p qapidoc_legacy \ + ../docs/sphinx/qapi_domain.py \ + ../docs/sphinx/qapidoc.py
Comment on flake8 applies.
python/tests/qapi-mypy.sh line 2
Quoted 8 lines
python/tests/qapi-mypy.sh new file +2
-0@@ -0,0 +1,2 @@ +#!/bin/sh -e +python3 -m mypy ../scripts/qapi
Not type-checking docs/sphinx/qapi_domain.py and docs/sphinx/qapidoc.py? Impractical due to us targeting an isanely wide Sphinx version range?
python/tests/qapi-pylint.sh line 6
Quoted 12 lines
python/tests/qapi-pylint.sh new file +6
-0@@ -0,0 +1,6 @@ +#!/bin/sh -e +SETUPTOOLS_USE_DISTUTILS=stdlib python3 -m pylint \ + --rcfile=../scripts/qapi/pylintrc \ + ../scripts/qapi/ \ + ../docs/sphinx/qapidoc.py \ + ../docs/sphinx/qapi_domain.py
Comment on flake8 applies.
scripts/qapi/pylintrc disable=consider-using-f-string, · line 25
Quoted 12 lines
scripts/qapi/pylintrc +1
-0@@ -19,6 +19,7 @@ disable=consider-using-f-string, too-many-instance-attributes, too-many-positional-arguments, too-many-statements, + unknown-option-value, useless-option-value, [REPORTS]
This wasn't there when I last saw this patch. PATCH 1 also updates this file. How did you decide which updates go where? Or is this an accident?
Re: [PATCH 4/5] python: add qapi static analysis tests
Quoted 37 lines · Markus Armbruster
Quoted 35 lines · John Snow
Update the python tests to also check qapi. No idea why I didn't do this before. I guess I was counting on moving it under python/ and then just forgot after that was NACKed. Oops, this turns out to be really easy.
flake8, isort and mypy use the tool configuration from the existing python directory (in setup.cfg). pylint continues to use the special configuration located in
scripts/qapi/- that configuration is more permissive. If we wish to unify the two configurations, that's a separate series and a discussion for a later date.As a result of this patch, one would be able to run any of the following tests locally from the qemu.git/python directory and have it cover the
scripts/qapi/module as well. All of the following options run the python tests, static analysis tests, and linter checks; but with different combinations of dependencies and interpreters.- "make check-minreqs" Run tests specifically under our oldest supported Python and our oldest supported dependencies. This is the test that runs on GitLab as "check-python-minreqs". This helps ensure we do not regress support on older platforms accidentally.
- "make check-tox" Runs the tests under the newest supported dependencies, but under each supported version of Python in turn. At time of writing, this is Python 3.8 to 3.13 inclusive. This test helps catch bleeding-edge problems before they become problems for developer workstations. This is the GitLab test "check-python-tox" and is an optionally run, may-fail test due to the unpredictable nature of new dependencies being released into the ecosystem that may cause regressions.
- "make check-dev" Runs the tests under the newest supported dependencies using whatever version of Python the user happens to have installed. This is a quick convenience check that does not map to any particular GitLab test.
Quoted 9 lines
Quoted 5 lines
(Note! check-dev may be busted on Fedora 41 and bleeding edge versions of setuptools. That's unrelated to this patch and I'll address it separately and soon. Thank you for your patience, --mgmt)
Signed-off-by John Snow <[email protected]>Let's mention this is a step towards having "make check" run the static analysis we want developers to run, but we're not there, yet.
It both is and isn't. That we can now check qapi and the qapi sphinx extensions from the same place as we do linting for python/ is sufficient justification in and of itself, regardless of how we improve and integrate this testing later on.
Quoted 59 lines
Quoted 59 lines
--- python/setup.cfg | 1 + python/tests/minreqs.txt | 21 +++++++++++++++++++++ python/tests/qapi-flake8.sh | 4 ++++ python/tests/qapi-isort.sh | 6 ++++++ python/tests/qapi-mypy.sh | 2 ++ python/tests/qapi-pylint.sh | 6 ++++++ scripts/qapi/pylintrc | 1 + 7 files changed, 41 insertions(+) create mode 100755 python/tests/qapi-flake8.sh create mode 100755 python/tests/qapi-isort.sh create mode 100755 python/tests/qapi-mypy.sh create mode 100755 python/tests/qapi-pylint.shpython/setup.cfg +1
-0@@ -47,6 +47,7 @@ devel = urwid >= 2.1.2 urwid-readline >= 0.13 Pygments >= 2.9.0 + sphinx >= 3.4.3 # Provides qom-fuse functionality fuse =python/tests/minreqs.txt +17
-0@@ -11,6 +11,9 @@ # When adding new dependencies, pin the very oldest non-yanked version # on PyPI that allows the test suite to pass. +# Dependencies for qapidoc/qapi_domain et al +sphinx==3.4.3 + # Dependencies for the TUI addon (Required for successful linting) urwid==2.1.2 urwid-readline==0.13@@ -49,3 +52,21 @@ platformdirs==2.2.0 toml==0.10.0 tomlkit==0.10.1 wrapt==1.14.0 + +# Transitive sphinx dependencies +Jinja2==2.7 +MarkupSafe==1.1.0 +alabaster==0.7.1 +babel==1.3 +docutils==0.12 +imagesize==0.5.0 +packaging==14.0 +pytz==2011b0 +requests==2.5.0 +snowballstemmer==1.1 +sphinxcontrib-applehelp==1.0.0 +sphinxcontrib-devhelp==1.0.0
Quoted 9 lines
python/tests/minreqs.txt platformdirs==2.2.0 · line 72
Quoted 4 lines
python/tests/minreqs.txt +4
-0+sphinxcontrib-htmlhelp==1.0.0 +sphinxcontrib-jsmath==1.0.0 +sphinxcontrib-qthelp==1.0.0 +sphinxcontrib-serializinghtml==1.0.0This wasn't there when I last saw this patch. The previous patch also updates this file. How did you decide which updates go where? Or is this an accident?
The previous patch pins dependencies that already existed, but we neglected to pin in this file. It's fixing an existing oversight.
This patch adds a bunch of new pinned dependencies for Sphinx, which we need for type-checking Sphinx extensions.
Quoted 17 lines
python/tests/qapi-flake8.sh line 4
Quoted 11 lines
python/tests/qapi-flake8.sh new file +4
-0@@ -0,0 +1,4 @@ +#!/bin/sh -e +python3 -m flake8 ../scripts/qapi/ \ + ../docs/sphinx/qapidoc.py \ + ../docs/sphinx/qapi_domain.pyNot linting
docs/sphinx/qapidoc_legacy.py. This is intentional (see its initial commit message). Since we plan to drop it soon, there's no real need for a comment here, but mentioning it in the commit message wouldn't hurt.
Alright.
Quoted 17 lines
python/tests/qapi-isort.sh line 3
Quoted 10 lines
python/tests/qapi-isort.sh new file +3
-0@@ -0,0 +1,6 @@ +#!/bin/sh -e +python3 -m isort --sp . -c ../scripts/qapi/ +# Force isort to recognize "compat" as a local module and notthird-party
python/tests/qapi-isort.sh line 6
Quoted 3 lines
python/tests/qapi-isort.sh new file +3
-0+python3 -m isort --sp . -c -p compat -p qapidoc_legacy \ + ../docs/sphinx/qapi_domain.py \ + ../docs/sphinx/qapidoc.pyComment on flake8 applies.
Quoted 12 lines
python/tests/qapi-mypy.sh line 2
Quoted 8 lines
python/tests/qapi-mypy.sh new file +2
-0@@ -0,0 +1,2 @@ +#!/bin/sh -e +python3 -m mypy ../scripts/qapiNot type-checking
docs/sphinx/qapi_domain.pyanddocs/sphinx/qapidoc.py? Impractical due to us targeting an isanely wide Sphinx version range?
Yes.
Quoted 27 lines
python/tests/qapi-pylint.sh line 6
Quoted 13 lines
python/tests/qapi-pylint.sh new file +6
-0@@ -0,0 +1,6 @@ +#!/bin/sh -e +SETUPTOOLS_USE_DISTUTILS=stdlib python3 -m pylint \ + --rcfile=../scripts/qapi/pylintrc \ + ../scripts/qapi/ \ + ../docs/sphinx/qapidoc.py \ + ../docs/sphinx/qapi_domain.pyComment on flake8 applies.
Quoted 11 lines
scripts/qapi/pylintrc +1
-0@@ -19,6 +19,7 @@ disable=consider-using-f-string, too-many-instance-attributes, too-many-positional-arguments, too-many-statements, + unknown-option-value, useless-option-value,
Quoted 5 lines
scripts/qapi/pylintrc disable=consider-using-f-string, · line 25
Quoted 1 line
scripts/qapi/pylintrc +0
-0[REPORTS]This wasn't there when I last saw this patch. PATCH 1 also updates this file. How did you decide which updates go where? Or is this an accident?
I didn't add the Sphinx extensions last time you saw this series, so that's new. This winds up being needed to tolerate the "too many positional arguments" option which only applies to newer pylint versions - older versions will complain about the option being unrecognized. In order to continue allowing a wide version of pylint versions, we need this option.
Re: [PATCH 4/5] python: add qapi static analysis tests
Quoted 45 lines · John Snow
Quoted 43 lines · Markus Armbruster
Quoted 40 lines · John Snow
Update the python tests to also check qapi. No idea why I didn't do this before. I guess I was counting on moving it under python/ and then just forgot after that was NACKed. Oops, this turns out to be really easy.
flake8, isort and mypy use the tool configuration from the existing python directory (in setup.cfg). pylint continues to use the special configuration located in
scripts/qapi/- that configuration is more permissive. If we wish to unify the two configurations, that's a separate series and a discussion for a later date.As a result of this patch, one would be able to run any of the following tests locally from the qemu.git/python directory and have it cover the
scripts/qapi/module as well. All of the following options run the python tests, static analysis tests, and linter checks; but with different combinations of dependencies and interpreters.- "make check-minreqs" Run tests specifically under our oldest supported Python and our oldest supported dependencies. This is the test that runs on GitLab as "check-python-minreqs". This helps ensure we do not regress support on older platforms accidentally.
- "make check-tox" Runs the tests under the newest supported dependencies, but under each supported version of Python in turn. At time of writing, this is Python 3.8 to 3.13 inclusive. This test helps catch bleeding-edge problems before they become problems for developer workstations. This is the GitLab test "check-python-tox" and is an optionally run, may-fail test due to the unpredictable nature of new dependencies being released into the ecosystem that may cause regressions.
- "make check-dev" Runs the tests under the newest supported dependencies using whatever version of Python the user happens to have installed. This is a quick convenience check that does not map to any particular GitLab test.
(Note! check-dev may be busted on Fedora 41 and bleeding edge versions of setuptools. That's unrelated to this patch and I'll address it separately and soon. Thank you for your patience, --mgmt)
Signed-off-by John Snow <[email protected]>
Quoted 8 lines
Quoted 3 lines
Let's mention this is a step towards having "make check" run the static analysis we want developers to run, but we're not there, yet.
It both is and isn't. That we can now check qapi and the qapi sphinx extensions from the same place as we do linting for python/ is sufficient justification in and of itself, regardless of how we improve and integrate this testing later on.
Alright.
Quoted 68 lines
Quoted 58 lines
Quoted 58 lines
--- python/setup.cfg | 1 + python/tests/minreqs.txt | 21 +++++++++++++++++++++ python/tests/qapi-flake8.sh | 4 ++++ python/tests/qapi-isort.sh | 6 ++++++ python/tests/qapi-mypy.sh | 2 ++ python/tests/qapi-pylint.sh | 6 ++++++ scripts/qapi/pylintrc | 1 + 7 files changed, 41 insertions(+) create mode 100755 python/tests/qapi-flake8.sh create mode 100755 python/tests/qapi-isort.sh create mode 100755 python/tests/qapi-mypy.sh create mode 100755 python/tests/qapi-pylint.shpython/setup.cfg +1
-0@@ -47,6 +47,7 @@ devel = urwid >= 2.1.2 urwid-readline >= 0.13 Pygments >= 2.9.0 + sphinx >= 3.4.3 # Provides qom-fuse functionality fuse =python/tests/minreqs.txt +17
-0@@ -11,6 +11,9 @@ # When adding new dependencies, pin the very oldest non-yanked version # on PyPI that allows the test suite to pass. +# Dependencies for qapidoc/qapi_domain et al +sphinx==3.4.3 + # Dependencies for the TUI addon (Required for successful linting) urwid==2.1.2 urwid-readline==0.13@@ -49,3 +52,21 @@ platformdirs==2.2.0 toml==0.10.0 tomlkit==0.10.1 wrapt==1.14.0 + +# Transitive sphinx dependencies +Jinja2==2.7 +MarkupSafe==1.1.0 +alabaster==0.7.1 +babel==1.3 +docutils==0.12 +imagesize==0.5.0 +packaging==14.0 +pytz==2011b0 +requests==2.5.0 +snowballstemmer==1.1 +sphinxcontrib-applehelp==1.0.0 +sphinxcontrib-devhelp==1.0.0Quoted 9 lines
python/tests/minreqs.txt platformdirs==2.2.0 · line 72
Quoted 4 lines
python/tests/minreqs.txt +4
-0+sphinxcontrib-htmlhelp==1.0.0 +sphinxcontrib-jsmath==1.0.0 +sphinxcontrib-qthelp==1.0.0 +sphinxcontrib-serializinghtml==1.0.0This wasn't there when I last saw this patch. The previous patch also updates this file. How did you decide which updates go where? Or is this an accident?
Quoted 5 lines
The previous patch pins dependencies that already existed, but we neglected to pin in this file. It's fixing an existing oversight.
This patch adds a bunch of new pinned dependencies for Sphinx, which we need for type-checking Sphinx extensions.
So... the previous patch fixes existing tests, and this one extends their coverage to the modern parts of docs/sphinx/. Correct?
Which tests exactly? I just asked that on the previous patch.
[...]
Quoted 18 lines
Quoted 11 lines
Quoted 11 lines
scripts/qapi/pylintrc +1
-0@@ -19,6 +19,7 @@ disable=consider-using-f-string, too-many-instance-attributes, too-many-positional-arguments, too-many-statements, + unknown-option-value, useless-option-value,Quoted 5 lines
scripts/qapi/pylintrc disable=consider-using-f-string, · line 25
Quoted 1 line
scripts/qapi/pylintrc +0
-0[REPORTS]This wasn't there when I last saw this patch. PATCH 1 also updates this file. How did you decide which updates go where? Or is this an accident?
Quoted 5 lines
I didn't add the Sphinx extensions last time you saw this series, so that's new. This winds up being needed to tolerate the "too many positional arguments" option which only applies to newer pylint versions - older versions will complain about the option being unrecognized. In order to continue allowing a wide version of pylint versions, we need this option.
Got it. Worth a comment?
Re: [PATCH 4/5] python: add qapi static analysis tests
Quoted 139 lines · Markus Armbruster
Quoted 1 line · John Snow
On Tue, Mar 25, 2025 at 3:53 AM Markus Armbruster <[email protected]>
Quoted 4 lines · wrote:
Quoted 4 lines
Quoted 1 line · John Snow
Update the python tests to also check qapi. No idea why I didn't do
this
Quoted 1 line
Quoted 1 line
Quoted 1 line
before. I guess I was counting on moving it under python/ and then
just
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
forgot after that was NACKed. Oops, this turns out to be really easy.
flake8, isort and mypy use the tool configuration from the existing python directory (in setup.cfg). pylint continues to use the special configuration located in
scripts/qapi/- that configuration is more permissive. If we wish to unify the two configurations, that's a separate series and a discussion for a later date.As a result of this patch, one would be able to run any of the
following
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
tests locally from the qemu.git/python directory and have it cover the
scripts/qapi/module as well. All of the following options run the python tests, static analysis tests, and linter checks; but with different combinations of dependencies and interpreters.- "make check-minreqs" Run tests specifically under our oldest
supported
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Python and our oldest supported dependencies. This is the test that runs on GitLab as "check-python-minreqs". This helps ensure we do
not
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
regress support on older platforms accidentally.
- "make check-tox" Runs the tests under the newest supported dependencies, but under each supported version of Python in turn. At time of writing, this is Python 3.8 to 3.13 inclusive. This test
helps
Quoted 1 line
Quoted 1 line
Quoted 1 line
catch bleeding-edge problems before they become problems for
developer
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
workstations. This is the GitLab test "check-python-tox" and is an optionally run, may-fail test due to the unpredictable nature of new dependencies being released into the ecosystem that may cause regressions.
- "make check-dev" Runs the tests under the newest supported dependencies using whatever version of Python the user happens to
have
Quoted 1 line
Quoted 1 line
Quoted 1 line
installed. This is a quick convenience check that does not map to
any
Quoted 8 lines
Quoted 8 lines
Quoted 7 lines
particular GitLab test.
(Note! check-dev may be busted on Fedora 41 and bleeding edge versions of setuptools. That's unrelated to this patch and I'll address it separately and soon. Thank you for your patience, --mgmt)
Signed-off-by John Snow <[email protected]>Quoted 7 lines
Quoted 3 lines
Let's mention this is a step towards having "make check" run the static analysis we want developers to run, but we're not there, yet.
It both is and isn't. That we can now check qapi and the qapi sphinx extensions from the same place as we do linting for python/ is sufficient justification in and of itself, regardless of how we improve and
integrate
Quoted 1 line
this testing later on.
Alright.
Quoted 24 lines
Quoted 24 lines
Quoted 24 lines
--- python/setup.cfg | 1 + python/tests/minreqs.txt | 21 +++++++++++++++++++++ python/tests/qapi-flake8.sh | 4 ++++ python/tests/qapi-isort.sh | 6 ++++++ python/tests/qapi-mypy.sh | 2 ++ python/tests/qapi-pylint.sh | 6 ++++++ scripts/qapi/pylintrc | 1 + 7 files changed, 41 insertions(+) create mode 100755 python/tests/qapi-flake8.sh create mode 100755 python/tests/qapi-isort.sh create mode 100755 python/tests/qapi-mypy.sh create mode 100755 python/tests/qapi-pylint.shpython/setup.cfg +1
-0@@ -47,6 +47,7 @@ devel = urwid >= 2.1.2 urwid-readline >= 0.13 Pygments >= 2.9.0 + sphinx >= 3.4.3Quoted 8 lines
Quoted 8 lines
python/tests/minreqs.txt line 11
Quoted 8 lines
python/setup.cfg +0
-0# Provides qom-fuse functionality fuse =python/tests/minreqs.txt +0
-0@@ -11,6 +11,9 @@ # When adding new dependencies, pin the very oldest non-yankedversion
Quoted 36 lines
Quoted 26 lines
Quoted 26 lines
python/tests/minreqs.txt +17
-0# on PyPI that allows the test suite to pass. +# Dependencies for qapidoc/qapi_domain et al +sphinx==3.4.3 + # Dependencies for the TUI addon (Required for successful linting) urwid==2.1.2 urwid-readline==0.13@@ -49,3 +52,21 @@ platformdirs==2.2.0 toml==0.10.0 tomlkit==0.10.1 wrapt==1.14.0 + +# Transitive sphinx dependencies +Jinja2==2.7 +MarkupSafe==1.1.0 +alabaster==0.7.1 +babel==1.3 +docutils==0.12 +imagesize==0.5.0 +packaging==14.0 +pytz==2011b0 +requests==2.5.0 +snowballstemmer==1.1 +sphinxcontrib-applehelp==1.0.0 +sphinxcontrib-devhelp==1.0.0Quoted 9 lines
python/tests/minreqs.txt platformdirs==2.2.0 · line 72
Quoted 4 lines
python/tests/minreqs.txt +4
-0+sphinxcontrib-htmlhelp==1.0.0 +sphinxcontrib-jsmath==1.0.0 +sphinxcontrib-qthelp==1.0.0 +sphinxcontrib-serializinghtml==1.0.0This wasn't there when I last saw this patch. The previous patch also updates this file. How did you decide which updates go where? Or is this an accident?
Quoted 1 line
The previous patch pins dependencies that already existed, but we
Quoted 9 lines
neglected
Quoted 4 lines
to pin in this file. It's fixing an existing oversight.
This patch adds a bunch of new pinned dependencies for Sphinx, which we need for type-checking Sphinx extensions.
So... the previous patch fixes existing tests, and this one extends their coverage to the modern parts of docs/sphinx/. Correct?
Yep.
Quoted 3 lines
Which tests exactly? I just asked that on the previous patch.
minreqs.txt concerns only make check-minreqs, but the shell script tests that just invoke a linter concern all launch avenues: make check, make check-dev, make check-tox, make check-minreqs.
Quoted 21 lines
[...]
Quoted 18 lines
Quoted 16 lines
scripts/qapi/pylintrc disable=consider-using-f-string, · line 25
Quoted 12 lines
scripts/qapi/pylintrc +1
-0@@ -19,6 +19,7 @@ disable=consider-using-f-string, too-many-instance-attributes, too-many-positional-arguments, too-many-statements, + unknown-option-value, useless-option-value, [REPORTS]This wasn't there when I last saw this patch. PATCH 1 also updates this file. How did you decide which updates go where? Or is this an accident?
Quoted 9 lines
Quoted 1 line
I didn't add the Sphinx extensions last time you saw this series, so
that's
Quoted 4 lines
new. This winds up being needed to tolerate the "too many positional arguments" option which only applies to newer pylint versions - older versions will complain about the option being unrecognized. In order to continue allowing a wide version of pylint versions, we need this option.
Got it. Worth a comment?
Yep, I can update the commit message.
PATCH 5/5 qapi: delete un-needed python static analysis configs
In reply to John SnowThe pylint config is being left in place because the settings differ enough from the python/ directory settings that we need a chit-chat on how to merge them O:-)
Everything else can go.
--- scripts/qapi/.flake8 | 3 --- scripts/qapi/.isort.cfg | 7 ------- scripts/qapi/mypy.ini | 4 ---- 3 files changed, 14 deletions(-) delete mode 100644 scripts/qapi/.flake8 delete mode 100644 scripts/qapi/.isort.cfg delete mode 100644 scripts/qapi/mypy.ini
scripts/qapi/.flake8 deleted +0 -3
@@ -1,3 +0,0 @@ -[flake8] -# Prefer pylint's bare-except checks to flake8's -extend-ignore = E722
scripts/qapi/.isort.cfg deleted +0 -7
@@ -1,7 +0,0 @@ -[settings] -force_grid_wrap=4 -force_sort_within_sections=True -include_trailing_comma=True -line_length=72 -lines_after_imports=2 -multi_line_output=3
scripts/qapi/mypy.ini deleted +0 -4
@@ -1,4 +0,0 @@ -[mypy] -strict = True -disallow_untyped_calls = False -python_version = 3.8
Signature · 1 line
2.48.1
Re: [PATCH 5/5] qapi: delete un-needed python static analysis configs
Quoted 22 lines · John Snow
The pylint config is being left in place because the settings differ enough from the python/ directory settings that we need a chit-chat on how to merge them O:-)
Everything else can go.
Signed-off-by John Snow <[email protected]>--- scripts/qapi/.flake8 | 3 --- scripts/qapi/.isort.cfg | 7 ------- scripts/qapi/mypy.ini | 4 ---- 3 files changed, 14 deletions(-) delete mode 100644 scripts/qapi/.flake8 delete mode 100644 scripts/qapi/.isort.cfg delete mode 100644 scripts/qapi/mypy.iniscripts/qapi/.flake8 deleted +0
-0
scripts/qapi/.flake8 line 3
Quoted 3 lines
scripts/qapi/.flake8 deleted +0
-3@@ -1,3 +0,0 @@ -[flake8] -# Prefer pylint's bare-except checks to flake8's -extend-ignore = E722
python/setup.cfg has:
[flake8]
# Prefer pylint's bare-except checks to flake8's extend-ignore = E722
exclude = __pycache__,
Good.
scripts/qapi/.isort.cfg line 7
Quoted 13 lines
scripts/qapi/.isort.cfg deleted +0
-7@@ -1,7 +0,0 @@ -[settings] -force_grid_wrap=4 -force_sort_within_sections=True -include_trailing_comma=True -line_length=72 -lines_after_imports=2 -multi_line_output=3
python/setup.cfg has:
[isort]
force_grid_wrap=4
force_sort_within_sections=True
include_trailing_comma=True
line_length=72
lines_after_imports=2
multi_line_output=3
Good.
scripts/qapi/mypy.ini line 4
Quoted 10 lines
scripts/qapi/mypy.ini deleted +0
-4@@ -1,4 +0,0 @@ -[mypy] -strict = True -disallow_untyped_calls = False -python_version = 3.8
python/setup.cfg has:
[mypy]
strict = True
python_version = 3.8
warn_unused_configs = True
namespace_packages = True
warn_unused_ignores = False
Also a bunch of [mypy-FOO] sections that don't apply here.
You explained the differences in review of a prior iteration. Recap:
} warn_unused_configs: Catches config values that aren't actually recognized } or used. Was helpful once upon a time when re-arranging the Python } directory to behave like a package to ensure that the conf files were } working correctly.
Could this be culled now?
Hmm, according to mypy(1), strict implies warn-unused-configs.
The question does not block this patch.
} namespace_packages: Needed for the python/ directory structure (nested } packages under a namespace, "qemu"). Doesn't impact scripts/qapi at all. } Read up on PEP420 if you are curious. Details in commit message, see below } if you're still curious.
mypy(1) makes me suspect this is the default. If that's true across the versions we care for, this could be culled.
Also does not block this patch.
} warn_unused_ignores: Needed once upon a time for cross-version mypy support } where some versions would warn in some cases and others would not. Adding } an ignore would effectively just invert which versions complained. Probably } still needed, but it's hard to measure.
Harmless enough.
} python_version: Changes mypy behavior regardless of the invoking python } interpreter to check the file as if it were to be executed by Python 3.8. I } actually want to remove this value from setup.cfg but haven't yet. I } removed it from the python-qemu-qmp repo and never added it for qapi. } Removing it is actually probably correct as it will catch errors specific } to various python versions we support, but there are some nits to iron out } in my neck of the woods. This is a case where scripts/qapi/ is stricter } than python/ :)
} (Not reasonable to solve for this series.)
Also present in the deleted file, so no change.
} lack of disallow_untyped_calls = False: I think this might be a remnant } from when we gradually typed qapi; it's evidently no longer needed since } qapi still checks fine without this affordance. The default under strict is } True.
Fair enough.
Let's mention the differences in the commit message. Here's my try:
Since the previous commit, python/setup.cfg applies to scripts/qapi/ as well. Configuration files in scripts/qapi/ override python/setup.cfg.
scripts/qapi/.flake8 and scripts/qapi/.isort.cfg actually match python/setup.cfg exactly, and can go.
The differences between scripts/qapi/mypy.ini and python/setup.cfg are harmless: [list the differences, explain why they're harmless as long as you can keep it brief, and if not, fall back to "trust me"]. So scripts/qapi/mypy.ini can go, too.
The pylint config is being left in place because the settings differ enough from the python/ directory settings that we need a chit-chat on how to merge them O:-)
With something like that
Re: [PATCH 5/5] qapi: delete un-needed python static analysis configs
Quoted 87 lines · Markus Armbruster
Quoted 22 lines · John Snow
The pylint config is being left in place because the settings differ enough from the python/ directory settings that we need a chit-chat on how to merge them O:-)
Everything else can go.
Signed-off-by John Snow <[email protected]>--- scripts/qapi/.flake8 | 3 --- scripts/qapi/.isort.cfg | 7 ------- scripts/qapi/mypy.ini | 4 ---- 3 files changed, 14 deletions(-) delete mode 100644 scripts/qapi/.flake8 delete mode 100644 scripts/qapi/.isort.cfg delete mode 100644 scripts/qapi/mypy.iniscripts/qapi/.flake8 deleted +0
-0scripts/qapi/.flake8 line 3
Quoted 3 lines
scripts/qapi/.flake8 deleted +0
-3@@ -1,3 +0,0 @@ -[flake8] -# Prefer pylint's bare-except checks to flake8's -extend-ignore = E722python/setup.cfg has:
[flake8]
# Prefer pylint's bare-except checks to flake8's extend-ignore = E722
exclude =__pycache__,Good.
scripts/qapi/.isort.cfg line 7
Quoted 13 lines
scripts/qapi/.isort.cfg deleted +0
-7@@ -1,7 +0,0 @@ -[settings] -force_grid_wrap=4 -force_sort_within_sections=True -include_trailing_comma=True -line_length=72 -lines_after_imports=2 -multi_line_output=3python/setup.cfg has:
[isort] force_grid_wrap=4 force_sort_within_sections=True include_trailing_comma=True line_length=72 lines_after_imports=2 multi_line_output=3Good.
scripts/qapi/mypy.ini line 4
Quoted 10 lines
scripts/qapi/mypy.ini deleted +0
-4@@ -1,4 +0,0 @@ -[mypy] -strict = True -disallow_untyped_calls = False -python_version = 3.8python/setup.cfg has:
[mypy]
strict = True
python_version= 3.8
warn_unused_configs= True
namespace_packages= True
warn_unused_ignores= FalseAlso a bunch of [mypy-FOO] sections that don't apply here.
You explained the differences in review of a prior iteration. Recap:
Quoted 8 lines
}
warn_unused_configs: Catches config values that aren't actually recognized
} or used. Was helpful once upon a time when re-arranging the Python } directory to behave like a package to ensure that the conf files were } working correctly.Could this be culled now?
Maybe!
Quoted 5 lines
Hmm, according to mypy(1), strict implies warn-unused-configs.
The question does not block this patch.
Send me a patch to drop it O:-)
Quoted 7 lines
}
namespace_packages: Needed for the python/ directory structure (nested } packages under a namespace, "qemu"). Doesn't impactscripts/qapiat all. } Read up on PEP420 if you are curious. Details in commit message, see below
} if you're still curious.
Quoted 5 lines
mypy(1) makes me suspect this is the default. If that's true across the versions we care for, this could be culled.
Also does not block this patch.
It definitely wasn't once upon a time. It may still not be true on the oldest mypy we currently support. We don't have a clear policy for what versions of python libraries we need to support - this is a muddy, gray area. So far I just try to avoid breaking support on older versions needlessly, but I don't have an upgrade policy.
If we want to integrate this directly into make check, we'll likely need to formalize this policy.
Quoted 46 lines
}
warn_unused_ignores: Needed once upon a time for cross-version mypy support
} where some versions would warn in some cases and others would not. Adding } an ignore would effectively just invert which versions complained. Probably
} still needed, but it's hard to measure.Harmless enough.
}
python_version: Changes mypy behavior regardless of the invoking python } interpreter to check the file as if it were to be executed by Python 3.8. I
} actually want to remove this value from setup.cfg but haven't yet. I } removed it from the python-qemu-qmp repo and never added it for qapi. } Removing it is actually probably correct as it will catch errors specific } to various python versions we support, but there are some nits to iron out
} in my neck of the woods. This is a case wherescripts/qapi/is stricter } than python/ :)
} (Not reasonable to solve for this series.)Also present in the deleted file, so no change.
} lack of
disallow_untyped_calls= False: I think this might be a remnant } from when we gradually typed qapi; it's evidently no longer needed since } qapi still checks fine without this affordance. The default under strict is
} True.Fair enough.
Let's mention the differences in the commit message. Here's my try:
Since the previous commit, python/setup.cfg applies to
scripts/qapi/as well. Configuration files inscripts/qapi/override python/setup.cfg.
scripts/qapi/.flake8andscripts/qapi/.isort.cfgactually match python/setup.cfg exactly, and can go.The differences between
scripts/qapi/mypy.iniand python/setup.cfg are harmless: [list the differences, explain why they're harmless as long as you can keep it brief, and if not, fall back to "trust me"]. Soscripts/qapi/mypy.inican go, too.
Quoted 7 lines
The pylint config is being left in place because the settings differ enough from the python/ directory settings that we need a chit-chat on how to merge them O:-)
With something like that
Reviewed-by Markus Armbruster <[email protected]>
okey-dokey, let me integrate this feedback and I'll re-send, but I'm going to wait until we hash everything else out - you had some questions on other bits in this series.
Re: [PATCH 5/5] qapi: delete un-needed python static analysis configs
Quoted 11 lines · John Snow
Quoted 9 lines · Markus Armbruster
Quoted 7 lines · John Snow
The pylint config is being left in place because the settings differ enough from the python/ directory settings that we need a chit-chat on how to merge them O:-)
Everything else can go.
Signed-off-by John Snow <[email protected]>
[...]
Quoted 36 lines
Quoted 22 lines
scripts/qapi/mypy.ini line 4
Quoted 10 lines
scripts/qapi/mypy.ini deleted +0
-4@@ -1,4 +0,0 @@ -[mypy] -strict = True -disallow_untyped_calls = False -python_version = 3.8python/setup.cfg has:
[mypy]
strict = True
python_version= 3.8
warn_unused_configs= True
namespace_packages= True
warn_unused_ignores= FalseAlso a bunch of [mypy-FOO] sections that don't apply here.
Quoted 9 lines
You explained the differences in review of a prior iteration. Recap:
}
warn_unused_configs: Catches config values that aren't actually recognized } or used. Was helpful once upon a time when re-arranging the Python } directory to behave like a package to ensure that the conf files were } working correctly.Could this be culled now?
Maybe!
Quoted 1 line
Quoted 6 lines
Quoted 4 lines
Hmm, according to mypy(1), strict implies warn-unused-configs.
The question does not block this patch.
Send me a patch to drop it O:-)
Done:
[PATCH] python: Drop redundant warn_unused_configs = True
Message-ID: <[email protected]>
Quoted 8 lines
Quoted 8 lines
}
namespace_packages: Needed for the python/ directory structure (nested } packages under a namespace, "qemu"). Doesn't impactscripts/qapiat all. } Read up on PEP420 if you are curious. Details in commit message, see below } if you're still curious.mypy(1) makes me suspect this is the default. If that's true across the versions we care for, this could be culled.
Quoted 8 lines
Quoted 2 lines
Also does not block this patch.
It definitely wasn't once upon a time. It may still not be true on the oldest mypy we currently support. We don't have a clear policy for what versions of python libraries we need to support - this is a muddy, gray area. So far I just try to avoid breaking support on older versions needlessly, but I don't have an upgrade policy.
So what is the oldest mypy we currently support? Unknown, best effort to fix any breakage we see? Wouldn't quite match my dictionary's definition of "support"...
Quoted 2 lines
If we want to integrate this directly into make check, we'll likely need to formalize this policy.
My gut feeling: supporting old mypy isn't worth much (if any) trouble.
Quoted 44 lines
Quoted 44 lines
}
warn_unused_ignores: Needed once upon a time for cross-version mypy support } where some versions would warn in some cases and others would not. Adding } an ignore would effectively just invert which versions complained. Probably } still needed, but it's hard to measure.Harmless enough.
}
python_version: Changes mypy behavior regardless of the invoking python } interpreter to check the file as if it were to be executed by Python 3.8. I } actually want to remove this value from setup.cfg but haven't yet. I } removed it from the python-qemu-qmp repo and never added it for qapi. } Removing it is actually probably correct as it will catch errors specific } to various python versions we support, but there are some nits to iron out } in my neck of the woods. This is a case wherescripts/qapi/is stricter } than python/ :)
} (Not reasonable to solve for this series.)Also present in the deleted file, so no change.
} lack of
disallow_untyped_calls= False: I think this might be a remnant } from when we gradually typed qapi; it's evidently no longer needed since } qapi still checks fine without this affordance. The default under strict is } True.Fair enough.
Let's mention the differences in the commit message. Here's my try:
Since the previous commit, python/setup.cfg applies to
scripts/qapi/as well. Configuration files inscripts/qapi/override python/setup.cfg.
scripts/qapi/.flake8andscripts/qapi/.isort.cfgactually match python/setup.cfg exactly, and can go.The differences between
scripts/qapi/mypy.iniand python/setup.cfg are harmless: [list the differences, explain why they're harmless as long as you can keep it brief, and if not, fall back to "trust me"]. Soscripts/qapi/mypy.inican go, too.The pylint config is being left in place because the settings differ enough from the python/ directory settings that we need a chit-chat on how to merge them O:-)
Quoted 7 lines
Quoted 3 lines
With something like that
Reviewed-by Markus Armbruster <[email protected]>okey-dokey, let me integrate this feedback and I'll re-send, but I'm going to wait until we hash everything else out - you had some questions on other bits in this series.
Thanks!
Re: [PATCH 5/5] qapi: delete un-needed python static analysis configs
Quoted 59 lines · Markus Armbruster
Quoted 1 line · John Snow
On Tue, Mar 25, 2025 at 4:05 AM Markus Armbruster <[email protected]>
Quoted 10 lines · wrote:
Quoted 10 lines
Quoted 7 lines · John Snow
The pylint config is being left in place because the settings differ enough from the python/ directory settings that we need a chit-chat on how to merge them O:-)
Everything else can go.
Signed-off-by John Snow <[email protected]>[...]
Quoted 20 lines
Quoted 20 lines
scripts/qapi/mypy.ini line 4
Quoted 10 lines
scripts/qapi/mypy.ini deleted +0
-4@@ -1,4 +0,0 @@ -[mypy] -strict = True -disallow_untyped_calls = False -python_version = 3.8python/setup.cfg has:
[mypy]
strict = True
python_version= 3.8
warn_unused_configs= True
namespace_packages= True
warn_unused_ignores= FalseQuoted 5 lines
Quoted 5 lines
Also a bunch of [mypy-FOO] sections that don't apply here.
You explained the differences in review of a prior iteration. Recap:
}
warn_unused_configs: Catches config values that aren't actuallyrecognized
Quoted 16 lines
Quoted 6 lines
} or used. Was helpful once upon a time when re-arranging the Python } directory to behave like a package to ensure that the conf files were } working correctly.
Could this be culled now?
Maybe!
Quoted 5 lines
Hmm, according to mypy(1), strict implies warn-unused-configs.
The question does not block this patch.
Quoted 7 lines
Quoted 1 line
Send me a patch to drop it O:-)
Done:
[PATCH] python: Drop redundant
warn_unused_configs= True
Message-ID: <[email protected]>
yay :)
Quoted 20 lines
Quoted 3 lines
Quoted 3 lines
}
namespace_packages: Needed for the python/ directory structure (nested } packages under a namespace, "qemu"). Doesn't impactscripts/qapiatall.
Quoted 1 line
Quoted 1 line
} Read up on PEP420 if you are curious. Details in commit message, see
below
Quoted 5 lines
Quoted 5 lines
} if you're still curious.
mypy(1) makes me suspect this is the default. If that's true across the versions we care for, this could be culled.
Quoted 8 lines
Quoted 2 lines
Also does not block this patch.
It definitely wasn't once upon a time. It may still not be true on the oldest mypy we currently support. We don't have a clear policy for what versions of python libraries we need to support - this is a muddy, gray area. So far I just try to avoid breaking support on older versions needlessly, but I don't have an upgrade policy.
Quoted 4 lines
So what is the oldest mypy we currently support? Unknown, best effort to fix any breakage we see? Wouldn't quite match my dictionary's definition of "support"...
from python/setup.cfg; mypy >= 1.4.0
from python/tests/minreqs.txt; mypy==1.4.0
This version is used for the minreqs test so I can be assured that we haven't broken anything for the old pythons and the old type checkers. I guarantee this version *will* work. I also guarantee the latest version will work. I don't necessarily guarantee everything in between, but I make my best effort.
Quoted 7 lines
Quoted 2 lines
If we want to integrate this directly into make check, we'll likely need
to
Quoted 1 line
formalize this policy.
My gut feeling: supporting old mypy isn't worth much (if any) trouble.
Very extremely likely true! If people are fine with "you need PyPI and and internet connection to run linter tests *at all*" we could move to the latest version(s) (as supported per-python interpreter, at least) and be done with it.
As I recall, some folks (Kevin Wolf?) had some requirements that they be able to run iotests and so forth outside of configure, which makes this kind of thing harder.
The tests grew out of ad-hoc scripts that had to work with whatever you happened to have installed, though, and so it still supports a wide range of linter versions for that reason. If I had my way, I'd stipulate we only ever use a specific version that I pin quite close to the current bleeding edge and then just periodically update the pin as needed for new features or bugfixes or compatibility for shifting python interpreter support windows.
If you want to discuss this and tackle it next release, I'm open to it, I've just lost a *lot* of appetite for making new package version requirement edicts in and around the build/doc/test system.
Quoted 70 lines
Quoted 2 lines
Quoted 2 lines
}
warn_unused_ignores: Needed once upon a time for cross-version mypysupport
Quoted 1 line
Quoted 1 line
} where some versions would warn in some cases and others would not.
Adding
Quoted 1 line
Quoted 1 line
} an ignore would effectively just invert which versions complained.
Probably
Quoted 5 lines
Quoted 5 lines
} still needed, but it's hard to measure.
Harmless enough.
}
python_version: Changes mypy behavior regardless of the invokingpython
Quoted 1 line
Quoted 1 line
} interpreter to check the file as if it were to be executed by Python
3.8. I
Quoted 3 lines
Quoted 3 lines
} actually want to remove this value from setup.cfg but haven't yet. I } removed it from the python-qemu-qmp repo and never added it for qapi. } Removing it is actually probably correct as it will catch errors
specific
Quoted 1 line
Quoted 1 line
} to various python versions we support, but there are some nits to
iron out
Quoted 1 line
Quoted 1 line
} in my neck of the woods. This is a case where
scripts/qapi/isstricter
Quoted 6 lines
Quoted 6 lines
} than python/ :)
} (Not reasonable to solve for this series.)Also present in the deleted file, so no change.
} lack of
disallow_untyped_calls= False: I think this might be aremnant
Quoted 1 line
Quoted 1 line
} from when we gradually typed qapi; it's evidently no longer needed
since
Quoted 1 line
Quoted 1 line
} qapi still checks fine without this affordance. The default under
strict is
Quoted 13 lines
Quoted 13 lines
} True.
Fair enough.
Let's mention the differences in the commit message. Here's my try:
Since the previous commit, python/setup.cfg applies to
scripts/qapi/as well. Configuration files inscripts/qapi/override python/setup.cfg.
scripts/qapi/.flake8andscripts/qapi/.isort.cfgactually match python/setup.cfg exactly, and can go.Quoted 7 lines
Quoted 7 lines
The differences between
scripts/qapi/mypy.iniand python/setup.cfg are harmless: [list the differences, explain why they're harmless as long as you can keep it brief, and if not, fall back to "trust me"]. Soscripts/qapi/mypy.inican go, too.The pylint config is being left in place because the settings differ enough from the python/ directory settings that we need a chit-chat
on
Quoted 7 lines
Quoted 5 lines
how to merge them O:-)
With something like that
Reviewed-by Markus Armbruster <[email protected]>okey-dokey, let me integrate this feedback and I'll re-send, but I'm
going
Quoted 1 line
to wait until we hash everything else out - you had some questions on
other
Quoted 1 line
bits in this series.
Thanks!