Skip to contentSkip to conversation

python: fix for mypy 0.930 and pylint dependencies

PATCH In review John Snow 2 participants 6 messages Jan 10, 2022

lore
Share & tools
lore

John Snow

author
#0 lore raw

cover python: fix for mypy 0.930 and pylint dependencies

GitLab: https://gitlab.com/jsnow/qemu/-/commits/python-ci-fix-2022-jan
CI: https://gitlab.com/jsnow/qemu/-/pipelines/445011098

Dependencies in the Python ecosystem were updated and caused CI failures immediately after leaving for PTO. This series fixes those breakages, thank you for your patience.

This patchset fixes the 'check-python-tox' job that was previously configured as 'allowed to fail'. I'm not re-enabling it to always run at present, maybe leaving it as an optional job is fine for now. The purpose of the job is primarily to find new breakages on cutting edge deployments of Python, which means that it's fairly likely to chirp as regressions are discovered.

Generally, I like seeing those regressions so I can keep on top of fixing them -- however, in practice, it seems as if what this really means is that every last time I go on PTO, they break for several weeks in an amazing coincidence.

Oh well.

When submitting Python changes in the future, please just check that this CI test ran to ensure that it works well on all supported Python interpreter versions.

John Snow (3):
python/aqmp: use absolute import statement Python/aqmp: fix type definitions for mypy 0.920
python: update type hints for mypy 0.930

 python/qemu/aqmp/aqmp_tui.py  | 3 ++-
 python/qemu/aqmp/protocol.py  | 5 +++--
 python/qemu/qmp/qom_common.py | 6 +-----
 3 files changed, 6 insertions(+), 8 deletions(-)
Signature · 1 line
2.31.1
author
#1 lore raw

PATCH 1/3 python/aqmp: use absolute import statement

pylint's dependency astroid appears to have bugs in 2.9.1 and 2.9.2 (Dec 31 and Jan 3) that appear to erroneously expect the qemu namespace to have an __init__.py file. astroid 2.9.3 (Jan 9) avoids that problem, but appears to not understand a relative import within a namespace package.

Update the relative import - it was worth changing anyway, because these packages will eventually be packaged and distributed separately.

---
 python/qemu/aqmp/aqmp_tui.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
python/qemu/aqmp/aqmp_tui.py +2 -1
index a2929f771c..f1e926dd75 100644
@@ -35,7 +35,8 @@
 import urwid
 import urwid_readline
 
-from ..qmp import QEMUMonitorProtocol, QMPBadPortError
+from qemu.qmp import QEMUMonitorProtocol, QMPBadPortError
+
 from .error import ProtocolError
 from .message import DeserializationError, Message, UnexpectedTypeError
 from .protocol import ConnectError, Runstate
Signature · 1 line
2.31.1
Reviewed-by on v1 1/3
#2 lore raw

Re: [PATCH 1/3] python/aqmp: use absolute import statement

Quoted 31 lines · John Snow

pylint's dependency astroid appears to have bugs in 2.9.1 and 2.9.2 (Dec 31 and Jan 3) that appear to erroneously expect the qemu namespace to have an __init__.py file. astroid 2.9.3 (Jan 9) avoids that problem, but appears to not understand a relative import within a namespace package.

Update the relative import - it was worth changing anyway, because these packages will eventually be packaged and distributed separately.

---
 python/qemu/aqmp/aqmp_tui.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
python/qemu/aqmp/aqmp_tui.py +2 -1
index a2929f771c..f1e926dd75 100644
@@ -35,7 +35,8 @@
 import urwid
 import urwid_readline
 
-from ..qmp import QEMUMonitorProtocol, QMPBadPortError
+from qemu.qmp import QEMUMonitorProtocol, QMPBadPortError
+
 from .error import ProtocolError
 from .message import DeserializationError, Message, UnexpectedTypeError
 from .protocol import ConnectError, Runstate
Signature · 1 line
2.31.1

--
Beraldo

author
#3 lore raw

PATCH 2/3 Python/aqmp: fix type definitions for mypy 0.920

In reply to John Snow

0.920 (Released 2021-12-15) is not entirely happy with the way that I was defining _FutureT:

qemu/aqmp/protocol.py:601: error: Item "object" of the upper bound "Optional[Future[Any]]" of type variable "_FutureT" has no attribute "done"

Update it with something a little mechanically simpler that works better across a wider array of mypy versions.

---
 python/qemu/aqmp/protocol.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
python/qemu/aqmp/protocol.py +3 -2
index 5190b33b13..c4fbe35a0e 100644
@@ -43,8 +43,8 @@
 
 
 T = TypeVar('T')
+_U = TypeVar('_U')
 _TaskFN = Callable[[], Awaitable[None]]  # aka ``async def func() -> None``
-_FutureT = TypeVar('_FutureT', bound=Optional['asyncio.Future[Any]'])
 
 
 class Runstate(Enum):
@@ -591,7 +591,8 @@ def _cleanup(self) -> None:
         """
         Fully reset this object to a clean state and return to `IDLE`.
         """
-        def _paranoid_task_erase(task: _FutureT) -> Optional[_FutureT]:
+        def _paranoid_task_erase(task: Optional['asyncio.Future[_U]']
+                                 ) -> Optional['asyncio.Future[_U]']:
             # Help to erase a task, ENSURING it is fully quiesced first.
             assert (task is None) or task.done()
             return None if (task and task.done()) else task
Signature · 1 line
2.31.1
author
#4 lore raw

PATCH 3/3 python: update type hints for mypy 0.930

In reply to John Snow

Mypy 0.930, released Dec 22, changes the way argparse objects are considered. Crafting a definition that works under Python 3.6 and an older mypy alongside newer versions simultaneously is ... difficult, so... eh. Stub it out with an 'Any' definition to get the CI moving again.

Oh well.

---
 python/qemu/qmp/qom_common.py | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)
python/qemu/qmp/qom_common.py +1 -5
index a59ae1a2a1..2e4c741f77 100644
@@ -30,10 +30,6 @@
 from . import QEMUMonitorProtocol, QMPError
 
 
-# The following is needed only for a type alias.
-Subparsers = argparse._SubParsersAction  # pylint: disable=protected-access
-
-
 class ObjectPropertyInfo:
     """
     Represents the return type from e.g. qom-list.
@@ -89,7 +85,7 @@ def __init__(self, args: argparse.Namespace):
         self.qmp.connect()
 
     @classmethod
-    def register(cls, subparsers: Subparsers) -> None:
+    def register(cls, subparsers: Any) -> None:
         """
         Register this command with the argument parser.
 
Signature · 1 line
2.31.1
Reviewed-by on v1 3/3
#5 lore raw

Re: [PATCH 3/3] python: update type hints for mypy 0.930

Quoted 41 lines · John Snow

Mypy 0.930, released Dec 22, changes the way argparse objects are considered. Crafting a definition that works under Python 3.6 and an older mypy alongside newer versions simultaneously is ... difficult, so... eh. Stub it out with an 'Any' definition to get the CI moving again.

Oh well.

---
 python/qemu/qmp/qom_common.py | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)
python/qemu/qmp/qom_common.py +1 -5
index a59ae1a2a1..2e4c741f77 100644
@@ -30,10 +30,6 @@
 from . import QEMUMonitorProtocol, QMPError
 
 
-# The following is needed only for a type alias.
-Subparsers = argparse._SubParsersAction  # pylint: disable=protected-access
-
-
 class ObjectPropertyInfo:
     """
     Represents the return type from e.g. qom-list.
@@ -89,7 +85,7 @@ def __init__(self, args: argparse.Namespace):
         self.qmp.connect()
 
     @classmethod
-    def register(cls, subparsers: Subparsers) -> None:
+    def register(cls, subparsers: Any) -> None:
         """
         Register this command with the argument parser.
 
Signature · 1 line
2.31.1

--
Beraldo