Skip to contentSkip to conversation

iotests: Fix pylint/mypy warnings on F33

PATCH In review Kevin Wolf 3 participants 8 messages Oct 27, 2020 – Oct 28, 2020

lore
Share & tools
lore

Kevin Wolf

author
#0 lore raw

cover iotests: Fix pylint/mypy warnings on F33

This series makes iotest 297 (which runs pylint and mypy on iotests.py) pass again on Fedora 33.

Kevin Wolf (3):
iotests.py: Fix type check errors in wait_migration()
iotests: Disable unsubscriptable-object in pylint
iotests: Use Python 3 style super()

 tests/qemu-iotests/iotests.py | 12 ++++++++----
 tests/qemu-iotests/pylintrc   |  2 ++
 2 files changed, 10 insertions(+), 4 deletions(-)
Signature · 1 line
2.28.0
author
#1 lore raw

PATCH 1/3 iotests.py: Fix type check errors in wait_migration()

Commit 1847a4a8c20 clarified that event_wait() can return None (though only with timeout=0) and commit f12a282ff47 annotated it as returning Optional[QMPMessage].

Type checks in wait_migration() fail because of the unexpected optional return type:

iotests.py:750: error: Value of type variable "Msg" of "log" cannot be "Optional[Dict[str, Any]]" iotests.py:751: error: Value of type "Optional[Dict[str, Any]]" is not indexable iotests.py:754: error: Value of type "Optional[Dict[str, Any]]" is not indexable

Fortunately, the non-zero default timeout is used in the event_wait() call, so we can make mypy happy by just asserting this.

---
 tests/qemu-iotests/iotests.py | 4 ++++
 1 file changed, 4 insertions(+)
tests/qemu-iotests/iotests.py +4 -0
index 63d2ace93c..28388a0fbc 100644
@@ -747,6 +747,10 @@ class VM(qtest.QEMUQtestMachine):
     def wait_migration(self, expect_runstate: Optional[str]) -> bool:
         while True:
             event = self.event_wait('MIGRATION')
+            # We use the default timeout, and with a timeout, event_wait()
+            # never returns None
+            assert event
+
             log(event, filters=[filter_qmp_event])
             if event['data']['status'] in ('completed', 'failed'):
                 break
Signature · 1 line
2.28.0
Reviewed-by on v1 1/3
#2 lore raw

Re: [PATCH 1/3] iotests.py: Fix type check errors in wait_migration()

Quoted 7 lines · Kevin Wolf

Commit 1847a4a8c20 clarified that event_wait() can return None (though only with timeout=0) and commit f12a282ff47 annotated it as returning Optional[QMPMessage].

Type checks in wait_migration() fail because of the unexpected optional return type:

Quoted 8 lines

iotests.py:750: error: Value of type variable "Msg" of "log" cannot be "Optional[Dict[str, Any]]" iotests.py:751: error: Value of type "Optional[Dict[str, Any]]" is not indexable iotests.py:754: error: Value of type "Optional[Dict[str, Any]]" is not indexable

Fortunately, the non-zero default timeout is used in the event_wait() call, so we can make mypy happy by just asserting this.

Quoted 10 lines
---
  tests/qemu-iotests/iotests.py | 4 ++++
  1 file changed, 4 insertions(+)
tests/qemu-iotests/iotests.py +0 -0
index 63d2ace93c..28388a0fbc 100644

tests/qemu-iotests/iotests.py class VM(qtest.QEMUQtestMachine): · line 756

Quoted 10 lines
tests/qemu-iotests/iotests.py +4 -0
@@ -747,6 +747,10 @@ class VM(qtest.QEMUQtestMachine):
      def wait_migration(self, expect_runstate: Optional[str]) -> bool:
          while True:
              event = self.event_wait('MIGRATION')
+            # We use the default timeout, and with a timeout, event_wait()
+            # never returns None
+            assert event
+
              log(event, filters=[filter_qmp_event])
              if event['data']['status'] in ('completed', 'failed'):
                  break

I tried, briefly, to see if I could overload the function to mypy to make it Do The Right Thing, but I don't think mypy supports overloading on float literals, or not well.

I tried to do this:

@overload
def events_wait(self, events: Sequence[Tuple[str, Any]]) -> QMPMessage: ...

@overload
def events_wait(self, events: Sequence[Tuple[str, Any]],

                timeout: Literal[0]) -> Optional[QMPMessage]: ...

@overload
def events_wait(self, events: Sequence[Tuple[str, Any]],

                timeout: float = 60.0) -> QMPMessage: ...

but ultimately mypy doesn't like this:

qemu/machine/machine.py:655: error: Overloaded function implementation does not accept all possible arguments of signature 2 Found 1 error in 1 file (checked 7 source files)

Trying literal 0.0 works even less well, because the Literal system does not appear to like floats at all. Hmph.

... So much for trying to be clever about this, I guess.

(The event system is pretty wonky anyway; especially type-wise. I think it's in need of an overhaul, so I'll put it on the list of things to investigate at some point. There might be something nice that can be done with asyncio and events that might make more sense type-wise. Problems for later.)

author
#3 lore raw

PATCH 2/3 iotests: Disable unsubscriptable-object in pylint

In reply to Kevin Wolf

When run with Python 3.9, pylint incorrectly warns about things like Optional[foo] because it doesn't recognise Optional as unsubscriptable. This is a known pylint bug:

    https://github.com/PyCQA/pylint/issues/3882

Just disable this check to get rid of the warnings.

Disabling this shouldn't make us miss any real bug because mypy also has a similar check ("... is not indexable").

---
 tests/qemu-iotests/pylintrc | 2 ++
 1 file changed, 2 insertions(+)
tests/qemu-iotests/pylintrc +2 -0
index 5481afe528..cd3702e23c 100644
@@ -17,6 +17,8 @@ disable=invalid-name,
         too-many-lines,
         too-many-locals,
         too-many-public-methods,
+        # pylint warns about Optional[] etc. as unsubscriptable in 3.9
+        unsubscriptable-object,
         # These are temporary, and should be removed:
         missing-docstring,
 
Signature · 1 line
2.28.0
Reviewed-by on v1 2/3
#4 lore raw

Re: [PATCH 2/3] iotests: Disable unsubscriptable-object in pylint

Quoted 22 lines · Kevin Wolf

When run with Python 3.9, pylint incorrectly warns about things like Optional[foo] because it doesn't recognise Optional as unsubscriptable. This is a known pylint bug:

     https://github.com/PyCQA/pylint/issues/3882

Just disable this check to get rid of the warnings.

Disabling this shouldn't make us miss any real bug because mypy also has a similar check ("... is not indexable").

---
  tests/qemu-iotests/pylintrc | 2 ++
  1 file changed, 2 insertions(+)
tests/qemu-iotests/pylintrc +0 -0
index 5481afe528..cd3702e23c 100644

tests/qemu-iotests/pylintrc disable=invalid-name, · line 24

Quoted 8 lines
tests/qemu-iotests/pylintrc +2 -0
@@ -17,6 +17,8 @@ disable=invalid-name,
          too-many-lines,
          too-many-locals,
          too-many-public-methods,
+        # pylint warns about Optional[] etc. as unsubscriptable in 3.9
+        unsubscriptable-object,
          # These are temporary, and should be removed:
          missing-docstring,
 
author
#5 lore raw

PATCH 3/3 iotests: Use Python 3 style super()

In reply to Kevin Wolf

pylint complains about the use of super with the current class and instance as arguments in VM.__init__():

iotests.py:546:8: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)

No reason not to follow the advice and make it happy, so let's do this.

---
 tests/qemu-iotests/iotests.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
tests/qemu-iotests/iotests.py +4 -4
index 28388a0fbc..814804a4c6 100644
@@ -543,10 +543,10 @@ class VM(qtest.QEMUQtestMachine):
 
     def __init__(self, path_suffix=''):
         name = "qemu%s-%d" % (path_suffix, os.getpid())
-        super(VM, self).__init__(qemu_prog, qemu_opts, name=name,
-                                 test_dir=test_dir,
-                                 socket_scm_helper=socket_scm_helper,
-                                 sock_dir=sock_dir)
+        super().__init__(qemu_prog, qemu_opts, name=name,
+                         test_dir=test_dir,
+                         socket_scm_helper=socket_scm_helper,
+                         sock_dir=sock_dir)
         self._num_drives = 0
 
     def add_object(self, opts):
Signature · 1 line
2.28.0
Reviewed-by on v1 3/3
#6 lore raw

Re: [PATCH 3/3] iotests: Use Python 3 style super()

Quoted 22 lines · Kevin Wolf

pylint complains about the use of super with the current class and instance as arguments in VM.__init__():

iotests.py:546:8: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)

No reason not to follow the advice and make it happy, so let's do this.

---
  tests/qemu-iotests/iotests.py | 8 ++++----
  1 file changed, 4 insertions(+), 4 deletions(-)
tests/qemu-iotests/iotests.py +0 -1
index 28388a0fbc..814804a4c6 100644
@@ -543,10 +543,10 @@ class VM(qtest.QEMUQtestMachine):
 
      def __init__(self, path_suffix=''):
          name = "qemu%s-%d" % (path_suffix, os.getpid())
-        super(VM, self).__init__(qemu_prog, qemu_opts, name=name,

tests/qemu-iotests/iotests.py class VM(qtest.QEMUQtestMachine): · line 552

Quoted 10 lines
tests/qemu-iotests/iotests.py +4 -3
-                                 test_dir=test_dir,
-                                 socket_scm_helper=socket_scm_helper,
-                                 sock_dir=sock_dir)
+        super().__init__(qemu_prog, qemu_opts, name=name,
+                         test_dir=test_dir,
+                         socket_scm_helper=socket_scm_helper,
+                         sock_dir=sock_dir)
          self._num_drives = 0
 
      def add_object(self, opts):
Reviewed-by on v1 3/3
#7 lore raw

Re: [PATCH 3/3] iotests: Use Python 3 style super()

In reply to Kevin Wolf
Quoted 11 lines · Kevin Wolf

pylint complains about the use of super with the current class and instance as arguments in VM.__init__():

iotests.py:546:8: R1725: Consider using Python 3 style super() without arguments (super-with-arguments)

No reason not to follow the advice and make it happy, so let's do this.

---
 tests/qemu-iotests/iotests.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)