Skip to contentSkip to conversation

python/qemu: strictly typed mypy conversion, pt3

PATCH In review John Snow 3 participants 10 messages Oct 9, 2020 – Oct 13, 2020

lore
Share & tools
lore

John Snow

author
#0 lore raw

cover python/qemu: strictly typed mypy conversion, pt3

This is actually quite short; it's already fully typed. Attached are two fixes for settimeout and error handling. There are actually more fixes that need to be made here, because use of readline() in non-blocking mode is actually undefined behavior, so a more thorough re-work of the error classes used by this library must be put on hold pending a more aggressive re-write.

That's a problem for later, so for now, call the initial conversion to the statically typed subset of python done so we can move on to adding the regression tests that will maintain this baseline for us.

John Snow (3):
python: add mypy config
python/qemu/qmp.py: re-raise OSError when encountered python/qemu/qmp.py: Fix settimeout operation

 python/mypy.ini    |  4 ++++
 python/qemu/qmp.py | 30 +++++++++++++++++++++---------
 2 files changed, 25 insertions(+), 9 deletions(-)
 create mode 100644 python/mypy.ini
Signature · 1 line
2.26.2
author
#1 lore raw

PATCH 1/3 python: add mypy config

Formalize the options used for checking the python library. You can run mypy from the directory that mypy.ini is in by typing mypy qemu/.

---
 python/mypy.ini | 4 ++++
 1 file changed, 4 insertions(+)
 create mode 100644 python/mypy.ini
python/mypy.ini new file +4 -0
new file mode 100644
index 00000000000..7a70eca47c6
@@ -0,0 +1,4 @@
+[mypy]
+strict = True
+python_version = 3.6
+warn_unused_configs = True
\ No newline at end of file
Signature · 1 line
2.26.2
#2 lore raw

Re: [PATCH 1/3] python: add mypy config

Quoted 16 lines · John Snow

Formalize the options used for checking the python library. You can run mypy from the directory that mypy.ini is in by typing mypy qemu/.

---
 python/mypy.ini | 4 ++++
 1 file changed, 4 insertions(+)
 create mode 100644 python/mypy.ini
python/mypy.ini new file +0 -0
new file mode 100644
index 00000000000..7a70eca47c6

python/mypy.ini line 4

Quoted 5 lines
python/mypy.ini new file +4 -0
@@ -0,0 +1,4 @@
+[mypy]
+strict = True
+python_version = 3.6
+warn_unused_configs = True
\ No newline at end of file

Missing "\n" at the end of file

Regards,
Bin

author
#3 lore raw

Re: [PATCH 1/3] python: add mypy config

Quoted 25 lines · Bin Meng
Quoted 16 lines · John Snow

Formalize the options used for checking the python library. You can run mypy from the directory that mypy.ini is in by typing mypy qemu/.

---
  python/mypy.ini | 4 ++++
  1 file changed, 4 insertions(+)

create mode 100644 python/mypy.ini

python/mypy.ini new file +0 -0
new file mode 100644
index 00000000000..7a70eca47c6

python/mypy.ini line 4

Quoted 5 lines
python/mypy.ini new file +4 -0
@@ -0,0 +1,4 @@
+[mypy]
+strict = True
+python_version = 3.6
+warn_unused_configs = True
\ No newline at end of file

Missing "\n" at the end of file

Quoted 3 lines

Regards,
Bin

Easily fixed, thanks!

author
#4 lore raw
In reply to John Snow
Quoted 16 lines · John Snow

This is actually quite short; it's already fully typed. Attached are two fixes for settimeout and error handling. There are actually more fixes that need to be made here, because use of readline() in non-blocking mode is actually undefined behavior, so a more thorough re-work of the error classes used by this library must be put on hold pending a more aggressive re-write.

That's a problem for later, so for now, call the initial conversion to the statically typed subset of python done so we can move on to adding the regression tests that will maintain this baseline for us.

John Snow (3):
python: add mypy config
python/qemu/qmp.py: re-raise OSError when encountered python/qemu/qmp.py: Fix settimeout operation

Quoted 5 lines
  python/mypy.ini    |  4 ++++
  python/qemu/qmp.py | 30 +++++++++++++++++++++---------
  2 files changed, 25 insertions(+), 9 deletions(-)

create mode 100644 python/mypy.ini

Based on: https://gitlab.com/jsnow/qemu/-/tree/python

author
#5 lore raw

PATCH 2/3 python/qemu/qmp.py: re-raise OSError when encountered

In reply to John Snow

Nested if conditions don't change when the exception block fires; we need to explicitly re-raise the error if we didn't intend to capture and suppress it.

---
 python/qemu/qmp.py | 11 ++++++-----
 1 file changed, 6 insertions(+), 5 deletions(-)
python/qemu/qmp.py +6 -5
index d911999da1f..4969e5741cb 100644
@@ -165,14 +165,15 @@ def __get_events(self, wait: Union[bool, float] = False) -> None:
         """
 
         # Check for new events regardless and pull them into the cache:
-        self.__sock.setblocking(False)
         try:
+            self.__sock.setblocking(False)
             self.__json_read()
         except OSError as err:
-            if err.errno == errno.EAGAIN:
-                # No data available
-                pass
-        self.__sock.setblocking(True)
+            # EAGAIN: No data available; not critical
+            if err.errno != errno.EAGAIN:
+                raise
+        finally:
+            self.__sock.setblocking(True)
 
         # Wait for new events, if needed.
         # if wait is 0.0, this means "no wait" and is also implicitly false.
Signature · 1 line
2.26.2
Reviewed-by on v1 2/3
#6 lore raw

Re: [PATCH 2/3] python/qemu/qmp.py: re-raise OSError when encountered

Quoted 8 lines · John Snow

Nested if conditions don't change when the exception block fires; we need to explicitly re-raise the error if we didn't intend to capture and suppress it.

---
  python/qemu/qmp.py | 11 ++++++-----
  1 file changed, 6 insertions(+), 5 deletions(-)
author
#7 lore raw

PATCH 3/3 python/qemu/qmp.py: Fix settimeout operation

In reply to John Snow

We enabled callers to interface directly with settimeout, but this reacts poorly with blocking/nonblocking operation; as they are using the same internal mechanism.

1. Whenever we change the blocking mechanism temporarily, always set it back to what it was afterwards.

2. Disallow callers from setting a timeout of "0", which means Non-blocking mode. This is going to create more weird problems than anybody wants, so just forbid it.

I opt not to coerce '0' to 'None' to maintain the principal of least surprise in mirroring the semantics of Python's interface.

---
 python/qemu/qmp.py | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)
python/qemu/qmp.py +17 -6
index 4969e5741cb..f64517fb0a7 100644
@@ -164,16 +164,19 @@ def __get_events(self, wait: Union[bool, float] = False) -> None:
                                 retrieved or if some other error occurred.
         """
 
+        # Current timeout and blocking status
+        current_timeout = self.__sock.gettimeout()
+
         # Check for new events regardless and pull them into the cache:
         try:
-            self.__sock.setblocking(False)
+            self.__sock.settimeout(0)  # i.e. setblocking(False)
             self.__json_read()
         except OSError as err:
             # EAGAIN: No data available; not critical
             if err.errno != errno.EAGAIN:
                 raise
         finally:
-            self.__sock.setblocking(True)
+            self.__sock.settimeout(current_timeout)
 
         # Wait for new events, if needed.
         # if wait is 0.0, this means "no wait" and is also implicitly false.
@@ -187,9 +190,11 @@ def __get_events(self, wait: Union[bool, float] = False) -> None:
             except Exception as err:
                 msg = "Error while reading from socket"
                 raise QMPConnectError(msg) from err
+            finally:
+                self.__sock.settimeout(current_timeout)
+
             if ret is None:
                 raise QMPConnectError("Error while reading from socket")
-            self.__sock.settimeout(None)
 
     def __enter__(self) -> 'QEMUMonitorProtocol':
         # Implement context manager enter function.
@@ -219,7 +224,7 @@ def connect(self, negotiate: bool = True) -> Optional[QMPMessage]:
             return self.__negotiate_capabilities()
         return None
 
-    def accept(self, timeout: float = 15.0) -> QMPMessage:
+    def accept(self, timeout: Optional[float] = 15.0) -> QMPMessage:
         """
         Await connection from QMP Monitor and perform capabilities negotiation.
 
@@ -338,13 +343,19 @@ def close(self) -> None:
         if self.__sockfile:
             self.__sockfile.close()
 
-    def settimeout(self, timeout: float) -> None:
+    def settimeout(self, timeout: Optional[float]) -> None:
         """
         Set the socket timeout.
 
-        @param timeout (float): timeout in seconds, or None.
+        @param timeout (float): timeout in seconds (non-zero), or None.
         @note This is a wrap around socket.settimeout
+
+        @raise ValueError: if timeout was set to 0.
         """
+        if timeout == 0:
+            msg = "timeout cannot be 0; this engages non-blocking mode."
+            msg += " Use 'None' instead to disable timeouts."
+            raise ValueError(msg)
         self.__sock.settimeout(timeout)
 
     def get_sock_fd(self) -> int:
Signature · 1 line
2.26.2
Reviewed-by on v1 3/3
#8 lore raw

Re: [PATCH 3/3] python/qemu/qmp.py: Fix settimeout operation

Quoted 11 lines · John Snow

We enabled callers to interface directly with settimeout, but this reacts poorly with blocking/nonblocking operation; as they are using the same internal mechanism.

1. Whenever we change the blocking mechanism temporarily, always set it back to what it was afterwards.

2. Disallow callers from setting a timeout of "0", which means Non-blocking mode. This is going to create more weird problems than anybody wants, so just forbid it.

Quoted 7 lines

I opt not to coerce '0' to 'None' to maintain the principal of least surprise in mirroring the semantics of Python's interface.

---
  python/qemu/qmp.py | 23 +++++++++++++++++------
  1 file changed, 17 insertions(+), 6 deletions(-)
author
#9 lore raw
In reply to John Snow
Quoted 16 lines · John Snow

This is actually quite short; it's already fully typed. Attached are two fixes for settimeout and error handling. There are actually more fixes that need to be made here, because use of readline() in non-blocking mode is actually undefined behavior, so a more thorough re-work of the error classes used by this library must be put on hold pending a more aggressive re-write.

That's a problem for later, so for now, call the initial conversion to the statically typed subset of python done so we can move on to adding the regression tests that will maintain this baseline for us.

John Snow (3):
python: add mypy config
python/qemu/qmp.py: re-raise OSError when encountered python/qemu/qmp.py: Fix settimeout operation

Quoted 5 lines
  python/mypy.ini    |  4 ++++
  python/qemu/qmp.py | 30 +++++++++++++++++++++---------
  2 files changed, 25 insertions(+), 9 deletions(-)

create mode 100644 python/mypy.ini

Thanks -- tentatively staging to my Python branch. I intend to send a PR on Friday, so if there's more feedback, there's plenty of time.

https://gitlab.com/jsnow/qemu.git python
https://gitlab.com/jsnow/qemu/-/commits/python