Skip to contentSkip to conversation

python: pylint and flake8 support

PATCH In review John Snow 2 participants 6 messages May 28, 2020 – May 30, 2020

lore
Share & tools
lore

John Snow

author
#0 lore raw

cover python: pylint and flake8 support

This is a quick series to delint the files under python/qemu, with one extra fix outside of that domain.

This was split out from my longer series attempting to package python/qemu. This part is a nice standalone chunk.

John Snow (4):
scripts/qmp: Fix shebang and imports
python/machine.py: remove bare except
python/qemu: delint and add pylintrc
python/qemu: delint; add flake8 config

 python/qemu/.flake8    |  2 ++
 python/qemu/accel.py   |  9 ++++---
 python/qemu/machine.py | 52 +++++++++++++++++++++++--------------
 python/qemu/pylintrc   | 58 ++++++++++++++++++++++++++++++++++++++++++
 python/qemu/qmp.py     |  4 +--
 python/qemu/qtest.py   | 42 +++++++++++++++++++-----------
 scripts/qmp/qmp        |  4 ++-
 scripts/qmp/qom-fuse   |  4 ++-
 scripts/qmp/qom-get    |  6 +++--
 scripts/qmp/qom-list   |  6 +++--
 scripts/qmp/qom-set    |  6 +++--
 scripts/qmp/qom-tree   |  6 +++--
 12 files changed, 150 insertions(+), 49 deletions(-)
 create mode 100644 python/qemu/.flake8
 create mode 100644 python/qemu/pylintrc
Signature · 1 line
2.21.3
author
#1 lore raw

PATCH 1/4 scripts/qmp: Fix shebang and imports

There's more wrong with these scripts; They are in various stages of disrepair. That's beyond the scope of this current patchset.

This just mechanically corrects the imports and the shebangs, as part of ensuring that the python/qemu/lib refactoring didn't break anything needlessly.

---
 scripts/qmp/qmp      | 4 +++-
 scripts/qmp/qom-fuse | 4 +++-
 scripts/qmp/qom-get  | 6 ++++--
 scripts/qmp/qom-list | 6 ++++--
 scripts/qmp/qom-set  | 6 ++++--
 scripts/qmp/qom-tree | 6 ++++--
 6 files changed, 22 insertions(+), 10 deletions(-)
scripts/qmp/qmp +3 -1
index 0625fc2abac..8e52e4a54de 100755
@@ -11,7 +11,9 @@
 # See the COPYING file in the top-level directory.
 
 import sys, os
-from qmp import QEMUMonitorProtocol
+
+sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
+from qemu.qmp import QEMUMonitorProtocol
 
 def print_response(rsp, prefix=[]):
     if type(rsp) == list:
scripts/qmp/qom-fuse +3 -1
index 6bada2c33d3..5fa6b3bf64d 100755
@@ -15,7 +15,9 @@ import fuse, stat
 from fuse import Fuse
 import os, posix
 from errno import *
-from qmp import QEMUMonitorProtocol
+
+sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
+from qemu.qmp import QEMUMonitorProtocol
 
 fuse.fuse_python_api = (0, 2)
 
scripts/qmp/qom-get +4 -2
index 007b4cd442e..666df718320 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 ##
 # QEMU Object Model test tools
 #
@@ -13,7 +13,9 @@
 
 import sys
 import os
-from qmp import QEMUMonitorProtocol
+
+sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
+from qemu.qmp import QEMUMonitorProtocol
 
 cmd, args = sys.argv[0], sys.argv[1:]
 socket_path = None
scripts/qmp/qom-list +4 -2
index 03bda3446b7..5074fd939f4 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 ##
 # QEMU Object Model test tools
 #
@@ -13,7 +13,9 @@
 
 import sys
 import os
-from qmp import QEMUMonitorProtocol
+
+sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
+from qemu.qmp import QEMUMonitorProtocol
 
 cmd, args = sys.argv[0], sys.argv[1:]
 socket_path = None
scripts/qmp/qom-set +4 -2
index c37fe78b000..240a78187f9 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 ##
 # QEMU Object Model test tools
 #
@@ -13,7 +13,9 @@
 
 import sys
 import os
-from qmp import QEMUMonitorProtocol
+
+sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
+from qemu.qmp import QEMUMonitorProtocol
 
 cmd, args = sys.argv[0], sys.argv[1:]
 socket_path = None
scripts/qmp/qom-tree +4 -2
index 1c8acf61e79..25b0781323c 100755
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 ##
 # QEMU Object Model test tools
 #
@@ -15,7 +15,9 @@
 
 import sys
 import os
-from qmp import QEMUMonitorProtocol
+
+sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..', 'python'))
+from qemu.qmp import QEMUMonitorProtocol
 
 cmd, args = sys.argv[0], sys.argv[1:]
 socket_path = None
Signature · 1 line
2.21.3
author
#2 lore raw

PATCH 2/4 python/machine.py: remove bare except

In reply to John Snow

Catch only the timeout error; if there are other problems, allow the stack trace to be visible.

---
 python/qemu/machine.py | 33 +++++++++++++++++++++------------
 1 file changed, 21 insertions(+), 12 deletions(-)
python/qemu/machine.py +21 -12
index b9a98e2c862..e3ea5235713 100644
@@ -342,7 +342,26 @@ def wait(self):
         self._load_io_log()
         self._post_shutdown()
 
-    def shutdown(self, has_quit=False):
+    def _issue_shutdown(self, has_quit: bool = False) -> None:
+        """
+        Shutdown the VM.
+        """
+        if not self.is_running():
+            return
+
+        if self._qmp is not None:
+            if not has_quit:
+                self._qmp.cmd('quit')
+            self._qmp.close()
+
+            try:
+                self._popen.wait(timeout=3)
+            except subprocess.TimeoutExpired:
+                self._popen.kill()
+
+        self._popen.wait()
+
+    def shutdown(self, has_quit: bool = False) -> None:
         """
         Terminate the VM and clean up
         """
@@ -353,17 +372,7 @@ def shutdown(self, has_quit=False):
             self._console_socket.close()
             self._console_socket = None
 
-        if self.is_running():
-            if self._qmp:
-                try:
-                    if not has_quit:
-                        self._qmp.cmd('quit')
-                    self._qmp.close()
-                    self._popen.wait(timeout=3)
-                except:
-                    self._popen.kill()
-            self._popen.wait()
-
+        self._issue_shutdown(has_quit)
         self._load_io_log()
         self._post_shutdown()
 
Signature · 1 line
2.21.3
author
#3 lore raw

PATCH 3/4 python/qemu: delint and add pylintrc

In reply to John Snow

Bring our these files up to speed with pylint 2.5.0. Add a pylintrc file to formalize which pylint subset we are targeting.

The similarity ignore is there to suppress similarity reports across imports, which for typing constants, are going to trigger this report erroneously.

---
 python/qemu/machine.py |  6 ++---
 python/qemu/pylintrc   | 58 ++++++++++++++++++++++++++++++++++++++++++
 python/qemu/qtest.py   | 42 +++++++++++++++++++-----------
 3 files changed, 88 insertions(+), 18 deletions(-)
 create mode 100644 python/qemu/pylintrc
python/qemu/machine.py +3 -3
index e3ea5235713..c79fc8fb89a 100644
@@ -58,7 +58,7 @@ def __init__(self, reply):
         self.reply = reply
 
 
-class QEMUMachine(object):
+class QEMUMachine:
     """
     A QEMU VM
 
@@ -242,7 +242,7 @@ def _base_args(self):
                          'chardev=mon,mode=control'])
         if self._machine is not None:
             args.extend(['-machine', self._machine])
-        for i in range(self._console_index):
+        for _ in range(self._console_index):
             args.extend(['-serial', 'null'])
         if self._console_set:
             self._console_address = os.path.join(self._sock_dir,
@@ -383,7 +383,7 @@ def shutdown(self, has_quit: bool = False) -> None:
                 command = ' '.join(self._qemu_full_args)
             else:
                 command = ''
-            LOG.warning(msg, -exitcode, command)
+            LOG.warning(msg, -int(exitcode), command)
 
         self._launched = False
 
python/qemu/pylintrc new file +58 -0

58 lines not on this page · raw message

python/qemu/qtest.py +27 -15

66 lines not on this page · raw message

Signature · 1 line
2.21.3
author
#4 lore raw

PATCH 4/4 python/qemu: delint; add flake8 config

In reply to John Snow

Mostly, ignore the "no bare except" rule, because flake8 is not contextual and cannot determine if we re-raise. Pylint can, though, so always prefer pylint for that.

---
 python/qemu/.flake8    |  2 ++
 python/qemu/accel.py   |  9 ++++++---
 python/qemu/machine.py | 13 +++++++++----
 python/qemu/qmp.py     |  4 ++--
 4 files changed, 19 insertions(+), 9 deletions(-)
 create mode 100644 python/qemu/.flake8
python/qemu/.flake8 new file +2 -0
new file mode 100644
index 00000000000..45d8146f3f5
@@ -0,0 +1,2 @@
+[flake8]
+extend-ignore = E722  # Pylint handles this, but smarter.
\ No newline at end of file
python/qemu/accel.py +6 -3
index 36ae85791ee..7fabe629208 100644
@@ -23,11 +23,12 @@
 # Mapping host architecture to any additional architectures it can
 # support which often includes its 32 bit cousin.
 ADDITIONAL_ARCHES = {
-    "x86_64" : "i386",
-    "aarch64" : "armhf",
-    "ppc64le" : "ppc64",
+    "x86_64": "i386",
+    "aarch64": "armhf",
+    "ppc64le": "ppc64",
 }
 
+
 def list_accel(qemu_bin):
     """
     List accelerators enabled in the QEMU binary.
@@ -47,6 +48,7 @@ def list_accel(qemu_bin):
     # Skip the first line which is the header.
     return [acc.strip() for acc in out.splitlines()[1:]]
 
+
 def kvm_available(target_arch=None, qemu_bin=None):
     """
     Check if KVM is available using the following heuristic:
@@ -69,6 +71,7 @@ def kvm_available(target_arch=None, qemu_bin=None):
         return False
     return True
 
+
 def tcg_available(qemu_bin):
     """
     Check if TCG is available.
python/qemu/machine.py +9 -4
index c79fc8fb89a..4b260fa2cb2 100644
@@ -29,6 +29,7 @@
 
 LOG = logging.getLogger(__name__)
 
+
 class QEMUMachineError(Exception):
     """
     Exception called when an error in QEMUMachine happens.
@@ -62,7 +63,8 @@ class QEMUMachine:
     """
     A QEMU VM
 
-    Use this object as a context manager to ensure the QEMU process terminates::
+    Use this object as a context manager to ensure
+    the QEMU process terminates::
 
         with VM(binary) as vm:
             ...
@@ -188,8 +190,10 @@ def send_fd_scm(self, fd=None, file_path=None):
             fd_param.append(str(fd))
 
         devnull = open(os.path.devnull, 'rb')
-        proc = subprocess.Popen(fd_param, stdin=devnull, stdout=subprocess.PIPE,
-                                stderr=subprocess.STDOUT, close_fds=False)
+        proc = subprocess.Popen(
+            fd_param, stdin=devnull, stdout=subprocess.PIPE,
+            stderr=subprocess.STDOUT, close_fds=False
+        )
         output = proc.communicate()[0]
         if output:
             LOG.debug(output)
@@ -491,7 +495,8 @@ def event_wait(self, name, timeout=60.0, match=None):
 
     def events_wait(self, events, timeout=60.0):
         """
-        events_wait waits for and returns a named event from QMP with a timeout.
+        events_wait waits for and returns a named event
+        from QMP with a timeout.
 
         events: a sequence of (name, match_criteria) tuples.
                 The match criteria are optional and may be None.
python/qemu/qmp.py +2 -2
index d6c9b2f4b12..6ae7693965a 100644
@@ -168,8 +168,8 @@ def accept(self, timeout=15.0):
 
         @param timeout: timeout in seconds (nonnegative float number, or
                         None). The value passed will set the behavior of the
-                        underneath QMP socket as described in [1]. Default value
-                        is set to 15.0.
+                        underneath QMP socket as described in [1].
+                        Default value is set to 15.0.
         @return QMP greeting dict
         @raise OSError on socket connection errors
         @raise QMPConnectError if the greeting is not received
Signature · 1 line
2.21.3
#5 lore raw
In reply to John Snow
Quoted 12 lines · John Snow

This is a quick series to delint the files under python/qemu, with one extra fix outside of that domain.

This was split out from my longer series attempting to package python/qemu. This part is a nice standalone chunk.

John Snow (4):
scripts/qmp: Fix shebang and imports
python/machine.py: remove bare except
python/qemu: delint and add pylintrc
python/qemu: delint; add flake8 config

Quoted 16 lines
 python/qemu/.flake8    |  2 ++
 python/qemu/accel.py   |  9 ++++---
 python/qemu/machine.py | 52 +++++++++++++++++++++++--------------
 python/qemu/pylintrc   | 58 ++++++++++++++++++++++++++++++++++++++++++
 python/qemu/qmp.py     |  4 +--
 python/qemu/qtest.py   | 42 +++++++++++++++++++-----------
 scripts/qmp/qmp        |  4 ++-
 scripts/qmp/qom-fuse   |  4 ++-
 scripts/qmp/qom-get    |  6 +++--
 scripts/qmp/qom-list   |  6 +++--
 scripts/qmp/qom-set    |  6 +++--
 scripts/qmp/qom-tree   |  6 +++--
 12 files changed, 150 insertions(+), 49 deletions(-)
 create mode 100644 python/qemu/.flake8
 create mode 100644 python/qemu/pylintrc

Thanks, patches 1/3/4 applied to my python-next tree:
https://gitlab.com/philmd/qemu/commits/python-next

I skipped patch #2 which doesn't apply on top on Vladimir's "hard-kill" patch:
https://www.mail-archive.com/[email protected]/msg680200.html