Skip to contentSkip to conversation

qapi: Bye-bye Python 2

PATCH In review Markus Armbruster 2 participants 11 messages Mar 4, 2020 – Mar 5, 2020

lore
Share & tools
lore
#0 lore raw

v2 cover qapi: Bye-bye Python 2

v2:
* PATCH 3: Don't mess up indentation
* PATCH 4: Additionally clean up a pycodestyle-3 "E713 test for membership should be 'not in'"

Markus Armbruster (4):
qapi: Inheriting from object is pointless with Python 3, drop
qapi: Drop conditionals for Python 2
qapi: Use super() now we have Python 3
qapi: Brush off some (py)lint

 scripts/qapi/commands.py       |  6 +--
 scripts/qapi/common.py         |  6 +--
 scripts/qapi/error.py          |  4 +-
 scripts/qapi/events.py         |  4 +-
 scripts/qapi/expr.py           |  3 +-
 scripts/qapi/gen.py            | 27 ++++++-------
 scripts/qapi/introspect.py     |  6 +--
 scripts/qapi/parser.py         | 20 ++++-----
 scripts/qapi/schema.py         | 74 +++++++++++++++++-----------------
 scripts/qapi/source.py         |  4 +-
 scripts/qapi/types.py          |  4 +-
 scripts/qapi/visit.py          |  4 +-
 tests/qapi-schema/test-qapi.py |  6 +--
 13 files changed, 74 insertions(+), 94 deletions(-)
Signature · 1 line
2.21.1
#1 lore raw

PATCH v2 1/4 qapi: Inheriting from object is pointless with Python 3, drop

---
 scripts/qapi/gen.py    |  2 +-
 scripts/qapi/parser.py |  6 +++---
 scripts/qapi/schema.py | 12 ++++++------
 scripts/qapi/source.py |  4 ++--
 4 files changed, 12 insertions(+), 12 deletions(-)
scripts/qapi/gen.py +1 -1
index 95afae0615..a53a705c73 100644
@@ -22,7 +22,7 @@ from qapi.common import *
 from qapi.schema import QAPISchemaVisitor
 
 
-class QAPIGen(object):
+class QAPIGen:
 
     def __init__(self, fname):
         self.fname = fname
scripts/qapi/parser.py +3 -3
index 342792e410..2e3a3c5d76 100644
@@ -23,7 +23,7 @@ from qapi.error import QAPIParseError, QAPISemError
 from qapi.source import QAPISourceInfo
 
 
-class QAPISchemaParser(object):
+class QAPISchemaParser:
 
     def __init__(self, fname, previously_included=None, incl_info=None):
         previously_included = previously_included or set()
@@ -293,7 +293,7 @@ class QAPISchemaParser(object):
         raise QAPIParseError(self, "documentation comment must end with '##'")
 
 
-class QAPIDoc(object):
+class QAPIDoc:
     """
     A documentation comment block, either definition or free-form
 
@@ -312,7 +312,7 @@ class QAPIDoc(object):
     Free-form documentation blocks consist only of a body section.
     """
 
-    class Section(object):
+    class Section:
         def __init__(self, name=None):
             # optional section name (argument/member or section name)
             self.name = name
scripts/qapi/schema.py +6 -6
index 5100110fa2..c8bcfe2c49 100644
@@ -24,7 +24,7 @@ from qapi.expr import check_exprs
 from qapi.parser import QAPISchemaParser
 
 
-class QAPISchemaEntity(object):
+class QAPISchemaEntity:
     meta = None
 
     def __init__(self, name, info, doc, ifcond=None, features=None):
@@ -89,7 +89,7 @@ class QAPISchemaEntity(object):
         return "%s '%s'" % (self.meta, self.name)
 
 
-class QAPISchemaVisitor(object):
+class QAPISchemaVisitor:
     def visit_begin(self, schema):
         pass
 
@@ -135,7 +135,7 @@ class QAPISchemaVisitor(object):
         pass
 
 
-class QAPISchemaModule(object):
+class QAPISchemaModule:
     def __init__(self, name):
         self.name = name
         self._entity_list = []
@@ -441,7 +441,7 @@ class QAPISchemaObjectType(QAPISchemaType):
                                        self.features)
 
 
-class QAPISchemaMember(object):
+class QAPISchemaMember:
     """ Represents object members, enum members and features """
     role = 'member'
 
@@ -519,7 +519,7 @@ class QAPISchemaObjectTypeMember(QAPISchemaMember):
                                         self.describe)
 
 
-class QAPISchemaObjectTypeVariants(object):
+class QAPISchemaObjectTypeVariants:
     def __init__(self, tag_name, info, tag_member, variants):
         # Flat unions pass tag_name but not tag_member.
         # Simple unions and alternates pass tag_member but not tag_name.
@@ -787,7 +787,7 @@ class QAPISchemaEvent(QAPISchemaEntity):
                             self.arg_type, self.boxed)
 
 
-class QAPISchema(object):
+class QAPISchema:
     def __init__(self, fname):
         self.fname = fname
         parser = QAPISchemaParser(fname)
scripts/qapi/source.py +2 -2
index 8956885033..e97b9a8e15 100644
@@ -13,7 +13,7 @@ import copy
 import sys
 
 
-class QAPISchemaPragma(object):
+class QAPISchemaPragma:
     def __init__(self):
         # Are documentation comments required?
         self.doc_required = False
@@ -23,7 +23,7 @@ class QAPISchemaPragma(object):
         self.name_case_whitelist = []
 
 
-class QAPISourceInfo(object):
+class QAPISourceInfo:
     def __init__(self, fname, line, parent):
         self.fname = fname
         self.line = line
Signature · 1 line
2.21.1
#3 lore raw

PATCH v2 2/4 qapi: Drop conditionals for Python 2

In reply to Markus Armbruster
---
 scripts/qapi/common.py         | 6 +-----
 scripts/qapi/gen.py            | 6 +-----
 scripts/qapi/parser.py         | 6 +-----
 tests/qapi-schema/test-qapi.py | 6 +-----
 4 files changed, 4 insertions(+), 20 deletions(-)
scripts/qapi/common.py +1 -5
index e00dcafce7..ba35abea47 100644
@@ -12,7 +12,6 @@
 # See the COPYING file in the top-level directory.
 
 import re
-import string
 
 
 # ENUMName -> ENUM_NAME, EnumName1 -> ENUM_NAME1
@@ -43,10 +42,7 @@ def c_enum_const(type_name, const_name, prefix=None):
     return camel_to_upper(type_name) + '_' + c_name(const_name, False).upper()
 
 
-if hasattr(str, 'maketrans'):
-    c_name_trans = str.maketrans('.-', '__')
-else:
-    c_name_trans = string.maketrans('.-', '__')
+c_name_trans = str.maketrans('.-', '__')
 
 
 # Map @name to a valid C identifier.
scripts/qapi/gen.py +1 -5
index a53a705c73..317cd72601 100644
@@ -15,7 +15,6 @@
 import errno
 import os
 import re
-import sys
 from contextlib import contextmanager
 
 from qapi.common import *
@@ -54,10 +53,7 @@ class QAPIGen:
                 if e.errno != errno.EEXIST:
                     raise
         fd = os.open(pathname, os.O_RDWR | os.O_CREAT, 0o666)
-        if sys.version_info[0] >= 3:
-            f = open(fd, 'r+', encoding='utf-8')
-        else:
-            f = os.fdopen(fd, 'r+')
+        f = open(fd, 'r+', encoding='utf-8')
         text = self.get_content()
         oldtext = f.read(len(text) + 1)
         if text != oldtext:
scripts/qapi/parser.py +1 -5
index 2e3a3c5d76..cf14e5426c 100644
@@ -16,7 +16,6 @@
 
 import os
 import re
-import sys
 from collections import OrderedDict
 
 from qapi.error import QAPIParseError, QAPISemError
@@ -30,10 +29,7 @@ class QAPISchemaParser:
         previously_included.add(os.path.abspath(fname))
 
         try:
-            if sys.version_info[0] >= 3:
-                fp = open(fname, 'r', encoding='utf-8')
-            else:
-                fp = open(fname, 'r')
+            fp = open(fname, 'r', encoding='utf-8')
             self.src = fp.read()
         except IOError as e:
             raise QAPISemError(incl_info or QAPISourceInfo(None, None, None),
tests/qapi-schema/test-qapi.py +1 -5
index 41232c11a3..bee18ee344 100755
@@ -16,15 +16,11 @@ import argparse
 import difflib
 import os
 import sys
+from io import StringIO
 
 from qapi.error import QAPIError
 from qapi.schema import QAPISchema, QAPISchemaVisitor
 
-if sys.version_info[0] < 3:
-    from cStringIO import StringIO
-else:
-    from io import StringIO
-
 
 class QAPISchemaTestVisitor(QAPISchemaVisitor):
 
Signature · 1 line
2.21.1
Reviewed-by on v2 2/4
#4 lore raw

Re: [PATCH v2 2/4] qapi: Drop conditionals for Python 2

Quoted 9 lines · Markus Armbruster
---
 scripts/qapi/common.py         | 6 +-----
 scripts/qapi/gen.py            | 6 +-----
 scripts/qapi/parser.py         | 6 +-----
 tests/qapi-schema/test-qapi.py | 6 +-----
 4 files changed, 4 insertions(+), 20 deletions(-)

Seeyanara.

#5 lore raw

PATCH v2 3/4 qapi: Use super() now we have Python 3

In reply to Markus Armbruster
---
 scripts/qapi/commands.py   |  4 +--
 scripts/qapi/error.py      |  4 +--
 scripts/qapi/events.py     |  4 +--
 scripts/qapi/gen.py        | 10 ++++----
 scripts/qapi/introspect.py |  4 +--
 scripts/qapi/parser.py     |  2 +-
 scripts/qapi/schema.py     | 51 +++++++++++++++++++-------------------
 scripts/qapi/types.py      |  4 +--
 scripts/qapi/visit.py      |  4 +--
 9 files changed, 43 insertions(+), 44 deletions(-)
scripts/qapi/commands.py +2 -2
index afa55b055c..8bb6316061 100644
@@ -237,8 +237,8 @@ void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds)
 class QAPISchemaGenCommandVisitor(QAPISchemaModularCVisitor):
 
     def __init__(self, prefix):
-        QAPISchemaModularCVisitor.__init__(
-            self, prefix, 'qapi-commands',
+        super().__init__(
+            prefix, 'qapi-commands',
             ' * Schema-defined QAPI/QMP commands', None, __doc__)
         self._regy = QAPIGenCCode(None)
         self._visited_ret_types = {}
scripts/qapi/error.py +2 -2
index b9f3751bea..ae60d9e2fe 100644
@@ -35,9 +35,9 @@ class QAPIParseError(QAPIError):
                 col = (col + 7) % 8 + 1
             else:
                 col += 1
-        QAPIError.__init__(self, parser.info, col, msg)
+        super().__init__(parser.info, col, msg)
 
 
 class QAPISemError(QAPIError):
     def __init__(self, info, msg):
-        QAPIError.__init__(self, info, None, msg)
+        super().__init__(info, None, msg)
scripts/qapi/events.py +2 -2
index 2bde3e6128..a98b9f5099 100644
@@ -138,8 +138,8 @@ def gen_event_send(name, arg_type, boxed, event_enum_name, event_emit):
 class QAPISchemaGenEventVisitor(QAPISchemaModularCVisitor):
 
     def __init__(self, prefix):
-        QAPISchemaModularCVisitor.__init__(
-            self, prefix, 'qapi-events',
+        super().__init__(
+            prefix, 'qapi-events',
             ' * Schema-defined QAPI/QMP events', None, __doc__)
         self._event_enum_name = c_name(prefix + 'QAPIEvent', protect=False)
         self._event_enum_members = []
scripts/qapi/gen.py +5 -5
index 317cd72601..e17354392b 100644
@@ -82,7 +82,7 @@ def _wrap_ifcond(ifcond, before, after):
 class QAPIGenCCode(QAPIGen):
 
     def __init__(self, fname):
-        QAPIGen.__init__(self, fname)
+        super().__init__(fname)
         self._start_if = None
 
     def start_if(self, ifcond):
@@ -102,13 +102,13 @@ class QAPIGenCCode(QAPIGen):
 
     def get_content(self):
         assert self._start_if is None
-        return QAPIGen.get_content(self)
+        return super().get_content()
 
 
 class QAPIGenC(QAPIGenCCode):
 
     def __init__(self, fname, blurb, pydoc):
-        QAPIGenCCode.__init__(self, fname)
+        super().__init__(fname)
         self._blurb = blurb
         self._copyright = '\n * '.join(re.findall(r'^Copyright .*', pydoc,
                                                   re.MULTILINE))
@@ -141,7 +141,7 @@ char qapi_dummy_%(name)s;
 class QAPIGenH(QAPIGenC):
 
     def _top(self):
-        return QAPIGenC._top(self) + guardstart(self.fname)
+        return super()._top() + guardstart(self.fname)
 
     def _bottom(self):
         return guardend(self.fname)
@@ -176,7 +176,7 @@ def ifcontext(ifcond, *args):
 class QAPIGenDoc(QAPIGen):
 
     def _top(self):
-        return (QAPIGen._top(self)
+        return (super()._top()
                 + '@c AUTOMATICALLY GENERATED, DO NOT MODIFY\n\n')
 
 
scripts/qapi/introspect.py +2 -2
index b3a463dd8b..0cc655fd9f 100644
@@ -76,8 +76,8 @@ def to_c_string(string):
 class QAPISchemaGenIntrospectVisitor(QAPISchemaMonolithicCVisitor):
 
     def __init__(self, prefix, unmask):
-        QAPISchemaMonolithicCVisitor.__init__(
-            self, prefix, 'qapi-introspect',
+        super().__init__(
+            prefix, 'qapi-introspect',
             ' * QAPI/QMP schema introspection', __doc__)
         self._unmask = unmask
         self._schema = None
scripts/qapi/parser.py +1 -1
index cf14e5426c..340f7c4633 100644
@@ -320,7 +320,7 @@ class QAPIDoc:
 
     class ArgSection(Section):
         def __init__(self, name):
-            QAPIDoc.Section.__init__(self, name)
+            super().__init__(name)
             self.member = None
 
         def connect(self, member):
scripts/qapi/schema.py +25 -26

197 lines not on this page · raw message

scripts/qapi/types.py +2 -2
index 99dcaf7074..3c83b6e4be 100644
@@ -241,8 +241,8 @@ void qapi_free_%(c_name)s(%(c_name)s *obj)
 class QAPISchemaGenTypeVisitor(QAPISchemaModularCVisitor):
 
     def __init__(self, prefix):
-        QAPISchemaModularCVisitor.__init__(
-            self, prefix, 'qapi-types', ' * Schema-defined QAPI types',
+        super().__init__(
+            prefix, 'qapi-types', ' * Schema-defined QAPI types',
             ' * Built-in QAPI types', __doc__)
 
     def _begin_system_module(self, name):
scripts/qapi/visit.py +2 -2
index 4efce62b0c..421e5bd8cd 100644
@@ -283,8 +283,8 @@ out:
 class QAPISchemaGenVisitVisitor(QAPISchemaModularCVisitor):
 
     def __init__(self, prefix):
-        QAPISchemaModularCVisitor.__init__(
-            self, prefix, 'qapi-visit', ' * Schema-defined QAPI visitors',
+        super().__init__(
+            prefix, 'qapi-visit', ' * Schema-defined QAPI visitors',
             ' * Built-in QAPI visitors', __doc__)
 
     def _begin_system_module(self, name):
Signature · 1 line
2.21.1
Reviewed-by on v2 3/4
#6 lore raw

Re: [PATCH v2 3/4] qapi: Use super() now we have Python 3

Quoted 13 lines · Markus Armbruster
---
 scripts/qapi/commands.py   |  4 +--
 scripts/qapi/error.py      |  4 +--
 scripts/qapi/events.py     |  4 +--
 scripts/qapi/gen.py        | 10 ++++----
 scripts/qapi/introspect.py |  4 +--
 scripts/qapi/parser.py     |  2 +-
 scripts/qapi/schema.py     | 51 +++++++++++++++++++-------------------
 scripts/qapi/types.py      |  4 +--
 scripts/qapi/visit.py      |  4 +--
 9 files changed, 43 insertions(+), 44 deletions(-)
#7 lore raw

PATCH v2 4/4 qapi: Brush off some (py)lint

In reply to Markus Armbruster
---
 scripts/qapi/commands.py   |  2 +-
 scripts/qapi/expr.py       |  3 +--
 scripts/qapi/gen.py        |  9 ++++++---
 scripts/qapi/introspect.py |  2 --
 scripts/qapi/parser.py     |  6 ++----
 scripts/qapi/schema.py     | 11 +++++------
 6 files changed, 15 insertions(+), 18 deletions(-)
scripts/qapi/commands.py +1 -1
index 8bb6316061..0e13e82989 100644
@@ -274,7 +274,7 @@ class QAPISchemaGenCommandVisitor(QAPISchemaModularCVisitor):
 
 void %(c_prefix)sqmp_init_marshal(QmpCommandList *cmds);
 ''',
-                       c_prefix=c_name(self._prefix, protect=False)))
+                             c_prefix=c_name(self._prefix, protect=False)))
         self._genc.preamble_add(mcgen('''
 #include "qemu/osdep.h"
 #include "%(prefix)sqapi-commands.h"
scripts/qapi/expr.py +1 -2
index d7a289eded..fecf466fa7 100644
@@ -35,7 +35,6 @@ def check_name_is_str(name, info, source):
 def check_name_str(name, info, source,
                    allow_optional=False, enum_member=False,
                    permit_upper=False):
-    global valid_name
     membername = name
 
     if allow_optional and name.startswith('*'):
@@ -249,7 +248,7 @@ def check_union(expr, info):
 def check_alternate(expr, info):
     members = expr['data']
 
-    if len(members) == 0:
+    if not members:
         raise QAPISemError(info, "'data' must not be empty")
     for (key, value) in members.items():
         source = "'data' member '%s'" % key
scripts/qapi/gen.py +6 -3
index e17354392b..33690bfa3b 100644
@@ -45,10 +45,10 @@ class QAPIGen:
 
     def write(self, output_dir):
         pathname = os.path.join(output_dir, self.fname)
-        dir = os.path.dirname(pathname)
-        if dir:
+        odir = os.path.dirname(pathname)
+        if odir:
             try:
-                os.makedirs(dir)
+                os.makedirs(odir)
             except os.error as e:
                 if e.errno != errno.EEXIST:
                     raise
@@ -261,6 +261,9 @@ class QAPISchemaModularCVisitor(QAPISchemaVisitor):
             genc.write(output_dir)
             genh.write(output_dir)
 
+    def _begin_system_module(self, name):
+        pass
+
     def _begin_user_module(self, name):
         pass
 
scripts/qapi/introspect.py +0 -2
index 0cc655fd9f..b5537eddc0 100644
@@ -10,8 +10,6 @@ This work is licensed under the terms of the GNU GPL, version 2.
 See the COPYING file in the top-level directory.
 """
 
-import string
-
 from qapi.common import *
 from qapi.gen import QAPISchemaMonolithicCVisitor
 from qapi.schema import (QAPISchemaArrayType, QAPISchemaBuiltinType,
scripts/qapi/parser.py +2 -4
index 340f7c4633..abadacbb0e 100644
@@ -282,8 +282,7 @@ class QAPISchemaParser:
                 doc.end_comment()
                 self.accept()
                 return doc
-            else:
-                doc.append(self.val)
+            doc.append(self.val)
             self.accept(False)
 
         raise QAPIParseError(self, "documentation comment must end with '##'")
@@ -492,7 +491,7 @@ class QAPIDoc:
             raise QAPIParseError(self._parser,
                                  "'%s' can't follow '%s' section"
                                  % (name, self.sections[0].name))
-        elif self._is_section_tag(name):
+        if self._is_section_tag(name):
             line = line[len(name)+1:]
             self._start_section(name[:-1])
 
@@ -556,7 +555,6 @@ class QAPIDoc:
             raise QAPISemError(feature.info,
                                "feature '%s' lacks documentation"
                                % feature.name)
-            self.features[feature.name] = QAPIDoc.ArgSection(feature.name)
         self.features[feature.name].connect(feature)
 
     def check_expr(self, expr):
scripts/qapi/schema.py +5 -6
index 87837e224e..d759308b4e 100644
@@ -19,7 +19,7 @@ import re
 from collections import OrderedDict
 
 from qapi.common import c_name, pointer_suffix
-from qapi.error import QAPIError, QAPIParseError, QAPISemError
+from qapi.error import QAPIError, QAPISemError
 from qapi.expr import check_exprs
 from qapi.parser import QAPISchemaParser
 
@@ -96,14 +96,14 @@ class QAPISchemaVisitor:
     def visit_end(self):
         pass
 
-    def visit_module(self, fname):
+    def visit_module(self, name):
         pass
 
     def visit_needed(self, entity):
         # Default to visiting everything
         return True
 
-    def visit_include(self, fname, info):
+    def visit_include(self, name, info):
         pass
 
     def visit_builtin_type(self, name, info, json_type):
@@ -576,7 +576,7 @@ class QAPISchemaObjectTypeVariants:
             assert self.tag_member.ifcond == []
         if self._tag_name:    # flat union
             # branches that are not explicitly covered get an empty type
-            cases = set([v.name for v in self.variants])
+            cases = {v.name for v in self.variants}
             for m in self.tag_member.type.members:
                 if m.name not in cases:
                     v = QAPISchemaObjectTypeVariant(m.name, self.info,
@@ -848,7 +848,7 @@ class QAPISchema:
 
     def _make_module(self, fname):
         name = self._module_name(fname)
-        if not name in self._module_dict:
+        if name not in self._module_dict:
             self._module_dict[name] = QAPISchemaModule(name)
         return self._module_dict[name]
 
@@ -1097,7 +1097,6 @@ class QAPISchema:
 
     def visit(self, visitor):
         visitor.visit_begin(self)
-        module = None
         for mod in self._module_dict.values():
             mod.visit(visitor)
         visitor.visit_end()
Signature · 1 line
2.21.1
Reviewed-by on v2 4/4
#8 lore raw

Re: [PATCH v2 4/4] qapi: Brush off some (py)lint

Quoted 10 lines · Markus Armbruster
---
 scripts/qapi/commands.py   |  2 +-
 scripts/qapi/expr.py       |  3 +--
 scripts/qapi/gen.py        |  9 ++++++---
 scripts/qapi/introspect.py |  2 --
 scripts/qapi/parser.py     |  6 ++----
 scripts/qapi/schema.py     | 11 +++++------
 6 files changed, 15 insertions(+), 18 deletions(-)

Looks okay. (I don't care as much about no-else-return being there or not, and this module is your baby.)

#9 lore raw

Re: [PATCH v2 4/4] qapi: Brush off some (py)lint

Quoted 12 lines · John Snow
Quoted 10 lines · Markus Armbruster
---
 scripts/qapi/commands.py   |  2 +-
 scripts/qapi/expr.py       |  3 +--
 scripts/qapi/gen.py        |  9 ++++++---
 scripts/qapi/introspect.py |  2 --
 scripts/qapi/parser.py     |  6 ++----
 scripts/qapi/schema.py     | 11 +++++------
 6 files changed, 15 insertions(+), 18 deletions(-)
Quoted 2 lines

Looks okay. (I don't care as much about no-else-return being there or not, and this module is your baby.)

I admit I personally prefer to elide the else there. But my main argument is consistent style. Picking the one pylint wants makes consistency easier.

Quoted 1 line

Thanks!

#10 lore raw

Re: [PATCH v2 4/4] qapi: Brush off some (py)lint

Quoted 17 lines · Markus Armbruster
Quoted 12 lines · John Snow
Quoted 10 lines · Markus Armbruster
---
 scripts/qapi/commands.py   |  2 +-
 scripts/qapi/expr.py       |  3 +--
 scripts/qapi/gen.py        |  9 ++++++---
 scripts/qapi/introspect.py |  2 --
 scripts/qapi/parser.py     |  6 ++----
 scripts/qapi/schema.py     | 11 +++++------
 6 files changed, 15 insertions(+), 18 deletions(-)
Quoted 2 lines

Looks okay. (I don't care as much about no-else-return being there or not, and this module is your baby.)

Quoted 8 lines

I admit I personally prefer to elide the else there. But my main argument is consistent style. Picking the one pylint wants makes consistency easier.

Quoted 1 line

Thanks!

I capitulated to Max and Kevin who disliked the same, and also personally agree I like to visually see the branches.

(If we want a tree-wide python style it might be useful to converge the linting configurations at some point -- but not yet, and it's not as important as the more nuts and bolts cleanups to remove Python3. RB stands, of course.)

--js