Skip to contentSkip to conversation

tools: Python 3 compatibility

PATCH Superseded Wei Liu 2 participants 16 messages Mar 11, 2019

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Wei Liu

author
#0 lore raw

v3 cover tools: Python 3 compatibility

This series makes tools work with Python 2 and 3

Tested on my testbox with Python 2 and 3. Unfortunately I didn't have a suitable disk image for pygrub to consume, but at least while running pygrub 2 and 3 it gave the same error message.

Andrew's "tools/xen-foreign: Update python scripts to be Py3 compatible" is a pre-requisite.

Wei.

Wei Liu (7):
Update python requirement
pygrub/fsimage: drop unused struct
pygrub: change tabs into spaces
build/m4: make python_devel.m4 work with both python 2 and 3
libxl: make python scripts work with python 2.6 and up
pygrub: make python scripts work with 2.6 and up pygrub/fsimage: make it work with python 3

 README                             | 10 +---
 m4/python_devel.m4                 | 23 +++------
 tools/configure                    | 72 +++++----------------------
 tools/configure.ac                 |  2 +-
 tools/libxl/gentest.py             |  4 +-
 tools/libxl/gentypes.py            | 12 +++--
 tools/libxl/idl.py                 | 15 +++---
 tools/pygrub/src/ExtLinuxConf.py   | 19 +++++---
 tools/pygrub/src/GrubConf.py       | 39 ++++++++-------
 tools/pygrub/src/LiloConf.py       | 19 +++++---
 tools/pygrub/src/fsimage/fsimage.c | 99 ++++++++++++++++++++++++++++++++------
 tools/pygrub/src/pygrub            | 75 ++++++++++++++---------------
 12 files changed, 205 insertions(+), 184 deletions(-)
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#1 lore raw

PATCH v3 1/7 Update python requirement

CentOS 5, which was the reason of the 2.4 restriction, is EOL. CentOS 6 ships 2.6.

Bump the version to 2.6 in README. Now that all scripts are 3 compatible, remove the restriction on python 2 as well.

Update the check in configure.ac.

---
 README             | 10 ++--------
 tools/configure    |  8 ++++----
 tools/configure.ac |  2 +-
 3 files changed, 7 insertions(+), 13 deletions(-)
README +2 -8
index c19409efa2..3049201d91 100644
@@ -46,7 +46,7 @@ provided by your OS distributor:
         - GCC 4.8 or later
         - GNU Binutils 2.24 or later
     * Development install of zlib (e.g., zlib-dev)
-    * Development install of Python 2, v2.4 or later (e.g., python-dev)
+    * Development install of Python v2.6 or later (e.g., python-dev)
     * Development install of curses (e.g., libncurses-dev)
     * Development install of openssl (e.g., openssl-dev)
     * Development install of x11 (e.g. xorg-x11-dev)
@@ -177,16 +177,10 @@ Python Runtime Libraries
 
 Various tools, such as pygrub, have the following runtime dependencies:
 
-    * Python 2, v2.4 or later.
+    * Python v2.6 or later.
           URL:    http://www.python.org/
           Debian: python
 
-Note that the build system expects `python` to be python2.  If your system
-has `python` pointing to python3 (as in the case of Arch Linux or Anaconda),
-you'll need to specify a path to a python2 binary when running configure:
-
-    PYTHON=/usr/bin/python2 ./configure
-
 Intel(R) Trusted Execution Technology Support
 =============================================
 
tools/configure +4 -4
index acc857510e..d0065b811d 100755
@@ -7002,15 +7002,15 @@ if test x"${PYTHONPATH}" = x"no"
 then
     as_fn_error $? "Unable to find $PYTHON, please install $PYTHON" "$LINENO" 5
 fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for python version >= 2.3 " >&5
-$as_echo_n "checking for python version >= 2.3 ... " >&6; }
-`$PYTHON -c 'import sys; sys.exit(eval("sys.version_info < (2, 3)"))'`
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for python version >= 2.6 " >&5
+$as_echo_n "checking for python version >= 2.6 ... " >&6; }
+`$PYTHON -c 'import sys; sys.exit(eval("sys.version_info < (2, 6)"))'`
 if test "$?" != "0"
 then
     python_version=`$PYTHON -V 2>&1`
     { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
 $as_echo "no" >&6; }
-    as_fn_error $? "$python_version is too old, minimum required version is 2.3" "$LINENO" 5
+    as_fn_error $? "$python_version is too old, minimum required version is 2.6" "$LINENO" 5
 else
     { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
 $as_echo "yes" >&6; }
tools/configure.ac +1 -1
index 1499344ce6..c9fd69ddfa 100644
@@ -358,7 +358,7 @@ AS_IF([echo "$PYTHON" | grep -q "^/"], [
 ],[test -z "$PYTHON"], [PYTHON="python"],
 [AC_MSG_ERROR([PYTHON specified, but is not an absolute path])])
 AX_PATH_PROG_OR_FAIL([PYTHONPATH], [$PYTHON])
-AX_CHECK_PYTHON_VERSION([2], [3])
+AX_CHECK_PYTHON_VERSION([2], [6])
 
 AS_IF([test "$cross_compiling" != yes], [
     AX_CHECK_PYTHON_DEVEL()
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#2 lore raw

Re: [PATCH v3 1/7] Update python requirement

Quoted 5 lines · Wei Liu

CentOS 5, which was the reason of the 2.4 restriction, is EOL. CentOS 6 ships 2.6.

Bump the version to 2.6 in README. Now that all scripts are 3 compatible, remove the restriction on python 2 as well.

I think the Python 2 restriction needs to stay until the end of the series.

Its fine to update the minimum to 2.6 before the changes are made, but it isn't really ok to claim that everything is 3-compatible at this point.

~Andrew

Quoted 4 lines

Update the check in configure.ac.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#3 lore raw

Re: [PATCH v3 1/7] Update python requirement

Quoted 11 lines · Andrew Cooper
Quoted 5 lines · Wei Liu

CentOS 5, which was the reason of the 2.4 restriction, is EOL. CentOS 6 ships 2.6.

Bump the version to 2.6 in README. Now that all scripts are 3 compatible, remove the restriction on python 2 as well.

I think the Python 2 restriction needs to stay until the end of the series.

Its fine to update the minimum to 2.6 before the changes are made, but it isn't really ok to claim that everything is 3-compatible at this point.

It was at the end of the series in previous version. I forgot to update the description.

I don't care if it is the first or last patch, really...

Wei.

Quoted 8 lines

~Andrew

Quoted 4 lines

Update the check in configure.ac.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#4 lore raw

PATCH v3 2/7 pygrub/fsimage: drop unused struct

In reply to Wei Liu
---
 tools/pygrub/src/fsimage/fsimage.c | 7 -------
 1 file changed, 7 deletions(-)
tools/pygrub/src/fsimage/fsimage.c +0 -7
index 743a3fb7b8..780207791c 100644
@@ -43,13 +43,6 @@ typedef struct fsimage_file {
 	fsi_file_t *file;
 } fsimage_file_t;
 
-struct foo {
-	int ref;
-	int size;
-	long hash;
-	int state;
-};
-
 static PyObject *
 fsimage_file_read(fsimage_file_t *file, PyObject *args, PyObject *kwargs)
 {
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#6 lore raw

PATCH v3 3/7 pygrub: change tabs into spaces

In reply to Wei Liu

Not sure why Python 2 never complained, but Python 3 does.

Change tabs to spaces.

---
Backport candidate.

---
 tools/pygrub/src/pygrub | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/pygrub/src/pygrub +2 -2
index 52a8965ad9..1189b1ca48 100755
@@ -858,7 +858,7 @@ if __name__ == "__main__":
             output_directory = a
 
     if debug:
-	logging.basicConfig(level=logging.DEBUG)
+        logging.basicConfig(level=logging.DEBUG)
 
 
     try:
@@ -917,7 +917,7 @@ if __name__ == "__main__":
             # IOErrors raised by fsimage.open
             # RuntimeErrors raised by run_grub if no menu.lst present
             if debug:
-		traceback.print_exc()
+                traceback.print_exc()
             fs = None
             continue
 
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Reviewed-by on v3 3/7
#7 lore raw

Re: [PATCH v3 3/7] pygrub: change tabs into spaces

Quoted 5 lines · Wei Liu

Not sure why Python 2 never complained, but Python 3 does.

Change tabs to spaces.

Tabs vs spaces was an issue with Py2 which got fixed in 3 when it was fine to break existing scripts.

The problem was that with certain editor settings, you could the indentation looking correct while evaluating differently with Pythons idea of indentation.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#8 lore raw

PATCH v3 4/7 build/m4: make python_devel.m4 work with both python 2 and 3

In reply to Wei Liu

Do the following:

1. Change the form of "print".
2. Use AC_CHECK_FUNC to avoid the need to generate library name.
3. Remove unused stuff.

---
 m4/python_devel.m4 | 23 +++++++-------------
 tools/configure    | 64 +++++++-----------------------------------------------
 2 files changed, 16 insertions(+), 71 deletions(-)
m4/python_devel.m4 +8 -15
index 05ea4ef7e2..f9cb23aee1 100644
@@ -1,38 +1,31 @@
 AC_DEFUN([AX_CHECK_PYTHON_DEVEL], [
 ac_previous_cppflags=$CPPFLAGS
 ac_previous_ldflags=$LDFLAGS
-ac_python_version=`$PYTHON -c 'import distutils.sysconfig; \
-    print distutils.sysconfig.get_config_var("VERSION")'`
 AC_PATH_PROG([pyconfig], [$PYTHON-config], [no])
 AS_IF([test x"$pyconfig" = x"no"], [
     dnl For those that don't have python-config
     CPPFLAGS="$CFLAGS `$PYTHON -c 'import distutils.sysconfig; \
         print "-I" + distutils.sysconfig.get_config_var("INCLUDEPY")'`"
     CPPFLAGS="$CPPFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("CFLAGS")'`"
-    PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("LIBS")'`"
-    PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("SYSLIBS")'`"
+        print(distutils.sysconfig.get_config_var("CFLAGS"))'`"
     LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print "-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\
-        standard_lib=1) + "/config"'`"
+        print("-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\
+        standard_lib=1) + "/config")'`"
     LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("LINKFORSHARED")'`"
+        print(distutils.sysconfig.get_config_var("LINKFORSHARED"))'`"
     LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \
-        print distutils.sysconfig.get_config_var("LDFLAGS")'`"
+        print(distutils.sysconfig.get_config_var("LDFLAGS"))'`"
 ], [
     dnl If python-config is found use it
     CPPFLAGS="$CFLAGS `$PYTHON-config --cflags`"
     LDFLAGS="$LDFLAGS `$PYTHON-config --ldflags`"
-    PYTHON_LIBS="$LIBS `$PYTHON-config --libs`"
 ])
 
 AC_CHECK_HEADER([Python.h], [],
     [AC_MSG_ERROR([Unable to find Python development headers])],)
-AC_CHECK_LIB(python$ac_python_version, PyArg_ParseTuple, [],
-    [AC_MSG_ERROR([Unable to find a suitable python development library])],
-    [$PYTHON_LIBS])
+AC_CHECK_FUNC([PyArg_ParseTuple], [],
+    [AC_MSG_ERROR([Unable to find a suitable python development library])]) 
+
 CPPFLAGS=$ac_previous_cppflags
 LDFLAGS=$ac_previous_ldflags
 ])
tools/configure +8 -56

95 lines not on this page · raw message

Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#9 lore raw

PATCH v3 5/7 libxl: make python scripts work with python 2.6 and up

In reply to Wei Liu

Go through transformations suggested by 2to3 and pick the necessary ones.

---
 tools/libxl/gentest.py  |  4 +++-
 tools/libxl/gentypes.py | 12 +++++++-----
 tools/libxl/idl.py      | 15 ++++++++-------
 3 files changed, 18 insertions(+), 13 deletions(-)
tools/libxl/gentest.py +3 -1
index 989959fc68..1cc7eebc82 100644
@@ -1,5 +1,7 @@
 #!/usr/bin/python
 
+from __future__ import print_function
+
 import os
 import sys
 import re
@@ -86,7 +88,7 @@ def gen_rand_init(ty, v, indent = "    ", parent = None):
 
 if __name__ == '__main__':
     if len(sys.argv) < 3:
-        print >>sys.stderr, "Usage: gentest.py <idl> <implementation>"
+        print("Usage: gentest.py <idl> <implementation>", file=sys.stderr)
         sys.exit(1)
 
     random.seed(os.getenv('LIBXL_TESTIDL_SEED'))
tools/libxl/gentypes.py +7 -5
index 88e5c5f30e..6417c9dd8c 100644
@@ -1,5 +1,7 @@
 #!/usr/bin/python
 
+from __future__ import print_function
+
 import sys
 import re
 
@@ -576,14 +578,14 @@ def libxl_C_enum_from_string(ty, str, e, indent = "    "):
 
 if __name__ == '__main__':
     if len(sys.argv) != 6:
-        print >>sys.stderr, "Usage: gentypes.py <idl> <header> <header-private> <header-json> <implementation>"
+        print("Usage: gentypes.py <idl> <header> <header-private> <header-json> <implementation>", file=sys.stderr)
         sys.exit(1)
 
     (_, idlname, header, header_private, header_json, impl) = sys.argv
 
     (builtins,types) = idl.parse(idlname)
 
-    print "outputting libxl type definitions to %s" % header
+    print("outputting libxl type definitions to %s" % header)
 
     f = open(header, "w")
 
@@ -633,7 +635,7 @@ if __name__ == '__main__':
     f.write("""#endif /* %s */\n""" % (header_define))
     f.close()
 
-    print "outputting libxl JSON definitions to %s" % header_json
+    print("outputting libxl JSON definitions to %s" % header_json)
 
     f = open(header_json, "w")
 
@@ -657,7 +659,7 @@ if __name__ == '__main__':
     f.write("""#endif /* %s */\n""" % header_json_define)
     f.close()
 
-    print "outputting libxl type internal definitions to %s" % header_private
+    print("outputting libxl type internal definitions to %s" % header_private)
 
     f = open(header_private, "w")
 
@@ -683,7 +685,7 @@ if __name__ == '__main__':
     f.write("""#endif /* %s */\n""" % header_json_define)
     f.close()
 
-    print "outputting libxl type implementations to %s" % impl
+    print("outputting libxl type implementations to %s" % impl)
 
     f = open(impl, "w")
     f.write("""
tools/libxl/idl.py +8 -7
index 2a7f3c44fe..d7367503b4 100644
@@ -1,3 +1,5 @@
+from __future__ import print_function
+
 import sys
 
 PASS_BY_VALUE = 1
@@ -11,7 +13,7 @@ DIR_BOTH = 3
 _default_namespace = ""
 def namespace(s):
     if type(s) != str:
-        raise TypeError, "Require a string for the default namespace."
+        raise TypeError("Require a string for the default namespace.")
     global _default_namespace
     _default_namespace = s
 
@@ -346,7 +348,7 @@ class OrderedDict(dict):
         return [(x,self[x]) for x in self.__ordered]
 
 def parse(f):
-    print >>sys.stderr, "Parsing %s" % f
+    print("Parsing %s" % f, file=sys.stderr)
 
     globs = {}
     locs = OrderedDict()
@@ -362,11 +364,10 @@ def parse(f):
             globs[n] = t
 
     try:
-        execfile(f, globs, locs)
-    except SyntaxError,e:
-        raise SyntaxError, \
-              "Errors were found at line %d while processing %s:\n\t%s"\
-              %(e.lineno,f,e.text)
+        exec(compile(open(f).read(), f, 'exec'), globs, locs)
+    except SyntaxError as e:
+        raise SyntaxError("Errors were found at line %d while processing %s:\n\t%s"
+                          % (e.lineno, f, e.text))
 
     types = [t for t in locs.ordered_values() if isinstance(t,Type)]
 
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#11 lore raw

PATCH v3 6/7 pygrub: make python scripts work with 2.6 and up

In reply to Wei Liu

Run 2to3 and pick the sensible suggestions.

Import print_function and absolute_import so 2.6 can work.

There has never been a curses.wrapper module according to 2.x and 3.x doc, only a function, so "import curses.wrapper" is not correct. It happened to work because 2.x implemented a (undocumented) module.

We only need to import curses to make curses.wrapper available to pygrub.

---
 tools/pygrub/src/ExtLinuxConf.py | 19 ++++++-----
 tools/pygrub/src/GrubConf.py     | 39 ++++++++++++----------
 tools/pygrub/src/LiloConf.py     | 19 ++++++-----
 tools/pygrub/src/pygrub          | 71 ++++++++++++++++++++--------------------
 4 files changed, 78 insertions(+), 70 deletions(-)
tools/pygrub/src/ExtLinuxConf.py +11 -8
index d1789bf020..9fd635b9cf 100644
@@ -10,9 +10,11 @@
 # along with this program; If not, see <http://www.gnu.org/licenses/>.
 #
 
+from __future__ import print_function, absolute_import
+
 import sys, re, os
 import logging
-import GrubConf
+from . import GrubConf
 
 class ExtLinuxImage(object):
     def __init__(self, lines, path):
@@ -32,7 +34,8 @@ class ExtLinuxImage(object):
         self.lines = []
         self.path = path
         self.root = ""
-        map(self.set_from_line, lines)
+        for line in lines:
+            self.set_from_line(line)
 
     def set_from_line(self, line, replace = None):
         (com, arg) = GrubConf.grub_exact_split(line, 2)
@@ -67,7 +70,7 @@ class ExtLinuxImage(object):
                         setattr(self, "initrd", a.replace("initrd=", ""))
                         arg = arg.replace(a, "")
 
-        if com is not None and self.commands.has_key(com):
+        if com is not None and com in self.commands:
             if self.commands[com] is not None:
                 setattr(self, self.commands[com], re.sub('^"(.+)"$', r"\1", arg.strip()))
             else:
@@ -136,7 +139,7 @@ class ExtLinuxConfigFile(object):
     def parse(self, buf = None):
         if buf is None:
             if self.filename is None:
-                raise ValueError, "No config file defined to parse!"
+                raise ValueError("No config file defined to parse!")
 
             f = open(self.filename, 'r')
             lines = f.readlines()
@@ -167,7 +170,7 @@ class ExtLinuxConfigFile(object):
 
             (com, arg) = GrubConf.grub_exact_split(l, 2)
             com = com.lower()
-            if self.commands.has_key(com):
+            if com in self.commands:
                 if self.commands[com] is not None:
                     setattr(self, self.commands[com], arg.strip())
                 else:
@@ -207,8 +210,8 @@ class ExtLinuxConfigFile(object):
         
 if __name__ == "__main__":
     if len(sys.argv) < 2:
-        raise RuntimeError, "Need a configuration file to read"
+        raise RuntimeError("Need a configuration file to read")
     g = ExtLinuxConfigFile(sys.argv[1])
     for i in g.images:
-        print i
-    print g.default
+        print(i)
+    print(g.default)
tools/pygrub/src/GrubConf.py +21 -18
index dc810d55cb..f8d3799dc0 100644
@@ -12,6 +12,8 @@
 # along with this program; If not, see <http://www.gnu.org/licenses/>.
 #
 
+from __future__ import print_function, absolute_import
+
 import os, sys
 import logging
 import re
@@ -44,7 +46,7 @@ def get_path(s):
         return (None, s)
     idx = s.find(')')
     if idx == -1:
-        raise ValueError, "Unable to find matching ')'"
+        raise ValueError("Unable to find matching ')'")
     d = s[:idx]
     return (GrubDiskPart(d), s[idx + 1:])
 
@@ -100,7 +102,8 @@ class _GrubImage(object):
                 "  initrd: %s\n" %(self.title, self.root, self.kernel,
                                    self.args, self.initrd))
     def _parse(self, lines):
-        map(self.set_from_line, lines)
+        for line in lines:
+            self.set_from_line(line)
 
     def reset(self, lines):
         self._root = self._initrd = self._kernel = self._args = None
@@ -141,7 +144,7 @@ class GrubImage(_GrubImage):
     def set_from_line(self, line, replace = None):
         (com, arg) = grub_exact_split(line, 2)
 
-        if self.commands.has_key(com):
+        if com in self.commands:
             if self.commands[com] is not None:
                 setattr(self, self.commands[com], arg.strip())
             else:
@@ -177,7 +180,7 @@ class _GrubConfigFile(object):
             self.parse()
 
     def parse(self, buf = None):
-        raise RuntimeError, "unimplemented parse function"   
+        raise RuntimeError("unimplemented parse function")
 
     def hasPasswordAccess(self):
         return self.passwordAccess
@@ -201,7 +204,7 @@ class _GrubConfigFile(object):
                 import crypt
                 if crypt.crypt(password, pwd[1]) == pwd[1]:
                     return True
-            except Exception, e:
+            except Exception as e:
                 self.passExc = "Can't verify password: %s" % str(e)
                 return False
 
@@ -213,7 +216,7 @@ class _GrubConfigFile(object):
 
     def set(self, line):
         (com, arg) = grub_exact_split(line, 2)
-        if self.commands.has_key(com):
+        if com in self.commands:
             if self.commands[com] is not None:
                 setattr(self, self.commands[com], arg.strip())
             else:
@@ -233,7 +236,7 @@ class _GrubConfigFile(object):
             self._default = val
 
         if self._default < 0:
-            raise ValueError, "default must be positive number"
+            raise ValueError("default must be positive number")
     default = property(_get_default, _set_default)
 
     def set_splash(self, val):
@@ -265,7 +268,7 @@ class GrubConfigFile(_GrubConfigFile):
     def parse(self, buf = None):
         if buf is None:
             if self.filename is None:
-                raise ValueError, "No config file defined to parse!"
+                raise ValueError("No config file defined to parse!")
 
             f = open(self.filename, 'r')
             lines = f.readlines()
@@ -296,7 +299,7 @@ class GrubConfigFile(_GrubConfigFile):
                 continue
 
             (com, arg) = grub_exact_split(l, 2)
-            if self.commands.has_key(com):
+            if com in self.commands:
                 if self.commands[com] is not None:
                     setattr(self, self.commands[com], arg.strip())
                 else:
@@ -328,7 +331,7 @@ class Grub2Image(_GrubImage):
         if com == "set":
             (com,arg) = grub2_handle_set(arg)
             
-        if self.commands.has_key(com):
+        if com in self.commands:
             if self.commands[com] is not None:
                 setattr(self, self.commands[com], arg.strip())
             else:
@@ -364,7 +367,7 @@ class Grub2ConfigFile(_GrubConfigFile):
     def parse(self, buf = None):
         if buf is None:
             if self.filename is None:
-                raise ValueError, "No config file defined to parse!"
+                raise ValueError("No config file defined to parse!")
 
             f = open(self.filename, 'r')
             lines = f.readlines()
@@ -398,7 +401,7 @@ class Grub2ConfigFile(_GrubConfigFile):
             title_match = re.match('^menuentry ["\'](.*?)["\'] (.*){', l)
             if title_match:
                 if img is not None:
-                    raise RuntimeError, "syntax error: cannot nest menuentry (%d %s)" % (len(img),img)
+                    raise RuntimeError("syntax error: cannot nest menuentry (%d %s)" % (len(img),img))
                 img = []
                 title = title_match.group(1)
                 continue
@@ -413,7 +416,7 @@ class Grub2ConfigFile(_GrubConfigFile):
                         menu_level -= 1
                         continue
                     else:
-                        raise RuntimeError, "syntax error: closing brace without menuentry"
+                        raise RuntimeError("syntax error: closing brace without menuentry")
 
                 self.add_image(Grub2Image(title, img))
                 img = None
@@ -428,7 +431,7 @@ class Grub2ConfigFile(_GrubConfigFile):
             if com == "set":
                 (com,arg) = grub2_handle_set(arg)
                 
-            if self.commands.has_key(com):
+            if com in self.commands:
                 if self.commands[com] is not None:
                     arg_strip = arg.strip()
                     if arg_strip == "${saved_entry}" or arg_strip == "${next_entry}":
@@ -443,7 +446,7 @@ class Grub2ConfigFile(_GrubConfigFile):
                 logging.warning("Unknown directive %s" %(com,))
             
         if img is not None:
-            raise RuntimeError, "syntax error: end of file with open menuentry(%d %s)" % (len(img),img)
+            raise RuntimeError("syntax error: end of file with open menuentry(%d %s)" % (len(img),img))
 
         if self.hasPassword():
             self.setPasswordAccess(False)
@@ -462,12 +465,12 @@ class Grub2ConfigFile(_GrubConfigFile):
         
 if __name__ == "__main__":
     if len(sys.argv) < 3:
-        raise RuntimeError, "Need a grub version (\"grub\" or \"grub2\") and a grub.conf or grub.cfg to read"
+        raise RuntimeError('Need a grub version ("grub" or "grub2") and a grub.conf or grub.cfg to read')
     if sys.argv[1] == "grub":
         g = GrubConfigFile(sys.argv[2])
     elif sys.argv[1] == "grub2":
         g = Grub2ConfigFile(sys.argv[2])
     else:
-        raise RuntimeError, "Unknown config type %s" % sys.argv[1]
+        raise RuntimeError("Unknown config type %s" % sys.argv[1])
     for i in g.images:
-        print i #, i.title, i.root, i.kernel, i.args, i.initrd
+        print(i) #, i.title, i.root, i.kernel, i.args, i.initrd
tools/pygrub/src/LiloConf.py +11 -8
index 2cb649f115..e3bfcb5244 100644
@@ -2,9 +2,11 @@
 #LiloConf.py
 #
 
+from __future__ import print_function, absolute_import
+
 import sys, re, os
 import logging
-import GrubConf
+from . import GrubConf
 
 class LiloImage(object):
     def __init__(self, lines, path):
@@ -24,12 +26,13 @@ class LiloImage(object):
         self.lines = []
         self.path = path
         self.root = ""
-        map(self.set_from_line, lines)
+        for line in lines:
+            self.set_from_line(line)
 
     def set_from_line(self, line, replace = None):
         (com, arg) = GrubConf.grub_exact_split(line, 2)
 
-        if self.commands.has_key(com):
+        if com in self.commands:
             if self.commands[com] is not None:
                 setattr(self, self.commands[com], re.sub('^"(.+)"$', r"\1", arg.strip()))
             else:
@@ -97,7 +100,7 @@ class LiloConfigFile(object):
     def parse(self, buf = None):
         if buf is None:
             if self.filename is None:
-                raise ValueError, "No config file defined to parse!"
+                raise ValueError("No config file defined to parse!")
 
             f = open(self.filename, 'r')
             lines = f.readlines()
@@ -127,7 +130,7 @@ class LiloConfigFile(object):
                 continue
 
             (com, arg) = GrubConf.grub_exact_split(l, 2)
-            if self.commands.has_key(com):
+            if com in self.commands:
                 if self.commands[com] is not None:
                     setattr(self, self.commands[com], arg.strip())
                 else:
@@ -170,8 +173,8 @@ class LiloConfigFile(object):
 
 if __name__ == "__main__":
     if len(sys.argv) < 2:
-        raise RuntimeError, "Need a lilo.conf to read"
+        raise RuntimeError("Need a lilo.conf to read")
     g = LiloConfigFile(sys.argv[1])
     for i in g.images:
-        print i #, i.title, i.root, i.kernel, i.args, i.initrd
-    print g.default
+        print(i) #, i.title, i.root, i.kernel, i.args, i.initrd
+    print(g.default)
tools/pygrub/src/pygrub +35 -36

187 lines not on this page · raw message

Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Reviewed-by on v3 6/7
#12 lore raw

Re: [PATCH v3 6/7] pygrub: make python scripts work with 2.6 and up

Quoted 4 lines · Wei Liu

Run 2to3 and pick the sensible suggestions.

Import print_function and absolute_import so 2.6 can work.

Quoted 8 lines

There has never been a curses.wrapper module according to 2.x and 3.x doc, only a function, so "import curses.wrapper" is not correct. It happened to work because 2.x implemented a (undocumented) module.

We only need to import curses to make curses.wrapper available to pygrub.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#13 lore raw

PATCH v3 7/7 pygrub/fsimage: make it work with python 3

In reply to Wei Liu

With the help of two porting guides and cpython source code:

1. Use PyBytes to replace PyString counterparts.
2. Use PyVarObject_HEAD_INIT.
3. Remove usage of Py_FindMethod.
4. Use new module initialisation routine.

For #3, Py_FindMethod was removed, yet an alternative wasn't documented. The code is the result of reverse-engineering cpython commit 6116d4a1d1

https://docs.python.org/3/howto/cporting.html
http://python3porting.com/cextensions.html

---
v3: use PyType_Ready to initialise types.
v2: use PyBytes.

---
 tools/pygrub/src/fsimage/fsimage.c | 92 ++++++++++++++++++++++++++++++++++----
 1 file changed, 84 insertions(+), 8 deletions(-)
tools/pygrub/src/fsimage/fsimage.c +84 -8
index 780207791c..057786d678 100644
@@ -26,11 +26,15 @@
 #include <xenfsimage.h>
 #include <stdlib.h>
 
+#if PY_MAJOR_VERSION < 3
 #if (PYTHON_API_VERSION >= 1011)
 #define PY_PAD 0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L
 #else
 #define PY_PAD 0L,0L,0L,0L
 #endif
+#else
+#define PY_PAD 0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L
+#endif
 
 typedef struct fsimage_fs {
 	PyObject_HEAD
@@ -59,12 +63,24 @@ fsimage_file_read(fsimage_file_t *file, PyObject *args, PyObject *kwargs)
 
 	bufsize = size ? size : 4096;
 
-	if ((buffer = PyString_FromStringAndSize(NULL, bufsize)) == NULL)
+	buffer =
+#if PY_MAJOR_VERSION < 3
+		PyString_FromStringAndSize(NULL, bufsize);
+#else
+		PyBytes_FromStringAndSize(NULL, bufsize);
+#endif
+
+	if (buffer == NULL)
 		return (NULL);
  
 	while (1) {
 		int err;
-		void *buf = PyString_AS_STRING(buffer) + bytesread;
+		void *buf =
+#if PY_MAJOR_VERSION < 3
+			PyString_AS_STRING(buffer) + bytesread;
+#else
+			PyBytes_AS_STRING(buffer) + bytesread;
+#endif
 
 		err = fsi_pread_file(file->file, buf, bufsize,
 		    bytesread + offset);
@@ -84,12 +100,20 @@ fsimage_file_read(fsimage_file_t *file, PyObject *args, PyObject *kwargs)
 			if (bufsize == 0)
 				break;
 		} else {
+#if PY_MAJOR_VERSION < 3
 			if (_PyString_Resize(&buffer, bytesread + bufsize) < 0)
+#else
+			if (_PyBytes_Resize(&buffer, bytesread + bufsize) < 0)
+#endif
 				return (NULL);
 		}
 	}
 
+#if PY_MAJOR_VERSION < 3
 	_PyString_Resize(&buffer, bytesread);
+#else
+	_PyBytes_Resize(&buffer, bytesread);
+#endif
 	return (buffer);
 }
 
@@ -106,11 +130,13 @@ static struct PyMethodDef fsimage_file_methods[] = {
 	{ NULL, NULL, 0, NULL }	
 };
 
+#if PY_MAJOR_VERSION < 3
 static PyObject *
 fsimage_file_getattr(fsimage_file_t *file, char *name)
 {
 	return (Py_FindMethod(fsimage_file_methods, (PyObject *)file, name));
 }
+#endif
 
 static void
 fsimage_file_dealloc(fsimage_file_t *file)
@@ -123,14 +149,17 @@ fsimage_file_dealloc(fsimage_file_t *file)
 
 static char fsimage_file_type__doc__[] = "Filesystem image file";
 PyTypeObject fsimage_file_type = {
-	PyObject_HEAD_INIT(&PyType_Type)
-	0,					/* ob_size */
+	PyVarObject_HEAD_INIT(&PyType_Type, 0)
 	"xenfsimage.file",			/* tp_name */
 	sizeof(fsimage_file_t),			/* tp_size */
 	0,					/* tp_itemsize */
 	(destructor) fsimage_file_dealloc, 	/* tp_dealloc */
 	0,					/* tp_print */
+#if PY_MAJOR_VERSION < 3
 	(getattrfunc) fsimage_file_getattr, 	/* tp_getattr */
+#else
+	0,					/* tp_getattr */
+#endif
 	0,					/* tp_setattr */
 	0,					/* tp_compare */
 	0,					/* tp_repr */
@@ -144,7 +173,16 @@ PyTypeObject fsimage_file_type = {
 	0,					/* tp_setattro */
 	0,					/* tp_as_buffer */
 	Py_TPFLAGS_DEFAULT,			/* tp_flags */
-	fsimage_file_type__doc__,
+	fsimage_file_type__doc__,		/* tp_doc */
+#if PY_MAJOR_VERSION >= 3
+	0,					/* tp_traverse */
+	0,					/* tp_clear */
+	0,					/* tp_richcompare */
+	0,					/* tp_weaklistoffset */
+	0,					/* tp_iter */
+	0,					/* tp_iternext */
+	fsimage_file_methods,			/* tp_methods */
+#endif
 	PY_PAD
 };
 
@@ -208,11 +246,13 @@ static struct PyMethodDef fsimage_fs_methods[] = {
 	{ NULL, NULL, 0, NULL }	
 };
 
+#if PY_MAJOR_VERSION < 3
 static PyObject *
 fsimage_fs_getattr(fsimage_fs_t *fs, char *name)
 {
 	return (Py_FindMethod(fsimage_fs_methods, (PyObject *)fs, name));
 }
+#endif
 
 static void
 fsimage_fs_dealloc (fsimage_fs_t *fs)
@@ -225,14 +265,17 @@ fsimage_fs_dealloc (fsimage_fs_t *fs)
 PyDoc_STRVAR(fsimage_fs_type__doc__, "Filesystem image");
 
 PyTypeObject fsimage_fs_type = {
-	PyObject_HEAD_INIT(&PyType_Type)
-	0,					/* ob_size */
+	PyVarObject_HEAD_INIT(&PyType_Type, 0)
 	"xenfsimage.fs",			/* tp_name */
 	sizeof(fsimage_fs_t),			/* tp_size */
 	0,					/* tp_itemsize */
 	(destructor) fsimage_fs_dealloc, 	/* tp_dealloc */
 	0,					/* tp_print */
+#if PY_MAJOR_VERSION < 3
 	(getattrfunc) fsimage_fs_getattr, 	/* tp_getattr */
+#else
+	0,					/* tp_getattr */
+#endif
 	0,					/* tp_setattr */
 	0,					/* tp_compare */
 	0,					/* tp_repr */
@@ -246,7 +289,16 @@ PyTypeObject fsimage_fs_type = {
 	0,					/* tp_setattro */
 	0,					/* tp_as_buffer */
 	Py_TPFLAGS_DEFAULT,			/* tp_flags */
-	fsimage_fs_type__doc__,
+	fsimage_fs_type__doc__,			/* tp_doc */
+#if PY_MAJOR_VERSION >= 3
+	0,					/* tp_traverse */
+	0,					/* tp_clear */
+	0,					/* tp_richcompare */
+	0,					/* tp_weaklistoffset */
+	0,					/* tp_iter */
+	0,					/* tp_iternext */
+	fsimage_fs_methods,			/* tp_methods */
+#endif
 	PY_PAD
 };
 
@@ -309,8 +361,32 @@ static struct PyMethodDef fsimage_module_methods[] = {
 	{ NULL, NULL, 0, NULL }
 };
 
+#if PY_MAJOR_VERSION >= 3
+static struct PyModuleDef fsimage_module_def = {
+	PyModuleDef_HEAD_INIT,
+	"xenfsimage",		/* m_name */
+	"",			/* m_doc */
+	-1,			/* m_size */
+	fsimage_module_methods,	/* m_methods */
+	NULL,			/* m_reload */
+	NULL,			/* m_traverse */
+	NULL,			/* m_clear */
+	NULL			/* m_free */
+};
+#endif
+
 PyMODINIT_FUNC
+#if PY_MAJOR_VERSION >= 3
+PyInit_xenfsimage(void)
+#else
 initxenfsimage(void)
+#endif
 {
+#if PY_MAJOR_VERSION < 3
 	Py_InitModule("xenfsimage", fsimage_module_methods);
+#else
+	if (PyType_Ready(&fsimage_fs_type) < 0 || PyType_Ready(&fsimage_file_type) < 0)
+		return NULL;
+	return PyModule_Create(&fsimage_module_def);
+#endif
 }
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#14 lore raw

[PATCH v3 8/7] pygrub/fsimage: use named initialisation in extension

... and drop PY_PAD.

---
This can be squashed into previous patch.

---
 tools/pygrub/src/fsimage/fsimage.c | 47 ++++++--------------------------------
 1 file changed, 7 insertions(+), 40 deletions(-)
tools/pygrub/src/fsimage/fsimage.c +7 -40
index cf8089a872..5483aefbfe 100644
@@ -26,16 +26,6 @@
 #include <xenfsimage.h>
 #include <stdlib.h>
 
-#if PY_MAJOR_VERSION < 3
-#if (PYTHON_API_VERSION >= 1011)
-#define PY_PAD 0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L
-#else
-#define PY_PAD 0L,0L,0L,0L
-#endif
-#else
-#define PY_PAD 0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L
-#endif
-
 typedef struct fsimage_fs {
 	PyObject_HEAD
 	fsi_t *fs;
@@ -150,40 +140,17 @@ fsimage_file_dealloc(fsimage_file_t *file)
 static char fsimage_file_type__doc__[] = "Filesystem image file";
 PyTypeObject fsimage_file_type = {
 	PyVarObject_HEAD_INIT(&PyType_Type, 0)
-	"xenfsimage.file",			/* tp_name */
-	sizeof(fsimage_file_t),			/* tp_size */
-	0,					/* tp_itemsize */
-	(destructor) fsimage_file_dealloc, 	/* tp_dealloc */
-	0,					/* tp_print */
+	.tp_name = "xenfsimage.file",
+	.tp_basicsize = sizeof(fsimage_file_t),
+	.tp_dealloc = (destructor) fsimage_file_dealloc,
 #if PY_MAJOR_VERSION < 3
-	(getattrfunc) fsimage_file_getattr, 	/* tp_getattr */
-#else
-	0,					/* tp_getattr */
+	.tp_getattr = (getattrfunc) fsimage_file_getattr,
 #endif
-	0,					/* tp_setattr */
-	0,					/* tp_compare */
-	0,					/* tp_repr */
-	0,					/* tp_as_number */
-	0,	 				/* tp_as_sequence */
-	0,					/* tp_as_mapping */
-	0,	   				/* tp_hash */
-	0,					/* tp_call */
-	0,					/* tp_str */
-	0,					/* tp_getattro */
-	0,					/* tp_setattro */
-	0,					/* tp_as_buffer */
-	Py_TPFLAGS_DEFAULT,			/* tp_flags */
-	fsimage_file_type__doc__,		/* tp_doc */
+	.tp_flags = Py_TPFLAGS_DEFAULT,
+	.tp_doc = fsimage_file_type__doc__,
 #if PY_MAJOR_VERSION >= 3
-	0,					/* tp_traverse */
-	0,					/* tp_clear */
-	0,					/* tp_richcompare */
-	0,					/* tp_weaklistoffset */
-	0,					/* tp_iter */
-	0,					/* tp_iternext */
-	fsimage_file_methods,			/* tp_methods */
+	.tp_methods = fsimage_file_methods,
 #endif
-	PY_PAD
 };
 
 static PyObject *
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel