tools: Python 3 compatibility
v2 cover tools: Python 3 compatibility
This series makes tools work with Python 2 and 3.
No more RFC because I have tested it on my testbox.
You need Andrew's "tools/xen-foreign: Update python scripts to be Py3 compatible" as well.
Wei.
Wei Liu (5):
build/m4: make python_devel.m4 work with both python 2 and 3
libxl: make python scripts work with python 2.6 and up
pygrub: convert python scripts to work with 2.6 and up
pygrub: make it build with python 3
Update python requirement
README | 10 +--- m4/python_devel.m4 | 23 ++++----- tools/configure | 72 +++++----------------------- tools/configure.ac | 2 +- tools/libxl/gentest.py | 2 +- tools/libxl/gentypes.py | 10 ++-- tools/libxl/idl.py | 13 +++--- tools/pygrub/src/ExtLinuxConf.py | 15 +++--- tools/pygrub/src/GrubConf.py | 36 +++++++------- tools/pygrub/src/LiloConf.py | 15 +++--- tools/pygrub/src/fsimage/fsimage.c | 96 ++++++++++++++++++++++++++++++++++---- 11 files changed, 157 insertions(+), 137 deletions(-)
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 1/5 build/m4: make python_devel.m4 work with both python 2 and 3
Do the following:
1. Change the form of "print".
2. Use AC_CHECK_FUNC to avoid the need to generate library name.
3. Remove unused stuff.
---
I doubt the non-python-config branch works, because the paths generated seem rather off. It definitely doesn't work on my machine, but I don't know how other systems could possibly be configured before the existence of python-config.
--- m4/python_devel.m4 | 23 +++++++------------- tools/configure | 64 +++++++----------------------------------------------- 2 files changed, 16 insertions(+), 71 deletions(-)
m4/python_devel.m4 +8 -15
@@ -1,38 +1,31 @@ AC_DEFUN([AX_CHECK_PYTHON_DEVEL], [ ac_previous_cppflags=$CPPFLAGS ac_previous_ldflags=$LDFLAGS -ac_python_version=`$PYTHON -c 'import distutils.sysconfig; \ - print distutils.sysconfig.get_config_var("VERSION")'` AC_PATH_PROG([pyconfig], [$PYTHON-config], [no]) AS_IF([test x"$pyconfig" = x"no"], [ dnl For those that don't have python-config CPPFLAGS="$CFLAGS `$PYTHON -c 'import distutils.sysconfig; \ print "-I" + distutils.sysconfig.get_config_var("INCLUDEPY")'`" CPPFLAGS="$CPPFLAGS `$PYTHON -c 'import distutils.sysconfig; \ - print distutils.sysconfig.get_config_var("CFLAGS")'`" - PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \ - print distutils.sysconfig.get_config_var("LIBS")'`" - PYTHON_LIBS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \ - print distutils.sysconfig.get_config_var("SYSLIBS")'`" + print(distutils.sysconfig.get_config_var("CFLAGS"))'`" LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \ - print "-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\ - standard_lib=1) + "/config"'`" + print("-L" + distutils.sysconfig.get_python_lib(plat_specific=1,\ + standard_lib=1) + "/config")'`" LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \ - print distutils.sysconfig.get_config_var("LINKFORSHARED")'`" + print(distutils.sysconfig.get_config_var("LINKFORSHARED"))'`" LDFLAGS="$LDFLAGS `$PYTHON -c 'import distutils.sysconfig; \ - print distutils.sysconfig.get_config_var("LDFLAGS")'`" + print(distutils.sysconfig.get_config_var("LDFLAGS"))'`" ], [ dnl If python-config is found use it CPPFLAGS="$CFLAGS `$PYTHON-config --cflags`" LDFLAGS="$LDFLAGS `$PYTHON-config --ldflags`" - PYTHON_LIBS="$LIBS `$PYTHON-config --libs`" ]) AC_CHECK_HEADER([Python.h], [], [AC_MSG_ERROR([Unable to find Python development headers])],) -AC_CHECK_LIB(python$ac_python_version, PyArg_ParseTuple, [], - [AC_MSG_ERROR([Unable to find a suitable python development library])], - [$PYTHON_LIBS]) +AC_CHECK_FUNC([PyArg_ParseTuple], [], + [AC_MSG_ERROR([Unable to find a suitable python development library])]) + CPPFLAGS=$ac_previous_cppflags LDFLAGS=$ac_previous_ldflags ])
tools/configure +8 -56
95 lines not on this page · raw message
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH for-next v2 v2 1/5] build/m4: make python_devel.m4 work with both python 2 and 3
Quoted 7 lines · Wei Liu
Do the following:
1. Change the form of "print".
2. UseAC_CHECK_FUNCto avoid the need to generate library name.
3. Remove unused stuff.Signed-off-by Wei Liu <[email protected]>
Thanks.
Quoted 5 lines
---
I doubt the non-python-config branch works, because the paths generated seem rather off. It definitely doesn't work on my machine, but I don't know how other systems could possibly be configured before the existence of python-config.
It's probably fine. On Arch, the paths for python2 looks fine. With python3, that would generates "-L/usr/lib/python3.7/config" which doesn't exist, but the python lib is in /usr/lib, so it's probably going to work just fine. Anyway, an installation of python3 without its python-config is probably broken.
Signature · 6 lines
Anthony PERARD _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 2/5 libxl: make python scripts work with python 2.6 and up
In reply to Wei LiuGo through the transformations suggested by 2to3 and pick the necessary ones.
Use sys.stderr.write to avoid importing from __future__.
--- tools/libxl/gentest.py | 2 +- tools/libxl/gentypes.py | 10 +++++----- tools/libxl/idl.py | 13 ++++++------- 3 files changed, 12 insertions(+), 13 deletions(-)
tools/libxl/gentest.py +1 -1
@@ -86,7 +86,7 @@ def gen_rand_init(ty, v, indent = " ", parent = None): if __name__ == '__main__': if len(sys.argv) < 3: - print >>sys.stderr, "Usage: gentest.py <idl> <implementation>" + sys.stderr.write("Usage: gentest.py <idl> <implementation>\n") sys.exit(1) random.seed(os.getenv('LIBXL_TESTIDL_SEED'))
tools/libxl/gentypes.py +5 -5
@@ -576,14 +576,14 @@ def libxl_C_enum_from_string(ty, str, e, indent = " "): if __name__ == '__main__': if len(sys.argv) != 6: - print >>sys.stderr, "Usage: gentypes.py <idl> <header> <header-private> <header-json> <implementation>" + sys.stderr.write("Usage: gentypes.py <idl> <header> <header-private> <header-json> <implementation>\n") sys.exit(1) (_, idlname, header, header_private, header_json, impl) = sys.argv (builtins,types) = idl.parse(idlname) - print "outputting libxl type definitions to %s" % header + print("outputting libxl type definitions to %s" % header) f = open(header, "w")
@@ -633,7 +633,7 @@ if __name__ == '__main__': f.write("""#endif /* %s */\n""" % (header_define)) f.close() - print "outputting libxl JSON definitions to %s" % header_json + print("outputting libxl JSON definitions to %s" % header_json) f = open(header_json, "w")
@@ -657,7 +657,7 @@ if __name__ == '__main__': f.write("""#endif /* %s */\n""" % header_json_define) f.close() - print "outputting libxl type internal definitions to %s" % header_private + print("outputting libxl type internal definitions to %s" % header_private) f = open(header_private, "w")
@@ -683,7 +683,7 @@ if __name__ == '__main__': f.write("""#endif /* %s */\n""" % header_json_define) f.close() - print "outputting libxl type implementations to %s" % impl + print("outputting libxl type implementations to %s" % impl) f = open(impl, "w") f.write("""
tools/libxl/idl.py +6 -7
@@ -11,7 +11,7 @@ DIR_BOTH = 3 _default_namespace = "" def namespace(s): if type(s) != str: - raise TypeError, "Require a string for the default namespace." + raise TypeError("Require a string for the default namespace.") global _default_namespace _default_namespace = s
@@ -346,7 +346,7 @@ class OrderedDict(dict): return [(x,self[x]) for x in self.__ordered] def parse(f): - print >>sys.stderr, "Parsing %s" % f + sys.stderr.write("Parsing %s\n" % f) globs = {} locs = OrderedDict()
@@ -362,11 +362,10 @@ def parse(f): globs[n] = t try: - execfile(f, globs, locs) - except SyntaxError,e: - raise SyntaxError, \ - "Errors were found at line %d while processing %s:\n\t%s"\ - %(e.lineno,f,e.text) + exec(compile(open(f).read(), f, 'exec'), globs, locs) + except SyntaxError as e: + raise SyntaxError("Errors were found at line %d while processing %s:\n\t%s"\ + %(e.lineno,f,e.text)) types = [t for t in locs.ordered_values() if isinstance(t,Type)]
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH for-next v2 v2 2/5] libxl: make python scripts work with python 2.6 and up
tools/libxl/idl.py def parse(f): · line 368
Quoted 13 lines · Wei Liu
tools/libxl/idl.py +4
-5@@ -362,11 +362,10 @@ def parse(f): globs[n] = t try: - execfile(f, globs, locs) - except SyntaxError,e: - raise SyntaxError, \ - "Errors were found at line %d while processing %s:\n\t%s"\ - %(e.lineno,f,e.text) + exec(compile(open(f).read(), f, 'exec'), globs, locs) + except SyntaxError as e: + raise SyntaxError("Errors were found at line %d while processing %s:\n\t%s"\ + %(e.lineno,f,e.text))
As you're editing this, the \ can go, and it would be nice to properly indent the second line. (Even better if we can get spaces in sensible places).
Otherwise, Reviewed-by: Andrew Cooper <[email protected]>
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH for-next v2 v2 2/5] libxl: make python scripts work with python 2.6 and up
In reply to Wei LiuQuoted 33 lines · Wei Liu
Go through the transformations suggested by 2to3 and pick the necessary ones.
Use sys.stderr.write to avoid importing from
__future__.Signed-off-by Wei Liu <[email protected]>--- tools/libxl/gentest.py | 2 +- tools/libxl/gentypes.py | 10 +++++----- tools/libxl/idl.py | 13 ++++++------- 3 files changed, 12 insertions(+), 13 deletions(-)tools/libxl/gentest.py +1
-1@@ -86,7 +86,7 @@ def gen_rand_init(ty, v, indent = " ", parent = None): if __name__ == '__main__': if len(sys.argv) < 3: - print >>sys.stderr, "Usage: gentest.py <idl> <implementation>" + sys.stderr.write("Usage: gentest.py <idl> <implementation>\n") sys.exit(1) random.seed(os.getenv('LIBXL_TESTIDL_SEED'))tools/libxl/gentypes.py +0
-0@@ -576,14 +576,14 @@ def libxl_C_enum_from_string(ty, str, e, indent = " "): if __name__ == '__main__': if len(sys.argv) != 6:
tools/libxl/gentypes.py def libxl_C_enum_from_string(ty, str, e, indent = " "): · line 586
Quoted 10 lines
tools/libxl/gentypes.py +2
-2- print >>sys.stderr, "Usage: gentypes.py <idl> <header> <header-private> <header-json> <implementation>" + sys.stderr.write("Usage: gentypes.py <idl> <header> <header-private> <header-json> <implementation>\n") sys.exit(1) (_, idlname, header, header_private, header_json, impl) = sys.argv (builtins,types) = idl.parse(idlname) - print "outputting libxl type definitions to %s" % header + print("outputting libxl type definitions to %s" % header)
Note that print is *not* a function in python 2, unless you..
from __future__ import print_function
...which should not be avoided.
It looks like print() just works as a function, but it's not, and I would recommend against introducing this kind of misleading code.
In print("hallo"), the parentheses are part of an expression, like in
((1+ 1) * 2)
Other syntax with parentheses creates tuples.
Python 2.7.13 (>>> replaced with ===)
=== type(())
<type 'tuple'>
=== type((1+1))
<type 'int'>
=== type((1, 2))
<type 'tuple'>
=== print () <- tuple
()
=== print (1+1) <- expression
2
=== print (1, 2) <- tuple
(1, 2)
=== from __future__ import print_function
=== print()
=== print(1+1)
2
=== print(1, 2)
1 2
Quoted 43 lines
tools/libxl/gentypes.py +3
-3f = open(header, "w")@@ -633,7 +633,7 @@ if __name__ == '__main__': f.write("""#endif /* %s */\n""" % (header_define)) f.close() - print "outputting libxl JSON definitions to %s" % header_json + print("outputting libxl JSON definitions to %s" % header_json) f = open(header_json, "w")@@ -657,7 +657,7 @@ if __name__ == '__main__': f.write("""#endif /* %s */\n""" % header_json_define) f.close() - print "outputting libxl type internal definitions to %s" % header_private + print("outputting libxl type internal definitions to %s" % header_private) f = open(header_private, "w")@@ -683,7 +683,7 @@ if __name__ == '__main__': f.write("""#endif /* %s */\n""" % header_json_define) f.close() - print "outputting libxl type implementations to %s" % impl + print("outputting libxl type implementations to %s" % impl) f = open(impl, "w") f.write("""tools/libxl/idl.py +1
-1@@ -11,7 +11,7 @@ DIR_BOTH = 3 _default_namespace = "" def namespace(s): if type(s) != str: - raise TypeError, "Require a string for the default namespace." + raise TypeError("Require a string for the default namespace.") global _default_namespace _default_namespace = s
tools/libxl/idl.py class OrderedDict(dict): · line 349
Quoted 5 lines
tools/libxl/idl.py +1
-1@@ -346,7 +346,7 @@ class OrderedDict(dict): return [(x,self[x]) for x in self.__ordered] def parse(f): - print >>sys.stderr, "Parsing %s" % f + sys.stderr.write("Parsing %s\n" % f)
And if you have it from future anyway...
print("Parsing", f, file=sys.stderr)
Quoted 9 lines
tools/libxl/idl.py +0
-2globs = {} locs = OrderedDict()@@ -362,11 +362,10 @@ def parse(f): globs[n] = t try: - execfile(f, globs, locs) - except SyntaxError,e:
tools/libxl/idl.py def parse(f): · line 371
Quoted 10 lines
tools/libxl/idl.py +4
-3- raise SyntaxError, \ - "Errors were found at line %d while processing %s:\n\t%s"\ - %(e.lineno,f,e.text) + exec(compile(open(f).read(), f, 'exec'), globs, locs) + except SyntaxError as e: + raise SyntaxError("Errors were found at line %d while processing %s:\n\t%s"\ + %(e.lineno,f,e.text)) types = [t for t in locs.ordered_values() if isinstance(t,Type)]
Hans
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH for-next v2 v2 2/5] libxl: make python scripts work with python 2.6 and up
Quoted 42 lines · Hans van Kranenburg
Quoted 41 lines · Wei Liu
Go through the transformations suggested by 2to3 and pick the necessary ones.
Use sys.stderr.write to avoid importing from
__future__.Signed-off-by Wei Liu <[email protected]>--- tools/libxl/gentest.py | 2 +- tools/libxl/gentypes.py | 10 +++++----- tools/libxl/idl.py | 13 ++++++------- 3 files changed, 12 insertions(+), 13 deletions(-)tools/libxl/gentest.py +1
-1@@ -86,7 +86,7 @@ def gen_rand_init(ty, v, indent = " ", parent = None): if __name__ == '__main__': if len(sys.argv) < 3: - print >>sys.stderr, "Usage: gentest.py <idl> <implementation>" + sys.stderr.write("Usage: gentest.py <idl> <implementation>\n") sys.exit(1) random.seed(os.getenv('LIBXL_TESTIDL_SEED'))tools/libxl/gentypes.py +1
-1@@ -576,14 +576,14 @@ def libxl_C_enum_from_string(ty, str, e, indent = " "): if __name__ == '__main__': if len(sys.argv) != 6: - print >>sys.stderr, "Usage: gentypes.py <idl> <header> <header-private> <header-json> <implementation>" + sys.stderr.write("Usage: gentypes.py <idl> <header> <header-private> <header-json> <implementation>\n") sys.exit(1) (_, idlname, header, header_private, header_json, impl) = sys.argv (builtins,types) = idl.parse(idlname)
Quoted 9 lines
tools/libxl/gentypes.py def libxl_C_enum_from_string(ty, str, e, indent = " "): · line 586
Quoted 2 lines
tools/libxl/gentypes.py +1
-1- print "outputting libxl type definitions to %s" % header + print("outputting libxl type definitions to %s" % header)Note that print is *not* a function in python 2, unless you..
from
__future__importprint_function...which should not be avoided.
I know. print in 2 is a statement but in 3 is a function.
Yet this transformation is done by 2to3 _without_ importing print_function. I thought it was a blessed way of doing stuff. I went through those transformations. They looked correct (although I agree they were confusing).
Importing print_function was specifically avoided because we wanted 2.4 compatibility at first. But now I propose to bump the requirement to 2.6, the from __future__ import print_function may become available. I will need to check if that works with 2.6.
Quoted 77 lines
It looks like
print()just works as a function, but it's not, and I would recommend against introducing this kind of misleading code.In
print("hallo"), the parentheses are part of an expression, like in
((1+ 1) * 2)Other syntax with parentheses creates tuples.
Python 2.7.13 (>>> replaced with ===)
=== type(())
<type 'tuple'>
=== type((1+1))
<type 'int'>
===type((1, 2))
<type 'tuple'>=== print () <- tuple
()
=== print (1+1) <- expression
2
=== print (1, 2) <- tuple
(1, 2)=== from
__future__importprint_function
===print()=== print(1+1)
2
===print(1, 2)
1 2Quoted 45 lines
tools/libxl/gentypes.py +3
-3f = open(header, "w")@@ -633,7 +633,7 @@ if __name__ == '__main__': f.write("""#endif /* %s */\n""" % (header_define)) f.close() - print "outputting libxl JSON definitions to %s" % header_json + print("outputting libxl JSON definitions to %s" % header_json) f = open(header_json, "w")@@ -657,7 +657,7 @@ if __name__ == '__main__': f.write("""#endif /* %s */\n""" % header_json_define) f.close() - print "outputting libxl type internal definitions to %s" % header_private + print("outputting libxl type internal definitions to %s" % header_private) f = open(header_private, "w")@@ -683,7 +683,7 @@ if __name__ == '__main__': f.write("""#endif /* %s */\n""" % header_json_define) f.close() - print "outputting libxl type implementations to %s" % impl + print("outputting libxl type implementations to %s" % impl) f = open(impl, "w") f.write("""tools/libxl/idl.py +1
-1@@ -11,7 +11,7 @@ DIR_BOTH = 3 _default_namespace = "" def namespace(s): if type(s) != str: - raise TypeError, "Require a string for the default namespace." + raise TypeError("Require a string for the default namespace.") global _default_namespace _default_namespace = s@@ -346,7 +346,7 @@ class OrderedDict(dict): return [(x,self[x]) for x in self.__ordered]
Quoted 8 lines
tools/libxl/idl.py class OrderedDict(dict): · line 349
Quoted 3 lines
tools/libxl/idl.py +1
-1def parse(f): - print >>sys.stderr, "Parsing %s" % f + sys.stderr.write("Parsing %s\n" % f)And if you have it from future anyway...
print("Parsing", f, file=sys.stderr)
Yeah, the first version had this. But then I deleted the import and rewrote the patch.
Wei.
Quoted 16 lines
Quoted 16 lines
tools/libxl/idl.py +4
-5globs = {} locs = OrderedDict()@@ -362,11 +362,10 @@ def parse(f): globs[n] = t try: - execfile(f, globs, locs) - except SyntaxError,e: - raise SyntaxError, \ - "Errors were found at line %d while processing %s:\n\t%s"\ - %(e.lineno,f,e.text) + exec(compile(open(f).read(), f, 'exec'), globs, locs) + except SyntaxError as e: + raise SyntaxError("Errors were found at line %d while processing %s:\n\t%s"\ + %(e.lineno,f,e.text))
Quoted 5 lines
tools/libxl/idl.py def parse(f): · line 371
Quoted 3 lines
tools/libxl/idl.py +0
-0types = [t for t in locs.ordered_values() if isinstance(t,Type)]Hans
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH for-next v2 v2 2/5] libxl: make python scripts work with python 2.6 and up
Quoted 5 lines · Wei Liu
Importing
print_functionwas specifically avoided because we wanted 2.4 compatibility at first. But now I propose to bump the requirement to 2.6, the from__future__importprint_functionmay become available. I will need to check if that works with 2.6.
According to:
https://docs.python.org/2.6/library/__future__.html
from __future__ import print_function should be available in 2.6.
I will use it and fix the sys.stderr.write in the next version.
Wei.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 3/5 pygrub: convert python scripts to work with 2.6 and up
In reply to Wei LiuRun 2to3 and pick the sensible suggestions.
--- tools/pygrub/src/ExtLinuxConf.py | 15 ++++++++------- tools/pygrub/src/GrubConf.py | 36 ++++++++++++++++++------------------ tools/pygrub/src/LiloConf.py | 15 ++++++++------- 3 files changed, 34 insertions(+), 32 deletions(-)
tools/pygrub/src/ExtLinuxConf.py +8 -7
@@ -32,7 +32,8 @@ class ExtLinuxImage(object): self.lines = [] self.path = path self.root = "" - map(self.set_from_line, lines) + for line in lines: + self.set_from_line(line) def set_from_line(self, line, replace = None): (com, arg) = GrubConf.grub_exact_split(line, 2)
@@ -67,7 +68,7 @@ class ExtLinuxImage(object): setattr(self, "initrd", a.replace("initrd=", "")) arg = arg.replace(a, "") - if com is not None and self.commands.has_key(com): + if com is not None and com in self.commands: if self.commands[com] is not None: setattr(self, self.commands[com], re.sub('^"(.+)"$', r"\1", arg.strip())) else:
@@ -136,7 +137,7 @@ class ExtLinuxConfigFile(object): def parse(self, buf = None): if buf is None: if self.filename is None: - raise ValueError, "No config file defined to parse!" + raise ValueError("No config file defined to parse!") f = open(self.filename, 'r') lines = f.readlines()
@@ -167,7 +168,7 @@ class ExtLinuxConfigFile(object): (com, arg) = GrubConf.grub_exact_split(l, 2) com = com.lower() - if self.commands.has_key(com): + if com in self.commands: if self.commands[com] is not None: setattr(self, self.commands[com], arg.strip()) else:
@@ -207,8 +208,8 @@ class ExtLinuxConfigFile(object): if __name__ == "__main__": if len(sys.argv) < 2: - raise RuntimeError, "Need a configuration file to read" + raise RuntimeError("Need a configuration file to read") g = ExtLinuxConfigFile(sys.argv[1]) for i in g.images: - print i - print g.default + print(i) + print(g.default)
tools/pygrub/src/GrubConf.py +18 -18
@@ -44,7 +44,7 @@ def get_path(s): return (None, s) idx = s.find(')') if idx == -1: - raise ValueError, "Unable to find matching ')'" + raise ValueError("Unable to find matching ')'") d = s[:idx] return (GrubDiskPart(d), s[idx + 1:])
@@ -100,7 +100,7 @@ class _GrubImage(object): " initrd: %s\n" %(self.title, self.root, self.kernel, self.args, self.initrd)) def _parse(self, lines): - map(self.set_from_line, lines) + list(map(self.set_from_line, lines)) def reset(self, lines): self._root = self._initrd = self._kernel = self._args = None
@@ -141,7 +141,7 @@ class GrubImage(_GrubImage): def set_from_line(self, line, replace = None): (com, arg) = grub_exact_split(line, 2) - if self.commands.has_key(com): + if com in self.commands: if self.commands[com] is not None: setattr(self, self.commands[com], arg.strip()) else:
@@ -177,7 +177,7 @@ class _GrubConfigFile(object): self.parse() def parse(self, buf = None): - raise RuntimeError, "unimplemented parse function" + raise RuntimeError("unimplemented parse function") def hasPasswordAccess(self): return self.passwordAccess
@@ -201,7 +201,7 @@ class _GrubConfigFile(object): import crypt if crypt.crypt(password, pwd[1]) == pwd[1]: return True - except Exception, e: + except Exception as e: self.passExc = "Can't verify password: %s" % str(e) return False
@@ -213,7 +213,7 @@ class _GrubConfigFile(object): def set(self, line): (com, arg) = grub_exact_split(line, 2) - if self.commands.has_key(com): + if com in self.commands: if self.commands[com] is not None: setattr(self, self.commands[com], arg.strip()) else:
@@ -233,7 +233,7 @@ class _GrubConfigFile(object): self._default = val if self._default < 0: - raise ValueError, "default must be positive number" + raise ValueError("default must be positive number") default = property(_get_default, _set_default) def set_splash(self, val):
@@ -265,7 +265,7 @@ class GrubConfigFile(_GrubConfigFile): def parse(self, buf = None): if buf is None: if self.filename is None: - raise ValueError, "No config file defined to parse!" + raise ValueError("No config file defined to parse!") f = open(self.filename, 'r') lines = f.readlines()
@@ -296,7 +296,7 @@ class GrubConfigFile(_GrubConfigFile): continue (com, arg) = grub_exact_split(l, 2) - if self.commands.has_key(com): + if com in self.commands: if self.commands[com] is not None: setattr(self, self.commands[com], arg.strip()) else:
@@ -328,7 +328,7 @@ class Grub2Image(_GrubImage): if com == "set": (com,arg) = grub2_handle_set(arg) - if self.commands.has_key(com): + if com in self.commands: if self.commands[com] is not None: setattr(self, self.commands[com], arg.strip()) else:
@@ -364,7 +364,7 @@ class Grub2ConfigFile(_GrubConfigFile): def parse(self, buf = None): if buf is None: if self.filename is None: - raise ValueError, "No config file defined to parse!" + raise ValueError("No config file defined to parse!") f = open(self.filename, 'r') lines = f.readlines()
@@ -398,7 +398,7 @@ class Grub2ConfigFile(_GrubConfigFile): title_match = re.match('^menuentry ["\'](.*?)["\'] (.*){', l) if title_match: if img is not None: - raise RuntimeError, "syntax error: cannot nest menuentry (%d %s)" % (len(img),img) + raise RuntimeError("syntax error: cannot nest menuentry (%d %s)" % (len(img),img)) img = [] title = title_match.group(1) continue
@@ -413,7 +413,7 @@ class Grub2ConfigFile(_GrubConfigFile): menu_level -= 1 continue else: - raise RuntimeError, "syntax error: closing brace without menuentry" + raise RuntimeError("syntax error: closing brace without menuentry") self.add_image(Grub2Image(title, img)) img = None
@@ -428,7 +428,7 @@ class Grub2ConfigFile(_GrubConfigFile): if com == "set": (com,arg) = grub2_handle_set(arg) - if self.commands.has_key(com): + if com in self.commands: if self.commands[com] is not None: arg_strip = arg.strip() if arg_strip == "${saved_entry}" or arg_strip == "${next_entry}":
@@ -443,7 +443,7 @@ class Grub2ConfigFile(_GrubConfigFile): logging.warning("Unknown directive %s" %(com,)) if img is not None: - raise RuntimeError, "syntax error: end of file with open menuentry(%d %s)" % (len(img),img) + raise RuntimeError("syntax error: end of file with open menuentry(%d %s)" % (len(img),img)) if self.hasPassword(): self.setPasswordAccess(False)
@@ -462,12 +462,12 @@ class Grub2ConfigFile(_GrubConfigFile): if __name__ == "__main__": if len(sys.argv) < 3: - raise RuntimeError, "Need a grub version (\"grub\" or \"grub2\") and a grub.conf or grub.cfg to read" + raise RuntimeError("Need a grub version (\"grub\" or \"grub2\") and a grub.conf or grub.cfg to read") if sys.argv[1] == "grub": g = GrubConfigFile(sys.argv[2]) elif sys.argv[1] == "grub2": g = Grub2ConfigFile(sys.argv[2]) else: - raise RuntimeError, "Unknown config type %s" % sys.argv[1] + raise RuntimeError("Unknown config type %s" % sys.argv[1]) for i in g.images: - print i #, i.title, i.root, i.kernel, i.args, i.initrd + print(i) #, i.title, i.root, i.kernel, i.args, i.initrd
tools/pygrub/src/LiloConf.py +8 -7
@@ -24,12 +24,13 @@ class LiloImage(object): self.lines = [] self.path = path self.root = "" - map(self.set_from_line, lines) + for line in lines: + self.set_from_line(line) def set_from_line(self, line, replace = None): (com, arg) = GrubConf.grub_exact_split(line, 2) - if self.commands.has_key(com): + if com in self.commands: if self.commands[com] is not None: setattr(self, self.commands[com], re.sub('^"(.+)"$', r"\1", arg.strip())) else:
@@ -97,7 +98,7 @@ class LiloConfigFile(object): def parse(self, buf = None): if buf is None: if self.filename is None: - raise ValueError, "No config file defined to parse!" + raise ValueError("No config file defined to parse!") f = open(self.filename, 'r') lines = f.readlines()
@@ -127,7 +128,7 @@ class LiloConfigFile(object): continue (com, arg) = GrubConf.grub_exact_split(l, 2) - if self.commands.has_key(com): + if com in self.commands: if self.commands[com] is not None: setattr(self, self.commands[com], arg.strip()) else:
@@ -170,8 +171,8 @@ class LiloConfigFile(object): if __name__ == "__main__": if len(sys.argv) < 2: - raise RuntimeError, "Need a lilo.conf to read" + raise RuntimeError("Need a lilo.conf to read") g = LiloConfigFile(sys.argv[1]) for i in g.images: - print i #, i.title, i.root, i.kernel, i.args, i.initrd - print g.default + print(i) #, i.title, i.root, i.kernel, i.args, i.initrd + print(g.default)
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH for-next v2 v2 3/5] pygrub: convert python scripts to work with 2.6 and up
tools/pygrub/src/GrubConf.py class _GrubImage(object): · line 103
Quoted 10 lines · Wei Liu
tools/pygrub/src/GrubConf.py +1
-1@@ -100,7 +100,7 @@ class _GrubImage(object): " initrd: %s\n" %(self.title, self.root, self.kernel, self.args, self.initrd)) def _parse(self, lines): - map(self.set_from_line, lines) + list(map(self.set_from_line, lines))
Another site which wants a transform into a for loop.
tools/pygrub/src/GrubConf.py class Grub2ConfigFile(_GrubConfigFile): · line 465
Quoted 9 lines
tools/pygrub/src/GrubConf.py +1
-1def reset(self, lines): self._root = self._initrd = self._kernel = self._args = None@@ -462,12 +462,12 @@ class Grub2ConfigFile(_GrubConfigFile): if __name__ == "__main__": if len(sys.argv) < 3: - raise RuntimeError, "Need a grub version (\"grub\" or \"grub2\") and a grub.conf or grub.cfg to read" + raise RuntimeError("Need a grub version (\"grub\" or \"grub2\") and a grub.conf or grub.cfg to read")
As you're editing this anyway, it would be better to switch to
RuntimeError('Need a grub version ("grub" or "grub2") and a grub.conf or grub.cfg to read')
To drop the unnecessary escaping.
Otherwise, Reviewed-by: Andrew Cooper <[email protected]>
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 4/5 pygrub: make it build with python 3
In reply to Wei LiuWith the help of two porting guides and cpython source code:
1. Use PyBytes to replace PyString counterparts.
2. Use PyVarObject_HEAD_INIT and provide compatibility for 2.5 and earlier.
3. Remove usage of Py_FindMethod.
4. Use new module initialisation routine.
For #3, Py_FindMethod was removed, yet an alternative wasn't documented. The code is the result of reverse-engineering cpython commit 6116d4a1d1
https://docs.python.org/3/howto/cporting.html
http://python3porting.com/cextensions.html
---
v2: use PyBytes.
--- tools/pygrub/src/fsimage/fsimage.c | 96 ++++++++++++++++++++++++++++++++++---- 1 file changed, 88 insertions(+), 8 deletions(-)
tools/pygrub/src/fsimage/fsimage.c +88 -8
@@ -26,11 +26,15 @@#include <xenfsimage.h> #include <stdlib.h> +#if PY_MAJOR_VERSION <3 #if (PYTHON_API_VERSION >=1011 )#define PY_PAD 0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L#else #define PY_PAD 0L,0L,0L,0L#endif +#else +#define PY_PAD 0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L,0L +#endif typedef struct fsimage_fs { PyObject_HEAD
@@ -66,12 +70,24 @@ fsimage_file_read(fsimage_file_t *file, PyObject *args, PyObject *kwargs) bufsize = size ? size :4096 ; -if ((buffer =PyString_FromStringAndSize (NULL , bufsize)) ==NULL ) + buffer = +#if PY_MAJOR_VERSION <3 +PyString_FromStringAndSize (NULL , bufsize); +#else +PyBytes_FromStringAndSize (NULL , bufsize); +#endif + +if (buffer ==NULL )return (NULL );while (1 ) {int err; -void *buf =PyString_AS_STRING (buffer) + bytesread; + void *buf = +#if PY_MAJOR_VERSION <3 +PyString_AS_STRING (buffer) + bytesread; +#else +PyBytes_AS_STRING (buffer) + bytesread; +#endif err =fsi_pread_file (file->file, buf, bufsize, bytesread + offset);
@@ -91,12 +107,20 @@ fsimage_file_read(fsimage_file_t *file, PyObject *args, PyObject *kwargs)if (bufsize ==0 )break ; }else { +#if PY_MAJOR_VERSION <3 if (_PyString_Resize (&buffer, bytesread + bufsize) <0 ) +#else + if (_PyBytes_Resize(&buffer, bytesread + bufsize) < 0) +#endif return (NULL ); } } +#if PY_MAJOR_VERSION <3 _PyString_Resize (&buffer, bytesread); +#else +_PyBytes_Resize (&buffer, bytesread); +#endif return (buffer); }
@@ -113,11 +137,13 @@ static struct PyMethodDef fsimage_file_methods[] = { { NULL, NULL, 0, NULL } }; +#if PY_MAJOR_VERSION <3 static PyObject *fsimage_file_getattr (fsimage_file_t *file,char *name) {return (Py_FindMethod (fsimage_file_methods, (PyObject *)file, name)); } +#endif static void fsimage_file_dealloc (fsimage_file_t *file)
@@ -128,16 +154,25 @@ fsimage_file_dealloc(fsimage_file_t *file)PyObject_DEL (file); } +/* Compatibility for 2.5 and earlier */ +#ifndef PyVarObject_HEAD_INIT +#define PyVarObject_HEAD_INIT (type, size) \ + PyObject_HEAD_INIT(type) size, +#endif +static char fsimage_file_type__doc__[] ="Filesystem image file" ;PyTypeObject fsimage_file_type = { -PyObject_HEAD_INIT (&PyType_Type) - 0, /* ob_size */ + PyVarObject_HEAD_INIT(&PyType_Type, 0)"xenfsimage.file" ,/* tp_name */ sizeof (fsimage_file_t),/* tp_size */ 0 ,/* tp_itemsize */ (destructor ) fsimage_file_dealloc,/* tp_dealloc */ 0 ,/* tp_print */ +#if PY_MAJOR_VERSION < 3 (getattrfunc) fsimage_file_getattr, /* tp_getattr */ +#else +0 ,/* tp_getattr */ +#endif0 ,/* tp_setattr */ 0 ,/* tp_compare */ 0 ,/* tp_repr */
@@ -151,7 +186,16 @@ PyTypeObject fsimage_file_type = {0 ,/* tp_setattro */ 0 ,/* tp_as_buffer */ Py_TPFLAGS_DEFAULT,/* tp_flags */ - fsimage_file_type__doc__, + fsimage_file_type__doc__,/* tp_doc */ +#if PY_MAJOR_VERSION >= 3 + 0, /* tp_traverse */ +0 ,/* tp_clear */ +0 ,/* tp_richcompare */ +0 ,/* tp_weaklistoffset */ +0 ,/* tp_iter */ +0 ,/* tp_iternext */ + fsimage_file_methods,/* tp_methods */ +#endifPY_PAD };
@@ -215,11 +259,13 @@ static struct PyMethodDef fsimage_fs_methods[] = { { NULL, NULL, 0, NULL } }; +#if PY_MAJOR_VERSION <3 static PyObject *fsimage_fs_getattr (fsimage_fs_t *fs,char *name) {return (Py_FindMethod (fsimage_fs_methods, (PyObject *)fs, name)); } +#endif static void fsimage_fs_dealloc (fsimage_fs_t *fs)
@@ -232,14 +278,17 @@ fsimage_fs_dealloc (fsimage_fs_t *fs)PyDoc_STRVAR (fsimage_fs_type__doc__,"Filesystem image" );PyTypeObject fsimage_fs_type = { -PyObject_HEAD_INIT (&PyType_Type) - 0, /* ob_size */ + PyVarObject_HEAD_INIT(&PyType_Type, 0)"xenfsimage.fs" ,/* tp_name */ sizeof (fsimage_fs_t),/* tp_size */ 0 ,/* tp_itemsize */ (destructor ) fsimage_fs_dealloc,/* tp_dealloc */ 0 ,/* tp_print */ +#if PY_MAJOR_VERSION < 3 (getattrfunc) fsimage_fs_getattr, /* tp_getattr */ +#else +0 ,/* tp_getattr */ +#endif0 ,/* tp_setattr */ 0 ,/* tp_compare */ 0 ,/* tp_repr */
@@ -253,7 +302,16 @@ PyTypeObject fsimage_fs_type = {0 ,/* tp_setattro */ 0 ,/* tp_as_buffer */ Py_TPFLAGS_DEFAULT,/* tp_flags */ - fsimage_fs_type__doc__, + fsimage_fs_type__doc__,/* tp_doc */ +#if PY_MAJOR_VERSION >= 3 + 0, /* tp_traverse */ +0 ,/* tp_clear */ +0 ,/* tp_richcompare */ +0 ,/* tp_weaklistoffset */ +0 ,/* tp_iter */ +0 ,/* tp_iternext */ + fsimage_fs_methods,/* tp_methods */ +#endifPY_PAD };
@@ -316,8 +374,30 @@ static struct PyMethodDef fsimage_module_methods[] = { { NULL, NULL, 0, NULL } }; +#if PY_MAJOR_VERSION >=3 +static struct PyModuleDef fsimage_module_def = { + PyModuleDef_HEAD_INIT, +"xenfsimage" ,/* m_name */ +"" ,/* m_doc */ +-1 ,/* m_size */ + fsimage_module_methods,/* m_methods */ +NULL ,/* m_reload */ +NULL ,/* m_traverse */ +NULL ,/* m_clear */ +NULL /* m_free */ +}; +#endif + PyMODINIT_FUNC +#if PY_MAJOR_VERSION >=3 +PyInit_xenfsimage (void ) +#else initxenfsimage(void) +#endif { +#if PY_MAJOR_VERSION <3 Py_InitModule ("xenfsimage" , fsimage_module_methods); +#else +return PyModule_Create (&fsimage_module_def); +#endif }
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH for-next v2 v2 4/5] pygrub: make it build with python 3
Quoted 34 lines · Wei Liu
tools/pygrub/src/fsimage/fsimage.c +12
-3@@ -128,16 +154,25 @@ fsimage_file_dealloc(fsimage_file_t *file)PyObject_DEL (file); } +/* Compatibility for 2.5 and earlier */ +#ifndef PyVarObject_HEAD_INIT +#define PyVarObject_HEAD_INIT (type, size) \ + PyObject_HEAD_INIT(type) size, +#endif +static char fsimage_file_type__doc__[] ="Filesystem image file" ;PyTypeObject fsimage_file_type = { -PyObject_HEAD_INIT (&PyType_Type) - 0, /* ob_size */ + PyVarObject_HEAD_INIT(&PyType_Type, 0)"xenfsimage.file" ,/* tp_name */ sizeof (fsimage_file_t),/* tp_size */ 0 ,/* tp_itemsize */ (destructor ) fsimage_file_dealloc,/* tp_dealloc */ 0 ,/* tp_print */ +#if PY_MAJOR_VERSION < 3 (getattrfunc) fsimage_file_getattr, /* tp_getattr */ +#else +0 ,/* tp_getattr */ +#endif0 ,/* tp_setattr */ 0 ,/* tp_compare */ 0 ,/* tp_repr */ @@ -151,7 +186,16 @@ PyTypeObject fsimage_file_type = {0 ,/* tp_setattro */ 0 ,/* tp_as_buffer */ Py_TPFLAGS_DEFAULT,/* tp_flags */ - fsimage_file_type__doc__, + fsimage_file_type__doc__, /* tp_doc */
tools/pygrub/src/fsimage/fsimage.c PyTypeObject fsimage_file_type = { · line 199
Quoted 10 lines
tools/pygrub/src/fsimage/fsimage.c +9
-0+#if PY_MAJOR_VERSION >=3 +0 ,/* tp_traverse */ +0 ,/* tp_clear */ +0 ,/* tp_richcompare */ +0 ,/* tp_weaklistoffset */ +0 ,/* tp_iter */ +0 ,/* tp_iternext */ + fsimage_file_methods,/* tp_methods */ +#endifPY_PAD
PY_PAD is very WTF. I've got no idea why it is necessary in the first place.
Either way, most of the fields are zero, so why not use named initialisation?
PyTypeObject fsimage_file_type = {
PyVarObject_HEAD_INIT(&PyType_Type, 0)
.tp_name = "xenfsimage.file",
.tp_size = sizeof(fsimage_file_t),
...
};
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 5/5 Update python requirement
In reply to Wei LiuCentOS 5, which was the reason of the 2.4 restriction, is EOL. CentOS 6 ships 2.6.
Bump the version to 2.6 in README. Now that all scripts are 3 compatible, remove the restriction on python 2 as well.
Update the check in configure.ac.
--- README | 10 ++-------- tools/configure | 8 ++++---- tools/configure.ac | 2 +- 3 files changed, 7 insertions(+), 13 deletions(-)
README +2 -8
@@ -46,7 +46,7 @@ provided by your OS distributor: - GCC 4.8 or later - GNU Binutils 2.24 or later * Development install of zlib (e.g., zlib-dev) - * Development install of Python 2, v2.4 or later (e.g., python-dev) + * Development install of Python v2.6 or later (e.g., python-dev) * Development install of curses (e.g., libncurses-dev) * Development install of openssl (e.g., openssl-dev) * Development install of x11 (e.g. xorg-x11-dev)
@@ -177,16 +177,10 @@ Python Runtime Libraries Various tools, such as pygrub, have the following runtime dependencies: - * Python 2, v2.4 or later. + * Python v2.6 or later. URL: http://www.python.org/ Debian: python -Note that the build system expects `python` to be python2. If your system -has `python` pointing to python3 (as in the case of Arch Linux or Anaconda), -you'll need to specify a path to a python2 binary when running configure: - - PYTHON=/usr/bin/python2 ./configure - Intel(R) Trusted Execution Technology Support =============================================
tools/configure +4 -4
@@ -7002,15 +7002,15 @@ if test x"${PYTHONPATH}" = x"no" then as_fn_error $? "Unable to find $PYTHON, please install $PYTHON" "$LINENO" 5 fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for python version >= 2.3 " >&5 -$as_echo_n "checking for python version >= 2.3 ... " >&6; } -`$PYTHON -c 'import sys; sys.exit(eval("sys.version_info < (2, 3)"))'` +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for python version >= 2.6 " >&5 +$as_echo_n "checking for python version >= 2.6 ... " >&6; } +`$PYTHON -c 'import sys; sys.exit(eval("sys.version_info < (2, 6)"))'` if test "$?" != "0" then python_version=`$PYTHON -V 2>&1` { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } - as_fn_error $? "$python_version is too old, minimum required version is 2.3" "$LINENO" 5 + as_fn_error $? "$python_version is too old, minimum required version is 2.6" "$LINENO" 5 else { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; }
tools/configure.ac +1 -1
@@ -358,7 +358,7 @@ AS_IF([echo "$PYTHON" | grep -q "^/"], [ ],[test -z "$PYTHON"], [PYTHON="python"], [AC_MSG_ERROR([PYTHON specified, but is not an absolute path])]) AX_PATH_PROG_OR_FAIL([PYTHONPATH], [$PYTHON]) -AX_CHECK_PYTHON_VERSION([2], [3]) +AX_CHECK_PYTHON_VERSION([2], [6]) AS_IF([test "$cross_compiling" != yes], [ AX_CHECK_PYTHON_DEVEL()
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH for-next v2 v2 5/5] Update python requirement
Quoted 9 lines · Wei Liu
CentOS 5, which was the reason of the 2.4 restriction, is EOL. CentOS 6 ships 2.6.
Bump the version to 2.6 in README. Now that all scripts are 3 compatible, remove the restriction on python 2 as well.
Update the check in configure.ac.
Signed-off-by Wei Liu <[email protected]>
I think this patch should be first in the series, otherwise there is going to be python code non-compatible with 2.4 before it is deemed too old ;-).
Signature · 6 lines
Anthony PERARD _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [PATCH for-next v2 v2 5/5] Update python requirement
Quoted 7 lines · Anthony PERARD
Quoted 6 lines · Wei Liu
CentOS 5, which was the reason of the 2.4 restriction, is EOL. CentOS 6 ships 2.6.
Bump the version to 2.6 in README. Now that all scripts are 3 compatible, remove the restriction on python 2 as well.
Quoted 7 lines
Quoted 3 lines
Update the check in configure.ac.
Signed-off-by Wei Liu <[email protected]>I think this patch should be first in the series, otherwise there is going to be python code non-compatible with 2.4 before it is deemed too old ;-).
Works for me.
Wei.
Quoted 3 lines
Signature · 2 lines
-- Anthony PERARD
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 3 lines · Wei Liu
This series makes tools work with Python 2 and 3.
No more RFC because I have tested it on my testbox.
Spoke too soon. The testbox wasn't using the files I installed. I will retest tomorrow.
Wei.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel