Skip to contentSkip to conversation

More python fixes

PATCH In review Wei Liu 4 participants 12 messages Apr 1, 2019 – Apr 2, 2019

lore
Share & tools
lore

Wei Liu

author
#0 lore raw

cover More python fixes

Wei Liu (4):
pygrub: fix message in grub parser pygrub/grub: always use integer for default entry
pygrub: decode string in Python 3 tools/ocaml: make python scripts 2 and 3 compatible

 tools/ocaml/libs/xentoollog/genlevels.py |  5 ++++-
 tools/ocaml/libs/xl/genwrap.py           | 17 ++++++++++-------
 tools/pygrub/src/GrubConf.py             |  8 ++++++--
 tools/pygrub/src/pygrub                  |  5 ++++-
 4 files changed, 24 insertions(+), 11 deletions(-)
Signature · 7 lines
2.20.1


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
author
#1 lore raw

PATCH 1/4 pygrub: fix message in grub parser

The code suggests 0 is allowed. Zero is not a positive number.

---
Backport candadiate.

---
 tools/pygrub/src/GrubConf.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/pygrub/src/GrubConf.py +1 -1
index f8d3799dc0..0204d410ac 100644
@@ -236,7 +236,7 @@ class _GrubConfigFile(object):
             self._default = val
 
         if self._default < 0:
-            raise ValueError("default must be positive number")
+            raise ValueError("default must be non-negative number")
     default = property(_get_default, _set_default)
 
     def set_splash(self, val):
Signature · 7 lines
2.20.1


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Acked-by on v1 1/4
#2 lore raw

Re: [PATCH 1/4] pygrub: fix message in grub parser

Quoted 3 lines · Wei Liu

The code suggests 0 is allowed. Zero is not a positive number.

I suspect you are opening a can of number theory worms with that commit message, but the overall change is an improvement.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#3 lore raw

PATCH 2/4 pygrub/grub: always use integer for default entry

In reply to Wei Liu

The original code set the default to either a string or an integer
(0) and relies on a Python 2 specific behaviour to work (integer is allowed to be compared to string in Python 2 but not 3).

Always use integer. The caller (pygrub) already has code to handle that.

---
 tools/pygrub/src/GrubConf.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
tools/pygrub/src/GrubConf.py +5 -1
index 0204d410ac..594139bac7 100644
@@ -233,7 +233,11 @@ class _GrubConfigFile(object):
         if val == "saved":
             self._default = 0
         else:
-            self._default = val
+            try:
+                self._default = int(val)
+            except ValueError:
+                logging.warning("Invalid value %s, setting default to 0" %(val,))
+                self._default = 0
 
         if self._default < 0:
             raise ValueError("default must be non-negative number")
Signature · 7 lines
2.20.1


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Acked-by on v1 2/4
#4 lore raw

Re: [PATCH 2/4] pygrub/grub: always use integer for default entry

Quoted 9 lines · Wei Liu

The original code set the default to either a string or an integer
(0) and relies on a Python 2 specific behaviour to work (integer is allowed to be compared to string in Python 2 but not 3).

Always use integer. The caller (pygrub) already has code to handle that.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#5 lore raw

PATCH 3/4 pygrub: decode string in Python 3

In reply to Wei Liu

String is unicode in 3 but bytes in 2. We need to call decode function when using Python 3.

---
 tools/pygrub/src/pygrub | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
tools/pygrub/src/pygrub +4 -1
index dbdce315c6..23312eae76 100755
@@ -457,7 +457,10 @@ class Grub:
         # limit read size to avoid pathological cases
         buf = f.read(FS_READ_MAX)
         del f
-        self.cf.parse(buf)
+        if sys.version_info[0] < 3:
+            self.cf.parse(buf)
+        else:
+            self.cf.parse(buf.decode())
 
     def image_index(self):
         if isinstance(self.cf.default, int):
Signature · 7 lines
2.20.1


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Acked-by on v1 3/4
#6 lore raw

Re: [PATCH 3/4] pygrub: decode string in Python 3

Quoted 5 lines · Wei Liu

String is unicode in 3 but bytes in 2. We need to call decode function when using Python 3.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#7 lore raw

PATCH 4/4 tools/ocaml: make python scripts 2 and 3 compatible

In reply to Wei Liu

1. Explicitly import reduce because that's required in 3.
2. Change print to function.
3. Eliminate invocations of has_key.

---
 tools/ocaml/libs/xentoollog/genlevels.py |  5 ++++-
 tools/ocaml/libs/xl/genwrap.py           | 17 ++++++++++-------
 2 files changed, 14 insertions(+), 8 deletions(-)
tools/ocaml/libs/xentoollog/genlevels.py +4 -1
index 8c233c59b1..f9cf853e26 100755
@@ -1,6 +1,9 @@
 #!/usr/bin/python
 
+from __future__ import print_function
+
 import sys
+from functools import reduce
 
 def read_levels():
 	f = open('../../../libs/toollog/include/xentoollog.h', 'r')
@@ -93,7 +96,7 @@ def autogen_header(open_comment, close_comment):
 
 if __name__ == '__main__':
 	if len(sys.argv) < 3:
-		print >>sys.stderr, "Usage: genlevels.py <mli> <ml> <c-inc>"
+		print("Usage: genlevels.py <mli> <ml> <c-inc>", file=sys.stderr)
 		sys.exit(1)
 
 	levels, olevels = read_levels()
tools/ocaml/libs/xl/genwrap.py +10 -7
index 815c1cb0e3..7bf26bdcd8 100644
@@ -1,6 +1,9 @@
 #!/usr/bin/python
 
+from __future__ import print_function
+
 import sys,os
+from functools import reduce
 
 import idl
 
@@ -78,7 +81,7 @@ def ocaml_type_of(ty):
     elif isinstance(ty,idl.Array):
         return "%s array" % ocaml_type_of(ty.elem_type)
     elif isinstance(ty,idl.Builtin):
-        if not builtins.has_key(ty.typename):
+        if ty.typename not in builtins:
             raise NotImplementedError("Unknown Builtin %s (%s)" % (ty.typename, type(ty)))
         typename,_,_ = builtins[ty.typename]
         if not typename:
@@ -251,7 +254,7 @@ def gen_ocaml_ml(ty, interface, indent=""):
             else:
                 s += "\texternal default : ctx -> %sunit -> t = \"stub_libxl_%s_init\"\n" % (union_args, ty.rawname)
 
-        if functions.has_key(ty.rawname):
+        if ty.rawname in functions:
             for name,args in functions[ty.rawname]:
                 s += "\texternal %s : " % name
                 s += " -> ".join(args)
@@ -278,7 +281,7 @@ def c_val(ty, c, o, indent="", parent = None):
         else:
             s += "%s = Int_val(%s);" % (c, o)
     elif isinstance(ty,idl.Builtin):
-        if not builtins.has_key(ty.typename):
+        if ty.typename not in builtins:
             raise NotImplementedError("Unknown Builtin %s (%s)" % (ty.typename, type(ty)))
         _,fn,_ = builtins[ty.typename]
         if not fn:
@@ -375,7 +378,7 @@ def ocaml_Val(ty, o, c, indent="", parent = None):
         else:
             s += "%s = Val_int(%s);" % (o, c)
     elif isinstance(ty,idl.Builtin):
-        if not builtins.has_key(ty.typename):
+        if ty.typename not in builtins:
             raise NotImplementedError("Unknown Builtin %s (%s)" % (ty.typename, type(ty)))
         _,_,fn = builtins[ty.typename]
         if not fn:
@@ -520,7 +523,7 @@ def autogen_header(open_comment, close_comment):
 
 if __name__ == '__main__':
     if len(sys.argv) < 4:
-        print >>sys.stderr, "Usage: genwrap.py <idl> <mli> <ml> <c-inc>"
+        print("Usage: genwrap.py <idl> <mli> <ml> <c-inc>", file=sys.stderr)
         sys.exit(1)
 
     (_,types) = idl.parse(sys.argv[1])
@@ -533,7 +536,7 @@ if __name__ == '__main__':
 
     for t in blacklist:
         if t not in [ty.rawname for ty in types]:
-            print "unknown type %s in blacklist" % t
+            print("unknown type %s in blacklist" % t)
 
     types = [ty for ty in types if not ty.rawname in blacklist]
 
@@ -564,7 +567,7 @@ if __name__ == '__main__':
             cinc.write("\n")
         cinc.write(gen_Val_ocaml(ty))
         cinc.write("\n")
-        if functions.has_key(ty.rawname):
+        if ty.rawname in functions:
             cinc.write(gen_c_stub_prototype(ty, functions[ty.rawname]))
             cinc.write("\n")
         if ty.init_fn is not None:
Signature · 7 lines
2.20.1


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Reviewed-by on v1 4/4
#8 lore raw

Re: [PATCH 4/4] tools/ocaml: make python scripts 2 and 3 compatible

Quoted 6 lines · Wei Liu

1. Explicitly import reduce because that's required in 3.
2. Change print to function.
3. Eliminate invocations of has_key.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Acked-by on v1 4/4
#9 lore raw

Re: [PATCH 4/4] tools/ocaml: make python scripts 2 and 3 compatible

In reply to Wei Liu
Quoted 10 lines · Wei Liu

1. Explicitly import reduce because that's required in 3.
2. Change print to function.
3. Eliminate invocations of has_key.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#10 lore raw
In reply to Wei Liu
Quoted 11 lines · Wei Liu

Wei Liu (4):
pygrub: fix message in grub parser pygrub/grub: always use integer for default entry
pygrub: decode string in Python 3 tools/ocaml: make python scripts 2 and 3 compatible

tools/ocaml/libs/xentoollog/genlevels.py |  5 ++++-
tools/ocaml/libs/xl/genwrap.py           | 17 ++++++++++-------
tools/pygrub/src/GrubConf.py             |  8 ++++++--
tools/pygrub/src/pygrub                  |  5 ++++-
4 files changed, 24 insertions(+), 11 deletions(-)

There is a bit missing compared to my original patch which is to convert the string back to bytes for the final write out and avoid the error

Traceback (most recent call last):
File "/tmp/xencode/usr/libexec/xen/bin/pygrub", line 967, in <module>
os.write(fd, ostring)
TypeError: a bytes-like object is required, not 'str'

The attached patch works for me in a quick test on python3 though I haven't tested it on python2 yet.

Michael Young

tools/pygrub/src/pygrub renamed from tools/pygrub/src/pygrub.orig +4 -1
@@ -963,5 +963,8 @@
         ostring = format_simple(bootcfg["kernel"], bootcfg["ramdisk"], args, "\0")
 
     sys.stdout.flush()
-    os.write(fd, ostring)
+    if sys.version_info[0] < 3:
+        os.write(fd, ostring)
+    else:
+        os.write(fd, ostring.encode())
     

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

author
#11 lore raw
Quoted 30 lines · MICHAEL A.
Quoted 11 lines · Wei Liu

Wei Liu (4):
pygrub: fix message in grub parser pygrub/grub: always use integer for default entry
pygrub: decode string in Python 3 tools/ocaml: make python scripts 2 and 3 compatible

tools/ocaml/libs/xentoollog/genlevels.py |  5 ++++-
tools/ocaml/libs/xl/genwrap.py           | 17 ++++++++++-------
tools/pygrub/src/GrubConf.py             |  8 ++++++--
tools/pygrub/src/pygrub                  |  5 ++++-
4 files changed, 24 insertions(+), 11 deletions(-)

There is a bit missing compared to my original patch which is to convert the string back to bytes for the final write out and avoid the error

Traceback (most recent call last):
File "/tmp/xencode/usr/libexec/xen/bin/pygrub", line 967, in <module>
os.write(fd, ostring)
TypeError: a bytes-like object is required, not 'str'

The attached patch works for me in a quick test on python3 though I haven't tested it on python2 yet.

(quoted diff) +0 -0
 	Michael Young

tools/pygrub/src/pygrub line 970

Quoted 9 lines
tools/pygrub/src/pygrub renamed from tools/pygrub/src/pygrub.orig +4 -1
@@ -963,5 +963,8 @@
         ostring = format_simple(bootcfg["kernel"], bootcfg["ramdisk"], args, "\0")
 
     sys.stdout.flush()
-    os.write(fd, ostring)
+    if sys.version_info[0] < 3:
+        os.write(fd, ostring)
+    else:
+        os.write(fd, ostring.encode())
 

This should work. I will fold this in to the pygrub patch.

Thanks for spotting my mistake.

Wei.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel