: add libxl python binding
PATCH 1/1 : add libxl python binding
Changes since last time:
- split auto-generated code in to c and h files
- un-break the build system
- fix ocaml binding due to libxl API change
- lot's of tidy-ups too numerous to mention
Please consider and apply :)
-----8<--------------------------------------------------------------- Introduce python binding for libxl. The binding is not yet complete but list_domains, domid_to_name, domain_shutdown and domain_destroy methods are implemented and tested. These functions provide examples of the two basic patterns that all future methods should follow.
About 5,000 lines of boilerplate is automatically generated to wrap and export all relevant libxl structure definitions. There are a few places where such code cannot be fully auto-generated and special hooks are declared and stubbed where, for example, conversion between libxl_file_reference and a python file object is required.
There are minor changes to libxltypes.py and the libxl.idl file and the new rules are documented. I have also removed the use of a nasty union in libxl_device_pci structure which was making the binding more complicated than necessary and for no good reason (exporting random bits of low level ABI that libxl would rather hide anyway). Finally there is a corresponding (untested) change to the ocaml binding to fix a compile error and maintain previous API in ocaml.
--
tools/python/genwrap.py | 346 +++++++++++++++++++++++++++++++ tools/python/xen/lowlevel/xl/xl.c | 397 ++++++++++++++++++++++++++++++++++++ tools/libxl/idl.txt | 7 tools/libxl/libxl.idl | 28 -- tools/libxl/libxl_pci.c | 13 - tools/libxl/libxltypes.py | 27 +- tools/ocaml/libs/xl/xl_stubs.c | 10 tools/python/Makefile | 6 tools/python/setup.py | 19 + 9 files changed, 823 insertions(+), 30 deletions(-)
--
tools/libxl/idl.txt +6 -1
@@ -13,7 +13,8 @@ contain the initial namespace element (e how to specify a namespace. The Type.typename contains the C name of the type _including_ the -namespace element. +namespace element while Type.rawname is always set to the 'base' name +of the type. The libxltypes.Type base class has several other properties which apply to all types. The properties are set by passing a named
@@ -48,6 +49,10 @@ Type.autogenerate_destructor: (default: Indicates if the above named Type.destructor_fn should be autogenerated. +Type.internal (default: False): + Indicates that type is not suitable for export, for example, via accessors + in auto-generated language bindings. + Other simple type-Classes -------------------------
tools/libxl/libxl.idl +11 -17
@@ -6,11 +6,11 @@ libxl_ctx = Builtin("ctx") libxl_uuid = Builtin("uuid") libxl_mac = Builtin("mac") -libxl_qemu_machine_type = Builtin("qemu_machine_type") -libxl_console_consback = Builtin("console_consback") -libxl_console_constype = Builtin("console_constype") -libxl_disk_phystype = Builtin("disk_phystype") -libxl_nic_type = Builtin("nic_type") +libxl_qemu_machine_type = Number("qemu_machine_type", namespace="libxl_") +libxl_console_consback = Number("console_consback", namespace="libxl_") +libxl_console_constype = Number("console_constype", namespace="libxl_") +libxl_disk_phystype = Number("disk_phystype", namespace="libxl_") +libxl_nic_type = Number("nic_type", namespace="libxl_") libxl_string_list = Builtin("string_list", destructor_fn="libxl_string_list_destroy", passby=PASS_BY_REFERENCE) libxl_key_value_list = Builtin("key_value_list", destructor_fn="libxl_key_value_list_destroy", passby=PASS_BY_REFERENCE)
@@ -85,7 +85,7 @@ libxl_file_reference = Struct("file_refe mapped is true then the actual file may already be unlinked."""), ("mapped", integer), ("data", void), - ("size", size_t)], autogenerate_destructor=False) + ("size", size_t)], autogenerate_destructor=False, internal=True) libxl_domain_build_info = Struct("domain_build_info",[ ("max_vcpus", integer),
@@ -240,17 +240,11 @@ libxl_device_net2 = Struct("device_net2" ]) libxl_device_pci = Struct("device_pci", [ - (None, Union(None, [("value", unsigned_integer), - (None, Struct(None,[("reserved1", BitField(unsigned_integer, 2)), - ("reg", BitField(unsigned_integer, 6)), - ("func", BitField(unsigned_integer, 3)), - ("dev", BitField(unsigned_integer, 5)), - ("bus", BitField(unsigned_integer, 8)), - ("reserved2", BitField(unsigned_integer, 7)), - ("enable", BitField(unsigned_integer, 1)), - ])), - ]) - ), + ("reg", uint8), + ("func", uint8), + ("dev", uint8), + ("bus", uint8), + ("enable", bool), ("domain", unsigned_integer), ("vdevfn", unsigned_integer), ("vfunc_mask", unsigned_integer),
tools/libxl/libxl_pci.c +11 -2
@@ -41,6 +41,15 @@#define PCI_BDF_SHORT "%02x:%02x.%01x"#define PCI_BDF_VDEVFN "%04x:%02x:%02x.%01x@%02x" +static unsigned int pcidev_value (libxl_device_pci *pcidev) +{ +return ((pcidev->reg &0x3f ) <<24 ) | + ((pcidev->func &0x7 ) <<21 ) | + ((pcidev->dev &0x1f ) <<16 ) | + ((pcidev->bus &0xff ) <<8 ) | + ((pcidev->enable) ?1 :0 ); +} +static int pcidev_init (libxl_device_pci *pcidev,unsigned int domain,unsigned int bus,unsigned int dev,unsigned int func,unsigned int vdevfn)
@@ -699,7 +708,7 @@ static int do_pci_add(libxl__gc *gc, uin } out:if (!libxl_is_stubdom (ctx, domid,NULL )) { - rc =xc_assign_device (ctx->xch, domid, pcidev->value); + rc =xc_assign_device (ctx->xch, domid,pcidev_value (pcidev));if (rc <0 && (hvm || errno !=ENOSYS )) {LIBXL__LOG_ERRNOVAL (ctx,LIBXL__LOG_ERROR , rc,"xc_assign_device failed" );return ERROR_FAIL ;
@@ -915,7 +924,7 @@ out: }if (!libxl_is_stubdom (ctx, domid,NULL )) { - rc =xc_deassign_device (ctx->xch, domid, pcidev->value); + rc =xc_deassign_device (ctx->xch, domid,pcidev_value (pcidev));if (rc <0 && (hvm || errno !=ENOSYS ))LIBXL__LOG_ERRNOVAL (ctx,LIBXL__LOG_ERROR , rc,"xc_deassign_device failed" ); }
tools/libxl/libxltypes.py +20 -7
@@ -14,10 +14,13 @@ class Type(object): if typename is None: # Anonymous type self.typename = None + self.rawname = None elif self.namespace is None: # e.g. system provided types self.typename = typename + self.rawname = typename else: self.typename = self.namespace + typename + self.rawname = typename if self.typename is not None: self.destructor_fn = kwargs.setdefault('destructor_fn', self.typename + "_destroy")
@@ -26,17 +29,27 @@ class Type(object): self.autogenerate_destructor = kwargs.setdefault('autogenerate_destructor', True) + self.internal = kwargs.setdefault('internal', False) + class Builtin(Type): """Builtin type""" def __init__(self, typename, **kwargs): kwargs.setdefault('destructor_fn', None) Type.__init__(self, typename, **kwargs) -class UInt(Type): +class Number(Builtin): + def __init__(self, ctype, **kwargs): + kwargs.setdefault('namespace', None) + kwargs.setdefault('destructor_fn', None) + kwargs.setdefault('signed', False) + self.signed = kwargs['signed'] + Builtin.__init__(self, ctype, **kwargs) + +class UInt(Number): def __init__(self, w, **kwargs): kwargs.setdefault('namespace', None) kwargs.setdefault('destructor_fn', None) - Type.__init__(self, "uint%d_t" % w, **kwargs) + Number.__init__(self, "uint%d_t" % w, **kwargs) self.width = w
@@ -128,12 +141,12 @@ class Reference(Type): void = Builtin("void *", namespace = None) bool = Builtin("bool", namespace = None) -size_t = Builtin("size_t", namespace = None) +size_t = Number("size_t", namespace = None) -integer = Builtin("int", namespace = None) -unsigned_integer = Builtin("unsigned int", namespace = None) -unsigned = Builtin("unsigned int", namespace = None) -unsigned_long = Builtin("unsigned long", namespace = None) +integer = Number("int", namespace = None, signed = True) +unsigned_integer = Number("unsigned int", namespace = None) +unsigned = Number("unsigned int", namespace = None) +unsigned_long = Number("unsigned long", namespace = None) uint8 = UInt(8) uint16 = UInt(16)
tools/ocaml/libs/xl/xl_stubs.c +9 -1
@@ -269,9 +269,17 @@ static int device_vfb_val(caml_gc *gc, lstatic int device_pci_val (caml_gc *gc,libxl_device_pci *c_val,value v) { +unsigned int bitmask;CAMLparam1 (v); - c_val->value =Int_val (Field (v,0 )); +/* FIXME: propagate API change to ocaml */ + bitmask =Int_val (Field (v,0 )); + c_val->reg = (bitmask & (0x3f <<24 )) >>24 ; + c_val->func = (bitmask & (0x7 <<21 )) >>21 ; + c_val->dev = (bitmask & (0x1f <<16 )) >>16 ; + c_val->bus = (bitmask & (0xff <<8 )) >>8 ; + c_val->enable = bitmask &1 ; + c_val->domain =Int_val (Field (v,1 )); c_val->vdevfn =Int_val (Field (v,2 )); c_val->msitranslate =Bool_val (Field (v,3 ));
tools/python/Makefile +6 -0
@@ -20,6 +20,10 @@ genpath-target = $(call buildmakevars2fi .PHONY: build buildpy buildpy: genpath + PYTHONPATH=$(XEN_ROOT)/tools/libxl $(PYTHON) genwrap.py \ + $(XEN_ROOT)/tools/libxl/libxl.idl \ + xen/lowlevel/xl/_pyxl_types.h \ + xen/lowlevel/xl/_pyxl_types.c CC="$(CC)" CFLAGS="$(CFLAGS)" $(PYTHON) setup.py build build: buildpy refresh-pot refresh-po $(CATALOGS)
@@ -85,6 +89,8 @@ test: clean: rm -f $(XENPATH) rm -rf build *.pyc *.pyo *.o *.a *~ $(CATALOGS) xen/util/auxbin.pyc + rm -f xen/lowlevel/xl/_pyxl_types.h + rm -f xen/lowlevel/xl/_pyxl_types.c rm -f $(DEPS) -include $(DEPS)
tools/python/genwrap.py new file +346 -0
346 lines not on this page · raw message
tools/python/setup.py +17 -2
@@ -9,14 +9,23 @@ extra_compile_args = [ "-fno-strict-ali include_dirs = [ XEN_ROOT + "/tools/libxc", XEN_ROOT + "/tools/xenstore", XEN_ROOT + "/tools/include", + XEN_ROOT + "/tools/libxl", ] library_dirs = [ XEN_ROOT + "/tools/libxc", XEN_ROOT + "/tools/xenstore", + XEN_ROOT + "/tools/libxl", + XEN_ROOT + "/tools/blktap2/control", ] libraries = [ "xenctrl", "xenguest", "xenstore" ] +plat = os.uname()[0] +if plat == 'Linux': + uuid_libs = ["uuid"] +else: + uuid_libs = [] + xc = Extension("xc", extra_compile_args = extra_compile_args, include_dirs = include_dirs + [ "xen/lowlevel/xc" ],
@@ -83,8 +92,14 @@ netlink = Extension("netlink", sources = [ "xen/lowlevel/netlink/netlink.c", "xen/lowlevel/netlink/libnetlink.c"]) -modules = [ xc, xs, ptsname, acm, flask ] -plat = os.uname()[0] +xl = Extension("xl", + extra_compile_args = extra_compile_args, + include_dirs = include_dirs + [ "xen/lowlevel/xl" ], + library_dirs = library_dirs, + libraries = libraries + ["xenlight", "blktapctl" ] + uuid_libs, + sources = [ "xen/lowlevel/xl/xl.c", "xen/lowlevel/xl/_pyxl_types.c" ]) + +modules = [ xc, xs, ptsname, acm, flask, xl ] if plat == 'SunOS': modules.extend([ scf, process ]) if plat == 'Linux':
tools/python/xen/lowlevel/xl/xl.c new file +397 -0
397 lines not on this page · raw message
Quoted 7 lines · Gianni Tedesco
Changes since last time:
- split auto-generated code in to c and h files
- un-break the build system
- fix ocaml binding due to libxl API change
- lot's of tidy-ups too numerous to mentionPlease consider and apply :)
Like it.
I think it'd be nice to separate out stuff like the PCI API change and IDL changes into preparatory patches. Patches which do one logical thing are generally easier to review (and useful for bisecting and such too).
Quoted 11 lines
-----8<--------------------------------------------------------------- Introduce python binding for libxl. The binding is not yet complete but
list_domains,domid_to_name,domain_shutdownanddomain_destroymethods are implemented and tested. These functions provide examples of the two basic patterns that all future methods should follow.About 5,000 lines of boilerplate is automatically generated to wrap and export all relevant libxl structure definitions. There are a few places where such code cannot be fully auto-generated and special hooks are declared and stubbed where, for example, conversion between
libxl_file_referenceand a python file object is required.
I think this is acceptable for Builtin() types and is really part of the meaning of builtin. (except as I read on I realise that libxl_file_reference isn't actually a Builtin like I thought, more on that later!)
Quoted 8 lines
There are minor changes to
libxltypes.pyand the libxl.idl file and the new rules are documented. I have also removed the use of a nasty union inlibxl_device_pcistructure which was making the binding more complicated than necessary and for no good reason (exporting random bits of low level ABI that libxl would rather hide anyway). Finally there is a corresponding (untested) change to the ocaml binding to fix a compile error and maintain previous API in ocaml.
Quoted 5 lines
Signed-off-by Gianni Tedesco <[email protected]>+Type.internal (default: False):
+ Indicates that type is not suitable for export, for example, via accessors
+ in auto-generated language bindings.
Maybe the issue here is that libxl_file_reference should really be a Builtin? It's suspicious that it is the only non-Builtin to which both this flag and autogenerate_destructor=False apply.
If that doesn't make sense then perhaps a new sub-class of Builtin might be more appropriate than an attribute (not sure what it would be called though, SuperBuiltin ;-)). I'm not sure that a non-Builtin should ever be internal and this would help prevent that.
tools/python/Makefile Thu Sep 09 12:06:50 2010 +0100 genpath-target = $(call buildmakevars2fi · line 27
Quoted 12 lines
diff -r
b19856f6dd76tools/python/Makefiletools/python/Makefile Thu Sep 09 12:06:50 2010 +0100 renamed from tools/python/Makefile Thu Sep 09 09:24:24 2010 +0100 +4
-0@@ -20,6 +20,10 @@ genpath-target = $(call buildmakevars2fi .PHONY: build buildpy buildpy: genpath + PYTHONPATH=$(XEN_ROOT)/tools/libxl $(PYTHON) genwrap.py \ + $(XEN_ROOT)/tools/libxl/libxl.idl \ + xen/lowlevel/xl/_pyxl_types.h \ + xen/lowlevel/xl/_pyxl_types.c CC="$(CC)" CFLAGS="$(CFLAGS)" $(PYTHON) setup.py build
Needs some additional dependencies on genwrap.py and $(XE_ROOT)/tools/libxl/libxl.idl?
Ian.
Quoted 7 lines · Ian Campbell
Quoted 6 lines · Gianni Tedesco
Changes since last time:
- split auto-generated code in to c and h files
- un-break the build system
- fix ocaml binding due to libxl API change
- lot's of tidy-ups too numerous to mention
Quoted 7 lines
Quoted 1 line
Please consider and apply :)
Like it.
I think it'd be nice to separate out stuff like the PCI API change and IDL changes into preparatory patches. Patches which do one logical thing are generally easier to review (and useful for bisecting and such too).
Yeah, that's not a problem, in this case I can just split it out by filename.
Quoted 27 lines
Quoted 11 lines
-----8<--------------------------------------------------------------- Introduce python binding for libxl. The binding is not yet complete but
list_domains,domid_to_name,domain_shutdownanddomain_destroymethods are implemented and tested. These functions provide examples of the two basic patterns that all future methods should follow.About 5,000 lines of boilerplate is automatically generated to wrap and export all relevant libxl structure definitions. There are a few places where such code cannot be fully auto-generated and special hooks are declared and stubbed where, for example, conversion between
libxl_file_referenceand a python file object is required.I think this is acceptable for
Builtin()types and is really part of the meaning of builtin. (except as I read on I realise thatlibxl_file_referenceisn't actually a Builtin like I thought, more on that later!)Quoted 9 lines
There are minor changes to
libxltypes.pyand the libxl.idl file and the new rules are documented. I have also removed the use of a nasty union inlibxl_device_pcistructure which was making the binding more complicated than necessary and for no good reason (exporting random bits of low level ABI that libxl would rather hide anyway). Finally there is a corresponding (untested) change to the ocaml binding to fix a compile error and maintain previous API in ocaml.Signed-off-by Gianni Tedesco <[email protected]>
Quoted 7 lines
Quoted 3 lines
+Type.internal (default: False):
+ Indicates that type is not suitable for export, for example, via accessors
+ in auto-generated language bindings.Maybe the issue here is that
libxl_file_referenceshould really be a Builtin? It's suspicious that it is the only non-Builtin to which both this flag and autogenerate_destructor=False apply.
I think your analysis is right and this in fact works for me. I had misunderstood full meaning of Builtins even though you had mentioned the (builtins,types) tuple thing. Fixed this and it works fine.
Quoted 4 lines
If that doesn't make sense then perhaps a new sub-class of Builtin might be more appropriate than an attribute (not sure what it would be called though, SuperBuiltin ;-)). I'm not sure that a non-Builtin should ever be internal and this would help prevent that.
Meh
Quoted 5 lines
Quoted 5 lines
diff -r
b19856f6dd76tools/python/Makefiletools/python/Makefile Thu Sep 09 12:06:50 2010 +0100 renamed from tools/python/Makefile Thu Sep 09 09:24:24 2010 +0100 +0
-0@@ -20,6 +20,10 @@ genpath-target = $(call buildmakevars2fi
Quoted 10 lines
tools/python/Makefile Thu Sep 09 12:06:50 2010 +0100 genpath-target = $(call buildmakevars2fi · line 27
Quoted 7 lines
tools/python/Makefile Thu Sep 09 12:06:50 2010 +0100 renamed from tools/python/Makefile Thu Sep 09 09:24:24 2010 +0100 +4
-0.PHONY: build buildpy buildpy: genpath + PYTHONPATH=$(XEN_ROOT)/tools/libxl $(PYTHON) genwrap.py \ + $(XEN_ROOT)/tools/libxl/libxl.idl \ + xen/lowlevel/xl/_pyxl_types.h \ + xen/lowlevel/xl/_pyxl_types.c CC="$(CC)" CFLAGS="$(CFLAGS)" $(PYTHON) setup.py buildNeeds some additional dependencies on
genwrap.pyand $(XE_ROOT)/tools/libxl/libxl.idl?
Yes, and libxltypes.py, fixed.