Skip to contentSkip to conversation

ocaml abi fixes

PATCH In review Ian Jackson 6 participants 25 messages Sep 10, 2019 Arm/actions semi architecture

lore
Share & tools
lore
#0 lore raw

v3 cover ocaml abi fixes

This now has further updates and improvements. Patches 1-3 are as before. Patches 4-12 are new. Thanks to Andy for his work (in particular, some debugging and commentary for my script) which I have incorporated here.

This is available as a git branch here
https://xenbits.xen.org/gitweb/?p=people/iwj/xen.git;a=summary
-b wip.ocaml.v3

Andrew Cooper (5):
tools/ocaml: Add missing CAP_PV
xen/domctl: Drop guest suffix from XEN_DOMCTL_CDF_hvm tools/ocaml: abi handling: Provide ocaml->C conversion/check tools/ocaml: Reformat domain_create_flag
tools/ocaml: abi: Use formal conversion and check in more places

Ian Jackson (7):
tools/ocaml: Add missing X86_EMU_VPCI
tools/ocaml: Introduce xenctrl ABI build-time checks tools/ocaml: abi-check: Add comments
tools/ocaml: abi-check: Improve output and error messages tools/ocaml: abi-check: Cope with multiple conversions of same type tools/ocaml: abi-check: Check properly.
tools/ocaml: tools/ocaml: Add missing CDF_* values

 .gitignore                          |   1 +
 tools/libxl/libxl_create.c          |   2 +-
 tools/ocaml/libs/xc/Makefile        |   5 ++
 tools/ocaml/libs/xc/abi-check       | 114 ++++++++++++++++++++++++++++++++++++
 tools/ocaml/libs/xc/xenctrl.ml      |   9 ++-
 tools/ocaml/libs/xc/xenctrl.mli     |  13 +++-
 tools/ocaml/libs/xc/xenctrl_stubs.c |  95 ++++++++++++++++++++++--------
 tools/python/xen/lowlevel/xc/xc.c   |   2 +-
 xen/arch/arm/domain.c               |   2 +-
 xen/arch/arm/domain_build.c         |   2 +-
 xen/arch/arm/setup.c                |   2 +-
 xen/arch/x86/domain.c               |   4 +-
 xen/arch/x86/setup.c                |   2 +-
 xen/common/domain.c                 |   4 +-
 xen/include/public/domctl.h         |   8 ++-
 xen/include/public/sysctl.h         |   4 ++
 xen/include/xen/sched.h             |   4 +-
 17 files changed, 232 insertions(+), 41 deletions(-)
 create mode 100755 tools/ocaml/libs/xc/abi-check
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw

PATCH v3 1/12 tools/ocaml: Add missing X86_EMU_VPCI

This was missing from x86_arch_emulation_flags.

---
 tools/ocaml/libs/xc/xenctrl.ml  | 1 +
 tools/ocaml/libs/xc/xenctrl.mli | 1 +
 2 files changed, 2 insertions(+)
tools/ocaml/libs/xc/xenctrl.ml +1 -0
index 35958b94d5..305625cb6c 100644
@@ -46,6 +46,7 @@ type x86_arch_emulation_flags =
 	| X86_EMU_IOMMU
 	| X86_EMU_PIT
 	| X86_EMU_USE_PIRQ
+	| X86_EMU_VPCI
 
 type xen_x86_arch_domainconfig =
 {
tools/ocaml/libs/xc/xenctrl.mli +1 -0
index 6c4268d453..da93160ed3 100644
@@ -40,6 +40,7 @@ type x86_arch_emulation_flags =
   | X86_EMU_IOMMU
   | X86_EMU_PIT
   | X86_EMU_USE_PIRQ
+  | X86_EMU_VPCI
 
 type xen_x86_arch_domainconfig = {
   emulation_flags: x86_arch_emulation_flags list;
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#2 lore raw

PATCH v3 2/12 tools/ocaml: Add missing CAP_PV

In reply to Ian Jackson

From: Andrew Cooper <[email protected]>

c/s f089fddd941 broke the Ocaml ABI by renumering XEN_SYSCTL_PHYSCAP_directio without adjusting the Ocaml physinfo_cap_flag enumeration. Fix this by inserting CAP_PV between CAP_HVM and CAP_DirectIO.

---
 tools/ocaml/libs/xc/xenctrl.ml  | 1 +
 tools/ocaml/libs/xc/xenctrl.mli | 5 ++++-
 2 files changed, 5 insertions(+), 1 deletion(-)
tools/ocaml/libs/xc/xenctrl.ml +1 -0
index 305625cb6c..097f39d5ce 100644
@@ -100,6 +100,7 @@ type sched_control =
 
 type physinfo_cap_flag =
 	| CAP_HVM
+	| CAP_PV
 	| CAP_DirectIO
 
 type physinfo =
tools/ocaml/libs/xc/xenctrl.mli +4 -1
index da93160ed3..957c9fdc2e 100644
@@ -83,7 +83,10 @@ type domaininfo = {
   arch_config : arch_domainconfig;
 }
 type sched_control = { weight : int; cap : int; }
-type physinfo_cap_flag = CAP_HVM | CAP_DirectIO
+type physinfo_cap_flag =
+  | CAP_HVM
+  | CAP_PV
+  | CAP_DirectIO
 type physinfo = {
   threads_per_core : int;
   cores_per_socket : int;
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#3 lore raw

PATCH v3 3/12 tools/ocaml: Introduce xenctrl ABI build-time checks

In reply to Ian Jackson

c/s f089fddd941 broke the Ocaml ABI by renumering XEN_SYSCTL_PHYSCAP_directio without adjusting the Ocaml physinfo_cap_flag enumeration.

Add build machinery which will check the ABI correspondence.

This will result in a compile time failure whenever constants get renumbered/added without a compatible adjustment to the Ocaml ABI.

---
 .gitignore                          |  1 +
 tools/ocaml/libs/xc/Makefile        |  5 +++
 tools/ocaml/libs/xc/abi-check       | 84 +++++++++++++++++++++++++++++++++++++
 tools/ocaml/libs/xc/xenctrl_stubs.c | 69 +++++++++++++++++++++---------
 xen/include/public/sysctl.h         |  4 ++
 5 files changed, 143 insertions(+), 20 deletions(-)
 create mode 100755 tools/ocaml/libs/xc/abi-check
.gitignore +1 -0
index 3c947ac948..3ada0c4f0b 100644
@@ -392,6 +392,7 @@ tools/ocaml/libs/xentoollog/_xtl_levels.*
 tools/ocaml/libs/xentoollog/xentoollog.ml
 tools/ocaml/libs/xentoollog/xentoollog.mli
 tools/ocaml/libs/xs/paths.ml
+tools/ocaml/libs/xc/xenctrl_abi_check.h
 tools/ocaml/xenstored/_paths.h
 tools/ocaml/xenstored/oxenstored
 tools/ocaml/xenstored/oxenstored.conf
tools/ocaml/libs/xc/Makefile +5 -0
index d24b0144d0..ac780627d2 100644
@@ -31,4 +31,9 @@ install: $(LIBS) META
 uninstall:
 	$(OCAMLFIND) remove -destdir $(OCAMLDESTDIR) xenctrl
 
+xenctrl_stubs.o: xenctrl_abi_check.h
+
+xenctrl_abi_check.h: abi-check xenctrl_stubs.c xenctrl.ml
+	$(PERL) -w $^ >[email protected] && mv -f [email protected] $@
+
 include $(TOPLEVEL)/Makefile.rules
tools/ocaml/libs/xc/abi-check new file +84 -0

84 lines not on this page · raw message

tools/ocaml/libs/xc/xenctrl_stubs.c +49 -20

99 lines not on this page · raw message

xen/include/public/sysctl.h +4 -0
index 36b3f8c429..5401f9c2fe 100644
@@ -90,6 +90,10 @@ struct xen_sysctl_tbuf_op {
  /* The platform supports direct access to I/O devices with IOMMU. */
 #define _XEN_SYSCTL_PHYSCAP_directio     2
 #define XEN_SYSCTL_PHYSCAP_directio  (1u<<_XEN_SYSCTL_PHYSCAP_directio)
+
+/* Max XEN_SYSCTL_PHYSCAP_* constant.  Used for ABI checking. */
+#define XEN_SYSCTL_PHYSCAP_MAX XEN_SYSCTL_PHYSCAP_directio
+
 struct xen_sysctl_physinfo {
     uint32_t threads_per_core;
     uint32_t cores_per_socket;
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Acked-by on v3 3/12
#4 lore raw

Re: [Xen-devel] [PATCH v3 03/12] tools/ocaml: Introduce xenctrl ABI build-time checks

Quoted 4 lines · Ian Jackson

c/s f089fddd941 broke the Ocaml ABI by renumering XEN_SYSCTL_PHYSCAP_directio without adjusting the Ocaml physinfo_cap_flag enumeration.

Quoted 8 lines

Add build machinery which will check the ABI correspondence.

This will result in a compile time failure whenever constants get renumbered/added without a compatible adjustment to the Ocaml ABI.

Just in case you want an extra ack for the tiny sysctl.h addition:

xen/include/public/sysctl.h struct xen_sysctl_tbuf_op { · line 95

Quoted 9 lines
xen/include/public/sysctl.h +3 -0
@@ -90,6 +90,10 @@ struct xen_sysctl_tbuf_op {
  /* The platform supports direct access to I/O devices with IOMMU. */
 #define _XEN_SYSCTL_PHYSCAP_directio     2
 #define XEN_SYSCTL_PHYSCAP_directio  (1u<<_XEN_SYSCTL_PHYSCAP_directio)
+
+/* Max XEN_SYSCTL_PHYSCAP_* constant.  Used for ABI checking. */
+#define XEN_SYSCTL_PHYSCAP_MAX XEN_SYSCTL_PHYSCAP_directio

I don't particularly like such "max" values put in (public) headers (as they require constant updating), but I can't see a good alternative either.

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#5 lore raw

Re: [Xen-devel] [PATCH v3 03/12] tools/ocaml: Introduce xenctrl ABI build-time checks

Quoted 16 lines · Jan Beulich
Quoted 4 lines · Ian Jackson

c/s f089fddd941 broke the Ocaml ABI by renumering XEN_SYSCTL_PHYSCAP_directio without adjusting the Ocaml physinfo_cap_flag enumeration.

Quoted 8 lines

Add build machinery which will check the ABI correspondence.

This will result in a compile time failure whenever constants get renumbered/added without a compatible adjustment to the Ocaml ABI.

Just in case you want an extra ack for the tiny sysctl.h addition:

Quoted 12 lines

xen/include/public/sysctl.h struct xen_sysctl_tbuf_op { · line 95

Quoted 9 lines
xen/include/public/sysctl.h +3 -0
@@ -90,6 +90,10 @@ struct xen_sysctl_tbuf_op {
  /* The platform supports direct access to I/O devices with IOMMU. */
 #define _XEN_SYSCTL_PHYSCAP_directio     2
 #define XEN_SYSCTL_PHYSCAP_directio  (1u<<_XEN_SYSCTL_PHYSCAP_directio)
+
+/* Max XEN_SYSCTL_PHYSCAP_* constant.  Used for ABI checking. */
+#define XEN_SYSCTL_PHYSCAP_MAX XEN_SYSCTL_PHYSCAP_directio

I don't particularly like such "max" values put in (public) headers (as they require constant updating), but I can't see a good alternative either.

This is the best I could come up with.  At least it is in a tools-restricted header.

~Andrew

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#6 lore raw

PATCH v3 4/12 xen/domctl: Drop guest suffix from XEN_DOMCTL_CDF_hvm

In reply to Ian Jackson

From: Andrew Cooper <[email protected]>

---
 tools/libxl/libxl_create.c        | 2 +-
 tools/python/xen/lowlevel/xc/xc.c | 2 +-
 xen/arch/arm/domain.c             | 2 +-
 xen/arch/arm/domain_build.c       | 2 +-
 xen/arch/arm/setup.c              | 2 +-
 xen/arch/x86/domain.c             | 4 ++--
 xen/arch/x86/setup.c              | 2 +-
 xen/common/domain.c               | 4 ++--
 xen/include/public/domctl.h       | 4 ++--
 xen/include/xen/sched.h           | 4 ++--
 10 files changed, 14 insertions(+), 14 deletions(-)
tools/libxl/libxl_create.c +1 -1
index 03ce166f4f..79e010da72 100644
@@ -557,7 +557,7 @@ int libxl__domain_make(libxl__gc *gc, libxl_domain_config *d_config,
         };
 
         if (info->type != LIBXL_DOMAIN_TYPE_PV) {
-            create.flags |= XEN_DOMCTL_CDF_hvm_guest;
+            create.flags |= XEN_DOMCTL_CDF_hvm;
             create.flags |=
                 libxl_defbool_val(info->hap) ? XEN_DOMCTL_CDF_hap : 0;
             create.flags |=
tools/python/xen/lowlevel/xc/xc.c +1 -1
index 9d53c4cf37..26b9a0fb74 100644
@@ -155,7 +155,7 @@ static PyObject *pyxc_domain_create(XcObject *self,
     }
 
 #if defined (__i386) || defined(__x86_64__)
-    if ( config.flags & XEN_DOMCTL_CDF_hvm_guest )
+    if ( config.flags & XEN_DOMCTL_CDF_hvm )
         config.arch.emulation_flags = (XEN_X86_EMU_ALL & ~XEN_X86_EMU_VPCI);
 #elif defined (__arm__) || defined(__aarch64__)
     config.arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE;
xen/arch/arm/domain.c +1 -1
index 941bbff4fe..a9c4113c26 100644
@@ -608,7 +608,7 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
 {
     unsigned int max_vcpus;
 
-    if ( config->flags != (XEN_DOMCTL_CDF_hvm_guest | XEN_DOMCTL_CDF_hap) )
+    if ( config->flags != (XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap) )
     {
         dprintk(XENLOG_INFO, "Unsupported configuration %#x\n", config->flags);
         return -EINVAL;
xen/arch/arm/domain_build.c +1 -1
index e79d4e204f..a0fee1ef13 100644
@@ -2070,7 +2070,7 @@ void __init create_domUs(void)
         struct xen_domctl_createdomain d_cfg = {
             .arch.gic_version = XEN_DOMCTL_CONFIG_GIC_NATIVE,
             .arch.nr_spis = 0,
-            .flags = XEN_DOMCTL_CDF_hvm_guest | XEN_DOMCTL_CDF_hap,
+            .flags = XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap,
             .max_evtchn_port = -1,
             .max_grant_frames = 64,
             .max_maptrack_frames = 1024,
xen/arch/arm/setup.c +1 -1
index fa6c110b11..1b303bde34 100644
@@ -797,7 +797,7 @@ void __init start_xen(unsigned long boot_phys_offset,
     struct bootmodule *xen_bootmodule;
     struct domain *dom0;
     struct xen_domctl_createdomain dom0_cfg = {
-        .flags = XEN_DOMCTL_CDF_hvm_guest | XEN_DOMCTL_CDF_hap,
+        .flags = XEN_DOMCTL_CDF_hvm | XEN_DOMCTL_CDF_hap,
         .max_evtchn_port = -1,
         .max_grant_frames = gnttab_dom0_frames(),
         .max_maptrack_frames = opt_max_maptrack_frames,
xen/arch/x86/domain.c +2 -2
index d538de8eae..4b0ad5e15d 100644
@@ -442,7 +442,7 @@ void arch_vcpu_destroy(struct vcpu *v)
 
 int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
 {
-    bool hvm = config->flags & XEN_DOMCTL_CDF_hvm_guest;
+    bool hvm = config->flags & XEN_DOMCTL_CDF_hvm;
     unsigned int max_vcpus;
 
     if ( hvm ? !hvm_enabled : !IS_ENABLED(CONFIG_PV) )
@@ -473,7 +473,7 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config)
         return -EINVAL;
     }
 
-    if ( !(config->flags & XEN_DOMCTL_CDF_hvm_guest) )
+    if ( !(config->flags & XEN_DOMCTL_CDF_hvm) )
         /*
          * It is only meaningful for XEN_DOMCTL_CDF_oos_off to be clear
          * for HVM guests.
xen/arch/x86/setup.c +1 -1
index 5a88ef368f..27981adc0b 100644
@@ -1723,7 +1723,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 
     if ( opt_dom0_pvh )
     {
-        dom0_cfg.flags |= (XEN_DOMCTL_CDF_hvm_guest |
+        dom0_cfg.flags |= (XEN_DOMCTL_CDF_hvm |
                            ((hvm_hap_supported() && !opt_dom0_shadow) ?
                             XEN_DOMCTL_CDF_hap : 0));
 
xen/common/domain.c +2 -2
index e9d2c613e0..09917b2885 100644
@@ -297,7 +297,7 @@ static void _domain_destroy(struct domain *d)
 
 static int sanitise_domain_config(struct xen_domctl_createdomain *config)
 {
-    if ( config->flags & ~(XEN_DOMCTL_CDF_hvm_guest |
+    if ( config->flags & ~(XEN_DOMCTL_CDF_hvm |
                            XEN_DOMCTL_CDF_hap |
                            XEN_DOMCTL_CDF_s3_integrity |
                            XEN_DOMCTL_CDF_oos_off |
@@ -313,7 +313,7 @@ static int sanitise_domain_config(struct xen_domctl_createdomain *config)
         return -EINVAL;
     }
 
-    if ( !(config->flags & XEN_DOMCTL_CDF_hvm_guest) &&
+    if ( !(config->flags & XEN_DOMCTL_CDF_hvm) &&
          (config->flags & XEN_DOMCTL_CDF_hap) )
     {
         dprintk(XENLOG_INFO, "HAP requested for non-HVM guest\n");
xen/include/public/domctl.h +2 -2
index 72d5133cba..ff9265f765 100644
@@ -50,8 +50,8 @@ struct xen_domctl_createdomain {
     uint32_t ssidref;
     xen_domain_handle_t handle;
  /* Is this an HVM guest (as opposed to a PV guest)? */
-#define _XEN_DOMCTL_CDF_hvm_guest     0
-#define XEN_DOMCTL_CDF_hvm_guest      (1U<<_XEN_DOMCTL_CDF_hvm_guest)
+#define _XEN_DOMCTL_CDF_hvm           0
+#define XEN_DOMCTL_CDF_hvm            (1U<<_XEN_DOMCTL_CDF_hvm)
  /* Use hardware-assisted paging if available? */
 #define _XEN_DOMCTL_CDF_hap           1
 #define XEN_DOMCTL_CDF_hap            (1U<<_XEN_DOMCTL_CDF_hap)
xen/include/xen/sched.h +2 -2
index d2bbe03bd9..e3601c1935 100644
@@ -912,7 +912,7 @@ void watchdog_domain_destroy(struct domain *d);
 static inline bool is_pv_domain(const struct domain *d)
 {
     return IS_ENABLED(CONFIG_PV) &&
-        evaluate_nospec(!(d->options & XEN_DOMCTL_CDF_hvm_guest));
+        evaluate_nospec(!(d->options & XEN_DOMCTL_CDF_hvm));
 }
 
 static inline bool is_pv_vcpu(const struct vcpu *v)
@@ -944,7 +944,7 @@ static inline bool is_pv_64bit_vcpu(const struct vcpu *v)
 static inline bool is_hvm_domain(const struct domain *d)
 {
     return IS_ENABLED(CONFIG_HVM) &&
-        evaluate_nospec(d->options & XEN_DOMCTL_CDF_hvm_guest);
+        evaluate_nospec(d->options & XEN_DOMCTL_CDF_hvm);
 }
 
 static inline bool is_hvm_vcpu(const struct vcpu *v)
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#7 lore raw

Re: [Xen-devel] [PATCH v3 04/12] xen/domctl: Drop guest suffix from XEN_DOMCTL_CDF_hvm

Quoted 3 lines · Ian Jackson

From: Andrew Cooper <[email protected]>

I should probably have finished writing my commit message before handing the branch off.

"The suffix is redundant, and dropping it helps to simplify the Ocaml/C ABI checking."

or something suitable.

~Andrew

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#8 lore raw

Re: [Xen-devel] [PATCH v3 04/12] xen/domctl: Drop guest suffix from XEN_DOMCTL_CDF_hvm

Andrew Cooper writes ("Re: [PATCH v3 04/12] xen/domctl: Drop guest suffix from XEN_DOMCTL_CDF_hvm"):

Quoted 2 lines

I should probably have finished writing my commit message before handing the branch off.

No problem. If you had left an XXX in it I would have known to ask you for the rest...

Quoted 2 lines

"The suffix is redundant, and dropping it helps to simplify the Ocaml/C ABI checking."

I have c&p that into my branch, thanks.

Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Acked-by
#9 lore raw

Re: [Xen-devel] [PATCH v3 04/12] xen/domctl: Drop guest suffix from XEN_DOMCTL_CDF_hvm

In reply to Andrew Cooper

Hi,

Quoted 5 lines · Andrew Cooper
Quoted 3 lines · Ian Jackson

From: Andrew Cooper <[email protected]>

Quoted 7 lines

I should probably have finished writing my commit message before handing the branch off.

"The suffix is redundant, and dropping it helps to simplify the Ocaml/C ABI checking."

or something suitable.

With the commit message updated:

Cheers,

Signature · 6 lines
Julien Grall

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#11 lore raw

PATCH v3 5/12 tools/ocaml: abi-check: Add comments

In reply to Ian Jackson

Provide interface documentation for this script.

Explain why we check .ml not .mli.

---
 tools/ocaml/libs/xc/abi-check | 15 +++++++++++++++
 1 file changed, 15 insertions(+)
tools/ocaml/libs/xc/abi-check +15 -0
index c987cd8454..ccc35e79e9 100755
@@ -5,6 +5,13 @@ use Data::Dumper;
 
 our %enums;
 
+# Usage: abi-check C-file Ocaml-file
+# Writes out a BUILD_BUG_ON() list to be included back into C.
+#
+# Ocaml-file should be the .ml file.  The ocaml compiler will check
+# that any declarations in a .mli correspond.  We check the .ml
+# rather than the .mli in case there are private types in future.
+
 @ARGV == 2 or die;
 our ($c, $o) = @ARGV;
 
@@ -13,6 +20,14 @@ open STDIN, "<", $c or die $!;
 our $cline = -1;
 our $ei;
 
+# Parse the C file looking for calls to:
+#   c_bitmap_to_ocaml_list()
+#
+# followed by anotations of the following form:
+#   /* ! OType OPrefix Mangle */
+#   /* ! CPrefix CFinal CFinalHow */
+#
+# The function definitions use /* ! */ which simply skips that instance.
 while (<>) {
     if ($cline == -1) {
         if (m/c_bitmap_to_ocaml_list/) {
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#12 lore raw

PATCH v3 6/12 tools/ocaml: abi handling: Provide ocaml->C conversion/check

In reply to Ian Jackson

From: Andrew Cooper <[email protected]>

No users of this yet so no overall change.

---
 tools/ocaml/libs/xc/abi-check       |  3 ++-
 tools/ocaml/libs/xc/xenctrl_stubs.c | 15 +++++++++++++++
 2 files changed, 17 insertions(+), 1 deletion(-)
tools/ocaml/libs/xc/abi-check +2 -1
index ccc35e79e9..a470c05e0b 100755
@@ -22,6 +22,7 @@ our $ei;
 
 # Parse the C file looking for calls to:
 #   c_bitmap_to_ocaml_list()
+#   ocaml_list_to_c_bitmap()
 #
 # followed by anotations of the following form:
 #   /* ! OType OPrefix Mangle */
@@ -30,7 +31,7 @@ our $ei;
 # The function definitions use /* ! */ which simply skips that instance.
 while (<>) {
     if ($cline == -1) {
-        if (m/c_bitmap_to_ocaml_list/) {
+        if (m/c_bitmap_to_ocaml_list|ocaml_list_to_c_bitmap/) {
             $cline = 0;
             $ei = { };
         }
tools/ocaml/libs/xc/xenctrl_stubs.c +15 -0
index 352a6bd2d6..c74c2b43a0 100644
@@ -153,6 +153,21 @@ static value c_bitmap_to_ocaml_list
 	CAMLreturn(list);
 }
 
+static unsigned int ocaml_list_to_c_bitmap(value l)
+             /* ! */
+             /*
+	      * All calls to this function must be in a form suitable
+	      * for xenctrl_abi_check.  The parsing there is ad-hoc.
+	      */
+{
+	unsigned int val;
+
+	for ( ; l != Val_none; l = Field(l, 1) )
+		val |= 1u << Int_val(Field(l, 0));
+
+	return val;
+}
+
 CAMLprim value stub_xc_domain_create(value xch, value config)
 {
 	CAMLparam2(xch, config);
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#13 lore raw

Re: [Xen-devel] [PATCH v3 06/12] tools/ocaml: abi handling: Provide ocaml->C conversion/check

Quoted 6 lines · Ian Jackson

From: Andrew Cooper <[email protected]>

No users of this yet so no overall change.

I'm afraid this doesn't bisect.  It needs to be in the same patch as one of its callers.

~Andrew

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#14 lore raw

Re: [Xen-devel] [PATCH v3 06/12] tools/ocaml: abi handling: Provide ocaml->C conversion/check

Andrew Cooper writes ("Re: [PATCH v3 06/12] tools/ocaml: abi handling: Provide ocaml->C conversion/check"):

Quoted 10 lines · Ian Jackson
Quoted 6 lines

From: Andrew Cooper <[email protected]>

No users of this yet so no overall change.

I'm afraid this doesn't bisect.  It needs to be in the same patch as one of its callers.

Oops. I will fix this with a temporary #if 0, since I don't like to mix up the general with the specific.

While checking for bisectability I discovered my `make clean' target was not working right. I have added
GENERATED_FILES += xenctrl_abi_check.h
to the Makefile patch. I trust that it's OK to keep everyone's acks.

Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#15 lore raw

PATCH v3 7/12 tools/ocaml: abi-check: Improve output and error messages

In reply to Ian Jackson

In the generated C, add some comments saying where we found the ocaml type. This helps with debugging. (I considered emitting #line directives but decided this would be more confusing than helpful.)

Improve two dies.

Use better-named filehandles (perl prints thier names when it dies).

---
 tools/ocaml/libs/xc/abi-check | 14 ++++++++------
 1 file changed, 8 insertions(+), 6 deletions(-)
tools/ocaml/libs/xc/abi-check +8 -6
index a470c05e0b..9450676429 100755
@@ -15,7 +15,7 @@ our %enums;
 @ARGV == 2 or die;
 our ($c, $o) = @ARGV;
 
-open STDIN, "<", $c or die $!;
+open C_FILE, "<", $c or die $!;
 
 our $cline = -1;
 our $ei;
@@ -29,7 +29,7 @@ our $ei;
 #   /* ! CPrefix CFinal CFinalHow */
 #
 # The function definitions use /* ! */ which simply skips that instance.
-while (<>) {
+while (<C_FILE>) {
     if ($cline == -1) {
         if (m/c_bitmap_to_ocaml_list|ocaml_list_to_c_bitmap/) {
             $cline = 0;
@@ -37,7 +37,8 @@ while (<>) {
         }
     } else {
         $cline++;
-        m{^\s+/\* \s+ ! \s+ (.*?) \s* \*/\s*$}x or die "$cline $_ ?";
+        m{^\s+/\* \s+ ! \s+ (.*?) \s* \*/\s*$}x or
+            die "at line $cline of annotation, did not expect $_ ?";
         my @vals = split /\s+/, $1;
         if ($cline == 1 && !@vals) {
             $cline = -1;
@@ -45,7 +46,7 @@ while (<>) {
             $ei->{$_} = shift @vals foreach qw(OType OPrefix Mangle);
         } elsif ($cline == 2 && @vals == 3) {
             $ei->{$_} = shift @vals foreach qw(CPrefix CFinal CFinalHow);
-            die if $enums{ $ei->{OType} };
+            die "redefining OType $ei->{OType}" if $enums{ $ei->{OType} };
             $enums{ $ei->{OType} } = $ei;
             $cline = -1;
         } else {
@@ -58,13 +59,14 @@ sub expect ($$) {
     printf "BUILD_BUG_ON( %-30s != %-10s );\n", @_ or die $!;
 }
 
-open STDIN, "<", $o or die $!;
+open OCAML_FILE, "<", $o or die $!;
 my $cval;
 $ei = undef;
 my $bitnum = 0;
-while (<>) {
+while (<OCAML_FILE>) {
     if (!$ei) {
         if (m{^type \s+ (\w+) \s* \= \s* $/}x && $enums{$1}) {
+            print "// found ocaml type $1 at $o:$.\n" or die $!;
             $ei = $enums{$1};
             $cval = '';
             $bitnum = 0;
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#16 lore raw

PATCH v3 8/12 tools/ocaml: abi-check: Cope with multiple conversions of same type

In reply to Ian Jackson
---
 tools/ocaml/libs/xc/abi-check | 5 +++++
 1 file changed, 5 insertions(+)
tools/ocaml/libs/xc/abi-check +5 -0
index 9450676429..abcd6ce6f1 100755
@@ -27,6 +27,8 @@ our $ei;
 # followed by anotations of the following form:
 #   /* ! OType OPrefix Mangle */
 #   /* ! CPrefix CFinal CFinalHow */
+# or, for subsequent invocations for the same OType, just
+#   /* ! OType */
 #
 # The function definitions use /* ! */ which simply skips that instance.
 while (<C_FILE>) {
@@ -42,6 +44,9 @@ while (<C_FILE>) {
         my @vals = split /\s+/, $1;
         if ($cline == 1 && !@vals) {
             $cline = -1;
+        } elsif ($cline == 1 && @vals == 1) {
+            my ($otype) = @vals;
+            die "reference to undefined OType $otype" unless $enums{$otype};
         } elsif ($cline == 1 && @vals == 3) {
             $ei->{$_} = shift @vals foreach qw(OType OPrefix Mangle);
         } elsif ($cline == 2 && @vals == 3) {
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#17 lore raw

PATCH v3 9/12 tools/ocaml: Reformat domain_create_flag

In reply to Ian Jackson

From: Andrew Cooper <[email protected]>

This will allow us to apply the abi checker soon.

No functional change.

---
 tools/ocaml/libs/xc/xenctrl.ml  | 4 +++-
 tools/ocaml/libs/xc/xenctrl.mli | 4 +++-
 2 files changed, 6 insertions(+), 2 deletions(-)
tools/ocaml/libs/xc/xenctrl.ml +3 -1
index 097f39d5ce..17174debb8 100644
@@ -57,7 +57,9 @@ type arch_domainconfig =
 	| ARM of xen_arm_arch_domainconfig
 	| X86 of xen_x86_arch_domainconfig
 
-type domain_create_flag = CDF_HVM | CDF_HAP
+type domain_create_flag =
+	| CDF_HVM
+	| CDF_HAP
 
 type domctl_create_config =
 {
tools/ocaml/libs/xc/xenctrl.mli +3 -1
index 957c9fdc2e..228bc00a1c 100644
@@ -50,7 +50,9 @@ type arch_domainconfig =
   | ARM of xen_arm_arch_domainconfig
   | X86 of xen_x86_arch_domainconfig
 
-type domain_create_flag = CDF_HVM | CDF_HAP
+type domain_create_flag =
+  | CDF_HVM
+  | CDF_HAP
 
 type domctl_create_config = {
   ssidref: int32;
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#18 lore raw

PATCH v3 10/12 tools/ocaml: abi-check: Check properly.

In reply to Ian Jackson

Fix a broken regexp which would mention $/' when it ought to have mentioned $'. The result would be that it would match lines like
type some_ocaml_type = Thing | Other_Thing
but ignore everything but the type name, giving wrong answers.

Check that we check mentioned types. Otherwise if we fail to spot some suitable thing in the ocaml, we would just omit checking this type !

---
 tools/ocaml/libs/xc/abi-check | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
tools/ocaml/libs/xc/abi-check +8 -1
index abcd6ce6f1..d532f37271 100755
@@ -47,6 +47,7 @@ while (<C_FILE>) {
         } elsif ($cline == 1 && @vals == 1) {
             my ($otype) = @vals;
             die "reference to undefined OType $otype" unless $enums{$otype};
+            $cline = -1;
         } elsif ($cline == 1 && @vals == 3) {
             $ei->{$_} = shift @vals foreach qw(OType OPrefix Mangle);
         } elsif ($cline == 2 && @vals == 3) {
@@ -70,7 +71,7 @@ $ei = undef;
 my $bitnum = 0;
 while (<OCAML_FILE>) {
     if (!$ei) {
-        if (m{^type \s+ (\w+) \s* \= \s* $/}x && $enums{$1}) {
+        if (m{^type \s+ (\w+) \s* \= \s* $}x && $enums{$1}) {
             print "// found ocaml type $1 at $o:$.\n" or die $!;
             $ei = $enums{$1};
             $cval = '';
@@ -96,6 +97,7 @@ while (<OCAML_FILE>) {
             } else {
                 die Dumper($ei)." ?";
             }
+            $ei->{Checked} = 1;
             $ei = undef;
         } elsif (!m{\S}) {
         } else {
@@ -104,4 +106,9 @@ while (<OCAML_FILE>) {
     }
 }
 
+foreach $ei (values %enums) {
+    next if $ei->{Checked};
+    die "did not find ocaml type definition for $ei->{OType} in $o";
+}
+
 close STDOUT or die $!;
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#19 lore raw

PATCH v3 11/12 tools/ocaml: tools/ocaml: Add missing CDF_* values

In reply to Ian Jackson
---
 tools/ocaml/libs/xc/xenctrl.ml  | 3 +++
 tools/ocaml/libs/xc/xenctrl.mli | 3 +++
 2 files changed, 6 insertions(+)
tools/ocaml/libs/xc/xenctrl.ml +3 -0
index 17174debb8..e544ef84da 100644
@@ -60,6 +60,9 @@ type arch_domainconfig =
 type domain_create_flag =
 	| CDF_HVM
 	| CDF_HAP
+	| CDF_S3_INTEGRITY
+	| CDF_OOS_OFF
+	| CDF_XS_DOMAIN
 
 type domctl_create_config =
 {
tools/ocaml/libs/xc/xenctrl.mli +3 -0
index 228bc00a1c..5a35000761 100644
@@ -53,6 +53,9 @@ type arch_domainconfig =
 type domain_create_flag =
   | CDF_HVM
   | CDF_HAP
+  | CDF_S3_INTEGRITY
+  | CDF_OOS_OFF
+  | CDF_XS_DOMAIN
 
 type domctl_create_config = {
   ssidref: int32;
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#20 lore raw

PATCH v3 12/12 tools/ocaml: abi: Use formal conversion and check in more places

In reply to Ian Jackson

From: Andrew Cooper <[email protected]>

---
 tools/ocaml/libs/xc/xenctrl_stubs.c | 15 +++++++++------
 xen/include/public/domctl.h         |  4 ++++
 2 files changed, 13 insertions(+), 6 deletions(-)
tools/ocaml/libs/xc/xenctrl_stubs.c +9 -6
index c74c2b43a0..f86ecc7b7e 100644
@@ -195,8 +195,10 @@ CAMLprim value stub_xc_domain_create(value xch, value config)
 
 	domain_handle_of_uuid_string(cfg.handle, String_val(VAL_HANDLE));
 
-	for ( l = VAL_FLAGS; l != Val_none; l = Field(l, 1) )
-		cfg.flags |= 1u << Int_val(Field(l, 0));
+	cfg.flags = ocaml_list_to_c_bitmap
+		/* ! domain_create_flag CDF_ lc */
+		/* ! XEN_DOMCTL_CDF_ XEN_DOMCTL_CDF_MAX max */
+		(VAL_FLAGS);
 
 	arch_domconfig = Field(VAL_ARCH, 0);
 	switch ( Tag_val(VAL_ARCH) )
@@ -211,8 +213,10 @@ CAMLprim value stub_xc_domain_create(value xch, value config)
         /* Mnemonics for the named fields inside xen_x86_arch_domainconfig */
 #define VAL_EMUL_FLAGS          Field(arch_domconfig, 0)
 
-		for ( l = VAL_EMUL_FLAGS; l != Val_none; l = Field(l, 1) )
-			cfg.arch.emulation_flags |= 1u << Int_val(Field(l, 0));
+		cfg.arch.emulation_flags = ocaml_list_to_c_bitmap
+			/* ! x86_arch_emulation_flags X86_EMU_ none */
+			/* ! XEN_X86_EMU_ XEN_X86_EMU_ALL all */
+			(VAL_EMUL_FLAGS);
 
 #undef VAL_EMUL_FLAGS
 
@@ -368,8 +372,7 @@ static value alloc_domaininfo(xc_domaininfo_t * info)
 	 * emulation_flags: x86_arch_emulation_flags list;
 	 */
 	emul_list = c_bitmap_to_ocaml_list
-		/* ! x86_arch_emulation_flags X86_EMU_ none */
-		/* ! XEN_X86_EMU_ XEN_X86_EMU_ALL all */
+		/* ! x86_arch_emulation_flags */
 		(info->arch_config.emulation_flags);
 
 	/* xen_x86_arch_domainconfig */
xen/include/public/domctl.h +4 -0
index ff9265f765..77f546cbb8 100644
@@ -64,6 +64,10 @@ struct xen_domctl_createdomain {
  /* Is this a xenstore domain? */
 #define _XEN_DOMCTL_CDF_xs_domain     4
 #define XEN_DOMCTL_CDF_xs_domain      (1U<<_XEN_DOMCTL_CDF_xs_domain)
+
+/* Max XEN_DOMCTL_CDF_* constant.  Used for ABI checking. */
+#define XEN_DOMCTL_CDF_MAX XEN_DOMCTL_CDF_xs_domain
+
     uint32_t flags;
 
     /*
Signature · 7 lines
2.11.0


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Acked-by on v3 12/12
#21 lore raw

Re: [Xen-devel] [PATCH v3 12/12] tools/ocaml: abi: Use formal conversion and check in more places

Quoted 4 lines · Ian Jackson

From: Andrew Cooper <[email protected]>

Same as for the sysctl one - if you want it, here you are:

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#22 lore raw
In reply to Ian Jackson

Ian Jackson writes ("[PATCH v3 00/12] ocaml abi fixes"):

Quoted 4 lines

This now has further updates and improvements. Patches 1-3 are as before. Patches 4-12 are new. Thanks to Andy for his work (in particular, some debugging and commentary for my script) which I have incorporated here.

I have fixed a few minor issues as discussed in this thread, and incorporated acks so far.

Andrew gave me his acked/reviewed-by on irc and I have added my own R-B to the patches of his I incorporated (and, in some cases wrote commit messages for).

The result is here:
https://xenbits.xen.org/gitweb/?p=people/iwj/xen.git;a=summary
-b wip.ocaml.v3.1
I can repost it if anyone thinks that would be useful.

Christian, are you happy with my other changes in tools/ocaml ? I don't feel I need a formal ocaml ack for the changes to add the missing entries to the enum types (since those are supposed to follow the hypervisor ABI) but I think I need your ack for the further script and xenctrl_stubs changes in v3 (or v3.1) of this series.

And I think there is one patch from Andy that could do with an ack from a hypervisor maintainer (hence THE REST in my "To" field):
xen/domctl: Drop guest suffix from XEN_DOMCTL_CDF_hvm

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#23 lore raw
Quoted 5 lines · Ian Jackson

Christian, are you happy with my other changes in tools/ocaml ? I don't feel I need a formal ocaml ack for the changes to add the missing entries to the enum types (since those are supposed to follow the hypervisor ABI) but I think I need your ack for the further script and xenctrl_stubs changes in v3 (or v3.1) of this series.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel